This is super awesome, one thing I would love is a way to integrate the stats for our schedular as well somehow.
Perhaps some sort of interface we can call to inform it we are running tasks outside of sidekiq?
“we are running tasks outside of sidekiq”
sorry, but could you explain this moment? You use resque or something else?
Yes we have this:
Which runs regular jobs for us in dedicated thread, it is initialized here:
I would like to have a way for schedular manager to inform statistics that these regular jobs ran…
Sounds good, can you expose it as an API ? that way it will be less fragile long term integration wise
Also is there some sort of storage limit here? Not sure if we need to keep all this information around beyond 1 month
yep, I can add this information to README file
I collect information per day and I haven’t any data live limits now.
So, I created section “how you can update statistic inside middleware”. You can find its here:
Also I have interesting idea. What do you think if I can collect any data from redis hash?
i.e. I will create table columns dynamically on index page.
For example: you add
yyyy-mm-dd:WorkerName:next-runtime key and on the index page there is a new column in the table.
Could you perform any job? It’s look like as empty statistic hash.
I mean that in theory I can display any information from redis hash. It’s mean that you can add any information (for example date of next run time) and you will see this in statistic table on index page.
Tons of jobs have been running in the last 15 minutes since we deployed it.
and what about table? it’s displaying correct?
Table is blank … maybe somehow… it is not talking to the right redis?
Can you try on a docker install? I can provide you with an image to play with on digital ocean if you wish.
Yeah I can see it working on local … its got to be talking incorrectly to redis.
… aha … this has to be related to the forking …
It is not being initialized … having a look
It’s really strange because I use only
Sidekiq#redis for calling redis
I see the bug, we fork out our Redis so we need to manually initialize on fork.
how I can to help you?
It’s fine, in theory this should fix it
Confirmed my patch fixed this.
I think @sam wanted something more along the lines of:
begin job.execute(args) finish = Time.now Sidekiq::Statstic.report_success(job.class, finish - start) rescue => e finish = Time.now Sidekiq::Statstic.report_failure(job.class, finish - start) end
Would be a lot more future-proof than manually inserting values into Redis.
has been included in discourse for ages, no point keeping this open forever