Forum updated, several bugs (sidekiq not running)

AI-generated summary

After updating to the February 2025 stable version, Shauny reported several bugs including missing notifications, manual page refreshes required, and inaccurate ‘Who’s Online’ counts. david suspected Sidekiq (background job processor) wasn’t running properly and asked for log and Sidekiq dashboard checks. Shauny shared error logs and a Sidekiq chart showing stalled jobs; after visiting the dashboard, notifications resumed — though later attributed to their host restarting the service Shauny. david noted that a high ‘enqueued’ job count suggests backlog, and confirmed restarting Sidekiq often resolves it. michaeld confirmed Sidekiq was stuck with ~1000 queued jobs post-update; restarting fixed it, suspecting the ‘Who’s Online’ plugin. mentalstring noted a similar Sidekiq loop issue and shared a screenshot of a related warning. mcdanlj also experienced stalled Sidekiq jobs (21 hours old, no new processing), fixed by a system restart; noted a dashboard warning about ‘Sidekiq not running’ that may have been overlooked. Visiting the dashboard alone didn’t fix it — restarts were key. Multiple users suspect the ‘Who’s Online’ plugin may be contributing to the issue.

Our forum updated 2 days ago to the “February 2025 stable” version (not sure how to find the exact version number).

Our members have noticed several bugs:

  • People aren’t getting notifications for replies.
  • People are having to manually refresh the page to see replies
  • The “Who’s Online” plugin is misbehaving, saying there are over 50 people online when there definitely aren’t.

How can we fix this? Is there going to be a hot fix or something?

Thanks
Shaun

We haven’t seen these issues on our hosting, or from any other self-hosters, so this is likely a problem specific to your server. We don’t have any relevant fixes in the pipeline.

From the symptoms, it sounds like Sidekiq (the background job processor) is not running correctly.

Do you see any recent errors when visiting /logs on your forum?

How does /sidekiq look? Does it show jobs being processed?

There are some recent errors:

Sidekiq looks like it’s working?

(Edit: actually I see it’s not working… never viewed this before lol)

I just got loads of notifications, seems it was stuck? Did visiting sidekiq give it a kick?

Ah it seems my host replied to my message and they gave it a kick… I’ll see what they say. Thanks!!

Interesting! To be honest I’m surprised that visiting the dashboard would give it a kick, but it’s possible!

What do the numbers look like at the top of the dashboard:

In a healthy state, the ‘enqueued’ number should be close to zero. I suspect if things have been stuck, your number is very high, and may take some time to process down to 0.

Ah great!

Sidekiq was (seemingly happily) running, but it seemed to be stuck since the update to 3.4 with about 1000 jobs queued. Since updates often do rebakes or async things, we didn’t really see that as something suspicious.

A restart of the process did the trick. I suspect the who’s online plugin might be the culprit but I’m not 100% sure.

Likely unrelated, but we also bumped into something that may also be sidekiq misbehaving after a recent update.

Also,

Coincidence or not, one of our admins saw this in the last 24h, but I haven’t been able to reproduce it yet so it may have been a single glitch.

I also ran into this problem.

The sidekiq dashboard showed nothing active, all five job slots were filled with 21-hour-old jobs, so sidekiq wasn’t pulling any new jobs of any sort, and the enqueued count was growing. I needed to apply system updates anyway, so I just restarted the system, which fixed the the problem.

The who’s online queue growing was my first tip-off.

Visiting the sidekiq dashboard did not “give it a kick”

Additionally, I noticed this warning on the Discourse dashboard:

I’m wondering whether that meant that there’s a warning that just didn’t show in the UI somewhere. I didn’t notice that warning immediately after my last update, to 176ee0bf60. I don’t check the dashboard often enough to know when it would have shown up.