More then 3 million enqueued jobs show in sidekiq? How to I stop it generate the queue?

Support
1

now 3k after I just cleaned all of them

The image shows a collection of jobs for a chat summary, with details about user IDs and force respect seen recently in settings. (Captioned by AI)
The image shows a collection of jobs for a chat summary, with details about user IDs and force respect seen recently in settings. (Captioned by AI)2452×342 112 KB

I try to look up any chat or summary settings in the admin dashboard but nothing was found.

Any idea where they come from? :sob:

P.s. those jobs make my site extremely slow :sob:

1 Like
2

What is in Retries tab of sidekiq?

The image shows the state of a task queue with 37,170.868 processed tasks, 27,055 failed tasks, 6 enqueued tasks, 4 scheduled tasks, and 4 dead tasks. (Captioned by AI)
The image shows the state of a task queue with 37,170.868 processed tasks, 27,055 failed tasks, 6 enqueued tasks, 4 scheduled tasks, and 4 dead tasks. (Captioned by AI)1356×120 15.2 KB

Here there are 6 jobs being retried. That’s an ok number and I know why.

If this is in the 1000’s that’s not good and you need to take action.

Have you got any errors in Email Logs or the Server Error Logs /logs?

Things to think about:

Is your email service provider throttling you? Have you exceeded their rate limits for the level of service you are on? Have you paid your bill with them?

3 Likes
3

A summary of job statuses shows 11,222,546 processed tasks, 6 scheduled, and no retries found. (Captioned by AI)
A summary of job statuses shows 11,222,546 processed tasks, 6 scheduled, and no retries found. (Captioned by AI)2486×602 120 KB

it’s zero

A screenshot displaying a log file with various system messages and error notifications. (Captioned by AI)
A screenshot displaying a log file with various system messages and error notifications. (Captioned by AI)1920×1064 224 KB

and doesn’t seem to have email-related issues in the log.

Is there any setting related to chat summary I can turn off or I should just turn of the chat ?

4

Is the entire queue all UserEmail jobs (for the most part)?

Check your email service dashboard (over at your email service provider, not on Discourse) and see if there is any information there

1 Like
5

yes, almost all of them are user email job

I checked mailgun (4 running instances under my account)

the first one is the only one that runs into this issue and the date doesn’t seem to be bad, I checked logs too but found nothing suspicious either.

A table displaying various metrics for different mail domains, such as percentage of accepted, delivered, bounced, opened, clicked, and complained emails. (Captioned by AI)
A table displaying various metrics for different mail domains, such as percentage of accepted, delivered, bounced, opened, clicked, and complained emails. (Captioned by AI)2536×960 117 KB

6

Do they have an audit trail, can you see a tick-over of continuously sent mails?

Last email sent?

7

A table displaying email delivery status, with all emails marked as "Delivered" or "Accepted" with the message "OK" or "(...)". (Captioned by AI)
A table displaying email delivery status, with all emails marked as "Delivered" or "Accepted" with the message "OK" or "(...)". (Captioned by AI)2518×1396 175 KB

like this?

and the last one should be 12 mins ago

8

So some are getting through, but is it processing them fast enough?

9

how do i check that?

on discourse (admin/email/sent) I saw the chat summary email was send with a low frequency like at most 5-6 per day, most time only 1 or none. :sob: how come it create so many jobs in sidekiq?

10

Take a look at your email logs in Admin … that might give you more clarity

11

One thing I noticed is that the sidekiq queue issue started right after I upgraded to 3.4.0.beta4-dev, the other instance I installed around the same time seems to have the issue too (confirming now)

also looked at the ids, it seems the system is trying to send chat summary to every users :thinking:
is it normal?

A table displaying the current jobs in low priority, including their arguments. (Captioned by AI)
A table displaying the current jobs in low priority, including their arguments. (Captioned by AI)1060×1128 91.2 KB

update:
Turning off chat prevents those jobs from being queued again, but my site kind of needs the chat function, and I am still looking into other solutions.

another update:
I turned off the chat function for 2 days, turned it on again, and changed the Chat channel retention days from 180 to 3 days. Now it seems to work fine, will keep an eye on it still.

4 Likes