I recently updated Discourse from version 3.4.0.beta1 to 3.4.0.beta3. After the update, we are noticing that the forum’s memory usage is gradually increasing, causing the application to go down. When checking the server, we can see that the redis-server is consuming 95% of the memory. We are running redis-cli flushall daily to temporarily fix the issue. The Discourse instance is hosted in Docker. I tried to downgrade to the previous version, but it was throwing a few errors while rebuilding. [image]

Sorry about that @Siraj_Moideen - it was indeed a bug that will be fixed by https://github.com/discourse/discourse/pull/31133 We have a query that list all the “users with unread” mentions or dms for chat, and, in trying to be too clever, it ended being wrong. Instead of enqueue a “chat summary” (aka. email notification) for mentions in channels or unread in dms, it was enqueueing a job for every messages :expressionless: Since that job ran every 5 minutes, it easily bloated sidekiq with useless jobs… (note: the email were thankfully not being sent as we would not be able to retrieve any mentions).

You cannot downgrade Discourse. Redis is not using 95% of the memory but 38.9%. Still a lot. What does your Sidekiq queue look like? /sidekiq/queues

Please find the details of /sidekiq/queues [image] Let me know if you need any other details

Are these emails jobs by any chance?

I doubt it. How can I check it?

[image] Siraj Moideen: May I know if you’re talking about this queues section? Yes, just click on the word “low” [image]

Please find the details below [image]

There’s an identical issue here: [image] More then 3 million enqueued jobs show in sidekiq? How to I stop it generate the queue? Support One thing I noticed is that the sidekiq queue issue started right after I upgraded to 3.4.0.beta4-dev, the other instance I installed around the same time seems to have the issue too (confirming now) also looked at the ids, it seems the system is trying to send chat summary to every users thinking is it normal? [A table displaying the current jobs in low priority, including their arguments. (Captioned by AI)] update: Turning off chat prevents those jobs from being queued again, but my si… With maybe a fix: [image]More then 3 million enqueued jobs show in sidekiq? How to I stop it generate the queue? I turned off the chat function for 2 days, turned it on again, and changed the Chat channel retention days from 180 to 3 days. Now it seems to work fine, will keep an eye on it still. Since you’re not the only one to experience this, that looks like a bug. :thinking:

Thanks, but I don’t think I can turn off the chat. I’ll try to figure out another way

Redis Memory Keeps Increasing in Discourse 3.4.0.beta3

Product Support

Siraj_Moideen (Siraj Moideen) January 16, 2025, 11:59am 9

May I know if you’re talking about this queues section?

One more thing is under /sidekiq/scheduler/history I can find Jobs::Chat::EmailNotifications still running for a longer period

Topic		Replies	Views
Redis Memory Keeps Increasing in Discourse 3.4.0.beta3 due to chat plugin bug Bug	2	221	February 3, 2025
Memory creep in last couple of updates Self-hosting hosting	27	3243	May 21, 2019
Sidekiq is consuming too much memory, restarting Self-hosting	38	9543	October 13, 2020
High Memory usage without traffic? Product Support	10	1369	June 1, 2022
Extreme memory usage due to bad mail credentials Self-hosting server-resources	14	1321	June 12, 2019

Redis Memory Keeps Increasing in Discourse 3.4.0.beta3

Related topics