We have an #Announcements category in which all our members are automatically subscribed to watch the first post. This allows us to schedule the publishing of posts in to that category on set times/dates. In turn, the announcement is then emailed to around 25k members. The problem we face is that the emails take over an hour to get sent, not ideal for time-critical announcements. If I watch Sidekiq I can see the “Scheduled” counter racking up all the individual emails one by one, once it gets to around 20,000 it then moves them over to the “Enqueued” tab, and then they eventually start sending out. Am I able to speed this process up at all? :thinking: For time critical emails it’d be nice to send them out around 100x faster than they currently go :blush:

[image] Richie: If I watch Sidekiq I can see the “Scheduled” counter racking up all the individual emails one by one, once it gets to around 20,000 it then moves them over to the “Enqueued” tab, and then they eventually start sending out. I timed it this evening. I scheduled a post to publish in to the #Announcements category at 18:30. At 18:40 there were 15,000 emails in the Scheduled queue. [IMG_3559.PNG] At 18:45, so 15 minutes after the post, there were 22,000 emails in the Scheduled queue. [IMG_3564.PNG] At 18:48 those emails then gradually started moving over to the Enqueued queue: [IMG_3568.PNG] At 18:51 they were still moving over: [IMG_3570.PNG] At 19:03 the emails were on their way out: [IMG_3571.PNG] By 19:10 there were just 10,000 emails remaining: [IMG_3572.PNG] At 19:27 there were just 569 emails remaining: [IMG_3574.PNG] And at 19:29 all the emails had gone out: [IMG_3575.PNG] So there we have it, one full hour to send 22,000 email notifications. Can anyone help me identify the bottleneck here? I’d very much like to be able to send these emails out faster than their current 22000-per-hour rate.

Shooting from the hip, Is it possible it’s an infrastructure load issue?

Perhaps? I genuinely don’t know :person_shrugging: The CPU spiked to around 44% on the server: [Screenshot 2025-09-18 at 22.17.08] And I use AWS SES for the SMTP.

There’s two things going on here: delay while email jobs are enqueued processing time for sending the actual email For the first, I’m not 100% confident on this but I think lowering email_time_window_mins means the notifications get queued sooner. Once the email jobs are scheduled, your sidekiq workers are working through them one at a time. Bumping up the sidekiq workers (set DISCOURSE_SIDEKIQ_WORKERS up from 5 to 10, 15, or 20 depending on server capacity) means more jobs get processed at the same time, so the queue gets emptied 2x/3x/4x faster.

[image] supermathie: For the first, I’m not 100% confident on this but I think lowering email_time_window_mins means the notifications get queued sooner. I don’t know how the backend works in the finest detail, but the email_time_window_mins is just a delay setting before the first email goes out. During this time, any user that is set to receive the email may “forfeit” the email if they happen to be active within the window of period (official terminology: email skipped; skip reasoning: User was seen recently). The default delay is 10 minutes, meaning the post must be live for 10 minutes, and only then will the emails be sent. Richie’s issue is the time difference between the first email and the last email… a delay of one hour. This is probably due to the sheer amount of emails that need to be sent, though I cannot say for certain either. Changing the setting above would only expedite the sending of the first email, but not address the completion duration of the entire batch of 22,000 emails. [image] supermathie: DISCOURSE_SIDEKIQ_WORKERS What would be the recommended setting, obviously dependent on infrastructure capability? Could a high setting result in server issues in terms of load or other?

[image] TempAccount: What would be the recommended setting, obviously dependent on infrastructure capability? Could a high setting result in server issues in terms of load or other? “As many as the server can handle”. It’s 100% dependent on the OP’s server capacity - too many and it’ll slow things down, too few and it’ll take longer to process. Given the CPU graph hitting 40% (is that of a single CPU or total capacity?) I’d probably start with bumping it either 2× (conservative) or 3× (aggressive) and see what happens, depending on risk tolerance for slowdowns.

[image] supermathie: Once the email jobs are scheduled, your sidekiq workers are working through them one at a time. Bumping up the sidekiq workers (set DISCOURSE_SIDEKIQ_WORKERS up from 5 to 10, 15, or 20 depending on server capacity) means more jobs get processed at the same time, so the queue gets emptied 2x/3x/4x faster. Great insight, thank you :slight_smile: Does that DISCOURSE_SIDEKIQ_WORKERS have to be a multiple of 5? Could I set it to 7 for example? I don’t have that parameter setting in my app.yml so I’ll assume it’s on a default of 5 somewhere. Can I just create that setting under the existing unicorn workers setting, then rebuild? Eg: expose: - “443:443” env: UNICORN_WORKERS: 8 DISCOURSE_SIDEKIQ_WORKERS: 7 Is it as simple as that? :thinking:

[image] Richie: Can I just create that setting under the existing unicorn workers setting, then rebuild? Eg: expose: - “443:443” env: UNICORN_WORKERS: 8 DISCOURSE_SIDEKIQ_WORKERS: 7 Is it as simple as that? :thinking: Is anyone able to confirm this is the correct change to make to my app.yml when the DISCOURSE_SIDEKIQ_WORKERS parameter does not currently exist?

I believe yes, given a brief inquiry to the AI bot. [image] Set Environmental Variables Self-Hosting notebook_with_decorative_cover This is a #how-to guide for setting up environmental variables in a self-hosted Discourse installation. person_raising_hand Required user level: Administrator Environmental variables play a crucial role in configuring your Discourse instance. They can store sensitive data like API keys and database passwords, making your installation more secure and flexible. In Discourse, environmental variables are set in the app.yml file within your Docker container setup…

Thanks @TempAccount Interestingly, DISCOURSE_SIDEKIQ_WORKERS only appears eleven times across the whole of meta: https://meta.discourse.org/search?q=%22DISCOURSE_SIDEKIQ_WORKERS%22%20order%3Alatest_topic …and one of those is in this topic :flushed_face:

Is there a way I can send email notifications faster?

Product Support Self-hosting

Richie (Richie Rich) October 20, 2025, 4:53pm 21

Ohhhhhh wait a minute. Is Discourse limiting itself somehow?

Have I missed a setting that’s restricting its CPU usage??

Topic		Replies	Views
How to set Email sending speed limit? Product Support	9	2409	December 8, 2020
Large number of queued jobs Product Support	5	1689	July 3, 2017
Send Emails immediately for new Post Product Support	3	818	January 23, 2022
Throttle Digest Email Product Support activity-summary	2	547	March 10, 2023
Discourse not sending Emails to All Users in mailing list mode Self-hosting	22	1541	April 1, 2020

Is there a way I can send email notifications faster?

Related topics