Is there a way I can send email notifications faster?

[Richie (Richie Rich)]

We have an #Announcements category in which all our members are automatically subscribed to watch the first post.

This allows us to schedule the publishing of posts in to that category on set times/dates.

In turn, the announcement is then emailed to around 25k members.

The problem we face is that the emails take over an hour to get sent, not ideal for time-critical announcements.

If I watch Sidekiq I can see the “Scheduled” counter racking up all the individual emails one by one, once it gets to around 20,000 it then moves them over to the “Enqueued” tab, and then they eventually start sending out.

Am I able to speed this process up at all?

For time critical emails it’d be nice to send them out around 100x faster than they currently go

[jahan_gagan]

How to set Email sending speed limit?

I have a forum on discourse but with slow sending speed limit (20/sec) Am I able to set sending speed limit on discourse? I’m asking this because I found that I got nearly 500,000 records everyday on unsubscribe_keys table, but only 9000 in mail_log and 1000 in skipped_mail_log, our forum have nearly 500,000 members, too. I’m not sure whether it’s caused by sending email too fast and make some weird problem happened, so I’m trying to throttle the sending speed but with no luck. Unsubscribe_k…

This discussion may help you, as it explains how to set DISCOURSE_MAX_DIGESTS_ENQUEUED_PER_30_MINS_PER_SITE, which defines the global limit for digests.

[Richie (Richie Rich)]

@Richie

If I watch Sidekiq I can see the “Scheduled” counter racking up all the individual emails one by one, once it gets to around 20,000 it then moves them over to the “Enqueued” tab, and then they eventually start sending out.

I timed it this evening.

I scheduled a post to publish in to the #Announcements category at 18:30.

---

At 18:40 there were 15,000 emails in the Scheduled queue.

---

At 18:45, so 15 minutes after the post, there were 22,000 emails in the Scheduled queue.

---

At 18:48 those emails then gradually started moving over to the Enqueued queue:

---

At 18:51 they were still moving over:

---

At 19:03 the emails were on their way out:

---

By 19:10 there were just 10,000 emails remaining:

---

At 19:27 there were just 569 emails remaining:

---

And at 19:29 all the emails had gone out:

---

So there we have it, one full hour to send 22,000 email notifications.

Can anyone help me identify the bottleneck here?

I’d very much like to be able to send these emails out faster than their current 22000-per-hour rate.

[TempAccount]

Shooting from the hip,
Is it possible it’s an infrastructure load issue?

[Richie (Richie Rich)]

Perhaps?

I genuinely don’t know

The CPU spiked to around 44% on the server:

And I use AWS SES for the SMTP.