How to set Email sending speed limit?

lisbethw1130 · February 7, 2020, 10:20am

I have a forum on discourse but with slow sending speed limit (20/sec)
Am I able to set sending speed limit on discourse?

I’m asking this because I found that I got nearly 500,000 records everyday on unsubscribe_keys table, but only 9000 in mail_log and 1000 in skipped_mail_log, our forum have nearly 500,000 members, too.
I’m not sure whether it’s caused by sending email too fast and make some weird problem happened, so I’m trying to throttle the sending speed but with no luck.

Unsubscribe_keys

email_log

pfaffman · February 7, 2020, 1:43pm

You need to get a faster mail server and/or change settings so that let mail gets sent.

You can change how often summary emails get sent and change the max emails per day setting.

There is no rate limit setting.

lisbethw1130 · February 7, 2020, 4:44pm

not sure whether sending rate is necessary…
because even we can increase to probably 60/sec，it’s still possible that the server send too fast and get blocked.

Stephen · February 7, 2020, 5:04pm

What are you using to send email?

lisbethw1130 · February 8, 2020, 2:08pm

I use AWS SES service

merefield · February 8, 2020, 3:43pm

Have you raised with their support?

codinghorror · February 8, 2020, 8:45pm

Why are so many Discourse members being mailed? What are they being mailed for, exactly? That’s the more relevant question.

On a typical Discourse install, the only “all users” email is the digest, and even that is specific to only users who have not been on the website in a week or more, but also haven’t been gone more than a year.

In general you only get emails when someone is speaking directly to you, and even then only when you haven’t touched the website with any web browser in the last 10 minutes.

lisbethw1130 · February 10, 2020, 3:08am

We had an activity before. (about several months ago) That activity attracted about 90 thousand members registered. And now they may leave for a few weeks. So system sent them the digest emails

j127 · June 16, 2020, 2:07am

Did you ever find a solution? I’m about to switch to Amazon SES and just realized that our initial limit is 14 emails per second. I’m not sure how quickly Discourse is going to send them into SES. The site sends a lot of email.

Does anyone know the maximum rate that Discourse sends emails? Will it try to send more than 14 per second?

If it sends too quickly, I’m wondering if it might be a good idea to write a script that evenly spaces the digest sending times for all the users across the week instead of sending the bulk of them on one day (our import anniversary).

Edit: I realized that I could figure out if this is a problem by checking the Mandrill logs. Unless I’m looking at some extra throttling from Mandrill, I think we have a maximum of about 8 emails per second, so it should be fine.

neounix · December 8, 2020, 12:54am

Hey Josh!

For digests you can limit this to 14 per second by setting this Discourse global to 420, as follows in your container build file:

 DISCOURSE_MAX_DIGESTS_ENQUEUED_PER_30_MINS_PER_SITE: 420

This is not “exactly” what you want but it is darn close.

Since 14 per second (from your post) times 30 seconds = 420, this global setting will limit digests to around 420 per every 30 minutes, just like the code “says”:

discourse/discourse/blob/main/app/jobs/scheduled/enqueue_digest_emails.rb#L37


      
              .where(
                "COALESCE(user_options.digest_after_minutes, ?) > 0",
                SiteSetting.default_email_digest_frequency,
              )
              .where("user_stats.bounce_score < ?", SiteSetting.bounce_score_threshold)
              .where("user_emails.primary")
              .where(
                "COALESCE(user_stats.digest_attempted_at, '2010-01-01') <= CURRENT_TIMESTAMP - ('1 MINUTE'::INTERVAL * COALESCE(user_options.digest_after_minutes, ?))",
                SiteSetting.default_email_digest_frequency,
              )
              .where(
                "COALESCE(last_seen_at, '2010-01-01') <= CURRENT_TIMESTAMP - ('1 MINUTE'::INTERVAL * COALESCE(user_options.digest_after_minutes, ?))",
                SiteSetting.default_email_digest_frequency,
              )
              .where(
                "COALESCE(last_seen_at, '2010-01-01') >= CURRENT_TIMESTAMP - ('1 DAY'::INTERVAL * ?)",
                SiteSetting.suppress_digest_email_after_days,
              )
              .order("user_stats.digest_attempted_at ASC NULLS FIRST")
          
          # If the site requires approval, make sure the user is approved

Naturally, if your SMTP channel is sending other emails, you will need to adjust this down; or you can set up a different, separate channel for digest emails (like we do).

Note: This global setting is set to 10,000 per 30 minutes by default; which is a lot of digests per hour if you ask me ;). So, we set this much lower so we have a “reasonable” SendGrid bill each month.

30 digests per 30 minutes = 1440 per day, or around 43,200 per month (because we were sending a lot of digests before I made this change a few days ago). Next month, I will bump this up to around 60 digests per 30 minutes, or around 86,400 digests per month plus or minus.

Hope this helps.

Topic		Replies	Views
Digests send rate Support activity-summary	2	516	August 17, 2022
Summary email limit sent in 1 hour Support	13	1360	December 11, 2020
Throttle Digest Email Support activity-summary	2	408	March 10, 2023
Just how much email does Discourse send? Hosting	12	2955	July 26, 2016
Optimize Discourse digest period Support activity-summary	12	681	April 21, 2020

How to set Email sending speed limit?

Related topics