Is it possible to optimize the digest period per thousands of community? Every Monday Discourse sends a few thousand emails in 1-2 seconds which leads to SPAM problem with our custom mail server.
How to reset “last_digest_time” per account and normalize it to the other thousand accounts?
The goal is to have 100-200 digests per day instead of 5K on Monday. We’ve been using Discourse for the last 4 years, maybe there is a problem with an upgrade which we do very often to the latest release.
Also, maybe there is a hint on how to access Postgress of Docker image and where “to update” records in DB.
What are your ideas? Server just sends letters to the destinations where they put our letter into SPAM. I would do the same if someone sent me 2k letters in 1 sec.
If you’re hosting your own SMTP then you’re falling into the usual associated pitfalls. Mail delivery companies like mailgun and sendgrid use a number of techniques to ensure their servers are trusted and seen as reputable by destination networks. That’s why the documentation recommends you use one of those products over self-hosting.
Exactly - while you only send those messages in spikes, reputable third party mailing services can reliably deliver hundreds of thousands of emails into third party mailboxes every hour.
Unfortunately we can’t offer any assistance if you insist on running your own mail server. If you decide not follow the recommendations, that you accept the additional complexity it creates.
Everything is OK with SMTP including DMARC, SPF, DKIM. I’ve been using own SMTP since the 2005 year. We did not have useless “money machine” services that time. Let’s imagine that tomorrow someone will try to charge for the air…
So, let’s back to the problem. I know that the easiest way for Senior Tester is to forward me to somewhere or something paid. Could someone provide a help on how to fix a critical bug in Discourse? I still do not agree that Discourse should send 10K letters in 1 sec. It should be rotation service or similar to this which normalizes “last_digest_timestamp” between accounts.
I’m not employed by CDCK, nor do I have any financial interest in pointing you to any third party services.
If you won’t want to follow the recommendation and documentation that’s your call. In this case it leads you off the path of the free support we provide here to the community, so your only other option is to engage with a consultant via the marketplace.
This isn’t a Discourse problem, this issue is down to your mail server IP not being seen as reputable. We can’t help you fix that and you refuse to consider the options which exist for that purpose.
Yes, I work with communities who send ten times that volume, many of the people running large communities operate at that scale or greater.
Your needs of email appear to now be exceeding your expertise with the protocol to reliably delivering email in volume, that’s why these transactional email companies exist. Nobody here is shilling for ‘big email’ - it’s just the challenge of delivering high-volume email.
You can have your mail server hold the mail and spread out the delivery. This really is an issue of your mail server rather than discourse. I ran mail servers from the early 1990s until around 200 5 when spam made it to hard. It’s much harder now.
I have some rate limits. If you even do not do this, typical SMTP servers have preconfigured settings for you. The mail servers are smarter today and can easily detect your delay between letters.
Yes, because they use the service which you mentioned before. You still do not understand the difference between those services and personal SMTP service. They use huge pool of IPs and simulate different senders. This is a good technique for companies that do “hard” SPAM. Of course, they will pay for this service.
I don’t see a reason why people should use these “money machines” for Discourse if Discourse does not send SPAM. Ops… sorry… due to a bug, it can send 100K on Monday in 1sec because of @Stephensays that this is TRUE behavior.
Ok, I don’t see sense to chat with people who have Crown on top of the head. Yes, Discourse is cool and thank you so much for your contribution.
I hope simple people without will have some ideas here.
According to the answers above from Discourse team they do not see any issues with Discourse. I resolve this issue in a dirty way with direct hacking of database. Here is my solution
cd /var/discourse
./launcher enter app
rails c
Topic.exec_sql("UPDATE users SET last_emailed_at = now() + interval '30' second * id WHERE last_emailed_at > ('2020-03-14'::date) AND last_emailed_at < ('2020-03-17'::date)")
Please update last_emailed_at range in SQL query. I had 4K last_emailed_at in 1 day on 16 of March. So, now all users were optimized to 3 days with interval 30 seconds.
Discourse is not designed to be used with a personal SMTP service. Sorry that wasn’t explained clearly; it can definitely work, but for a community with regular email activity it is not a good idea. This is a limitation of the state of email, and we are all reacting to it as we can.
on top of the head. Yes, Discourse is cool and thank you so much for your contribution.