Handling bouncing e-mails


(Régis Hanol) #1

Setting up bounces handling

Discourse uses the VERP technique to handle bouncing e-mails.

To enable VERP, set the “reply by email address” site setting with an email address template that uses the + sign. The default of

replies+%{reply_key}@discourse.example.com

… works fine. If you are using your own or your company’s email server, then, that’s it :tada:

If, however, you are using a third party email service, you will need to enable VERP, or activate their webhooks as follows:

MailGun

  • Log in to MailGun and go to your dashboard; look in the right column for API Keys.
  • in Discourse, set the mailgun api key site setting with your Secret API Key
  • go to the MailGun domains list and click the one your discourse instance is using
  • click on the webhooks tab and click the gear icon next to both Dropped messages and Hard bounces events; edit the URL of both webhooks to http://your.discourse/webhooks/mailgun

SendGrid

  • Log in to SendGrid and go to Mail Settings
  • expand the Event Notification setting and enable it if it isn’t already
  • click edit to set the HTTP POST URL to http://your.discourse/webhooks/sendgrid
  • make sure you selected the following 3 actions: Dropped, Deferred and Bounced
  • save your changes by clicking the :white_check_mark:

MailJet

  • Log in to MailJet and go to event tracking
  • check the bounce event
  • set the endpoint URL to http://your.discourse/webhooks/mailjet
  • check the :white_medium_square: in the group events column
  • click the save button

SparkPost

  • Login to SparkPost and go to your dashboard.
  • Select Account from left menu icons
  • Select Webhooks from menu ( https://app.sparkpost.com/account/webhooks )
  • Select New Webhook.
  • Enter a friendly value for Webhook Name e.g. Discourse Forum
  • Enter a Target URL as http://your.discourse/webhooks/sparkpost
  • Select Add Webhook.

Amazon Simple Email Service (AWS SES)

  • Log in to the AWS console, and go to Simple Email Service (SES)
  • Under Domains, find the one you use for your forum, and click it
  • Under the Notifications section, click “Edit Configuration”
  • All of the SNS settings can be set to “No SNS Topic”
  • Under “Email Feedback Forwarding”, choose Enabled
  • Save Config

Testing Bounces

To ensure that everything is working, do this:

  1. Sign up for a new account on your site with an obviously incorrect email, of the form bademail@obviously-invalid-domain.com or clearly.nobody.has.this.email.address@gmail.com
  2. Wait a while for the emails to fully bounce (check your provider’s logs, if you want – they will usually be shown under the “warn” or “error” log level)
  3. Check /admin/email/bounced on your site to confirm that the bounce was picked up. Bear in mind this may take up to 48 hours depending on how many retries and the particular logic of your email provider.

Configuring Bounces

Bounces handling can be customized using these site settings

  • soft bounce score: points added to the user’s “bounce score” when we receive a temporary bounce from their email address
  • hard bounce score: points added to the user’s “bounce score” when we receive a permanent (or unknown) bounce from their email address
  • reset bounce score after days: number of days during which we must receive no bounce before we can reset the user’s “bounce score” back to 0
  • bounce score threshold: score threshold at which point we will stop sending non urgent emails to the user
  • bounce score threshold deactivate: score threshold at which point we will automatically deactivate the user

How discourse manages bounces and unsubscribe while sending email notifications?
Email-in replies thread wrongly
Postmark bounce support?
Surface data about users with too many bounces
What mail service do you recommend?
Mailgun "temporarily disabled due to high bounce rate"
(AstonJ) #2

This is great @zogstrip :slight_smile:

What actually happens when an email is bounced, is the user put in to some sort of moderated state and gets displayed a notice asking them to update their email?


(Jesse Perry) #3

Thanks for explaining this. Any guidance for those of us suckers still using & paying for Mandrill?


(Jeff Atwood) #4

If you can share credentials privately with @zogstrip, perhaps; we cancelled our Mandrill account with extreme prejudice. And there is no free tier to test on…


(Jeff Atwood) #5

We should update this to be clear that accounts will automatically be set to Deactivated state after enough bounces over enough time.


(Régis Hanol) #6

Added the configure section :wink:


#7

This webhook only works for bounced emails right? No support for inbound emails from Mailgun/other providers?


(Régis Hanol) #8

It’s only for bounces because it’s the only event we lose when you’re using a mail service like Mailgun etc…


(Anton) #9

Please add configuration for SparkPost

While there is no such a configuration, is there any other easy-to-do way to handle SparkPost bouncing emails webhooks?


(Dean Taylor) #10

Support for SparkPost was recently added (28 days ago) and later fixed (8 days ago).

Pretty sure the setup instructions are as follows, this is what I’m currently testing:

SparkPost

  • Login to SparkPost and go to your dashboard.
  • Select Account from left menu icons
  • Select Webhooks from menu ( https://app.sparkpost.com/account/webhooks )
  • Select New Webhook.
  • Enter a friendly value for Webhook Name e.g. Discourse Forum
  • Enter a Target URL as http://your.discourse/webhooks/sparkpost
  • Select Add Webhook.

(Dean Taylor) #11

@zogstrip could you confirm my SparkPost configuration instructions above?

Sadly I’m not seeing expected results in Discourse.

I send a PM to a user with a invalid email address…

I can see the email sent via Discourse in /admin/email/sent.

I’m seeing the “Message Events” in SparkPost indicating a “bounce” was received:

And I’m also seeing that the Webhook was called and was “successful”. ie. 200 response received by SparkPost :

However my “Bounced” log is empty:

Any ideas? (thanks)


(Régis Hanol) #12

Could you PM me the payload sent by SparkPost? (you should be able to retrieve it from the logs)


(OG) #13

I’m little bit confused… I’m using self-hosted mail server… How do I give access to bounce catching mailbox to Discourse? Or should it be done using mail server logs?

Is it same as box for polling?


(Anton) #14

@DeanMarkTaylor, may you please share the updated configuration instructions once your problem is solved and you can get SparkPost functioning?
Many thanks.


(Dean Taylor) #15

Sure can do - if I can find it…

Obviously the access log doesn’t give away the payload info for POST requests:

[25/Oct/2016:02:39:38 +0000] 54.244.54.128 "POST /webhooks/sparkpost HTTP/1.1" "SparkPost" "-" 200 358 "-" 0.016 0.168 "-"

Is there a specific log I should be looking in?

Sure will do @meglio.


(Rafael dos Santos Silva) #16

Maybe the rails log on /var/discourse/shared/standalone/log/rails/production.log ?


(Dean Taylor) #17

Yeap - thanks @Falco

This has been done - thanks for your help.


(OG) #18

Sorry if this was mentioned else where, but reply by email address replaces From header in email or does it use separate bounce header like following one?

Errors-To: <bounced@ourdomain.com>
Return-Path: <bounced@ourdomain.com>

(Dean Taylor) #19

I can now confirm that my previous Spark Post instructions are correct and have now been added to the original post. However you will need to update to latest to get a couple of fixes from the last couple of days.

A big thanks to @zogstrip for completing those of fixes so fast!


(Anton) #20

@DeanMarkTaylor , what about Authentication and Events list (in SparkPost configuration)?