Handling bouncing e-mails

Setting up bounces handling

Discourse uses the VERP technique to handle bouncing e-mails.

To enable VERP, set the “reply by email address” site setting with an email address template that uses the + sign. The default of

replies+%{reply_key}@discourse.example.com

… works fine. If you are using your own or your company’s email server, then, that’s it :tada:

If, however, you are using a third party email service, you will need to enable VERP, or activate their webhooks as follows:

MailGun

  1. Log in to MailGun and go to your dashboard; look in the left column for the SettingsSecurity & Users page. Then go to the API Security tab.
  2. In Discourse, set the mailgun api key site setting with your HTTP webhook signing key
  3. Go to the MailGun sending domains list and click the one your Discourse instance is using
  4. Expand the “Sending” navigation by clicking the disclosure triangle next to it (if it’s not already expanded), then click the Webhooks item in the left navigation
  5. Click the “Add Webhook” button, set the “Event Type” menu to “Permanent Failure”, and set the URL to http://your.discourse/webhooks/mailgun
  6. Repeat the “Add Webhook” portion, choosing “Temporary Failure” as the “Event Type”

SendGrid

  • Log in to SendGrid and go to Mail Settings
  • expand the Event Webhook setting and enable it if it isn’t already
  • click edit to set the HTTP POST URL to http://your.discourse/webhooks/sendgrid
  • make sure you selected the following 3 actions: Dropped, Deferred and Bounced
  • save your changes by clicking the :white_check_mark:

MailJet

  • Log in to MailJet and go to event tracking
  • check the bounce event
  • set the endpoint URL to http://your.discourse/webhooks/mailjet
  • check the :white_medium_square: in the group events column
  • click the save button

SparkPost

  • Login to SparkPost and go to your dashboard.
  • Select Account from left menu icons
  • Select Webhooks from menu ( SparkPost )
  • Select New Webhook.
  • Enter a friendly value for Webhook Name e.g. Discourse Forum
  • Enter a Target URL as http://your.discourse/webhooks/sparkpost
  • Select Add Webhook.

Amazon Simple Email Service (AWS SES)

  • Configure Discourse for VERP, as described at the top of this post
  • Log in to the AWS console, and go to Simple Email Service (SES)
  • Under Domains, find the one you use for your forum, and click it
  • Under the Notifications section, click “Edit Configuration”
  • All of the SNS settings can be set to “No SNS Topic”
  • Under “Email Feedback Forwarding”, choose Enabled
  • Save Config

Postmark

  • Login to Postmark and then select the server that is used for sending Discourse emails
  • Select the message stream associated with Discourse emails, typically Default Transactional Stream
  • Click Webhooks
  • Click Add webhook
  • Enter Webhook URL as http://your.discourse/webhooks/postmark
  • Under, Which events should we send?, check Delivery and Spam complaint
  • Scroll down and click Save webhook

Testing Bounces

To ensure that everything is working, do this:

  1. Sign up for a new account on your site with an obviously incorrect email, of the form bademail@obviously-invalid-domain.com or clearly.nobody.has.this.email.address@gmail.com
  2. Wait a while for the emails to fully bounce (check your provider’s logs, if you want – they will usually be shown under the “warn” or “error” log level)
  3. Check /admin/email/bounced on your site to confirm that the bounce was picked up. Bear in mind this may take up to 48 hours depending on how many retries and the particular logic of your email provider.

Configuring Bounces

Bounces handling can be customized using these site settings

  • soft bounce score: points added to the user’s “bounce score” when we receive a temporary bounce from their email address
  • hard bounce score: points added to the user’s “bounce score” when we receive a permanent (or unknown) bounce from their email address
  • reset bounce score after days: number of days during which we must receive no bounce before we can reset the user’s “bounce score” back to 0
  • bounce score threshold: score threshold at which point we will stop sending non urgent emails to the user
44 Likes

This is great @zogstrip :slight_smile:

What actually happens when an email is bounced, is the user put in to some sort of moderated state and gets displayed a notice asking them to update their email?

1 Like

Thanks for explaining this. Any guidance for those of us suckers still using & paying for Mandrill?

If you can share credentials privately with @zogstrip, perhaps; we cancelled our Mandrill account with extreme prejudice. And there is no free tier to test on…

1 Like

We should update this to be clear that accounts will automatically be set to Deactivated state after enough bounces over enough time.

3 Likes

Added the configure section :wink:

3 Likes

This webhook only works for bounced emails right? No support for inbound emails from Mailgun/other providers?

It’s only for bounces because it’s the only event we lose when you’re using a mail service like Mailgun etc…

3 Likes

Please add configuration for SparkPost

While there is no such a configuration, is there any other easy-to-do way to handle SparkPost bouncing emails webhooks?

Support for SparkPost was recently added (28 days ago) and later fixed (8 days ago).

Pretty sure the setup instructions are as follows, this is what I’m currently testing:

SparkPost

  • Login to SparkPost and go to your dashboard.
  • Select Account from left menu icons
  • Select Webhooks from menu ( SparkPost )
  • Select New Webhook.
  • Enter a friendly value for Webhook Name e.g. Discourse Forum
  • Enter a Target URL as http://your.discourse/webhooks/sparkpost
  • Select Add Webhook.
4 Likes

@zogstrip could you confirm my SparkPost configuration instructions above?

Sadly I’m not seeing expected results in Discourse.

I send a PM to a user with a invalid email address…

I can see the email sent via Discourse in /admin/email/sent.

I’m seeing the “Message Events” in SparkPost indicating a “bounce” was received:

And I’m also seeing that the Webhook was called and was “successful”. ie. 200 response received by SparkPost :

However my “Bounced” log is empty:

Any ideas? (thanks)

4 Likes

Could you PM me the payload sent by SparkPost? (you should be able to retrieve it from the logs)

I’m little bit confused… I’m using self-hosted mail server… How do I give access to bounce catching mailbox to Discourse? Or should it be done using mail server logs?

Is it same as box for polling?

2 Likes

@DeanMarkTaylor, may you please share the updated configuration instructions once your problem is solved and you can get SparkPost functioning?
Many thanks.

Sure can do - if I can find it…

Obviously the access log doesn’t give away the payload info for POST requests:

[25/Oct/2016:02:39:38 +0000] 54.244.54.128 "POST /webhooks/sparkpost HTTP/1.1" "SparkPost" "-" 200 358 "-" 0.016 0.168 "-"

Is there a specific log I should be looking in?

Sure will do @meglio.

Maybe the rails log on /var/discourse/shared/standalone/log/rails/production.log ?

2 Likes

Yeap - thanks @Falco

This has been done - thanks for your help.

3 Likes

Sorry if this was mentioned else where, but reply by email address replaces From header in email or does it use separate bounce header like following one?

Errors-To: <bounced@ourdomain.com>
Return-Path: <bounced@ourdomain.com>

I can now confirm that my previous Spark Post instructions are correct and have now been added to the original post. However you will need to update to latest to get a couple of fixes from the last couple of days.

A big thanks to @zogstrip for completing those of fixes so fast!

3 Likes

@DeanMarkTaylor , what about Authentication and Events list (in SparkPost configuration)?