Handling bouncing e-mails

I’ve set up Mailgun hooks as described above, but I sometimes also get incoming bounce emails to an address like bounce+5215e4.9df4bb-user=domain@intfiction.org The email subject will be something like “Undeliverable: Topic title”, and the user/domain is for a real user’s email address that was sent that email. But Discourse doesn’t seem to do anything with these incoming bounce emails. Do I need to enable something?

Mailgun recently (2019-03-12) released a new Webhooks API, and the instructions here don’t seem applicable any longer:

Should we continue to use the legacy webhooks & API? If so, how do we set those up with the changes to Mailgun’s web UI? I realize this is more of a question for Mailgun support, but first I want to know if Discourse has added support for the new webhooks yet.

[edit]
Nevermind, looks like support was added here:

[/edit]

@zogstrip per my message above, I recommend editing the

to this:

1 Like

Thanks Jeff! The OP is a wiki, feel free to edit your changes right in.

3 Likes

I’m using mailgun. Could someone please help me? I think the instructions are out of date.

So I go into /admin/site_settings/category/email (this could be mentioned in the instructions) and set

“reply by email address”

to

replies+%{reply_key}@discourse.example.com

replacing discourse.example.com with my own domain that points to my discourse instance.

After that I go to Settings in mailgun, API Security. I think the user interface must have changed, because we now have Private API key, Public validation key, and HTTP webhook signing key. Which one do we set to mailgun api key?

Regarding http://your.discourse/webhooks/mailgun, should this be, for example, https://discourse.example.com/webhooks/mailgun, where discourse.example.com is your own domain pointing to your instance? (Also, as I understand it, the install process now gives you https automatically. Can you run a discourse instance without https? Not that I would.)

Thank you for helping me.

1 Like

Could you please let me know which of the three keys in Mailgun I need to use to configure bounce handling? I think the user interface in Mailgun has changed so it’s no longer obvious.

1 Like

I think I figured it out. I used the HTTP webhook signing key in Mailgun, which is under API Security in Settings. It seems to be working. I can see the bounced email in the admin dashboard in my Discourse instance. I also didn’t need to set the reply by email address site setting.

4 Likes

I had to cycle my API keys, and now am not getting any bounce reports. I too think you need to use the “HTTP webhook signing key” instead. But when I try to set that for the “mailgun api key” setting, it won’t let me, saying “mailgun_api_key: Value doesn’t match the required format.”

This commit added support for the new general API key: Support new mailgun's API key format · discourse/discourse@474a01f · GitHub

I’d guess it also needs to support the HTTP webhook signing key, which is a plain 32 digit hexadecimal number.

3 Likes

PR ready (this was an easy one):

4 Likes

I added incoming mail server using this tutorial Straightforward direct-delivery incoming mail
It seems to work fine, I can reply by email, but when I tried to sign up with an invalid email I only got one failure in Mailgun logs (which says that envelope.sender is replies+verp-3e19668e2fcbd604f59a9964c79151ad@reply.example.com, as expected) but I don’t see any incoming emails from the invalid address in /admin/email. Is this normal?

It might show up at admin/email/rejected. But I’m not clear on what is happening in your description. Are you saying you signed up with an invalid email address, and you want to see the failure from Mailgun in Discourse?

If so, double-check your webhooks, via:

1 Like

As I understand I don’t need webhooks if I set up my own server for incoming emails via Straightforward direct-delivery incoming mail? Because it should arrive to it and pushed to Discourse via API.

I don’t know what supposed to happen, I just want Discourse to handle the bouncing emails correctly, that is stop sending to such emails.

You mentioned Mailgun logs, so I thought you were using Mailgun. If you want Mailgun to inform Discourse about bounces, you need to configure webhooks, per the directions in this topic.

I’m not sure what you ought to do in your case; but it also isn’t clear how it is setup. :thinking:

2 Likes

I am using Mailgun to send emails from Discourse, and the server linked above to receive emails.

2 Likes

If you want Discourse to stop sending mail after hitting a bounce threshold, you need a webhook from Mailgun to fire at your Discourse, and then configure the threshold in the settings.

I’m unsure what this all means at this point, but it sounds like you have two questions:

  1. How to handle bounced email, and
  2. Seeing all mail logs from incoming mail

These are two different things. This topic here is for setting up webhooks so your outgoing mail providers let Discourse know what’s bouncing.

The topic you followed (at Straightforward direct-delivery incoming mail) has a troubleshooting section. But receiving mail to your server and bouncing messages sent from your server are two separate issues.

4 Likes

It means that the incoming mail server can receive incoming emails. As I understand, it should be enough to handle bounces? Or am I missing something and I still need webhooks?
As I understand, to handle bounces I just need to set the “reply by” address and then when the delivery fails my server will receive a reply (with the verp key in the address) and push it to Discourse.

An email can fail to be delivered because of many different reasons, and there are several methods those failures can be reported back to your forum. Most of the time the sending email service is able to detect the failure as it happens, this is what the webhooks are for. (Possibly Mailgun etc use VERP internally, I’m not sure).

VERP is another method which can handle bounces when Mailgun etc think the email was successfully sent but the final target mail server rejects it. I’m not sure if Mailgun etc will decide that the email was successfully sent after only one email relay succeeds or if it can detect failures further away. I’m also not sure when VERP will be used; from my experience it is not a common method. And, unfortunately, it seems to me that it isn’t actually handled by Discourse (at least not if you’re also using Mailgun and webhooks).

So yes, you need to set up the webhooks or else you will miss out on 99.5% of your bounces.

1 Like