Bad CSRF with mailgun replies

[Jeff_Vienneau]

Hi,

I am trying to setup inbound email handling with mailgun and the http post API.

Things I have configured:
Mailgun API key.
manual polling enabled = checked
reply email address replay@mydomain
reply by email enabled = checked

Added a mailgun route:
match_recipient(".*@mydomain")
forward"(http://mydomain/admin/email/handle_mail&type=mime")

MX record and am receiving replys in mailgun logs.

Is there anything else I need to configure to fix “BAD CSRF”?
Do I need a plugin or API web hook or something?

[Jeff_Vienneau]

Does inbound mail use “handle_mail” endpoint when configured for mailgun?

[Jeff_Vienneau]

I think I solved the Bad CSRF issue by adding user and api key to the URL but now I am seeing and error in the logs: admin/email_controller.rb error at line 86, missing required parameter "email’.

Can I assume this is a json parameter that mailgun is not populating because of wrong API or something?

[Jeff_Vienneau]

The logs:

/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.8/lib/action_controller/metal/strong_parameters.rb:251:in `require'
/var/www/discourse/app/controllers/admin/email_controller.rb:86:in `handle_mail'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.8/lib/action_controller/metal/implicit_render.rb:4:in `send_action'

[DeanMarkTaylor (Dean Taylor)]

TL;DR:

I don’t believe Mailgun’s mail forwarding will work as-is with Discourse. It doesn’t pass the email data in the right way and there doesn’t appear to be a way to change that on either end Mailgun nor Discourse.

---

The detail

Within Mailgun’s documentation there is a Django code example which indicates what is passed to the HTTP endpoint by mailguns forward functionality.

An example payload at the bottom of this page confirms this.

For reference here is the code sample:

Consider this Django code:

# Handler for HTTP POST to http://myhost.com/messages for the route defined above
def on_incoming_message(request):
     if request.method == 'POST':
         sender    = request.POST.get('sender')
         recipient = request.POST.get('recipient')
         subject   = request.POST.get('subject', '')

         body_plain = request.POST.get('body-plain', '')
         body_without_quotes = request.POST.get('stripped-text', '')
         # note: other MIME headers are also posted here...

         # attachments:
         for key in request.FILES:
             file = request.FILES[key]
             # do something with the file

     # Returned text is ignored but HTTP status code matters:
     # Mailgun wants to see 2xx, otherwise it will make another attempt in 5 minutes
     return HttpResponse('OK')

Note that this means that many POST parameters including sender, recipient, subject, body-plain, stripped-text and files as separate “chunks”.

This is not what Discourse expects to be sent to the /admin/email/handle_mail endpoint - it simply wants the complete raw email as the email parameter.

[Jeff_Vienneau]

Thanks a bunch, @DeanMarkTaylor.

I’ll pursue different method then.

[jomaxro (Joshua Rosenfeld)]

Take a look at

Direct-delivery incoming email for self-hosted sites

Discourse is all about enabling civilized discussion. While plenty of people like a web interface, e-mail is still the “hub” of many people’s online lives. That’s why sending e-mail is so important, and when you’re sending e-mail, you really want to be able to receive it, too. There are several reasons why: If e-mails “bounce” (they can’t be delivered for some reason), you need to know about that. Repeatedly sending e-mails that bounce will get your e-mails flagged as spam. Receiving e-ma…

[dfriestedt (Drew Friestedt)]

I followed these directions exactly, and am getting the following error at mailgun. Should I assume that I need to migrate away from mailgun to receive reply emails?

Also, I’m confused what the mailgun api key is used for. Says it’s used to verify webhook messages. Can someone elaborate?

{
"severity": "temporary",
"tags": [],
"storage": {
    "url": "https://se.api.mailgun.net/v3/domains/reply.volyoom.fit/messages/eyJwIjpmYWxzZSwiayI6IjhiYTFjMTI0LWVhNmQtNDcxZC05MzU5LTNiNzhjZGY2MGJmYSIsInMiOiJkNDFhMjFlMDRmIiwiYyI6InRhbmtiIn0=",
    "key": "REMOVED"
},
"delivery-status": {
    "attempt-no": 1,
    "description": "[\"BAD CSRF\"]",
    "session-seconds": 0.07819604873657227,
    "retry-seconds": 600,
    "code": 403,
    "message": "[\"BAD CSRF\"]"
},
"recipient-domain": "REMOVED",
"id": "QRcFjJ3NSKu0Hn9pfPnGyQ",
"campaigns": [],
"reason": "generic",
"user-variables": {},
"flags": {
    "is-routed": false,
    "is-authenticated": false,
    "is-system-test": false,
    "is-test-mode": false
},
"log-level": "warn",
"timestamp": 1508087126.237296,
"envelope": {
    "sender": "REMOVED",
    "transport": "http",
    "targets": "REMOVED/admin/email/handle_mail"
},
"message": {
    "headers": {
        "to": "REMOVED Forum <replies+ddcd9246b485715a803756190c86e6e5@REMOVED>",
        "message-id": "ADBE049D-0179-4B3C-813A-517A02741FBD@monroeresidential.com",
        "from": "Drew Friestedt <REMOVED>",
        "subject": "Re: [REMOVED Forum] [Zwift] This is a test topic with email reply"
    },
    "attachments": [],
    "size": 7044
},
"recipient": "REMOVED/admin/email/handle_mail",
"event": "failed"

[DeanMarkTaylor (Dean Taylor)]

Should I assume that I need to migrate away from mailgun to receive reply emails?

Yes, see my earlier post:

I don’t believe Mailgun’s mail forwarding will work as-is with Discourse. It doesn’t pass the email data in the right way and there doesn’t appear to be a way to change that on either end Mailgun nor Discourse.

[dfriestedt (Drew Friestedt)]

Got it. I did see that but was hoping it was wrong. Can you recommend a solution that does work to deliver reply messages with mail-receiver?

[DeanMarkTaylor (Dean Taylor)]

@dfriestedt

Can you recommend a solution that does work to deliver reply messages with mail-receiver?

The recommended solution has already been posted too:

Bad CSRF with mailgun replies

Take a look at https://meta.discourse.org/t/straightforward-direct-delivery-incoming-mail/49487

Direct-delivery incoming email for self-hosted sites

Discourse is all about enabling civilized discussion. While plenty of people like a web interface, e-mail is still the “hub” of many people’s online lives. That’s why sending e-mail is so important, and when you’re sending e-mail, you really want to be able to receive it, too. There are several reasons why: If e-mails “bounce” (they can’t be delivered for some reason), you need to know about that. Repeatedly sending e-mails that bounce will get your e-mails flagged as spam. Receiving e-ma…

[dfriestedt (Drew Friestedt)]

I did follow those directions exactly. I’m getting the following message in the log file, so it’s working perfectly.

<22>Oct 15 16:29:08 postfix/master[1]: daemon started – version 3.1.1, configuration /etc/postfix

However, mailgun is still unable to deliver the messages. The log file posted above is from mailgun after setting up mail-receiver.

[dfriestedt (Drew Friestedt)]

I’m reviewing my other setting to make sure I have not missed anything:
manual polling enabled = True
mailgun api key = Provided
reply by email enabled = True
reply by email address = replies+%{reply_key}@example.com
at mailgun - Forward to: http://example.com/admin/email/handle_mail

[DeanMarkTaylor (Dean Taylor)]

I did follow those directions exactly.

However, mailgun is still unable to deliver the messages.

at mailgun - Forward to: http://example.com/admin/email/handle_mail

Please re-read the instructions linked as they have nothing to do with mailgun - mailgun will not be used for receiving mail when following those instructions correctly.

[dfriestedt (Drew Friestedt)]

Sorry - I see. Pure stupidity on my part. Thx for pointing me in the right direction. Inbound email gets delivered and processed by your own server and as you mentioned, does not hit mailgun. Need to open port 25 on EC2 instance.

Thank you!

[pfaffman (Jay Pfaffman)]

I don’t believe Mailgun’s mail forwarding will work as-is with Discourse. It doesn’t pass the email data in the right way and there doesn’t appear to be a way to change that on either end Mailgun nor Discourse.

I’m using mailgun to forward discourse+%{reply_key}@literatecomputing.com to a gmail inbox that is retrieved via pop3 and I believe that it’s working just fine.

[DeanMarkTaylor (Dean Taylor)]

I’m using mailgun to forward discourse+%{reply_key}@literatecomputing.com to a gmail inbox that is retrieved via pop3 and I believe that it’s working just fine.

That’s not directly to Discourse via the HTTP POST API which is what this topic was about…

GMail is acting as a middle man and this delays the processing of emails into Discourse by 1 to 2 times the POP3 polling time.

There are a few other POP3/Gmail issues which use of the direct mechanism resolves.

One thing you don’t get is Gmail’s superb SPAM processing. However there is already a decent level of protection both in the mail handler and Discourse especially if Akismet is installed.

[Jeff_Vienneau]

Thanks, got it working with the mail receiver method.

[kerray (Jaromír Matýšek)]

I wanted to make reply via email work from Mailgun, and found out that this plugin fork works: https://github.com/halkeye/discourse-mailgun

So Reply by email is possible just with Mailgun, you just need to install this plugin.