Mailgun leaking server's real IP behind Cloudflare

I want the solution indexed here in the event someone else runs into the same issue in the future.

I did submit a support ticket there first and I’m waiting on the response. Sometimes it takes Mailgun a long time to respond or they don’t respond at all. Either way, I intend to discover and/or leave a solution to the problem ITT.

Mailgun is commonly used in conjunction with Discourse.

2 Likes

cough:

$ dig @ns1.digitalocean.com sposting.com +short
138.197.91.157

Looks like you setup DNS on DigitalOcean before moving to Cloudflare, then forgot to remove the zone from DO.

2 Likes

Derp. Thanks.

That won’t prevent the IP from leaking via Mailgun though, will it? I think I might be limited to Mailgun API only. No idea how I’m going to integrate that if so.

Mailgun is a separate problem, but you’ve got to close off every avenue of leakage if you want to remain DDoS proof.

3 Likes

I warned you for that:

I don’t think Mailgun has any options or settings for this. It’s best to set up a separate relay server inbetween Discourse and Mailgun, that removes those headers. See this article: Remove sensitive information from email headers with postfix · major.io

5 Likes

If you did a standard install, you’re delivering mail via SMTP, not the API.

2 Likes

Going to try to implement this. I’ll let you know how I make out ITT.

Yes naturally. I’m either going to set up a separate relay server or try to locate and alter the email send code in Discourse; which I understand vanishes upon updates and fresh installs. Uncharted territory. My first attempt will be the former method as it seems the simplest approach.

1 Like

Yeah. The relay server is the cleaner of those solutions. You could offer a bounty for someone to write a send-with-mailgun-api plugin. I have no idea how hard it’d be, but there are hooks to use mailgun webhooks.

4 Likes

Solved.

The quick and dirty solution was indeed a mail relay server. I set up a cheap $5/mo DigitalOcean droplet running Postfix as a Mailgun relay and a custom header_checks file to prevent the original requesting IP, i.e. my Discourse droplet, from being displayed in the email headers. Discourse now talks to that mailserver opposed to connecting directly to smtp.mailgun.org

A non-SMTP Mailgun API solution be it a plugin, or as an official Discourse option would be incredibly useful for anyone using Cloudflare. An additional fix could be pushed internally at Mailgun as well if they added an option to hide the original requesting IP as is done automatically through their API.

In the meantime this works fine enough.

Thanks to @michaeld, @pfaffman and @mpalmer for the sound advice and I hope this thread serves as reference in the future for anyone experiencing similar issues.

4 Likes

Just for curiosity I checked what Mandrill does, and yes it leaks my server IP address too. So if I were concerned hiding the original IP gainst DOS, I would setup a mailrelay .

2 Likes

For those who are looking at this today, someone who wants to DDOS you can drop anything that discourse tries to fetch, point the url of the file to a server they control and get your IP address. Relay for the mail server isn’t enough, you need a rule to send new outbound connections from discourse to go through a proxy or similar.

We had several false starts with trying to get this working against all bad actors out there and ended up with Flokinet in Romania, they have a proprietary DDOS shield to the tune of 1.4 Tb/ps IIRC.

2 Likes

You could also use firewall rules to disable all connections except from cloudflare.

I don’t know how feasible it is but if during installation there was a means to specificy those kinds of requests to a different gateway or something that would be cool. Also, before discourse error handles this it can be used by bad actors to grab someone’s IP address in a PM, a setting to not list the link until it had been processed could prevent that maybe?

Last I looked the latter problem was “turn off embed content globally or live with it”.

It wouldn’t stop someone clicking on a random link though that wasn’t embedded so I dunno.

Not really, Cloudflare is only used by the client connection, any oneboxing or image embedding by URL is done directly. It’s very easy to bypass and precisely why we regularly have to stress that Cloudflare isn’t a substitute to a firewall.

1 Like

I think my point was that you’d firewall all http requests except from cloudflare (and firewall everything else too). Then it wouldn’t matter if they had your ip. Security through obscurity is not security.

How would that protect you if the attacker fakes their origin? Spoofing that kind of thing is trivial.

1 Like

Oh. Right. I totally forgot spoofing the source IP address for a DDOS attack.

Honestly there are piles of vectors for this kind of situation. If a server is directly connected to the internet with no true firewall between it and the rest of the world, then it’s impossible to be certain of it’s security.

You do not understand. If they can flood your pipe with infinite requests you are doomed, “dropping in firewall” does nothing at all for that scenario. Once your IP is exposed the game is over.

2 Likes