Proposed changes to discourse-setup because Digital Ocean blocks outbound smtp


(Jay Pfaffman) #1

Continuing the discussion from Digital Ocean is blocking outgoing mail!:

It seems that Digital Ocean is blocking out bound SMTP traffic for at least some customers. My guess is that soon you’ll have to jump through hoops to be able to access a remote SMTP port. Simple solution: Mailgun, Sendgrid, and Sparkpost all accept TLS traffic on port 2525.

I propose these changes to discourse-setup

  • changing default SMTP port for those services to 2525.
  • adding a test (e.g., nc -z -w5 smtp.mailgun.org 25) to see if the chosen SMTP port is open

Why those 3 services services? They’re the ones that discourse-setup currently knows about and recommends smtp_user_name for).

Unless someone (i.e., @codinghorror) has some reason not to, I’ll try to do this later this week.

I’ll also see about changing the scary “your ip doesn’t resolve to your hostname” message too. It continues to confuse people. Or maybe I’ll look again at finding a more reliable way to test that the resolved host IP connects to the server.


(Jeff Atwood) #2

I don’t agree with any of this. Unless you have actual evidence that DO is globally blocking the smtp port for all new signups, not just some of them…


(Eli the Bearded) #3

You don’t agree with any of it? I’d say having the set-up script test that the named mail server can be reached on the named port is a good thing and will help people identify problems faster. Maybe do nothing else is done to help people get around the blockage, lest DO get more aggressive.


(Jay Pfaffman) #4

The response that I just got from Digital Ocean didn’t say “Yeah, we’re blocking everyone by default”, so perhaps my knee-jerk reaction was out of line, but adding a simple “can we connect to the smtp server at all” test still seems like a reasonable idea.

Here’s their response, which I find unsatisfying. Note that this DO tech says that his only way to know if a droplet has been blocked is to try to telnet to google’s smtp servers.


(Matt Palmer) #5

We shouldn’t be sending mail on port 25 anyway. 587 is the “submission” port, for MUAs (which is what Discourse is) to submit mail to a fully-fledged MTA, using an encrypted and authenticated connection. AFAIK all decent ESPs support submission, and if a provider blocks 587 outbound, I’m more than happy to go full rage-womble on them, because there is absolutely zero reason to do so on anti-abuse grounds.


(Jay Pfaffman) #6

I wasn’t clear, then. They are also blocking 587. It seems to be the case that they are blocking all standard ports that transmit mail.


(Matt Palmer) #7

download

I’ll spin up a test droplet, and if I can reproduce, then it will be rage-womble: ENGAGED.

Addendum: drat, they’re wise to my plan and my test droplet could do outbound 587. If anyone gets a direct contact at DO who wants to argue the point, feel free to point them my way, otherwise it’s a bit weird to be raising a ticket for a problem I’m not actually having…


(Jay Pfaffman) #8

I doubt you’ll reproduce. With a new account, maybe. Maybe with a new account? Maybe a new account that doesn’t have a credit card?

Among my frustrations is that they won’t say what makes you get blocked. I guess if that were public it’d tell spammers how to sneak by.


(Jay Pfaffman) #9

I’ll include a link to this post in my next exchange with them.


(Matt Palmer) #10

OK then, let me give you the full spiel, then:

Port 25 outbound is an obvious candidate for filtering, because if you can hit the Internet on port 25, you can deliver e-mail to practically anyone. If that e-mail is unwanted, the only useful “identity” information that the recipient has to try and prevent further abuse is the sending IP address. That identity is tied, in general, to a hosting provider, not an individual subscriber, so any reputation hit for spamming goes, in the first instance, onto the hosting provider. For a cloud provider, whose only customer contact is a (potentially stolen) credit card, there are practically zero consequences that can be imposed on the actual responsible party (the customer). So, the hosting provider ends up wearing the blame. It’s no surprise, therefore, that cloud providers are somewhat reticent to let everyone have unfettered outbound port 25. It’s a shame, but it’s not a surprise.

The thing is, port 587 is a whole different kettle of fish. You cannot deliver e-mail to any address on the Internet by connecting to port 587. In order to get an e-mail sent through port 587, you need to, at the very least, authenticate yourself. That provides an identity, separate from the sending IP address, on which to apply consequences. If the e-mail ends up being spam, then it’s the e-mail provider (as the party which made the connection to port 25) which takes the reputation hit. They know this, and there’s all sorts of stuff they do to balance “getting paid to deliver e-mail” and “not getting blacklisted to hell and back” (with varying levels of success). My point here is that the reputation hit from delivering an e-mail that was submitted over port 587 lands on the intermediary, not the hosting provider.

Now, you might say, “aha! but the Received: headers will show it came from our IP space originally, and we’ll still get blamed”. There’s two responses to that. First off, nobody who’s been doing e-mail for more than five minutes believes any Received header they didn’t generate, because they’re trivial to spoof. Much fun was had in the good old days blaming spam on all sorts of innocent parties via that trick. Second of all, if you’re worried about multi-hop reputation problems, you need to block 2525 as well, because guess what? 2525 is a de facto standard submission port, as well, because some people can’t remember two different numbers, or something. They behave practically identically, and any problems you’d have if you allowed submission over the standard port of 587 also apply to submission over 2525.

So, yeah. Unblock outbound 587, please. The Internet thanks you.


(Jay Pfaffman) #11

I’ll submit that next time I get to a computer.

I suspect it’s new accounts that are getting filtered, which is a significant proportion of my installs. A while back, mailgun started requiring credit cards; I told people to give credit card numbers. I can’t suggest a fix here (Other than use 2525).

Your logic about filtering 587 seems sound. We’ll see.


(Jeff Atwood) #12

Probably fine to quick test the 587 port during setup and warn if it is not working, but that is not what was proposed in first post.


(Bruno) #13

I had some emails forwarding through my server (at DO) and they indeed got blocked. Nothing related to Discourse. But they are indeed cracking down on some servers and they are going by the policy shoot… then ask.

So we had to go through all these hoops to get our email back.

Just my 2c


(Rafael dos Santos Silva) #14

Well, I can’t say this is new:

This was on 2014-12-25, and I didn’t get my Discourse for Christmas.


(Jay Pfaffman) #15

Not new, but I’ve not seen it until now, and the install that I just did. . .

root@online:~# telnet smtp.mailgun.org 587
Trying 50.17.85.209...
^C
root@online:~# ^C
root@online:~# telnet smtp.mailgun.org 2525
Trying 52.24.219.230...
Connected to smtp.mailgun.org.
Escape character is '^]'.
220 ak47 ESMTP ready


(Rafael dos Santos Silva) #16

Yeah, they say it depends on your context. If you come from a 3rd world country IP, you get all sorts of problems like that. At OVH I can’t even get a server.

It’s like credit, DO will analyze all the info they have on you, and if you check enough boxes, you are a potential spammer / abuser and gets a different config.

That’s why me, @brunoedigital and @nixie are affected but not you guys from the north.


(Bruno) #17

Could we sue then for this prejudice against us? :wink:

Just kidding.

But indeed I got blocked here. So that’s probably one of the reasons.


(Jay Pfaffman) #18

Lovely explication, @mpalmer. Crickets from Digital Ocean.

I’ve changed my install script to use 2525 since there’s no easy way to tell whether 587 is blocked (my script blindly pipes values into ./discourse-setup so that I’m continually making sure that it’s working correctly).

It appears that people who use Digital Ocean and want to get their ports un-blocked need to file a ticket to request that, which is not something I’m willing to ask my install customers to do if at all possible.

I’ll try to add such a test to ./discourse-setup in the next week.