Discourse doctor parsing SMTP credentials not working (+ admin registration email not working)

simbleau · December 28, 2020, 11:38pm

Well, to start off, this is production, and I can’t get the first registration email to send.

I am sure my SMTP settings are correct.
I am able to ping and telnet my mail server with the port used, but as soon as i do “EHLO mailserver” the connection is closed by the foreign host.

At the tail of my logs, it says:

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.2ms | Allocations: 32)
Delivered mail f3853b94-b772-48c2-b1b7-a78bbcbfc5c1@discourse.mywebsite.com (60177.6ms)
Job exception: Net::ReadTimeout

And I tried ensuring my SMTP settings were correct so I ran /var/discourse/discourse-doctor, but it cannot validate my settings. It says:

==================== YML SETTINGS ====================
awk: not an option: --field-separator=:
DISCOURSE_HOSTNAME=
awk: not an option: --field-separator=:
SMTP_ADDRESS=
awk: not an option: --field-separator=:
DEVELOPER_EMAILS=
awk: not an option: --field-separator=:
SMTP_PASSWORD=
awk: not an option: --field-separator=:
SMTP_PORT=
awk: not an option: --field-separator=:
SMTP_USER_NAME=
awk: not an option: --field-separator=:
LETSENCRYPT_ACCOUNT_EMAIL=

But my YAML (./containers/app.yml) seems perfectly formatted.

  ## TODO: The SMTP mail server used to validate new accounts and send notifications
  # SMTP ADDRESS, username, and password are required
  # WARNING the char '#' in SMTP password can cause problems!
  DISCOURSE_SMTP_ADDRESS: mail.redacted.com
  DISCOURSE_SMTP_PORT: 465
  DISCOURSE_SMTP_USER_NAME: redacted@redacted.com
  DISCOURSE_SMTP_PASSWORD: "redacted"
  #DISCOURSE_SMTP_ENABLE_START_TLS: true           # (optional, default true)

  ## If you added the Lets Encrypt template, uncomment below to get a free SSL certificate
  LETSENCRYPT_ACCOUNT_EMAIL: redacted@redacted.com

pfaffman · December 29, 2020, 12:22am

That sounds like something that needs to be solved. That’s consistent with the network error you described.

My guess is that special characters on your password are confusing Discourse-doctor.

irwinstar · December 29, 2020, 1:26pm

This address seems not like a SMTP address.
I guess this “mail.redacted.com” is the http address.
I think U need check it.

simbleau · December 29, 2020, 1:39pm

@irwinstar, I removed the actual mail server for the sake of not exposing my mail server on a public forum. But I can guarantee it is correct because I copied the settings from my mail client (working) and also compared them to my actual mail server’s SMTP settings.

Thank you for your offer to help. I think the issue is SPF/DKIM records so I’m trying to fix those.

Nonetheless, discourse-doctor still should be parsing it correctly. Not sure what’s going on there, since I didn’t edit the file myself in vim/nano. To be clear, the YAML was generated using discourse-setup.

irwinstar · December 29, 2020, 1:45pm

And I think discourse not support the protocol in 465 now. If your mail server support 587, you can change it to 587 instead.
Unfortunately, my mail server can not use 587, so I add a mail proxy out side discourse.

simbleau · December 29, 2020, 1:46pm

@irwinstar, This is extremely important information if this is correct, and would explain why my email setup isn’t working. Do you have a source for this information?

irwinstar · December 29, 2020, 1:55pm

I’m not sure about it, but I try 465 some times.
Finally, I ues this “hieulq/mailproxy” docker image as a mail proxy.
And use this app.yml setting for discourse.
This usage seems bad, but it works.
I try to find a mail plugin today, but I did not find any one.

  DISCOURSE_SMTP_ADDRESS: x.x.x.x
  DISCOURSE_SMTP_PORT: 2525
  DISCOURSE_SMTP_AUTHENTICATION: none
  DISCOURSE_SMTP_OPENSSL_VERIFY_MODE: none
  DISCOURSE_SMTP_ENABLE_START_TLS: false

neounix · December 29, 2020, 2:30pm

Hey @simbleau

My experience is that SendGrid works well (port 587).

You can also test with SendGrid for free!

simbleau · December 29, 2020, 2:32pm

Appreciate the suggestions.

I’m trying to avoid purchasing any cloud mail servers, or using proxies (because if I wanted a dirty solution I’d just rake an admin account at this point) when I already have a mail server on hostgator.

I’m sure the SPF records are set for my discourse server’s IP. I’m really just trying to validate the DKIM records now.

~~If someone can confirm 465 is not allowed, that would be fantastic news as it would stop me from going too far down the rabbit hole of a non-issue.~~ 465 was an issue, noted in the post below.

simbleau · December 29, 2020, 3:41pm

I’ve made some leeway on the installation.

Using port 587 I can successfully telnet an entire email and authenticate fine. So I switched.

Now, when the email gets sent, I am receiving the following error:

Delivered mail d86b48cc-b0f4-4df3-8960-ab5ff96613a0@discourse.imbleau.com (251.0ms)
Job exception: hostname "mail.redacted.com" does not match the server certificate

But what I have noticed is that this is no longer a connection timeout (triggered after 60 whole seconds), but rather, this is a different exception received in 250ms.

This is probably because I’ve used an email which is system@discourse.redacted.com, but this email internally uses mail.redacted.com as the email server.

I’m not sure how to fix this, honestly. I tried the following:

./launcher enter app
rails r "SiteSetting.notification_email = 'system@mail.redacted.com'"
exit

and rebuilding, as to use it just this once. Still no avail. The same error.

simbleau · December 29, 2020, 3:55pm

FIXED THE EMAIL ISSUE! (discourse-doctor still parses incorrectly)

I researched a bit more about ruby and why this error was happening behind the scenes which led me to this stackoverflow issue

This was the culprit. The solution involved changing the SSL negotiation, which gave me the intuition to turn this setting off in the app.yml file
DISCOURSE_SMTP_ENABLE_START_TLS: false # (optional, default true)

Afterwards, SUCCESS!

RGJ · December 29, 2020, 5:16pm

That will disable TLS encryption, while the “only” issue is that you are using an incorrect hostname, i.e. the mail server is presenting a certificate which says something else than mail.redacted.com. Many email servers tend to be known by multiple different hostnames, you only need to find out which one it is using for its certificate.

You could try to find out which hostname it says it is and adjust your configuration accordingly.

echo | openssl s_client -connect mail.redacted.com:587 -starttls smtp -servername mail.redacted.com 2>/dev/null |openssl x509 -noout -subject

If that does not work, you can leave TLS encryption enabled and ONLY disable certificate verification:

DISCOURSE_SMTP_ENABLE_START_TLS: true
DISCOURSE_SMTP_OPENSSL_VERIFY_MODE: false

simbleau · December 29, 2020, 5:35pm

Thanks @RGJ! Weirdly enough, it’s saying the CN is one of my subdomains, completely unrelated to mail. That’s worth investigating. Thank you for the guidance.

pfaffman · December 30, 2020, 5:19pm

All of your users will have the same information.

If you’ll share with me privately your app.yml I’ll see if I can figure out what’s confusing discourse-doctor.

simbleau · December 30, 2020, 9:26pm

Sent. Take a crack at it.

pfaffman · December 30, 2020, 9:38pm

Oh. The problem is that you have some version of awk other than GNU awk and the --field-separator option is called something else in your awk.

What version of awk do you have? What OS are you running?

simbleau · January 1, 2021, 12:55am

man awk is telling me Version 1.3.4 2019-12-31 MAWK(1)

cat /etc/os-release tells me:

~$ cat /etc/os-release 
NAME="Ubuntu"
VERSION="20.04.1 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.1 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

pfaffman · January 4, 2021, 7:42pm

Would you try editing discourse-doctor and replacing --field-separator with -F?

It looks like that will solve the problem. If you’ll confirm, I’ll submit a PR.

simbleau · January 5, 2021, 12:15pm

That partially worked. The error went away but it stopped returning anything.

==================== YML SETTINGS ====================
DISCOURSE_HOSTNAME=
SMTP_ADDRESS=
DEVELOPER_EMAILS=
SMTP_PASSWORD=
SMTP_PORT=
SMTP_USER_NAME=
LETSENCRYPT_ACCOUNT_EMAIL=

simbleau · January 5, 2021, 12:29pm

I solved it.

In discourse-doctor, change line 213 to the following:

  read_config_result=$(echo $config_line | awk  -F ":" '{print $2}')

This works because mawk hates -F=":", instead it literally demands a space instead like -F ":".

Opened a PR : https://github.com/discourse/discourse_docker/pull/513