Amazon SES: "Must issue a STARTTLS command first"

Hi,

New discourse setup with Amazon SES as the email provider. I have stepped through the Troubleshooting email howto.

There is no activation email received, and when i use ./discourse-doctor I get a STARTTLS error:

======================================== ERROR ========================================
                                    UNEXPECTED ERROR

530 Must issue a STARTTLS command first

Configuration:

  DISCOURSE_DEVELOPER_EMAILS: 'admin@XXX.com'
  DISCOURSE_SMTP_ADDRESS: email-smtp.us-east-1.amazonaws.com
  DISCOURSE_SMTP_PORT: 587
  DISCOURSE_SMTP_USER_NAME: XXX
  DISCOURSE_SMTP_PASSWORD: "XXX"
  DISCOURSE_SMTP_AUTHENTICATION: "login"
  DISCOURSE_SMTP_ENABLE_START_TLS: true           # (optional, default true)

Checklist:

  • Tried the “Troubleshooting email” howto in this forum
  • Domain is verified in Amazon with TXT and DKIM
  • SES is out of sandbox mode
  • The Test Email button in amazon sends a message successfully.
  • Tried DISCOURSE_SMTP_AUTHENTICATION = “plain”, no effect
  • Tried removing DISCOURSE_SMTP_AUTHENTICATION, no effect

I am grateful for any thoughts!

Andrew

Having trouble getting email up and running on AWS.
I am using Amazon Simple Email Service since this is the only option when running AWS FreeTier.
I have a couple of verified email adresses - and sending from the AWS SES console works fine.
I see no errors in any of the logs - but running discourse-doctor reveals the following.

==================== MAIL TEST ====================
For a robust test, get an address from http://www.mail-tester.com/
Sending mail to REDACTED  . . 
Testing sending to <my-aws-verified-email> using <aws-ses-smtp-user>:<aws-ses-smtp-password>@email-smtp.eu-west-1.amazonaws.com:587.
======================================== ERROR ========================================
                                    UNEXPECTED ERROR

530 Must issue a STARTTLS command first

How can I get Discourse to send a STARTTLS command?

The logs contains no error messages
In /var/discourse/shared/standalone/log/rails/production.log

Started GET "/" for <my-local-ip> at 2019-01-12 22:43:39 +0000
Processing by FinishInstallationController#index as HTML
Rendering finish_installation/index.html.erb within layouts/finish_installation
Rendered finish_installation/index.html.erb within layouts/finish_installation (5.9ms)
Rendered layouts/_head.html.erb (12.9ms)
Completed 200 OK in 378ms (Views: 214.7ms | ActiveRecord: 74.6ms)
Started GET "/stylesheets/wizard_a7d668c8ceb59963af0b700e223fb4dda180a82d.css?__ws=<my-domain>" for <my-local-ip> at 2019-01-12 22:43:40 +0000
Processing by StylesheetsController#show as CSS
Parameters: {"__ws"=&gt;"<my-domain>", "name"=&gt;"wizard_a7d668c8ceb59963af0b700e223fb4dda180a82d"}
Sent file /var/www/discourse/tmp/stylesheet-cache/wizard_a7d668c8ceb59963af0b700e223fb4dda180a82d.css (0.2ms)
Completed 200 OK in 6ms (ActiveRecord: 0.4ms)
Started GET "/finish-installation/register" for <my-local-ip> at 2019-01-12 22:43:45 +0000
Processing by FinishInstallationController#register as HTML
Rendering finish_installation/register.html.erb within layouts/finish_installation
Rendered finish_installation/register.html.erb within layouts/finish_installation (3.0ms)
Rendered layouts/_head.html.erb (6.7ms)
Completed 200 OK in 39ms (Views: 18.9ms | ActiveRecord: 8.3ms)
Started POST "/finish-installation/register" for 84.215.136.87 at 2019-01-12 22:44:07 +0000
Processing by FinishInstallationController#register as HTML
Parameters: {"utf8"=&gt;"✓", "authenticity_token"=&gt;"Qg1xkp9v9nDhtNk0G4EPwApHsk219BJAZOohJssVtGGfz95m5PhIzleCDedRdh11Fxv7ETHP5WzH2N7DYQI+6g==", "email"=&gt;"trillebar@gmail.com", "username"=&gt;"MalaRF", "password"=&gt;"[FILTERED]", "commit"=&gt;"Register"}
Redirected to http://testforum.mx5-miata.no/finish-installation/confirm-email
Completed 302 Found in 292ms (ActiveRecord: 71.8ms)
Started GET "/finish-installation/confirm-email" for <my-local-ip> at 2019-01-12 22:44:07 +0000
Processing by FinishInstallationController#confirm_email as HTML
Rendering finish_installation/confirm_email.html.erb within layouts/finish_installation
Rendered finish_installation/confirm_email.html.erb within layouts/finish_installation (1.2ms)
Rendered layouts/_head.html.erb (1.5ms)
Completed 200 OK in 9ms (Views: 3.3ms | ActiveRecord: 0.4ms)
Sent mail to <my-email>@gmail.com (474.7ms)

Regards, Tor

1 Like

Finally got my discourse test-site up.
I had to verify my domain in AWS Console “SES Home, Identity Management, Domains”
I used with DKIM settings generated and added the entries in our DNS at one.com
But discourse-doctor still fails when trying to send an email

3 Likes

Thanks. Can you post your config settings for email? My domain is verified too but still not working :frowning:

With verified domain - all you should need to do as well is to verify the email you intend to send from. Thats all I have done. Maybe recheck the details in the config?
BTW: discourse-doctor still reports the mail-error… But I get verification emails sent

With SES, can You confirm that Amazon support has moved your SES account out of sandbox mode?

Once that’s done, You just need the following:

So please comment out any other SMTP related settings as those aren’t really required.

Aha! Well, at least it doesn’t crap out. I guess discourse-doctor should just ignore 530 as an error? Or print something like “Got 530 error. If you’re using Amazon SES you can ignore this”?

You could still use something like Mailgun as long as AWS isn’t blocking port 2525.

3 Likes

Yeah would be nice to have a hint here, I recommend that approach.

2 Likes

The point is:
Discourse itself sends emails to the same recipient without any errors (at least not in production.log).
Is discourse-doctor not meant to perform a “health check” of the Discourse installation?
I don’t see this problem being specific to AWS. Except if the from-address is different than the one being used by the installed Discourse - SES in sandboxed mode allows only to and from email adresses that are verified in SES. When discourse-doctor fails sending emails - but Discourse succeed - I just stop trusting the doctor.

1 Like

Similar thoughts here - it didn’t occur to me that discourse-doctor could fail at sending email, but that discourse itself would succeed.

1 Like

I believe this is the fault of the rake task not checking and respecting the START_TLS environment variable.

EDIT: let me see if I can make that work

5 Likes

OK! We do a preliminary connection test using Net::SMTP to first try and diagnose some common problems, but that was never trying to use STARTTLS so it would always fail if that was in play.

I tried enabling it, but Net::SMTP in our container doesn’t work with that (I tested against my server and gmail, both failed). My server complained:

Jan 15 03:54:36 ralakrid postfix/submission/smtpd[2046]: connect from unknown[178.128.235.9]
Jan 15 03:54:36 ralakrid postfix/submission/smtpd[2046]: SSL_accept error from unknown[178.128.235.9]: -1
Jan 15 03:54:36 ralakrid postfix/submission/smtpd[2046]: warning: TLS library problem: error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../ssl/record/ssl3_record.c:252:
Jan 15 03:54:36 ralakrid postfix/submission/smtpd[2046]: lost connection after STARTTLS from unknown[178.128.235.9]
Jan 15 03:54:36 ralakrid postfix/submission/smtpd[2046]: disconnect from unknown[178.128.235.9] ehlo=1 starttls=0/1 commands=1/2

It isn’t really worth mucking about with Net::SMTP when this is the only place it’s used, so that error message is bypassed and we try using ActionMailer (which DOES reflect what Discourse will do) with this: https://github.com/discourse/discourse/pull/6883

5 Likes

Hey, @supermathie. So it looks like you’ve fixed the rake task so that discourse-doctor does not need to be updated for this. Thanks.

(And was it me who included a spurious j? I’ve spent too much time looking at history. I’ll try to let it go.)

3 Likes

:+1: Yep, that case just wasn’t handled in our rake task.

Nope, it leaked here.

2 Likes

Well, I can’t quite tell on mobile, but I’m thinking that it was me who submitted that PR. I thought that I’d tested all the paths though.

2 Likes

It’s a rather crude tool. It is resigned to help identify the most common problems for people who don’t know what bash is. If you can make aws work you’re not the target audience. Presumably you’d not be running it to test a site they wasn’t broken. :wink:

I have been trying to get AWS SES working for on a brand new discourse install for 2 days!

I’ve tried everything in the initial guide:

Discourse doctor also fails with 530 issue STARTTLS command.

My SES credentials are correct since I can send email from command line curl with the same settings, however discourse won’t send me my initial admin email.

 DISCOURSE_DEVELOPER_EMAILS: 'MYEMAIL'
 DISCOURSE_SMTP_ADDRESS: email-smtp.us-west-2.amazonaws.com
 DISCOURSE_SMTP_PORT: 587
 DISCOURSE_SMTP_USER_NAME: XXX
 DISCOURSE_SMTP_PASSWORD: "SESPASSWORD"

SES is not in sandbox. I can’t get the first initial email sent and can not login as admin. There are no errors in /var/discourse/shared/standalone/log/rails/production.log

Unless you change it, Discourse is sending from noreply@your.host.name. If SES won’t accept mail from that address, it won’t send. There’s a line at the end of app.yml where you can set it.

A fix is pending for this: Amazon SES: "Must issue a STARTTLS command first"

Do SES logs show that it accepted the email?

If you get into the rails console, you can Create Admin Account from Console and then follow up with testing the email setup.

3 Likes

Thank you!

I managed to login as admin and send test emails which all worked.

It was just the discourse doctor that was broken.