Troubleshooting email on a new Discourse install

thanks that got me back on track informational wise :slight_smile:
I would suggest to extend the comment by saying that this email can be changed after the installation by going to {discourse-domain/admin/site_settings/category/required
(but saying that the SMTP log- in can only be changed by a rebuild, they are read-only in the settings).
How to configure Mailgun Email Server on Discourse Forum correct way? - Detailed and Verified Steps • Crunchify is illustrating the relationships not too bad.
I don’t mind doing the PR, if it could be accepted as an improvement.


I find some good guides for the telnet work, explaining it well what has to be done when TLS is used (second covers Zoho Mail, what I currently use).



and good explanation for What is the difference between the HELO/EHLO commands? | RedinSkala

Thank you so much for posting this. Was driving me mad not getting email working. DNS records etc etc when all that was needed was:

DISCOURSE_SMTP_OPENSSL_VERIFY_MODE: none

2 Likes

so I’m done, took me a while so now it comes.
I would say production.log needs some update on how entries in there are populated
I was able to reproduce what Online SMTP tool - Test your Mail Server on the fly does or tells me, respectively using OpenSSL but I had to overcome some obstacles.

My list of obstacles:

Before I continue, a little excurse first. Can’t remember the reason but I was irritated by the pro and cons of Port 465 and 587. During my research, I came across Sending email failed with SMTPS port 465 what refers to RFC 8314 - Cleartext Considered Obsolete: Use of Transport Layer Security (TLS) for Email Submission and Access what says that port 465 should be the new standard port by using “Implicit TLS” as improper configured STARTTLS can lead to plain-text communication. More information are given in:

At first, I tested the communication manually.
The guidance was given to me by

Here is an example for mailgun
S…what I typed in the console
R…what the console returned

  1. converting username into a base64 encoded string

    S: echo -n test@mailgun.mydomain.com | base64
    R: dGVzdEBtYWlsZ3VuLm15ZG9tYWluLmNvbQ==
    
  2. converting password into a base64 encoded string

    S: echo -n mypassword | base64
    R: bXlwYXNzd29yZA==
    
  3. establishing OpenSSL including an automatic timeout to quit the connection after 30 seconds if nothing happens.
    timeout 30 o openssl s_client -connect smtp.eu.mailgun.org:587 -starttls smtp -tls1_2 -crlf -ign_eof

  4. sending EHLO and AUTH LOGIN (encrypted login)

    S: EHLO test@mailgun.mydomain.com
    R: 250-ak47
    R: 250-AUTH PLAIN LOGIN
    R: 250-SIZE 52428800
    R: 250-8BITMIME
    R: 250-ENHANCEDSTATUSCODES
    R: 250 SMTPUTF8
    S: AUTH LOGIN
    R: 334 VXNlcm5hbWU6
    
  5. do the log-in and define the sender and recipient
    VXNlcm5hbWU6 decoded means Username
    UGFzc3dvcmQ6decoded means Password
    You can decode it by the above-mentioend website or do

    S: base64 -d
    S: VXNlcm5hbWU6
    S: *press the key CTRL+d*
    R: Username:base64: invalid input
    

    the text before :base 64 is the translation.
    So let get ahead with the log-in:

    S: AUTH LOGIN
    R: 334 VXNlcm5hbWU6
    S: dGVzdEBtYWlsZ3VuLm15ZG9tYWluLmNvbQ==
    R: 334 UGFzc3dvcmQ6
    S: bXlwYXNzd29yZA==
    R: 235 2.0.0 OK
    S: mail from:<SENDER@domain.com>
    R: 250 Sender address accepted
    S: rcpt to:<receiver@gmail.com>
    R: 250 Recipient address accepted
    
  6. create the email

    S: DATA
    R: 354 Continue
    S: from: test_sender@mailgun.test.ga
    S: to: RECIPIENT@domain.com
    S: subject: OpenSSL Mailgun Test
    S: this is the email body
    S: .
    R: 250 Great success
    S: quit
    R: 221 See you later. Yours truly, Mailgun
    

Quickly after my first tests, I got sick and tired of writing again and again. Luckily I found out about the HERE document:

So I created a HERE document for each smtp service:

mailgun
openssl s_client -connect smtp.eu.mailgun.org:587 -starttls smtp -tls1_2 -crlf -ign_eof <<EOF 
EHLO test@mailgun.mydomain.com
AUTH LOGIN
dGVzdEBtYWlsZ3VuLm15ZG9tYWluLmNvbQ==
bXlwYXNzd29yZA==
mail from:<SENDER@domain.com>
rcpt to:<MyPrivateEmail@gmail.com>
DATA
from: test_sender@somedomain.ga
to: RECIPIENT@domain.com
subject: OpenSSL Siteground Mail
Mime-Version: 1.0;
Content-Type: text/html; charset="ISO-8859-1";
Content-Transfer-Encoding: 7bit;

<html>
	<body>
		 <ul>
			<li>
				openssl s_client -connect smtp.eu.mailgun.org:587 -starttls smtp -tls1_2 -crlf -ign_eof	
			</li>
			<li>
				EHLO test@mailgun.mydomain.com
			</li>
			<li>
				mail from:&ltSENDER@domain.com&gt
			</li>
			<li>
				rcpt to:&ltMyPrivateEmail@gmail.com&gt
			</li>
			<li>
				from: test_sender@somedomain.ga
			</li>
			<li>
				to: RECIPIENT@domain.com
			</li>

		</ul> 	
	</body>
</html>
.
quit
EOF	

Here I figured out that it accpets pretty much an sender & receipent as your are registerd by your credit card

siteground

openssl s_client -connect itm13.siteground.biz:587 -starttls smtp -tls1_2 -crlf -ign_eof <<EOF
EHLO MyEmailAccount@SitegroundHostedDomain.com
AUTH LOGIN
TXlFbWFpbEFjY291bnRAU2l0ZWdyb3VuZEhvc3RlZERvbWFpbi5jb20=
bXlwYXNzd29yZA==
mail from:AnySender@SitegroundHostedDomain.com
rcpt to:MyPrivateEmail@gmail.com
DATA
from: SENDER
to: RECIPIENT
subject: OpenSSL Siteground Mail
Mime-Version: 1.0;
Content-Type: text/html; charset=“ISO-8859-1”;
Content-Transfer-Encoding: 7bit;

  • openssl s_client -connect smtp.zoho.eu:587 -starttls smtp -tls1_2 -crlf -ign_eof
  • mail from:&ltAnySender@SitegroundHostedDomain.com&gt
  • rcpt to:&ltMyPrivateEmail@gmail.com&gt
  • from: SENDER
  • to: RECIPIENT
	</ul> 	
</body>
. quit EOF

Here I figured out that you have to use a domain hosted on siteground or you get

siteground error

550-Sorry 157.245.67.230. The “MAIL FROM” email address
550-AnySender@NotRegisteredSubdomain.SitegroundHostedDomain.com / NotRegisteredSubdomain.SitegroundHostedDomain.com you are using is not
550-hosted on this server. You authenticated as
550-MyEmailAccount@SitegroundHostedDomain.com but tried to use
550-AnySender@NotRegisteredSubdomain.SitegroundHostedDomain.com / NotRegisteredSubdomain.SitegroundHostedDomain.com as mail from address.
550-Sending mails from addresses not hosted on our server is not allowed.
550-Please check the “MAIL FROM” settings of your mail client and make sure you
550-set the “MAIL FROM” address to an email account which is hosted on the
550 server.

zoho

openssl s_client -connect smtp.zoho.eu:587 -starttls smtp -tls1_2 -crlf -ign_eof <<EOF
EHLO Discourse@zoho.MyDomain.com
AUTH LOGIN
ZGlzY291cnNlQHpvaG8uTXlEb21haW4uY29t
bXlwYXNzd29yZA==
mail from:
rcpt to:MyPrivateEmail@gmail.com
DATA
from: discourse@zoho.fairbnb.community
to: RECIPIENT
subject: OpenSSL Zoho Mail
Mime-Version: 1.0;
Content-Type: text/html; charset=“ISO-8859-1”;
Content-Transfer-Encoding: 7bit;

  • openssl s_client -connect smtp.zoho.eu:587 -starttls smtp -tls1_2 -crlf -ign_eof
  • EHLO Discourse@zoho.MyDomain.com
  • mail from:&ltDiscourse@zoho.MyDomain.com&gt
  • rcpt to:&ltMyPrivateEmail@gmail.com&gt
  • from: Discourse@zoho.MyDomain.com
  • to: RECIPIENT
	</ul> 	
</body>

.

EOF

Here I figured out that if:

  1. to: does not match rcpt to: recipient is put in BBC and to is ignored
  2. from: does not match Username: sending is refused and you get
    553 Relaying disallowed as Sender@zoho.MyDomain.com

So I all my test passed, the same as I repated them after I entered the Dicourse docker container, as per How do I get into a Docker container's shell? - Stack Overflow


After all my test I suggest that the production.log should also log the answers from the smtp servers as that makes life much easier. Some of my troubles would have been solved much quicker.

  1. It quite confusion if sent or failed emails are logged with the same message. In all my logs I have
    Delivered mail 6c663176-73a2-4feb-83f9-32b6113cec27@forum.fairbnb.community (308.9ms)
    sometimes it is followed by
    Job exception: end of file reached
    although Online SMTP tool - Test your Mail Server on the fly tells me differently, using the same settings.
  2. using the email defined in
    ## If you want to set the 'From' email address for your first registration, uncomment and change:
    ## After getting the first signup email, re-comment the line. It only needs to run once.
    - exec: rails r "SiteSetting.notification_email='Discourse@zoho.MyDomain.com'"
    
    what is not listed after
    ./launcher rebuild app
     tail shared/standalone/log/rails/production.log
    
    but at least the emails failed to be delivered in the first registration process (I did it twice for whatever reason) were still queued and thus send after the change in the app.yml.
    After I received the emails from Discourse@zoho.MyDomain.com I did more shared/standalone/log/rails/production.log and found
production.log extracts, newest at the bottom
Delivered mail 63ac46aa-3faf-486d-b51c-92668f7a06fe@forum.fairbnb.community (717.5ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.4ms | Allocations: 141)
Delivered mail 19cd2ae0-6aba-4f3f-9726-883102cb1dc0@forum.fairbnb.community (462.7ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.3ms | Allocations: 139)
Delivered mail 0319f11e-9748-4167-bb3e-3aea2e0ac1a5@forum.fairbnb.community (283.9ms)
Job exception: end of file reached
...
  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.3ms | Allocations: 139)
Delivered mail 0319f11e-9748-4167-bb3e-3aea2e0ac1a5@forum.fairbnb.community (283.9ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.3ms | Allocations: 139)
Delivered mail 6c663176-73a2-4feb-83f9-32b6113cec27@forum.fairbnb.community (308.9ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.5ms | Allocations: 138)
Delivered mail 3315fa23-8f66-4a44-a39d-f367ce1409ba@forum.fairbnb.community (360.0ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 12.3ms | Allocations: 141)
Delivered mail 79e7b7de-f92f-4e39-bd01-f428044a0da3@forum.fairbnb.community (383.9ms)
....
  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.3ms | Allocations: 139)
Delivered mail 0319f11e-9748-4167-bb3e-3aea2e0ac1a5@forum.fairbnb.community (283.9ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.3ms | Allocations: 139)
Delivered mail 6c663176-73a2-4feb-83f9-32b6113cec27@forum.fairbnb.community (308.9ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.5ms | Allocations: 138)
Delivered mail 3315fa23-8f66-4a44-a39d-f367ce1409ba@forum.fairbnb.community (360.0ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 12.3ms | Allocations: 141)
Delivered mail 79e7b7de-f92f-4e39-bd01-f428044a0da3@forum.fairbnb.community (383.9ms)
Job exception: end of file reached
...
Started POST "/admin/email/test" for 185.45.238.16 at 2019-12-21 16:30:14 +0000
Processing by Admin::EmailController#test as */*
  Parameters: {"email_address"=>"Stefan.Mueller.83@gmail.com"}
  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 2.3ms | Allocations: 336)
Delivered mail ef530971-de4e-4460-9e2d-b5a64f88b8b6@forum.fairbnb.community (1169.4ms)
...
Delivered mail 48f1381d-b5e4-4095-8aef-d381001f8a54@forum.fairbnb.community (1143.5ms)
Delivered mail a343fac0-5af3-407b-addc-606a133ff003@forum.fairbnb.community (1216.2ms)
Delivered mail 3713349b-203e-4454-b32b-17522997e8c1@forum.fairbnb.community (1223.0ms)
Delivered mail bce49105-f8e4-4a7a-ab23-db91c1b416ba@forum.fairbnb.community (1268.5ms)
1 Like

is not advised see ruby - SSL mode flags - verification of certificates: is it safe to use :none? - Stack Overflow
May you play around with OpenSSL command?

1 Like

2019-12-23T08:14:13Z
This was blocked until I made the device have a fully qualified domain name (FQDN)

Edit: issues is solved, turned out discourse doctor gave an error while everything was fine.

hello, I hope someone can help me because I’m breaking my head over this and can’t figure out what is wrong.

I have setup the forum successfully, however it doesn’t send mails. For example, mail to reset password. I tried a test mail from the mail test in admin CP and it didn’t arrive.

error: There was a problem sending the test email. Please double-check your mail settings, verify that your host is not blocking mail connections, and try again.

I did all the steps in the troubleshooting first.

i checked the logs and email is getting sent but refused:

Connection refused - connect(2) for “[domainname]" port 587

I have setup the email account at discourse@[domainname], also added a noreply@[domainname].

here are the settings in app.yml.

DISCOURSE_SMTP_ADDRESS: [domainname]
DISCOURSE_SMTP_PORT: 587
DISCOURSE_SMTP_USER_NAME: discourse@[domainname]
DISCOURSE_SMTP_PASSWORD: [password]
DISCOURSE_SMTP_ENABLE_START_TLS: true
DISCOURSE_SMTP_OPENSSL_VERIFY_MODE: none

what do I need to change/where?

thank you so very much in advance!

  # WARNING the char '#' in SMTP password can cause problems!
  DISCOURSE_SMTP_ADDRESS: smtp.zoho.com
  DISCOURSE_SMTP_PORT: 587
  DISCOURSE_SMTP_USER_NAME: hi@yourdomain.com
  DISCOURSE_SMTP_PASSWORD: "password8888"
  DISCOURSE_SMTP_ENABLE_START_TLS: true           # (optional, default true)
  DISCOURSE_SMTP_AUTHENTICATION: login
  DISCOURSE_SMTP_OPENSSL_VERIFY_MODE: none

##- exec: rails r "SiteSetting.notification_email... change to - exec: rails r "SiteSetting.notification_email... (remove the ##)

run:
  - exec: echo "Beginning of custom commands"
  ## If you want to set the 'From' email address for your first registration, uncomment and change:
  ## After getting the first signup email, re-comment the line. It only needs to run once.
  - exec: rails r "SiteSetting.notification_email='hi@yourdomain.com'"
  - exec: echo "End of custom commands"

I solved the email problem through the above method, I hope this method can help other people.

2 Likes

so registration email was send as shown in the guide discourse/INSTALL-cloud.md at master · discourse/discourse · GitHub?
So the setup should be correct.

Have you tried to send an email doing telnet/openssl as described in Troubleshooting email on a new Discourse install

Thank you for your reply! The issue has been solved (sorry I forgot to post here, it’s in another topic) it turned out everything worked ok, but bc I used discourse doctor from the shell to check the settings (I was changing app.yml every time) - it turned out discourse doctor gave a sign something was wrong with mail while in fact it worked…

1 Like

you are welcome. I’m dropping by here frequently; quite often as a last resort :grin:.
Quick answers keeping me motivated.

where? I could be helpful in future cases.

Fair enough, I was too lazy to look it up when I posted.

Here it is: Discourse-doctor Error 503 AUTH command used when not advertised - but email works fine from admin CP

1 Like