Troubleshooting email on a new Discourse install

I mean no offense to you or your abilities. Many people here know, and want to know, very little about intricate details about discourse or system administration.

You figured out that you can do the destroy start to save some time, so your problem is solved.

You asked why a bunch of people who provide free support do it a particular way. I provided an answer for why it’s a little hard to find out something that few people need isn’t made more apparent.

Maybe I should have said just that most people don’t want to know that.

Having helped hundreds of people get their site up and running, I can tell you that most of them do not want to know. You are not the typical first time installer.

You didn’t ask about that,at least not in the post I was responding to.

2 Likes

Ok, a few things here.

First - you tagged someone directly when asking for help. Unless you’re a paying customer please never do that. Community support is provided on a best-effort basis and is completely voluntary. You don’t have the right to summon anybody to help you for free.

You’ve chosen to deviate from the recommendations laid out in the email guide. Which means that you assume responsibility for all additional technical complexity your change creates. That’s on you 100%, whatever point you think you need to make here.

Differentiating between the settings picked up in a rebuild and those which are used after a destroy is far from obvious for many users. People who are asking for assistance with email setup typically aren’t on the right point on the curve for this to make much sense. In that regard I agree with Jay that it’s far easier to simply tell people to rebuild each time. Confuse the two and you just end up with two periods of downtime and maybe a pause between while you ask for help. Suggesting that he doesn’t want people to understand the difference is a little ridiculous, take a look at his profile and you can see that he has responded to hundreds of support topics helping people find their way.

Look at this from our perspective, we can try to educate people on additional bits of launcher-fu earlier on and risk people using the wrong argument, which increases the time to fix, or keep it simple even if that means the odd unnecessary rebuild occurs. If you don’t like that then your alternative is to post over on the #marketplace asking for a consultant to work with you, then you can expect more detail and reasoning because it will be on your dime.

4 Likes

Thank you all so much for all your responses. I will give them a try when I’m able to hop back on the server. Much appreciated!

1 Like

After a new Discourse installed refrenced from github install tutorial, then register administrator of discoure, but the email is not working. The command telnet smtp.myserver.com 465 is working fine. mail-tester website can also recieved the email from my smtp user account. What’s the issue here?

======================================== ERROR ========================================
UNEXPECTED ERROR

end of file reached

====================================== SOLUTION =======================================
This is not a common error. No recommended solution exists!

Please report the exact error message above to https://meta.discourse.org/
(And a solution, if you find one!)

It looks like there is an issue with discourse-docker. If mail-tester is receiving mail then mail is working. Why are you using discourse-doctor?

I fully with you that all happens on a voluntary base as long as you don’t pay for it. It has been like this and will be. I’m doing my best contributing where I can.
Involving Jay was due to my quote of a very specific statement of him.
We all learning every day another thing, so I’ve just figured that it was once the same for him What's the different between rebuild and bootstrap.
I tried once to understand it in more detail too and it is still on my agenda, The launcher command in the /var/discourse folder are not self-explaining.
I lost on focus on this as the folks I’m doing all this on a purely voluntary base hadn’t been sure yet if discourse is the right platform for them. It has changed recently, what put me back on this track.
So I’m caught in the same situation to help others to make the vision prospering, without asking for any compensation than a thank you.
I have only the personal demand to explain and document things I do and for that purpose, I need to understand some settings in more detail than the standard first installer as the may can life with occasional downtime or running only with the default settings.
I’m trying to spread to my findings as good as possible here but if I lack understanding, I cannot explain as good as I wish to do.

I’m helping the folks from http://fairbnb.coop to run their own hosted community platform to allow them to leave their paid Mighty Networks platform, what rises fees significantly for next year and slack.
The wish is to replace it by mattermost and discourse where I’m currently the only system administrator trying to get it run (and keeping their budget untouched to spend it on the platform and all related).
As the platform is very important I have to run the community platform on decoupled from their platform (for the time being) what comes with limitations, what causes me to deviate from the recommendations laid out in the email guide.

All that I would like to know is if the email sent by SiteSetting.notification_email is the address is the only one to send any kind of notification, using the login credentials given by

SMTP server address? [smtp.example.com]: 
SMTP port? [587]: 
SMTP user name? [user@example.com]: 
SMTP password? [pa$word]: 

If so, I don’t agree that the line

After getting the first signup email, re-comment the line. It only needs to run once.

as it indicates that email given there is used only once, so notifications are sent using an email address as described by

The default email from address is based on the install domain plus subdomain, so if your URL is discourse.example.com it will be:

noreply@discourse.example.com
what will cause the same issue again.

If this is not the case, the line may not be re-commented as, for whatever reason, I have to run ./launcher rebuild app, what will apply all settings given in app.yml what ends me up in the same issue over again.

I hope that I can specify the email address used for sending notifications by setting SiteSetting.notification_email and hoping that discourse will tell the sender alias as specified in RFC 2822 Internet Message Format , using SMTP user name and SMTP password?.

I tested the smtp services by using smtper.net and will do telnet as described e.g. by

to ensure that not the hosting service or host is blocking sending emails.

Last but not least I would be happy to know which one of Delivered mail or Sent mail is the right output in tail shared/standalone/log/rails/production.log to confirm that the email is sent and what is the difference, respectively. Currently, both outputs make me thing, that discourse thinks that the email was sent.

I going to publish all my actions here to make life easier for others

The SiteSetting.notification_email changes the value of the site setting. It needs to be done only once. If you leave it in, it’ll set it at every rebuild, which could be very confusing if you later change it in the web interface and then weeks later do a rebuild and then wonder why the value has reverted. Changing this value will have affect only when the container is rebuilt, so if you have in your head a rule like “if I change just the mail settings, I don’t need to do a full rebuild”, you’ll be wrong. (The correct rule is “if I change only environment settings I don’t need to do a full rebuild”, but it’s a very subtle difference.)

1 Like

I forget to mentioned that I have currently time to do so but available time will drop significantly in two months time. Who knows how much support I’m able to provide than or if I’m able to do so in a couple of months time.
That is the reason why I’m asking such detailed question, do have it recorded properly to avoid to get stuck at the same point again, if they need to make changes on their own and giving them a better feeling when managing the system by giving them some extra information.

thanks that got me back on track informational wise :slight_smile:
I would suggest to extend the comment by saying that this email can be changed after the installation by going to {discourse-domain/admin/site_settings/category/required
(but saying that the SMTP log- in can only be changed by a rebuild, they are read-only in the settings).
How to configure Mailgun Email Server on Discourse Forum correct way? - Detailed and Verified Steps • Crunchify is illustrating the relationships not too bad.
I don’t mind doing the PR, if it could be accepted as an improvement.


I find some good guides for the telnet work, explaining it well what has to be done when TLS is used (second covers Zoho Mail, what I currently use).



and good explanation for What is the difference between the HELO/EHLO commands? | RedinSkala

Thank you so much for posting this. Was driving me mad not getting email working. DNS records etc etc when all that was needed was:

DISCOURSE_SMTP_OPENSSL_VERIFY_MODE: none

2 Likes

so I’m done, took me a while so now it comes.
I would say production.log needs some update on how entries in there are populated
I was able to reproduce what Online SMTP tool - Test your Mail Server on the fly does or tells me, respectively using OpenSSL but I had to overcome some obstacles.

My list of obstacles:

Before I continue, a little excurse first. Can’t remember the reason but I was irritated by the pro and cons of Port 465 and 587. During my research, I came across Sending email failed with SMTPS port 465 what refers to RFC 8314 - Cleartext Considered Obsolete: Use of Transport Layer Security (TLS) for Email Submission and Access what says that port 465 should be the new standard port by using “Implicit TLS” as improper configured STARTTLS can lead to plain-text communication. More information are given in:

At first, I tested the communication manually.
The guidance was given to me by

Here is an example for mailgun
S…what I typed in the console
R…what the console returned

  1. converting username into a base64 encoded string

    S: echo -n test@mailgun.mydomain.com | base64
    R: dGVzdEBtYWlsZ3VuLm15ZG9tYWluLmNvbQ==
    
  2. converting password into a base64 encoded string

    S: echo -n mypassword | base64
    R: bXlwYXNzd29yZA==
    
  3. establishing OpenSSL including an automatic timeout to quit the connection after 30 seconds if nothing happens.
    timeout 30 o openssl s_client -connect smtp.eu.mailgun.org:587 -starttls smtp -tls1_2 -crlf -ign_eof

  4. sending EHLO and AUTH LOGIN (encrypted login)

    S: EHLO test@mailgun.mydomain.com
    R: 250-ak47
    R: 250-AUTH PLAIN LOGIN
    R: 250-SIZE 52428800
    R: 250-8BITMIME
    R: 250-ENHANCEDSTATUSCODES
    R: 250 SMTPUTF8
    S: AUTH LOGIN
    R: 334 VXNlcm5hbWU6
    
  5. do the log-in and define the sender and recipient
    VXNlcm5hbWU6 decoded means Username
    UGFzc3dvcmQ6decoded means Password
    You can decode it by the above-mentioend website or do

    S: base64 -d
    S: VXNlcm5hbWU6
    S: *press the key CTRL+d*
    R: Username:base64: invalid input
    

    the text before :base 64 is the translation.
    So let get ahead with the log-in:

    S: AUTH LOGIN
    R: 334 VXNlcm5hbWU6
    S: dGVzdEBtYWlsZ3VuLm15ZG9tYWluLmNvbQ==
    R: 334 UGFzc3dvcmQ6
    S: bXlwYXNzd29yZA==
    R: 235 2.0.0 OK
    S: mail from:<SENDER@domain.com>
    R: 250 Sender address accepted
    S: rcpt to:<receiver@gmail.com>
    R: 250 Recipient address accepted
    
  6. create the email

    S: DATA
    R: 354 Continue
    S: from: test_sender@mailgun.test.ga
    S: to: RECIPIENT@domain.com
    S: subject: OpenSSL Mailgun Test
    S: this is the email body
    S: .
    R: 250 Great success
    S: quit
    R: 221 See you later. Yours truly, Mailgun
    

Quickly after my first tests, I got sick and tired of writing again and again. Luckily I found out about the HERE document:

So I created a HERE document for each smtp service:

mailgun
openssl s_client -connect smtp.eu.mailgun.org:587 -starttls smtp -tls1_2 -crlf -ign_eof <<EOF 
EHLO test@mailgun.mydomain.com
AUTH LOGIN
dGVzdEBtYWlsZ3VuLm15ZG9tYWluLmNvbQ==
bXlwYXNzd29yZA==
mail from:<SENDER@domain.com>
rcpt to:<MyPrivateEmail@gmail.com>
DATA
from: test_sender@somedomain.ga
to: RECIPIENT@domain.com
subject: OpenSSL Siteground Mail
Mime-Version: 1.0;
Content-Type: text/html; charset="ISO-8859-1";
Content-Transfer-Encoding: 7bit;

<html>
	<body>
		 <ul>
			<li>
				openssl s_client -connect smtp.eu.mailgun.org:587 -starttls smtp -tls1_2 -crlf -ign_eof	
			</li>
			<li>
				EHLO test@mailgun.mydomain.com
			</li>
			<li>
				mail from:&ltSENDER@domain.com&gt
			</li>
			<li>
				rcpt to:&ltMyPrivateEmail@gmail.com&gt
			</li>
			<li>
				from: test_sender@somedomain.ga
			</li>
			<li>
				to: RECIPIENT@domain.com
			</li>

		</ul> 	
	</body>
</html>
.
quit
EOF	

Here I figured out that it accpets pretty much an sender & receipent as your are registerd by your credit card

siteground

openssl s_client -connect itm13.siteground.biz:587 -starttls smtp -tls1_2 -crlf -ign_eof <<EOF
EHLO MyEmailAccount@SitegroundHostedDomain.com
AUTH LOGIN
TXlFbWFpbEFjY291bnRAU2l0ZWdyb3VuZEhvc3RlZERvbWFpbi5jb20=
bXlwYXNzd29yZA==
mail from:AnySender@SitegroundHostedDomain.com
rcpt to:MyPrivateEmail@gmail.com
DATA
from: SENDER
to: RECIPIENT
subject: OpenSSL Siteground Mail
Mime-Version: 1.0;
Content-Type: text/html; charset=“ISO-8859-1”;
Content-Transfer-Encoding: 7bit;

  • openssl s_client -connect smtp.zoho.eu:587 -starttls smtp -tls1_2 -crlf -ign_eof
  • mail from:&ltAnySender@SitegroundHostedDomain.com&gt
  • rcpt to:&ltMyPrivateEmail@gmail.com&gt
  • from: SENDER
  • to: RECIPIENT
	</ul> 	
</body>
. quit EOF

Here I figured out that you have to use a domain hosted on siteground or you get

siteground error

550-Sorry 157.245.67.230. The “MAIL FROM” email address
550-AnySender@NotRegisteredSubdomain.SitegroundHostedDomain.com / NotRegisteredSubdomain.SitegroundHostedDomain.com you are using is not
550-hosted on this server. You authenticated as
550-MyEmailAccount@SitegroundHostedDomain.com but tried to use
550-AnySender@NotRegisteredSubdomain.SitegroundHostedDomain.com / NotRegisteredSubdomain.SitegroundHostedDomain.com as mail from address.
550-Sending mails from addresses not hosted on our server is not allowed.
550-Please check the “MAIL FROM” settings of your mail client and make sure you
550-set the “MAIL FROM” address to an email account which is hosted on the
550 server.

zoho

openssl s_client -connect smtp.zoho.eu:587 -starttls smtp -tls1_2 -crlf -ign_eof <<EOF
EHLO Discourse@zoho.MyDomain.com
AUTH LOGIN
ZGlzY291cnNlQHpvaG8uTXlEb21haW4uY29t
bXlwYXNzd29yZA==
mail from:
rcpt to:MyPrivateEmail@gmail.com
DATA
from: discourse@zoho.fairbnb.community
to: RECIPIENT
subject: OpenSSL Zoho Mail
Mime-Version: 1.0;
Content-Type: text/html; charset=“ISO-8859-1”;
Content-Transfer-Encoding: 7bit;

  • openssl s_client -connect smtp.zoho.eu:587 -starttls smtp -tls1_2 -crlf -ign_eof
  • EHLO Discourse@zoho.MyDomain.com
  • mail from:&ltDiscourse@zoho.MyDomain.com&gt
  • rcpt to:&ltMyPrivateEmail@gmail.com&gt
  • from: Discourse@zoho.MyDomain.com
  • to: RECIPIENT
	</ul> 	
</body>

.

EOF

Here I figured out that if:

  1. to: does not match rcpt to: recipient is put in BBC and to is ignored
  2. from: does not match Username: sending is refused and you get
    553 Relaying disallowed as Sender@zoho.MyDomain.com

So I all my test passed, the same as I repated them after I entered the Dicourse docker container, as per How do I get into a Docker container's shell? - Stack Overflow


After all my test I suggest that the production.log should also log the answers from the smtp servers as that makes life much easier. Some of my troubles would have been solved much quicker.

  1. It quite confusion if sent or failed emails are logged with the same message. In all my logs I have
    Delivered mail 6c663176-73a2-4feb-83f9-32b6113cec27@forum.fairbnb.community (308.9ms)
    sometimes it is followed by
    Job exception: end of file reached
    although Online SMTP tool - Test your Mail Server on the fly tells me differently, using the same settings.
  2. using the email defined in
    ## If you want to set the 'From' email address for your first registration, uncomment and change:
    ## After getting the first signup email, re-comment the line. It only needs to run once.
    - exec: rails r "SiteSetting.notification_email='Discourse@zoho.MyDomain.com'"
    
    what is not listed after
    ./launcher rebuild app
     tail shared/standalone/log/rails/production.log
    
    but at least the emails failed to be delivered in the first registration process (I did it twice for whatever reason) were still queued and thus send after the change in the app.yml.
    After I received the emails from Discourse@zoho.MyDomain.com I did more shared/standalone/log/rails/production.log and found
production.log extracts, newest at the bottom
Delivered mail 63ac46aa-3faf-486d-b51c-92668f7a06fe@forum.fairbnb.community (717.5ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.4ms | Allocations: 141)
Delivered mail 19cd2ae0-6aba-4f3f-9726-883102cb1dc0@forum.fairbnb.community (462.7ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.3ms | Allocations: 139)
Delivered mail 0319f11e-9748-4167-bb3e-3aea2e0ac1a5@forum.fairbnb.community (283.9ms)
Job exception: end of file reached
...
  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.3ms | Allocations: 139)
Delivered mail 0319f11e-9748-4167-bb3e-3aea2e0ac1a5@forum.fairbnb.community (283.9ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.3ms | Allocations: 139)
Delivered mail 6c663176-73a2-4feb-83f9-32b6113cec27@forum.fairbnb.community (308.9ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.5ms | Allocations: 138)
Delivered mail 3315fa23-8f66-4a44-a39d-f367ce1409ba@forum.fairbnb.community (360.0ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 12.3ms | Allocations: 141)
Delivered mail 79e7b7de-f92f-4e39-bd01-f428044a0da3@forum.fairbnb.community (383.9ms)
....
  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.3ms | Allocations: 139)
Delivered mail 0319f11e-9748-4167-bb3e-3aea2e0ac1a5@forum.fairbnb.community (283.9ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.3ms | Allocations: 139)
Delivered mail 6c663176-73a2-4feb-83f9-32b6113cec27@forum.fairbnb.community (308.9ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.5ms | Allocations: 138)
Delivered mail 3315fa23-8f66-4a44-a39d-f367ce1409ba@forum.fairbnb.community (360.0ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 12.3ms | Allocations: 141)
Delivered mail 79e7b7de-f92f-4e39-bd01-f428044a0da3@forum.fairbnb.community (383.9ms)
Job exception: end of file reached
...
Started POST "/admin/email/test" for 185.45.238.16 at 2019-12-21 16:30:14 +0000
Processing by Admin::EmailController#test as */*
  Parameters: {"email_address"=>"Stefan.Mueller.83@gmail.com"}
  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 2.3ms | Allocations: 336)
Delivered mail ef530971-de4e-4460-9e2d-b5a64f88b8b6@forum.fairbnb.community (1169.4ms)
...
Delivered mail 48f1381d-b5e4-4095-8aef-d381001f8a54@forum.fairbnb.community (1143.5ms)
Delivered mail a343fac0-5af3-407b-addc-606a133ff003@forum.fairbnb.community (1216.2ms)
Delivered mail 3713349b-203e-4454-b32b-17522997e8c1@forum.fairbnb.community (1223.0ms)
Delivered mail bce49105-f8e4-4a7a-ab23-db91c1b416ba@forum.fairbnb.community (1268.5ms)
1 Like

is not advised see ruby - SSL mode flags - verification of certificates: is it safe to use :none? - Stack Overflow
May you play around with OpenSSL command?

2 Likes

2019-12-23T08:14:13Z
This was blocked until I made the device have a fully qualified domain name (FQDN)

Edit: issues is solved, turned out discourse doctor gave an error while everything was fine.

hello, I hope someone can help me because I’m breaking my head over this and can’t figure out what is wrong.

I have setup the forum successfully, however it doesn’t send mails. For example, mail to reset password. I tried a test mail from the mail test in admin CP and it didn’t arrive.

error: There was a problem sending the test email. Please double-check your mail settings, verify that your host is not blocking mail connections, and try again.

I did all the steps in the troubleshooting first.

i checked the logs and email is getting sent but refused:

Connection refused - connect(2) for “[domainname]" port 587

I have setup the email account at discourse@[domainname], also added a noreply@[domainname].

here are the settings in app.yml.

DISCOURSE_SMTP_ADDRESS: [domainname]
DISCOURSE_SMTP_PORT: 587
DISCOURSE_SMTP_USER_NAME: discourse@[domainname]
DISCOURSE_SMTP_PASSWORD: [password]
DISCOURSE_SMTP_ENABLE_START_TLS: true
DISCOURSE_SMTP_OPENSSL_VERIFY_MODE: none

what do I need to change/where?

thank you so very much in advance!

  # WARNING the char '#' in SMTP password can cause problems!
  DISCOURSE_SMTP_ADDRESS: smtp.zoho.com
  DISCOURSE_SMTP_PORT: 587
  DISCOURSE_SMTP_USER_NAME: hi@yourdomain.com
  DISCOURSE_SMTP_PASSWORD: "password8888"
  DISCOURSE_SMTP_ENABLE_START_TLS: true           # (optional, default true)
  DISCOURSE_SMTP_AUTHENTICATION: login
  DISCOURSE_SMTP_OPENSSL_VERIFY_MODE: none

##- exec: rails r "SiteSetting.notification_email... change to - exec: rails r "SiteSetting.notification_email... (remove the ##)

run:
  - exec: echo "Beginning of custom commands"
  ## If you want to set the 'From' email address for your first registration, uncomment and change:
  ## After getting the first signup email, re-comment the line. It only needs to run once.
  - exec: rails r "SiteSetting.notification_email='hi@yourdomain.com'"
  - exec: echo "End of custom commands"

I solved the email problem through the above method, I hope this method can help other people.

3 Likes

so registration email was send as shown in the guide discourse/INSTALL-cloud.md at master · discourse/discourse · GitHub?
So the setup should be correct.

Have you tried to send an email doing telnet/openssl as described in Troubleshooting email on a new Discourse install

Thank you for your reply! The issue has been solved (sorry I forgot to post here, it’s in another topic) it turned out everything worked ok, but bc I used discourse doctor from the shell to check the settings (I was changing app.yml every time) - it turned out discourse doctor gave a sign something was wrong with mail while in fact it worked…

2 Likes

you are welcome. I’m dropping by here frequently; quite often as a last resort :grin:.
Quick answers keeping me motivated.

where? I could be helpful in future cases.

1 Like

Fair enough, I was too lazy to look it up when I posted.

Here it is: Discourse-doctor Error 503 AUTH command used when not advertised - but email works fine from admin CP

1 Like

To anyone trying to set up Discourse with an email from their shared hosting service, even if you’re instructed to use port 465, you need to use port 587 when setting up Discourse.

This is from my cPanel
image
No matter what I tried, I could not get Discourse to send me the initial registration email. Then I found this post that saved my life, used port 587, and suddenly everything worked, even though it is impossible for me to send mail from port 587.

So if you’ve set up Discourse to use port 465 and can’t get the email to send, skip all the telnet, discourse-doctor, production.log, etc. stuff and just try it with port 587 and see if that fixes it.

3 Likes