Troubleshooting email on a new Discourse install

In case it helps anyone here, this was the solve for an installation unable to get emails to deliver.
They wanted to use their CPanel server for outgoing mail and it just wasn’t working.

Cpanel gives you details like this:

Which led to an app.yml like this:

DISCOURSE_SMTP_ADDRESS: example.com
DISCOURSE_SMTP_PORT: 465
DISCOURSE_SMTP_USER_NAME: me@example.com
DISCOURSE_SMTP_PASSWORD: password

way too much hacking later and I got that working by doing this in the rails console.

Discourse::Application.config.action_mailer.smtp_settings[:tls]=true

:bulb: :blush: ok, so now I actually read up on the fun that is STARTTLS

The correct configuration is to use the scary looking NOT Recommended outgoing port of 587 (yes, the discourse default) and let STARTTLS do it’s thing.

Also, people often don’t know how to setup SSL on their shared cpanel servers for mail so you will likely experience certificate warnings which will also stop this from working.

Final settings needed:

DISCOURSE_SMTP_ADDRESS: example.com
#DISCOURSE_SMTP_PORT:                    # (optional, default 587)
DISCOURSE_SMTP_USER_NAME: me@example.com
DISCOURSE_SMTP_PASSWORD: password
#DISCOURSE_SMTP_ENABLE_START_TLS:         # (optional, default true)
DISCOURSE_SMTP_OPENSSL_VERIFY_MODE: none

Seems obvious but there was nothing in the logs that helped identify the reason for the problem other than it just doesn’t work.

5 Likes

thank you, this does the job.
I spend hours and hours with my own email server configured with TLS only on port 587.
struggling, no helpful log.
Finally, I added this parameter and it works :sunny:

By the way, rebuilding app takes so long, just for a small change of parameter value.
I wish it may exist a simpler way, without forcing rebuilding all but taking account of changes in settings : containers/app.yml

For a quick way to try different parameter without having to rebuild the app:

If you look at /var/discourse/shared/standalone/log/var-log, there are several files with a .txt suffix.
(for example, on my system I have MDJlYzAyNzY5NzcxOWY2.txt and a couple more)

If you grep /usr/bin/docker * in that directory you can see how the container is started.For me the command showed up in one of these .txt file (although I seem to remember seeing it in the syslog file the first I went looking for it.
Note that most of the parameters come directly from the containers/app.yml file.

I grabbed the command line and put it in a file. I can edit it, and start a container with different parameters to try things out (port mapping, hostname, SMTP parameters…)

That seems to work as long as you only change variables that are passed as arguments there.

For this to work, you need to stop the container if it is already running, and remove it. By default the container is called app.

docker stop app
docker rm app

Here’s what the command looks like for me (sensitive info replaced with xxx)
I saved it to a file I called start_container.sh and made the file executable (and added the stop and rm commands so I don’t forget)

#!/bin/bash
/usr/bin/docker stop app
/usr/bin/docker rm app
/usr/bin/docker run --shm-size=512m -d --restart=always -e LANG=en_US.UTF-8 -e RAILS_ENV=production -e UNICORN_WORKERS=4 -e UNICORN_SIDEKIQS=1 -e RUBY_GLOBAL_METHOD_CACHE_SIZE=131072 -e RUBY_GC_HEAP_GROWTH_MAX_SLOTS=40000 -e RUBY_GC_HEAP_INIT_SLOTS=400000 -e RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR=1.5 -e DISCOURSE_DB_SOCKET=/var/run/postgresql -e DISCOURSE_DB_HOST= -e DISCOURSE_DB_PORT= -e DISCOURSE_HOSTNAME=xx.xx.xx -e DISCOURSE_DEVELOPER_EMAILS=xx@xx.xx.xx -e DISCOURSE_SMTP_ADDRESS=in-v3.mailjet.com -e DISCOURSE_SMTP_PORT=587 -e DISCOURSE_SMTP_USER_NAME=xxx -e DISCOURSE_SMTP_PASSWORD=xxxxx -e DISCOURSE_SMTP_ENABLE_START_TLS=true -h lrose-app -e DOCKER_HOST_IP=172.17.0.1 --name app -t -p 8888:80 -p 443:443 -v /var/discourse/shared/standalone:/shared -v /var/discourse/shared/standalone/log/var-log:/var/log --mac-address xxxxx local_discourse/app /sbin/boot
1 Like

For a quicker way, try:

./launcher destroy app
./launcher start app
4 Likes

I had properly the same problem

tail -f /var/discourse/shared/standalone/log/rails/production.log
Sent mail to user@mail.com (61113.9ms)
Job exception: Net::ReadTimeout

mentioned in


and here

as solution is on

maybe 465 works but there is a bug as described on


so verify username and password that they don’t begin with “Q” or “R”, using https://www.base64encode.net/

5 posts were split to a new topic: Can’t get Siteground Email and one-click droplet to work

Hello,

I am trying your solution but no success :frowning:
I have created the mail account (and also the discourse install) in a subdomain.

so it looks something like

DISCOURSE_SMTP_ADDRESS: subdomain.example.com
DISCOURSE_SMTP_PORT: 465
DISCOURSE_SMTP_USER_NAME: discourse@subdomain.example.com
DISCOURSE_SMTP_PASSWORD: password

I have changed the app.yml to the final settings you have supplied (commenting out the smtp_port anddiscourse_smtp_enable_start_tls and adding discourse_smtp_openssl_verify_mode:none.

But now I don’t know what do do.
the discourse-doctor says the email is sent but I never receive it. I have used mail-tester.com and they too say it’s never received. But I don’t know what I have to change in Cpanel or (where do I need to change it??) to let it send my mail.

can anyone help?

At the bottom of app.yml is a commented out line that lets you set the notification address. You likely need to change it to whatever your mark server is configured for. You could check the mail server logs to see the error.

thank you @pfaffman :slight_smile:
to answer my own question (and maybe help others), problem was with the ‘no-reply’ email address that didn’t exist as a sender, and the server blocking those. I created a no-reply account and that fixed it.

3 Likes

With more shared/standalone/log/rails/production.log, I got to know the connection is getting refused to my admin mail address admin@mydomain.com. But that mail address is working absolutely fine because it is receiving mails from other mail clients while in Discourse installation even I am not receiving an activation mail. Also, my cloud server on DigitalOcean is getting through the email server with this telnet mail.mydomain.com 465 but why the telnet connection with forums.mydomain.com is not getting established?
I am using CPanel by the way. Kindly assist!!

Looks like your digitalocean account is currently under probation and they’re blocking SMTP ports (25,254,587). To solve this, you can either configure your SMTP to listen to a non-standard port (e.g. 2525) and then update this in discourse so that it uses the new port for email transport.

4 Likes

There is no option to modify SMTP ports in my CPanel. Anyway, my gmail account is working. Is it advisable to use gmail as admin account with mail SMTP server from mailgun?

You can use any email address as admin account as long as it is a valid email address able to receive emails.

On mailgun SMTP, use port 2525 instead of 587 and you should be good to go.

1 Like

If you think that your mail configuration is fully ok, and yet your mails aren’t delivering. You can segregate the fault by trying to send mails thru your smtp server, but without the intervention of discourse. You just need access to your droplet thru a terminal.

I was able to send mails thru Terminal in the below given way. SSH to your droplet and go ahead as told below.

  • telnet smtp.mailgun.org 25
  • If ready msg comes, then: ehlo smtp.*****.com (replacing stars with your own smtp server name)
  • 6-7 msgs should come up containing 250 code.
  • auth login (you should get 334 VXNlcm5hbWU6 )
  • base64 encoded id: cd9zewhateveryouridisnUueHl6 (if your user id is ok, you should get 334 some-code).
  • base64 encoded pw: OnU2Njk7NWE1yourownpasswordherejllMzg= (you should get 235 2.0.0 ok msg)
  • mail from: <anyname@yoursendingdomainname> (you should get ‘250 sender address accepted’ msg)
  • rcpt to: <recipientname@gmail.com> (you should get ‘250 recipient address accepted’ msg)
  • data (you should receive ‘354 continue’ msg)
  • And now type your msg, whatever. And in the end just a period plus enter. At this, you should get ‘250 Great Success’ msg).

To come out, you can ‘quit’ or press ‘escape character+enter+quit’. You should get ‘221 See you later. Connection closed by foreign host’ msg).

What are the correct settings passed to ./discourse-setup for connecting to an smtp server on localhost:25 without auth?

I’m very surprised that this isn’t supported OOTB; it’s the default config on most linux installs…

My server runs postfix locally; it is not accessible from the Internet. It works fine, for example, when running the mail command. I found a few unofficial guides on the Internet suggesting changes to /var/discourse/containers/app.yml, and I finally got it to install & start with the following settings:

  DISCOURSE_SMTP_ADDRESS: localhost
  DISCOURSE_SMTP_PORT: 25
  DISCOURSE_SMTP_USER_NAME: discourse@opensouceecology.org
  DISCOURSE_SMTP_PASSWORD: "none"
  DISCOURSE_SMTP_AUTHENTICATION: none
  DISCOURSE_SMTP_OPENSSL_VERIFY_MODE: none
  DISCOURSE_SMTP_ENABLE_START_TLS: false

Note that if I omit the DISCOURSE_SMTP_USER_NAME or DISCOURSE_SMTP_PASSWORD variables, your install script yells at me stating that they’re required (bug?).

And now when I click the “Resend Activation Email” button in the Discourse wui, this entry pops-up in the log file (/var/discourse/shared/standalone/log/rails/production.log):

Started PUT "/finish-installation/resend-email" for 127.0.0.1 at 2019-11-07 13:15:31 +0000
Processing by FinishInstallationController#resend_email as HTML
  Parameters: {"authenticity_token"=>"SzQCvRWiqdXsBKzOjIB0X7KkvXro7Od6SdP8Qa8vvrskPeNYZNos5ORHJfyDUrHiKShZR/txM6NHuqHHCQCR1w=="}
  Rendering finish_installation/resend_email.html.erb within layouts/finish_installation
  Rendered finish_installation/resend_email.html.erb within layouts/finish_installation (Duration: 0.7ms | Allocations: 103)
  Rendered layouts/_head.html.erb (Duration: 0.5ms | Allocations: 103)
Completed 200 OK in 98ms (Views: 3.0ms | ActiveRecord: 0.0ms | Allocations: 4763)
  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.5ms | Allocations: 141)
Delivered mail c4ca58ca-345e-46c4-81bc-6d0eac7afa04@discourse.opensourceecology.org (11.3ms)
Job exception: wrong authentication type none

…But my authentication type is ‘none’. What should the correct setting be for no authentication?

EDIT: also, can someone link me to the doc that defines all of the possible “DISCOURSE_SMTP_*” variables and all of their valid values?

EDIT2: this is proving to be far more difficult than it should be. I think ‘localhost’ is resolving inside the docker container to the Discourse docker container itself (app) – not the docker host that is running my postfix smtp server. That’s further complicated by postfix’s mynetworks and iptables (which were configured by the discourse-setup script or its children scripts). What’s the correct config here to just have Discourse use the smtp server on which I want to run Discourse, with no smtp auth?

I think that hasn’t been very true for about 20 years.

You can’t use discourse-setup for situations like yours because few people have non password protected smtp servers, even behind a firewall.

What I would do is configure smtp passwords for my mail server. There really isn’t much downside.

If you don’t want to do that I think that instead of “none” for authentication you might want “” (and similarly for the password and username).

I think so too. You might try using the container name. I think that you need to see that they are both on the same docker network.

2 Likes

In 2019, it’s the default config for postfix on RHEL/CentOS. Postfix binds only to the loopback interface and drops all smtp requests that don’t originate from 127.0.0.0/8. No auth required. I’m not sure about debian, but I’d imagine exim has a similar default config.

A couple relevant topics on these forums from other users who hit this issue:

There doesn’t appear to be a topic for how to set this up on RHEL/Cent OS with the necessary changes to both Discourse and postfix, so I’m documenting this here.

This does not appear to be possible with the discourse-setup script, but I did get this to work.

First, I had to figure out the IP address of the docker host as the docker container sees it. Using 127.0.0.1 won’t work because the docker container will see 127.0.0.1 as itself. Rather, we need to specify the IP address or hostname of the docker host that is running the postfix SMTP server, and one that is addressible by the docker container (so not your docker host’s Internet-facing IP address if you want your SMTP server to not be Internet-accessible, for example).

I extracted the relevant IP address of the docker host (172.17.0.1) from the docker0 interface by executing this on the docker host:

[maltfield@osestaging1 ~]$ ip address show docker0
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP 
    link/ether 02:42:80:35:65:a1 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:80ff:fe35:65a1/64 scope link 
       valid_lft forever preferred_lft forever
[maltfield@osestaging1 ~]$

Then I edited my Discourse app’s yaml file, setting the “DISCOURSE_SMTP_ADDRESS” to 172.17.0.1 from above.

[maltfield@osestaging1 ~]$ cd /var/discourse/
[maltfield@osestaging1 discourse]$ grep SMTP containers/app.yml
  DISCOURSE_SMTP_ADDRESS: 172.17.0.1
  DISCOURSE_SMTP_PORT: 25
  DISCOURSE_SMTP_AUTHENTICATION: none
  DISCOURSE_SMTP_OPENSSL_VERIFY_MODE: none
  DISCOURSE_SMTP_ENABLE_START_TLS: false
[maltfield@osestaging1 discourse]$ 

Note that I first tried to use the internal docker hostname host.docker.internal for this, but apparently this hostname isn’t available to linux docker users

Because the default postfix configuration in RHEL/Cent OS binds only to 127.0.0.1 (which is good), you’ll need to change /etc/postfix/main.cf so it also binds to the docker0 interface and add that subnet to the mynetworks group so that SMTP traffic coming from docker containers will be accepted by postfix.

[maltfield@osestaging1 postfix]$ grep -ir '172.17' /etc/postfix/*
/etc/postfix/main.cf:inet_interfaces = 127.0.0.1, 172.17.0.1
/etc/postfix/main.cf:mynetworks = 127.0.0.0/8, 172.17.0.0/16
[maltfield@osestaging1 postfix]$ 

After those changes, rebuild Discourse and it should now be able to send emails out through your docker host’s postfix.

/var/discourse/launcher rebuild app

While this works, I have a few questions:

  1. Is there some other environment variable or hostname that already points to the docker host (172.17.0.1 in this case)?

I noticed that there is a DISCOURSE_HOST_IP environment variable “injected” by launcher. Is it possible to set this DISCOURSE_SMTP_ADDRESS yaml key to the same value as the other’s yaml key with something like this?

DISCOURSE_SMTP_ADDRESS: $DISCOURSE_HOST_IP
  1. In general, how durable is the 172.17.0.1 IP of the docker host? Is it always this IP on RHEL/Cent OS systems? Will it ever change on me?

Hi all,

So, I’m running into a brick wall with the final email configuration, and I’m hoping someone can point me in the right direction.
Here are the discourse-doctor logs:

DISCOURSE DOCTOR Sat Nov 30 14:46:40 UTC 2019
OS: Linux ip-172-31-83-113.ec2.internal 3.10.0-1062.4.3.el7.x86_64 #1 SMP Wed Nov 13 23:58:53 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux


Found containers/app.yml

==================== YML SETTINGS ====================
DISCOURSE_HOSTNAME=discourse.a2ztrainingsystem.com
SMTP_ADDRESS=smtp.mail.us-east-1.awsapps.com
DEVELOPER_EMAILS=REDACTED
SMTP_PASSWORD=REDACTED
SMTP_PORT=465
SMTP_USER_NAME=ariel@a2ztrainingsystem.com
LETSENCRYPT_ACCOUNT_EMAIL=REDACTED

==================== DOCKER INFO ====================
DOCKER VERSION: Docker version 19.03.5, build 633a0ea

DOCKER PROCESSES (docker ps -a)

CONTAINER ID        IMAGE                 COMMAND             CREATED             STATUS              PORTS                                      NAMES
25b9f765fadc        local_discourse/app   "/sbin/boot"        20 minutes ago      Up 20 minutes       0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp   app


Discourse container app is running


==================== PLUGINS ====================
          - git clone https://github.com/discourse/docker_manager.git

No non-official plugins detected.

See https://github.com/discourse/discourse/blob/master/lib/plugin/metadata.rb for the official list.

========================================
Discourse version at discourse.a2ztrainingsystem.com: NOT FOUND
Discourse version at localhost: NOT FOUND


==================== MEMORY INFORMATION ====================
OS: Linux
RAM (MB): 1880

              total        used        free      shared  buff/cache   available
Mem:           1836        1026          91          26         718         600
Swap:          2047          32        2015

==================== DISK SPACE CHECK ====================
---------- OS Disk Space ----------
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1       20G   13G  7.7G  62% /

==================== DISK INFORMATION ====================

Disk /dev/xvda: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000b956b

    Device Boot      Start         End      Blocks   Id  System
/dev/xvda1   *        2048    41943006    20970479+  83  Linux

==================== END DISK INFORMATION ====================

==================== MAIL TEST ====================
For a robust test, get an address from http://www.mail-tester.com/
Sending mail to REDACTED  . .
Testing sending to dan@devopsidiot.com using ariel@a2ztrainingsystem.com:REDACTEDPASSWORD@smtp.mail.us-east-1.awsapps.com:465.
======================================== ERROR ========================================
                                    UNEXPECTED ERROR

Net::ReadTimeout

====================================== SOLUTION =======================================
This is not a common error. No recommended solution exists!

Please report the exact error message above to https://meta.discourse.org/
(And a solution, if you find one!)
=======================================================================================

==================== DONE! ==================== 

And here is, what I think, is tripping me up when I tail the prod.log

Delivered mail 789c7c29-29c5-4f11-8d27-ec175f600592@discourse.a2ztrainingsystem.com (60119.5ms)

Now, I went into containers/app.yml and uncommented out the default email to make it:

- exec: rails r "SiteSetting.notification_email='ariel@a2ztrainingsystem.com'"

I did rebuild the app after I saved that setting -
./launcher rebuild app

I’m running out of AWS with AWS’ Workmail set up. I can connect to telnet w/o issue, so I don’t think I’m being blocked.

Any assistance or direction would be appreciated!

I think

should look like

or was the message changed once and Delivered mail is the same then Send mail to?

You may have the same problem as described by @jehrlich.
I’ve got stuck at the same point, based on the description

Is the email domain correct?

I tested the from-address used by discourse via https://www.smtper.net/ and got

"Relaying disallowed as noreply@…

what led me to SMTP - Simple Mail Transfer Protocol Configuration in Zoho Mail, https://help.zoho.com/portal/community/topic/enable-relaying and What is the difference between the “From” and “Sender”? – Mailgun Help Center what it turn ended in the conclusion that I need to change notification email what can be any email address accepted by your email relaying service, as described in Can domain name of discourse hosting and sending emails be different?.
I’m only wondering why I shall do

as far as I understand, email notification always sent using the email address specified in SiteSetting.notification_email=.
It is nowhere mentioned that discourse is going to use the address specified in DISCOURSE_SMTP_USER_NAME:.
@pfaffman or anyone else here around who can enlighten us?

why is that not recommended in the troubleshooter as ./launcher rebuild app is so time-consuming