I made this mistake, and left discourse.example.com in the mail-receiver.yml file.
I have fixed this now, but mail-receiver doesn’t seem to be ‘getting’ the new details.
How do I ‘reset’ mail-receiver (e.g. what is the equivalent command to ./launcher rebuild app?).
Edit: I didn’t read the previous post closely enough, the command is ./launcher rebuild mail_receiver.
I’m running into a further issue now where mail-receiver won’t deliver mail to Discourse — I’ve tried searching for help, but no luck.
Logs:
Starting Postfix
Dec 14 03:12:32 forum-mail-receiver postfix/master[1]: daemon started -- version 3.5.6, configuration /etc/postfix
Dec 14 03:15:47 forum-mail-receiver postfix/smtpd[113]: connect from mail-pl1-f169.google.com[209.85.214.169]
Dec 14 03:15:47 forum-mail-receiver postfix/smtpd[113]: 821CB37A659: client=mail-pl1-f169.google.com[209.85.214.169]
Dec 14 03:15:47 forum-mail-receiver postfix/cleanup[120]: 821CB37A659: message-id=<602f2194be912e92b969eacf5eac26e2@frontapp.com>
Dec 14 03:15:47 forum-mail-receiver postfix/qmgr[98]: 821CB37A659: from=<[my personal email address]>, size=4086, nrcpt=1 (queue active)
<23>Dec 14 03:15:47 receive-mail[122]: Recipient: nobody@[my forum URL]Dec 14 03:16:20 forum-mail-receiver postfix/smtpd[113]: disconnect from mail-pl1-f169.google.com[209.85.214.169] ehlo=1 mail=1 rcpt=1 bdat=1 quit=1 commands=5
<19>Dec 14 03:16:47 receive-mail[122]: Failed to POST the e-mail to [my forum URL]/admin/email/handle_mail: execution expired (Net::OpenTimeout)<19>Dec 14 03:16:47 receive-mail[122]: /usr/lib/ruby/2.7.0/net/http.rb:960:in `initialize'
/usr/lib/ruby/2.7.0/net/http.rb:960:in `open'
/usr/lib/ruby/2.7.0/net/http.rb:960:in `block in connect'
/usr/lib/ruby/2.7.0/timeout.rb:105:in `timeout'
/usr/lib/ruby/2.7.0/net/http.rb:958:in `connect'
/usr/lib/ruby/2.7.0/net/http.rb:943:in `do_start'
/usr/lib/ruby/2.7.0/net/http.rb:932:in `start'
/usr/lib/ruby/2.7.0/net/http.rb:1483:in `request'
/usr/local/lib/site_ruby/mail_receiver/discourse_mail_receiver.rb:43:in `process'
/usr/local/bin/receive-mail:13:in `<main>'Dec 14 03:16:47 forum-mail-receiver postfix/pipe[121]: 821CB37A659: to=<nobody@[my forum URL]>, relay=discourse, delay=60, delays=0.17/0.01/0/60, dsn=4.3.0, status=deferred (temporary failure)
Dec 14 03:17:32 forum-mail-receiver postfix/qmgr[98]: 7C67437A663: from=<[my personal email address]>, size=4093, nrcpt=1 (queue active)
Any idea what could be causing this?
The mail-receiver.yml file is valid, and I’ve checked for typos:
This is my API key scope:
The mail is making it into mail-receiver, but it is just sitting in mailq:
Alternately, is there any way to delete the mail-receiver container entirely and start over?
The issue may be that you don’t have the API key set
Thanks for the response @pfaffman… it is definitely set in my mail-receiver.yml config. Should it be in quotes?
(Net::OpenTimeout)
That’s your issue. The mail receiver can’t access your forum URL. So either you have the wrong one somehow or there’s some network issue in docker between the mail receiver and your forum., I think.
How can I troubleshoot further?
ping forum.[mydomain].co.nz
from inside mailq shows:
64 bytes from [ip].vultrusercontent.com ([ip]): icmp_seq=1 ttl=64 time=0.113 ms
64 bytes from [ip].vultrusercontent.com ([ip]): icmp_seq=2 ttl=64 time=0.074 ms
64 bytes from [ip].vultrusercontent.com ([ip]): icmp_seq=3 ttl=64 time=0.069 ms
and so on, showing that a connection is successful.
forum.[mydomain].co.nz is where the forum is hosted, and this same URL is used in MAIL_DOMAIN and DISCOURSE_MAIL_ENDPOINT.
Looking at the mail-receiver.yml settings closer, am I missing quotes or https:// anywhere that should have it?
## this is the incoming mail receiver container template
##
## After making changes to this file, you MUST rebuild
## /var/discourse/launcher rebuild mail-receiver
##
## BE *VERY* CAREFUL WHEN EDITING!
## YAML FILES ARE SUPER SUPER SENSITIVE TO MISTAKES IN WHITESPACE OR ALIGNMENT!
## visit http://www.yamllint.com/ to validate this file as needed
base_image: discourse/mail-receiver:release
update_pups: false
expose:
- "25:25" # SMTP
env:
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LANGUAGE: en_US.UTF-8
## Where e-mail to your forum should be sent. In general, it's perfectly fine
## to use the same domain as the forum itself here.
MAIL_DOMAIN: forum.[domain].co.nz
# uncomment these (and the volume below!) to support TLS
# POSTCONF_smtpd_tls_key_file: /letsencrypt/discourse.example.com/discourse.example.com.key
# POSTCONF_smtpd_tls_cert_file: /letsencrypt/discourse.example.com/fullchain.cer
# POSTCONF_smtpd_tls_security_level: may
## The URL of the mail processing endpoint of your Discourse forum.
## This is simply your forum's base URL, with `/admin/email/handle_mail`
## appended. Be careful if you're running a subfolder setup -- in that case,
## the URL needs to have the subfolder included!
DISCOURSE_MAIL_ENDPOINT: 'https://forum.[domain].co.nz/admin/email/handle_mail'
## The master API key of your Discourse forum. You can get this from
## the "API" tab of your admin panel.
DISCOURSE_API_KEY: 639[rest of API key]884ef
## The username to use for processing incoming e-mail. Unless you have
## renamed the `system` user, you should leave this as-is.
DISCOURSE_API_USERNAME: system
volumes:
- volume:
host: /var/discourse/shared/mail-receiver/postfix-spool
guest: /var/spool/postfix
# uncomment to support TLS
# - volume:
# host: /var/discourse/shared/standalone/letsencrypt
# guest: /letsencrypt
Are you running the ping inside the container, i.e. after first running ./launcher enter mail-receiver?
It’s also worth noting that ping (typically ICMP) is different from connecting to http/https (TCP) and may behave differently depending on lots of factors in network configuration.
I would try using curl after entering the container to see if it can connect to your forum over https, e.g.
cd /var/discourse
./launcher enter mail-receiver
curl -v https://forum.[domain].co.nz
If it’s working, it’ll print a bunch of HTML. If it’s not, it’ll show some error and -v will make it print lots of information along the way which might help reveal why it failed.
If it does fail, it is also worth trying to run the same curl command outside of the container to identify whether it’s specific to the container or to the host system in general.
Thanks @Simon_Manning, your help is very much appreciated! I didn’t know that connections through ping aren’t necessarily the same as connections through curl.
I was running ping inside the container, and it succeeded.
I followed your instructions and ran curl inside the container, and it failed:
root@forum:/var/discourse# ./launcher enter mail-receiver
x86_64 arch detected.
WARNING: containers/mail-receiver.yml file is world-readable. You can secure this file by running: chmod o-rwx containers/mail-receiver.yml
bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
root@forum-mail-receiver:/# curl -v https://forum.[domain].co.nz
* Trying [IPv4 address]:443...
* Trying [IPv6 address]:443...
* Immediate connect fail for [IPv6 address]: Cannot assign requested address
* connect to [IPv4 address] port 443 failed: Connection timed out
* Failed to connect to forum.[domain].co.nz port 443: Connection timed out
* Closing connection 0
curl: (28) Failed to connect to forum.[domain].co.nz port 443: Connection timed out
Then I ran exit and then curl again, and got:
root@forum:/var/discourse# curl -v https://forum.[domain].co.nz
* Trying 127.0.1.1:443...
* Connected to forum.[domain].co.nz (127.0.1.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
...and so on.
It looks like it’s specific to the container, and not to the host system — any ideas?
I have also opened a support ticket with Vultr (VPS provider for this instance) to see if it is a problem on their end!
Docker creates virtual networks for the containers and in the absence of specifying one, containers will use the default network. This default network does not allow container-to-container communication.
Usually this is fine for mail-receiver because your Discourse container will have port 443 exposed outside of that network and when mail-receiver tries to connect to 1.2.3.4, it will leave the Docker network. The host system (or some networking further out) will realise it just needs to come back again and it will end up entering the Discourse container from outside.
Two possibilities come to mind. One is that mail-receiver is somehow aware of the Discourse container IP when looking up the domain name and therefore an intra-container connection is being blocked. I think this is unlikely.
The other is that a firewall on the host system is blocking connections from leaving a container and entering a different one. Vultr may use default firewall rules that cause this or I also vaguely remember that Docker installs some rules in UFW by default, so that could be related if that’s used.
You can’t use https be because you didn’t uncomment these:
That only applies to TLS support on the mail server side, i.e. for other mail servers to be able to deliver emails to mail-receiver over TLS.
Worth doing since the Discourse container evidently does have a certificate but shouldn’t affect mail-receiver connecting to Discourse. Potentially the rebuild could though if it happens to correct something in the container’s networking.
Thank you, I uncommented those lines, and the line for the volume.
My mail-receiver.yml file now looks like this:
root@forum:/var/discourse# cat containers/mail-receiver.yml
## this is the incoming mail receiver container template
##
## After making changes to this file, you MUST rebuild
## /var/discourse/launcher rebuild mail-receiver
##
## BE *VERY* CAREFUL WHEN EDITING!
## YAML FILES ARE SUPER SUPER SENSITIVE TO MISTAKES IN WHITESPACE OR ALIGNMENT!
## visit http://www.yamllint.com/ to validate this file as needed
base_image: discourse/mail-receiver:release
update_pups: false
expose:
- "25:25" # SMTP
env:
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LANGUAGE: en_US.UTF-8
## Where e-mail to your forum should be sent. In general, it's perfectly fine
## to use the same domain as the forum itself here.
MAIL_DOMAIN: forum.[domain].co.nz
# uncomment these (and the volume below!) to support TLS
POSTCONF_smtpd_tls_key_file: /letsencrypt/forum.[domain].co.nz/forum.[domain].co.nz.key
POSTCONF_smtpd_tls_cert_file: /letsencrypt/forum.[domain].co.nz/fullchain.cer
POSTCONF_smtpd_tls_security_level: may
## The URL of the mail processing endpoint of your Discourse forum.
## This is simply your forum's base URL, with `/admin/email/handle_mail`
## appended. Be careful if you're running a subfolder setup -- in that case,
## the URL needs to have the subfolder included!
DISCOURSE_MAIL_ENDPOINT: 'https://forum.[domain].co.nz/admin/email/handle_mail'
## The master API key of your Discourse forum. You can get this from
## the "API" tab of your admin panel.
DISCOURSE_API_KEY: '074[rest of API key - yes I generated a new one limited to the system user]d98'
## The username to use for processing incoming e-mail. Unless you have
## renamed the `system` user, you should leave this as-is.
DISCOURSE_API_USERNAME: system
volumes:
- volume:
host: /var/discourse/shared/mail-receiver/postfix-spool
guest: /var/spool/postfix
# uncomment to support TLS
- volume:
host: /var/discourse/shared/standalone/letsencrypt
guest: /letsencrypt
When I send a new email and run ./launcher logs mail-receiver, here is what I see:
Dec 21 22:41:21 forum-mail-receiver postfix/smtpd[132]: connect from mail-pj1-f54.google.com[209.85.216.54]
Dec 21 22:41:23 forum-mail-receiver postfix/smtpd[132]: 16DAC379E42: client=mail-pj1-f54.google.com[209.85.216.54]
Dec 21 22:41:23 forum-mail-receiver postfix/cleanup[139]: 16DAC379E42: message-id=<94fc2bef18b410ae8b121c6af2da2df4@frontapp.com>
Dec 21 22:41:23 forum-mail-receiver postfix/qmgr[100]: 16DAC379E42: from=<[my email address]>, size=5585, nrcpt=1 (queue active)
<23>Dec 21 22:41:23 receive-mail[141]: Recipient: nobody@forum.[domain].co.nzDec 21 22:41:50 forum-mail-receiver postfix/smtpd[143]: connect from mail-oa1-f50.google.com[209.85.160.50]
Dec 21 22:41:52 forum-mail-receiver postfix/smtpd[143]: 2E445379E48: client=mail-oa1-f50.google.com[209.85.160.50]
Dec 21 22:41:52 forum-mail-receiver postfix/cleanup[139]: 2E445379E48: message-id=<6b2f9d646dc46f4fec4af006de01d3ae@frontapp.com>
Dec 21 22:41:52 forum-mail-receiver postfix/qmgr[100]: 2E445379E48: from=<[my email address]>, size=4100, nrcpt=1 (queue active)
<23>Dec 21 22:41:52 receive-mail[147]: Recipient: nobody@forum.[domain].co.nzDec 21 22:41:53 forum-mail-receiver postfix/smtpd[132]: disconnect from mail-pj1-f54.google.com[209.85.216.54] ehlo=2 starttls=1 mail=1 rcpt=1 bdat=1 quit=1 commands=7
Dec 21 22:41:58 forum-mail-receiver postfix/qmgr[100]: 1194937A670: from=<double-bounce@forum-mail-receiver.localdomain>, size=942, nrcpt=1 (queue active)
Dec 21 22:41:58 forum-mail-receiver postfix/smtp[149]: fatal: unknown service: smtp/tcp
Dec 21 22:41:59 forum-mail-receiver postfix/qmgr[100]: warning: private/smtp socket: malformed response
Dec 21 22:41:59 forum-mail-receiver postfix/qmgr[100]: warning: transport smtp failure -- see a previous warning/fatal/panic logfile record for the problem description
Dec 21 22:41:59 forum-mail-receiver postfix/master[1]: warning: process /usr/lib/postfix/sbin/smtp pid 149 exit status 1
Dec 21 22:41:59 forum-mail-receiver postfix/master[1]: warning: /usr/lib/postfix/sbin/smtp: bad command startup -- throttling
Dec 21 22:41:59 forum-mail-receiver postfix/error[150]: 1194937A670: to=<postmaster@forum-mail-receiver.localdomain>, orig_to=<postmaster>, relay=none, delay=1192, delays=1191/1/0/0.01, dsn=4.3.0, status=deferred (unknown mail transport error)
<19>Dec 21 22:42:23 receive-mail[141]: Failed to POST the e-mail to https://forum.sobercheck.co.nz/admin/email/handle_mail: execution expired (Net::OpenTimeout)<19>Dec 21 22:42:23 receive-mail[141]: /usr/lib/ruby/2.7.0/net/http.rb:960:in `initialize'
/usr/lib/ruby/2.7.0/net/http.rb:960:in `open'
/usr/lib/ruby/2.7.0/net/http.rb:960:in `block in connect'
/usr/lib/ruby/2.7.0/timeout.rb:105:in `timeout'
/usr/lib/ruby/2.7.0/net/http.rb:958:in `connect'
/usr/lib/ruby/2.7.0/net/http.rb:943:in `do_start'
/usr/lib/ruby/2.7.0/net/http.rb:932:in `start'
/usr/lib/ruby/2.7.0/net/http.rb:1483:in `request'
/usr/local/lib/site_ruby/mail_receiver/discourse_mail_receiver.rb:43:in `process'
/usr/local/bin/receive-mail:13:in `<main>'Dec 21 22:42:23 forum-mail-receiver postfix/pipe[140]: 16DAC379E42: to=<nobody@forum.[domain].co.nz>, relay=discourse, delay=60, delays=0.23/0.01/0/60, dsn=4.3.0, status=deferred (temporary failure)
Dec 21 22:42:25 forum-mail-receiver postfix/smtpd[143]: disconnect from mail-oa1-f50.google.com[209.85.160.50] ehlo=2 starttls=1 mail=1 rcpt=1 bdat=1 quit=1 commands=7
<19>Dec 21 22:42:52 receive-mail[147]: Failed to POST the e-mail to https://forum.[domain].co.nz/admin/email/handle_mail: execution expired (Net::OpenTimeout)<19>Dec 21 22:42:52 receive-mail[147]: /usr/lib/ruby/2.7.0/net/http.rb:960:in `initialize'
/usr/lib/ruby/2.7.0/net/http.rb:960:in `open'
/usr/lib/ruby/2.7.0/net/http.rb:960:in `block in connect'
/usr/lib/ruby/2.7.0/timeout.rb:105:in `timeout'
/usr/lib/ruby/2.7.0/net/http.rb:958:in `connect'
/usr/lib/ruby/2.7.0/net/http.rb:943:in `do_start'
/usr/lib/ruby/2.7.0/net/http.rb:932:in `start'
/usr/lib/ruby/2.7.0/net/http.rb:1483:in `request'
/usr/local/lib/site_ruby/mail_receiver/discourse_mail_receiver.rb:43:in `process'
/usr/local/bin/receive-mail:13:in `<main>'Dec 21 22:42:52 forum-mail-receiver postfix/pipe[146]: 2E445379E48: to=<nobody@forum.[domain].co.nz>, relay=discourse, delay=60, delays=0.15/0.01/0/60, dsn=4.3.0, status=deferred (temporary failure)
Dec 21 22:45:45 forum-mail-receiver postfix/anvil[135]: statistics: max connection rate 1/60s for (smtp:209.85.216.54) at Dec 21 22:41:21
Dec 21 22:45:45 forum-mail-receiver postfix/anvil[135]: statistics: max connection count 1 for (smtp:209.85.216.54) at Dec 21 22:41:21
Dec 21 22:45:45 forum-mail-receiver postfix/anvil[135]: statistics: max cache size 2 at Dec 21 22:41:50
I’m really stuck now, does anyone have any ideas what could be causing this? 
Doh! Yes. I conflated the TLS and the https.
This still isn’t working at all, no emails are getting passed on from mail-receiver to Discourse.
Can I ‘undo’ mail-receiver back to the start (reset it entirely) and start again, in the hopes that it will work?
How can I do this?
You can just edit the file and rebuild the mail container.
Thanks for the tip about the firewall! I also encountered troubles similar to @MathiasFoster, with the mail-receiver container unable to reach the forum site in the app container. A bit puzzling at first, since containers are being allowed to listen to the outside world without issue.
I am also using Vultr as my VPS provider with their Ubuntu OS image. Some combination of the OS image defaults plus Docker does indeed seem to block communication across containers.
Anyway, in my case, it was enough to allow HTTPS on the host:
$ ufw allow https
After that, the mail-receiver was able to deliver mail as expected.