Configure direct-delivery incoming email for self-hosted sites with Mail-Receiver

Hello! I have a weird issue where I’ve set this up following the guide, and it works great! However, something has gone wrong with outbound email, which I thought wouldn’t be affected by any of this. Sidekiq gives the following error for every attempted email (all of them stuck in the Retries list) since I turned on mail-receiver:

Jobs::HandledExceptionWrapper: Wrapped OpenSSL::SSL::SSLError: SSL_read: unexpected eof while reading

My searching leads me to believe that this is related to TLS somehow. I had uncommented the TLS-related lines in the .yml file, but re-commenting them didn’t fix the issue either. I tried the instructions in the guide for resolving Postfix conflicts, but I apparently don’t have Postfix? (The /etc/postfix directory in the guide doesn’t exist on my instance, nor does it recognize postfix as a service.) And according to the netstat results, only docker-proxy is using port 25.

We’re using Gmail as the outbound SMTP service, and in fact we were using Gmail for inbound POP3 polling before this. I did delete a bunch of MX records pointing to Google, but the guide said to do that.

This is my mail-receiver.yml, with certain details redacted, of course:

## this is the incoming mail receiver container template
##
## After making changes to this file, you MUST rebuild
## /var/discourse/launcher rebuild mail-receiver
##
## BE *VERY* CAREFUL WHEN EDITING!
## YAML FILES ARE SUPER SUPER SENSITIVE TO MISTAKES IN WHITESPACE OR ALIGNMENT!
## visit http://www.yamllint.com/ to validate this file as needed

base_image: discourse/mail-receiver:release
update_pups: false

expose:
  - "25:25"   # SMTP

env:
  LC_ALL: en_US.UTF-8
  LANG: en_US.UTF-8
  LANGUAGE: en_US.UTF-8

  ## Where e-mail to your forum should be sent.  In general, it's perfectly fine
  ## to use the same domain as the forum itself here.
  MAIL_DOMAIN: discourse.[mydomain].org
# uncomment these (and the volume below!) to support TLS
  POSTCONF_smtpd_tls_key_file:  /letsencrypt/discourse.[mydomain].org/discourse.[mydomain].org.key
  POSTCONF_smtpd_tls_cert_file:  /letsencrypt/discourse.[mydomain].org/fullchain.cer
  POSTCONF_smtpd_tls_security_level: may


  ## The base URL for this Discourse instance.
  ## This will be whatever your Discourse site URL is. For example,
  ## https://discourse.example.com. If you're running a subfolder setup,
  ## be sure to account for that (ie https://example.com/forum).
DISCOURSE_BASE_URL: 'https://discourse.[mydomain].org'

  ## The master API key of your Discourse forum.  You can get this from
  ## the "API" tab of your admin panel.
  DISCOURSE_API_KEY: [myapikey]

  ## The username to use for processing incoming e-mail.  Unless you have
  ## renamed the `system` user, you should leave this as-is.
  DISCOURSE_API_USERNAME: system

volumes:
  - volume:
      host: /var/discourse/shared/mail-receiver/postfix-spool
      guest: /var/spool/postfix
# uncomment to support TLS
  - volume:
      host: /var/discourse/shared/standalone/letsencrypt
      guest: /letsencrypt

Email tech is a bit out of my expertise, so I appreciate any advice, even if it’s to point out that I missed something stupid when I was setting this up. Thanks!

As you thought, it’s nothing to do with the neck receiver. The host that you’re sending mail through has a broken ssl cert.

Well, I figured it out after much troubleshooting. The issue likely came from the fact that the domain we host our Discourse instance on is not the same as the domain our MX records were on. Once I got past that confusion, it all came together.

It’s definitely my own stupid mistake, but the guide contributed to my confusion a bit with this:

It’s not super clear that the two forum.example.com entries don’t have to be identical, and in my case they needed to be different. That may be something that people using this guide should be experienced enough to know, but I was not. So I’m leaving this here for anyone else who may run into a similar problem. I learned a few things about DNS that I didn’t know, so this was a good learning experience, and it’s all working great now. :slight_smile:

Well, I spoke too soon. Outbound email works fine, inbound replies seem to work fine, but posting to a category’s email address is failing silently. I copy/pasted the address straight from the settings into a new email, so I know there are no typos.

My mail-receiver’s logs basically have three types of entries. The successful one, which was an emailed reply to an existing post, looks like this:

Sep 20 16:59:44 discourse-mail-receiver postfix/smtpd[277]: connect from server168-1.web-hosting.com[68.65.122.144]
Sep 20 16:59:45 discourse-mail-receiver postfix/smtpd[277]: NOQUEUE: reject: RCPT from server168-1.web-hosting.com[68.65.122.144]: 454 4.7.1 <[category]@discourse.[domain].org>: Relay access denied; from=<ryan@[redacted].com> to=<[category]@discourse.[domain].org> proto=ESMTP helo=<server168-1.web-hosting.com>
<22>Sep 20 16:59:45 policyd-spf[288]: : prepend Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=[redacted]; helo=server168-1.web-hosting.com; envelope-from=ryan@[redacted].com; receiver=discourse.[domain].org Sep 20 16:59:45 discourse-mail-receiver postfix/cleanup[281]: 4CCED114200: message-id=<20240920165945.4CCED114200@discourse-mail-receiver.localdomain>
Sep 20 16:59:45 discourse-mail-receiver postfix/smtpd[277]: disconnect from server168-1.web-hosting.com[68.65.122.144] ehlo=1 starttls=0/1 mail=1 rcpt=0/1 data=0/1 quit=1 commands=3/6

Other than that, there are two types of (what I assume are) errors, each of which repeats quite a lot. The first one looks like:

Sep 20 17:00:23 discourse-mail-receiver postfix/qmgr[124]: 5D162FC26D: from=<double-bounce@discourse-mail-receiver.localdomain>, size=960, nrcpt=1 (queue active)

And the other:

Sep 20 17:00:23 discourse-mail-receiver postfix/error[293]: 8DC3BFC141: to=<postmaster@discourse-mail-receiver.localdomain>, orig_to=<postmaster>, relay=none, delay=126622, delays=126622/0.05/0/0, dsn=4.4.3, status=deferred (delivery temporarily suspended: Host or domain name not found. Name service error for name=discourse-mail-receiver.localdomain type=MX: Host not found, try again)

And here’s what my mailq looks like, just entries like this one over and over and over:

3D07BFC23D      960 Fri Sep 20 06:42:23  double-bounce@discourse-mail-receiver.localdomain
(delivery temporarily suspended: Host or domain name not found. Name service error for name=discourse-mail-receiver.localdomain type=MX: Host not found, try again)
                                         postmaster@discourse-mail-receiver.localdomain

Some of this looks like it has to do with emails that Discourse sends, which then get bounced back for whatever reason. Does mail-receiver have any sort of functionality to processes these bounces, or are they going to sit in the mailq forever?

Secondly, why do replies work, but email posting directly to a category does not? Thanks again for your help and your patience. :slight_smile:

That looks like the log entries for failure to a category’s address rather than success replying to a post.

I’m not 100% sure but I think relay access denied suggests discourse.[domain].org is probably not the domain used for MAIL_DOMAIN in mail-receiver.yml. Possibly the reply address is allowed through other means.

I know what’s used in MAIL_DOMAIN ends up in at least one place in a postfix config file so changing it probably requires rebuilding the container. Have you changed MAIL_DOMAIN and if so, did you run ./launcher rebuild mail-receiver afterwards?

2 Likes

[Sorry for accidentally hitting enter prematurely before finishing my previous post]

I am still banging my head against this issue. But I have a new idea on what the issue might be. I am working with two domains, let’s call them [domain1] and [domain2]. My Gmail SMTP relay is hosted on [domain1]. My Discourse instance, as well as my mail-receiver is hosted on [domain2].

How do I set the reply-by-email-address setting in Discourse to force a reply-to address in [domain2], when the email is being sent out from [domain1]? I get the SSL EOF error mentioned above when trying to do this. I assume there is some DNS authentication trickery or something I’m missing.

Looks like I’ve figured this out finally. In order to make the ‘reply-to’ address be in a different domain than the SMTP relay, there were some settings I needed to loosen up in Google Workspace. Everything seems to be working as intended in both directions.

1 Like

One final question here. Even though everything works correctly now, I still have a bunch of old entries in my mailq. These are most likely emails that were generated with the wrong settings and so will be stuck in limbo forever. I’d rather just delete them and move on. So, how do I clear the mailq?