Configure direct-delivery incoming email for self-hosted sites

No. It was just temporarily down; it’s back up now.

I will doublecheck the API key now.

1 Like

I didn’t notice that the old API key had been revoked. I created a new one and I have evidence that mail is now getting to the right place. Thank you for directing me to check.

2 Likes

Thank you for taking the effort to write this guide. I wonder if with slight modification could be adapted to host the outgoing email too.

I see this guide which isn’t very complicated but it seems redundant to have one container running for receiving and another for sending.

No. Sending mail is very complicated. If you want to do that you’ll need to look elsewhere for instructions and support.

3 Likes

Then maybe it’s the other way around. I did the sending setup and now I don’t want to have a different container to do the receiving. probably I should ask the other side :slight_smile:

1 Like

You can look in the mail-receiver container for the bit that pushes the message to Discourse via the API.

Hm. I have the same Vultr setup as @MathiasFoster and @jryans here, and I ran into the same (Net::OpenTimeout) issue. ufw allow https got my incoming email working.

But now the site won’t send any mail. Before setting up incoming email, outgoing email had been working fine.

I don’t have anything fancy going on:
notification_email is admin@tasat.org.
Outgoing mail uses smtp.titan.email at Hostinger.

mail_receiver.yml contains:

`MAIL_DOMAIN` = forum.tasat.org
`DISCOURSE_MAIL_ENDPOINT` = https://forum.tasat.org/admin/email/handle_mail
`DISCOURSE_API_KEY` = [redacted]

In skipped email, I’m seeing <replies+verp-14c9cc6eb915b08d4983c90c744ba4b4@forum.tasat.org>: Sender address rejected: not owned by user admin@tasat.org

I’m new to email sausage-making. Should I take forum. out of the mail_receiver.yml strings and make “replies@tasat.org” a sending alias of “admin@tasat.org”…?

The mail receiver is independent of Discourse and email sending, it just acts as a simple email server configured only to receive emails, using the Discourse API to push those emails in to Discourse.

The only thing I can think of in setting it up that might affect sending is setting reply by email address, though that should just set Reply-To on the outbound emails and not affect the sender address.

Just to confirm, it sounds like you have these (among the rest) in app.yml:

DISCOURSE_SMTP_USER_NAME: admin@tasat.org
DISCOURSE_NOTIFICATION_EMAIL: admin@tasat.org

And this in reply by email address:

replies+%{reply_key}@forum.tasat.org

Is that right?

If so, do email notifications that don’t accept replies still work? I believe the email verification is an example of that, so you could try creating an account and see if that email is sent successfully.

With that arrangement, what should happen for emails that can accept replies is that the sent email uses:
From: admin@tasat.org (also sender address)
Reply-To: replies+abc123@forum.tasat.org

Typically Reply-To is not treated as part of the sender information, it just provides a default To address to use when people reply, but perhaps Hostinger do treat it as such. You might be able to add a sending alias for replies@forum.tasat.org.

1 Like

What happens if you try sending a test email to an address you get from https://www.mail-tester.com/?

I can’t see how changing ufw allow https could affect incoming mail.

It could be that vultr isn’t allowing outgoing connections to port 25. That would explain outgoing mail not working.

Outgoing and incoming email are largely separate from each other.

I believe Vultr (or maybe just installing Docker when ufw is present) have some rules that prevent containers talking to each other, meaning mail receiver can’t connect to Discourse. ufw allow https gets around that.

1 Like

But docker bypasses ufw, isn’t it?

1 Like

On the default network, only if a container connects directly to another container by local IP address, that is the local IP address of the container itself.

When mail receiver looks up your Discourse domain name, it won’t get that local IP so it has to leave its Docker network and will pass through ufw at least once to reach Discourse.

1 Like

I was thinking this topic, where you were participating too:

1 Like

That’s a separate situation, albeit related. That’s connections coming in from the outside and Docker adds rules to allow exposed ports through.

I’m not enormously familiar with netfilter/iptables chain rules but I believe the above is showing:

  1. If the connection is coming in from docker0, i.e. from the default docker network, return to the previous chain (stop processing rules in that chain).
  2. Otherwise if the connection is coming in from anything except docker0, if it’s also https or http, specify DNAT causing it to move on to the FORWARD chain.

So with the arrangement shown in the other topic, what happens is that if https or http traffic comes in from outside, it gets directed in to docker. If the traffic comes from the docker network however, it will be returned and rejected or dropped by the INPUT chain.

What ufw allow https does is adds a rule to the INPUT chain accepting it. That way when the connection is returned to the INPUT chain as above, it will be accepted and find docker listening, ultimately being routed to the container.

1 Like

@Simon_Manning @pfaffman

EDIT: see end of message

Thanks for the responses! I’ve had to be away for a bit, but am digging back into this…

Yes, that’s what I have at the moment.

When I try creating an account now, the submit button doesn’t do anything. As if it knows it won’t work. (And nothing shows up in Skipped or elsewhere.)

Edit: I have set “replies@tasat.org” as a sending alias for admin@tasat.org, and confirmed that it works for sending and receiving. I’ve also confirmed delivery of an email sent from an external client addressed to replies+verp-174bc7d8411bc4ec2cfa84c55bd31425@forum.tasat.org

In the interest of trying something, I changed reply by email address:
from replies+%{reply_key}@forum.tasat.org
to replies+%{reply_key}@tasat.org

But it doesn’t change the results.

It doesn’t arrive at mail-tester. All outgoing attempts end up under “skipped” with a variety of this message:

553 5.7.1 <replies+verp-8c79cd4e83023bda6df0624c2cacd36e@tasat.org>: 
Sender address rejected: not owned by user admin@tasat.org 

Maybe this is interesting…? When I run discourse-doctor, outbound email fails as follows:

==================== MAIL TEST ====================
For a robust test, get an address from http://www.mail-tester.com/
Sending mail to REDACTED  . . 
Testing sending to admin@tasat.org using smtp.titan.email:587, 
username:admin@tasat.org with plain auth.
SMTP server connection successful.
Sending to admin@tasat.org. . . 
Mail was not sent.

Reason: 553 5.7.1 <replies+verp-3cc19f7b135e6f56219e030999db9e29@tasat.org>: 
Sender address rejected: not owned by user admin@tasat.org

Sending directly to the replies+ address (or a forum.tasat.org address) from an email client works – it follows the “replies” alias and ends up in the admin inbox. Where is the rejection coming from?

I had a look at the article section “Prevent outgoing host email from interfering.” I have no path /etc/postfix – but here’s my netstat output:

root@forum:/var/discourse# netstat -tulpn | grep :25
tcp        0      0 0.0.0.0:25              0.0.0.0:*               LISTEN      1487576/docker-prox
tcp6       0      0 :::25                   :::*                    LISTEN      1487581/docker-prox

MAJOR EDIT – I heard back from Titan support tonight: “The reply-to address and from address should be the same or else the email won’t go through.” So I guess all the troubleshooting is out the window and I have to look for a new email host that doesn’t impose that requirement.

It’s certainly not exhaustive but there are some sending service recommendations in the install documentation along with some basic information about using them in Discourse: https://github.com/discourse/discourse/blob/main/docs/INSTALL-email.md

The following topic (linked at the bottom of that document) also has information about bounce handling for those and others:

I use Mailgun flex plan (entirely within free allowance) myself but I know there has been confusion around their pricing and potentially things have changed for new users since I joined. The last I saw (no idea if still true), it was possible to move to flex after the trial ended, it was just incredibly unclear.

Right.

You can switch to mailgun flex, though they make it hard to figure out. About once a month I email them asking about why it’s so hard to find.