Self-hosted mail-receiver update following Let's Encrypt root certificate change

Product Support Self-hosting
,

AI-generated summary

This thread addresses a critical update for self-hosted Discourse users running the mail-receiver Docker image, which broke due to Let’s Encrypt’s root certificate expiration. david explains that only direct-delivery self-hosters are affected and provides steps to update: pull the new discourse/mail-receiver:release image and rebuild via ./launcher rebuild mail-receiver. Mails are queued on sender servers and should redeliver after the fix. bartv reports a git conflict during rebuild; david advises checking git diff and resetting changes, which bartv confirms resolves the issue. pfaffman shares a quick check to verify if the container runs Alpine (old) or Debian 11 (new). md-misko reports broken anti-spam measures post-update, possibly due to chroot or missing dnsutils, and links to related issues. Nacho_Caballero and Jonathan5 ask how to identify undelivered emails since October 1st; david suggests bounced messages would return to senders, and Nacho_Caballero suggests checking the incoming_emails PostgreSQL table — though Jonathan5 notes it’s empty for the relevant period. Simon_Manning proposes automating image pulls during rebuilds, which david agrees with, seeking input from @Falco. The update was prompted by wlandgraf and tested with community help.

1

This announcement only affects self-hosters who are running Configure direct-delivery incoming email for self-hosted sites with Mail-Receiver. Managed hosting customers, and self-hosters using POP3 for incoming mail are not affected.

As you may be aware, Let’s Encrypt recently changed their root certificate. The old root certificate expired today, and has caused a number of issues for out-of-date clients across the web. All of our systems at CDCK are up-to-date, and were not affected by today’s expiration. Unfortunately we missed the public mail-receiver docker image, which has not been updated for a few months.

This means that existing self-hosted mail-receiver installations will be unable to deliver mail to letsencrypt-secured sites.

We just pushed an updated version to DockerHub, which includes the new root certificate. Assuming you followed the official installation instructions, you can update your installation by running

docker pull discourse/mail-receiver:release
cd /var/discourse
./launcher rebuild mail-receiver

Mails received by broken installations will have been temporarily queued on the sending server. Those servers should attempt to redeliver the mail periodically, so any missed mails should arrive soon after you update the image.

If you’re still seeing issues after following these steps, make sure you’re running the release version of mail-receiver. You can find information on that in this topic.

Sorry for the disruption here! Big thanks to @wlandgraf for initially reporting the issue, and helping us test the fix :heart:

2

Thanks for the fix! My update got stuck with the following error message. I haven’t made any changes to this template, so I’m not sure what to do?

root@ba /var/discourse # ./launcher rebuild mail-receiver
Ensuring launcher is up to date
Fetching origin
Updating Launcher...
Updating 46d899f..990519e
error: Your local changes to the following files would be overwritten by merge:
	templates/web.letsencrypt.ssl.template.yml
Please commit your changes or stash them before you merge.
Aborting
failed to update
3

What happens if you

cd /var/discourse
git diff

Does it show any differences in the web.letsencrypt.ssl.template.yml file?

If so, you can reset them using git reset --hard

4

Ah, I did make a change :innocent: I got it to upgrade now, thanks!

5

You can test whether you’re running the old mail-receiver like this.

docker exec mail-receiver bash -c "cat /etc/issue|grep Alpine"

If it’s Alpine, it’s the old one.

docker exec mail-receiver bash -c "cat /etc/issue|grep 'Debian GNU/Linux 11'"

If it’s Debian, it’s the new one.

6

@david, would you mind having a look at the issues below, the latest update to the mail-receiver removed the ability to implement additional anti-spam measures, which worked wonders in the previous version, and my forum is under increasing spam pressure again due to the last change.

I tried to figure out what the root issue is, but I don’t have deep enough knowledge of postfix to figure it out. I found similar reports (client=unknown even when PTR record exists in DNS) regarding postfix in a chroot jail, so this might have something to do with it. Also, dnsutils seem to be missing from the new mail-receiver Debian image, but installing them still doesn’t fix the issue?

This one should be fixed easily:

7

@david Thanks for this fix! I just ran it and I can see it’s working. :+1:

Is there a way to see which emails were not able to be delivered since October 1st?

I’ve tried this, but I only see 5 requests (the earliest was received on Nov 26):

/var/discourse/launcher enter mail-receiver
mailq

I also tried looking at the logs, but I only got the warning reported here:

8

I think anything which isn’t still in the queue should have been bounced back to the sender. I’m afraid I’m not sure whether the container has any logs which would go back further than what you’ve found.

9

Would simply adding pull_image after line 657 resolve the need for explicitly pulling the image before the rebuild? I.e.

  # If the image being bootstrapped is not the base Discourse image
  if [[ ! X"" = X"$base_image" ]]; then
    image=$base_image
    # Attempt to bring the image up to date
    pull_image
  fi

This will cause a docker pull $image to always happen during a bootstrap/rebuild of a container with base_image set in its yml file. I don’t think that hurts for mail-receiver if it happens to already be up to date but I’m not sure if there are other situations where that could be a problem.

10

Yeah, an unconditional pull probably makes sense here. It’s a little strange that it only applies to our main base image at the moment. What do you think @Falco?

11

I’d be interested to hear a definitive answer to this question :slight_smile:

12

I think you can see it by querying the incoming_emails postgres table

13

Unfortunately that doesn’t show anything in the relevant time period (October 2021 to February 2022).

Would there be any logs in the mail-receiver container itself?