Cannot upgrade / send emails / fetch themes... Docker problem?

Product Support

AI-generated summary

Cannot upgrade / send emails / fetch themes… Docker problem?
The discussion revolves around Paracelsus experiencing issues with sending emails, upgrading, and fetching themes on their Discourse platform, suspecting a Docker problem. Ed_S suggests checking DNS on the host and in the Docker container, which Paracelsus does, revealing a DNS resolution issue within the Docker container.

Further investigation, including checking the Docker version and running a hello-world container, is suggested by Ed_S. Paracelsus discovers that the issue is related to Docker’s Bridge network configuration and considers resetting Docker networking, but is unsure about its safety in a production environment.

Ultimately, Paracelsus resolves the issue with help from ChatGPT, identifying a permissions or security policy problem and fixing it by checking iptables rules and manually adding Docker-related rules, followed by a Docker restart. The solution involves running a container with the --privileged flag and updating iptables rules to allow Docker forwarding, as shown in post 9.

1

Hi,

In the last days we started suddenly having an issue with sending emails… then we’ve tried to upgrade/update without success (the update page hangs and every component - docker, discourse, plugins - get stuck at “Checking for new version…”) and then we tried to update everything via root but the rebuild fails giving the following message (among other things):

fatal: unable to access 'https://github.com/discourse/discourse.git/': Could not resolve host: github.com`

Our host has confirmed that there are no problems with the server connectivity, SSL, firewall or any other thing.

I suspect is something with docker. Even checked this topic: Docker sometimes fails to connect to github.com - Installation - Discourse Meta and tried:

# cd /var/discourse/
# ./launcher enter app
# wget -O - -v https://github.com/discourse/logster.git/ |wc

and got the following:

--2025-02-28 09:45:51--  https://github.com/discourse/logster.git
Resolving github.com (github.com)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘github.com’
      0       0       0

Any ideas on how to address this issue?

2

Are you operating in a special territory such as mainland China?

I would check DNS on your host and then check DNS in your docker.

check DNS on your host:

resolvectl status|egrep Servers
nslookup github.com

check DNS in your docker

docker exec -it app bash -c "cat /etc/resolv.conf"
docker exec -it app bash -c "wget -O - -v https://github.com/discourse/logster.git/ |wc"
3

Ok, so:

on host DNS:

#resolvectl status|egrep Servers
DNS Servers: 8.8.8.8
Fallback DNS Servers: 8.8.8.8
DNS Servers: 8.8.8.8 8.8.4.4

#nslookup github.com
Server: 8.8.8.8
Address: 8.8.8.8#53

Non-authoritative answer:
Name:   github.com
Address: 140.82.112.3

On Docker DNS:

#docker exec -it app bash -c "cat /etc/resolv.conf"
# Generated by Docker Engine.
# This file can be edited; Docker Engine will not make further changes once it
# has been modified.

nameserver 8.8.8.8
nameserver 8.8.4.4

# Based on host file: '/etc/resolv.conf' (legacy)
# Overrides: [nameservers]

#docker exec -it app bash -c "wget -O - -v https://github.com/discourse/logster.git/ |wc"
--2025-02-28 12:20:17--  https://github.com/discourse/logster.git/
Resolving github.com (github.com)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘github.com’
      0       0       0

P.S: Our host operates in Canada.

4

Thanks… I have no idea how this might happen, but I’m not a docker expert.

What’s your host OS, and how did you install Discourse? Did you use a package supplied by the hosting company, or is it a standard install? (Or a dev install??) How did you install Docker?

But this all used to work, and then stopped working, so possibly it is docker related. Perhaps you can update your docker installation??

5

It’s a standard install that have been working very well so far in AlmaLinux in the past few months (and before that well too in CentOS).

How can I update the docker installation (without the usual rebuild instruction with launcher or the through the website as none of these methods seem to be working)?

6

Before you go any further, please take a backup (if you can) and download a local copy. (Perhaps try Move a Discourse site to another VPS with rsync)

Please try these, they might give a hint:

docker --version
docker run hello-world
7

With a bit of help from ChatGPT I found that it is the Docker’s Bridge network configuration or whatever that means that is the problem.

I’ve tried several fixes but so far without success. Haven’t tried this one yet:

The image provides steps to manually reset Docker networking by stopping Docker, deleting the network directory, and restarting Docker. (Captioned by AI)
The image provides steps to manually reset Docker networking by stopping Docker, deleting the network directory, and restarting Docker. (Captioned by AI)1170×727 54.6 KB

Is this safe for a production environment? (even if it creates a short downtime)

8

I’m afraid I don’t know. I trust you have a safe recent backup. The least disruptive thing to do might be a reboot.

9

I’m just posting to update on the situation. It got solved with help from ChatGPT.

After trying different solutions, this part showed that something was wrong with the permissions or security policies:

  1. Try Running a Container with the --privileged Flag

This bypasses some restrictions that might be affecting networking.

docker run --rm --privileged alpine ping -c 4 github.com

If this works, it suggests a permissions or security policy issue.

After that, applied the next suggestion:

  1. Check iptables Rules Again

Since Docker relies on iptables, let’s confirm it has the correct forwarding rules.

Run:

sudo iptables -t nat -L -v -n | grep -i docker

If no Docker-related rules appear, manually add them:

sudo iptables -P FORWARD ACCEPT

sudo iptables -A FORWARD -i docker0 -o eth0 -j ACCEPT

sudo iptables -t nat -A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE

Then restart Docker:

sudo systemctl restart docker

And test again.

… which totally did it. :grin: