Cannot upgrade / send emails / fetch themes... Docker problem?

[Paracelsus]

Hi,

In the last days we started suddenly having an issue with sending emails… then we’ve tried to upgrade/update without success (the update page hangs and every component - docker, discourse, plugins - get stuck at “Checking for new version…”) and then we tried to update everything via root but the rebuild fails giving the following message (among other things):

fatal: unable to access 'https://github.com/discourse/discourse.git/': Could not resolve host: github.com`

Our host has confirmed that there are no problems with the server connectivity, SSL, firewall or any other thing.

I suspect is something with docker. Even checked this topic: Docker sometimes fails to connect to github.com - Installation - Discourse Meta and tried:

# cd /var/discourse/
# ./launcher enter app
# wget -O - -v https://github.com/discourse/logster.git/ |wc

and got the following:

--2025-02-28 09:45:51--  https://github.com/discourse/logster.git
Resolving github.com (github.com)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘github.com’
      0       0       0

Any ideas on how to address this issue?

[Ed_S]

Are you operating in a special territory such as mainland China?

I would check DNS on your host and then check DNS in your docker.

check DNS on your host:

resolvectl status|egrep Servers
nslookup github.com

check DNS in your docker

docker exec -it app bash -c "cat /etc/resolv.conf"
docker exec -it app bash -c "wget -O - -v https://github.com/discourse/logster.git/ |wc"

[Paracelsus]

Ok, so:

on host DNS:

#resolvectl status|egrep Servers
DNS Servers: 8.8.8.8
Fallback DNS Servers: 8.8.8.8
DNS Servers: 8.8.8.8 8.8.4.4

#nslookup github.com
Server: 8.8.8.8
Address: 8.8.8.8#53

Non-authoritative answer:
Name:   github.com
Address: 140.82.112.3

On Docker DNS:

#docker exec -it app bash -c "cat /etc/resolv.conf"
# Generated by Docker Engine.
# This file can be edited; Docker Engine will not make further changes once it
# has been modified.

nameserver 8.8.8.8
nameserver 8.8.4.4

# Based on host file: '/etc/resolv.conf' (legacy)
# Overrides: [nameservers]

#docker exec -it app bash -c "wget -O - -v https://github.com/discourse/logster.git/ |wc"
--2025-02-28 12:20:17--  https://github.com/discourse/logster.git/
Resolving github.com (github.com)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘github.com’
      0       0       0

P.S: Our host operates in Canada.

[Ed_S]

Thanks… I have no idea how this might happen, but I’m not a docker expert.

What’s your host OS, and how did you install Discourse? Did you use a package supplied by the hosting company, or is it a standard install? (Or a dev install??) How did you install Docker?

But this all used to work, and then stopped working, so possibly it is docker related. Perhaps you can update your docker installation??

[Paracelsus]

It’s a standard install that have been working very well so far in AlmaLinux in the past few months (and before that well too in CentOS).

How can I update the docker installation (without the usual rebuild instruction with launcher or the through the website as none of these methods seem to be working)?

[Ed_S]

Before you go any further, please take a backup (if you can) and download a local copy. (Perhaps try Move a Discourse site to another VPS with rsync)

Please try these, they might give a hint:

docker --version
docker run hello-world

[Paracelsus]

With a bit of help from ChatGPT I found that it is the Docker’s Bridge network configuration or whatever that means that is the problem.

I’ve tried several fixes but so far without success. Haven’t tried this one yet:

Is this safe for a production environment? (even if it creates a short downtime)

[Ed_S]

I’m afraid I don’t know. I trust you have a safe recent backup. The least disruptive thing to do might be a reboot.

[Paracelsus]

I’m just posting to update on the situation. It got solved with help from ChatGPT.

After trying different solutions, this part showed that something was wrong with the permissions or security policies:

3. Try Running a Container with the --privileged Flag

This bypasses some restrictions that might be affecting networking.

docker run --rm --privileged alpine ping -c 4 github.com

If this works, it suggests a permissions or security policy issue.

After that, applied the next suggestion:

4. Check iptables Rules Again

Since Docker relies on iptables, let’s confirm it has the correct forwarding rules.

Run:

sudo iptables -t nat -L -v -n | grep -i docker

If no Docker-related rules appear, manually add them:

sudo iptables -P FORWARD ACCEPT

sudo iptables -A FORWARD -i docker0 -o eth0 -j ACCEPT

sudo iptables -t nat -A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE

Then restart Docker:

sudo systemctl restart docker

And test again.

… which totally did it.