Could not resolve host: github.com for SamSaffron/pups.git

docker

(Max Resnick) #1

Hi all -

We’re enjoying discourse thus far. I’m having an issue with running a rebuild of the docker container. The container won’t rebuild because for some reason it can’t resolve github.com. I know the host system (debian jessie) can resolve at that time via dig && ping. However if I issue a systemctl restart docker the issue goes away.

I turned to the goog or alphabet machine which suggested starting the docker daemon using the goog’s dns which I’ve done and confirmed that it seems to have taken via a systemctl status docker.

I’ve been able to reproduce on a debian droplet and a virtual box vm.

Also I’ve been able to successfully pull a Ubuntu image, and run the container.

I’m not sure if there’s something in launcher or any other suggestions? Working off of git master, and is up to date.

shortlist:

  • Debian Jessie Kernel 3.16
  • Docker version 1.6.2, build 7c8fca2

launch rebuild app

root@alpha:/var/discourse# ./launcher rebuild app
Ensuring discourse docker is up to date
Fetching origin
Discourse Docker is up-to-date
Stopping old container
+ /usr/bin/docker stop -t 10 app
app
cd /pups && git pull && /pups/bin/pups --stdin
fatal: unable to access 'https://github.com/SamSaffron/pups.git/': Could not resolve host: github.com
210c421dbe03e0339d110f1eec9eb3c985b0a0d4e32709d569c9cd1f235a7964
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one

systemctl status - shows --dns

root@alpha:/var/discourse# systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled)
   Active: active (running) since Thu 2015-08-13 03:42:17 UTC; 26min ago
     Docs: http://docs.docker.com
 Main PID: 22232 (docker)
   CGroup: /system.slice/docker.service
           └─22232 /usr/bin/docker -d -H fd:// --dns 8.8.8.8 --dns 8.8.4.4

Aug 13 04:03:40 alpha docker[22232]: time="2015-08-13T04:03:40Z" level=info msg="+job wait(7d602b0addff82e7626b659c5e30131a11ab7f35034198983a19e2b8861111b5)"
Aug 13 04:03:40 alpha docker[22232]: time="2015-08-13T04:03:40Z" level=info msg="-job wait(7d602b0addff82e7626b659c5e30131a11ab7f35034198983a19e2b88... = OK (0)"
Aug 13 04:03:40 alpha docker[22232]: time="2015-08-13T04:03:40Z" level=info msg="DELETE /v1.18/containers/7d602b0addff82e7626b659c5e30131a11ab7f3503...8861111b5"
Aug 13 04:03:40 alpha docker[22232]: time="2015-08-13T04:03:40Z" level=info msg="+job rm(7d602b0addff82e7626b659c5e30131a11ab7f35034198983a19e2b8861111b5)"
Aug 13 04:03:40 alpha docker[22232]: time="2015-08-13T04:03:40Z" level=info msg="+job log(destroy, 7d602b0addff82e7626b659c5e30131a11ab7f35034198983...e:1.0.12)"
Aug 13 04:03:40 alpha docker[22232]: time="2015-08-13T04:03:40Z" level=info msg="-job log(destroy, 7d602b0addff82e7626b659c5e30131a11ab7f35034198983... = OK (0)"
Aug 13 04:03:40 alpha docker[22232]: time="2015-08-13T04:03:40Z" level=info msg="-job rm(7d602b0addff82e7626b659c5e30131a11ab7f35034198983a19e2b8861... = OK (0)"
Aug 13 04:04:15 alpha docker[22232]: time="2015-08-13T04:04:15Z" level=info msg="GET /v1.18/containers/json"
Aug 13 04:04:15 alpha docker[22232]: time="2015-08-13T04:04:15Z" level=info msg="+job containers()"
Aug 13 04:04:15 alpha docker[22232]: time="2015-08-13T04:04:15Z" level=info msg="-job containers() = OK (0)"
Hint: Some lines were ellipsized, use -l to show in full.

docker info

root@alpha:/var/discourse# docker info
Containers: 3
Images: 18
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 24
 Dirperm1 Supported: true
Execution Driver: native-0.2
Kernel Version: 3.16.0-4-amd64
Operating System: Debian GNU/Linux 8 (jessie)
CPUs: 1
Total Memory: 2.879 GiB
Name: alpha
ID: WF4H:KABY:JPTX:RW2U:FWCT:PNKZ:DR2P:2ZFE:ZLUD:AB4R:D3CH:EXCA
WARNING: No memory limit support
WARNING: No swap limit support

edit: formatting fix code block


Cant send emails anymore
(Sam Saffron) #2

This is very likely the solution


(Max Resnick) #3

Hi @sam thanks for taking the time to review.
I have that option uncommented in /etc/defaults/docker, and I’ve restarted the docker service many times.

To clarify, after restarting the docker service the laucher rebuild will work once, after that I have to restart the service.

You can see this via my systemd status.

CGroup: /system.slice/docker.service
           └─22232 /usr/bin/docker -d -H fd:// --dns 8.8.8.8 --dns 8.8.4.4

#4

Same issue, I went to add the cloudflare template and when rebuilding it could not resolve github.com.

I have already tried the solution of using googles DNS servers in dockers configuration file with no success.

doing ping github.com works but again, it cant seem to resolve when rebuilding.

Here is the error:

Here is me pinging github.com
https://i.imgur.com/oUjgq8p.png

Interestingly though, when I ping6 github.com I of course do not recieve a response because github does not have AAAA records i believe.
https://i.imgur.com/Qu9OJg9.png

Any idea how to fix this?


(Matt Palmer) #5

This isn’t a Discourse problem, it’s a Docker problem. You’ll probably get more knowledgeable help at the Docker support forums (which, helpfully, also run Discourse).


#6

I figured out the issue and solution to this problem at least for me.

The issue was that Docker uses 8.8.8.8 and 8.8.4.4 as its DNS and I had an OVH network based firewall blocking ALL IPv4 traffic that did not match port 80, 443, and a few other specific rules. I had to authorize IPv4 traffic for 8.8.8.8 and 8.8.4.4