The one thing that is non-standard about my docker install, is that I don’t like it opening up my containers to the world, so I have added DOCKER_OPTS="--icc=true --iptables=false"
to /etc/default/docker
.
I then manually add iptables rules to allow the containers to communicate out. I don’t think this is causing the issue with the container not starting or even bootstrapping on 1.5.0 though. Everything works just fine on 1.4.0.
I’m a bit stumped with what’s going on here.
If nobody can give see any issues with what I’m doing, I’ll try set the daemon to default startup params and see if it’s still having an issue. Last resort will be to upgrade the OS to 14.10 and see if it makes any difference.
This is what happens after upgrading docker to 1.5.0 and rebooting:
Docker Info:
Containers: 4
Images: 42
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 53
Execution Driver: native-0.2
Kernel Version: 3.13.0-45-generic
Operating System: Ubuntu 14.04.2 LTS
CPUs: 8
Total Memory: 31.29 GiB
Name: nephilim
ID: 5R4W:3547:JCVF:OAML:6SGH:6FQL:EUKZ:QRCI:NENU:LAEC:FRXD:MOKS
WARNING: No swap limit support
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e3c6c3f66a2c local_discourse/web:latest "/sbin/boot" 32 hours ago Exited (143) 6 minutes ago web
f77454d020ee local_discourse/data:latest "/sbin/boot" 33 hours ago Up 4 minutes 0.0.0.0:2223->22/tcp, 0.0.0.0:5432->5432/tcp, 0.0.0.0:6379->6379/tcp
The data container auto-starts with no issue. Web throws a flappy.
Try rebuild:
user@nephilim:/var/discourse$ sudo git pull
Already up-to-date.
user@nephilim:/var/discourse$ sudo ./launcher rebuild web
WARNING: No swap limit support
Updating discourse docker
Already up-to-date.
Stopping old container
e3c6c3f66a2ce58bffff438259c87277cbc70d0ac717f249b63de5dcf791219e
Calculated ENV: -e LANG=en_US.UTF-8 -e HOME=/root -e RAILS_ENV=production -e UNICORN_WORKERS=6 -e UNICORN_SIDEKIQS=1 -e RUBY_GC_MALLOC_LIMIT=40000000 -e RUBY_HEAP_MIN_SLOTS=800000 -e DISCOURSE_DB_SOCKET= -e DISCOURSE_DB_HOST=data -e DISCOURSE_DB_PORT= -e DISCOURSE_DB_PASSWORD=xxx -e DISCOURSE_REDIS_HOST=data -e DISCOURSE_DEVELOPER_EMAILS=xxx -e DISCOURSE_HOSTNAME=xxx -e DISCOURSE_SMTP_ADDRESS=xxx -e DISCOURSE_SMTP_PORT=587 -e DISCOURSE_SMTP_USER_NAME=xxx -e DISCOURSE_SMTP_PASSWORD=xxx
cd /pups && git pull && /pups/bin/pups --stdin
FATA[0000] Error response from daemon: Cannot start container ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8: (exit status 1)
ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8
FAILED TO BOOTSTRAP
Here’s a bit from upstart/docker.log, not sure if this is helpful at all:
INFO[0343] -job attach(51c945b878e65c39e367af42b4f6847a95a27c4882cb676ce8deb7f39929579c) = OK (0)
INFO[0343] POST /v1.17/containers/51c945b878e65c39e367af42b4f6847a95a27c4882cb676ce8deb7f39929579c/wait
INFO[0343] +job wait(51c945b878e65c39e367af42b4f6847a95a27c4882cb676ce8deb7f39929579c)
INFO[0343] -job wait(51c945b878e65c39e367af42b4f6847a95a27c4882cb676ce8deb7f39929579c) = OK (0)
INFO[0343] GET /v1.17/containers/51c945b878e65c39e367af42b4f6847a95a27c4882cb676ce8deb7f39929579c/json
INFO[0343] +job container_inspect(51c945b878e65c39e367af42b4f6847a95a27c4882cb676ce8deb7f39929579c)
INFO[0343] -job container_inspect(51c945b878e65c39e367af42b4f6847a95a27c4882cb676ce8deb7f39929579c) = OK (0)
INFO[0343] DELETE /v1.17/containers/51c945b878e65c39e367af42b4f6847a95a27c4882cb676ce8deb7f39929579c?v=1
INFO[0343] +job rm(51c945b878e65c39e367af42b4f6847a95a27c4882cb676ce8deb7f39929579c)
INFO[0343] +job log(destroy, 51c945b878e65c39e367af42b4f6847a95a27c4882cb676ce8deb7f39929579c, samsaffron/discourse:1.0.7)
INFO[0343] -job log(destroy, 51c945b878e65c39e367af42b4f6847a95a27c4882cb676ce8deb7f39929579c, samsaffron/discourse:1.0.7) = OK (0)
INFO[0343] -job rm(51c945b878e65c39e367af42b4f6847a95a27c4882cb676ce8deb7f39929579c) = OK (0)
INFO[0343] POST /v1.17/containers/create
INFO[0343] +job create()
INFO[0343] +job log(create, ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e, samsaffron/discourse:1.0.7)
INFO[0343] -job log(create, ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e, samsaffron/discourse:1.0.7) = OK (0)
INFO[0343] -job create() = OK (0)
INFO[0343] POST /v1.17/containers/ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e/attach?stdin=1&stdout=1&stream=1
INFO[0343] +job container_inspect(ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e)
INFO[0343] -job container_inspect(ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e) = OK (0)
INFO[0343] +job attach(ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e)
INFO[0343] POST /v1.17/containers/ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e/start
INFO[0343] +job start(ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e)
INFO[0343] +job allocate_interface(ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e)
INFO[0343] -job allocate_interface(ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e) = OK (0)
INFO[0343] +job log(start, ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e, samsaffron/discourse:1.0.7)
INFO[0343] -job log(start, ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e, samsaffron/discourse:1.0.7) = OK (0)
INFO[0343] -job start(ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e) = OK (0)
INFO[0343] +job log(die, ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e, samsaffron/discourse:1.0.7)
INFO[0343] -job log(die, ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e, samsaffron/discourse:1.0.7) = OK (0)
INFO[0343] +job release_interface(ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e)
INFO[0343] -job release_interface(ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e) = OK (0)
INFO[0343] -job attach(ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e) = OK (0)
INFO[0343] POST /v1.17/containers/ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e/wait
INFO[0343] +job wait(ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e)
INFO[0343] -job wait(ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e) = OK (0)
INFO[0343] GET /v1.17/containers/ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e/json
INFO[0343] +job container_inspect(ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e)
INFO[0343] -job container_inspect(ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e) = OK (0)
INFO[0343] DELETE /v1.17/containers/ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e?v=1
INFO[0343] +job rm(ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e)
INFO[0343] +job log(destroy, ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e, samsaffron/discourse:1.0.7)
INFO[0343] -job log(destroy, ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e, samsaffron/discourse:1.0.7) = OK (0)
INFO[0343] -job rm(ede95d1be077d962c39e451f787d8c0ea2c4d8b5e4357fea3fd43d3c7315c63e) = OK (0)
INFO[0343] POST /v1.17/containers/create
INFO[0343] +job create()
INFO[0343] +job log(create, ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8, samsaffron/discourse:1.0.7)
INFO[0343] -job log(create, ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8, samsaffron/discourse:1.0.7) = OK (0)
INFO[0343] -job create() = OK (0)
INFO[0343] POST /v1.17/containers/ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8/attach?stderr=1&stdin=1&stdout=1&stream=1
INFO[0343] +job container_inspect(ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8)
INFO[0343] -job container_inspect(ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8) = OK (0)
INFO[0343] +job attach(ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8)
INFO[0343] POST /v1.17/containers/ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8/start
INFO[0343] +job start(ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8)
INFO[0343] +job allocate_interface(ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8)
INFO[0343] -job allocate_interface(ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8) = OK (0)
INFO[0343] +job link(-A)
iptables failed: iptables --wait -t filter -A DOCKER -i docker0 -o docker0 -p tcp -s 172.17.0.12 -d 172.17.0.2 --dport 22 -j ACCEPT: iptables: No chain/target/match by that name.
(exit status 1)
INFO[0343] -job link(-A) = ERR (1)
INFO[0343] +job link(-D)
INFO[0343] -job link(-D) = OK (0)
INFO[0343] +job release_interface(ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8)
INFO[0343] -job release_interface(ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8) = OK (0)
INFO[0343] +job log(die, ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8, samsaffron/discourse:1.0.7)
INFO[0343] -job log(die, ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8, samsaffron/discourse:1.0.7) = OK (0)
Cannot start container ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8: (exit status 1)
INFO[0343] -job start(ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8) = ERR (1)
ERRO[0343] Handler for POST /containers/{name:.*}/start returned error: Cannot start container ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8: (exit status 1)
ERRO[0343] HTTP Error: statusCode=500 Cannot start container ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8: (exit status 1)
INFO[0343] DELETE /v1.17/containers/ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8
INFO[0343] +job rm(ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8)
INFO[0344] +job log(destroy, ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8, samsaffron/discourse:1.0.7)
INFO[0344] -job log(destroy, ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8, samsaffron/discourse:1.0.7) = OK (0)
INFO[0344] -job rm(ed1c0d1c8dc38588cef767b4a70452637e0379c47db74399f81909cde7fcf8d8) = OK (0)