Container fails startup after yum update

What does docker info prints?

1 Like
[root@xxx ~]# docker info
Containers: 1
 Running: 0
 Paused: 0
 Stopped: 1
Images: 38
Server Version: 1.12.5
Storage Driver: devicemapper
 Pool Name: vg_alder_main-docker--pool
 Pool Blocksize: 524.3 kB
 Base Device Size: 107.4 GB
 Backing Filesystem: xfs
 Data file:
 Metadata file:
 Data Space Used: 15.88 GB
 Data Space Total: 28.5 GB
 Data Space Available: 12.63 GB
 Metadata Space Used: 3.076 MB
 Metadata Space Total: 67.11 MB
 Metadata Space Available: 64.03 MB
 Thin Pool Minimum Free Space: 2.85 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: true
 Deferred Deletion Enabled: true
 Deferred Deleted Device Count: 0
 Library Version: 1.02.135-RHEL7 (2016-11-16)
Logging Driver: json-file
Cgroup Driver: systemd
Plugins:
 Volume: local
 Network: host bridge null overlay
Swarm: inactive
Runtimes: docker-runc runc
Default Runtime: docker-runc
Security Options: seccomp
Kernel Version: 3.10.0-514.6.1.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 2
CPUs: 2
Total Memory: 3.86 GiB
Name: alder.ndc.nasa.gov
ID: 4HCD:CXPT:UOD3:TH6S:6CZ3:XOEV:FYIF:GHBZ:OCIR:JBTX:UVG6:G37X
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
Insecure Registries:
 127.0.0.0/8
Registries: docker.io (secure)

You’re not running the same Docker binary as ./launcher is:

What happens if you try to run /usr/bin/docker.io run -it --rm busybox sh?

1 Like

[root@xxx ~]# /usr/bin/docker.io run -it --rm busybox sh
/ #

No errors

“Curiouser and curiouser!” cried Alice (she was so much surprised, that for the moment she quite forgot how to speak good English).

What about /usr/bin/docker.io start app? Also, does docker.io info give different results to docker info? Are they symlinked, or at least provided by the same RPM? If they’re in different RPMs, try removing the older one (it should be the one providing docker.io, because that’s a legacy name), and see if ./launcher becomes infinitely happier.

They are symlinked.

/usr/bin/docker.io start app gives the same error as ./launcher start app.

Oooooh… have you done a ./launcher rebuild app? Or at least a docker rm app? Try the latter first, and if that still doesn’t work, give the former a shot.

2 Likes

I had to rebuild, but now it works.

Thanks

1 Like

Isn’t it great how Docker makes everything so much easier…

5 Likes

Right… sooo much

:mdr:

You are using devicemapper, and this isn’t what we recommend as a storage drive, so keep that in mind.

3 Likes

Thanks, note to self.

You will experience drive corruption with devicemapper, it is just a matter of time. We block installs if devicemapper is detected.

3 Likes

Is this something you see as a permanent problem, or will later OS versions possibly allow reliable devmapper? Is it open bugs?

Devicemapper is not stable in our experience, so we don’t support it.

I’m wondering if that is an over simplified policy? Today turned ugly when I tried to update Discourse. It said I needed to update Docker. I was reminded that I never switched devmapper from loop mode to direct-lvm. According to the Docker site devmapper direct-lvm is the recommended method on Redhat and CentOS. Because AUFS is not supported. Now I have to use --skip-prereqs using the Docker recommended storage driver. Is the default loop mode actually the problem because people never do the xtra steps to direct-lvm (block level)?

> [root@xxxx ~]# docker info
> Containers: 2
>  Running: 0
>  Paused: 0
>  Stopped: 2
> Images: 39
> Server Version: 1.12.6
> Storage Driver: devicemapper
>  Pool Name: docker-thinpool
>  Pool Blocksize: 524.3 kB
>  Base Device Size: 10.74 GB
>  Backing Filesystem: xfs
>  Data file:
>  Metadata file:
>  Data Space Used: 490.7 MB
>  Data Space Total: 490.7 MB
>  Data Space Available: 0 B
>  Metadata Space Used: 122.9 kB
>  Metadata Space Total: 4.194 MB
>  Metadata Space Available: 4.071 MB
>  Thin Pool Minimum Free Space: 48.76 MB
>  Udev Sync Supported: true
>  Deferred Removal Enabled: true
>  Deferred Deletion Enabled: true
>  Deferred Deleted Device Count: 0
>  Library Version: 1.02.135-RHEL7 (2016-11-16)
> Logging Driver: json-file
> Cgroup Driver: systemd
> Plugins:
>  Volume: local
>  Network: host bridge null overlay
> Swarm: inactive
> Runtimes: docker-runc runc
> Default Runtime: docker-runc
> Security Options: seccomp
> Kernel Version: 3.10.0-514.10.2.el7.x86_64
> Operating System: CentOS Linux 7 (Core)
> OSType: linux
> Architecture: x86_64
> Number of Docker Hooks: 2
> CPUs: 2
> Total Memory: 3.86 GiB
> Name: xxxx
> ID: 4HCD:CXPT:UOD3:TH6S:6CZ3:XOEV:FYIF:GHBZ:OCIR:JBTX:UVG6:G37X
> Docker Root Dir: /var/lib/docker
> Debug Mode (client): false
> Debug Mode (server): false
> Registry: https://index.docker.io/v1/
> Insecure Registries:
>  127.0.0.0/8
> Registries: docker.io (secure)

Devicemapper in general was a dumpster fire for a long, long time. It may be that these days dm+loop is still bad but dm+lvm is OK, but… once bitten and all that.

Yesterday I had to drop to command line to update Discourse. This was the first time seeing the need to start using --ignore-prereqs. Will the web based updater fail from now on?

The web based updated will ask for a command line rebuild every time we make a change that depends on new OS level software, like updated nginx, ruby, postgresql, etc.

We upgrade database every two years, and ruby almost every year, for reference.

2 Likes

That wasn’t exactly the question. Yesterday the launcher command started refusing to work without --ignore-prereqs. But I’m using the default docker setup for redhat/centos. The ones you suggest are not available on my platform according to the docker site. So I wondered if the command line started demanding --ignore-prereqs, will the web updater fail now because you don’t have a way to “ignore” there.