My docker installation when I tried to update the docker manager via web-ui. I know my docker installation is broken because I followed sam’s recommendation and got this:
$ docker run -it --rm hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
1b930d010525: Pull complete
Digest: sha256:9572f7cdcee8591948c2963463447a53466950b3fc15a247fcad1917ca215a2f
Status: Downloaded newer image for hello-world:latest
docker: Error response from daemon: no status provided on response: unknown.
ERRO[0003] error waiting for container: context canceled
I then found this Github issue which seems to match what I’m seeing:
Since I’m on Ubuntu 16.04, it looks like this is the solultion:
Add a deb repo:
sudo add-apt-repository 'deb http://security.ubuntu.com/ubuntu xenial-security main universe'
Install runc (note the ~ s, github messed them up in @vietthang207 's original post):
Installing runc causes docker-ce to be removed, hence reinstall it:
sudo apt-get install docker-ce
Could someone confirm that it is save to do this or what the consequences will be? I’m a bit scared that it will uninstall docker. I suppose that means my site will be down during the process (strangely enough, it is currently still up and running, despite docker being broken). But is my discourse installation safe or will my containers also disappear?
Edit: This is how far I have dared to go:
$ sudo apt-get install runc=1.0.0~rc7+git20190403.029124da-0ubuntu1~16.04.4
[sudo] password for christoph:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
aufs-tools cgroupfs-mount docker-ce-cli libltdl7
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
containerd.io docker-ce
The following NEW packages will be installed:
runc
0 upgraded, 1 newly installed, 2 to remove and 21 not upgraded.
1 not fully installed or removed.
Need to get 1,890 kB of archives.
After this operation, 192 MB disk space will be freed.
Do you want to continue? [Y/n] n
Abort.
$
Have you rebooted? I’ve had trouble with Docker if I’ve done an upgrade and didn’t reboot.
Yeah. That’s scary-sounding. I know that I have sites on 16.04 and know that I haven’t done anything with runc.
If you’re concerned about down time and you’re on Digital Ocean, you might just
spin up a new droplet with 18.04 and do a clean install.
Get an floating IP, point it at the old server, and point DNS there.
Put the old forum in read-only
back up
restore to the new server
re-point the floating IP to the new server.
You’ll have no down time and just the read-only time it takes to back up and restore. You can practice a time or two before you do the read-only bit. You could also rsync uploads rather than including them in the backup, so you’d need to backup/restore just the database.
You’d probably want to pull over your existing let’s encrypt certs too, though, or, I think, you’ll have to rebuild to get them.
I was contemplating it but decided against it as I thought that since my docker installation is apparently broken but my forum still works, rebooting the server might also break the forum. What are the chances? Should I try rebooting?
Thanks for your explanation for minimising downtime. But no, I’m not on DO. But downtime is not a problem at all as long as I know it wont be for many hours or days.
If you upgraded Docker, it’s still installed, and this problem appeared when you upgraded, then I’m pretty sure that rebooting will solve your problem. Note: Free advice is worth what you pay for it.
Make a backup before you reboot. Worst case, with a backup and your app.yml you can spin up a new server and start anew in under an hour (unless you have 10s of GB of data, perhaps).
My plan would likely work for a number of hosting services.
If you’re super paranoid you could spin up a new server somewhere and pull a backup over there so if something went horribly wrong you’d have a fallback plan.
I then realized that I probably should use a different command because I have separate data and web_only containers so I tried ./launcher bootstrap web_only but it gave me the same error.
I guess I’ll try rebooting this weekend when I have the time to fix possible follow problems. Because: no, although I have set up three discourse instances that was years ago and “spinning up a new server and starting anew” would probably keep me busy for an entire day. Pointing my domain at the new IP address alone would be a potential nightmare to accomplish. Or maybe not, but then finding out that it’s not a big deal would also take some time…
Well, since I’ve spend hundreds of hours (much more time than if I’d just done “artisinal installations” provided by my predecessor) if you do know how to configure DNS, my installations are fully automated and take under an hour (most of the DNS propagation time).
