Fix Discourse installation on Digital Ocean that broke during Docker update

We didn’t spend any time looking into what caused the problem because I wanted to get @waffleslop up and running as soon as possible. I’ve upgraded my Discourse (hosted on DigitalOcean) servers without a problem. However, I do use the command line rather than the GUI since I have a non-standard install.

I can recommend a few things to minimize the risk of extended downtime:

  1. Make a backup before doing anything! I wonder if there should be a warning in the interface strongly recommending a backup before you can do an update. A recent backup gives me comfort that at least we can spin up a new Droplet and restore things in the worst case.
  2. Make sure you can get to your backup! @waffleslop and I spent a considerable part of our time figuring out how to get a copy of /var/discourse to the new Droplet. Something very weird was going on with the original Droplet and we weren’t able to just scp the files to the new Droplet. For my own servers, I put backups on S3 and I copy them to my local machine every night. Is that excessive? Probably. But it does give me plenty of options when things stop working for some reason.
  3. Test your backups from time to time. When your production servers are down, you want to have confidence that you know what you are doing. Ideally you’d test a backup just before doing an update so that you have a place to fall back on if anything goes wrong with production. But it’s usually enough to try a backup as often as needed to keep the process fresh in your mind.
  4. Two heads are better than one. Maybe this is self-interest talking, but it can be a lot easier to get through an emergency if you can share your screen on a call with someone who has experience with this sort of situation. Ideally you want someone who knows how to use the command line.

As long as you make a backup, you should be pretty safe to upgrade.

5 Likes