Warning: If you are not comfortable working as a Linux systems administrator, and do not have experience with docker containers, moving to a multi-container deployment will cause you difficulty, and both staff and volunteer help here will appropriately ask you to return to a standalone single-container deployment fully managed by the
If you move to a multi-container deployment and your system breaks as a result, you are likely to experience the opportunity to keep both broken pieces. If you read the instructions below and it feels like magic, rather than clarifying how things actually work inside the containers, run, don’t walk to your nearest default standalone deployment and you’ll do yourself a favor.
- Back up your discourse
- Throw the whole thing away
- Start over from scratch with a multi-container deployment
- Restore your backup
If, like me, you have a large site that takes hours to restore, you might wonder whether there is a faster way. Wonder no more! I migrated from a standalone deployment to a three-container web, data, and redis deployment in less time than it typically takes to
./launcher rebuild app for that site. (12 minutes of total downtime, when rebuilding the app has sometimes taken more than 30 minutes.) Based on my experience, I would keep Redis with Postgres in a single
data container in the future.
If you do this, you are taking responsibility for knowing when you need to rebuild your other containers (data, and if you are silly like me and split out redis, redis as well). You will no longer get free upgrades of everything with
./launcher rebuild app — if you don’t have the resources to manage this process, use a standalone deployment, or purchase hosted Discourse.
Do not use this process to migrate to multiple containers unless, after reading it, you now understand also how it would also tell you how to migrate quickly from multiple containers to a single container. If that is not obvious to you after reading this, then this post is sufficiently advanced technology (that is, indistinguishable from magic), and it is possible that you also might not recognize if this process breaks partway through, so you might end up with a broken Discourse that you don’t recognize until much later. If that happens, you will get to keep both broken pieces. You break it, you buy it, as they say!
Back up first, and turn on thumbnail backups first so that you don’t have to rebuild them all on restore. If you make a mistake here, you will easily get into a situation where the easiest, safest, and fastest way to recover is to switch to the normal method. Be ready to fall back to the recommended method if anything goes wrong.
Download your backup. The commands below involve moving files around in the Discourse data, and if you make a mistake, maybe you will have deleted your backup. So download it. And, if your backup doesn’t include uploads, back them up too. They are also located where you’ll be moving files around.
Seriously, back up.
When I did this, first I did a backup, then I did a remote system backup, before I went any further.
You will need at least
containers/data.yml and if you also want to split out redis, also
containers/redis.yml. Start by copying
samples/data.yml (and optionally
samples/redis.yml) to the
If you are deploying redis separately, remove the redis template from the top of the
containers/data.yml file. (But don’t do that without a good reason; it’s just extra work.)
You have two ways to create web_only.yml.
containers/; then compare both of them to
containers/app.yml, preserving any postgres configuration in
params:in your new
- Copy any
params:for postgres from
- Create a unique password to replace
- Alternatively, copy
containers/web_only.ymland compare it to
- Remove any references to the postgres and redis templates
- Remove the entire
params:section that had only postgres settings]
- Add a
links:section, verbatim from
samples/web_only.ymlor modified (see below) if you are deploying redis in a separate container
- Add a database section from
samples/web_only.ymland create a unique password to replace
- Change the volume definitions from
Here is the
links: section to use if you are splitting redis out into its own container instead of using the reasonable default of bundling it with postgres in the data container:
# Use 'links' key to link containers together, aka use Docker --link flag. links: - link: name: data alias: data - link: name: redis alias: redis
redis link is not needed if you are combining the redis and postgress containers as a single data container; this is to show what you would do.
Here is a copy of the current postgres settings in
samples/data.yml that you will need to change
## TODO: configure connectivity to the databases DISCOURSE_DB_SOCKET: '' #DISCOURSE_DB_USERNAME: discourse DISCOURSE_DB_PASSWORD: SOME_SECRET DISCOURSE_DB_HOST: data ## If you use a single data+redis container, the following will be "data" DISCOURSE_REDIS_HOST: redis
Note that for a normal deployment (not multisite) you will not need to modify any other lines.
DISCOURSE_DB_SOCKET is for a Unix domain socket for Postgres, it’s not a port number.
Here is an example of the change to the volumes definition at the end of web_only.yml that you will need to use if you copy it from app.yml instead of from
@@ -75,10 +80,10 @@ ## The Docker container is stateless; all data is stored in /shared volumes: - volume: - host: /var/discourse/shared/standalone + host: /var/discourse/shared/web_only guest: /shared - volume: - host: /var/discourse/shared/standalone/log/var-log + host: /var/discourse/shared/web_only/log/var-log guest: /var/log
Now set the same secret password you used in
containers/data.yml instead of
Now you are ready for the migration.
Now is when you take, and download, your final backup before you try the fast migration. Remember, if anything goes wrong here, you immediately go to the recommended method. I can’t stress this enough.
Separate data (postgres) and redis conainers:
cd /var/discourse ./launcher stop app cd shared mkdir data mkdir redis mv standalone/postgres_* data/ mv standalone/redis_data/ redis/ mv standalone web_only mkdir -p data/log/var-log mkdir -p redis/log/var-log cd .. ./launcher destroy app ./launcher bootstrap data ./launcher bootstrap redis ./launcher start redis ./launcher start data ./launcher bootstrap web_only ./launcher start web_only
Combined postgres+redis data container:
cd /var/discourse ./launcher stop app cd shared mkdir data mv standalone/postgres_* data/ mv standalone/redis_data/ data/ mv standalone web_only mkdir -p data/log/var-log cd .. ./launcher destroy app ./launcher bootstrap data ./launcher start data ./launcher bootstrap web_only ./launcher start web_only
Also note that if you have previously set up external nginx, you will need to change the
proxy_pass path to match the new
web_only socket location; say from
For me, on a 2-core VM with 4GB RAM and a site with 600MB backups without downloads, this process resulted in 12 minutes of downtime. Your milage may vary.
Note that none of this so far updates the launcher. You may not be up to date. (For example, I ran this after the postgres 12 update was available, but before I had applied it. This process left me with postgres 10. Then the very next thing I did was rebuild the data app, which updated launcher and took me successfully through the postgres 12 update process.)
After this migration, if you need to update redis or data, you need to first stop the web app. This would look something like this:
./launcher stop web_only ./launcher rebuild data # and/or redis ./launcher rebuild web_only
Note that if you rebuild the
redis) container, you will need to create a new web container to re-connect it with the new data container. You can do this either by rebuilding
web_only or, if you don’t think you need to rebuild it, a
./launcher destroy web_only; ./launcher start web_only will do the trick (and if you get an error about “missing data container” or similar, this is what you need to do).
However, when neither postgres nor redis needs an update, it’s much faster not to have to rebuild those containers, and most app rebuilds are just
./launcher rebuild web_only.
Alternatively, for even less downtime (variously reported between 15 seconds and 2 minutes):
./launcher bootstrap web_only ./launcher destroy web_only && ./launcher start web_only
Again, by moving to a multi-container deployment, keeping track of when that is appropriate is now your job. You’ll get notifications in the Admin console about updates, but they will apply only to the
web_only container. Nothing will tell you when you need to update postgres or redis. If you do this, read the Announcements category before every version upgrade you do, and read the release notes for every new version you are upgrading to or through. That is, if you skip a version update, do not skip reading the release notes for the version you skipped updating to. (Consider setting up a watch on release notes, or subscribing your feed reader to
https://meta.discourse.org/tag/release-notes.rss to stay up to date.)
Note that rebuilding the web_only container requires that the database be running, so you cannot speed things up by rebuilding all two or three containers in parallel. If you are going to rebuild them all every time, stick with the standard recommended standalone deployment; it will be faster than juggling multiple containers.
If you back up uploads separately from the database, I hope you have a file-based remote backup regime for uploads so that it’s being backed up to restore in case of disaster.
Review your remote backup implementation to make sure that it will back up uploads in /var/discourse/shared/web_only instead of in /var/discourse/shared/standalone so that you keep your backups up to date in your new multi-container implementation.