Move from standalone container to separate web and data containers

[RichardNooooh]

I assume it’s not strictly necessary to download it. Most people are running a single cloud instance, who may or may not have a backup S3 bucket. Downloading the backup would be the only way for this group to make an off-site backup, like you mentioned.

I would even go as far as to make an additional backup system on a completely different provider as well, especially after what recently happened to this open source dev. Regardless of its validity, it’s a great warning to have a weekly or monthly backup to an entirely separate storage location/provider.

[merefield (Robert)]

@5minpause

./discourse-setup --two-container

Thanks @pfaffman for adding this, really painless now! (except for restoring a large backup which is always a little bit of a tense and long wait whatever the setup).

@5minpause

/var/discourse/shared/web_only/backups/default

btw I think this path is incorrect: should be web-only (by default) I think?:

volumes:
  - volume:
      host: /var/discourse/shared/web-only
      guest: /shared
  - volume:
      host: /var/discourse/shared/web-only/log/var-log
      guest: /var/log

PS I’ve edited the OP accordingly

[Jagster (Jakke Lehtonen)]

@5minpause

var/discourse/shared/web_only/backups/default

Yeah. web-only and that is a bit confusing when in every other context it is web_only. But perhaps when something is a directory or a container-what-ever is different thing.

What the I know… I just wasted 4 hours fighting with SearXNG and that should be 5 mins jobs (I really dislike docker things)

edit and off topic

Really he and ck is a banned word? It was teached to us in school as a non-offended word. So, they were wrong, obvious

[pfaffman (Jay Pfaffman)]

@Jagster

Really he and ck is a banned word?

I think that might have been added when watched words were first being tested and just never got removed.

@Jagster

Yeah. web-only and that is a bit confusing when in every other context it is web_only. But perhaps when something is a directory or a container-what-ever is different thing.

Yeah. I get confused about that often. In fact, I think that’s probably why once when I tried to just rename things to switch from single to double-container I got the underscore/dash thing wrong and it failed.

And worse, I am pretty sure it’s my fault. I got some error when I created the two-container option in discourse-setup (maybe containers couldn’t have underscores?) Ruby likes underscores in filenames, so maybe that’s why I used an underscore there? I think that’s it–and I think web_only can’t work as a docker container name since they also need to be valid hostnames.

[merefield (Robert)]

I prefer hyphens in directory paths, so all good as it is, and underscore in the container name honestly, makes sense, so leave it as it is.

btw, I think there should be a title or a self-certified badge on meta for those rocking the two container setup Once you’ve been here for a year, I think it should be mandatory for your standard installs to be migrated.

[pfaffman (Jay Pfaffman)]

@merefield

Once you’ve been here for a year, I think it should be mandatory for your standard installs to be migrated.

If not for so much existing documentation about the single-container setup, I’d almost argue that it should be the default, though there’d need to be some tooling to let people know that the database might need attention, or something.

I often see lots of people unhappy about and otherwise scared of two-container installs. (Recently someone wanted the two-container install I’d create when I did their install moved to a single container, for example.) It’s so very rare that it’s a problem, and the one time that it causes problems, it actually saves some hassle since it makes it easy to put off a Postgres Upgrade until you’re ready to do it. You can usually put off a PG upgrade for a good while (except when the AI plugin got added to core and required that extension).

[merefield (Robert)]

my backups started failing after this:

[2025-09-09 09:34:50] Creating archive: blah-forum-2025-09-09-093246-v20250828181952.tar.gz
[2025-09-09 09:34:50] Making sure archive does not already exist...
[2025-09-09 09:34:50] Creating empty archive...
[2025-09-09 09:34:50] EXCEPTION: tar --create --file /var/www/discourse/public/backups/default/blah-forum-2025-09-09-093246-v20250828181952.tar --files-from /dev/null
tar: /var/www/discourse/public/backups/default/blah-forum-2025-09-09-093246-v20250828181952.tar: Cannot open: Permission denied
tar: Error is not recoverable: exiting now

looking at permissions from within the web_only container:

/var/www/discourse/public/backups# ls -l
total 4
drwxr-xr-x 2 root root 4096 Sep  6 12:37 default

if I look at another instance (standard install), the ownership is different:

/var/www/discourse/public/backups# ls -l
total 4
drwxr-xr-x 2 discourse www-data 4096 Sep  9 03:46 default

what’s gone wrong here and what should I change on that directory for the web_only container - should it be the same as for standard install?

[pfaffman (Jay Pfaffman)]

@merefield

what’s gone wrong here and what should I change on that directory for the web_only container - should it be the same as for standard install?

tl;dr maybe try

docker exec -it web_only bash
chown  -R discourse:www-data /shared/backups

And more words.

Without looking, I’d next try rebuilding the data container, hoping that whatever change was made also got made to (or affects) the data container.

The bad-advice answer is to make ...backups/default world writable and seeing the ownership of the backup.

@merefield

drwxr-xr-x 2 root root 4096 Sep  6 12:37 default

@merefield

drwxr-xr-x 2 discourse www-data 4096 Sep  9 03:46 default

So I think what you want to do is chown default to discourse.www-data in the web container (that’s the one doing the backups).

Here’s a recent single-container:

root@forum.mbse-capella.org(app):~$ docker exec -it app  bash
root@new-app:/# grep www /etc/passwd
www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin
root@new-app:/# grep discourse /etc/passwd
discourse:x:1000:1000::/home/discourse:/bin/bash

At some points in the past the build process would chown all the files, but it can take a really long time, so I think that may have been removed at some point (this is more than a feeling than anything based on paying attention to commits).

[merefield (Robert)]

@pfaffman

docker exec -it web_only bash

this is equivalent to ./launcher enter web_only?

[pfaffman (Jay Pfaffman)]

@merefield

this is equivalent to ./launcher enter web_only?

Mostly. ./launcher will do a git pull (at least I thought so, but maybe not?) first and you’re more likely to have tab-completion working for docker than ./launcher.

[merefield (Robert)]

It also places you at the root, whereas launcher puts you in the discourse directory

[merefield (Robert)]

@pfaffman

docker exec -it web_only bash
chown  -R discourse:www-data /shared/backups`

/var/www/discourse/public/backups# ls -l
total 4
drwxr-xr-x 2 discourse www-data 4096 Sep  6 12:37 default

that seems to make the right tweak, so lets see how we get on with next backup, thanks!

[pfaffman (Jay Pfaffman)]

You can just run one now from the command line or the UX to see if it worked.

Also, if I’d been more clever:

docker exec -it web_only bash -c "chown  -R discourse:www-data /shared/backups"

[merefield (Robert)]

I was patient and waited for the scheduled job and that appears to have worked, so many thanks for your help @pfaffman

I guess I’m just left wondering if this will happen to others and if there needs to be a change somewhere to prevent that?