Apologies ahead of time for some lack of details… I really got myself into it here…
If you know a lot about the file permission requirements in /var/discourse/shared/standalone you might be able to skip down to the bottom of this post for the core question & skip the details that got me here 
The latest web-based upgrade hosed my install - I think it was similar to Error Updating from 3.4.0.beta3-dev to 3.4.0.beta3 but I think I was on 3.4.0.beta4-dev
I’m also on Ubuntu 20, so now ./launcher rebuild app returns the error:
ERROR: Docker version 19.03.13 not supported, please upgrade to at least 20.10.0, or recommended 24.0.7
I don’t believe this is possible, so I created a new server and followed the steps in Move a Discourse site to another VPS with rsync with the additional command to bring over the volumes from my old server as well:
rsync -rvz root@104.236.77.249:/var/lib/docker/volumes/ /var/lib/docker/volumes
Now, on the new Ubuntu 24 server, ./launcher rebuild app is returning this error:
Stopping PostgreSQL 15 database server: mainError: Data directory /shared/postgres_data must not be owned by root … failed!
full command output
x86_64 arch detected.
WARNING: containers/app.yml file is world-readable. You can secure this file by running: chmod o-rwx containers/app.yml
Ensuring launcher is up to date
Launcher is up-to-date
2.0.20250129-0720: Pulling from discourse/base
Digest: sha256:01b8516e5504c0e9bc3707773015ff4407be03a89154194ff3b5b8699291bc26
Status: Image is up to date for discourse/base:2.0.20250129-0720
docker.io/discourse/base:2.0.20250129-0720
/usr/local/lib/ruby/gems/3.3.0/gems/pups-1.2.1/lib/pups.rb
/usr/local/bin/pups --stdin
I, [2025-02-05T15:28:12.350842 #1] INFO -- : Reading from stdin
I, [2025-02-05T15:28:12.364293 #1] INFO -- : File > /etc/service/postgres/run chmod: +x chown:
I, [2025-02-05T15:28:12.369358 #1] INFO -- : File > /etc/service/postgres/log/run chmod: +x chown:
I, [2025-02-05T15:28:12.373684 #1] INFO -- : File > /etc/runit/3.d/99-postgres chmod: +x chown:
I, [2025-02-05T15:28:12.378720 #1] INFO -- : File > /root/install_postgres chmod: +x chown:
I, [2025-02-05T15:28:12.383793 #1] INFO -- : File > /root/upgrade_postgres chmod: +x chown:
I, [2025-02-05T15:28:12.384811 #1] INFO -- : Replacing data_directory = '/var/lib/postgresql/15/main' with data_directory = '/shared/postgres_data' in /etc/postgresql/15/main/postgresql.conf
I, [2025-02-05T15:28:12.386020 #1] INFO -- : Replacing (?-mix:#?listen_addresses *=.*) with listen_addresses = '*' in /etc/postgresql/15/main/postgresql.conf
I, [2025-02-05T15:28:12.391549 #1] INFO -- : Replacing (?-mix:#?synchronous_commit *=.*) with synchronous_commit = $db_synchronous_commit in /etc/postgresql/15/main/postgresql.conf
I, [2025-02-05T15:28:12.392476 #1] INFO -- : Replacing (?-mix:#?shared_buffers *=.*) with shared_buffers = $db_shared_buffers in /etc/postgresql/15/main/postgresql.conf
I, [2025-02-05T15:28:12.393539 #1] INFO -- : Replacing (?-mix:#?work_mem *=.*) with work_mem = $db_work_mem in /etc/postgresql/15/main/postgresql.conf
I, [2025-02-05T15:28:12.394234 #1] INFO -- : Replacing (?-mix:#?default_text_search_config *=.*) with default_text_search_config = '$db_default_text_search_config' in /etc/postgresql/15/main/postgresql.conf
I, [2025-02-05T15:28:12.394860 #1] INFO -- : Replacing (?-mix:#?checkpoint_segments *=.*) with checkpoint_segments = $db_checkpoint_segments in /etc/postgresql/15/main/postgresql.conf
I, [2025-02-05T15:28:12.395701 #1] INFO -- : Replacing (?-mix:#?logging_collector *=.*) with logging_collector = $db_logging_collector in /etc/postgresql/15/main/postgresql.conf
I, [2025-02-05T15:28:12.396354 #1] INFO -- : Replacing (?-mix:#?log_min_duration_statement *=.*) with log_min_duration_statement = $db_log_min_duration_statement in /etc/postgresql/15/main/postgresql.conf
I, [2025-02-05T15:28:12.396997 #1] INFO -- : Replacing (?-mix:^#local +replication +postgres +peer$) with local replication postgres peer in /etc/postgresql/15/main/pg_hba.conf
I, [2025-02-05T15:28:12.397677 #1] INFO -- : Replacing (?-mix:^host.*all.*all.*127.*$) with host all all 0.0.0.0/0 md5 in /etc/postgresql/15/main/pg_hba.conf
I, [2025-02-05T15:28:12.398272 #1] INFO -- : Replacing (?-mix:^host.*all.*all.*::1\/128.*$) with host all all ::/0 md5 in /etc/postgresql/15/main/pg_hba.conf
I, [2025-02-05T15:28:12.398841 #1] INFO -- : > if [ -f /root/install_postgres ]; then
/root/install_postgres && rm -f /root/install_postgres
elif [ -e /shared/postgres_run/.s.PGSQL.5432 ]; then
socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1
fi
initdb: warning: enabling "trust" authentication for local connections
initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.
W: https://dl.yarnpkg.com/debian/dists/stable/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
debconf: delaying package configuration, since apt-utils is not installed
I, [2025-02-05T15:28:32.940792 #1] INFO -- : Generating locales (this might take a while)...
en_US.UTF-8... done
Generation complete.
Upgrading PostgreSQL from version 13 to 15
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /shared/postgres_data_new ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
/usr/lib/postgresql/15/bin/pg_ctl -D /shared/postgres_data_new -l logfile start
Get:1 http://deb.debian.org/debian bookworm-backports InRelease [59.0 kB]
Get:2 http://deb.debian.org/debian bookworm InRelease [151 kB]
Get:3 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
Get:4 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
Get:5 https://deb.nodesource.com/node_22.x nodistro InRelease [12.1 kB]
Get:6 https://dl.yarnpkg.com/debian stable InRelease [17.1 kB]
Get:7 http://deb.debian.org/debian bookworm-backports/main amd64 Packages [282 kB]
Get:8 http://deb.debian.org/debian bookworm/main amd64 Packages [8,792 kB]
Get:9 https://apt.postgresql.org/pub/repos/apt bookworm-pgdg InRelease [129 kB]
Get:10 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [13.5 kB]
Get:11 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [243 kB]
Get:12 https://deb.nodesource.com/node_22.x nodistro/main amd64 Packages [5,274 B]
Get:13 https://dl.yarnpkg.com/debian stable/main all Packages [10.9 kB]
Get:14 https://dl.yarnpkg.com/debian stable/main amd64 Packages [10.9 kB]
Get:15 https://apt.postgresql.org/pub/repos/apt bookworm-pgdg/main amd64 Packages [360 kB]
Fetched 10.2 MB in 2s (4,214 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
postgresql-client-13
Suggested packages:
postgresql-doc-13
The following NEW packages will be installed:
postgresql-13 postgresql-13-pgvector postgresql-client-13
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 17.3 MB of archives.
After this operation, 56.7 MB of additional disk space will be used.
Get:1 https://apt.postgresql.org/pub/repos/apt bookworm-pgdg/main amd64 postgresql-client-13 amd64 13.18-1.pgdg120+1 [1,523 kB]
Get:2 https://apt.postgresql.org/pub/repos/apt bookworm-pgdg/main amd64 postgresql-13 amd64 13.18-1.pgdg120+1 [15.4 MB]
Get:3 https://apt.postgresql.org/pub/repos/apt bookworm-pgdg/main amd64 postgresql-13-pgvector amd64 0.8.0-1.pgdg120+1 [297 kB]
Fetched 17.3 MB in 2s (8,411 kB/s)
Selecting previously unselected package postgresql-client-13.
(Reading database ... 33363 files and directories currently installed.)
Preparing to unpack .../postgresql-client-13_13.18-1.pgdg120+1_amd64.deb ...
Unpacking postgresql-client-13 (13.18-1.pgdg120+1) ...
Selecting previously unselected package postgresql-13.
Preparing to unpack .../postgresql-13_13.18-1.pgdg120+1_amd64.deb ...
Unpacking postgresql-13 (13.18-1.pgdg120+1) ...
Selecting previously unselected package postgresql-13-pgvector.
Preparing to unpack .../postgresql-13-pgvector_0.8.0-1.pgdg120+1_amd64.deb ...
Unpacking postgresql-13-pgvector (0.8.0-1.pgdg120+1) ...
Setting up postgresql-client-13 (13.18-1.pgdg120+1) ...
Setting up postgresql-13 (13.18-1.pgdg120+1) ...
Creating new PostgreSQL cluster 13/main ...
/usr/lib/postgresql/13/bin/initdb -D /var/lib/postgresql/13/main --auth-local peer --auth-host md5
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "C.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/13/main ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
pg_ctlcluster 13 main start
Warning: The selected stats_temp_directory /var/run/postgresql/13-main.pg_stat_tmp
is not writable for the cluster owner. Not adding this setting in
postgresql.conf.
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
Setting up postgresql-13-pgvector (0.8.0-1.pgdg120+1) ...
Processing triggers for postgresql-common (267.pgdg120+1) ...
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
Removing obsolete dictionary files:
Stopping PostgreSQL 13 database server: main.
Stopping PostgreSQL 15 database server: mainError: Data directory /shared/postgres_data must not be owned by root ... failed!
failed!
could not open version file "/shared/postgres_data/PG_VERSION": Permission denied
Failure, exiting
-------------------------------------------------------------------------------------
UPGRADE OF POSTGRES FAILED
Please visit https://meta.discourse.org/t/postgresql-15-update/349515 for support.
You can run ./launcher start app to restart your app in the meanwhile
-------------------------------------------------------------------------------------
FAILED
--------------------
Pups::ExecError: if [ -f /root/install_postgres ]; then
/root/install_postgres && rm -f /root/install_postgres
elif [ -e /shared/postgres_run/.s.PGSQL.5432 ]; then
socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1
fi
failed with return #<Process::Status: pid 18 exit 1>
Location of failure: /usr/local/lib/ruby/gems/3.3.0/gems/pups-1.2.1/lib/pups/exec_command.rb:132:in `spawn'
exec failed with the params {"tag"=>"db", "cmd"=>"if [ -f /root/install_postgres ]; then\n /root/install_postgres && rm -f /root/install_postgres\nelif [ -e /shared/postgres_run/.s.PGSQL.5432 ]; then\n socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1\nfi\n"}
bootstrap failed with exit code 1
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.
The /var/discourse/shared folder is owned by root on both old and new servers. Digging in some more levels, though, I notice that there were some permission differences in some folders in /var/discourse/shared/standalone…
Old server:
/var/discourse/shared/standalone# ls -alh
total 56K
drwxr-xr-x 14 root root 4.0K Jan 22 2021 .
drwxr-xr-x 3 root root 4.0K Mar 12 2016 ..
drwxr-xr-x 3 1000 www-data 4.0K Sep 21 2015 backups
drwxr-xr-x 8 root root 4.0K Dec 10 19:36 letsencrypt
drwxr-xr-x 4 root root 4.0K Jun 12 2015 log
drwxr-xr-x 2 syslog syslog 4.0K Jun 12 2015 postgres_backup
drwx------ 19 syslog syslog 4.0K Feb 4 20:03 postgres_data
drwx------ 20 colord landscape 4.0K Jan 22 2021 postgres_data_old
drwxrwsr-x 7 syslog syslog 4.0K Feb 4 20:03 postgres_run
drwxr-xr-x 2 landscape messagebus 4.0K Feb 4 20:03 redis_data
drwxr-xr-x 2 root root 4.0K Nov 7 2019 ssl
drwxr-xr-x 4 root root 4.0K Jun 12 2015 state
drwxr-xr-x 4 1000 www-data 4.0K Feb 4 20:03 tmp
drwxr-xr-x 4 1000 www-data 4.0K Jun 16 2017 uploads
New server:
/var/discourse/shared/standalone# ls -alh
total 60K
drwxr-xr-x 15 root root 4.0K Feb 5 15:32 .
drwxr-xr-x 3 root root 4.0K Feb 5 01:59 ..
drwxr-xr-x 3 root root 4.0K Feb 5 01:59 backups
drwxr-xr-x 8 root root 4.0K Feb 5 02:00 letsencrypt
drwxr-xr-x 4 root root 4.0K Feb 5 01:59 log
drwxr-xr-x 2 root root 4.0K Feb 5 01:59 postgres_backup
drwx------ 20 root root 4.0K Feb 5 02:42 postgres_data
drwx------ 20 messagebus tss 4.0K Feb 5 15:33 postgres_data_new
drwx------ 20 root root 4.0K Feb 5 02:04 postgres_data_older
drwxrwxr-x 8 messagebus tss 4.0K Feb 5 02:41 postgres_run
drwxr-xr-x 2 root root 4.0K Feb 5 02:05 redis_data
drwxr-xr-x 2 root root 4.0K Feb 5 02:05 ssl
drwxr-xr-x 4 root root 4.0K Feb 5 02:04 state
drwxr-xr-x 4 root root 4.0K Feb 5 02:04 tmp
drwxr-xr-x 4 root root 4.0K Feb 5 02:04 uploads
So I changed ownership on two of them in the new server:
chown -R syslog: postgres_data
chown -R syslog: postgres_backup
This changed the error message to something that I don’t know what to do with:
Stopping PostgreSQL 15 database server: mainError: Config owner (postgres:101) and data owner (messagebus:102) do not match, and config owner is not root … failed!
To clarify, my current permission in the new server are:
/var/discourse/shared/standalone# ls -alh
total 60K
drwxr-xr-x 15 root root 4.0K Feb 5 15:32 .
drwxr-xr-x 3 root root 4.0K Feb 5 01:59 ..
drwxr-xr-x 3 root root 4.0K Feb 5 01:59 backups
drwxr-xr-x 8 root root 4.0K Feb 5 02:00 letsencrypt
drwxr-xr-x 4 root root 4.0K Feb 5 01:59 log
drwxr-xr-x 2 syslog syslog 4.0K Feb 5 01:59 postgres_backup
drwx------ 20 syslog syslog 4.0K Feb 5 02:42 postgres_data
drwx------ 20 messagebus tss 4.0K Feb 5 15:33 postgres_data_new
drwx------ 20 root root 4.0K Feb 5 02:04 postgres_data_older
drwxrwxr-x 8 messagebus tss 4.0K Feb 5 02:41 postgres_run
drwxr-xr-x 2 root root 4.0K Feb 5 02:05 redis_data
drwxr-xr-x 2 root root 4.0K Feb 5 02:05 ssl
drwxr-xr-x 4 root root 4.0K Feb 5 02:04 state
drwxr-xr-x 4 root root 4.0K Feb 5 02:04 tmp
drwxr-xr-x 4 root root 4.0K Feb 5 02:04 uploads
I’m assuming some chown is going to fix this up for me, but I don’t know what/why/how…
