### Author:  [@CamilleRoux](https://meta.discourse.org/u/CamilleRoux)
#### Post date: [June 6, 2021, 5:03pm UTC](https://meta.discourse.org/t/postgresql-13-update-from-postgresql-10-fails/192865/1 "2021-06-06T17:03:46Z")
Continuing the discussion from [PostgreSQL 13 update](https://meta.discourse.org/t/postgresql-13-update/172563):
I’m in a multisite with 2 container configuration (only `postgres.10` and `redis` templates on the data container). The current version of postgres is 10 and I’d like to update to 13. Here is the error I get when I rebuild the `data` container:
```plaintext
fixing permissions on existing directory /shared/postgres_data_new ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
/usr/lib/postgresql/13/bin/pg_ctl -D /shared/postgres_data_new -l logfile start
Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
Hit:2 http://deb.debian.org/debian buster InRelease
Get:3 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]
Get:4 http://apt.postgresql.org/pub/repos/apt buster-pgdg InRelease [104 kB]
Get:5 http://security.debian.org/debian-security buster/updates/main amd64 Packages [291 kB]
Get:6 http://apt.postgresql.org/pub/repos/apt buster-pgdg/main amd64 Packages [231 kB]
Hit:7 https://deb.nodesource.com/node_15.x buster InRelease
Fetched 743 kB in 1s (944 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
postgresql-client-10
Suggested packages:
postgresql-doc-10
The following NEW packages will be installed:
postgresql-10 postgresql-client-10
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 6,441 kB of archives.
After this operation, 30.6 MB of additional disk space will be used.
Get:1 http://apt.postgresql.org/pub/repos/apt buster-pgdg/main amd64 postgresql-client-10 amd64 10.17-1.pgdg100+1 [1,439 kB]
Get:2 http://apt.postgresql.org/pub/repos/apt buster-pgdg/main amd64 postgresql-10 amd64 10.17-1.pgdg100+1 [5,002 kB]
Fetched 6,441 kB in 0s (34.9 MB/s)
Selecting previously unselected package postgresql-client-10.
(Reading database ... 43021 files and directories currently installed.)
Preparing to unpack .../postgresql-client-10_10.17-1.pgdg100+1_amd64.deb ...
Unpacking postgresql-client-10 (10.17-1.pgdg100+1) ...
Selecting previously unselected package postgresql-10.
Preparing to unpack .../postgresql-10_10.17-1.pgdg100+1_amd64.deb ...
Unpacking postgresql-10 (10.17-1.pgdg100+1) ...
Setting up postgresql-client-10 (10.17-1.pgdg100+1) ...
update-alternatives: warning: forcing reinstallation of alternative /usr/share/postgresql/13/man/man1/psql.1.gz because link group psql.1.gz is broken
Setting up postgresql-10 (10.17-1.pgdg100+1) ...
Creating new PostgreSQL cluster 10/main ...
/usr/lib/postgresql/10/bin/initdb -D /var/lib/postgresql/10/main --auth-local peer --auth-host md5
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "C.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/10/main ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default timezone ... Etc/UTC
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
pg_ctlcluster 10 main start
Ver Cluster Port Status Owner Data directory Log file
10 main 5433 down postgres /var/lib/postgresql/10/main /var/log/postgresql/postgresql-10-main.log
update-alternatives: warning: forcing reinstallation of alternative /usr/share/postgresql/13/man/man1/postmaster.1.gz because link group postmaster.1.gz is broken
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
Processing triggers for postgresql-common (226.pgdg100+1) ...
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
Removing obsolete dictionary files:
Stopping PostgreSQL 10 database server: main.
Stopping PostgreSQL 13 database server: main.
Performing Consistency Checks
-----------------------------
Checking cluster versions ok
The source cluster was not shut down cleanly.
Failure, exiting
-------------------------------------------------------------------------------------
UPGRADE OF POSTGRES FAILED
Please visit https://meta.discourse.org/t/postgresql-13-update/172563 for support.
You can run ./launcher start app to restart your app in the meanwhile
FAILED
--------------------
Pups::ExecError: /root/upgrade_postgres failed with return #
Location of failure: /pups/lib/pups/exec_command.rb:112:in `spawn'
exec failed with the params "/root/upgrade_postgres"
1f253827e5700e1861c4e586213aaffa8994e452e43b9336301dcd02072e00f4
**FAILED TO BOOTSTRAP** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.
```
If I set postgres.10 template again, everything works.
Do you know what I can do to upgrade to Postgres 13?
PS: Upgrade to postgres 12 is failing too
---
### Author:  [@CamilleRoux](https://meta.discourse.org/u/CamilleRoux)
#### Post date: [June 6, 2021, 5:28pm UTC](https://meta.discourse.org/t/postgresql-13-update-from-postgresql-10-fails/192865/2 "2021-06-06T17:28:00Z")
I saw that it’s require to stop de data container before:
> [@PostgreSQL 13 update](https://meta.discourse.org/t/postgresql-13-update/172563/1):
>
> If you are running a setup with a dedicated data container based in the sample supplied in our discourse\_docker repository, you want to be sure you are shutting down PostgreSQL in a safe and clean way.
>
> Nowadays, we have background jobs running queries spanning several minutes, so shutting down the web container will help the data container be shutdown safely.
>
> ```plaintext
> ./launcher stop web_only
> ./launcher stop data
> ./launcher rebuild data
> ./launcher rebuild data
> ./launcher rebuild web_only
>
> ```
Here is what I get I run `tail -f shared/data/log/var-log/postgres/current` after stopping the 2 containers:
```plaintext
2021-06-06 16:38:37.411 UTC [113] HINT: The server must be started by the user that owns the data directory.
2021-06-06 16:38:38.424 GMT [114] LOG: skipping missing configuration file "/shared/postgres_data/postgresql.auto.conf"
2021-06-06 16:38:38.424 UTC [114] FATAL: data directory "/shared/postgres_data" has wrong ownership
2021-06-06 16:38:38.424 UTC [114] HINT: The server must be started by the user that owns the data directory.
2021-06-06 16:38:39.439 GMT [115] LOG: skipping missing configuration file "/shared/postgres_data/postgresql.auto.conf"
2021-06-06 16:38:39.439 UTC [115] FATAL: data directory "/shared/postgres_data" has wrong ownership
2021-06-06 16:38:39.439 UTC [115] HINT: The server must be started by the user that owns the data directory.
2021-06-06 16:38:40.461 GMT [116] LOG: skipping missing configuration file "/shared/postgres_data/postgresql.auto.conf"
2021-06-06 16:38:40.461 UTC [116] FATAL: data directory "/shared/postgres_data" has wrong ownership
2021-06-06 16:38:40.461 UTC [116] HINT: The server must be started by the user that owns the data directory.
```
It’s probably the source of the problem, how can I fix that?
---
### Author:  [@Benjamin\_D](https://meta.discourse.org/u/Benjamin_D)
#### Post date: [June 6, 2021, 5:35pm UTC](https://meta.discourse.org/t/postgresql-13-update-from-postgresql-10-fails/192865/3 "2021-06-06T17:35:33Z")
out of curiosity, what is the owner of your `postgres_data` directory ?
---
### Author:  [@CamilleRoux](https://meta.discourse.org/u/CamilleRoux)
#### Post date: [June 6, 2021, 5:37pm UTC](https://meta.discourse.org/t/postgresql-13-update-from-postgresql-10-fails/192865/4 "2021-06-06T17:37:37Z")
Here it is
```plaintext
root@forum:/var/discourse/shared/data# ls -al
total 32
drwxr-xr-x 8 root root 4096 Jun 6 16:56 .
drwxr-xr-x 4 root root 4096 Mar 3 2019 ..
drwxr-xr-x 3 root root 4096 Jul 8 2018 log
drwxr-xr-x 2 _apt netdev 4096 Jul 7 2018 postgres_backup
drwx------ 20 _apt netdev 4096 Jun 6 17:29 postgres_data
drwx------ 19 _apt netdev 4096 Jun 6 16:56 postgres_data_new
drwxrwsr-x 5 _apt netdev 4096 Jun 6 17:29 postgres_run
drwxr-xr-x 2 lxd lxd 4096 Jun 6 17:34 redis_data
```
It is OK?
---
### Author:  [@Benjamin\_D](https://meta.discourse.org/u/Benjamin_D)
#### Post date: [June 6, 2021, 7:10pm UTC](https://meta.discourse.org/t/postgresql-13-update-from-postgresql-10-fails/192865/5 "2021-06-06T19:10:52Z")
for my setup (2 containers, postgres 13) group for postgres\_\* is render not netdev, and postgres\_run group right is x, not setgid. but I don’t really know what it implicates 😅
ps. and redis\_data is uuidd syslog …
---
### Author:  [@CamilleRoux](https://meta.discourse.org/u/CamilleRoux)
#### Post date: [June 6, 2021, 10:34pm UTC](https://meta.discourse.org/t/postgresql-13-update-from-postgresql-10-fails/192865/6 "2021-06-06T22:34:10Z")
Thanks, I don’t know neither 🙂
I hope someone could help!
---
### Author:  [@fitzy](https://meta.discourse.org/u/fitzy)
#### Post date: [June 8, 2021, 12:21am UTC](https://meta.discourse.org/t/postgresql-13-update-from-postgresql-10-fails/192865/7 "2021-06-08T00:21:30Z")
Could you please run the same command with `ls -lan`? That will help with determining ownership (`-n` converts the username to UID and group name to GID). Here’s an example of my own site with 2 container setup:
```plaintext
[root@/var/discourse/shared/data]$ ls -lan
total 28
drwxr-xr-x. 7 0 0 4096 Jun 7 22:15 .
drwxr-xr-x. 5 0 0 4096 Jun 7 22:28 ..
drwxr-xr-x. 3 0 0 4096 Jun 7 22:15 log
drwxr-xr-x. 2 105 109 4096 Jun 7 22:15 postgres_backup
drwx------. 19 105 109 4096 Jun 7 22:27 postgres_data
drwxrwxr-x. 3 105 109 4096 Jun 7 22:27 postgres_run
drwxr-xr-x. 2 106 110 4096 Jun 8 00:17 redis_data
```
---
### Author:  [@CamilleRoux](https://meta.discourse.org/u/CamilleRoux)
#### Post date: [June 9, 2021, 2:13pm UTC](https://meta.discourse.org/t/postgresql-13-update-from-postgresql-10-fails/192865/8 "2021-06-09T14:13:03Z")
sure! Here it is:
```plaintext
root@forum:/var/discourse/shared/data# ls -lan
total 32
drwxr-xr-x 8 0 0 4096 Jun 6 16:56 .
drwxr-xr-x 4 0 0 4096 Mar 3 2019 ..
drwxr-xr-x 3 0 0 4096 Jul 8 2018 log
drwxr-xr-x 2 105 109 4096 Jul 7 2018 postgres_backup
drwx------ 20 105 109 4096 Jun 6 17:29 postgres_data
drwx------ 19 105 109 4096 Jun 6 16:56 postgres_data_new
drwxrwsr-x 5 105 109 4096 Jun 6 17:29 postgres_run
drwxr-xr-x 2 106 110 4096 Jun 9 14:10 redis_data
```
---
### Author:  [@amotl](https://meta.discourse.org/u/amotl)
#### Post date: [October 13, 2021, 6:38pm UTC](https://meta.discourse.org/t/postgresql-13-update-from-postgresql-10-fails/192865/9 "2021-10-13T18:38:57Z")
Hi there,
we are in the same situation, yet on a standalone setup. `shared/standalone/log/var-log/` does not display any files at all, however after entering the container, we see identical results:
```plaintext
root@pulp-discourse-iptools:/var/www/discourse# tail -n 3 /var/log/postgres/current
2021-10-13 18:33:04.027 GMT [917] LOG: skipping missing configuration file "/shared/postgres_data/postgresql.auto.conf"
2021-10-13 18:33:04.028 UTC [917] FATAL: data directory "/shared/postgres_data" has wrong ownership
2021-10-13 18:33:04.028 UTC [917] HINT: The server must be started by the user that owns the data directory.
```
```plaintext
root@pulp-discourse-iptools:/var/www/discourse# ls -lan /shared
total 8
drwxr-xr-x 12 0 0 178 Oct 13 18:01 .
drwxr-xr-x 56 0 0 167 May 10 2020 ..
drwxr-xr-x 3 1000 33 21 Apr 3 2019 backups
drwxr-xr-x 4 0 0 34 Apr 3 2019 log
drwxr-xr-x 2 106 110 6 Apr 3 2019 postgres_backup
drwx------ 20 105 109 4096 Oct 13 17:57 postgres_data
drwx------ 19 105 109 4096 Oct 13 18:02 postgres_data_new
drwxrwsr-x 4 105 109 60 Oct 13 17:56 postgres_run
drwxr-xr-x 2 108 111 41 Oct 13 18:26 redis_data
drwxr-xr-x 4 0 0 44 Apr 3 2019 state
drwxr-xr-x 4 1000 33 37 Oct 13 18:26 tmp
drwxr-xr-x 4 1000 33 38 Apr 3 2019 uploads
```
However, that still looks weird:
```plaintext
root@pulp-discourse-iptools:/var/www/discourse# ls -alF /shared | grep postgres_
drwxr-xr-x 2 postgres postgres 6 Apr 3 2019 postgres_backup/
drwx------ 20 Debian-exim ssh 4096 Oct 13 18:53 postgres_data/
drwx------ 19 Debian-exim ssh 4096 Oct 13 18:02 postgres_data_new/
drwxrwsr-x 4 Debian-exim ssh 60 Oct 13 17:56 postgres_run/
```
In order to be more safe than sorry, we switched back to `"templates/postgres.10.template.yml"` in `app.yml` and currently only aim to bring the old instance up again.
Maybe it is sensible to share that we are already on the most recent commit of `https://github.com/discourse/discourse_docker.git`, after switching to the `main` branch. However, because the rebuild did not succeed, this probably has no impact when starting the previous image, right?
If anyone has any idea how to resolve this situation, we would be very grateful. Otherwise, we might come back with further reports.
With kind regards,
Andreas.
---
### Author:  [@amotl](https://meta.discourse.org/u/amotl)
#### Post date: [October 13, 2021, 7:01pm UTC](https://meta.discourse.org/t/postgresql-13-update-from-postgresql-10-fails/192865/10 "2021-10-13T19:01:55Z")
Hi again,
> [@amotl](#):
>
> `FATAL: data directory "/shared/postgres_data" has wrong ownership`
following the advice [Update failed (postgresql) - #8 by noezDE](https://meta.discourse.org/t/update-failed-postgresql/153713/8) by @noezDE (thanks a stack!), we have been able to recover from this situation.
Now, the site is running again and we will look into upgrading once again.
With kind regards,
Andreas.
* * *
### Recovery procedure
At first, progress was made by invoking
```plaintext
chown -R postgres /shared/postgres_data
sv restart postgres
```
After that, the logs said
```plaintext
2021-10-13 18:52:12.437 UTC [3357] FATAL: could not create lock file "/var/run/postgresql/.s.PGSQL.5432.lock": Permission denied
```
So, we went full-on and invoked
```plaintext
chown -R postgres:postgres /shared/postgres_*
sv restart postgres
```
Voilà.
---
### Author:  [@amotl](https://meta.discourse.org/u/amotl)
#### Post date: [October 13, 2021, 9:28pm UTC](https://meta.discourse.org/t/postgresql-13-update-from-postgresql-10-fails/192865/11 "2021-10-13T21:28:53Z")
> [@amotl](#):
>
> Now, the site is running again and we will look into upgrading once again.
Trying once more, with `"templates/postgres.template.yml"` again, everything went really smooth and the upgrade from PostgreSQL 10 to PostgreSQL 13 happened successfully. Thanks again to the whole Discourse team for their great work on this piece of software.