Cannot rebuild app because UPGRADE OF POSTGRES FAILED

Installation
1

I am single container and installed by official way

here is the log

Upgrading PostgreSQL from version 13 to 15
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /shared/postgres_data_new ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok


Success. You can now start the database server using:

    /usr/lib/postgresql/15/bin/pg_ctl -D /shared/postgres_data_new -l logfile start

Get:1 https://dl.yarnpkg.com/debian stable InRelease [17.1 kB]
Get:2 https://deb.nodesource.com/node_22.x nodistro InRelease [12.1 kB]
Get:3 http://deb.debian.org/debian bookworm-backports InRelease [59.0 kB]
Get:4 http://deb.debian.org/debian bookworm InRelease [151 kB]
Get:5 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
Get:6 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
Get:7 https://dl.yarnpkg.com/debian stable/main amd64 Packages [10.9 kB]
Get:8 https://dl.yarnpkg.com/debian stable/main all Packages [10.9 kB]
Get:9 https://deb.nodesource.com/node_22.x nodistro/main amd64 Packages [5,416 B]
Get:10 http://deb.debian.org/debian bookworm-backports/main amd64 Packages [283 kB]
Get:11 http://deb.debian.org/debian bookworm/main amd64 Packages [8,792 kB]
Get:12 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [13.5 kB]
Get:13 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [245 kB]
Get:14 https://apt.postgresql.org/pub/repos/apt bookworm-pgdg InRelease [129 kB]
Get:15 https://apt.postgresql.org/pub/repos/apt bookworm-pgdg/main amd64 Packages [360 kB]
Fetched 10.2 MB in 24s (417 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  postgresql-client-13
Suggested packages:
  postgresql-doc-13
The following NEW packages will be installed:
  postgresql-13 postgresql-13-pgvector postgresql-client-13
0 upgraded, 3 newly installed, 0 to remove and 3 not upgraded.
Need to get 17.3 MB of archives.
After this operation, 56.7 MB of additional disk space will be used.
Get:1 https://apt.postgresql.org/pub/repos/apt bookworm-pgdg/main amd64 postgresql-client-13 amd64 13.18-1.pgdg120+1 [1,523 kB]
Get:2 https://apt.postgresql.org/pub/repos/apt bookworm-pgdg/main amd64 postgresql-13 amd64 13.18-1.pgdg120+1 [15.4 MB]
Get:3 https://apt.postgresql.org/pub/repos/apt bookworm-pgdg/main amd64 postgresql-13-pgvector amd64 0.8.0-1.pgdg120+1 [297 kB]
Fetched 17.3 MB in 1min 12s (238 kB/s)
Selecting previously unselected package postgresql-client-13.
(Reading database ... 33363 files and directories currently installed.)
Preparing to unpack .../postgresql-client-13_13.18-1.pgdg120+1_amd64.deb ...
Unpacking postgresql-client-13 (13.18-1.pgdg120+1) ...
Selecting previously unselected package postgresql-13.
Preparing to unpack .../postgresql-13_13.18-1.pgdg120+1_amd64.deb ...
Unpacking postgresql-13 (13.18-1.pgdg120+1) ...
Selecting previously unselected package postgresql-13-pgvector.
Preparing to unpack .../postgresql-13-pgvector_0.8.0-1.pgdg120+1_amd64.deb ...
Unpacking postgresql-13-pgvector (0.8.0-1.pgdg120+1) ...
Setting up postgresql-client-13 (13.18-1.pgdg120+1) ...
Setting up postgresql-13 (13.18-1.pgdg120+1) ...
Creating new PostgreSQL cluster 13/main ...
/usr/lib/postgresql/13/bin/initdb -D /var/lib/postgresql/13/main --auth-local peer --auth-host md5
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "C.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/13/main ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

Success. You can now start the database server using:

    pg_ctlcluster 13 main start

invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
Setting up postgresql-13-pgvector (0.8.0-1.pgdg120+1) ...
Processing triggers for postgresql-common (267.pgdg120+1) ...
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
Removing obsolete dictionary files:
Stopping PostgreSQL 13 database server: main.
Stopping PostgreSQL 15 database server: main.
Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok

The source cluster was not shut down cleanly.
Failure, exiting
-------------------------------------------------------------------------------------
UPGRADE OF POSTGRES FAILED

Please visit https://meta.discourse.org/t/postgresql-15-update/349515 for support.

You can run ./launcher start app to restart your app in the meanwhile
-------------------------------------------------------------------------------------



FAILED
--------------------
Pups::ExecError: if [ -f /root/install_postgres ]; then
  /root/install_postgres && rm -f /root/install_postgres
elif [ -e /shared/postgres_run/.s.PGSQL.5432 ]; then
  socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1
fi
 failed with return #<Process::Status: pid 18 exit 1>
Location of failure: /usr/local/lib/ruby/gems/3.3.0/gems/pups-1.2.1/lib/pups/exec_command.rb:132:in `spawn'
exec failed with the params {"tag"=>"db", "cmd"=>"if [ -f /root/install_postgres ]; then\n  /root/install_postgres && rm -f /root/install_postgres\nelif [ -e /shared/postgres_run/.s.PGSQL.5432 ]; then\n  socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1\nfi\n"}
bootstrap failed with exit code 1
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.
82e5245f6cce6139328d093702db6d9cc57686bd1ef10025bdabff6c5f8151fe
2

This means you need to

and then ./launcher stop app afterwards and try a rebuild again.

3

Hello, I did

1 ./launcher start app
2 ./launcher stop app
3 ./launcher rebuild app

and the error was same

4

I may found the reason, after I run ./launcher stop app
The DB system did not shut down.
I waited for 3 mins already. any ideas?

2025-02-11 19:24:08.044 UTC [550] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2025-02-11 19:24:08.044 UTC [550] LOG:  listening on IPv6 address "::", port 5432
2025-02-11 19:24:08.046 UTC [550] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2025-02-11 19:24:08.065 UTC [569] LOG:  database system was interrupted; last known up at 2025-02-11 19:19:31 UTC
2025-02-11 19:24:08.323 UTC [569] LOG:  database system was not properly shut down; automatic recovery in progress
2025-02-11 19:24:08.326 UTC [569] LOG:  redo starts at 3/2BE89C0
2025-02-11 19:24:08.326 UTC [569] LOG:  invalid record length at 3/2BEFFC8: wanted 24, got 0
2025-02-11 19:24:08.326 UTC [569] LOG:  redo done at 3/2BEFFA0
2025-02-11 19:24:08.344 UTC [550] LOG:  database system is ready to accept connections
5

You want to start the container, wait for the database to be on a slow period and try again.

6

There are almost no user is using my discourse instance now. so I guess the database is on a slow period…

7

Could you try these steps?

2 Likes
8

It works. Thank you very much!