Failed upgrade from 2.5.0beta4 to 2.5.0beta5

Hello there,

the upgrade from 2.5.0beta4 to 2.5.0beta5 failed.

A docker container upgrade was noted (when I attempted the upgrade through the web UI, and then I hit the page “You are running an old version of the Discourse image.” and I need to go the the underling Ubuntu console to run the launcher rebuild app command.

From what I understand, after starting the postgresql server, attemptions to access it fail. This is the output of the launcher command. We use this setup for two years now, without experiencing any issues during discourse image upgrades until now.

root@host:/var/discourse# ./launcher rebuild my-app
Stopping old container
+ /usr/bin/docker stop -t 60 my-app
my-app
cd /pups && git pull && /pups/bin/pups --stdin
Already up to date.
I, [2020-05-30T10:14:28.336323 #1]  INFO -- : Loading --stdin
I, [2020-05-30T10:14:28.347207 #1]  INFO -- : > apt-get remove -y postgresql-10 postgresql-client-10 postgresql-contrib-10
I, [2020-05-30T10:14:30.056220 #1]  INFO -- : Reading package lists...
Building dependency tree...
Reading state information...
Package 'postgresql-10' is not installed, so not removed
Package 'postgresql-client-10' is not installed, so not removed
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

I, [2020-05-30T10:14:30.056724 #1]  INFO -- : > apt-get update && apt-get install -y postgresql-9.5 postgresql-client-9.5 postgresql-contrib-9.5
debconf: delaying package configuration, since apt-utils is not installed
I, [2020-05-30T10:14:51.560662 #1]  INFO -- : Hit:1 http://deb.debian.org/debian buster InRelease
Get:2 http://deb.debian.org/debian buster-updates InRelease [49.3 kB]
Get:3 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
Hit:4 https://deb.nodesource.com/node_10.x buster InRelease
Get:5 http://security.debian.org/debian-security buster/updates/main amd64 Packages [201 kB]
Get:6 http://apt.postgresql.org/pub/repos/apt buster-pgdg InRelease [84.6 kB]
Get:7 http://apt.postgresql.org/pub/repos/apt buster-pgdg/main amd64 Packages [171 kB]
Fetched 572 kB in 5s (127 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
Suggested packages:
  postgresql-doc-9.5 libdbd-pg-perl
The following NEW packages will be installed:
  postgresql-9.5 postgresql-client-9.5 postgresql-contrib-9.5
0 upgraded, 3 newly installed, 0 to remove and 9 not upgraded.
Need to get 5,700 kB of archives.
After this operation, 27.2 MB of additional disk space will be used.
Get:1 http://apt.postgresql.org/pub/repos/apt buster-pgdg/main amd64 postgresql-client-9.5 amd64 9.5.22-1.pgdg100+1 [1,252 kB]
Get:2 http://apt.postgresql.org/pub/repos/apt buster-pgdg/main amd64 postgresql-9.5 amd64 9.5.22-1.pgdg100+1 [3,985 kB]
Get:3 http://apt.postgresql.org/pub/repos/apt buster-pgdg/main amd64 postgresql-contrib-9.5 amd64 9.5.22-1.pgdg100+1 [463 kB]
Fetched 5,700 kB in 7s (828 kB/s)
Selecting previously unselected package postgresql-client-9.5.
(Reading database ... 43929 files and directories currently installed.)
Preparing to unpack .../postgresql-client-9.5_9.5.22-1.pgdg100+1_amd64.deb ...
Unpacking postgresql-client-9.5 (9.5.22-1.pgdg100+1) ...
Selecting previously unselected package postgresql-9.5.
Preparing to unpack .../postgresql-9.5_9.5.22-1.pgdg100+1_amd64.deb ...
Unpacking postgresql-9.5 (9.5.22-1.pgdg100+1) ...
Selecting previously unselected package postgresql-contrib-9.5.
Preparing to unpack .../postgresql-contrib-9.5_9.5.22-1.pgdg100+1_amd64.deb ...
Unpacking postgresql-contrib-9.5 (9.5.22-1.pgdg100+1) ...
Setting up postgresql-client-9.5 (9.5.22-1.pgdg100+1) ...
update-alternatives: warning: forcing reinstallation of alternative /usr/share/postgresql/12/man/man1/psql.1.gz because link group psql.1.gz is broken
Setting up postgresql-9.5 (9.5.22-1.pgdg100+1) ...
Creating new PostgreSQL cluster 9.5/main ...
/usr/lib/postgresql/9.5/bin/initdb -D /var/lib/postgresql/9.5/main --auth-local peer --auth-host md5
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "C.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/9.5/main ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default timezone ... Etc/UTC
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
creating template1 database in /var/lib/postgresql/9.5/main/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating collations ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
loading PL/pgSQL server-side language ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok
syncing data to disk ... ok

Success. You can now start the database server using:

    /usr/lib/postgresql/9.5/bin/pg_ctl -D /var/lib/postgresql/9.5/main -l logfile start

Ver Cluster Port Status Owner    Data directory               Log file
9.5 main    5433 down   postgres /var/lib/postgresql/9.5/main /var/log/postgresql/postgresql-9.5-main.log
update-alternatives: warning: forcing reinstallation of alternative /usr/share/postgresql/12/man/man1/postmaster.1.gz because link group postmaster.1.gz is broken
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
Setting up postgresql-contrib-9.5 (9.5.22-1.pgdg100+1) ...
Processing triggers for postgresql-common (213.pgdg100+1) ...
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
Removing obsolete dictionary files:

I, [2020-05-30T10:14:51.560954 #1]  INFO -- : > mkdir -p /shared/postgres_run
I, [2020-05-30T10:14:51.565020 #1]  INFO -- :
I, [2020-05-30T10:14:51.565382 #1]  INFO -- : > chown postgres:postgres /shared/postgres_run
I, [2020-05-30T10:14:51.568493 #1]  INFO -- :
I, [2020-05-30T10:14:51.568683 #1]  INFO -- : > chmod 775 /shared/postgres_run
I, [2020-05-30T10:14:51.571643 #1]  INFO -- :
I, [2020-05-30T10:14:51.571905 #1]  INFO -- : > rm -fr /var/run/postgresql
I, [2020-05-30T10:14:51.575191 #1]  INFO -- :
I, [2020-05-30T10:14:51.575460 #1]  INFO -- : > ln -s /shared/postgres_run /var/run/postgresql
I, [2020-05-30T10:14:51.578698 #1]  INFO -- :
I, [2020-05-30T10:14:51.578886 #1]  INFO -- : > socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1
2020/05/30 10:14:51 socat[1566] E connect(6, AF=1 "/shared/postgres_run/.s.PGSQL.5432", 36): No such file or directory
I, [2020-05-30T10:14:51.586239 #1]  INFO -- :
I, [2020-05-30T10:14:51.586497 #1]  INFO -- : > rm -fr /shared/postgres_run/.s*
I, [2020-05-30T10:14:51.590775 #1]  INFO -- :
I, [2020-05-30T10:14:51.591006 #1]  INFO -- : > rm -fr /shared/postgres_run/*.pid
I, [2020-05-30T10:14:51.595283 #1]  INFO -- :
I, [2020-05-30T10:14:51.595499 #1]  INFO -- : > mkdir -p /shared/postgres_run/9.5-main.pg_stat_tmp
I, [2020-05-30T10:14:51.598982 #1]  INFO -- :
I, [2020-05-30T10:14:51.599223 #1]  INFO -- : > chown postgres:postgres /shared/postgres_run/9.5-main.pg_stat_tmp
I, [2020-05-30T10:14:51.602578 #1]  INFO -- :
I, [2020-05-30T10:14:51.612032 #1]  INFO -- : File > /etc/service/postgres/run  chmod: +x  chown:
I, [2020-05-30T10:14:51.620151 #1]  INFO -- : File > /etc/runit/3.d/99-postgres  chmod: +x  chown:
I, [2020-05-30T10:14:51.620486 #1]  INFO -- : > chown -R root /var/lib/postgresql/9.5/main
I, [2020-05-30T10:14:51.636135 #1]  INFO -- :
I, [2020-05-30T10:14:51.636458 #1]  INFO -- : > [ ! -e /shared/postgres_data ] && install -d -m 0755 -o postgres -g postgres /shared/postgres_data && sudo -E -u postgres /usr/lib/postgresql/9.5/bin/initdb -D /shared/postgres_data || exit 0
I, [2020-05-30T10:14:51.640512 #1]  INFO -- :
I, [2020-05-30T10:14:51.640611 #1]  INFO -- : > chown -R postgres:postgres /shared/postgres_data
I, [2020-05-30T10:14:51.660141 #1]  INFO -- :
I, [2020-05-30T10:14:51.660411 #1]  INFO -- : > chown -R postgres:postgres /var/run/postgresql
I, [2020-05-30T10:14:51.665486 #1]  INFO -- :
I, [2020-05-30T10:14:51.665922 #1]  INFO -- : Replacing data_directory = '/var/lib/postgresql/9.5/main' with data_directory = '/shared/postgres_data' in /etc/postgresql/9.5/main/postgresql.conf
I, [2020-05-30T10:14:51.666711 #1]  INFO -- : Replacing (?-mix:#?listen_addresses *=.*) with listen_addresses = '*' in /etc/postgresql/9.5/main/postgresql.conf
I, [2020-05-30T10:14:51.667296 #1]  INFO -- : Replacing (?-mix:#?synchronous_commit *=.*) with synchronous_commit = $db_synchronous_commit in /etc/postgresql/9.5/main/postgresql.conf
I, [2020-05-30T10:14:51.667777 #1]  INFO -- : Replacing (?-mix:#?shared_buffers *=.*) with shared_buffers = $db_shared_buffers in /etc/postgresql/9.5/main/postgresql.conf
I, [2020-05-30T10:14:51.668237 #1]  INFO -- : Replacing (?-mix:#?work_mem *=.*) with work_mem = $db_work_mem in /etc/postgresql/9.5/main/postgresql.conf
I, [2020-05-30T10:14:51.668720 #1]  INFO -- : Replacing (?-mix:#?default_text_search_config *=.*) with default_text_search_config = '$db_default_text_search_config' in /etc/postgresql/9.5/main/postgresql.conf
I, [2020-05-30T10:14:51.669161 #1]  INFO -- : > install -d -m 0755 -o postgres -g postgres /shared/postgres_backup
I, [2020-05-30T10:14:51.673715 #1]  INFO -- :
I, [2020-05-30T10:14:51.674143 #1]  INFO -- : Replacing (?-mix:#?max_wal_senders *=.*) with max_wal_senders = $db_max_wal_senders in /etc/postgresql/9.5/main/postgresql.conf
I, [2020-05-30T10:14:51.674678 #1]  INFO -- : Replacing (?-mix:#?wal_level *=.*) with wal_level = $db_wal_level in /etc/postgresql/9.5/main/postgresql.conf
I, [2020-05-30T10:14:51.675160 #1]  INFO -- : Replacing (?-mix:#?checkpoint_segments *=.*) with checkpoint_segments = $db_checkpoint_segments in /etc/postgresql/9.5/main/postgresql.conf
I, [2020-05-30T10:14:51.675617 #1]  INFO -- : Replacing (?-mix:#?logging_collector *=.*) with logging_collector = $db_logging_collector in /etc/postgresql/9.5/main/postgresql.conf
I, [2020-05-30T10:14:51.676082 #1]  INFO -- : Replacing (?-mix:#?log_min_duration_statement *=.*) with log_min_duration_statement = $db_log_min_duration_statement in /etc/postgresql/9.5/main/postgresql.conf
I, [2020-05-30T10:14:51.676537 #1]  INFO -- : Replacing (?-mix:^#local +replication +postgres +peer$) with local replication postgres  peer in /etc/postgresql/9.5/main/pg_hba.conf
I, [2020-05-30T10:14:51.676970 #1]  INFO -- : Replacing (?-mix:^host.*all.*all.*127.*$) with host all all 0.0.0.0/0 md5 in /etc/postgresql/9.5/main/pg_hba.conf
I, [2020-05-30T10:14:51.677419 #1]  INFO -- : > HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/9.5/bin/postmaster -D /etc/postgresql/9.5/main
I, [2020-05-30T10:14:51.680769 #1]  INFO -- : > sleep 5
2020-05-30 10:14:51.788 UTC [1585] LOG:  database system was shut down at 2020-05-30 10:12:23 UTC
2020-05-30 10:14:51.791 UTC [1585] LOG:  MultiXact member wraparound protections are now enabled
2020-05-30 10:14:51.796 UTC [1582] LOG:  database system is ready to accept connections
2020-05-30 10:14:51.796 UTC [1589] LOG:  autovacuum launcher started
I, [2020-05-30T10:14:56.684099 #1]  INFO -- :
I, [2020-05-30T10:14:56.684539 #1]  INFO -- : > su postgres -c 'createdb discourse' || true
createdb: error: could not connect to database template1: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I, [2020-05-30T10:14:56.764870 #1]  INFO -- :
I, [2020-05-30T10:14:56.765721 #1]  INFO -- : > su postgres -c 'psql discourse -c "create user discourse;"' || true
psql: error: could not connect to server: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I, [2020-05-30T10:14:56.854570 #1]  INFO -- :
I, [2020-05-30T10:14:56.854902 #1]  INFO -- : > su postgres -c 'psql discourse -c "grant all privileges on database discourse to discourse;"' || true
psql: error: could not connect to server: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I, [2020-05-30T10:14:56.949901 #1]  INFO -- :
I, [2020-05-30T10:14:56.950243 #1]  INFO -- : > su postgres -c 'psql discourse -c "alter schema public owner to discourse;"'
psql: error: could not connect to server: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I, [2020-05-30T10:14:57.035955 #1]  INFO -- :
I, [2020-05-30T10:14:57.036482 #1]  INFO -- : Terminating async processes
I, [2020-05-30T10:14:57.036547 #1]  INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/9.5/bin/postmaster -D /etc/postgresql/9.5/main pid: 1582
2020-05-30 10:14:57.036 UTC [1582] LOG:  received fast shutdown request
2020-05-30 10:14:57.036 UTC [1582] LOG:  aborting any active transactions
2020-05-30 10:14:57.036 UTC [1589] LOG:  autovacuum launcher shutting down
2020-05-30 10:14:57.041 UTC [1586] LOG:  shutting down
2020-05-30 10:14:57.060 UTC [1586] LOG:  database system is shut down


FAILED
--------------------
Pups::ExecError: su postgres -c 'psql discourse -c "alter schema public owner to discourse;"' failed with return #<Process::Status: pid 1614 exit 2>
Location of failure: /pups/lib/pups/exec_command.rb:112:in `spawn'
exec failed with the params "su postgres -c 'psql $db_name -c \"alter schema public owner to $db_user;\"'"
cbc7ddb5e9371753be15c5763802fa765625651c985b3c5c4fe15109edec218d
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.

discourse-doctor wasn’t able to do anything:

DISCOURSE DOCTOR Sa 30. Mai 12:43:36 CEST 2020
OS: Linux cusanus.net 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux


Can't find app.yml or web_only.yml.
Giving up.

Our configuration is called my-app.yml. We need custom names, because we have several configurations.

Docker version:

docker --version
Docker version 19.03.10, build 9424aeaee9

Any help?

1 Like

We had a major Postgres version Upgrade.

I’d suggest you try editing your config to rebuild with postgres 10 template.

2 Likes

In a number of templates, the values: app.yml, web-only.yml and data.yml have been hard-coded.

2 Likes

Thanks, @itsbhanusharma! This helped me. We had been using "templates/postgres.9.5.template.yml". Changing to "templates/postgres.10.template.yml" did not work, but changing to the most recent "templates/postgres.template.yml" worked flawlessly. Thanks for the help!

5 Likes

That’s discourse-doctor. It is only designed to test setup deployed with ./discourse-setup … OP is using a renamed template

2 Likes

Correct.

FYI only, app.yml is still hardcoded into some comments (not a big deal of course), just FYI: see:

2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.