Urgent, rebuild failed from old version

Hi, our forum is running on an old version (for some front-end reasons).

version: 7647cf6a22aa7f9454123b24262ca04290b4d774  # v2.9.0.beta9

Today I tried to rebuild the app but failed with the following error:

2023-05-08 15:04:08.108 UTC [55] postgres@postgres ERROR:  database "discourse" already exists
2023-05-08 15:04:08.108 UTC [55] postgres@postgres STATEMENT:  CREATE DATABASE discourse;
createdb: error: database creation failed: ERROR:  database "discourse" already exists
I, [2023-05-08T15:04:08.110303 #1]  INFO -- :
I, [2023-05-08T15:04:08.110584 #1]  INFO -- : > su postgres -c 'psql discourse -c "create user discourse;"' || true
2023-05-08 15:04:08.178 UTC [59] postgres@discourse ERROR:  role "discourse" already exists
2023-05-08 15:04:08.178 UTC [59] postgres@discourse STATEMENT:  create user discourse;
ERROR:  role "discourse" already exists
FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle install --retry 3 --jobs 4' failed with return #<Process::Status: pid 455 exit 16>
Location of failure: /usr/local/lib/ruby/gems/3.2.0/gems/pups-1.1.1/lib/pups/exec_command.rb:117:in `spawn'
exec failed with the params {"cd"=>"$home", "hook"=>"bundle_exec", "cmd"=>["su discourse -c 'bundle config --local deployment true'", "su discourse -c 'bundle config --local without \"development test\"'", "su discourse -c 'bundle install --retry 3 --jobs 4'"]}
bootstrap failed with exit code 16
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.

Is there any way to rebuild successfully?
If it doesn’t work, I think it’s okay to restore the state of the last successful operation.
I tried to start the old container with the command ./launcher restart appbut I got a 502 error.
Thanks very much!

Output for docker images

REPOSITORY            TAG                 IMAGE ID       CREATED        SIZE
discourse/base        2.0.20230502-0058   e57f7d864cd3   6 days ago     3.41GB
local_discourse/app   latest              008ff91a3095   6 months ago   3.56GB
<none>                <none>              377935ab3fc8   6 months ago   3.56GB
<none>                <none>              91c1d188a205   6 months ago   3.43GB
<none>                <none>              89ddf039d845   7 months ago   3.4GB
<none>                <none>              e3160006f3d9   8 months ago   3.19GB

Outout for docker ps

CONTAINER ID   IMAGE                 COMMAND        CREATED        STATUS         PORTS                                                                      NAMES
1ab23a15e3c6   local_discourse/app   "/sbin/boot"   6 months ago   Up 6 minutes   0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp   app

Why would you rebuild if you didn’t want to upgrade? Or are you now ready to upgrade?

I think all you need to do may be to checkout discourse_docker (aka /var/discourse) to a version that uses a base image that works with 2.9.0.beta9.

You’ll need to include more of the output of your rebuild. I don’t think you’ve provided the output of the actual problem.

My guess is that the database is migrated part way and something (perhaps duplicate tags due to case mismatch) caused the database not to migrate all the say. If you do not want to upgrade to 3.1.0beta4 then you’ll need to restore to your latest backup.

Because I made some modifications to my own plugins :worried:

I think all you need to do may be to checkout discourse_docker (aka /var/discourse) to a version that uses a base image that works with 2.9.0.beta9.
Could u tell me how I can checkout a version that uses a base image works with 2.9.0.beta9? Thanks!

You’ll need to include more of the output of your rebuild. I don’t think you’ve provided the output of the actual problem.

Full output show below. It would be great if you could take a look

x86_64 arch detected.
Ensuring launcher is up to date
Fetching origin
remote: Enumerating objects: 258, done.
remote: Counting objects: 100% (224/224), done.
remote: Compressing objects: 100% (92/92), done.
remote: Total 258 (delta 155), reused 174 (delta 127), pack-reused 34
Receiving objects: 100% (258/258), 70.72 KiB | 14.14 MiB/s, done.
Resolving deltas: 100% (160/160), completed with 17 local objects.
From https://github.com/discourse/discourse_docker
   30be122..225c3d0  main               -> origin/main
 * [new branch]      build-27           -> origin/build-27
 * [new branch]      drop-env           -> origin/drop-env
 * [new branch]      gabe/before-pr-672 -> origin/gabe/before-pr-672
 * [new branch]      pgvector           -> origin/pgvector
 * [new branch]      revert-654-old-image-for-old-discourse -> origin/revert-654-old-image-for-old-discourse
 * [new branch]      yarn-timeout       -> origin/yarn-timeout
Updating Launcher...
Updating 30be122..225c3d0
Fast-forward
 .github/workflows/build.yml                   | 32 +++++++++++-----------
 image/auto_build.rb                           |  4 +--
 image/base/VERSION                            |  1 -
 image/base/install-imagemagick                | 20 +++++++++-----
 image/base/install-jemalloc                   | 10 +++----
 image/base/install-nginx                      |  4 +--
 image/base/install-oxipng                     | 27 ++++++++++---------
 image/base/install-redis                      |  4 +--
 image/base/install-ruby                       |  7 ++++-
 image/base/install-rust                       | 30 +++++++++++++++++++++
 image/base/release.Dockerfile                 |  2 +-
 image/base/slim.Dockerfile                    | 27 +++++++++++--------
 image/discourse_dev/Dockerfile                |  2 ++
 image/discourse_dev/postgres_dev.template.yml |  1 +
 image/discourse_test/Dockerfile               |  3 ++-
 launcher                                      | 21 ++++++++++-----
 samples/standalone.yml                        |  4 ++-
 samples/web_only.yml                          |  4 ++-
 templates/enable-ruby-yjit.yml                |  2 ++
 templates/web.ipv6.template.yml               |  8 ++++++
 templates/web.template.yml                    | 38 ++++++++++++++++++++-------
 21 files changed, 173 insertions(+), 78 deletions(-)
 delete mode 100644 image/base/VERSION
 create mode 100755 image/base/install-rust
 create mode 100644 templates/enable-ruby-yjit.yml
 create mode 100644 templates/web.ipv6.template.yml
Launcher updated, restarting...
x86_64 arch detected.

WARNING: We are about to start downloading the Discourse base image
This process may take anywhere between a few minutes to an hour, depending on your network speed

Please be patient

2.0.20230502-0058: Pulling from discourse/base
26c5c85e47da: Pull complete
d4326dc5fac5: Pull complete
9d375d0492ab: Pull complete
Digest: sha256:fa95da36c3d3a582d644b139ec678f5778d745697454bc86f598c689031b30aa
Status: Downloaded newer image for discourse/base:2.0.20230502-0058
docker.io/discourse/base:2.0.20230502-0058
Ensuring launcher is up to date
Fetching origin
Launcher is up-to-date
Stopping old container
+ /usr/bin/docker stop -t 600 app
app
2.0.20230502-0058: Pulling from discourse/base
Digest: sha256:fa95da36c3d3a582d644b139ec678f5778d745697454bc86f598c689031b30aa
Status: Image is up to date for discourse/base:2.0.20230502-0058
docker.io/discourse/base:2.0.20230502-0058
/usr/local/lib/ruby/gems/3.2.0/gems/pups-1.1.1/lib/pups.rb
/usr/local/bin/pups --stdin
I, [2023-05-08T15:03:54.075526 #1]  INFO -- : Reading from stdin
I, [2023-05-08T15:03:54.079505 #1]  INFO -- : > locale-gen $LANG && update-locale
I, [2023-05-08T15:03:54.155831 #1]  INFO -- : Generating locales (this might take a while)...
Generation complete.

I, [2023-05-08T15:03:54.156017 #1]  INFO -- : > mkdir -p /shared/postgres_run
I, [2023-05-08T15:03:54.168292 #1]  INFO -- :
I, [2023-05-08T15:03:54.168941 #1]  INFO -- : > chown postgres:postgres /shared/postgres_run
I, [2023-05-08T15:03:54.173737 #1]  INFO -- :
I, [2023-05-08T15:03:54.173986 #1]  INFO -- : > chmod 775 /shared/postgres_run
I, [2023-05-08T15:03:54.177575 #1]  INFO -- :
I, [2023-05-08T15:03:54.177705 #1]  INFO -- : > rm -fr /var/run/postgresql
I, [2023-05-08T15:03:54.179718 #1]  INFO -- :
I, [2023-05-08T15:03:54.179901 #1]  INFO -- : > ln -s /shared/postgres_run /var/run/postgresql
I, [2023-05-08T15:03:54.185548 #1]  INFO -- :
I, [2023-05-08T15:03:54.185724 #1]  INFO -- : > socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1
2023/05/08 15:03:54 socat[19] E connect(6, AF=1 "/shared/postgres_run/.s.PGSQL.5432", 36): No such file or directory
I, [2023-05-08T15:03:54.232120 #1]  INFO -- :
I, [2023-05-08T15:03:54.232474 #1]  INFO -- : > rm -fr /shared/postgres_run/.s*
I, [2023-05-08T15:03:54.235563 #1]  INFO -- :
I, [2023-05-08T15:03:54.235758 #1]  INFO -- : > rm -fr /shared/postgres_run/*.pid
I, [2023-05-08T15:03:54.237831 #1]  INFO -- :
I, [2023-05-08T15:03:54.237947 #1]  INFO -- : > mkdir -p /shared/postgres_run/13-main.pg_stat_tmp
I, [2023-05-08T15:03:54.239931 #1]  INFO -- :
I, [2023-05-08T15:03:54.240157 #1]  INFO -- : > chown postgres:postgres /shared/postgres_run/13-main.pg_stat_tmp
I, [2023-05-08T15:03:54.241689 #1]  INFO -- :
I, [2023-05-08T15:03:54.245869 #1]  INFO -- : File > /etc/service/postgres/run  chmod: +x  chown:
I, [2023-05-08T15:03:54.250215 #1]  INFO -- : File > /etc/service/postgres/log/run  chmod: +x  chown:
I, [2023-05-08T15:03:54.254535 #1]  INFO -- : File > /etc/runit/3.d/99-postgres  chmod: +x  chown:
I, [2023-05-08T15:03:54.259662 #1]  INFO -- : File > /root/upgrade_postgres  chmod: +x  chown:
I, [2023-05-08T15:03:54.259874 #1]  INFO -- : > chown -R root /var/lib/postgresql/13/main
I, [2023-05-08T15:04:02.637596 #1]  INFO -- :
I, [2023-05-08T15:04:02.637810 #1]  INFO -- : > [ ! -e /shared/postgres_data ] && install -d -m 0755 -o postgres -g postgres /shared/postgres_data && sudo -E -u postgres /usr/lib/postgresql/13/bin/initdb -D /shared/postgres_data || exit 0
I, [2023-05-08T15:04:02.640269 #1]  INFO -- :
I, [2023-05-08T15:04:02.640354 #1]  INFO -- : > chown -R postgres:postgres /shared/postgres_data
I, [2023-05-08T15:04:02.982260 #1]  INFO -- :
I, [2023-05-08T15:04:02.982484 #1]  INFO -- : > chown -R postgres:postgres /var/run/postgresql
I, [2023-05-08T15:04:02.984804 #1]  INFO -- :
I, [2023-05-08T15:04:02.984959 #1]  INFO -- : > /root/upgrade_postgres
I, [2023-05-08T15:04:02.988475 #1]  INFO -- :
I, [2023-05-08T15:04:02.988618 #1]  INFO -- : > rm /root/upgrade_postgres
I, [2023-05-08T15:04:02.990173 #1]  INFO -- :
I, [2023-05-08T15:04:02.993550 #1]  INFO -- : Replacing data_directory = '/var/lib/postgresql/13/main' with data_directory = '/shared/postgres_data' in /etc/postgresql/13/main/postgresql.conf
I, [2023-05-08T15:04:02.994262 #1]  INFO -- : Replacing (?-mix:#?listen_addresses *=.*) with listen_addresses = '*' in /etc/postgresql/13/main/postgresql.conf
I, [2023-05-08T15:04:02.994532 #1]  INFO -- : Replacing (?-mix:#?synchronous_commit *=.*) with synchronous_commit = $db_synchronous_commit in /etc/postgresql/13/main/postgresql.conf
I, [2023-05-08T15:04:02.994879 #1]  INFO -- : Replacing (?-mix:#?shared_buffers *=.*) with shared_buffers = $db_shared_buffers in /etc/postgresql/13/main/postgresql.conf
I, [2023-05-08T15:04:02.996787 #1]  INFO -- : Replacing (?-mix:#?work_mem *=.*) with work_mem = $db_work_mem in /etc/postgresql/13/main/postgresql.conf
I, [2023-05-08T15:04:02.998967 #1]  INFO -- : Replacing (?-mix:#?default_text_search_config *=.*) with default_text_search_config = '$db_default_text_search_config' in /etc/postgresql/13/main/postgresql.conf
I, [2023-05-08T15:04:02.999477 #1]  INFO -- : > install -d -m 0755 -o postgres -g postgres /shared/postgres_backup
I, [2023-05-08T15:04:03.006201 #1]  INFO -- :
I, [2023-05-08T15:04:03.006621 #1]  INFO -- : Replacing (?-mix:#?checkpoint_segments *=.*) with checkpoint_segments = $db_checkpoint_segments in /etc/postgresql/13/main/postgresql.conf
I, [2023-05-08T15:04:03.007035 #1]  INFO -- : Replacing (?-mix:#?logging_collector *=.*) with logging_collector = $db_logging_collector in /etc/postgresql/13/main/postgresql.conf
I, [2023-05-08T15:04:03.007382 #1]  INFO -- : Replacing (?-mix:#?log_min_duration_statement *=.*) with log_min_duration_statement = $db_log_min_duration_statement in /etc/postgresql/13/main/postgresql.conf
I, [2023-05-08T15:04:03.008361 #1]  INFO -- : Replacing (?-mix:^#local +replication +postgres +peer$) with local replication postgres  peer in /etc/postgresql/13/main/pg_hba.conf
I, [2023-05-08T15:04:03.008605 #1]  INFO -- : Replacing (?-mix:^host.*all.*all.*127.*$) with host all all 0.0.0.0/0 md5 in /etc/postgresql/13/main/pg_hba.conf
I, [2023-05-08T15:04:03.009930 #1]  INFO -- : Replacing (?-mix:^host.*all.*all.*::1\/128.*$) with host all all ::/0 md5 in /etc/postgresql/13/main/pg_hba.conf
I, [2023-05-08T15:04:03.010209 #1]  INFO -- : > HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main
I, [2023-05-08T15:04:03.011794 #1]  INFO -- : > sleep 5
2023-05-08 15:04:03.241 UTC [42] LOG:  starting PostgreSQL 13.10 (Debian 13.10-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2023-05-08 15:04:03.241 UTC [42] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2023-05-08 15:04:03.241 UTC [42] LOG:  listening on IPv6 address "::", port 5432
2023-05-08 15:04:03.257 UTC [42] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-05-08 15:04:03.338 UTC [45] LOG:  database system was shut down at 2023-05-08 15:03:36 UTC
2023-05-08 15:04:03.358 UTC [42] LOG:  database system is ready to accept connections
I, [2023-05-08T15:04:08.013914 #1]  INFO -- :
I, [2023-05-08T15:04:08.014151 #1]  INFO -- : > su postgres -c 'createdb discourse' || true
2023-05-08 15:04:08.108 UTC [55] postgres@postgres ERROR:  database "discourse" already exists
2023-05-08 15:04:08.108 UTC [55] postgres@postgres STATEMENT:  CREATE DATABASE discourse;
createdb: error: database creation failed: ERROR:  database "discourse" already exists
I, [2023-05-08T15:04:08.110303 #1]  INFO -- :
I, [2023-05-08T15:04:08.110584 #1]  INFO -- : > su postgres -c 'psql discourse -c "create user discourse;"' || true
2023-05-08 15:04:08.178 UTC [59] postgres@discourse ERROR:  role "discourse" already exists
2023-05-08 15:04:08.178 UTC [59] postgres@discourse STATEMENT:  create user discourse;
ERROR:  role "discourse" already exists
I, [2023-05-08T15:04:08.181863 #1]  INFO -- :
I, [2023-05-08T15:04:08.182271 #1]  INFO -- : > su postgres -c 'psql discourse -c "grant all privileges on database discourse to discourse;"' || true
I, [2023-05-08T15:04:08.227959 #1]  INFO -- : GRANT

I, [2023-05-08T15:04:08.228478 #1]  INFO -- : > su postgres -c 'psql discourse -c "alter schema public owner to discourse;"'
I, [2023-05-08T15:04:08.270180 #1]  INFO -- : ALTER SCHEMA

I, [2023-05-08T15:04:08.270520 #1]  INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists hstore;"'
NOTICE:  extension "hstore" already exists, skipping
I, [2023-05-08T15:04:08.317406 #1]  INFO -- : CREATE EXTENSION

I, [2023-05-08T15:04:08.317969 #1]  INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists pg_trgm;"'
NOTICE:  extension "pg_trgm" already exists, skipping
I, [2023-05-08T15:04:08.365325 #1]  INFO -- : CREATE EXTENSION

I, [2023-05-08T15:04:08.365687 #1]  INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists hstore;"'
NOTICE:  extension "hstore" already exists, skipping
I, [2023-05-08T15:04:08.410053 #1]  INFO -- : CREATE EXTENSION

I, [2023-05-08T15:04:08.410506 #1]  INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists pg_trgm;"'
NOTICE:  extension "pg_trgm" already exists, skipping
I, [2023-05-08T15:04:08.456640 #1]  INFO -- : CREATE EXTENSION

I, [2023-05-08T15:04:08.457086 #1]  INFO -- : > sudo -u postgres psql discourse
I, [2023-05-08T15:04:08.459410 #1]  INFO -- : update pg_database set encoding = pg_char_to_encoding('UTF8') where datname = 'discourse' AND encoding = pg_char_to_encoding('SQL_ASCII');

I, [2023-05-08T15:04:08.530021 #1]  INFO -- : File > /var/lib/postgresql/take-database-backup  chmod: +x  chown: postgres:postgres
I, [2023-05-08T15:04:08.532414 #1]  INFO -- : File > /var/spool/cron/crontabs/postgres  chmod:   chown:
I, [2023-05-08T15:04:08.532561 #1]  INFO -- : > echo postgres installed!
I, [2023-05-08T15:04:08.534589 #1]  INFO -- : postgres installed!

I, [2023-05-08T15:04:08.537818 #1]  INFO -- : File > /etc/service/redis/run  chmod: +x  chown:
I, [2023-05-08T15:04:08.540694 #1]  INFO -- : File > /etc/service/redis/log/run  chmod: +x  chown:
I, [2023-05-08T15:04:08.543387 #1]  INFO -- : File > /etc/runit/3.d/10-redis  chmod: +x  chown:
I, [2023-05-08T15:04:08.544184 #1]  INFO -- : Replacing daemonize yes with  in /etc/redis/redis.conf
I, [2023-05-08T15:04:08.544804 #1]  INFO -- : Replacing (?-mix:^pidfile.*$) with  in /etc/redis/redis.conf
I, [2023-05-08T15:04:08.545240 #1]  INFO -- : > install -d -m 0755 -o redis -g redis /shared/redis_data
I, [2023-05-08T15:04:08.548065 #1]  INFO -- :
I, [2023-05-08T15:04:08.548532 #1]  INFO -- : Replacing (?-mix:^logfile.*$) with logfile "" in /etc/redis/redis.conf
I, [2023-05-08T15:04:08.549102 #1]  INFO -- : Replacing (?-mix:^bind .*$) with  in /etc/redis/redis.conf
I, [2023-05-08T15:04:08.549628 #1]  INFO -- : Replacing (?-mix:^dir .*$) with dir /shared/redis_data in /etc/redis/redis.conf
I, [2023-05-08T15:04:08.551579 #1]  INFO -- : Replacing (?-mix:^protected-mode yes) with protected-mode no in /etc/redis/redis.conf
I, [2023-05-08T15:04:08.552183 #1]  INFO -- : Replacing # io-threads 4 with io-threads $redis_io_threads in /etc/redis/redis.conf
I, [2023-05-08T15:04:08.552661 #1]  INFO -- : > echo redis installed
I, [2023-05-08T15:04:08.554960 #1]  INFO -- : redis installed

I, [2023-05-08T15:04:08.555271 #1]  INFO -- : > cat /etc/redis/redis.conf | grep logfile
I, [2023-05-08T15:04:08.559569 #1]  INFO -- : logfile ""

I, [2023-05-08T15:04:08.559811 #1]  INFO -- : > exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf
I, [2023-05-08T15:04:08.560965 #1]  INFO -- : > sleep 10
103:C 08 May 2023 15:04:08.575 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
103:C 08 May 2023 15:04:08.575 # Redis version=7.0.7, bits=64, commit=00000000, modified=0, pid=103, just started
103:C 08 May 2023 15:04:08.575 # Configuration loaded
103:M 08 May 2023 15:04:08.575 * monotonic clock: POSIX clock_gettime
103:M 08 May 2023 15:04:08.578 * Running mode=standalone, port=6379.
103:M 08 May 2023 15:04:08.578 # Server initialized
103:M 08 May 2023 15:04:08.578 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
103:M 08 May 2023 15:04:08.580 * Loading RDB produced by version 6.2.6
103:M 08 May 2023 15:04:08.580 * RDB age 147 seconds
103:M 08 May 2023 15:04:08.580 * RDB memory usage when created 544.32 Mb
103:M 08 May 2023 15:04:10.774 * Done loading RDB, keys loaded: 47520, keys expired: 1006.
103:M 08 May 2023 15:04:10.774 * DB loaded from disk: 2.194 seconds
103:M 08 May 2023 15:04:10.774 * Ready to accept connections
I, [2023-05-08T15:04:18.562281 #1]  INFO -- :
I, [2023-05-08T15:04:18.562734 #1]  INFO -- : > thpoff echo "thpoff is installed!"
I, [2023-05-08T15:04:18.568341 #1]  INFO -- : thpoff is installed!

I, [2023-05-08T15:04:18.568575 #1]  INFO -- : > /usr/local/bin/ruby -e 'if ENV["DISCOURSE_SMTP_ADDRESS"] == "smtp.example.com"; puts "Aborting! Mail is not configured!"; exit 1; end'
I, [2023-05-08T15:04:18.618078 #1]  INFO -- :
I, [2023-05-08T15:04:18.618251 #1]  INFO -- : > /usr/local/bin/ruby -e 'if ENV["DISCOURSE_HOSTNAME"] == "discourse.example.com"; puts "Aborting! Domain is not configured!"; exit 1; end'
I, [2023-05-08T15:04:18.667604 #1]  INFO -- :
I, [2023-05-08T15:04:18.667782 #1]  INFO -- : > /usr/local/bin/ruby -e 'if (ENV["DISCOURSE_CDN_URL"] || "")[0..1] == "//"; puts "Aborting! CDN must have a protocol specified. Once fixed you should rebake your posts now to correct all posts."; exit 1; end'
I, [2023-05-08T15:04:18.715845 #1]  INFO -- :
I, [2023-05-08T15:04:18.716052 #1]  INFO -- : > rm -f /etc/cron.d/anacron
I, [2023-05-08T15:04:18.718365 #1]  INFO -- :
I, [2023-05-08T15:04:18.720298 #1]  INFO -- : File > /etc/cron.d/anacron  chmod:   chown:
I, [2023-05-08T15:04:18.722840 #1]  INFO -- : File > /etc/runit/1.d/copy-env  chmod: +x  chown:
I, [2023-05-08T15:04:18.725135 #1]  INFO -- : File > /etc/service/unicorn/run  chmod: +x  chown:
I, [2023-05-08T15:04:18.727462 #1]  INFO -- : File > /etc/service/nginx/run  chmod: +x  chown:
I, [2023-05-08T15:04:18.729626 #1]  INFO -- : File > /etc/runit/3.d/01-nginx  chmod: +x  chown:
I, [2023-05-08T15:04:18.732235 #1]  INFO -- : File > /etc/runit/3.d/02-unicorn  chmod: +x  chown:
I, [2023-05-08T15:04:18.732382 #1]  INFO -- : Replacing # postgres with sv start postgres || exit 1 in /etc/service/unicorn/run
I, [2023-05-08T15:04:18.732558 #1]  INFO -- : > exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf
I, [2023-05-08T15:04:18.734089 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git reset --hard
130:C 08 May 2023 15:04:18.737 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
130:C 08 May 2023 15:04:18.737 # Redis version=7.0.7, bits=64, commit=00000000, modified=0, pid=130, just started
130:C 08 May 2023 15:04:18.737 # Configuration loaded
130:M 08 May 2023 15:04:18.738 * monotonic clock: POSIX clock_gettime
130:M 08 May 2023 15:04:18.738 # Warning: Could not create server TCP listening socket *:6379: bind: Address already in use
130:M 08 May 2023 15:04:18.738 # Failed listening on port 6379 (TCP), aborting.
Updating files: 100% (33085/33085), done.
I, [2023-05-08T15:04:21.888178 #1]  INFO -- : HEAD is now at a8e28060d FIX: rename notify_about_flags_after to notify_about_reviewable_item_after (#21320)

I, [2023-05-08T15:04:21.888268 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git clean -f
I, [2023-05-08T15:04:21.984973 #1]  INFO -- :
I, [2023-05-08T15:04:21.985267 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse bash -c '
  set -o errexit
  if [ $(git rev-parse --is-shallow-repository) == "true" ]; then
      git remote set-branches --add origin main
      git remote set-branches origin 7647cf6a22aa7f9454123b24262ca04290b4d774
      git fetch --depth 1 origin 7647cf6a22aa7f9454123b24262ca04290b4d774
  else
      git fetch --tags --prune-tags --prune --force origin
  fi
'
From https://github.com/discourse/discourse
 - [deleted]             (none)     -> origin/a11y-select-kit-close-on-focus-out
 - [deleted]             (none)     -> origin/app-inject
 - [deleted]             (none)     -> origin/bump-eslint
 - [deleted]             (none)     -> origin/dependabot/npm_and_yarn/app/assets/javascripts/babel/core-7.21.5
 - [deleted]             (none)     -> origin/dependabot/npm_and_yarn/app/assets/javascripts/babel/standalone-7.21.7
 - [deleted]             (none)     -> origin/dependabot/npm_and_yarn/app/assets/javascripts/jsdom-21.1.2
 - [deleted]             (none)     -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/aws-s3-3.1.0
 - [deleted]             (none)     -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/aws-s3-multipart-3.2.0
 - [deleted]             (none)     -> origin/dev_fix_test
 - [deleted]             (none)     -> origin/feature_notification_admin_problems
 - [deleted]             (none)     -> origin/fix-username-validation
 - [deleted]             (none)     -> origin/loic-set-limits-for-reviewables
 - [deleted]             (none)     -> origin/old-styles
   a8e28060d..afbeeea09  main                   -> origin/main
 * [new branch]          chat/streaming         -> origin/chat/streaming
 + e5efacf69...b3cfecbd9 default-to-subcategory-on-read-only-categories -> origin/default-to-subcategory-on-read-only-categories  (forced update)
 * [new branch]          dependabot/bundler/omniauth-2.1.1 -> origin/dependabot/bundler/omniauth-2.1.1
 * [new branch]          dependabot/bundler/redis-5.0.6 -> origin/dependabot/bundler/redis-5.0.6
 * [new branch]          dependabot/bundler/sidekiq-7.1.0 -> origin/dependabot/bundler/sidekiq-7.1.0
 * [new branch]          dependabot/npm_and_yarn/app/assets/javascripts/ember-auto-import-chunks-json-generator-1.1.0 -> origin/dependabot/npm_and_yarn/app/assets/javascripts/ember-auto-import-chunks-json-generator-1.1.0
 * [new branch]          dependabot/npm_and_yarn/app/assets/javascripts/uppy/aws-s3-3.1.1 -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/aws-s3-3.1.1
 * [new branch]          dependabot/npm_and_yarn/app/assets/javascripts/uppy/aws-s3-multipart-3.3.0 -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/aws-s3-multipart-3.3.0
 + b8e4996b9...f16673f79 dependabot/npm_and_yarn/app/assets/javascripts/uppy/core-3.2.0 -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/core-3.2.0  (forced update)
 + 3ed9e6173...4ae8c98dd dependabot/npm_and_yarn/app/assets/javascripts/uppy/utils-5.3.0 -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/utils-5.3.0  (forced update)
 + 3bb7b79f0...8e7c9354f dependabot/npm_and_yarn/app/assets/javascripts/uppy/xhr-upload-3.2.0 -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/xhr-upload-3.2.0  (forced update)
   43bbc443c..0532a1c3d  dev-new-user-menu-default-enable -> origin/dev-new-user-menu-default-enable
 * [new branch]          dev/delete-old-personal-message-settings -> origin/dev/delete-old-personal-message-settings
 + 895956cb1...10ea7a4f0 ember-this-fallback    -> origin/ember-this-fallback  (forced update)
   10dda035f..4838c2950  feature/add-user-status-to-inline-mentions-on-chat -> origin/feature/add-user-status-to-inline-mentions-on-chat
 * [new branch]          feature/add-user-status-to-inline-mentions-on-chat-in-progress -> origin/feature/add-user-status-to-inline-mentions-on-chat-in-progress
 * [new branch]          feature/status-on-mentions-on-updated-chat-messages -> origin/feature/status-on-mentions-on-updated-chat-messages
 * [new branch]          feature_user_tips      -> origin/feature_user_tips
 * [new branch]          fix-select-kit-click-outside-regression -> origin/fix-select-kit-click-outside-regression
 * [new branch]          fix-tests              -> origin/fix-tests
 * [new branch]          fix_inactive_user_card -> origin/fix_inactive_user_card
 * [new branch]          fix_utf8               -> origin/fix_utf8
 + 4e8828636...c7ddb4f19 generic-import         -> origin/generic-import  (forced update)
 + 0eeb01892...f5f7e58fa headless-new           -> origin/headless-new  (forced update)
 * [new branch]          implicit-injections-v2 -> origin/implicit-injections-v2
 + ec1a0d83f...99f0e8336 modal-named-outlet     -> origin/modal-named-outlet  (forced update)
 * [new branch]          offline-app-testing    -> origin/offline-app-testing
 * [new branch]          sk-loading-slider      -> origin/sk-loading-slider
 * [new branch]          sk-tab-accessibility   -> origin/sk-tab-accessibility
   845000a6c..784006c71  stable                 -> origin/stable
 * [new branch]          tag-displayName        -> origin/tag-displayName
   a8e28060d..afbeeea09  tests-passed           -> origin/tests-passed
 * [new branch]          try-buildjet           -> origin/try-buildjet
 + 42b7246b2...bf6c559f0 ux_welcome_topic       -> origin/ux_welcome_topic  (forced update)
I, [2023-05-08T15:04:22.883268 #1]  INFO -- :
I, [2023-05-08T15:04:22.883432 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse bash -c '
  set -o errexit
  if [[ $(git symbolic-ref --short HEAD) == 7647cf6a22aa7f9454123b24262ca04290b4d774 ]] ; then
      git pull
  else
      git -c advice.detachedHead=false checkout 7647cf6a22aa7f9454123b24262ca04290b4d774
  fi
'
HEAD is now at 7647cf6a2 Version bump to v2.9.0.beta9
I, [2023-05-08T15:04:28.752035 #1]  INFO -- :
I, [2023-05-08T15:04:28.752208 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git config user.discourse-version 7647cf6a22aa7f9454123b24262ca04290b4d774
I, [2023-05-08T15:04:28.758701 #1]  INFO -- :
I, [2023-05-08T15:04:28.758774 #1]  INFO -- : > cd /var/www/discourse && mkdir -p tmp
I, [2023-05-08T15:04:28.760445 #1]  INFO -- :
I, [2023-05-08T15:04:28.760494 #1]  INFO -- : > cd /var/www/discourse && chown discourse:www-data tmp
I, [2023-05-08T15:04:28.762144 #1]  INFO -- :
I, [2023-05-08T15:04:28.762226 #1]  INFO -- : > cd /var/www/discourse && mkdir -p tmp/pids
I, [2023-05-08T15:04:28.764309 #1]  INFO -- :
I, [2023-05-08T15:04:28.764405 #1]  INFO -- : > cd /var/www/discourse && mkdir -p tmp/sockets
I, [2023-05-08T15:04:28.766300 #1]  INFO -- :
I, [2023-05-08T15:04:28.766378 #1]  INFO -- : > cd /var/www/discourse && touch tmp/.gitkeep
I, [2023-05-08T15:04:28.769841 #1]  INFO -- :
I, [2023-05-08T15:04:28.769903 #1]  INFO -- : > cd /var/www/discourse && mkdir -p                    /shared/log/rails
I, [2023-05-08T15:04:28.771724 #1]  INFO -- :
I, [2023-05-08T15:04:28.771796 #1]  INFO -- : > cd /var/www/discourse && bash -c "touch -a           /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log"
I, [2023-05-08T15:04:28.774160 #1]  INFO -- :
I, [2023-05-08T15:04:28.774259 #1]  INFO -- : > cd /var/www/discourse && bash -c "ln    -s           /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log /var/www/discourse/log"
I, [2023-05-08T15:04:28.776695 #1]  INFO -- :
I, [2023-05-08T15:04:28.776822 #1]  INFO -- : > cd /var/www/discourse && bash -c "mkdir -p           /shared/{uploads,backups}"
I, [2023-05-08T15:04:28.779275 #1]  INFO -- :
I, [2023-05-08T15:04:28.779373 #1]  INFO -- : > cd /var/www/discourse && bash -c "ln    -s           /shared/{uploads,backups} /var/www/discourse/public"
I, [2023-05-08T15:04:28.781533 #1]  INFO -- :
I, [2023-05-08T15:04:28.781656 #1]  INFO -- : > cd /var/www/discourse && bash -c "mkdir -p           /shared/tmp/{backups,restores}"
I, [2023-05-08T15:04:28.784890 #1]  INFO -- :
I, [2023-05-08T15:04:28.785073 #1]  INFO -- : > cd /var/www/discourse && bash -c "ln    -s           /shared/tmp/{backups,restores} /var/www/discourse/tmp"
I, [2023-05-08T15:04:28.788425 #1]  INFO -- :
I, [2023-05-08T15:04:28.788609 #1]  INFO -- : > cd /var/www/discourse && chown -R discourse:www-data /shared/log/rails /shared/uploads /shared/backups /shared/tmp
I, [2023-05-08T15:04:36.271931 #1]  INFO -- :
I, [2023-05-08T15:04:36.272148 #1]  INFO -- : > cd /var/www/discourse && [ ! -d public/plugins ] || find public/plugins/ -maxdepth 1 -xtype l -delete
I, [2023-05-08T15:04:36.281275 #1]  INFO -- :
I, [2023-05-08T15:04:36.281620 #1]  INFO -- : Replacing # redis with sv start redis || exit 1 in /etc/service/unicorn/run
I, [2023-05-08T15:04:36.282185 #1]  INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/docker_manager.git
Cloning into 'docker_manager'...
I, [2023-05-08T15:04:37.096659 #1]  INFO -- :
I, [2023-05-08T15:04:37.096927 #1]  INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/paviliondev/discourse-custom-wizard.git
Cloning into 'discourse-custom-wizard'...
I, [2023-05-08T15:04:38.116130 #1]  INFO -- :
I, [2023-05-08T15:04:38.116534 #1]  INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/discourse-follow.git
Cloning into 'discourse-follow'...
I, [2023-05-08T15:04:38.859416 #1]  INFO -- :
I, [2023-05-08T15:04:38.859732 #1]  INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/discourse-spoiler-alert.git
Cloning into 'discourse-spoiler-alert'...
I, [2023-05-08T15:04:39.432270 #1]  INFO -- :
I, [2023-05-08T15:04:39.432632 #1]  INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/ttlclub/discourse-rewards.git
Cloning into 'discourse-rewards'...
I, [2023-05-08T15:04:39.953657 #1]  INFO -- :
I, [2023-05-08T15:04:39.953880 #1]  INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/discourse-data-explorer.git
Cloning into 'discourse-data-explorer'...
I, [2023-05-08T15:04:40.587627 #1]  INFO -- :
I, [2023-05-08T15:04:40.587802 #1]  INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/paviliondev/discourse-subscription-client.git
Cloning into 'discourse-subscription-client'...
I, [2023-05-08T15:04:41.127896 #1]  INFO -- :
I, [2023-05-08T15:04:41.128239 #1]  INFO -- : > cp /var/www/discourse/config/nginx.sample.conf /etc/nginx/conf.d/discourse.conf
I, [2023-05-08T15:04:41.131797 #1]  INFO -- :
I, [2023-05-08T15:04:41.132014 #1]  INFO -- : > rm /etc/nginx/sites-enabled/default
I, [2023-05-08T15:04:41.134080 #1]  INFO -- :
I, [2023-05-08T15:04:41.134204 #1]  INFO -- : > mkdir -p /var/nginx/cache
I, [2023-05-08T15:04:41.136417 #1]  INFO -- :
I, [2023-05-08T15:04:41.137261 #1]  INFO -- : Replacing pid /run/nginx.pid; with daemon off; in /etc/nginx/nginx.conf
I, [2023-05-08T15:04:41.137571 #1]  INFO -- : Replacing (?m-ix:upstream[^\}]+\}) with upstream discourse { server 127.0.0.1:3000; } in /etc/nginx/conf.d/discourse.conf
I, [2023-05-08T15:04:41.137823 #1]  INFO -- : Replacing (?-mix:server_name.+$) with server_name _ ; in /etc/nginx/conf.d/discourse.conf
I, [2023-05-08T15:04:41.138068 #1]  INFO -- : Replacing (?-mix:client_max_body_size.+$) with client_max_body_size $upload_size ; in /etc/nginx/conf.d/discourse.conf
I, [2023-05-08T15:04:41.138332 #1]  INFO -- : Replacing (?-mix:worker_connections.+$) with worker_connections $nginx_worker_connections ; in /etc/nginx/nginx.conf
I, [2023-05-08T15:04:41.138566 #1]  INFO -- : > echo "done configuring web"
I, [2023-05-08T15:04:41.139638 #1]  INFO -- : done configuring web

I, [2023-05-08T15:04:41.139844 #1]  INFO -- : > cd /var/www/discourse && gem install bundler --conservative -v $(awk '/BUNDLED WITH/ { getline; gsub(/ /,""); print $0 }' Gemfile.lock)
I, [2023-05-08T15:04:41.583118 #1]  INFO -- : Successfully installed bundler-2.3.18
1 gem installed

I, [2023-05-08T15:04:41.583318 #1]  INFO -- : > cd /var/www/discourse && find /var/www/discourse ! -user discourse -exec chown discourse {} \+
I, [2023-05-08T15:05:12.625273 #1]  INFO -- :
I, [2023-05-08T15:05:12.625541 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'yarn install --frozen-lockfile && yarn cache clean'
warning "eslint-config-discourse > eslint-plugin-lodash@7.1.0" has unmet peer dependency "lodash@>=4".
warning " > @mixer/parallel-prettier@2.0.1" has unmet peer dependency "prettier@^2.0.0".
I, [2023-05-08T15:05:50.702874 #1]  INFO -- : yarn install v1.22.19
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
[4/5] Linking dependencies...
[5/5] Building fresh packages...
$ yarn --cwd app/assets/javascripts/discourse $(node -e 'if(JSON.parse(process.env.npm_config_argv).original.includes(`--frozen-lockfile`)){console.log(`--frozen-lockfile`)}')
yarn install v1.22.19
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
Done in 19.55s.
Done in 34.07s.
yarn cache v1.22.19
success Cleared cache.
Done in 3.58s.

I, [2023-05-08T15:05:50.705897 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle config --local deployment true'
I, [2023-05-08T15:05:50.955400 #1]  INFO -- :
I, [2023-05-08T15:05:50.955569 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle config --local without "development test"'
I, [2023-05-08T15:05:51.070186 #1]  INFO -- : You are replacing the current local value of without, which is currently "test:development"

I, [2023-05-08T15:05:51.070344 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle install --retry 3 --jobs 4'
You are trying to install in deployment mode after changing
your Gemfile. Run `bundle install` elsewhere and add the
updated Gemfile.lock to version control.

If this is a development machine, remove the /var/www/discourse/Gemfile freeze
by running `bundle config unset deployment`.

The dependencies in your gemfile changed

You have added to the Gemfile:
* net-smtp (~> 0.2.1)
* net-imap (~> 0.2.1)
* net-pop (~> 0.1.1)
* digest (= 3.0.0)
I, [2023-05-08T15:05:51.373703 #1]  INFO -- :
I, [2023-05-08T15:05:51.374113 #1]  INFO -- : Terminating async processes
I, [2023-05-08T15:05:51.374194 #1]  INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main pid: 42
I, [2023-05-08T15:05:51.374263 #1]  INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 103
103:signal-handler (1683558351) Received SIGTERM scheduling shutdown...
2023-05-08 15:05:51.374 UTC [42] LOG:  received fast shutdown request
2023-05-08 15:05:51.405 UTC [42] LOG:  aborting any active transactions
2023-05-08 15:05:51.407 UTC [42] LOG:  background worker "logical replication launcher" (PID 51) exited with exit code 1
2023-05-08 15:05:51.408 UTC [46] LOG:  shutting down
103:M 08 May 2023 15:05:51.445 # User requested shutdown...
103:M 08 May 2023 15:05:51.445 * Saving the final RDB snapshot before exiting.
2023-05-08 15:05:51.678 UTC [42] LOG:  database system is shut down
103:M 08 May 2023 15:05:53.628 * DB saved on disk
103:M 08 May 2023 15:05:53.628 # Redis is now ready to exit, bye bye...


FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle install --retry 3 --jobs 4' failed with return #<Process::Status: pid 455 exit 16>
Location of failure: /usr/local/lib/ruby/gems/3.2.0/gems/pups-1.1.1/lib/pups/exec_command.rb:117:in `spawn'
exec failed with the params {"cd"=>"$home", "hook"=>"bundle_exec", "cmd"=>["su discourse -c 'bundle config --local deployment true'", "su discourse -c 'bundle config --local without \"development test\"'", "su discourse -c 'bundle install --retry 3 --jobs 4'"]}
bootstrap failed with exit code 16
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.
e542e80828c60c1b8bd1fcaa6908f3ecb6ea33f1988178c5c7e893e2feeeb4b4

If you do not want to upgrade to 3.1.0beta4 then you’ll need to restore to your latest backup.

Our last backup was two days ago, so we want to get back to the current state as much as possible. Thank you for your help!

I think this might be your isse:

Is one or more of your plugins adding those gems?

Another issue you’re likely to have is that the plugins may be incompatible with the current version of Discourse. This doesn’t seem like a problem that’s likely to get solved via discussion here, though someone else might be more clever than I (or just notice something that I overlooked).

The first thing you should do is to solve whatever your problem with the discoure “front end” is. You already have a bunch of security issues and bugs that you’re dealing with.

1 Like

Thanks very much for your help! Unfortunately, I’ve been busy with real-life stuff lately, so I don’t have much time to troubleshoot forum issues. Is it possible to return to the state before the rebuild for the time being?