Bootstrap failure during most recent upgrade

nskerl · January 31, 2023, 2:43am

Hello,
After a failed upgrade I am receiving the below errors. It’s a bit hard to tell if the errors during the db upgrade (database "discourse" already exists) are expected or not, as they are logged as INFO severity.

Can someone help me find the actual errors here? Thanks!



WARNING: Docker version 17.05.0-ce deprecated, recommend upgrade to 17.06.2 or newer.
x86_64 arch detected.
WARNING: containers/app.yml file is world-readable. You can secure this file by running: chmod o-rwx containers/app.yml
Ensuring launcher is up to date
Fetching origin
Launcher is up-to-date
Stopping old container
+ /usr/bin/docker stop -t 60 app
app
2.0.20230116-0051: Pulling from discourse/base
Digest: sha256:9e75876adb52427902a6d76dab731c4888af811d7e5ad9ee6efcec4ce6db986f
Status: Image is up to date for discourse/base:2.0.20230116-0051
/usr/local/lib/ruby/gems/3.1.0/gems/pups-1.1.1/lib/pups.rb
/usr/local/bin/pups --stdin
I, [2023-01-31T02:12:29.507746 #1]  INFO -- : Reading from stdin
I, [2023-01-31T02:12:29.512327 #1]  INFO -- : > locale-gen $LANG && update-locale
I, [2023-01-31T02:12:29.587516 #1]  INFO -- : Generating locales (this might take a while)...
Generation complete.

I, [2023-01-31T02:12:29.587714 #1]  INFO -- : > mkdir -p /shared/postgres_run
I, [2023-01-31T02:12:29.604283 #1]  INFO -- :
I, [2023-01-31T02:12:29.605702 #1]  INFO -- : > chown postgres:postgres /shared/postgres_run
I, [2023-01-31T02:12:29.609468 #1]  INFO -- :
I, [2023-01-31T02:12:29.609613 #1]  INFO -- : > chmod 775 /shared/postgres_run
I, [2023-01-31T02:12:29.612628 #1]  INFO -- :
I, [2023-01-31T02:12:29.612758 #1]  INFO -- : > rm -fr /var/run/postgresql
I, [2023-01-31T02:12:29.615359 #1]  INFO -- :
I, [2023-01-31T02:12:29.615527 #1]  INFO -- : > ln -s /shared/postgres_run /var/run/postgresql
I, [2023-01-31T02:12:29.619262 #1]  INFO -- :
I, [2023-01-31T02:12:29.619376 #1]  INFO -- : > socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1
2023/01/31 02:12:29 socat[18] E connect(6, AF=1 "/shared/postgres_run/.s.PGSQL.5432", 36): No such file or directory
I, [2023-01-31T02:12:29.686531 #1]  INFO -- :
I, [2023-01-31T02:12:29.686750 #1]  INFO -- : > rm -fr /shared/postgres_run/.s*
I, [2023-01-31T02:12:29.689986 #1]  INFO -- :
I, [2023-01-31T02:12:29.690104 #1]  INFO -- : > rm -fr /shared/postgres_run/*.pid
I, [2023-01-31T02:12:29.692595 #1]  INFO -- :
I, [2023-01-31T02:12:29.692745 #1]  INFO -- : > mkdir -p /shared/postgres_run/13-main.pg_stat_tmp
I, [2023-01-31T02:12:29.694709 #1]  INFO -- :
I, [2023-01-31T02:12:29.694945 #1]  INFO -- : > chown postgres:postgres /shared/postgres_run/13-main.pg_stat_tmp
I, [2023-01-31T02:12:29.696655 #1]  INFO -- :
I, [2023-01-31T02:12:29.701107 #1]  INFO -- : File > /etc/service/postgres/run  chmod: +x  chown:
I, [2023-01-31T02:12:29.704891 #1]  INFO -- : File > /etc/service/postgres/log/run  chmod: +x  chown:
I, [2023-01-31T02:12:29.708854 #1]  INFO -- : File > /etc/runit/3.d/99-postgres  chmod: +x  chown:
I, [2023-01-31T02:12:29.713274 #1]  INFO -- : File > /root/upgrade_postgres  chmod: +x  chown:
I, [2023-01-31T02:12:29.713515 #1]  INFO -- : > chown -R root /var/lib/postgresql/13/main
I, [2023-01-31T02:12:30.105952 #1]  INFO -- :
I, [2023-01-31T02:12:30.106215 #1]  INFO -- : > [ ! -e /shared/postgres_data ] && install -d -m 0755 -o postgres -g postgres /shared/postgres_data && sudo -E -u postgres /usr/lib/postgresql/13/bin/initdb -D /shared/postgres_data || exit 0
I, [2023-01-31T02:12:30.108090 #1]  INFO -- :
I, [2023-01-31T02:12:30.108148 #1]  INFO -- : > chown -R postgres:postgres /shared/postgres_data
I, [2023-01-31T02:12:30.136161 #1]  INFO -- :
I, [2023-01-31T02:12:30.136283 #1]  INFO -- : > chown -R postgres:postgres /var/run/postgresql
I, [2023-01-31T02:12:30.138599 #1]  INFO -- :
I, [2023-01-31T02:12:30.138746 #1]  INFO -- : > /root/upgrade_postgres
I, [2023-01-31T02:12:30.143732 #1]  INFO -- :
I, [2023-01-31T02:12:30.143854 #1]  INFO -- : > rm /root/upgrade_postgres
I, [2023-01-31T02:12:30.145536 #1]  INFO -- :
I, [2023-01-31T02:12:30.147152 #1]  INFO -- : Replacing data_directory = '/var/lib/postgresql/13/main' with data_directory = '/shared/postgres_data' in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-31T02:12:30.147564 #1]  INFO -- : Replacing (?-mix:#?listen_addresses *=.*) with listen_addresses = '*' in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-31T02:12:30.148006 #1]  INFO -- : Replacing (?-mix:#?synchronous_commit *=.*) with synchronous_commit = $db_synchronous_commit in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-31T02:12:30.148448 #1]  INFO -- : Replacing (?-mix:#?shared_buffers *=.*) with shared_buffers = $db_shared_buffers in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-31T02:12:30.149047 #1]  INFO -- : Replacing (?-mix:#?work_mem *=.*) with work_mem = $db_work_mem in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-31T02:12:30.149522 #1]  INFO -- : Replacing (?-mix:#?default_text_search_config *=.*) with default_text_search_config = '$db_default_text_search_config' in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-31T02:12:30.149948 #1]  INFO -- : > install -d -m 0755 -o postgres -g postgres /shared/postgres_backup
I, [2023-01-31T02:12:30.158389 #1]  INFO -- :
I, [2023-01-31T02:12:30.158681 #1]  INFO -- : Replacing (?-mix:#?checkpoint_segments *=.*) with checkpoint_segments = $db_checkpoint_segments in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-31T02:12:30.159009 #1]  INFO -- : Replacing (?-mix:#?logging_collector *=.*) with logging_collector = $db_logging_collector in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-31T02:12:30.159522 #1]  INFO -- : Replacing (?-mix:#?log_min_duration_statement *=.*) with log_min_duration_statement = $db_log_min_duration_statement in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-31T02:12:30.160469 #1]  INFO -- : Replacing (?-mix:^#local +replication +postgres +peer$) with local replication postgres  peer in /etc/postgresql/13/main/pg_hba.conf
I, [2023-01-31T02:12:30.160736 #1]  INFO -- : Replacing (?-mix:^host.*all.*all.*127.*$) with host all all 0.0.0.0/0 md5 in /etc/postgresql/13/main/pg_hba.conf
I, [2023-01-31T02:12:30.161169 #1]  INFO -- : Replacing (?-mix:^host.*all.*all.*::1\/128.*$) with host all all ::/0 md5 in /etc/postgresql/13/main/pg_hba.conf
I, [2023-01-31T02:12:30.161557 #1]  INFO -- : > HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main
I, [2023-01-31T02:12:30.162911 #1]  INFO -- : > sleep 5
2023-01-31 02:12:30.417 UTC [41] LOG:  starting PostgreSQL 13.9 (Debian 13.9-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2023-01-31 02:12:30.418 UTC [41] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2023-01-31 02:12:30.418 UTC [41] LOG:  listening on IPv6 address "::", port 5432
2023-01-31 02:12:30.420 UTC [41] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-01-31 02:12:30.425 UTC [44] LOG:  database system was shut down at 2023-01-31 02:12:07 UTC
2023-01-31 02:12:30.447 UTC [41] LOG:  database system is ready to accept connections
I, [2023-01-31T02:12:35.167092 #1]  INFO -- :
I, [2023-01-31T02:12:35.167602 #1]  INFO -- : > su postgres -c 'createdb discourse' || true
2023-01-31 02:12:35.279 UTC [54] postgres@postgres ERROR:  database "discourse" already exists
2023-01-31 02:12:35.279 UTC [54] postgres@postgres STATEMENT:  CREATE DATABASE discourse;
createdb: error: database creation failed: ERROR:  database "discourse" already exists
I, [2023-01-31T02:12:35.281216 #1]  INFO -- :
I, [2023-01-31T02:12:35.281539 #1]  INFO -- : > su postgres -c 'psql discourse -c "create user discourse;"' || true
2023-01-31 02:12:35.352 UTC [58] postgres@discourse ERROR:  role "discourse" already exists
2023-01-31 02:12:35.352 UTC [58] postgres@discourse STATEMENT:  create user discourse;
ERROR:  role "discourse" already exists
I, [2023-01-31T02:12:35.354851 #1]  INFO -- :
I, [2023-01-31T02:12:35.355126 #1]  INFO -- : > su postgres -c 'psql discourse -c "grant all privileges on database discourse to discourse;"' || true
I, [2023-01-31T02:12:35.402281 #1]  INFO -- : GRANT

I, [2023-01-31T02:12:35.402506 #1]  INFO -- : > su postgres -c 'psql discourse -c "alter schema public owner to discourse;"'
I, [2023-01-31T02:12:35.458595 #1]  INFO -- : ALTER SCHEMA

I, [2023-01-31T02:12:35.458802 #1]  INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists hstore;"'
NOTICE:  extension "hstore" already exists, skipping
I, [2023-01-31T02:12:35.504923 #1]  INFO -- : CREATE EXTENSION

I, [2023-01-31T02:12:35.505151 #1]  INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists pg_trgm;"'
NOTICE:  extension "pg_trgm" already exists, skipping
I, [2023-01-31T02:12:35.550581 #1]  INFO -- : CREATE EXTENSION

I, [2023-01-31T02:12:35.550770 #1]  INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists hstore;"'
NOTICE:  extension "hstore" already exists, skipping
I, [2023-01-31T02:12:35.596669 #1]  INFO -- : CREATE EXTENSION

I, [2023-01-31T02:12:35.596907 #1]  INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists pg_trgm;"'
NOTICE:  extension "pg_trgm" already exists, skipping
I, [2023-01-31T02:12:35.642037 #1]  INFO -- : CREATE EXTENSION

I, [2023-01-31T02:12:35.642257 #1]  INFO -- : > sudo -u postgres psql discourse
I, [2023-01-31T02:12:35.645526 #1]  INFO -- : update pg_database set encoding = pg_char_to_encoding('UTF8') where datname = 'discourse' AND encoding = pg_char_to_encoding('SQL_ASCII');

I, [2023-01-31T02:12:35.725665 #1]  INFO -- : File > /var/lib/postgresql/take-database-backup  chmod: +x  chown: postgres:postgres
I, [2023-01-31T02:12:35.731072 #1]  INFO -- : File > /var/spool/cron/crontabs/postgres  chmod:   chown:
I, [2023-01-31T02:12:35.731177 #1]  INFO -- : > echo postgres installed!
I, [2023-01-31T02:12:35.732848 #1]  INFO -- : postgres installed!

I, [2023-01-31T02:12:35.736463 #1]  INFO -- : File > /etc/service/redis/run  chmod: +x  chown:
I, [2023-01-31T02:12:35.740012 #1]  INFO -- : File > /etc/service/redis/log/run  chmod: +x  chown:
I, [2023-01-31T02:12:35.743464 #1]  INFO -- : File > /etc/runit/3.d/10-redis  chmod: +x  chown:
I, [2023-01-31T02:12:35.747236 #1]  INFO -- : Replacing daemonize yes with  in /etc/redis/redis.conf
I, [2023-01-31T02:12:35.747785 #1]  INFO -- : Replacing (?-mix:^pidfile.*$) with  in /etc/redis/redis.conf
I, [2023-01-31T02:12:35.748548 #1]  INFO -- : > install -d -m 0755 -o redis -g redis /shared/redis_data
I, [2023-01-31T02:12:35.750643 #1]  INFO -- :
I, [2023-01-31T02:12:35.750946 #1]  INFO -- : Replacing (?-mix:^logfile.*$) with logfile "" in /etc/redis/redis.conf
I, [2023-01-31T02:12:35.751409 #1]  INFO -- : Replacing (?-mix:^bind .*$) with  in /etc/redis/redis.conf
I, [2023-01-31T02:12:35.753357 #1]  INFO -- : Replacing (?-mix:^dir .*$) with dir /shared/redis_data in /etc/redis/redis.conf
I, [2023-01-31T02:12:35.754130 #1]  INFO -- : Replacing (?-mix:^protected-mode yes) with protected-mode no in /etc/redis/redis.conf
I, [2023-01-31T02:12:35.755226 #1]  INFO -- : Replacing # io-threads 4 with io-threads $redis_io_threads in /etc/redis/redis.conf
I, [2023-01-31T02:12:35.756092 #1]  INFO -- : > echo redis installed
I, [2023-01-31T02:12:35.757841 #1]  INFO -- : redis installed

I, [2023-01-31T02:12:35.758062 #1]  INFO -- : > cat /etc/redis/redis.conf | grep logfile
I, [2023-01-31T02:12:35.768360 #1]  INFO -- : logfile ""

I, [2023-01-31T02:12:35.768588 #1]  INFO -- : > exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf
I, [2023-01-31T02:12:35.770179 #1]  INFO -- : > sleep 10
102:C 31 Jan 2023 02:12:35.800 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
102:C 31 Jan 2023 02:12:35.800 # Redis version=7.0.7, bits=64, commit=00000000, modified=0, pid=102, just started
102:C 31 Jan 2023 02:12:35.800 # Configuration loaded
102:M 31 Jan 2023 02:12:35.801 * monotonic clock: POSIX clock_gettime
102:M 31 Jan 2023 02:12:35.804 * Running mode=standalone, port=6379.
102:M 31 Jan 2023 02:12:35.804 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
102:M 31 Jan 2023 02:12:35.804 # Server initialized
102:M 31 Jan 2023 02:12:35.804 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
102:M 31 Jan 2023 02:12:35.805 * Loading RDB produced by version 7.0.7
102:M 31 Jan 2023 02:12:35.805 * RDB age 1476 seconds
102:M 31 Jan 2023 02:12:35.805 * RDB memory usage when created 21.94 Mb
102:M 31 Jan 2023 02:12:35.865 * Done loading RDB, keys loaded: 11301, keys expired: 5.
102:M 31 Jan 2023 02:12:35.865 * DB loaded from disk: 0.061 seconds
102:M 31 Jan 2023 02:12:35.865 * Ready to accept connections
I, [2023-01-31T02:12:45.772254 #1]  INFO -- :
I, [2023-01-31T02:12:45.772587 #1]  INFO -- : > thpoff echo "thpoff is installed!"
I, [2023-01-31T02:12:45.780195 #1]  INFO -- : thpoff is installed!

I, [2023-01-31T02:12:45.780804 #1]  INFO -- : > /usr/local/bin/ruby -e 'if ENV["DISCOURSE_SMTP_ADDRESS"] == "smtp.example.com"; puts "Aborting! Mail is not configured!"; exit 1; end'
I, [2023-01-31T02:12:45.861042 #1]  INFO -- :
I, [2023-01-31T02:12:45.861480 #1]  INFO -- : > /usr/local/bin/ruby -e 'if ENV["DISCOURSE_HOSTNAME"] == "discourse.example.com"; puts "Aborting! Domain is not configured!"; exit 1; end'
I, [2023-01-31T02:12:45.948241 #1]  INFO -- :
I, [2023-01-31T02:12:45.948568 #1]  INFO -- : > /usr/local/bin/ruby -e 'if (ENV["DISCOURSE_CDN_URL"] || "")[0..1] == "//"; puts "Aborting! CDN must have a protocol specified. Once fixed you should rebake your posts now to correct all posts."; exit 1; end'
I, [2023-01-31T02:12:46.025938 #1]  INFO -- :
I, [2023-01-31T02:12:46.026222 #1]  INFO -- : > rm -f /etc/cron.d/anacron
I, [2023-01-31T02:12:46.029873 #1]  INFO -- :
I, [2023-01-31T02:12:46.033179 #1]  INFO -- : File > /etc/cron.d/anacron  chmod:   chown:
I, [2023-01-31T02:12:46.038163 #1]  INFO -- : File > /etc/runit/1.d/copy-env  chmod: +x  chown:
I, [2023-01-31T02:12:46.042404 #1]  INFO -- : File > /etc/service/unicorn/run  chmod: +x  chown:
I, [2023-01-31T02:12:46.046360 #1]  INFO -- : File > /etc/service/nginx/run  chmod: +x  chown:
I, [2023-01-31T02:12:46.050691 #1]  INFO -- : File > /etc/runit/3.d/01-nginx  chmod: +x  chown:
I, [2023-01-31T02:12:46.055487 #1]  INFO -- : File > /etc/runit/3.d/02-unicorn  chmod: +x  chown:
I, [2023-01-31T02:12:46.055640 #1]  INFO -- : Replacing # postgres with sv start postgres || exit 1 in /etc/service/unicorn/run
I, [2023-01-31T02:12:46.055980 #1]  INFO -- : > exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf
I, [2023-01-31T02:12:46.058230 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git reset --hard
130:C 31 Jan 2023 02:12:46.064 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
130:C 31 Jan 2023 02:12:46.064 # Redis version=7.0.7, bits=64, commit=00000000, modified=0, pid=130, just started
130:C 31 Jan 2023 02:12:46.064 # Configuration loaded
130:M 31 Jan 2023 02:12:46.065 * monotonic clock: POSIX clock_gettime
130:M 31 Jan 2023 02:12:46.065 # Warning: Could not create server TCP listening socket *:6379: bind: Address already in use
130:M 31 Jan 2023 02:12:46.065 # Failed listening on port 6379 (TCP), aborting.
Updating files: 100% (32677/32677), done.
I, [2023-01-31T02:12:49.255454 #1]  INFO -- : HEAD is now at 2eb0a300 FIX: Use hashtags in channel archive PMs if available (#19859)

I, [2023-01-31T02:12:49.255881 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git clean -f
I, [2023-01-31T02:12:49.379823 #1]  INFO -- :
I, [2023-01-31T02:12:49.380017 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git remote set-branches --add origin main
I, [2023-01-31T02:12:49.389501 #1]  INFO -- :
I, [2023-01-31T02:12:49.389646 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git remote set-branches origin tests-passed
I, [2023-01-31T02:12:49.399086 #1]  INFO -- :
I, [2023-01-31T02:12:49.399174 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git fetch --depth 1 origin tests-passed
From https://github.com/discourse/discourse
 * branch              tests-passed -> FETCH_HEAD
 * [new branch]        tests-passed -> origin/tests-passed
I, [2023-01-31T02:12:52.277887 #1]  INFO -- :
I, [2023-01-31T02:12:52.278062 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git checkout tests-passed
Switched to a new branch 'tests-passed'
I, [2023-01-31T02:12:53.014674 #1]  INFO -- : Branch 'tests-passed' set up to track remote branch 'tests-passed' from 'origin'.

I, [2023-01-31T02:12:53.014831 #1]  INFO -- : > cd /var/www/discourse && mkdir -p tmp
I, [2023-01-31T02:12:53.018549 #1]  INFO -- :
I, [2023-01-31T02:12:53.018665 #1]  INFO -- : > cd /var/www/discourse && chown discourse:www-data tmp
I, [2023-01-31T02:12:53.021355 #1]  INFO -- :
I, [2023-01-31T02:12:53.021476 #1]  INFO -- : > cd /var/www/discourse && mkdir -p tmp/pids
I, [2023-01-31T02:12:53.024361 #1]  INFO -- :
I, [2023-01-31T02:12:53.024471 #1]  INFO -- : > cd /var/www/discourse && mkdir -p tmp/sockets
I, [2023-01-31T02:12:53.027261 #1]  INFO -- :
I, [2023-01-31T02:12:53.027345 #1]  INFO -- : > cd /var/www/discourse && touch tmp/.gitkeep
I, [2023-01-31T02:12:53.033167 #1]  INFO -- :
I, [2023-01-31T02:12:53.033279 #1]  INFO -- : > cd /var/www/discourse && mkdir -p                    /shared/log/rails
I, [2023-01-31T02:12:53.036071 #1]  INFO -- :
I, [2023-01-31T02:12:53.036165 #1]  INFO -- : > cd /var/www/discourse && bash -c "touch -a           /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log"
I, [2023-01-31T02:12:53.040309 #1]  INFO -- :
I, [2023-01-31T02:12:53.040425 #1]  INFO -- : > cd /var/www/discourse && bash -c "ln    -s           /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log /var/www/discourse/log"
I, [2023-01-31T02:12:53.044220 #1]  INFO -- :
I, [2023-01-31T02:12:53.044322 #1]  INFO -- : > cd /var/www/discourse && bash -c "mkdir -p           /shared/{uploads,backups}"
I, [2023-01-31T02:12:53.048231 #1]  INFO -- :
I, [2023-01-31T02:12:53.048331 #1]  INFO -- : > cd /var/www/discourse && bash -c "ln    -s           /shared/{uploads,backups} /var/www/discourse/public"
I, [2023-01-31T02:12:53.052267 #1]  INFO -- :
I, [2023-01-31T02:12:53.052372 #1]  INFO -- : > cd /var/www/discourse && bash -c "mkdir -p           /shared/tmp/{backups,restores}"
I, [2023-01-31T02:12:53.056297 #1]  INFO -- :
I, [2023-01-31T02:12:53.056404 #1]  INFO -- : > cd /var/www/discourse && bash -c "ln    -s           /shared/tmp/{backups,restores} /var/www/discourse/tmp"
I, [2023-01-31T02:12:53.059875 #1]  INFO -- :
I, [2023-01-31T02:12:53.059989 #1]  INFO -- : > cd /var/www/discourse && chown -R discourse:www-data /shared/log/rails /shared/uploads /shared/backups /shared/tmp
I, [2023-01-31T02:12:53.380014 #1]  INFO -- :
I, [2023-01-31T02:12:53.380324 #1]  INFO -- : > cd /var/www/discourse && [ ! -d public/plugins ] || find public/plugins/ -maxdepth 1 -xtype l -delete
I, [2023-01-31T02:12:53.391099 #1]  INFO -- :
I, [2023-01-31T02:12:53.391261 #1]  INFO -- : Replacing # redis with sv start redis || exit 1 in /etc/service/unicorn/run
I, [2023-01-31T02:12:53.391586 #1]  INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/docker_manager.git
Cloning into 'docker_manager'...
I, [2023-01-31T02:12:55.309191 #1]  INFO -- :
I, [2023-01-31T02:12:55.309701 #1]  INFO -- : > cp /var/www/discourse/config/nginx.sample.conf /etc/nginx/conf.d/discourse.conf
I, [2023-01-31T02:12:55.320678 #1]  INFO -- :
I, [2023-01-31T02:12:55.321055 #1]  INFO -- : > rm /etc/nginx/sites-enabled/default
I, [2023-01-31T02:12:55.323561 #1]  INFO -- :
I, [2023-01-31T02:12:55.323666 #1]  INFO -- : > mkdir -p /var/nginx/cache
I, [2023-01-31T02:12:55.325959 #1]  INFO -- :
I, [2023-01-31T02:12:55.326652 #1]  INFO -- : Replacing pid /run/nginx.pid; with daemon off; in /etc/nginx/nginx.conf
I, [2023-01-31T02:12:55.326977 #1]  INFO -- : Replacing (?m-ix:upstream[^\}]+\}) with upstream discourse { server 127.0.0.1:3000; } in /etc/nginx/conf.d/discourse.conf
I, [2023-01-31T02:12:55.327230 #1]  INFO -- : Replacing (?-mix:server_name.+$) with server_name _ ; in /etc/nginx/conf.d/discourse.conf
I, [2023-01-31T02:12:55.327666 #1]  INFO -- : Replacing (?-mix:client_max_body_size.+$) with client_max_body_size $upload_size ; in /etc/nginx/conf.d/discourse.conf
I, [2023-01-31T02:12:55.328108 #1]  INFO -- : Replacing (?-mix:worker_connections.+$) with worker_connections $nginx_worker_connections ; in /etc/nginx/nginx.conf
I, [2023-01-31T02:12:55.328304 #1]  INFO -- : > echo "done configuring web"
I, [2023-01-31T02:12:55.330097 #1]  INFO -- : done configuring web

I, [2023-01-31T02:12:55.330393 #1]  INFO -- : > cd /var/www/discourse && gem install bundler --conservative -v $(awk '/BUNDLED WITH/ { getline; gsub(/ /,""); print $0 }' Gemfile.lock)
ERROR:  Loading command: install (RuntimeError)
        failed to get urandom
        /usr/local/lib/ruby/3.1.0/securerandom.rb:75:in `urandom'
        /usr/local/lib/ruby/3.1.0/securerandom.rb:75:in `singleton class'
        /usr/local/lib/ruby/3.1.0/securerandom.rb:42:in `<module:SecureRandom>'
        /usr/local/lib/ruby/3.1.0/securerandom.rb:41:in `<top (required)>'
        <internal:/usr/local/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        <internal:/usr/local/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        /usr/local/lib/ruby/3.1.0/resolv.rb:8:in `<top (required)>'
        <internal:/usr/local/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        <internal:/usr/local/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        /usr/local/lib/ruby/3.1.0/net/http.rb:25:in `<top (required)>'
        <internal:/usr/local/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        <internal:/usr/local/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/request.rb:2:in `<top (required)>'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/remote_fetcher.rb:3:in `require_relative'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/remote_fetcher.rb:3:in `<top (required)>'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/spec_fetcher.rb:2:in `require_relative'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/spec_fetcher.rb:2:in `<top (required)>'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/dependency_installer.rb:6:in `require_relative'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/dependency_installer.rb:6:in `<top (required)>'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/commands/install_command.rb:4:in `require_relative'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/commands/install_command.rb:4:in `<top (required)>'
        <internal:/usr/local/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        <internal:/usr/local/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/command_manager.rb:234:in `load_and_instantiate'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/command_manager.rb:135:in `[]'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/command_manager.rb:207:in `find_command'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/command_manager.rb:249:in `invoke_command'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/command_manager.rb:191:in `process_args'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/command_manager.rb:149:in `run'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/gem_runner.rb:51:in `run'
        /usr/local/bin/gem:10:in `<main>'
ERROR:  While executing gem ... (NoMethodError)
    undefined method `deprecated?' for nil:NilClass

    cmd.deprecation_warning if cmd.deprecated?
                                  ^^^^^^^^^^^^
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/command_manager.rb:250:in `invoke_command'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/command_manager.rb:191:in `process_args'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/command_manager.rb:149:in `run'
        /usr/local/lib/ruby/site_ruby/3.1.0/rubygems/gem_runner.rb:51:in `run'
        /usr/local/bin/gem:10:in `<main>'
I, [2023-01-31T02:12:55.589105 #1]  INFO -- :
I, [2023-01-31T02:12:55.589425 #1]  INFO -- : Terminating async processes
I, [2023-01-31T02:12:55.589488 #1]  INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main pid: 41
I, [2023-01-31T02:12:55.589533 #1]  INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 102
2023-01-31 02:12:55.589 UTC [41] LOG:  received fast shutdown request
102:signal-handler (1675131175) Received SIGTERM scheduling shutdown...
2023-01-31 02:12:55.592 UTC [41] LOG:  aborting any active transactions
2023-01-31 02:12:55.594 UTC [41] LOG:  background worker "logical replication launcher" (PID 50) exited with exit code 1
2023-01-31 02:12:55.594 UTC [45] LOG:  shutting down
2023-01-31 02:12:55.611 UTC [41] LOG:  database system is shut down
102:M 31 Jan 2023 02:12:55.618 # User requested shutdown...
102:M 31 Jan 2023 02:12:55.619 * Saving the final RDB snapshot before exiting.
102:M 31 Jan 2023 02:12:55.702 * DB saved on disk
102:M 31 Jan 2023 02:12:55.702 # Redis is now ready to exit, bye bye...


FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && gem install bundler --conservative -v $(awk '/BUNDLED WITH/ { getline; gsub(/ /,""); print $0 }' Gemfile.lock) failed with return #<Process::Status: pid 243 exit 1>
Location of failure: /usr/local/lib/ruby/gems/3.1.0/gems/pups-1.1.1/lib/pups/exec_command.rb:117:in `spawn'
exec failed with the params {"cd"=>"$home", "hook"=>"web", "cmd"=>["gem install bundler --conservative -v $(awk '/BUNDLED WITH/ { getline; gsub(/ /,\"\"); print $0 }' Gemfile.lock)", "find $home ! -user discourse -exec chown discourse {} \\+"]}
bootstrap failed with exit code 1
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.

Falco · January 31, 2023, 3:13am

This error usually happens when you are using an outdated Kernel, which is usually the case when running an end-of-life distro version. Can you share your kernel and OS version ?

nskerl · January 31, 2023, 3:29am

Yes, that might be my issue.

Linux 3.16.0-77-generic x86_64
Ubuntu 14.04.5 LTS

nskerl · January 31, 2023, 4:11am

In this case, since our nightly backups are up to v3.0.0.beta15 (which worked ok up until now), should we rebuild with latest and attempt to restore that? Or, should we rebuild with that specific version and restore then upgrade to latest.

If you know of any documentation on installing a fresh version from a specific tag (like 3.0.0.beta15) I would appreciate any guidance.

thanks.

Ed_S · January 31, 2023, 5:01am

Just to be clear, in this case, this is the error. Thanks for posting the log.

nskerl · January 31, 2023, 5:48am

Thank you for confirming that.

From reading other posts on this topic, it does appear that it is supported path to restore a (slightly) older backup to a new version. We will pursue this tomorrow.

nskerl · February 3, 2023, 12:04am

Just wanted to report back that a clean install on Ubuntu 22.04 got us back up, and we had no issues restoring the v3.0.0.beta15 backup on Discourse latest.

Thanks all!

system · March 5, 2023, 12:04am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Failed to Bootstrap, due to discourse-alt-logo theme component Installation	8	927	July 30, 2021
Upgrading 2.7 to 3.1 failing: "hotlinked_media_status" already exists Installation	30	1064	October 1, 2023
Unable to boostrap Discourse since last upgrade Installation	5	607	January 22, 2023
Updating failed, unable to rebuild Installation	4	1063	July 9, 2021
Build failed for specific commit hash Installation	9	618	April 12, 2023

Bootstrap failure during most recent upgrade

Related topics