### Author:  [@littleviolette](https://meta.discourse.org/u/littleviolette)
#### Post date: [8 januari 2023 om 10:15 UTC](https://meta.discourse.org/t/help-discourse-update-broke-app-fails-to-rebuild/251043/1 "2023-01-08T10:15:04Z")
When I tried to update to the latest version today, I encountered the follow error:
```plaintext
fatal: unable to access 'https://github.com/discourse/discourse.git/': Could not resolve host: github.com
```
I am running the app on a DigitalOcean Droplet Ubuntu 18.04.6 LTS (GNU/Linux 3.13.0-52-generic x86\_64)
I am not familiar with docker or server-side programming. I would really appreciate your help! đ
Here is the error code in its entirety:
````plaintext
root@forum:/var/discourse# ./launcher rebuild app
x86_64 arch detected.
WARNING: containers/app.yml file is world-readable. You can secure this file by running: chmod o-rwx containers/app.yml
Ensuring launcher is up to date
Fetching origin
Launcher is up-to-date
Stopping old container
+ /usr/bin/docker stop -t 60 app
app
2.0.20221221-0050: Pulling from discourse/base
Digest: sha256:7ac6489e34a0f40f32457bffaa331940a98447b46451a6a1279a79868ee5420b
Status: Image is up to date for discourse/base:2.0.20221221-0050
docker.io/discourse/base:2.0.20221221-0050
/usr/local/lib/ruby/gems/3.1.0/gems/pups-1.1.1/lib/pups.rb
/usr/local/bin/pups --stdin
I, [2023-01-08T10:02:23.027325 #1] INFO -- : Reading from stdin
I, [2023-01-08T10:02:23.037278 #1] INFO -- : > locale-gen $LANG && update-locale
I, [2023-01-08T10:02:23.114042 #1] INFO -- : Generating locales (this might take a while)...
Generation complete.
I, [2023-01-08T10:02:23.114335 #1] INFO -- : > mkdir -p /shared/postgres_run
I, [2023-01-08T10:02:23.121579 #1] INFO -- :
I, [2023-01-08T10:02:23.122417 #1] INFO -- : > chown postgres:postgres /shared/postgres_run
I, [2023-01-08T10:02:23.126437 #1] INFO -- :
I, [2023-01-08T10:02:23.126670 #1] INFO -- : > chmod 775 /shared/postgres_run
I, [2023-01-08T10:02:23.129755 #1] INFO -- :
I, [2023-01-08T10:02:23.129992 #1] INFO -- : > rm -fr /var/run/postgresql
I, [2023-01-08T10:02:23.133413 #1] INFO -- :
I, [2023-01-08T10:02:23.133646 #1] INFO -- : > ln -s /shared/postgres_run /var/run/postgresql
I, [2023-01-08T10:02:23.137014 #1] INFO -- :
I, [2023-01-08T10:02:23.137231 #1] INFO -- : > socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1
2023/01/08 10:02:23 socat[18] E connect(6, AF=1 "/shared/postgres_run/.s.PGSQL.5432", 36): No such file or directory
I, [2023-01-08T10:02:23.163463 #1] INFO -- :
I, [2023-01-08T10:02:23.163683 #1] INFO -- : > rm -fr /shared/postgres_run/.s*
I, [2023-01-08T10:02:23.167367 #1] INFO -- :
I, [2023-01-08T10:02:23.167548 #1] INFO -- : > rm -fr /shared/postgres_run/*.pid
I, [2023-01-08T10:02:23.170830 #1] INFO -- :
I, [2023-01-08T10:02:23.171037 #1] INFO -- : > mkdir -p /shared/postgres_run/13-main.pg_stat_tmp
I, [2023-01-08T10:02:23.174338 #1] INFO -- :
I, [2023-01-08T10:02:23.174539 #1] INFO -- : > chown postgres:postgres /shared/postgres_run/13-main.pg_stat_tmp
I, [2023-01-08T10:02:23.177456 #1] INFO -- :
I, [2023-01-08T10:02:23.183562 #1] INFO -- : File > /etc/service/postgres/run chmod: +x chown:
I, [2023-01-08T10:02:23.189225 #1] INFO -- : File > /etc/service/postgres/log/run chmod: +x chown:
I, [2023-01-08T10:02:23.195374 #1] INFO -- : File > /etc/runit/3.d/99-postgres chmod: +x chown:
I, [2023-01-08T10:02:23.201124 #1] INFO -- : File > /root/upgrade_postgres chmod: +x chown:
I, [2023-01-08T10:02:23.201450 #1] INFO -- : > chown -R root /var/lib/postgresql/13/main
I, [2023-01-08T10:02:23.616900 #1] INFO -- :
I, [2023-01-08T10:02:23.617006 #1] INFO -- : > [! -e /shared/postgres_data] && install -d -m 0755 -o postgres -g postgres /shared/postgres_data && sudo -E -u postgres /usr/lib/postgresql/13/bin/initdb -D /shared/postgres_data || exit 0
I, [2023-01-08T10:02:23.619878 #1] INFO -- :
I, [2023-01-08T10:02:23.619967 #1] INFO -- : > chown -R postgres:postgres /shared/postgres_data
I, [2023-01-08T10:02:23.673160 #1] INFO -- :
I, [2023-01-08T10:02:23.673318 #1] INFO -- : > chown -R postgres:postgres /var/run/postgresql
I, [2023-01-08T10:02:23.677342 #1] INFO -- :
I, [2023-01-08T10:02:23.677523 #1] INFO -- : > /root/upgrade_postgres
I, [2023-01-08T10:02:23.683764 #1] INFO -- :
I, [2023-01-08T10:02:23.683966 #1] INFO -- : > rm /root/upgrade_postgres
I, [2023-01-08T10:02:23.686507 #1] INFO -- :
I, [2023-01-08T10:02:23.687630 #1] INFO -- : Replacing data_directory = '/var/lib/postgresql/13/main' with data_directory = '/shared/postgres_data' in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-08T10:02:23.688285 #1] INFO -- : Replacing (?-mix:#?listen_addresses *=.*) with listen_addresses = '*' in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-08T10:02:23.688707 #1] INFO -- : Replacing (?-mix:#?synchronous_commit *=.*) with synchronous_commit = $db_synchronous_commit in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-08T10:02:23.689138 #1] INFO -- : Replacing (?-mix:#?shared_buffers *=.*) with shared_buffers = $db_shared_buffers in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-08T10:02:23.689521 #1] INFO -- : Replacing (?-mix:#?work_mem *=.*) with work_mem = $db_work_mem in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-08T10:02:23.689954 #1] INFO -- : Replacing (?-mix:#?default_text_search_config *=.*) with default_text_search_config = '$db_default_text_search_config' in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-08T10:02:23.690327 #1] INFO -- : > install -d -m 0755 -o postgres -g postgres /shared/postgres_backup
I, [2023-01-08T10:02:23.696176 #1] INFO -- :
I, [2023-01-08T10:02:23.696554 #1] INFO -- : Replacing (?-mix:#?checkpoint_segments *=.*) with checkpoint_segments = $db_checkpoint_segments in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-08T10:02:23.696973 #1] INFO -- : Replacing (?-mix:#?logging_collector *=.*) with logging_collector = $db_logging_collector in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-08T10:02:23.697400 #1] INFO -- : Replacing (?-mix:#?log_min_duration_statement *=.*) with log_min_duration_statement = $db_log_min_duration_statement in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-08T10:02:23.698030 #1] INFO -- : Replacing (?-mix:^#local +replication +postgres +peer$) with local replication postgres peer in /etc/postgresql/13/main/pg_hba.conf
I, [2023-01-08T10:02:23.698390 #1] INFO -- : Replacing (?-mix:^host.*all.*all.*127.*$) with host all all 0.0.0.0/0 md5 in /etc/postgresql/13/main/pg_hba.conf
I, [2023-01-08T10:02:23.698734 #1] INFO -- : Replacing (?-mix:^host.*all.*all.*::1\/128.*$) with host all all ::/0 md5 in /etc/postgresql/13/main/pg_hba.conf
I, [2023-01-08T10:02:23.699085 #1] INFO -- : > HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main
I, [2023-01-08T10:02:23.701001 #1] INFO -- : > sleep 5
2023-01-08 10:02:23.892 UTC [41] LOG: starting PostgreSQL 13.9 (Debian 13.9-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2023-01-08 10:02:23.893 UTC [41] LOG: listening on IPv4 address "0.0.0.0", port 5432
2023-01-08 10:02:23.893 UTC [41] LOG: listening on IPv6 address "::", port 5432
2023-01-08 10:02:23.895 UTC [41] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-01-08 10:02:23.900 UTC [44] LOG: database system was shut down at 2023-01-08 10:02:12 UTC
2023-01-08 10:02:23.914 UTC [41] LOG: database system is ready to accept connections
I, [2023-01-08T10:02:28.704562 #1] INFO -- :
I, [2023-01-08T10:02:28.705165 #1] INFO -- : > su postgres -c 'createdb discourse' || true
2023-01-08 10:02:28.812 UTC [54] postgres@postgres ERROR: database "discourse" already exists
2023-01-08 10:02:28.812 UTC [54] postgres@postgres STATEMENT: CREATE DATABASE discourse;
createdb: error: database creation failed: ERROR: database "discourse" already exists
I, [2023-01-08T10:02:28.815155 #1] INFO -- :
I, [2023-01-08T10:02:28.815612 #1] INFO -- : > su postgres -c 'psql discourse -c "create user discourse;"' || true
2023-01-08 10:02:28.891 UTC [58] postgres@discourse ERROR: role "discourse" already exists
2023-01-08 10:02:28.891 UTC [58] postgres@discourse STATEMENT: create user discourse;
ERROR: role "discourse" already exists
I, [2023-01-08T10:02:28.894445 #1] INFO -- :
I, [2023-01-08T10:02:28.894925 #1] INFO -- : > su postgres -c 'psql discourse -c "grant all privileges on database discourse to discourse;"' || true
I, [2023-01-08T10:02:28.966522 #1] INFO -- : GRANT
I, [2023-01-08T10:02:28.966978 #1] INFO -- : > su postgres -c 'psql discourse -c "alter schema public owner to discourse;"'
I, [2023-01-08T10:02:29.031133 #1] INFO -- : ALTER SCHEMA
I, [2023-01-08T10:02:29.031397 #1] INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists hstore;"'
NOTICE: extension "hstore" already exists, skipping
I, [2023-01-08T10:02:29.102970 #1] INFO -- : CREATE EXTENSION
I, [2023-01-08T10:02:29.103264 #1] INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists pg_trgm;"'
NOTICE: extension "pg_trgm" already exists, skipping
I, [2023-01-08T10:02:29.167237 #1] INFO -- : CREATE EXTENSION
I, [2023-01-08T10:02:29.167506 #1] INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists hstore;"'
NOTICE: extension "hstore" already exists, skipping
I, [2023-01-08T10:02:29.233713 #1] INFO -- : CREATE EXTENSION
I, [2023-01-08T10:02:29.233931 #1] INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists pg_trgm;"'
NOTICE: extension "pg_trgm" already exists, skipping
I, [2023-01-08T10:02:29.296228 #1] INFO -- : CREATE EXTENSION
I, [2023-01-08T10:02:29.296631 #1] INFO -- : > sudo -u postgres psql discourse
I, [2023-01-08T10:02:29.300403 #1] INFO -- : update pg_database set encoding = pg_char_to_encoding('UTF8') where datname = 'discourse' AND encoding = pg_char_to_encoding('SQL_ASCII');
I, [2023-01-08T10:02:29.392287 #1] INFO -- : File > /var/lib/postgresql/take-database-backup chmod: +x chown: postgres:postgres
I, [2023-01-08T10:02:29.396585 #1] INFO -- : File > /var/spool/cron/crontabs/postgres chmod: chown:
I, [2023-01-08T10:02:29.396782 #1] INFO -- : > echo postgres installed!
I, [2023-01-08T10:02:29.399217 #1] INFO -- : postgres installed!
I, [2023-01-08T10:02:29.404860 #1] INFO -- : File > /etc/service/redis/run chmod: +x chown:
I, [2023-01-08T10:02:29.410365 #1] INFO -- : File > /etc/service/redis/log/run chmod: +x chown:
I, [2023-01-08T10:02:29.415874 #1] INFO -- : File > /etc/runit/3.d/10-redis chmod: +x chown:
I, [2023-01-08T10:02:29.417263 #1] INFO -- : Replacing daemonize yes with in /etc/redis/redis.conf
I, [2023-01-08T10:02:29.418070 #1] INFO -- : Replacing (?-mix:^pidfile.*$) with in /etc/redis/redis.conf
I, [2023-01-08T10:02:29.419197 #1] INFO -- : > install -d -m 0755 -o redis -g redis /shared/redis_data
I, [2023-01-08T10:02:29.422449 #1] INFO -- :
I, [2023-01-08T10:02:29.422962 #1] INFO -- : Replacing (?-mix:^logfile.*$) with logfile "" in /etc/redis/redis.conf
I, [2023-01-08T10:02:29.423516 #1] INFO -- : Replacing (?-mix:^bind .*$) with in /etc/redis/redis.conf
I, [2023-01-08T10:02:29.424221 #1] INFO -- : Replacing (?-mix:^dir .*$) with dir /shared/redis_data in /etc/redis/redis.conf
I, [2023-01-08T10:02:29.424956 #1] INFO -- : Replacing (?-mix:^protected-mode yes) with protected-mode no in /etc/redis/redis.conf
I, [2023-01-08T10:02:29.425580 #1] INFO -- : Replacing # io-threads 4 with io-threads $redis_io_threads in /etc/redis/redis.conf
I, [2023-01-08T10:02:29.426230 #1] INFO -- : > echo redis installed
I, [2023-01-08T10:02:29.428842 #1] INFO -- : redis installed
I, [2023-01-08T10:02:29.429031 #1] INFO -- : > cat /etc/redis/redis.conf | grep logfile
I, [2023-01-08T10:02:29.435746 #1] INFO -- : logfile ""
I, [2023-01-08T10:02:29.435909 #1] INFO -- : > exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf
I, [2023-01-08T10:02:29.438431 #1] INFO -- : > sleep 10
102:C 08 Jan 2023 10:02:29.468 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
102:C 08 Jan 2023 10:02:29.468 # Redis version=7.0.7, bits=64, commit=00000000, modified=0, pid=102, just started
102:C 08 Jan 2023 10:02:29.468 # Configuration loaded
102:M 08 Jan 2023 10:02:29.469 * monotonic clock: POSIX clock_gettime
102:M 08 Jan 2023 10:02:29.471 * Running mode=standalone, port=6379.
102:M 08 Jan 2023 10:02:29.471 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
102:M 08 Jan 2023 10:02:29.471 # Server initialized
102:M 08 Jan 2023 10:02:29.471 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
102:M 08 Jan 2023 10:02:29.471 # WARNING You have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo madvise > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled (set to 'madvise' or 'never').
102:M 08 Jan 2023 10:02:29.472 * Loading RDB produced by version 7.0.7
102:M 08 Jan 2023 10:02:29.472 * RDB age 4308 seconds
102:M 08 Jan 2023 10:02:29.472 * RDB memory usage when created 269.18 Mb
102:M 08 Jan 2023 10:02:31.177 * Done loading RDB, keys loaded: 43865, keys expired: 3782.
102:M 08 Jan 2023 10:02:31.177 * DB loaded from disk: 1.706 seconds
102:M 08 Jan 2023 10:02:31.177 * Ready to accept connections
I, [2023-01-08T10:02:39.441240 #1] INFO -- :
I, [2023-01-08T10:02:39.442074 #1] INFO -- : > thpoff echo "thpoff is installed!"
I, [2023-01-08T10:02:39.447520 #1] INFO -- : thpoff is installed!
I, [2023-01-08T10:02:39.447934 #1] INFO -- : > /usr/local/bin/ruby -e 'if ENV["DISCOURSE_SMTP_ADDRESS"] == "smtp.example.com"; puts "Aborting! Mail is not configured!"; exit 1; end'
I, [2023-01-08T10:02:39.639965 #1] INFO -- :
I, [2023-01-08T10:02:39.640606 #1] INFO -- : > /usr/local/bin/ruby -e 'if ENV["DISCOURSE_HOSTNAME"] == "discourse.example.com"; puts "Aborting! Domain is not configured!"; exit 1; end'
I, [2023-01-08T10:02:39.784645 #1] INFO -- :
I, [2023-01-08T10:02:39.785173 #1] INFO -- : > /usr/local/bin/ruby -e 'if (ENV["DISCOURSE_CDN_URL"] || "")[0..1] == "//"; puts "Aborting! CDN must have a protocol specified. Once fixed you should rebake your posts now to correct all posts."; exit 1; end'
I, [2023-01-08T10:02:39.919869 #1] INFO -- :
I, [2023-01-08T10:02:39.920430 #1] INFO -- : > rm -f /etc/cron.d/anacron
I, [2023-01-08T10:02:39.924493 #1] INFO -- :
I, [2023-01-08T10:02:39.928198 #1] INFO -- : File > /etc/cron.d/anacron chmod: chown:
I, [2023-01-08T10:02:39.935098 #1] INFO -- : File > /etc/runit/1.d/copy-env chmod: +x chown:
I, [2023-01-08T10:02:39.941466 #1] INFO -- : File > /etc/service/unicorn/run chmod: +x chown:
I, [2023-01-08T10:02:39.947679 #1] INFO -- : File > /etc/service/nginx/run chmod: +x chown:
I, [2023-01-08T10:02:39.953746 #1] INFO -- : File > /etc/runit/3.d/01-nginx chmod: +x chown:
I, [2023-01-08T10:02:39.959831 #1] INFO -- : File > /etc/runit/3.d/02-unicorn chmod: +x chown:
I, [2023-01-08T10:02:39.960073 #1] INFO -- : Replacing # postgres with sv start postgres || exit 1 in /etc/service/unicorn/run
I, [2023-01-08T10:02:39.960480 #1] INFO -- : > exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf
I, [2023-01-08T10:02:39.963937 #1] INFO -- : > cd /var/www/discourse && git reset --hard
129:C 08 Jan 2023 10:02:39.973 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
129:C 08 Jan 2023 10:02:39.973 # Redis version=7.0.7, bits=64, commit=00000000, modified=0, pid=129, just started
129:C 08 Jan 2023 10:02:39.973 # Configuration loaded
129:M 08 Jan 2023 10:02:39.973 * monotonic clock: POSIX clock_gettime
129:M 08 Jan 2023 10:02:39.974 # Warning: Could not create server TCP listening socket *:6379: bind: Address already in use
129:M 08 Jan 2023 10:02:39.974 # Failed listening on port 6379 (TCP), aborting.
Updating files: 100% (32637/32637), done.
I, [2023-01-08T10:02:44.070762 #1] INFO -- : HEAD is now at e775ed1c DEV: Properly include spec examples (#19532)
I, [2023-01-08T10:02:44.070958 #1] INFO -- : > cd /var/www/discourse && git clean -f
I, [2023-01-08T10:02:44.243596 #1] INFO -- :
I, [2023-01-08T10:02:44.243778 #1] INFO -- : > cd /var/www/discourse && git remote set-branches --add origin main
I, [2023-01-08T10:02:44.248754 #1] INFO -- :
I, [2023-01-08T10:02:44.248928 #1] INFO -- : > cd /var/www/discourse && git remote set-branches origin tests-passed
I, [2023-01-08T10:02:44.253401 #1] INFO -- :
I, [2023-01-08T10:02:44.253516 #1] INFO -- : > cd /var/www/discourse && git fetch --depth 1 origin tests-passed
fatal: unable to access 'https://github.com/discourse/discourse.git/': Could not resolve host: github.com
I, [2023-01-08T10:03:04.333061 #1] INFO -- :
I, [2023-01-08T10:03:04.333557 #1] INFO -- : Terminating async processes
I, [2023-01-08T10:03:04.333603 #1] INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main pid: 41
I, [2023-01-08T10:03:04.333661 #1] INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 102
2023-01-08 10:03:04.333 UTC [41] LOG: received fast shutdown request
102:signal-handler (1673172184) Received SIGTERM scheduling shutdown...
2023-01-08 10:03:04.335 UTC [41] LOG: aborting any active transactions
2023-01-08 10:03:04.340 UTC [41] LOG: background worker "logical replication launcher" (PID 50) exited with exit code 1
2023-01-08 10:03:04.340 UTC [45] LOG: shutting down
102:M 08 Jan 2023 10:03:04.375 # User requested shutdown...
102:M 08 Jan 2023 10:03:04.375 * Saving the final RDB snapshot before exiting.
2023-01-08 10:03:04.387 UTC [41] LOG: database system is shut down
102:M 08 Jan 2023 10:03:05.898 * DB saved on disk
102:M 08 Jan 2023 10:03:05.898 # Redis is now ready to exit, bye bye...
FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && git fetch --depth 1 origin tests-passed failed with return #
Location of failure: /usr/local/lib/ruby/gems/3.1.0/gems/pups-1.1.1/lib/pups/exec_command.rb:117:in `spawn'
exec failed with the params {"cd"=>"$home", "hook"=>"code", "cmd"=>["git reset --hard", "git clean -f", "git remote set-branches --add origin main", "git remote set-branches origin $version", "git fetch --depth 1 origin $version", "git checkout $version", "mkdir -p tmp", "chown discourse:www-data tmp", "mkdir -p tmp/pids", "mkdir -p tmp/sockets", "touch tmp/.gitkeep", "mkdir -p /shared/log/rails", "bash -c \"touch -a /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log\"", "bash -c \"ln -s /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log $home/log\"", "bash -c \"mkdir -p /shared/{uploads,backups}\"", "bash -c \"ln -s /shared/{uploads,backups} $home/public\"", "bash -c \"mkdir -p /shared/tmp/{backups,restores}\"", "bash -c \"ln -s /shared/tmp/{backups,restores} $home/tmp\"", "chown -R discourse:www-data /shared/log/rails /shared/uploads /shared/backups /shared/tmp", "[! -d public/plugins] || find public/plugins/ -maxdepth 1 -xtype l -delete"]}
bootstrap failed with exit code 128
**FAILED TO BOOTSTRAP** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.
021af933a82ed69c2ea566b7ac1e5cdee45b11d1da210d1359d43e849f4b5025
root@forum:/var/discourse# ```
````
---
### Author:  [@hiddenseal](https://meta.discourse.org/u/hiddenseal)
#### Post date: [8 januari 2023 om 10:57 UTC](https://meta.discourse.org/t/help-discourse-update-broke-app-fails-to-rebuild/251043/2 "2023-01-08T10:57:22Z")
Hm. I had the exact same issue with a different hosting provider and resolved it by moving to digital ocean.
> [@Docker sometimes fails to connect to github.com](https://meta.discourse.org/t/docker-sometimes-fails-to-connect-to-github-com/250643):
>
> While trying to upgrade Discourse via ./launcher rebuild app I get an issue with fatal: unable to access 'https://github.com/discourse/REPO.git/': Failed to connect to github.com port 443: Connection timed out However, the problem is: sometimes this fails even for discourse. But if you try to do it again, immediately after, with no changes, it may go through, but fail on one of the plugins. If I remove all the plugins, the discourse connection may fail or it may go through and actually build tâŚ
What geographic location did you chose for the droplet?
---
### Author:  [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [8 januari 2023 om 12:30 UTC](https://meta.discourse.org/t/help-discourse-update-broke-app-fails-to-rebuild/251043/3 "2023-01-08T12:30:14Z")
Exit code 128 is out of memory. Increase your ram or swap.
---
### Author:  [@RGJ](https://meta.discourse.org/u/RGJ)
#### Post date: [8 januari 2023 om 15:58 UTC](https://meta.discourse.org/t/help-discourse-update-broke-app-fails-to-rebuild/251043/4 "2023-01-08T15:58:26Z")
> [@pfaffman](#):
>
> Exit code 128 is out of memory.
No itâs not? 128 is a generic error code. Out of memory results in 137 most of the time.
`could not resolve host` is a network of DNS issue.
---
### Author:  [@littleviolette](https://meta.discourse.org/u/littleviolette)
#### Post date: [8 januari 2023 om 17:25 UTC](https://meta.discourse.org/t/help-discourse-update-broke-app-fails-to-rebuild/251043/5 "2023-01-08T17:25:57Z")
@pfaffman After increasing the vCPU from 6 - 8 (the only thing I can upgrade on DigitalOcean), I am still getting the same `Exit Code 128` error.
My serverâs spec: 8vCPUs/16GB Memory/320GB SSD/6TB Transfer
```plaintext
System load: 0.2
Usage of /: 55.6% of 314.85GB
Memory usage: 2%
Swap usage: 0%
Processes: 165
```
Here is the complete error log:
```plaintext
root@forum:/var/discourse# ./launcher rebuild app
x86_64 arch detected.
WARNING: containers/app.yml file is world-readable. You can secure this file by running: chmod o-rwx containers/app.yml
Ensuring launcher is up to date
Fetching origin
Launcher is up-to-date
Stopping old container
+ /usr/bin/docker stop -t 60 app
app
2.0.20221221-0050: Pulling from discourse/base
Digest: sha256:7ac6489e34a0f40f32457bffaa331940a98447b46451a6a1279a79868ee5420b
Status: Image is up to date for discourse/base:2.0.20221221-0050
docker.io/discourse/base:2.0.20221221-0050
/usr/local/lib/ruby/gems/3.1.0/gems/pups-1.1.1/lib/pups.rb
/usr/local/bin/pups --stdin
I, [2023-01-08T17:17:51.784634 #1] INFO -- : Reading from stdin
I, [2023-01-08T17:17:51.789935 #1] INFO -- : > locale-gen $LANG && update-locale
I, [2023-01-08T17:17:51.866583 #1] INFO -- : Generating locales (this might take a while)...
Generation complete.
I, [2023-01-08T17:17:51.866813 #1] INFO -- : > mkdir -p /shared/postgres_run
I, [2023-01-08T17:17:51.873210 #1] INFO -- :
I, [2023-01-08T17:17:51.874108 #1] INFO -- : > chown postgres:postgres /shared/postgres_run
I, [2023-01-08T17:17:51.877428 #1] INFO -- :
I, [2023-01-08T17:17:51.877600 #1] INFO -- : > chmod 775 /shared/postgres_run
I, [2023-01-08T17:17:51.880335 #1] INFO -- :
I, [2023-01-08T17:17:51.880503 #1] INFO -- : > rm -fr /var/run/postgresql
I, [2023-01-08T17:17:51.883210 #1] INFO -- :
I, [2023-01-08T17:17:51.883316 #1] INFO -- : > ln -s /shared/postgres_run /var/run/postgresql
I, [2023-01-08T17:17:51.886051 #1] INFO -- :
I, [2023-01-08T17:17:51.886148 #1] INFO -- : > socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1
2023/01/08 17:17:51 socat[18] E connect(6, AF=1 "/shared/postgres_run/.s.PGSQL.5432", 36): No such file or directory
I, [2023-01-08T17:17:51.909808 #1] INFO -- :
I, [2023-01-08T17:17:51.909975 #1] INFO -- : > rm -fr /shared/postgres_run/.s*
I, [2023-01-08T17:17:51.913236 #1] INFO -- :
I, [2023-01-08T17:17:51.913423 #1] INFO -- : > rm -fr /shared/postgres_run/*.pid
I, [2023-01-08T17:17:51.916137 #1] INFO -- :
I, [2023-01-08T17:17:51.916229 #1] INFO -- : > mkdir -p /shared/postgres_run/13-main.pg_stat_tmp
I, [2023-01-08T17:17:51.918500 #1] INFO -- :
I, [2023-01-08T17:17:51.918680 #1] INFO -- : > chown postgres:postgres /shared/postgres_run/13-main.pg_stat_tmp
I, [2023-01-08T17:17:51.920791 #1] INFO -- :
I, [2023-01-08T17:17:51.925959 #1] INFO -- : File > /etc/service/postgres/run chmod: +x chown:
I, [2023-01-08T17:17:51.930666 #1] INFO -- : File > /etc/service/postgres/log/run chmod: +x chown:
I, [2023-01-08T17:17:51.935630 #1] INFO -- : File > /etc/runit/3.d/99-postgres chmod: +x chown:
I, [2023-01-08T17:17:51.940501 #1] INFO -- : File > /root/upgrade_postgres chmod: +x chown:
I, [2023-01-08T17:17:51.940815 #1] INFO -- : > chown -R root /var/lib/postgresql/13/main
I, [2023-01-08T17:17:52.327524 #1] INFO -- :
I, [2023-01-08T17:17:52.327809 #1] INFO -- : > [! -e /shared/postgres_data] && install -d -m 0755 -o postgres -g postgres /shared/postgres_data && sudo -E -u postgres /usr/lib/postgresql/13/bin/initdb -D /shared/postgres_data || exit 0
I, [2023-01-08T17:17:52.330153 #1] INFO -- :
I, [2023-01-08T17:17:52.330206 #1] INFO -- : > chown -R postgres:postgres /shared/postgres_data
I, [2023-01-08T17:17:52.385884 #1] INFO -- :
I, [2023-01-08T17:17:52.386017 #1] INFO -- : > chown -R postgres:postgres /var/run/postgresql
I, [2023-01-08T17:17:52.388853 #1] INFO -- :
I, [2023-01-08T17:17:52.389039 #1] INFO -- : > /root/upgrade_postgres
I, [2023-01-08T17:17:52.394749 #1] INFO -- :
I, [2023-01-08T17:17:52.394868 #1] INFO -- : > rm /root/upgrade_postgres
I, [2023-01-08T17:17:52.397391 #1] INFO -- :
I, [2023-01-08T17:17:52.398496 #1] INFO -- : Replacing data_directory = '/var/lib/postgresql/13/main' with data_directory = '/shared/postgres_data' in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-08T17:17:52.399030 #1] INFO -- : Replacing (?-mix:#?listen_addresses *=.*) with listen_addresses = '*' in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-08T17:17:52.399339 #1] INFO -- : Replacing (?-mix:#?synchronous_commit *=.*) with synchronous_commit = $db_synchronous_commit in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-08T17:17:52.399638 #1] INFO -- : Replacing (?-mix:#?shared_buffers *=.*) with shared_buffers = $db_shared_buffers in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-08T17:17:52.399932 #1] INFO -- : Replacing (?-mix:#?work_mem *=.*) with work_mem = $db_work_mem in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-08T17:17:52.400191 #1] INFO -- : Replacing (?-mix:#?default_text_search_config *=.*) with default_text_search_config = '$db_default_text_search_config' in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-08T17:17:52.400449 #1] INFO -- : > install -d -m 0755 -o postgres -g postgres /shared/postgres_backup
I, [2023-01-08T17:17:52.405233 #1] INFO -- :
I, [2023-01-08T17:17:52.405420 #1] INFO -- : Replacing (?-mix:#?checkpoint_segments *=.*) with checkpoint_segments = $db_checkpoint_segments in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-08T17:17:52.405722 #1] INFO -- : Replacing (?-mix:#?logging_collector *=.*) with logging_collector = $db_logging_collector in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-08T17:17:52.406030 #1] INFO -- : Replacing (?-mix:#?log_min_duration_statement *=.*) with log_min_duration_statement = $db_log_min_duration_statement in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-08T17:17:52.406566 #1] INFO -- : Replacing (?-mix:^#local +replication +postgres +peer$) with local replication postgres peer in /etc/postgresql/13/main/pg_hba.conf
I, [2023-01-08T17:17:52.406842 #1] INFO -- : Replacing (?-mix:^host.*all.*all.*127.*$) with host all all 0.0.0.0/0 md5 in /etc/postgresql/13/main/pg_hba.conf
I, [2023-01-08T17:17:52.407086 #1] INFO -- : Replacing (?-mix:^host.*all.*all.*::1\/128.*$) with host all all ::/0 md5 in /etc/postgresql/13/main/pg_hba.conf
I, [2023-01-08T17:17:52.407305 #1] INFO -- : > HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main
I, [2023-01-08T17:17:52.409278 #1] INFO -- : > sleep 5
2023-01-08 17:17:52.593 UTC [41] LOG: starting PostgreSQL 13.9 (Debian 13.9-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2023-01-08 17:17:52.594 UTC [41] LOG: listening on IPv4 address "0.0.0.0", port 5432
2023-01-08 17:17:52.594 UTC [41] LOG: listening on IPv6 address "::", port 5432
2023-01-08 17:17:52.598 UTC [41] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-01-08 17:17:52.605 UTC [44] LOG: database system was shut down at 2023-01-08 17:17:42 UTC
2023-01-08 17:17:52.621 UTC [41] LOG: database system is ready to accept connections
I, [2023-01-08T17:17:57.412189 #1] INFO -- :
I, [2023-01-08T17:17:57.412815 #1] INFO -- : > su postgres -c 'createdb discourse' || true
2023-01-08 17:17:57.500 UTC [54] postgres@postgres ERROR: database "discourse" already exists
2023-01-08 17:17:57.500 UTC [54] postgres@postgres STATEMENT: CREATE DATABASE discourse;
createdb: error: database creation failed: ERROR: database "discourse" already exists
I, [2023-01-08T17:17:57.502683 #1] INFO -- :
I, [2023-01-08T17:17:57.503102 #1] INFO -- : > su postgres -c 'psql discourse -c "create user discourse;"' || true
2023-01-08 17:17:57.571 UTC [58] postgres@discourse ERROR: role "discourse" already exists
2023-01-08 17:17:57.571 UTC [58] postgres@discourse STATEMENT: create user discourse;
ERROR: role "discourse" already exists
I, [2023-01-08T17:17:57.573448 #1] INFO -- :
I, [2023-01-08T17:17:57.573809 #1] INFO -- : > su postgres -c 'psql discourse -c "grant all privileges on database discourse to discourse;"' || true
I, [2023-01-08T17:17:57.632967 #1] INFO -- : GRANT
I, [2023-01-08T17:17:57.633283 #1] INFO -- : > su postgres -c 'psql discourse -c "alter schema public owner to discourse;"'
I, [2023-01-08T17:17:57.683299 #1] INFO -- : ALTER SCHEMA
I, [2023-01-08T17:17:57.683765 #1] INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists hstore;"'
NOTICE: extension "hstore" already exists, skipping
I, [2023-01-08T17:17:57.737889 #1] INFO -- : CREATE EXTENSION
I, [2023-01-08T17:17:57.738056 #1] INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists pg_trgm;"'
NOTICE: extension "pg_trgm" already exists, skipping
I, [2023-01-08T17:17:57.788026 #1] INFO -- : CREATE EXTENSION
I, [2023-01-08T17:17:57.788254 #1] INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists hstore;"'
NOTICE: extension "hstore" already exists, skipping
I, [2023-01-08T17:17:57.834234 #1] INFO -- : CREATE EXTENSION
I, [2023-01-08T17:17:57.834647 #1] INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists pg_trgm;"'
NOTICE: extension "pg_trgm" already exists, skipping
I, [2023-01-08T17:17:57.882288 #1] INFO -- : CREATE EXTENSION
I, [2023-01-08T17:17:57.882631 #1] INFO -- : > sudo -u postgres psql discourse
I, [2023-01-08T17:17:57.885847 #1] INFO -- : update pg_database set encoding = pg_char_to_encoding('UTF8') where datname = 'discourse' AND encoding = pg_char_to_encoding('SQL_ASCII');
I, [2023-01-08T17:17:57.954282 #1] INFO -- : File > /var/lib/postgresql/take-database-backup chmod: +x chown: postgres:postgres
I, [2023-01-08T17:17:57.957842 #1] INFO -- : File > /var/spool/cron/crontabs/postgres chmod: chown:
I, [2023-01-08T17:17:57.957973 #1] INFO -- : > echo postgres installed!
I, [2023-01-08T17:17:57.959975 #1] INFO -- : postgres installed!
I, [2023-01-08T17:17:57.964538 #1] INFO -- : File > /etc/service/redis/run chmod: +x chown:
I, [2023-01-08T17:17:57.969401 #1] INFO -- : File > /etc/service/redis/log/run chmod: +x chown:
I, [2023-01-08T17:17:57.974012 #1] INFO -- : File > /etc/runit/3.d/10-redis chmod: +x chown:
I, [2023-01-08T17:17:57.975225 #1] INFO -- : Replacing daemonize yes with in /etc/redis/redis.conf
I, [2023-01-08T17:17:57.975797 #1] INFO -- : Replacing (?-mix:^pidfile.*$) with in /etc/redis/redis.conf
I, [2023-01-08T17:17:57.977653 #1] INFO -- : > install -d -m 0755 -o redis -g redis /shared/redis_data
I, [2023-01-08T17:17:57.980144 #1] INFO -- :
I, [2023-01-08T17:17:57.980429 #1] INFO -- : Replacing (?-mix:^logfile.*$) with logfile "" in /etc/redis/redis.conf
I, [2023-01-08T17:17:57.980926 #1] INFO -- : Replacing (?-mix:^bind .*$) with in /etc/redis/redis.conf
I, [2023-01-08T17:17:57.981625 #1] INFO -- : Replacing (?-mix:^dir .*$) with dir /shared/redis_data in /etc/redis/redis.conf
I, [2023-01-08T17:17:57.982227 #1] INFO -- : Replacing (?-mix:^protected-mode yes) with protected-mode no in /etc/redis/redis.conf
I, [2023-01-08T17:17:57.982908 #1] INFO -- : Replacing # io-threads 4 with io-threads $redis_io_threads in /etc/redis/redis.conf
I, [2023-01-08T17:17:57.983543 #1] INFO -- : > echo redis installed
I, [2023-01-08T17:17:57.985612 #1] INFO -- : redis installed
I, [2023-01-08T17:17:57.985800 #1] INFO -- : > cat /etc/redis/redis.conf | grep logfile
I, [2023-01-08T17:17:57.992316 #1] INFO -- : logfile ""
I, [2023-01-08T17:17:57.992577 #1] INFO -- : > exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf
I, [2023-01-08T17:17:57.994136 #1] INFO -- : > sleep 10
102:C 08 Jan 2023 17:17:58.023 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
102:C 08 Jan 2023 17:17:58.023 # Redis version=7.0.7, bits=64, commit=00000000, modified=0, pid=102, just started
102:C 08 Jan 2023 17:17:58.023 # Configuration loaded
102:M 08 Jan 2023 17:17:58.023 * monotonic clock: POSIX clock_gettime
102:M 08 Jan 2023 17:17:58.025 * Running mode=standalone, port=6379.
102:M 08 Jan 2023 17:17:58.025 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
102:M 08 Jan 2023 17:17:58.025 # Server initialized
102:M 08 Jan 2023 17:17:58.025 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
102:M 08 Jan 2023 17:17:58.025 # WARNING You have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo madvise > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled (set to 'madvise' or 'never').
102:M 08 Jan 2023 17:17:58.026 * Loading RDB produced by version 7.0.7
102:M 08 Jan 2023 17:17:58.026 * RDB age 26094 seconds
102:M 08 Jan 2023 17:17:58.026 * RDB memory usage when created 267.24 Mb
102:M 08 Jan 2023 17:17:59.205 * Done loading RDB, keys loaded: 28568, keys expired: 15293.
102:M 08 Jan 2023 17:17:59.205 * DB loaded from disk: 1.179 seconds
102:M 08 Jan 2023 17:17:59.205 * Ready to accept connections
I, [2023-01-08T17:18:07.996578 #1] INFO -- :
I, [2023-01-08T17:18:07.997276 #1] INFO -- : > thpoff echo "thpoff is installed!"
I, [2023-01-08T17:18:08.001894 #1] INFO -- : thpoff is installed!
I, [2023-01-08T17:18:08.002222 #1] INFO -- : > /usr/local/bin/ruby -e 'if ENV["DISCOURSE_SMTP_ADDRESS"] == "smtp.example.com"; puts "Aborting! Mail is not configured!"; exit 1; end'
I, [2023-01-08T17:18:08.163344 #1] INFO -- :
I, [2023-01-08T17:18:08.163791 #1] INFO -- : > /usr/local/bin/ruby -e 'if ENV["DISCOURSE_HOSTNAME"] == "discourse.example.com"; puts "Aborting! Domain is not configured!"; exit 1; end'
I, [2023-01-08T17:18:08.256476 #1] INFO -- :
I, [2023-01-08T17:18:08.257079 #1] INFO -- : > /usr/local/bin/ruby -e 'if (ENV["DISCOURSE_CDN_URL"] || "")[0..1] == "//"; puts "Aborting! CDN must have a protocol specified. Once fixed you should rebake your posts now to correct all posts."; exit 1; end'
I, [2023-01-08T17:18:08.335922 #1] INFO -- :
I, [2023-01-08T17:18:08.336428 #1] INFO -- : > rm -f /etc/cron.d/anacron
I, [2023-01-08T17:18:08.340179 #1] INFO -- :
I, [2023-01-08T17:18:08.343109 #1] INFO -- : File > /etc/cron.d/anacron chmod: chown:
I, [2023-01-08T17:18:08.348517 #1] INFO -- : File > /etc/runit/1.d/copy-env chmod: +x chown:
I, [2023-01-08T17:18:08.353394 #1] INFO -- : File > /etc/service/unicorn/run chmod: +x chown:
I, [2023-01-08T17:18:08.357978 #1] INFO -- : File > /etc/service/nginx/run chmod: +x chown:
I, [2023-01-08T17:18:08.362553 #1] INFO -- : File > /etc/runit/3.d/01-nginx chmod: +x chown:
I, [2023-01-08T17:18:08.367045 #1] INFO -- : File > /etc/runit/3.d/02-unicorn chmod: +x chown:
I, [2023-01-08T17:18:08.367288 #1] INFO -- : Replacing # postgres with sv start postgres || exit 1 in /etc/service/unicorn/run
I, [2023-01-08T17:18:08.367561 #1] INFO -- : > exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf
I, [2023-01-08T17:18:08.370035 #1] INFO -- : > cd /var/www/discourse && git reset --hard
129:C 08 Jan 2023 17:18:08.376 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
129:C 08 Jan 2023 17:18:08.376 # Redis version=7.0.7, bits=64, commit=00000000, modified=0, pid=129, just started
129:C 08 Jan 2023 17:18:08.376 # Configuration loaded
129:M 08 Jan 2023 17:18:08.377 * monotonic clock: POSIX clock_gettime
129:M 08 Jan 2023 17:18:08.377 # Warning: Could not create server TCP listening socket *:6379: bind: Address already in use
129:M 08 Jan 2023 17:18:08.377 # Failed listening on port 6379 (TCP), aborting.
Updating files: 100% (32637/32637), done.
I, [2023-01-08T17:18:11.718607 #1] INFO -- : HEAD is now at e775ed1c DEV: Properly include spec examples (#19532)
I, [2023-01-08T17:18:11.718853 #1] INFO -- : > cd /var/www/discourse && git clean -f
I, [2023-01-08T17:18:11.867916 #1] INFO -- :
I, [2023-01-08T17:18:11.868312 #1] INFO -- : > cd /var/www/discourse && git remote set-branches --add origin main
I, [2023-01-08T17:18:11.873068 #1] INFO -- :
I, [2023-01-08T17:18:11.873230 #1] INFO -- : > cd /var/www/discourse && git remote set-branches origin tests-passed
I, [2023-01-08T17:18:11.877351 #1] INFO -- :
I, [2023-01-08T17:18:11.877407 #1] INFO -- : > cd /var/www/discourse && git fetch --depth 1 origin tests-passed
fatal: unable to access 'https://github.com/discourse/discourse.git/': Could not resolve host: github.com
I, [2023-01-08T17:18:31.945158 #1] INFO -- :
I, [2023-01-08T17:18:31.945552 #1] INFO -- : Terminating async processes
I, [2023-01-08T17:18:31.945652 #1] INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main pid: 41
2023-01-08 17:18:31.945 UTC [41] LOG: received fast shutdown request
I, [2023-01-08T17:18:31.945947 #1] INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 102
102:signal-handler (1673198311) Received SIGTERM scheduling shutdown...
2023-01-08 17:18:31.949 UTC [41] LOG: aborting any active transactions
2023-01-08 17:18:31.952 UTC [41] LOG: background worker "logical replication launcher" (PID 50) exited with exit code 1
2023-01-08 17:18:31.952 UTC [45] LOG: shutting down
102:M 08 Jan 2023 17:18:31.994 # User requested shutdown...
102:M 08 Jan 2023 17:18:31.994 * Saving the final RDB snapshot before exiting.
2023-01-08 17:18:32.003 UTC [41] LOG: database system is shut down
102:M 08 Jan 2023 17:18:33.194 * DB saved on disk
102:M 08 Jan 2023 17:18:33.194 # Redis is now ready to exit, bye bye...
FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && git fetch --depth 1 origin tests-passed failed with return #
Location of failure: /usr/local/lib/ruby/gems/3.1.0/gems/pups-1.1.1/lib/pups/exec_command.rb:117:in `spawn'
exec failed with the params {"cd"=>"$home", "hook"=>"code", "cmd"=>["git reset --hard", "git clean -f", "git remote set-branches --add origin main", "git remote set-branches origin $version", "git fetch --depth 1 origin $version", "git checkout $version", "mkdir -p tmp", "chown discourse:www-data tmp", "mkdir -p tmp/pids", "mkdir -p tmp/sockets", "touch tmp/.gitkeep", "mkdir -p /shared/log/rails", "bash -c \"touch -a /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log\"", "bash -c \"ln -s /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log $home/log\"", "bash -c \"mkdir -p /shared/{uploads,backups}\"", "bash -c \"ln -s /shared/{uploads,backups} $home/public\"", "bash -c \"mkdir -p /shared/tmp/{backups,restores}\"", "bash -c \"ln -s /shared/tmp/{backups,restores} $home/tmp\"", "chown -R discourse:www-data /shared/log/rails /shared/uploads /shared/backups /shared/tmp", "[! -d public/plugins] || find public/plugins/ -maxdepth 1 -xtype l -delete"]}
bootstrap failed with exit code 128
**FAILED TO BOOTSTRAP** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.
75d71c3f22d42c746becd5f6fbeb5529fd01ff4a7176d2b942930aad6f973b81
```
---
### Author:  [@littleviolette](https://meta.discourse.org/u/littleviolette)
#### Post date: [8 januari 2023 om 17:26 UTC](https://meta.discourse.org/t/help-discourse-update-broke-app-fails-to-rebuild/251043/6 "2023-01-08T17:26:27Z")
@RGJ How can I resolve the DNS issue? Are there other issues at play?
---
### Author:  [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [8 januari 2023 om 18:01 UTC](https://meta.discourse.org/t/help-discourse-update-broke-app-fails-to-rebuild/251043/7 "2023-01-08T18:01:27Z")
Yeah. Sorry. I didnât know you had that much ram.
`fatal: unable to access 'https://github.com/discourse/discourse.git/': Could not resolve host: github.com`
Thatâs the issue. Donât know how youâd get that on a digital ocean droplet. Must be something with your docker network. But I donât have any ideas.
Edit: hereâs one, make sure docker is up to date and reboot. See if you can
```
docker run hello-world
```
---
### Author:  [@littleviolette](https://meta.discourse.org/u/littleviolette)
#### Post date: [8 januari 2023 om 18:50 UTC](https://meta.discourse.org/t/help-discourse-update-broke-app-fails-to-rebuild/251043/9 "2023-01-08T18:50:40Z")
SFO1 DigitalOcean server
---
### Author:  [@littleviolette](https://meta.discourse.org/u/littleviolette)
#### Post date: [8 januari 2023 om 21:40 UTC](https://meta.discourse.org/t/help-discourse-update-broke-app-fails-to-rebuild/251043/10 "2023-01-08T21:40:51Z")
Docker seems to be running correctly. Anything else I can try?
```plaintext
root@forum:~# docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
2db29710123e: Pull complete
Digest: sha256:94ebc7edf3401f299cd3376a1669bc0a49aef92d6d2669005f9bc5ef028dc333
Status: Downloaded newer image for hello-world:latest
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/get-started/
```
---
### Author:  [@littleviolette](https://meta.discourse.org/u/littleviolette)
#### Post date: [8 januari 2023 om 21:52 UTC](https://meta.discourse.org/t/help-discourse-update-broke-app-fails-to-rebuild/251043/11 "2023-01-08T21:52:16Z")
The below didnât work. Still getting the same error. đ
```plaintext
git config --global --unset https.proxy
```
---
### Author:  [@littleviolette](https://meta.discourse.org/u/littleviolette)
#### Post date: [9 januari 2023 om 00:29 UTC](https://meta.discourse.org/t/help-discourse-update-broke-app-fails-to-rebuild/251043/12 "2023-01-09T00:29:35Z")
Could this be the cause of the issue? How could I resolve it?
```plaintext
# Warning: Could not create server TCP listening socket *:6379: bind: Address already in use
# Failed listening on port 6379 (TCP), aborting.
```
Thank you!
---
### Author:  [@littleviolette](https://meta.discourse.org/u/littleviolette)
#### Post date: [9 januari 2023 om 01:44 UTC](https://meta.discourse.org/t/help-discourse-update-broke-app-fails-to-rebuild/251043/13 "2023-01-09T01:44:46Z")
I restored a backup of our droplet and try rebuilding the app again. I was returned with the error:
```plaintext
bootstrap failed with exit code 1
```
This old image runs on Ubuntu 16.04.7 LTS (GNU/Linux 3.13.0-52-generic x86\_64).
Anyone would be able to help? Where else can I seek help? Thank you!
Here is the full log:
````plaintext
root@lh-forum-4:/var/discourse# ./launcher rebuild app
x86_64 arch detected.
WARNING: containers/app.yml file is world-readable. You can secure this file by running: chmod o-rwx containers/app.yml
Ensuring launcher is up to date
Fetching origin
remote: Enumerating objects: 75, done.
remote: Counting objects: 100% (46/46), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 75 (delta 38), reused 42 (delta 38), pack-reused 29
Unpacking objects: 100% (75/75), done.
From https://github.com/discourse/discourse_docker
e0f27d1..e6cb71c main -> origin/main
* [new branch] build-27 -> origin/build-27
* [new branch] revert-654-old-image-for-old-discourse -> origin/revert-654-old-image-for-old-discourse
+ 4fae4a0...2e142d4 ruby-3.2 -> origin/ruby-3.2 (forced update)
bdc1b59..4ffccf2 yarn-timeout -> origin/yarn-timeout
Updating Launcher...
Updating e0f27d1..e6cb71c
Fast-forward
.github/workflows/build.yml | 26 +++++++++++++-------------
image/base/VERSION | 1 -
image/base/install-imagemagick | 4 ++--
image/base/install-jemalloc | 10 +++++-----
image/base/install-nginx | 4 ++--
image/base/install-oxipng | 4 ++--
image/base/install-redis | 4 ++--
image/base/install-ruby | 7 ++++++-
image/base/install-rust | 30 ++++++++++++++++++++++++++++++
image/base/slim.Dockerfile | 12 +++++++++---
image/discourse_dev/postgres_dev.template.yml | 1 +
image/discourse_test/Dockerfile | 3 ++-
launcher | 11 ++++++++++-
templates/web.template.yml | 4 ++--
14 files changed, 86 insertions(+), 35 deletions(-)
delete mode 100644 image/base/VERSION
create mode 100755 image/base/install-rust
Launcher updated, restarting...
x86_64 arch detected.
WARNING: We are about to start downloading the Discourse base image
This process may take anywhere between a few minutes to an hour, depending on your network speed
Please be patient
2.0.20221221-0050: Pulling from discourse/base
025c56f98b67: Pull complete
4daec98b5622: Pull complete
05576b07f5fb: Pull complete
Digest: sha256:7ac6489e34a0f40f32457bffaa331940a98447b46451a6a1279a79868ee5420b
Status: Downloaded newer image for discourse/base:2.0.20221221-0050
WARNING: containers/app.yml file is world-readable. You can secure this file by running: chmod o-rwx containers/app.yml
Ensuring launcher is up to date
Fetching origin
Launcher is up-to-date
Stopping old container
+ /usr/bin/docker stop -t 60 app
app
2.0.20221221-0050: Pulling from discourse/base
Digest: sha256:7ac6489e34a0f40f32457bffaa331940a98447b46451a6a1279a79868ee5420b
Status: Image is up to date for discourse/base:2.0.20221221-0050
/usr/local/lib/ruby/gems/3.1.0/gems/pups-1.1.1/lib/pups.rb
/usr/local/bin/pups --stdin
I, [2023-01-09T01:24:11.633490 #1] INFO -- : Reading from stdin
I, [2023-01-09T01:24:11.638961 #1] INFO -- : > locale-gen $LANG && update-locale
I, [2023-01-09T01:24:11.697592 #1] INFO -- : Generating locales (this might take a while)...
Generation complete.
I, [2023-01-09T01:24:11.697856 #1] INFO -- : > mkdir -p /shared/postgres_run
I, [2023-01-09T01:24:11.704671 #1] INFO -- :
I, [2023-01-09T01:24:11.705524 #1] INFO -- : > chown postgres:postgres /shared/postgres_run
I, [2023-01-09T01:24:11.710149 #1] INFO -- :
I, [2023-01-09T01:24:11.710385 #1] INFO -- : > chmod 775 /shared/postgres_run
I, [2023-01-09T01:24:11.713993 #1] INFO -- :
I, [2023-01-09T01:24:11.714224 #1] INFO -- : > rm -fr /var/run/postgresql
I, [2023-01-09T01:24:11.717249 #1] INFO -- :
I, [2023-01-09T01:24:11.717484 #1] INFO -- : > ln -s /shared/postgres_run /var/run/postgresql
I, [2023-01-09T01:24:11.720899 #1] INFO -- :
I, [2023-01-09T01:24:11.721113 #1] INFO -- : > socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1
2023/01/09 01:24:11 socat[19] E connect(6, AF=1 "/shared/postgres_run/.s.PGSQL.5432", 36): Connection refused
I, [2023-01-09T01:24:11.747602 #1] INFO -- :
I, [2023-01-09T01:24:11.747838 #1] INFO -- : > rm -fr /shared/postgres_run/.s*
I, [2023-01-09T01:24:11.752658 #1] INFO -- :
I, [2023-01-09T01:24:11.752860 #1] INFO -- : > rm -fr /shared/postgres_run/*.pid
I, [2023-01-09T01:24:11.756646 #1] INFO -- :
I, [2023-01-09T01:24:11.756769 #1] INFO -- : > mkdir -p /shared/postgres_run/13-main.pg_stat_tmp
I, [2023-01-09T01:24:11.759946 #1] INFO -- :
I, [2023-01-09T01:24:11.760120 #1] INFO -- : > chown postgres:postgres /shared/postgres_run/13-main.pg_stat_tmp
I, [2023-01-09T01:24:11.763132 #1] INFO -- :
I, [2023-01-09T01:24:11.769380 #1] INFO -- : File > /etc/service/postgres/run chmod: +x chown:
I, [2023-01-09T01:24:11.775772 #1] INFO -- : File > /etc/service/postgres/log/run chmod: +x chown:
I, [2023-01-09T01:24:11.782041 #1] INFO -- : File > /etc/runit/3.d/99-postgres chmod: +x chown:
I, [2023-01-09T01:24:11.788362 #1] INFO -- : File > /root/upgrade_postgres chmod: +x chown:
I, [2023-01-09T01:24:11.788781 #1] INFO -- : > chown -R root /var/lib/postgresql/13/main
I, [2023-01-09T01:24:11.886134 #1] INFO -- :
I, [2023-01-09T01:24:11.886778 #1] INFO -- : > [! -e /shared/postgres_data] && install -d -m 0755 -o postgres -g postgres /shared/postgres_data && sudo -E -u postgres /usr/lib/postgresql/13/bin/initdb -D /shared/postgres_data || exit 0
I, [2023-01-09T01:24:11.890560 #1] INFO -- :
I, [2023-01-09T01:24:11.890642 #1] INFO -- : > chown -R postgres:postgres /shared/postgres_data
I, [2023-01-09T01:24:11.937820 #1] INFO -- :
I, [2023-01-09T01:24:11.937984 #1] INFO -- : > chown -R postgres:postgres /var/run/postgresql
I, [2023-01-09T01:24:11.941646 #1] INFO -- :
I, [2023-01-09T01:24:11.941855 #1] INFO -- : > /root/upgrade_postgres
I, [2023-01-09T01:24:11.948144 #1] INFO -- :
I, [2023-01-09T01:24:11.948323 #1] INFO -- : > rm /root/upgrade_postgres
I, [2023-01-09T01:24:11.950684 #1] INFO -- :
I, [2023-01-09T01:24:11.951813 #1] INFO -- : Replacing data_directory = '/var/lib/postgresql/13/main' with data_directory = '/shared/postgres_data' in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-09T01:24:11.952432 #1] INFO -- : Replacing (?-mix:#?listen_addresses *=.*) with listen_addresses = '*' in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-09T01:24:11.952806 #1] INFO -- : Replacing (?-mix:#?synchronous_commit *=.*) with synchronous_commit = $db_synchronous_commit in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-09T01:24:11.953169 #1] INFO -- : Replacing (?-mix:#?shared_buffers *=.*) with shared_buffers = $db_shared_buffers in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-09T01:24:11.953567 #1] INFO -- : Replacing (?-mix:#?work_mem *=.*) with work_mem = $db_work_mem in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-09T01:24:11.954485 #1] INFO -- : Replacing (?-mix:#?default_text_search_config *=.*) with default_text_search_config = '$db_default_text_search_config' in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-09T01:24:11.954943 #1] INFO -- : > install -d -m 0755 -o postgres -g postgres /shared/postgres_backup
I, [2023-01-09T01:24:11.960686 #1] INFO -- :
I, [2023-01-09T01:24:11.960968 #1] INFO -- : Replacing (?-mix:#?checkpoint_segments *=.*) with checkpoint_segments = $db_checkpoint_segments in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-09T01:24:11.961288 #1] INFO -- : Replacing (?-mix:#?logging_collector *=.*) with logging_collector = $db_logging_collector in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-09T01:24:11.961749 #1] INFO -- : Replacing (?-mix:#?log_min_duration_statement *=.*) with log_min_duration_statement = $db_log_min_duration_statement in /etc/postgresql/13/main/postgresql.conf
I, [2023-01-09T01:24:11.962361 #1] INFO -- : Replacing (?-mix:^#local +replication +postgres +peer$) with local replication postgres peer in /etc/postgresql/13/main/pg_hba.conf
I, [2023-01-09T01:24:11.962631 #1] INFO -- : Replacing (?-mix:^host.*all.*all.*127.*$) with host all all 0.0.0.0/0 md5 in /etc/postgresql/13/main/pg_hba.conf
I, [2023-01-09T01:24:11.962854 #1] INFO -- : Replacing (?-mix:^host.*all.*all.*::1\/128.*$) with host all all ::/0 md5 in /etc/postgresql/13/main/pg_hba.conf
I, [2023-01-09T01:24:11.963079 #1] INFO -- : > HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main
I, [2023-01-09T01:24:11.964769 #1] INFO -- : > sleep 5
2023-01-09 01:24:12.155 UTC [42] LOG: starting PostgreSQL 13.9 (Debian 13.9-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2023-01-09 01:24:12.155 UTC [42] LOG: listening on IPv4 address "0.0.0.0", port 5432
2023-01-09 01:24:12.155 UTC [42] LOG: listening on IPv6 address "::", port 5432
2023-01-09 01:24:12.159 UTC [42] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-01-09 01:24:12.180 UTC [45] LOG: database system was interrupted; last known up at 2023-01-09 01:19:52 UTC
2023-01-09 01:24:12.328 UTC [45] LOG: database system was not properly shut down; automatic recovery in progress
2023-01-09 01:24:12.332 UTC [45] LOG: redo starts at 104B/AF57BD60
2023-01-09 01:24:12.399 UTC [45] LOG: redo done at 104B/B014B070
2023-01-09 01:24:12.510 UTC [42] LOG: database system is ready to accept connections
I, [2023-01-09T01:24:16.968330 #1] INFO -- :
I, [2023-01-09T01:24:16.969333 #1] INFO -- : > su postgres -c 'createdb discourse' || true
2023-01-09 01:24:17.073 UTC [55] postgres@postgres ERROR: database "discourse" already exists
2023-01-09 01:24:17.073 UTC [55] postgres@postgres STATEMENT: CREATE DATABASE discourse;
createdb: error: database creation failed: ERROR: database "discourse" already exists
I, [2023-01-09T01:24:17.075782 #1] INFO -- :
I, [2023-01-09T01:24:17.076143 #1] INFO -- : > su postgres -c 'psql discourse -c "create user discourse;"' || true
2023-01-09 01:24:17.158 UTC [59] postgres@discourse ERROR: role "discourse" already exists
2023-01-09 01:24:17.158 UTC [59] postgres@discourse STATEMENT: create user discourse;
ERROR: role "discourse" already exists
I, [2023-01-09T01:24:17.161838 #1] INFO -- :
I, [2023-01-09T01:24:17.162221 #1] INFO -- : > su postgres -c 'psql discourse -c "grant all privileges on database discourse to discourse;"' || true
I, [2023-01-09T01:24:17.225658 #1] INFO -- : GRANT
I, [2023-01-09T01:24:17.225947 #1] INFO -- : > su postgres -c 'psql discourse -c "alter schema public owner to discourse;"'
I, [2023-01-09T01:24:17.287774 #1] INFO -- : ALTER SCHEMA
I, [2023-01-09T01:24:17.288208 #1] INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists hstore;"'
NOTICE: extension "hstore" already exists, skipping
I, [2023-01-09T01:24:17.355297 #1] INFO -- : CREATE EXTENSION
I, [2023-01-09T01:24:17.355532 #1] INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists pg_trgm;"'
NOTICE: extension "pg_trgm" already exists, skipping
I, [2023-01-09T01:24:17.414737 #1] INFO -- : CREATE EXTENSION
I, [2023-01-09T01:24:17.414981 #1] INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists hstore;"'
NOTICE: extension "hstore" already exists, skipping
I, [2023-01-09T01:24:17.480559 #1] INFO -- : CREATE EXTENSION
I, [2023-01-09T01:24:17.480906 #1] INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists pg_trgm;"'
NOTICE: extension "pg_trgm" already exists, skipping
I, [2023-01-09T01:24:17.543581 #1] INFO -- : CREATE EXTENSION
I, [2023-01-09T01:24:17.543962 #1] INFO -- : > sudo -u postgres psql discourse
I, [2023-01-09T01:24:17.548022 #1] INFO -- : update pg_database set encoding = pg_char_to_encoding('UTF8') where datname = 'discourse' AND encoding = pg_char_to_encoding('SQL_ASCII');
I, [2023-01-09T01:24:17.629332 #1] INFO -- : File > /var/lib/postgresql/take-database-backup chmod: +x chown: postgres:postgres
I, [2023-01-09T01:24:17.633132 #1] INFO -- : File > /var/spool/cron/crontabs/postgres chmod: chown:
I, [2023-01-09T01:24:17.633279 #1] INFO -- : > echo postgres installed!
I, [2023-01-09T01:24:17.635614 #1] INFO -- : postgres installed!
I, [2023-01-09T01:24:17.641703 #1] INFO -- : File > /etc/service/redis/run chmod: +x chown:
I, [2023-01-09T01:24:17.647615 #1] INFO -- : File > /etc/service/redis/log/run chmod: +x chown:
I, [2023-01-09T01:24:17.653516 #1] INFO -- : File > /etc/runit/3.d/10-redis chmod: +x chown:
I, [2023-01-09T01:24:17.654485 #1] INFO -- : Replacing daemonize yes with in /etc/redis/redis.conf
I, [2023-01-09T01:24:17.655248 #1] INFO -- : Replacing (?-mix:^pidfile.*$) with in /etc/redis/redis.conf
I, [2023-01-09T01:24:17.656560 #1] INFO -- : > install -d -m 0755 -o redis -g redis /shared/redis_data
I, [2023-01-09T01:24:17.659732 #1] INFO -- :
I, [2023-01-09T01:24:17.660109 #1] INFO -- : Replacing (?-mix:^logfile.*$) with logfile "" in /etc/redis/redis.conf
I, [2023-01-09T01:24:17.660734 #1] INFO -- : Replacing (?-mix:^bind .*$) with in /etc/redis/redis.conf
I, [2023-01-09T01:24:17.661486 #1] INFO -- : Replacing (?-mix:^dir .*$) with dir /shared/redis_data in /etc/redis/redis.conf
I, [2023-01-09T01:24:17.662997 #1] INFO -- : Replacing (?-mix:^protected-mode yes) with protected-mode no in /etc/redis/redis.conf
I, [2023-01-09T01:24:17.663728 #1] INFO -- : Replacing # io-threads 4 with io-threads $redis_io_threads in /etc/redis/redis.conf
I, [2023-01-09T01:24:17.664352 #1] INFO -- : > echo redis installed
I, [2023-01-09T01:24:17.667002 #1] INFO -- : redis installed
I, [2023-01-09T01:24:17.667147 #1] INFO -- : > cat /etc/redis/redis.conf | grep logfile
I, [2023-01-09T01:24:17.673285 #1] INFO -- : logfile ""
I, [2023-01-09T01:24:17.673495 #1] INFO -- : > exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf
I, [2023-01-09T01:24:17.675679 #1] INFO -- : > sleep 10
103:C 09 Jan 2023 01:24:17.694 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
103:C 09 Jan 2023 01:24:17.694 # Redis version=7.0.7, bits=64, commit=00000000, modified=0, pid=103, just started
103:C 09 Jan 2023 01:24:17.694 # Configuration loaded
103:M 09 Jan 2023 01:24:17.694 * monotonic clock: POSIX clock_gettime
103:M 09 Jan 2023 01:24:17.696 * Running mode=standalone, port=6379.
103:M 09 Jan 2023 01:24:17.696 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
103:M 09 Jan 2023 01:24:17.696 # Server initialized
103:M 09 Jan 2023 01:24:17.696 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
103:M 09 Jan 2023 01:24:17.696 # WARNING You have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo madvise > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled (set to 'madvise' or 'never').
103:M 09 Jan 2023 01:24:17.697 * Loading RDB produced by version 6.2.6
103:M 09 Jan 2023 01:24:17.697 * RDB age 24 seconds
103:M 09 Jan 2023 01:24:17.697 * RDB memory usage when created 223.12 Mb
103:M 09 Jan 2023 01:24:18.488 * Done loading RDB, keys loaded: 20712, keys expired: 2.
103:M 09 Jan 2023 01:24:18.488 * DB loaded from disk: 0.792 seconds
103:M 09 Jan 2023 01:24:18.488 * Ready to accept connections
I, [2023-01-09T01:24:27.678833 #1] INFO -- :
I, [2023-01-09T01:24:27.679564 #1] INFO -- : > thpoff echo "thpoff is installed!"
I, [2023-01-09T01:24:27.685101 #1] INFO -- : thpoff is installed!
I, [2023-01-09T01:24:27.685300 #1] INFO -- : > /usr/local/bin/ruby -e 'if ENV["DISCOURSE_SMTP_ADDRESS"] == "smtp.example.com"; puts "Aborting! Mail is not configured!"; exit 1; end'
I, [2023-01-09T01:24:27.821400 #1] INFO -- :
I, [2023-01-09T01:24:27.821650 #1] INFO -- : > /usr/local/bin/ruby -e 'if ENV["DISCOURSE_HOSTNAME"] == "discourse.example.com"; puts "Aborting! Domain is not configured!"; exit 1; end'
I, [2023-01-09T01:24:27.922447 #1] INFO -- :
I, [2023-01-09T01:24:27.922729 #1] INFO -- : > /usr/local/bin/ruby -e 'if (ENV["DISCOURSE_CDN_URL"] || "")[0..1] == "//"; puts "Aborting! CDN must have a protocol specified. Once fixed you should rebake your posts now to correct all posts."; exit 1; end'
I, [2023-01-09T01:24:28.027881 #1] INFO -- :
I, [2023-01-09T01:24:28.028111 #1] INFO -- : > rm -f /etc/cron.d/anacron
I, [2023-01-09T01:24:28.031718 #1] INFO -- :
I, [2023-01-09T01:24:28.035410 #1] INFO -- : File > /etc/cron.d/anacron chmod: chown:
I, [2023-01-09T01:24:28.041535 #1] INFO -- : File > /etc/runit/1.d/copy-env chmod: +x chown:
I, [2023-01-09T01:24:28.047538 #1] INFO -- : File > /etc/service/unicorn/run chmod: +x chown:
I, [2023-01-09T01:24:28.053666 #1] INFO -- : File > /etc/service/nginx/run chmod: +x chown:
I, [2023-01-09T01:24:28.059855 #1] INFO -- : File > /etc/runit/3.d/01-nginx chmod: +x chown:
I, [2023-01-09T01:24:28.065788 #1] INFO -- : File > /etc/runit/3.d/02-unicorn chmod: +x chown:
I, [2023-01-09T01:24:28.065958 #1] INFO -- : Replacing # postgres with sv start postgres || exit 1 in /etc/service/unicorn/run
I, [2023-01-09T01:24:28.066269 #1] INFO -- : > exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf
I, [2023-01-09T01:24:28.069486 #1] INFO -- : > cd /var/www/discourse && git reset --hard
130:C 09 Jan 2023 01:24:28.076 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
130:C 09 Jan 2023 01:24:28.076 # Redis version=7.0.7, bits=64, commit=00000000, modified=0, pid=130, just started
130:C 09 Jan 2023 01:24:28.076 # Configuration loaded
130:M 09 Jan 2023 01:24:28.077 * monotonic clock: POSIX clock_gettime
130:M 09 Jan 2023 01:24:28.077 # Warning: Could not create server TCP listening socket *:6379: bind: Address already in use
130:M 09 Jan 2023 01:24:28.077 # Failed listening on port 6379 (TCP), aborting.
Updating files: 100% (32637/32637), done.
I, [2023-01-09T01:24:30.874219 #1] INFO -- : HEAD is now at e775ed1c DEV: Properly include spec examples (#19532)
I, [2023-01-09T01:24:30.874398 #1] INFO -- : > cd /var/www/discourse && git clean -f
I, [2023-01-09T01:24:31.062146 #1] INFO -- :
I, [2023-01-09T01:24:31.062530 #1] INFO -- : > cd /var/www/discourse && git remote set-branches --add origin main
I, [2023-01-09T01:24:31.069712 #1] INFO -- :
I, [2023-01-09T01:24:31.069909 #1] INFO -- : > cd /var/www/discourse && git remote set-branches origin tests-passed
I, [2023-01-09T01:24:31.076412 #1] INFO -- :
I, [2023-01-09T01:24:31.076586 #1] INFO -- : > cd /var/www/discourse && git fetch --depth 1 origin tests-passed
From https://github.com/discourse/discourse
* branch tests-passed -> FETCH_HEAD
* [new branch] tests-passed -> origin/tests-passed
I, [2023-01-09T01:24:33.068037 #1] INFO -- :
I, [2023-01-09T01:24:33.068288 #1] INFO -- : > cd /var/www/discourse && git checkout tests-passed
Switched to a new branch 'tests-passed'
I, [2023-01-09T01:24:34.269503 #1] INFO -- : Branch 'tests-passed' set up to track remote branch 'tests-passed' from 'origin'.
I, [2023-01-09T01:24:34.269940 #1] INFO -- : > cd /var/www/discourse && mkdir -p tmp
I, [2023-01-09T01:24:34.275253 #1] INFO -- :
I, [2023-01-09T01:24:34.275637 #1] INFO -- : > cd /var/www/discourse && chown discourse:www-data tmp
I, [2023-01-09T01:24:34.280161 #1] INFO -- :
I, [2023-01-09T01:24:34.280520 #1] INFO -- : > cd /var/www/discourse && mkdir -p tmp/pids
I, [2023-01-09T01:24:34.284884 #1] INFO -- :
I, [2023-01-09T01:24:34.285261 #1] INFO -- : > cd /var/www/discourse && mkdir -p tmp/sockets
I, [2023-01-09T01:24:34.289641 #1] INFO -- :
I, [2023-01-09T01:24:34.289776 #1] INFO -- : > cd /var/www/discourse && touch tmp/.gitkeep
I, [2023-01-09T01:24:34.294382 #1] INFO -- :
I, [2023-01-09T01:24:34.294585 #1] INFO -- : > cd /var/www/discourse && mkdir -p /shared/log/rails
I, [2023-01-09T01:24:34.298440 #1] INFO -- :
I, [2023-01-09T01:24:34.298630 #1] INFO -- : > cd /var/www/discourse && bash -c "touch -a /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log"
I, [2023-01-09T01:24:34.305505 #1] INFO -- :
I, [2023-01-09T01:24:34.305635 #1] INFO -- : > cd /var/www/discourse && bash -c "ln -s /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log /var/www/discourse/log"
I, [2023-01-09T01:24:34.311597 #1] INFO -- :
I, [2023-01-09T01:24:34.311732 #1] INFO -- : > cd /var/www/discourse && bash -c "mkdir -p /shared/{uploads,backups}"
I, [2023-01-09T01:24:34.317886 #1] INFO -- :
I, [2023-01-09T01:24:34.318011 #1] INFO -- : > cd /var/www/discourse && bash -c "ln -s /shared/{uploads,backups} /var/www/discourse/public"
I, [2023-01-09T01:24:34.324093 #1] INFO -- :
I, [2023-01-09T01:24:34.324477 #1] INFO -- : > cd /var/www/discourse && bash -c "mkdir -p /shared/tmp/{backups,restores}"
I, [2023-01-09T01:24:34.330970 #1] INFO -- :
I, [2023-01-09T01:24:34.331318 #1] INFO -- : > cd /var/www/discourse && bash -c "ln -s /shared/tmp/{backups,restores} /var/www/discourse/tmp"
I, [2023-01-09T01:24:34.338719 #1] INFO -- :
I, [2023-01-09T01:24:34.338924 #1] INFO -- : > cd /var/www/discourse && chown -R discourse:www-data /shared/log/rails /shared/uploads /shared/backups /shared/tmp
I, [2023-01-09T01:24:35.406006 #1] INFO -- :
I, [2023-01-09T01:24:35.406244 #1] INFO -- : > cd /var/www/discourse && [! -d public/plugins] || find public/plugins/ -maxdepth 1 -xtype l -delete
I, [2023-01-09T01:24:35.415072 #1] INFO -- :
I, [2023-01-09T01:24:35.415458 #1] INFO -- : Replacing # redis with sv start redis || exit 1 in /etc/service/unicorn/run
I, [2023-01-09T01:24:35.416573 #1] INFO -- : > cd /var/www/discourse/plugins && mkdir -p plugins
I, [2023-01-09T01:24:35.422220 #1] INFO -- :
I, [2023-01-09T01:24:35.422355 #1] INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/docker_manager.git
Cloning into 'docker_manager'...
I, [2023-01-09T01:24:36.585720 #1] INFO -- :
I, [2023-01-09T01:24:36.586299 #1] INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/discourse-solved.git
Cloning into 'discourse-solved'...
I, [2023-01-09T01:24:37.349634 #1] INFO -- :
I, [2023-01-09T01:24:37.350162 #1] INFO -- : > cd /var/www/discourse/plugins && sudo -E -u discourse git clone https://github.com/discourse/discourse-canned-replies.git
Cloning into 'discourse-canned-replies'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2023-01-09T01:24:38.160428 #1] INFO -- :
I, [2023-01-09T01:24:38.160771 #1] INFO -- : > cd /var/www/discourse/plugins && sudo -E -u discourse git clone https://github.com/discourse/discourse-saved-searches.git
Cloning into 'discourse-saved-searches'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2023-01-09T01:24:38.831562 #1] INFO -- :
I, [2023-01-09T01:24:38.831772 #1] INFO -- : > cd /var/www/discourse/plugins && sudo -E -u discourse git clone https://github.com/discourse/discourse-cakeday.git
Cloning into 'discourse-cakeday'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2023-01-09T01:24:39.532439 #1] INFO -- :
I, [2023-01-09T01:24:39.532805 #1] INFO -- : > cd /var/www/discourse/plugins && sudo -E -u discourse git clone https://github.com/discourse/discourse-bcc.git
Cloning into 'discourse-bcc'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2023-01-09T01:24:40.154314 #1] INFO -- :
I, [2023-01-09T01:24:40.154550 #1] INFO -- : > cd /var/www/discourse/plugins && sudo -E -u discourse git clone https://github.com/discourse/discourse-footnote.git
Cloning into 'discourse-footnote'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2023-01-09T01:24:40.815415 #1] INFO -- :
I, [2023-01-09T01:24:40.815710 #1] INFO -- : > cd /var/www/discourse/plugins && sudo -E -u discourse git clone https://github.com/discourse/discourse-no-bump.git
Cloning into 'discourse-no-bump'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2023-01-09T01:24:41.457886 #1] INFO -- :
I, [2023-01-09T01:24:41.458104 #1] INFO -- : > cd /var/www/discourse/plugins && sudo -E -u discourse git clone https://github.com/discourse/discourse-user-notes.git
Cloning into 'discourse-user-notes'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2023-01-09T01:24:42.132928 #1] INFO -- :
I, [2023-01-09T01:24:42.133386 #1] INFO -- : > cd /var/www/discourse/plugins && sudo -E -u discourse git clone https://github.com/discourse/discourse-tooltips.git
Cloning into 'discourse-tooltips'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2023-01-09T01:24:42.837714 #1] INFO -- :
I, [2023-01-09T01:24:42.837921 #1] INFO -- : > cd /var/www/discourse/plugins && sudo -E -u discourse git clone https://github.com/discourse/discourse-adplugin.git
Cloning into 'discourse-adplugin'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2023-01-09T01:24:43.578307 #1] INFO -- :
I, [2023-01-09T01:24:43.578602 #1] INFO -- : > cd /var/www/discourse/plugins && sudo -E -u discourse git clone https://github.com/discourse/discourse-yearly-review.git
Cloning into 'discourse-yearly-review'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2023-01-09T01:24:44.295040 #1] INFO -- :
I, [2023-01-09T01:24:44.295608 #1] INFO -- : > cp /var/www/discourse/config/nginx.sample.conf /etc/nginx/conf.d/discourse.conf
I, [2023-01-09T01:24:44.300575 #1] INFO -- :
I, [2023-01-09T01:24:44.300991 #1] INFO -- : > rm /etc/nginx/sites-enabled/default
I, [2023-01-09T01:24:44.304367 #1] INFO -- :
I, [2023-01-09T01:24:44.304686 #1] INFO -- : > mkdir -p /var/nginx/cache
I, [2023-01-09T01:24:44.308212 #1] INFO -- :
I, [2023-01-09T01:24:44.308941 #1] INFO -- : Replacing pid /run/nginx.pid; with daemon off; in /etc/nginx/nginx.conf
I, [2023-01-09T01:24:44.309341 #1] INFO -- : Replacing (?m-ix:upstream[^\}]+\}) with upstream discourse { server 127.0.0.1:3000; } in /etc/nginx/conf.d/discourse.conf
I, [2023-01-09T01:24:44.309679 #1] INFO -- : Replacing (?-mix:server_name.+$) with server_name _ ; in /etc/nginx/conf.d/discourse.conf
I, [2023-01-09T01:24:44.310410 #1] INFO -- : Replacing (?-mix:client_max_body_size.+$) with client_max_body_size $upload_size ; in /etc/nginx/conf.d/discourse.conf
I, [2023-01-09T01:24:44.310844 #1] INFO -- : Replacing (?-mix:worker_connections.+$) with worker_connections $nginx_worker_connections ; in /etc/nginx/nginx.conf
I, [2023-01-09T01:24:44.311086 #1] INFO -- : > echo "done configuring web"
I, [2023-01-09T01:24:44.313609 #1] INFO -- : done configuring web
I, [2023-01-09T01:24:44.313969 #1] INFO -- : > cd /var/www/discourse && gem install bundler --conservative -v $(awk '/BUNDLED WITH/ { getline; gsub(/ /,""); print $0 }' Gemfile.lock)
ERROR: Loading command: install (RuntimeError)
failed to get urandom
/usr/local/lib/ruby/3.1.0/securerandom.rb:75:in `urandom'
/usr/local/lib/ruby/3.1.0/securerandom.rb:75:in `singleton class'
/usr/local/lib/ruby/3.1.0/securerandom.rb:42:in `'
/usr/local/lib/ruby/3.1.0/securerandom.rb:41:in `'
:85:in `require'
:85:in `require'
/usr/local/lib/ruby/3.1.0/resolv.rb:8:in `'
:85:in `require'
:85:in `require'
/usr/local/lib/ruby/3.1.0/net/http.rb:25:in `'
:85:in `require'
:85:in `require'
/usr/local/lib/ruby/3.1.0/rubygems/request.rb:2:in `'
/usr/local/lib/ruby/3.1.0/rubygems/remote_fetcher.rb:3:in `require_relative'
/usr/local/lib/ruby/3.1.0/rubygems/remote_fetcher.rb:3:in `'
/usr/local/lib/ruby/3.1.0/rubygems/spec_fetcher.rb:2:in `require_relative'
/usr/local/lib/ruby/3.1.0/rubygems/spec_fetcher.rb:2:in `'
/usr/local/lib/ruby/3.1.0/rubygems/dependency_installer.rb:6:in `require_relative'
/usr/local/lib/ruby/3.1.0/rubygems/dependency_installer.rb:6:in `'
/usr/local/lib/ruby/3.1.0/rubygems/commands/install_command.rb:4:in `require_relative'
/usr/local/lib/ruby/3.1.0/rubygems/commands/install_command.rb:4:in `'
:85:in `require'
:85:in `require'
/usr/local/lib/ruby/3.1.0/rubygems/command_manager.rb:228:in `load_and_instantiate'
/usr/local/lib/ruby/3.1.0/rubygems/command_manager.rb:135:in `[]'
/usr/local/lib/ruby/3.1.0/rubygems/command_manager.rb:201:in `find_command'
/usr/local/lib/ruby/3.1.0/rubygems/command_manager.rb:183:in `process_args'
/usr/local/lib/ruby/3.1.0/rubygems/command_manager.rb:149:in `run'
/usr/local/lib/ruby/3.1.0/rubygems/gem_runner.rb:51:in `run'
/usr/local/bin/gem:21:in `'
ERROR: While executing gem ... (NoMethodError)
undefined method `deprecated?' for nil:NilClass
cmd.deprecation_warning if cmd.deprecated?
^^^^^^^^^^^^
/usr/local/lib/ruby/3.1.0/rubygems/command_manager.rb:184:in `process_args'
/usr/local/lib/ruby/3.1.0/rubygems/command_manager.rb:149:in `run'
/usr/local/lib/ruby/3.1.0/rubygems/gem_runner.rb:51:in `run'
/usr/local/bin/gem:21:in `'
I, [2023-01-09T01:24:44.603237 #1] INFO -- :
I, [2023-01-09T01:24:44.603710 #1] INFO -- : Terminating async processes
I, [2023-01-09T01:24:44.603794 #1] INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main pid: 42
I, [2023-01-09T01:24:44.603862 #1] INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 103
2023-01-09 01:24:44.603 UTC [42] LOG: received fast shutdown request
103:signal-handler (1673227484) Received SIGTERM scheduling shutdown...
2023-01-09 01:24:44.608 UTC [42] LOG: aborting any active transactions
2023-01-09 01:24:44.610 UTC [42] LOG: background worker "logical replication launcher" (PID 51) exited with exit code 1
2023-01-09 01:24:44.613 UTC [46] LOG: shutting down
103:M 09 Jan 2023 01:24:44.669 # User requested shutdown...
103:M 09 Jan 2023 01:24:44.669 * Saving the final RDB snapshot before exiting.
2023-01-09 01:24:44.697 UTC [42] LOG: database system is shut down
103:M 09 Jan 2023 01:24:45.821 * DB saved on disk
103:M 09 Jan 2023 01:24:45.821 # Redis is now ready to exit, bye bye...
FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && gem install bundler --conservative -v $(awk '/BUNDLED WITH/ { getline; gsub(/ /,""); print $0 }' Gemfile.lock) failed with return #
Location of failure: /usr/local/lib/ruby/gems/3.1.0/gems/pups-1.1.1/lib/pups/exec_command.rb:117:in `spawn'
exec failed with the params {"cd"=>"$home", "hook"=>"web", "cmd"=>["gem install bundler --conservative -v $(awk '/BUNDLED WITH/ { getline; gsub(/ /,\"\"); print $0 }' Gemfile.lock)", "find $home ! -user discourse -exec chown discourse {} \\+"]}
bootstrap failed with exit code 1
**FAILED TO BOOTSTRAP** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.```
````
---
### Author:  [@hiddenseal](https://meta.discourse.org/u/hiddenseal)
#### Post date: [9 januari 2023 om 02:07 UTC](https://meta.discourse.org/t/help-discourse-update-broke-app-fails-to-rebuild/251043/14 "2023-01-09T02:07:52Z")
Did you try to search for ways to resolve
```plaintext
unable to access '/root/.config/git/attributes': Permission denied
```
? There are some posts on Stackoverflow related to this.
> [@littleviolette](#):
>
> Anyone would be able to help?
I understand what you feel. While you wait for someone to reply to your #Marketplace request, maybe you can try to create a new droplet, install Docker, install Discourse & restore it from backup (the discourse backup, not the whole server image)? That wouldnât solve your issue _per se_ and will not give you an understanding of why things broke, but at least that _may_ give you a working instance sooner that youâll get help. Iâm suggesting this as I was initially hesitant and was genuinely surprised at how easy it is to migrate.
---
### Author:  [@pacharanero](https://meta.discourse.org/u/pacharanero)
#### Post date: [9 januari 2023 om 13:12 UTC](https://meta.discourse.org/t/help-discourse-update-broke-app-fails-to-rebuild/251043/15 "2023-01-09T13:12:16Z")
I would agree, definitely worth trying a full reinstall from a backup. You desperately need to upgrade that server to at least 22.04LTS, it is significantly out of date and running a kernel 2 major versions behind current. Itâs likely to have numerous incompatibilities with newer versions of Docker, Redis, Postgres possibly even., as well as being likely to have known security vulnerabilities.
A new Droplet would be the ideal way, this will of course change your IP address so you would need to be able to make changes to DNS settings to wire this up to your domain name.
If you donât have external backups on AWS S3 then they will be on your existing droplet in the directory `/var/discourse/shared/standalone/backups/default`. SFTP can be used to get the backup off your server and onto the new one.
---
### Author:  [@littleviolette](https://meta.discourse.org/u/littleviolette)
#### Post date: [10 januari 2023 om 17:43 UTC](https://meta.discourse.org/t/help-discourse-update-broke-app-fails-to-rebuild/251043/16 "2023-01-10T17:43:53Z")
Update: We fixed the issue by copying the Discourse instance to a new droplet with @pfaffmanâs help. Thank you all for your advice!
---
### Author:  [@system](https://meta.discourse.org/u/system)
#### Post date: [9 februari 2023 om 17:44 UTC](https://meta.discourse.org/t/help-discourse-update-broke-app-fails-to-rebuild/251043/17 "2023-02-09T17:44:24Z")
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.