Failed to bootstrap on first install

I followed the tutorial at 超级详细教学安装discourse+控制面板 to deploy Discourse on a cloud server. I installed Docker and started the container as instructed. However, after completing the configuration, during the app build phase (where my logs are interrupted), the SSH connection to the server becomes unresponsive. The server status shows 100% disk read, and it freezes. I can only restart it from the control panel. Does anyone know how to resolve this?
I will post the logs in the reply below.

1 Like

logs of ./launcher

x86_64 arch detected.
2.0.20231218-0429: Pulling from discourse/base
Digest: sha256:468f70b9bb4c6d0c6c2bbb3efc1a5e12d145eae57bdb6946b7fe5558beb52dc1
Status: Image is up to date for discourse/base:2.0.20231218-0429
docker.io/discourse/base:2.0.20231218-0429
/usr/local/lib/ruby/gems/3.2.0/gems/pups-1.2.1/lib/pups.rb
/usr/local/bin/pups --stdin
I, [2024-02-15T10:27:53.783742 #1]  INFO -- : Reading from stdin
I, [2024-02-15T10:27:53.788732 #1]  INFO -- : > locale-gen $LANG && update-locale
I, [2024-02-15T10:27:53.819777 #1]  INFO -- : Generating locales (this might take a while)...
Generation complete.
I, [2024-02-15T10:27:53.819971 #1]  INFO -- : > mkdir -p /shared/postgres_run
I, [2024-02-15T10:27:53.822342 #1]  INFO -- : 
I, [2024-02-15T10:27:53.822640 #1]  INFO -- : > chown postgres:postgres /shared/postgres_run
I, [2024-02-15T10:27:53.824680 #1]  INFO -- : 
I, [2024-02-15T10:27:53.824924 #1]  INFO -- : > chmod 775 /shared/postgres_run
I, [2024-02-15T10:27:53.826606 #1]  INFO -- : 
I, [2024-02-15T10:27:53.826773 #1]  INFO -- : > rm -fr /var/run/postgresql
I, [2024-02-15T10:27:53.828655 #1]  INFO -- : 
I, [2024-02-15T10:27:53.828815 #1]  INFO -- : > ln -s /shared/postgres_run /var/run/postgresql
I, [2024-02-15T10:27:53.831027 #1]  INFO -- : 
I, [2024-02-15T10:27:53.831176 #1]  INFO -- : > socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1
2024/02/15 10:27:53 socat[19] E connect(6, AF=1 "/shared/postgres_run/.s.PGSQL.5432", 36): No such file or directory
I, [2024-02-15T10:27:53.838657 #1]  INFO -- : 
I, [2024-02-15T10:27:53.838799 #1]  INFO -- : > rm -fr /shared/postgres_run/.s*
I, [2024-02-15T10:27:53.841286 #1]  INFO -- : 
I, [2024-02-15T10:27:53.841411 #1]  INFO -- : > rm -fr /shared/postgres_run/*.pid
I, [2024-02-15T10:27:53.843857 #1]  INFO -- : 
I, [2024-02-15T10:27:53.844057 #1]  INFO -- : > mkdir -p /shared/postgres_run/13-main.pg_stat_tmp
I, [2024-02-15T10:27:53.845883 #1]  INFO -- : 
I, [2024-02-15T10:27:53.846049 #1]  INFO -- : > chown postgres:postgres /shared/postgres_run/13-main.pg_stat_tmp
I, [2024-02-15T10:27:53.847952 #1]  INFO -- : 
I, [2024-02-15T10:27:53.852041 #1]  INFO -- : File > /etc/service/postgres/run  chmod: +x  chown: 
I, [2024-02-15T10:27:53.855952 #1]  INFO -- : File > /etc/service/postgres/log/run  chmod: +x  chown: 
I, [2024-02-15T10:27:53.859781 #1]  INFO -- : File > /etc/runit/3.d/99-postgres  chmod: +x  chown: 
I, [2024-02-15T10:27:53.863808 #1]  INFO -- : File > /root/upgrade_postgres  chmod: +x  chown: 
I, [2024-02-15T10:27:53.864070 #1]  INFO -- : > chown -R root /var/lib/postgresql/13/main
I, [2024-02-15T10:27:55.115302 #1]  INFO -- : 
I, [2024-02-15T10:27:55.115451 #1]  INFO -- : > [ ! -e /shared/postgres_data ] && install -d -m 0755 -o postgres -g postgres /shared/postgres_data && sudo -E -u postgres /usr/lib/postgresql/13/bin/initdb -D /shared/postgres_data || exit 0
I, [2024-02-15T10:27:55.117522 #1]  INFO -- : 
I, [2024-02-15T10:27:55.117656 #1]  INFO -- : > chown -R postgres:postgres /shared/postgres_data
I, [2024-02-15T10:27:55.128602 #1]  INFO -- : 
I, [2024-02-15T10:27:55.128937 #1]  INFO -- : > chown -R postgres:postgres /var/run/postgresql
I, [2024-02-15T10:27:55.131289 #1]  INFO -- : 
I, [2024-02-15T10:27:55.131550 #1]  INFO -- : > /root/upgrade_postgres
I, [2024-02-15T10:27:55.136105 #1]  INFO -- : 
I, [2024-02-15T10:27:55.136339 #1]  INFO -- : > rm /root/upgrade_postgres
I, [2024-02-15T10:27:55.138143 #1]  INFO -- : 
I, [2024-02-15T10:27:55.138557 #1]  INFO -- : Replacing data_directory = '/var/lib/postgresql/13/main' with data_directory = '/shared/postgres_data' in /etc/postgresql/13/main/postgresql.conf
I, [2024-02-15T10:27:55.139064 #1]  INFO -- : Replacing (?-mix:#?listen_addresses *=.*) with listen_addresses = '*' in /etc/postgresql/13/main/postgresql.conf
I, [2024-02-15T10:27:55.142593 #1]  INFO -- : Replacing (?-mix:#?synchronous_commit *=.*) with synchronous_commit = $db_synchronous_commit in /etc/postgresql/13/main/postgresql.conf
I, [2024-02-15T10:27:55.143215 #1]  INFO -- : Replacing (?-mix:#?shared_buffers *=.*) with shared_buffers = $db_shared_buffers in /etc/postgresql/13/main/postgresql.conf
I, [2024-02-15T10:27:55.143821 #1]  INFO -- : Replacing (?-mix:#?work_mem *=.*) with work_mem = $db_work_mem in /etc/postgresql/13/main/postgresql.conf
I, [2024-02-15T10:27:55.144347 #1]  INFO -- : Replacing (?-mix:#?default_text_search_config *=.*) with default_text_search_config = '$db_default_text_search_config' in /etc/postgresql/13/main/postgresql.conf
I, [2024-02-15T10:27:55.144822 #1]  INFO -- : > install -d -m 0755 -o postgres -g postgres /shared/postgres_backup
I, [2024-02-15T10:27:55.149060 #1]  INFO -- : 
I, [2024-02-15T10:27:55.149426 #1]  INFO -- : Replacing (?-mix:#?checkpoint_segments *=.*) with checkpoint_segments = $db_checkpoint_segments in /etc/postgresql/13/main/postgresql.conf
I, [2024-02-15T10:27:55.149729 #1]  INFO -- : Replacing (?-mix:#?logging_collector *=.*) with logging_collector = $db_logging_collector in /etc/postgresql/13/main/postgresql.conf
I, [2024-02-15T10:27:55.150305 #1]  INFO -- : Replacing (?-mix:#?log_min_duration_statement *=.*) with log_min_duration_statement = $db_log_min_duration_statement in /etc/postgresql/13/main/postgresql.conf
I, [2024-02-15T10:27:55.150833 #1]  INFO -- : Replacing (?-mix:^#local +replication +postgres +peer$) with local replication postgres  peer in /etc/postgresql/13/main/pg_hba.conf
I, [2024-02-15T10:27:55.151186 #1]  INFO -- : Replacing (?-mix:^host.*all.*all.*127.*$) with host all all 0.0.0.0/0 md5 in /etc/postgresql/13/main/pg_hba.conf
I, [2024-02-15T10:27:55.151662 #1]  INFO -- : Replacing (?-mix:^host.*all.*all.*::1\/128.*$) with host all all ::/0 md5 in /etc/postgresql/13/main/pg_hba.conf
I, [2024-02-15T10:27:55.152088 #1]  INFO -- : > HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main
I, [2024-02-15T10:27:55.155843 #1]  INFO -- : > sleep 5
2024-02-15 10:27:55.190 UTC [42] LOG:  starting PostgreSQL 13.13 (Debian 13.13-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2024-02-15 10:27:55.190 UTC [42] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2024-02-15 10:27:55.191 UTC [42] LOG:  listening on IPv6 address "::", port 5432
2024-02-15 10:27:55.194 UTC [42] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2024-02-15 10:27:55.200 UTC [45] LOG:  database system was shut down at 2024-02-15 10:26:58 UTC
2024-02-15 10:27:55.204 UTC [42] LOG:  database system is ready to accept connections
I, [2024-02-15T10:28:00.163378 #1]  INFO -- : 
I, [2024-02-15T10:28:00.163700 #1]  INFO -- : > su postgres -c 'createdb discourse' || true
2024-02-15 10:28:00.223 UTC [55] postgres@postgres ERROR:  database "discourse" already exists
2024-02-15 10:28:00.223 UTC [55] postgres@postgres STATEMENT:  CREATE DATABASE discourse;
createdb: error: database creation failed: ERROR:  database "discourse" already exists
I, [2024-02-15T10:28:00.224612 #1]  INFO -- : 
I, [2024-02-15T10:28:00.225093 #1]  INFO -- : > su postgres -c 'psql discourse -c "create user discourse;"' || true
2024-02-15 10:28:00.273 UTC [59] postgres@discourse ERROR:  role "discourse" already exists
2024-02-15 10:28:00.273 UTC [59] postgres@discourse STATEMENT:  create user discourse;
ERROR:  role "discourse" already exists
I, [2024-02-15T10:28:00.274926 #1]  INFO -- : 
I, [2024-02-15T10:28:00.275392 #1]  INFO -- : > su postgres -c 'psql discourse -c "grant all privileges on database discourse to discourse;"' || true
I, [2024-02-15T10:28:00.321218 #1]  INFO -- : GRANT
I, [2024-02-15T10:28:00.321681 #1]  INFO -- : > su postgres -c 'psql discourse -c "alter schema public owner to discourse;"'
I, [2024-02-15T10:28:00.368758 #1]  INFO -- : ALTER SCHEMA
I, [2024-02-15T10:28:00.369222 #1]  INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists hstore;"'
NOTICE:  extension "hstore" already exists, skipping
I, [2024-02-15T10:28:00.419312 #1]  INFO -- : CREATE EXTENSION
I, [2024-02-15T10:28:00.419764 #1]  INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists pg_trgm;"'
NOTICE:  extension "pg_trgm" already exists, skipping
I, [2024-02-15T10:28:00.467914 #1]  INFO -- : CREATE EXTENSION
I, [2024-02-15T10:28:00.468449 #1]  INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists vector;"'
NOTICE:  extension "vector" already exists, skipping
I, [2024-02-15T10:28:00.517577 #1]  INFO -- : CREATE EXTENSION
I, [2024-02-15T10:28:00.518046 #1]  INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists hstore;"'
NOTICE:  extension "hstore" already exists, skipping
I, [2024-02-15T10:28:00.563004 #1]  INFO -- : CREATE EXTENSION
I, [2024-02-15T10:28:00.563470 #1]  INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists pg_trgm;"'
NOTICE:  extension "pg_trgm" already exists, skipping
I, [2024-02-15T10:28:00.618027 #1]  INFO -- : CREATE EXTENSION
I, [2024-02-15T10:28:00.618552 #1]  INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists vector;"'
NOTICE:  extension "vector" already exists, skipping
I, [2024-02-15T10:28:00.669078 #1]  INFO -- : CREATE EXTENSION
I, [2024-02-15T10:28:00.670057 #1]  INFO -- : > sudo -u postgres psql discourse
I, [2024-02-15T10:28:00.672371 #1]  INFO -- : update pg_database set encoding = pg_char_to_encoding('UTF8') where datname = 'discourse' AND encoding = pg_char_to_encoding('SQL_ASCII');
I, [2024-02-15T10:28:00.730978 #1]  INFO -- : File > /var/lib/postgresql/take-database-backup  chmod: +x  chown: postgres:postgres
I, [2024-02-15T10:28:00.733740 #1]  INFO -- : File > /var/spool/cron/crontabs/postgres  chmod:   chown: 
I, [2024-02-15T10:28:00.733836 #1]  INFO -- : > echo postgres installed!
I, [2024-02-15T10:28:00.735684 #1]  INFO -- : postgres installed!
I, [2024-02-15T10:28:00.740680 #1]  INFO -- : File > /etc/service/redis/run  chmod: +x  chown: 
I, [2024-02-15T10:28:00.745130 #1]  INFO -- : File > /etc/service/redis/log/run  chmod: +x  chown: 
I, [2024-02-15T10:28:00.749422 #1]  INFO -- : File > /etc/runit/3.d/10-redis  chmod: +x  chown: 
I, [2024-02-15T10:28:00.749951 #1]  INFO -- : Replacing daemonize yes with  in /etc/redis/redis.conf
I, [2024-02-15T10:28:00.750813 #1]  INFO -- : Replacing (?-mix:^pidfile.*$) with  in /etc/redis/redis.conf
I, [2024-02-15T10:28:00.751445 #1]  INFO -- : > install -d -m 0755 -o redis -g redis /shared/redis_data
I, [2024-02-15T10:28:00.756305 #1]  INFO -- : 
I, [2024-02-15T10:28:00.756731 #1]  INFO -- : Replacing (?-mix:^logfile.*$) with logfile "" in /etc/redis/redis.conf
I, [2024-02-15T10:28:00.757237 #1]  INFO -- : Replacing (?-mix:^bind .*$) with  in /etc/redis/redis.conf
I, [2024-02-15T10:28:00.758178 #1]  INFO -- : Replacing (?-mix:^dir .*$) with dir /shared/redis_data in /etc/redis/redis.conf
I, [2024-02-15T10:28:00.758990 #1]  INFO -- : Replacing (?-mix:^protected-mode yes) with protected-mode no in /etc/redis/redis.conf
I, [2024-02-15T10:28:00.759791 #1]  INFO -- : Replacing # io-threads 4 with io-threads $redis_io_threads in /etc/redis/redis.conf
I, [2024-02-15T10:28:00.760914 #1]  INFO -- : > echo redis installed
I, [2024-02-15T10:28:00.763943 #1]  INFO -- : redis installed
I, [2024-02-15T10:28:00.764240 #1]  INFO -- : > cat /etc/redis/redis.conf | grep logfile
I, [2024-02-15T10:28:00.769404 #1]  INFO -- : logfile ""
I, [2024-02-15T10:28:00.769851 #1]  INFO -- : > exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf
I, [2024-02-15T10:28:00.771248 #1]  INFO -- : > sleep 10
111:C 15 Feb 2024 10:28:00.779 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
111:C 15 Feb 2024 10:28:00.779 # Redis version=7.0.7, bits=64, commit=00000000, modified=0, pid=111, just started
111:C 15 Feb 2024 10:28:00.779 # Configuration loaded
111:M 15 Feb 2024 10:28:00.780 * monotonic clock: POSIX clock_gettime
111:M 15 Feb 2024 10:28:00.781 * Running mode=standalone, port=6379.
111:M 15 Feb 2024 10:28:00.781 # Server initialized
111:M 15 Feb 2024 10:28:00.782 * Loading RDB produced by version 7.0.7
111:M 15 Feb 2024 10:28:00.782 * RDB age 62 seconds
111:M 15 Feb 2024 10:28:00.782 * RDB memory usage when created 0.98 Mb
111:M 15 Feb 2024 10:28:00.782 * Done loading RDB, keys loaded: 34, keys expired: 0.
111:M 15 Feb 2024 10:28:00.782 * DB loaded from disk: 0.001 seconds
111:M 15 Feb 2024 10:28:00.782 * Ready to accept connections
I, [2024-02-15T10:28:10.773983 #1]  INFO -- : 
I, [2024-02-15T10:28:10.774312 #1]  INFO -- : > thpoff echo "thpoff is installed!"
I, [2024-02-15T10:28:10.777770 #1]  INFO -- : thpoff is installed!
I, [2024-02-15T10:28:10.777992 #1]  INFO -- : > /usr/local/bin/ruby -e 'if ENV["DISCOURSE_SMTP_ADDRESS"] == "smtp.example.com"; puts "Aborting! Mail is not configured!"; exit 1; end'
I, [2024-02-15T10:28:10.841325 #1]  INFO -- : 
I, [2024-02-15T10:28:10.841704 #1]  INFO -- : > /usr/local/bin/ruby -e 'if ENV["DISCOURSE_HOSTNAME"] == "discourse.example.com"; puts "Aborting! Domain is not configured!"; exit 1; end'
I, [2024-02-15T10:28:10.912056 #1]  INFO -- : 
I, [2024-02-15T10:28:10.912495 #1]  INFO -- : > /usr/local/bin/ruby -e 'if (ENV["DISCOURSE_CDN_URL"] || "")[0..1] == "//"; puts "Aborting! CDN must have a protocol specified. Once fixed you should rebake your posts now to correct all posts."; exit 1; end'
I, [2024-02-15T10:28:10.975717 #1]  INFO -- : 
I, [2024-02-15T10:28:10.976065 #1]  INFO -- : > rm -f /etc/cron.d/anacron
I, [2024-02-15T10:28:10.980294 #1]  INFO -- : 
I, [2024-02-15T10:28:10.983044 #1]  INFO -- : File > /etc/cron.d/anacron  chmod:   chown: 
I, [2024-02-15T10:28:10.987324 #1]  INFO -- : File > /etc/runit/1.d/copy-env  chmod: +x  chown: 
I, [2024-02-15T10:28:10.991444 #1]  INFO -- : File > /etc/service/unicorn/run  chmod: +x  chown: 
I, [2024-02-15T10:28:10.995677 #1]  INFO -- : File > /etc/service/nginx/run  chmod: +x  chown: 
I, [2024-02-15T10:28:10.999661 #1]  INFO -- : File > /etc/runit/3.d/01-nginx  chmod: +x  chown: 
I, [2024-02-15T10:28:11.003463 #1]  INFO -- : File > /etc/runit/3.d/02-unicorn  chmod: +x  chown: 
I, [2024-02-15T10:28:11.003703 #1]  INFO -- : Replacing # postgres with sv start postgres || exit 1 in /etc/service/unicorn/run
I, [2024-02-15T10:28:11.004008 #1]  INFO -- : > exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf
I, [2024-02-15T10:28:11.005833 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git reset --hard
138:C 15 Feb 2024 10:28:11.012 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
138:C 15 Feb 2024 10:28:11.012 # Redis version=7.0.7, bits=64, commit=00000000, modified=0, pid=138, just started
138:C 15 Feb 2024 10:28:11.012 # Configuration loaded
138:M 15 Feb 2024 10:28:11.012 * monotonic clock: POSIX clock_gettime
138:M 15 Feb 2024 10:28:11.014 # Warning: Could not create server TCP listening socket *:6379: bind: Address already in use
138:M 15 Feb 2024 10:28:11.014 # Failed listening on port 6379 (TCP), aborting.
Updating files:  36% (12265/33715)
Updating files:  37% (12475/33715)
Updating files:  38% (12812/33715)
Updating files:  39% (13149/33715)
Updating files:  40% (13486/33715)
Updating files:  41% (13824/33715)
Updating files:  42% (14161/33715)
Updating files:  43% (14498/33715)
Updating files:  44% (14835/33715)
Updating files:  45% (15172/33715)
Updating files:  46% (15509/33715)
Updating files:  47% (15847/33715)
Updating files:  48% (16184/33715)
Updating files:  49% (16521/33715)
Updating files:  50% (16858/33715)
Updating files:  51% (17195/33715)
Updating files:  52% (17532/33715)
Updating files:  53% (17869/33715)
Updating files:  54% (18207/33715)
Updating files:  55% (18544/33715)
Updating files:  56% (18881/33715)
Updating files:  57% (19218/33715)
Updating files:  58% (19555/33715)
Updating files:  59% (19892/33715)
Updating files:  60% (20229/33715)
Updating files:  61% (20567/33715)
Updating files:  62% (20904/33715)
Updating files:  63% (21241/33715)
Updating files:  64% (21578/33715)
Updating files:  65% (21915/33715)
Updating files:  66% (22252/33715)
Updating files:  67% (22590/33715)
Updating files:  68% (22927/33715)
Updating files:  69% (23264/33715)
Updating files:  70% (23601/33715)
Updating files:  71% (23938/33715)
Updating files:  72% (24275/33715)
Updating files:  73% (24612/33715)
Updating files:  74% (24950/33715)
Updating files:  75% (25287/33715)
Updating files:  76% (25624/33715)
Updating files:  77% (25961/33715)
Updating files:  78% (26298/33715)
Updating files:  79% (26635/33715)
Updating files:  80% (26972/33715)
Updating files:  81% (27310/33715)
Updating files:  82% (27647/33715)
Updating files:  82% (27668/33715)
Updating files:  83% (27984/33715)
Updating files:  84% (28321/33715)
Updating files:  85% (28658/33715)
Updating files:  86% (28995/33715)
Updating files:  87% (29333/33715)
Updating files:  88% (29670/33715)
Updating files:  89% (30007/33715)
Updating files:  90% (30344/33715)
Updating files:  91% (30681/33715)
Updating files:  92% (31018/33715)
Updating files:  93% (31355/33715)
Updating files:  94% (31693/33715)
Updating files:  95% (32030/33715)
Updating files:  96% (32367/33715)
Updating files:  97% (32704/33715)
Updating files:  98% (33041/33715)
Updating files:  99% (33378/33715)
Updating files: 100% (33715/33715)
Updating files: 100% (33715/33715), done.
I, [2024-02-15T10:28:13.867424 #1]  INFO -- : HEAD is now at 6ab1a19e9 DEV: Convert min_trust_level_to_allow_invite to groups (#24893)
I, [2024-02-15T10:28:13.870245 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git clean -f
I, [2024-02-15T10:28:14.033356 #1]  INFO -- : 
I, [2024-02-15T10:28:14.033496 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse bash -c '
  set -o errexit
  if [ $(git rev-parse --is-shallow-repository) == "true" ]; then
      git remote set-branches --add origin main
      git remote set-branches origin tests-passed
      git fetch --depth 1 origin tests-passed
  else
      git fetch --tags --prune-tags --prune --force origin
  fi
'
From https://github.com/discourse/discourse
 - [deleted]             (none)     -> origin/0-drop-code
 - [deleted]             (none)     -> origin/bulk-button-class
 - [deleted]             (none)     -> origin/dependabot/bundler/excon-0.108.0
 - [deleted]             (none)     -> origin/dependabot/bundler/hashdiff-1.1.0
 - [deleted]             (none)     -> origin/dependabot/bundler/parallel-1.24.0
 - [deleted]             (none)     -> origin/dependabot/bundler/rubocop-discourse-3.6.0
 - [deleted]             (none)     -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/drop-target-2.0.2
 - [deleted]             (none)     -> origin/dev/add-new-chat-metrics
 - [deleted]             (none)     -> origin/dev/allow-chat-mentions-to-have-several-notifications
 - [deleted]             (none)     -> origin/dev/convert-min-trust-level-to-create-tag-to-group
 - [deleted]             (none)     -> origin/dev/convert-min-trust-level-to-ignore-to-groups
 - [deleted]             (none)     -> origin/dev/rethink-mass-mentions-polymorphic
 - [deleted]             (none)     -> origin/feature/migrations-schema-generator
 - [deleted]             (none)     -> origin/fix-join-filter-upload
 - [deleted]             (none)     -> origin/fix-modal-flash-minor
 - [deleted]             (none)     -> origin/global-notice-reactive
 - [deleted]             (none)     -> origin/labeler
 - [deleted]             (none)     -> origin/low-ram-help
 - [deleted]             (none)     -> origin/multi-invite
 - [deleted]             (none)     -> origin/plugin-extra-js
 - [deleted]             (none)     -> origin/table-builder-modal-modifier
 - [deleted]             (none)     -> origin/tests-passed-broken
 - [deleted]             (none)     -> origin/wip_categories
 - [deleted]             (none)     -> origin/wizard-split
   6ab1a19e9..c30aeafd9  main                    -> origin/main
 * [new branch]          0-ember-bump            -> origin/0-ember-bump
 * [new branch]          0-event-helper          -> origin/0-event-helper
 * [new branch]          0-extend-native         -> origin/0-extend-native
 * [new branch]          0-fix-perf-watched-words -> origin/0-fix-perf-watched-words
 * [new branch]          0-rev                   -> origin/0-rev
 * [new branch]          0-screen-track          -> origin/0-screen-track
 * [new branch]          add-api-for-forced-auth -> origin/add-api-for-forced-auth
 * [new branch]          asciinema-iframe        -> origin/asciinema-iframe
 * [new branch]          async-category-search   -> origin/async-category-search
 * [new branch]          bulk-select-dropdown-items -> origin/bulk-select-dropdown-items
 * [new branch]          convert-header-to-glimmer -> origin/convert-header-to-glimmer
 * [new branch]          csp-strict-dynamic      -> origin/csp-strict-dynamic
 * [new branch]          decorator-transforms    -> origin/decorator-transforms
 * [new branch]          dependabot/bundler/omniauth-2.0.0 -> origin/dependabot/bundler/omniauth-2.0.0
 * [new branch]          dependabot/bundler/parallel_tests-4.4.0 -> origin/dependabot/bundler/parallel_tests-4.4.0
 * [new branch]          dependabot/bundler/rack-mini-profiler-3.3.1 -> origin/dependabot/bundler/rack-mini-profiler-3.3.1
 * [new branch]          dev-convert-topic-map-expanded-widget-to-component -> origin/dev-convert-topic-map-expanded-widget-to-component
 * [new branch]          dev/add-modifier-user-action-stream-builder -> origin/dev/add-modifier-user-action-stream-builder
 * [new branch]          dev/check-whether-online-user-ids-may-be-memoized-between-job-runs -> origin/dev/check-whether-online-user-ids-may-be-memoized-between-job-runs
 + 79ea38776...bd0faffec dev/drop-notification_id-column-on-chat-mentions -> origin/dev/drop-notification_id-column-on-chat-mentions  (forced update)
 * [new branch]          dev/drop-the-user-id-column-from-chat-mentions -> origin/dev/drop-the-user-id-column-from-chat-mentions
 + e0c0191bb...6751ac529 dev/dry-up-user-status-serializers -> origin/dev/dry-up-user-status-serializers  (forced update)
 * [new branch]          dev/experiment-theme-settings-free-ui -> origin/dev/experiment-theme-settings-free-ui
 * [new branch]          dev/groundwork-theme-settings-editor -> origin/dev/groundwork-theme-settings-editor
 * [new branch]          dev/make-admins-tl4-in-tests -> origin/dev/make-admins-tl4-in-tests
 * [new branch]          dev/promote-plugin-post-deploy-migrations -> origin/dev/promote-plugin-post-deploy-migrations
 * [new branch]          dev/proof-of-concept-rewriting-mentions-notification-logic -> origin/dev/proof-of-concept-rewriting-mentions-notification-logic
 * [new branch]          dev_category_order_database -> origin/dev_category_order_database
 * [new branch]          dev_update_category_position -> origin/dev_update_category_position
 * [new branch]          discover_enrollment     -> origin/discover_enrollment
 * [new branch]          extra-js-revert         -> origin/extra-js-revert
 * [new branch]          fast-edit-expand-regex  -> origin/fast-edit-expand-regex
   8780daeee..d92b6b19b  feature/bookmark-menu-and-modal-redesign -> origin/feature/bookmark-menu-and-modal-redesign
 * [new branch]          feature/custom-date-range -> origin/feature/custom-date-range
 * [new branch]          feature/paste-upload-on-first-upload-input-templates -> origin/feature/paste-upload-on-first-upload-input-templates
 * [new branch]          feature_async_load_colors -> origin/feature_async_load_colors
 * [new branch]          feature_track_mentions  -> origin/feature_track_mentions
 * [new branch]          fix_category_chooser    -> origin/fix_category_chooser
 * [new branch]          fix_in_any_groups       -> origin/fix_in_any_groups
 * [new branch]          fix_reviewable_translation -> origin/fix_reviewable_translation
   7f2bdca65..4db506cea  generic-import-latest   -> origin/generic-import-latest
 * [new branch]          hijacks                 -> origin/hijacks
 * [new branch]          issue/respect-homepage-admin-sidebar-back -> origin/issue/respect-homepage-admin-sidebar-back
 * [new branch]          issue/run-admin-sidebar-init-after-logs-notice -> origin/issue/run-admin-sidebar-init-after-logs-notice
 * [new branch]          loic-rubocop-plugin-rules -> origin/loic-rubocop-plugin-rules
 + 90a3535b7...54d0135bc migrations-convert      -> origin/migrations-convert  (forced update)
 * [new branch]          promote-migration-tweak -> origin/promote-migration-tweak
 * [new branch]          redesign-my-threads-index -> origin/redesign-my-threads-index
 * [new branch]          revert-25009-dev/convert-min-trust-to-allow-self-wiki-to-groups -> origin/revert-25009-dev/convert-min-trust-to-allow-self-wiki-to-groups
 * [new branch]          revert-bulk-topic-actions -> origin/revert-bulk-topic-actions
 * [new branch]          scrolling               -> origin/scrolling
 * [new branch]          sidebar-above-outlet    -> origin/sidebar-above-outlet
 * [new branch]          site-logo-height        -> origin/site-logo-height
   d7f7f79b6..92749b738  stable                  -> origin/stable
 * [new branch]          system-test-for-soft-load-search -> origin/system-test-for-soft-load-search
   6ab1a19e9..c30aeafd9  tests-passed            -> origin/tests-passed
 * [new branch]          tmp/move-notifiers-specs -> origin/tmp/move-notifiers-specs
 * [new branch]          unhide_lazy_load_categories -> origin/unhide_lazy_load_categories
 * [new branch]          universal-nonce         -> origin/universal-nonce
 * [new branch]          user-menu-full-name     -> origin/user-menu-full-name
 * [new branch]          ux-label-bold           -> origin/ux-label-bold
 * [new branch]          virtual-scroll-list     -> origin/virtual-scroll-list
 t [tag update]          beta                    -> beta
 t [tag update]          latest-release          -> latest-release
 * [new tag]             v3.1.4                  -> v3.1.4
 * [new tag]             v3.1.5                  -> v3.1.5
 * [new tag]             v3.2.0                  -> v3.2.0
 * [new tag]             v3.2.0.beta4            -> v3.2.0.beta4
 * [new tag]             v3.2.0.beta5            -> v3.2.0.beta5
I, [2024-02-15T10:28:16.245050 #1]  INFO -- : 
I, [2024-02-15T10:28:16.245159 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse bash -c '
  set -o errexit
  if [[ $(git symbolic-ref --short HEAD) == tests-passed ]] ; then
      git pull
  else
      git -c advice.detachedHead=false checkout tests-passed
  fi
'
Switched to a new branch 'tests-passed'
I, [2024-02-15T10:28:23.253872 #1]  INFO -- : Branch 'tests-passed' set up to track remote branch 'tests-passed' from 'origin'.
I, [2024-02-15T10:28:23.254007 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git config user.discourse-version tests-passed
I, [2024-02-15T10:28:23.264578 #1]  INFO -- : 
I, [2024-02-15T10:28:23.264750 #1]  INFO -- : > cd /var/www/discourse && mkdir -p tmp
I, [2024-02-15T10:28:23.268499 #1]  INFO -- : 
I, [2024-02-15T10:28:23.268683 #1]  INFO -- : > cd /var/www/discourse && chown discourse:www-data tmp
I, [2024-02-15T10:28:23.272081 #1]  INFO -- : 
I, [2024-02-15T10:28:23.272162 #1]  INFO -- : > cd /var/www/discourse && mkdir -p tmp/pids
I, [2024-02-15T10:28:23.275033 #1]  INFO -- : 
I, [2024-02-15T10:28:23.275255 #1]  INFO -- : > cd /var/www/discourse && mkdir -p tmp/sockets
I, [2024-02-15T10:28:23.278006 #1]  INFO -- : 
I, [2024-02-15T10:28:23.278205 #1]  INFO -- : > cd /var/www/discourse && touch tmp/.gitkeep
I, [2024-02-15T10:28:23.281859 #1]  INFO -- : 
I, [2024-02-15T10:28:23.281991 #1]  INFO -- : > cd /var/www/discourse && mkdir -p                    /shared/log/rails
I, [2024-02-15T10:28:23.284816 #1]  INFO -- : 
I, [2024-02-15T10:28:23.284885 #1]  INFO -- : > cd /var/www/discourse && bash -c "touch -a           /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log"
I, [2024-02-15T10:28:23.288517 #1]  INFO -- : 
I, [2024-02-15T10:28:23.288699 #1]  INFO -- : > cd /var/www/discourse && bash -c "ln    -s           /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log /var/www/discourse/log"
I, [2024-02-15T10:28:23.292668 #1]  INFO -- : 
I, [2024-02-15T10:28:23.292739 #1]  INFO -- : > cd /var/www/discourse && bash -c "mkdir -p           /shared/{uploads,backups}"
I, [2024-02-15T10:28:23.296906 #1]  INFO -- : 
I, [2024-02-15T10:28:23.296988 #1]  INFO -- : > cd /var/www/discourse && bash -c "ln    -s           /shared/{uploads,backups} /var/www/discourse/public"
I, [2024-02-15T10:28:23.300824 #1]  INFO -- : 
I, [2024-02-15T10:28:23.301054 #1]  INFO -- : > cd /var/www/discourse && bash -c "mkdir -p           /shared/tmp/{backups,restores}"
I, [2024-02-15T10:28:23.305107 #1]  INFO -- : 
I, [2024-02-15T10:28:23.305304 #1]  INFO -- : > cd /var/www/discourse && bash -c "ln    -s           /shared/tmp/{backups,restores} /var/www/discourse/tmp"
I, [2024-02-15T10:28:23.309337 #1]  INFO -- : 
I, [2024-02-15T10:28:23.309640 #1]  INFO -- : > cd /var/www/discourse && chown -R discourse:www-data /shared/log/rails /shared/uploads /shared/backups /shared/tmp
I, [2024-02-15T10:28:23.313054 #1]  INFO -- : 
I, [2024-02-15T10:28:23.313172 #1]  INFO -- : > cd /var/www/discourse && [ ! -d public/plugins ] || find public/plugins/ -maxdepth 1 -xtype l -delete
I, [2024-02-15T10:28:23.315102 #1]  INFO -- : 
I, [2024-02-15T10:28:23.316685 #1]  INFO -- : Replacing # redis with sv start redis || exit 1 in /etc/service/unicorn/run
I, [2024-02-15T10:28:23.317733 #1]  INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/docker_manager.git
Cloning into 'docker_manager'...
I, [2024-02-15T10:28:26.376448 #1]  INFO -- : 
I, [2024-02-15T10:28:26.377014 #1]  INFO -- : > cp /var/www/discourse/config/nginx.sample.conf /etc/nginx/conf.d/discourse.conf
I, [2024-02-15T10:28:26.382672 #1]  INFO -- : 
I, [2024-02-15T10:28:26.382999 #1]  INFO -- : > rm /etc/nginx/sites-enabled/default
I, [2024-02-15T10:28:26.385923 #1]  INFO -- : 
I, [2024-02-15T10:28:26.386108 #1]  INFO -- : > mkdir -p /var/nginx/cache
I, [2024-02-15T10:28:26.388292 #1]  INFO -- : 
I, [2024-02-15T10:28:26.388901 #1]  INFO -- : Replacing pid /run/nginx.pid; with daemon off; in /etc/nginx/nginx.conf
I, [2024-02-15T10:28:26.391324 #1]  INFO -- : Replacing (?m-ix:upstream[^\}]+\}) with upstream discourse { server 127.0.0.1:3000; } in /etc/nginx/conf.d/discourse.conf
I, [2024-02-15T10:28:26.391633 #1]  INFO -- : Replacing (?-mix:server_name.+$) with server_name _ ; in /etc/nginx/conf.d/discourse.conf
I, [2024-02-15T10:28:26.392107 #1]  INFO -- : Replacing (?-mix:client_max_body_size.+$) with client_max_body_size $upload_size ; in /etc/nginx/conf.d/discourse.conf
I, [2024-02-15T10:28:26.392579 #1]  INFO -- : Replacing (?-mix:worker_connections.+$) with worker_connections $nginx_worker_connections ; in /etc/nginx/nginx.conf
I, [2024-02-15T10:28:26.392893 #1]  INFO -- : > echo "done configuring web"
I, [2024-02-15T10:28:26.394573 #1]  INFO -- : done configuring web
I, [2024-02-15T10:28:26.394967 #1]  INFO -- : > cd /var/www/discourse && gem install bundler --conservative -v $(awk '/BUNDLED WITH/ { getline; gsub(/ /,""); print $0 }' Gemfile.lock)
I, [2024-02-15T10:28:27.261440 #1]  INFO -- : Successfully installed bundler-2.5.3
1 gem installed
I, [2024-02-15T10:28:27.261620 #1]  INFO -- : > cd /var/www/discourse && find /var/www/discourse ! -user discourse -exec chown discourse {} \+
I, [2024-02-15T10:28:31.381167 #1]  INFO -- : 
I, [2024-02-15T10:28:31.381815 #1]  INFO -- : > cd /var/www/discourse && if [ "tests-passed" != "tests-passed" ]; then
  rm -rf app/assets/javascripts/node_modules
fi
I, [2024-02-15T10:28:31.385573 #1]  INFO -- : 
I, [2024-02-15T10:28:31.385994 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'yarn install --frozen-lockfile && yarn cache clean'
warning "@discourse/lint-configs > eslint-plugin-ember > ember-eslint-parser@0.2.5" has unmet peer dependency "@typescript-eslint/parser@^6.15.0".
warning "@discourse/lint-configs > eslint-plugin-ember > ember-eslint-parser@0.2.5" has incorrect peer dependency "typescript@^5.3.3".
warning " > @glint/environment-ember-loose@1.3.0" has unmet peer dependency "@glimmer/component@^1.1.2".
I, [2024-02-15T10:30:20.264448 #1]  INFO -- : Terminating async processes
I, [2024-02-15T10:30:20.265295 #1]  INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main pid: 42
2024-02-15 10:30:20.265 UTC [42] LOG:  received fast shutdown request
I, [2024-02-15T10:30:20.265390 #1]  INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 111
111:signal-handler (1707993020) Received SIGTERM scheduling shutdown...
111:M 15 Feb 2024 10:30:20.284 # User requested shutdown...
111:M 15 Feb 2024 10:30:20.285 * Saving the final RDB snapshot before exiting.
2024-02-15 10:30:20.303 UTC [42] LOG:  aborting any active transactions
2024-02-15 10:30:20.307 UTC [42] LOG:  background worker "logical replication launcher" (PID 51) exited with exit code 1
2024-02-15 10:30:20.309 UTC [46] LOG:  shutting down
111:M 15 Feb 2024 10:30:20.335 * DB saved on disk
111:M 15 Feb 2024 10:30:20.344 # Redis is now ready to exit, bye bye...
2024-02-15 10:30:20.463 UTC [42] LOG:  database system is shut down
bootstrap failed with exit code 1
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.
c861827aef2d0dda3f1fc07c2cfe615a91eef803ab3fb36b5d71698adadc21e1

logs of ./discourse-doctor

DISCOURSE DOCTOR Thu 15 Feb 2024 06:32:48 PM CST
OS: Linux iZj6cgi365ov99veqodfgnZ 5.4.0-171-generic #189-Ubuntu SMP Fri Jan 5 14:23:02 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux


Found containers/app.yml

==================== YML SETTINGS ====================
DISCOURSE_HOSTNAME=*
SMTP_ADDRESS=*
DEVELOPER_EMAILS=*
SMTP_PASSWORD=*
SMTP_PORT=*
SMTP_USER_NAME=*
LETSENCRYPT_ACCOUNT_EMAIL=*

==================== DOCKER INFO ====================
DOCKER VERSION: Docker version 25.0.3, build 4debf41

DOCKER PROCESSES (docker ps -a)

CONTAINER ID   IMAGE                              COMMAND                  CREATED       STATUS                        PORTS     NAMES
e34a707c2cba   discourse/base:2.0.20231218-0429   "/bin/bash -c '/usr/…"   2 hours ago   Exited (255) 18 minutes ago             quirky_fermat

==================== SERIOUS PROBLEM!!!! ====================
app not running!
Attempting to rebuild
==================== REBUILD LOG ====================
x86_64 arch detected.
Ensuring launcher is up to date
Fetching origin
Launcher is up-to-date
2.0.20231218-0429: Pulling from discourse/base
Digest: sha256:468f70b9bb4c6d0c6c2bbb3efc1a5e12d145eae57bdb6946b7fe5558beb52dc1
Status: Image is up to date for discourse/base:2.0.20231218-0429
docker.io/discourse/base:2.0.20231218-0429
/usr/local/lib/ruby/gems/3.2.0/gems/pups-1.2.1/lib/pups.rb
/usr/local/bin/pups --stdin
I, [2024-02-15T10:32:58.066976 #1]  INFO -- : Reading from stdin
I, [2024-02-15T10:32:58.072322 #1]  INFO -- : > locale-gen $LANG && update-locale
I, [2024-02-15T10:32:58.144173 #1]  INFO -- : Generating locales (this might take a while)...
Generation complete.

I, [2024-02-15T10:32:58.144356 #1]  INFO -- : > mkdir -p /shared/postgres_run
I, [2024-02-15T10:32:58.151295 #1]  INFO -- : 
I, [2024-02-15T10:32:58.152268 #1]  INFO -- : > chown postgres:postgres /shared/postgres_run
I, [2024-02-15T10:32:58.155924 #1]  INFO -- : 
I, [2024-02-15T10:32:58.156188 #1]  INFO -- : > chmod 775 /shared/postgres_run
I, [2024-02-15T10:32:58.159244 #1]  INFO -- : 
I, [2024-02-15T10:32:58.159572 #1]  INFO -- : > rm -fr /var/run/postgresql
I, [2024-02-15T10:32:58.162267 #1]  INFO -- : 
I, [2024-02-15T10:32:58.162579 #1]  INFO -- : > ln -s /shared/postgres_run /var/run/postgresql
I, [2024-02-15T10:32:58.165310 #1]  INFO -- : 
I, [2024-02-15T10:32:58.165607 #1]  INFO -- : > socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1
2024/02/15 10:32:58 socat[19] E connect(6, AF=1 "/shared/postgres_run/.s.PGSQL.5432", 36): No such file or directory
I, [2024-02-15T10:32:58.188054 #1]  INFO -- : 
I, [2024-02-15T10:32:58.188429 #1]  INFO -- : > rm -fr /shared/postgres_run/.s*
I, [2024-02-15T10:32:58.191661 #1]  INFO -- : 
I, [2024-02-15T10:32:58.191844 #1]  INFO -- : > rm -fr /shared/postgres_run/*.pid
I, [2024-02-15T10:32:58.194737 #1]  INFO -- : 
I, [2024-02-15T10:32:58.195096 #1]  INFO -- : > mkdir -p /shared/postgres_run/13-main.pg_stat_tmp
I, [2024-02-15T10:32:58.197379 #1]  INFO -- : 
I, [2024-02-15T10:32:58.197747 #1]  INFO -- : > chown postgres:postgres /shared/postgres_run/13-main.pg_stat_tmp
I, [2024-02-15T10:32:58.199854 #1]  INFO -- : 
I, [2024-02-15T10:32:58.204841 #1]  INFO -- : File > /etc/service/postgres/run  chmod: +x  chown: 
I, [2024-02-15T10:32:58.209036 #1]  INFO -- : File > /etc/service/postgres/log/run  chmod: +x  chown: 
I, [2024-02-15T10:32:58.214787 #1]  INFO -- : File > /etc/runit/3.d/99-postgres  chmod: +x  chown: 
I, [2024-02-15T10:32:58.218929 #1]  INFO -- : File > /root/upgrade_postgres  chmod: +x  chown: 
I, [2024-02-15T10:32:58.219263 #1]  INFO -- : > chown -R root /var/lib/postgresql/13/main
I, [2024-02-15T10:32:59.721972 #1]  INFO -- : 
I, [2024-02-15T10:32:59.722380 #1]  INFO -- : > [ ! -e /shared/postgres_data ] && install -d -m 0755 -o postgres -g postgres /shared/postgres_data && sudo -E -u postgres /usr/lib/postgresql/13/bin/initdb -D /shared/postgres_data || exit 0
I, [2024-02-15T10:32:59.724421 #1]  INFO -- : 
I, [2024-02-15T10:32:59.724554 #1]  INFO -- : > chown -R postgres:postgres /shared/postgres_data
I, [2024-02-15T10:32:59.751357 #1]  INFO -- : 
I, [2024-02-15T10:32:59.751770 #1]  INFO -- : > chown -R postgres:postgres /var/run/postgresql
I, [2024-02-15T10:32:59.754527 #1]  INFO -- : 
I, [2024-02-15T10:32:59.754836 #1]  INFO -- : > /root/upgrade_postgres
I, [2024-02-15T10:32:59.760384 #1]  INFO -- : 
I, [2024-02-15T10:32:59.760719 #1]  INFO -- : > rm /root/upgrade_postgres
I, [2024-02-15T10:32:59.762618 #1]  INFO -- : 
I, [2024-02-15T10:32:59.763346 #1]  INFO -- : Replacing data_directory = '/var/lib/postgresql/13/main' with data_directory = '/shared/postgres_data' in /etc/postgresql/13/main/postgresql.conf
I, [2024-02-15T10:32:59.763898 #1]  INFO -- : Replacing (?-mix:#?listen_addresses *=.*) with listen_addresses = '*' in /etc/postgresql/13/main/postgresql.conf
I, [2024-02-15T10:32:59.764407 #1]  INFO -- : Replacing (?-mix:#?synchronous_commit *=.*) with synchronous_commit = $db_synchronous_commit in /etc/postgresql/13/main/postgresql.conf
I, [2024-02-15T10:32:59.764961 #1]  INFO -- : Replacing (?-mix:#?shared_buffers *=.*) with shared_buffers = $db_shared_buffers in /etc/postgresql/13/main/postgresql.conf
I, [2024-02-15T10:32:59.765445 #1]  INFO -- : Replacing (?-mix:#?work_mem *=.*) with work_mem = $db_work_mem in /etc/postgresql/13/main/postgresql.conf
I, [2024-02-15T10:32:59.765966 #1]  INFO -- : Replacing (?-mix:#?default_text_search_config *=.*) with default_text_search_config = '$db_default_text_search_config' in /etc/postgresql/13/main/postgresql.conf
I, [2024-02-15T10:32:59.766437 #1]  INFO -- : > install -d -m 0755 -o postgres -g postgres /shared/postgres_backup
I, [2024-02-15T10:32:59.772080 #1]  INFO -- : 
I, [2024-02-15T10:32:59.772535 #1]  INFO -- : Replacing (?-mix:#?checkpoint_segments *=.*) with checkpoint_segments = $db_checkpoint_segments in /etc/postgresql/13/main/postgresql.conf
I, [2024-02-15T10:32:59.772939 #1]  INFO -- : Replacing (?-mix:#?logging_collector *=.*) with logging_collector = $db_logging_collector in /etc/postgresql/13/main/postgresql.conf
I, [2024-02-15T10:32:59.773467 #1]  INFO -- : Replacing (?-mix:#?log_min_duration_statement *=.*) with log_min_duration_statement = $db_log_min_duration_statement in /etc/postgresql/13/main/postgresql.conf
I, [2024-02-15T10:32:59.774259 #1]  INFO -- : Replacing (?-mix:^#local +replication +postgres +peer$) with local replication postgres  peer in /etc/postgresql/13/main/pg_hba.conf
I, [2024-02-15T10:32:59.774561 #1]  INFO -- : Replacing (?-mix:^host.*all.*all.*127.*$) with host all all 0.0.0.0/0 md5 in /etc/postgresql/13/main/pg_hba.conf
I, [2024-02-15T10:32:59.775081 #1]  INFO -- : Replacing (?-mix:^host.*all.*all.*::1\/128.*$) with host all all ::/0 md5 in /etc/postgresql/13/main/pg_hba.conf
I, [2024-02-15T10:32:59.775507 #1]  INFO -- : > HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main
I, [2024-02-15T10:32:59.778511 #1]  INFO -- : > sleep 5
2024-02-15 10:32:59.875 UTC [42] LOG:  starting PostgreSQL 13.13 (Debian 13.13-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2024-02-15 10:32:59.876 UTC [42] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2024-02-15 10:32:59.877 UTC [42] LOG:  listening on IPv6 address "::", port 5432
2024-02-15 10:32:59.880 UTC [42] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2024-02-15 10:32:59.886 UTC [45] LOG:  database system was shut down at 2024-02-15 10:30:20 UTC
2024-02-15 10:32:59.891 UTC [42] LOG:  database system is ready to accept connections
I, [2024-02-15T10:33:04.781190 #1]  INFO -- : 
I, [2024-02-15T10:33:04.781481 #1]  INFO -- : > su postgres -c 'createdb discourse' || true
2024-02-15 10:33:04.885 UTC [55] postgres@postgres ERROR:  database "discourse" already exists
2024-02-15 10:33:04.885 UTC [55] postgres@postgres STATEMENT:  CREATE DATABASE discourse;
createdb: error: database creation failed: ERROR:  database "discourse" already exists
I, [2024-02-15T10:33:04.887989 #1]  INFO -- : 
I, [2024-02-15T10:33:04.888519 #1]  INFO -- : > su postgres -c 'psql discourse -c "create user discourse;"' || true
2024-02-15 10:33:04.950 UTC [59] postgres@discourse ERROR:  role "discourse" already exists
2024-02-15 10:33:04.950 UTC [59] postgres@discourse STATEMENT:  create user discourse;
ERROR:  role "discourse" already exists
I, [2024-02-15T10:33:04.952839 #1]  INFO -- : 
I, [2024-02-15T10:33:04.953301 #1]  INFO -- : > su postgres -c 'psql discourse -c "grant all privileges on database discourse to discourse;"' || true
I, [2024-02-15T10:33:05.002768 #1]  INFO -- : GRANT

I, [2024-02-15T10:33:05.003148 #1]  INFO -- : > su postgres -c 'psql discourse -c "alter schema public owner to discourse;"'
I, [2024-02-15T10:33:05.053723 #1]  INFO -- : ALTER SCHEMA

I, [2024-02-15T10:33:05.053931 #1]  INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists hstore;"'
NOTICE:  extension "hstore" already exists, skipping
I, [2024-02-15T10:33:05.110606 #1]  INFO -- : CREATE EXTENSION

I, [2024-02-15T10:33:05.111079 #1]  INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists pg_trgm;"'
NOTICE:  extension "pg_trgm" already exists, skipping
I, [2024-02-15T10:33:05.156719 #1]  INFO -- : CREATE EXTENSION

I, [2024-02-15T10:33:05.157309 #1]  INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists vector;"'
NOTICE:  extension "vector" already exists, skipping
I, [2024-02-15T10:33:05.202622 #1]  INFO -- : CREATE EXTENSION

I, [2024-02-15T10:33:05.202859 #1]  INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists hstore;"'
NOTICE:  extension "hstore" already exists, skipping
I, [2024-02-15T10:33:05.253597 #1]  INFO -- : CREATE EXTENSION

I, [2024-02-15T10:33:05.254116 #1]  INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists pg_trgm;"'
NOTICE:  extension "pg_trgm" already exists, skipping
I, [2024-02-15T10:33:05.298429 #1]  INFO -- : CREATE EXTENSION

I, [2024-02-15T10:33:05.298902 #1]  INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists vector;"'
NOTICE:  extension "vector" already exists, skipping
I, [2024-02-15T10:33:05.348357 #1]  INFO -- : CREATE EXTENSION

I, [2024-02-15T10:33:05.348865 #1]  INFO -- : > sudo -u postgres psql discourse
I, [2024-02-15T10:33:05.352505 #1]  INFO -- : update pg_database set encoding = pg_char_to_encoding('UTF8') where datname = 'discourse' AND encoding = pg_char_to_encoding('SQL_ASCII');

I, [2024-02-15T10:33:05.434503 #1]  INFO -- : File > /var/lib/postgresql/take-database-backup  chmod: +x  chown: postgres:postgres
I, [2024-02-15T10:33:05.439741 #1]  INFO -- : File > /var/spool/cron/crontabs/postgres  chmod:   chown: 
I, [2024-02-15T10:33:05.439953 #1]  INFO -- : > echo postgres installed!
I, [2024-02-15T10:33:05.442035 #1]  INFO -- : postgres installed!

I, [2024-02-15T10:33:05.447602 #1]  INFO -- : File > /etc/service/redis/run  chmod: +x  chown: 
I, [2024-02-15T10:33:05.453493 #1]  INFO -- : File > /etc/service/redis/log/run  chmod: +x  chown: 
I, [2024-02-15T10:33:05.458965 #1]  INFO -- : File > /etc/runit/3.d/10-redis  chmod: +x  chown: 
I, [2024-02-15T10:33:05.459960 #1]  INFO -- : Replacing daemonize yes with  in /etc/redis/redis.conf
I, [2024-02-15T10:33:05.460629 #1]  INFO -- : Replacing (?-mix:^pidfile.*$) with  in /etc/redis/redis.conf
I, [2024-02-15T10:33:05.461384 #1]  INFO -- : > install -d -m 0755 -o redis -g redis /shared/redis_data
I, [2024-02-15T10:33:05.465040 #1]  INFO -- : 
I, [2024-02-15T10:33:05.465645 #1]  INFO -- : Replacing (?-mix:^logfile.*$) with logfile "" in /etc/redis/redis.conf
I, [2024-02-15T10:33:05.466438 #1]  INFO -- : Replacing (?-mix:^bind .*$) with  in /etc/redis/redis.conf
I, [2024-02-15T10:33:05.467357 #1]  INFO -- : Replacing (?-mix:^dir .*$) with dir /shared/redis_data in /etc/redis/redis.conf
I, [2024-02-15T10:33:05.468322 #1]  INFO -- : Replacing (?-mix:^protected-mode yes) with protected-mode no in /etc/redis/redis.conf
I, [2024-02-15T10:33:05.469224 #1]  INFO -- : Replacing # io-threads 4 with io-threads $redis_io_threads in /etc/redis/redis.conf
I, [2024-02-15T10:33:05.469833 #1]  INFO -- : > echo redis installed
I, [2024-02-15T10:33:05.473332 #1]  INFO -- : redis installed

I, [2024-02-15T10:33:05.473556 #1]  INFO -- : > cat /etc/redis/redis.conf | grep logfile
I, [2024-02-15T10:33:05.482371 #1]  INFO -- : logfile ""

I, [2024-02-15T10:33:05.482879 #1]  INFO -- : > exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf
I, [2024-02-15T10:33:05.486036 #1]  INFO -- : > sleep 10
111:C 15 Feb 2024 10:33:05.494 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
111:C 15 Feb 2024 10:33:05.494 # Redis version=7.0.7, bits=64, commit=00000000, modified=0, pid=111, just started
111:C 15 Feb 2024 10:33:05.494 # Configuration loaded
111:M 15 Feb 2024 10:33:05.495 * monotonic clock: POSIX clock_gettime
111:M 15 Feb 2024 10:33:05.496 * Running mode=standalone, port=6379.
111:M 15 Feb 2024 10:33:05.496 # Server initialized
111:M 15 Feb 2024 10:33:05.497 * Loading RDB produced by version 7.0.7
111:M 15 Feb 2024 10:33:05.497 * RDB age 165 seconds
111:M 15 Feb 2024 10:33:05.497 * RDB memory usage when created 0.96 Mb
111:M 15 Feb 2024 10:33:05.498 * Done loading RDB, keys loaded: 34, keys expired: 0.
111:M 15 Feb 2024 10:33:05.498 * DB loaded from disk: 0.001 seconds
111:M 15 Feb 2024 10:33:05.498 * Ready to accept connections
I, [2024-02-15T10:33:15.489334 #1]  INFO -- : 
I, [2024-02-15T10:33:15.489635 #1]  INFO -- : > thpoff echo "thpoff is installed!"
I, [2024-02-15T10:33:15.493689 #1]  INFO -- : thpoff is installed!

I, [2024-02-15T10:33:15.493894 #1]  INFO -- : > /usr/local/bin/ruby -e 'if ENV["DISCOURSE_SMTP_ADDRESS"] == "smtp.example.com"; puts "Aborting! Mail is not configured!"; exit 1; end'
I, [2024-02-15T10:33:15.558077 #1]  INFO -- : 
I, [2024-02-15T10:33:15.558490 #1]  INFO -- : > /usr/local/bin/ruby -e 'if ENV["DISCOURSE_HOSTNAME"] == "discourse.example.com"; puts "Aborting! Domain is not configured!"; exit 1; end'
I, [2024-02-15T10:33:15.637632 #1]  INFO -- : 
I, [2024-02-15T10:33:15.638063 #1]  INFO -- : > /usr/local/bin/ruby -e 'if (ENV["DISCOURSE_CDN_URL"] || "")[0..1] == "//"; puts "Aborting! CDN must have a protocol specified. Once fixed you should rebake your posts now to correct all posts."; exit 1; end'
I, [2024-02-15T10:33:15.705466 #1]  INFO -- : 
I, [2024-02-15T10:33:15.705909 #1]  INFO -- : > rm -f /etc/cron.d/anacron
I, [2024-02-15T10:33:15.710220 #1]  INFO -- : 
I, [2024-02-15T10:33:15.713372 #1]  INFO -- : File > /etc/cron.d/anacron  chmod:   chown: 
I, [2024-02-15T10:33:15.718596 #1]  INFO -- : File > /etc/runit/1.d/copy-env  chmod: +x  chown: 
I, [2024-02-15T10:33:15.722935 #1]  INFO -- : File > /etc/service/unicorn/run  chmod: +x  chown: 
I, [2024-02-15T10:33:15.727457 #1]  INFO -- : File > /etc/service/nginx/run  chmod: +x  chown: 
I, [2024-02-15T10:33:15.732032 #1]  INFO -- : File > /etc/runit/3.d/01-nginx  chmod: +x  chown: 
I, [2024-02-15T10:33:15.736451 #1]  INFO -- : File > /etc/runit/3.d/02-unicorn  chmod: +x  chown: 
I, [2024-02-15T10:33:15.736788 #1]  INFO -- : Replacing # postgres with sv start postgres || exit 1 in /etc/service/unicorn/run
I, [2024-02-15T10:33:15.737110 #1]  INFO -- : > exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf
I, [2024-02-15T10:33:15.739873 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git reset --hard
138:C 15 Feb 2024 10:33:15.744 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
138:C 15 Feb 2024 10:33:15.744 # Redis version=7.0.7, bits=64, commit=00000000, modified=0, pid=138, just started
138:C 15 Feb 2024 10:33:15.744 # Configuration loaded
138:M 15 Feb 2024 10:33:15.746 * monotonic clock: POSIX clock_gettime
138:M 15 Feb 2024 10:33:15.746 # Warning: Could not create server TCP listening socket *:6379: bind: Address already in use
138:M 15 Feb 2024 10:33:15.746 # Failed listening on port 6379 (TCP), aborting.
Updating files:  39% (13472/33715)
Updating files:  40% (13486/33715)
Updating files:  41% (13824/33715)
Updating files:  42% (14161/33715)
Updating files:  43% (14498/33715)
Updating files:  44% (14835/33715)
Updating files:  45% (15172/33715)
Updating files:  46% (15509/33715)
Updating files:  47% (15847/33715)
Updating files:  48% (16184/33715)
Updating files:  49% (16521/33715)
Updating files:  50% (16858/33715)
Updating files:  51% (17195/33715)
Updating files:  52% (17532/33715)
Updating files:  53% (17869/33715)
Updating files:  54% (18207/33715)
Updating files:  55% (18544/33715)
Updating files:  56% (18881/33715)
Updating files:  57% (19218/33715)
Updating files:  58% (19555/33715)
Updating files:  59% (19892/33715)
Updating files:  60% (20229/33715)
Updating files:  61% (20567/33715)
Updating files:  62% (20904/33715)
Updating files:  63% (21241/33715)
Updating files:  64% (21578/33715)
Updating files:  65% (21915/33715)
Updating files:  66% (22252/33715)
Updating files:  67% (22590/33715)
Updating files:  68% (22927/33715)
Updating files:  69% (23264/33715)
Updating files:  70% (23601/33715)
Updating files:  71% (23938/33715)
Updating files:  72% (24275/33715)
Updating files:  73% (24612/33715)
Updating files:  74% (24950/33715)
Updating files:  75% (25287/33715)
Updating files:  76% (25624/33715)
Updating files:  77% (25961/33715)
Updating files:  78% (26298/33715)
Updating files:  79% (26635/33715)
Updating files:  80% (26972/33715)
Updating files:  81% (27310/33715)
Updating files:  82% (27647/33715)
Updating files:  83% (27984/33715)
Updating files:  84% (28321/33715)
Updating files:  85% (28658/33715)
Updating files:  86% (28995/33715)
Updating files:  87% (29333/33715)
Updating files:  88% (29670/33715)
Updating files:  89% (30007/33715)
Updating files:  90% (30344/33715)
Updating files:  91% (30681/33715)
Updating files:  92% (31018/33715)
Updating files:  93% (31355/33715)
Updating files:  94% (31693/33715)
Updating files:  94% (32000/33715)
Updating files:  95% (32030/33715)
Updating files:  96% (32367/33715)
Updating files:  97% (32704/33715)
Updating files:  98% (33041/33715)
Updating files:  99% (33378/33715)
Updating files: 100% (33715/33715)
Updating files: 100% (33715/33715), done.
I, [2024-02-15T10:33:18.782143 #1]  INFO -- : HEAD is now at 6ab1a19e9 DEV: Convert min_trust_level_to_allow_invite to groups (#24893)

I, [2024-02-15T10:33:18.782611 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git clean -f
I, [2024-02-15T10:33:18.995525 #1]  INFO -- : 
I, [2024-02-15T10:33:18.995981 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse bash -c '
  set -o errexit
  if [ $(git rev-parse --is-shallow-repository) == "true" ]; then
      git remote set-branches --add origin main
      git remote set-branches origin tests-passed
      git fetch --depth 1 origin tests-passed
  else
      git fetch --tags --prune-tags --prune --force origin
  fi
'
From https://github.com/discourse/discourse
 - [deleted]             (none)     -> origin/0-drop-code
 - [deleted]             (none)     -> origin/bulk-button-class
 - [deleted]             (none)     -> origin/dependabot/bundler/excon-0.108.0
 - [deleted]             (none)     -> origin/dependabot/bundler/hashdiff-1.1.0
 - [deleted]             (none)     -> origin/dependabot/bundler/parallel-1.24.0
 - [deleted]             (none)     -> origin/dependabot/bundler/rubocop-discourse-3.6.0
 - [deleted]             (none)     -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/drop-target-2.0.2
 - [deleted]             (none)     -> origin/dev/add-new-chat-metrics
 - [deleted]             (none)     -> origin/dev/allow-chat-mentions-to-have-several-notifications
 - [deleted]             (none)     -> origin/dev/convert-min-trust-level-to-create-tag-to-group
 - [deleted]             (none)     -> origin/dev/convert-min-trust-level-to-ignore-to-groups
 - [deleted]             (none)     -> origin/dev/rethink-mass-mentions-polymorphic
 - [deleted]             (none)     -> origin/feature/migrations-schema-generator
 - [deleted]             (none)     -> origin/fix-join-filter-upload
 - [deleted]             (none)     -> origin/fix-modal-flash-minor
 - [deleted]             (none)     -> origin/global-notice-reactive
 - [deleted]             (none)     -> origin/labeler
 - [deleted]             (none)     -> origin/low-ram-help
 - [deleted]             (none)     -> origin/multi-invite
 - [deleted]             (none)     -> origin/plugin-extra-js
 - [deleted]             (none)     -> origin/table-builder-modal-modifier
 - [deleted]             (none)     -> origin/tests-passed-broken
 - [deleted]             (none)     -> origin/wip_categories
 - [deleted]             (none)     -> origin/wizard-split
   6ab1a19e9..cb8958f20  main                    -> origin/main
 * [new branch]          0-ember-bump            -> origin/0-ember-bump
 * [new branch]          0-event-helper          -> origin/0-event-helper
 * [new branch]          0-extend-native         -> origin/0-extend-native
 * [new branch]          0-fix-perf-watched-words -> origin/0-fix-perf-watched-words
 * [new branch]          0-rev                   -> origin/0-rev
 * [new branch]          0-screen-track          -> origin/0-screen-track
 * [new branch]          add-api-for-forced-auth -> origin/add-api-for-forced-auth
 * [new branch]          asciinema-iframe        -> origin/asciinema-iframe
 * [new branch]          async-category-search   -> origin/async-category-search
 * [new branch]          bulk-select-dropdown-items -> origin/bulk-select-dropdown-items
 * [new branch]          convert-header-to-glimmer -> origin/convert-header-to-glimmer
 * [new branch]          csp-strict-dynamic      -> origin/csp-strict-dynamic
 * [new branch]          decorator-transforms    -> origin/decorator-transforms
 * [new branch]          dependabot/bundler/omniauth-2.0.0 -> origin/dependabot/bundler/omniauth-2.0.0
 * [new branch]          dependabot/bundler/parallel_tests-4.4.0 -> origin/dependabot/bundler/parallel_tests-4.4.0
 * [new branch]          dev-convert-topic-map-expanded-widget-to-component -> origin/dev-convert-topic-map-expanded-widget-to-component
 * [new branch]          dev/add-modifier-user-action-stream-builder -> origin/dev/add-modifier-user-action-stream-builder
 * [new branch]          dev/check-whether-online-user-ids-may-be-memoized-between-job-runs -> origin/dev/check-whether-online-user-ids-may-be-memoized-between-job-runs
 + 79ea38776...bd0faffec dev/drop-notification_id-column-on-chat-mentions -> origin/dev/drop-notification_id-column-on-chat-mentions  (forced update)
 * [new branch]          dev/drop-the-user-id-column-from-chat-mentions -> origin/dev/drop-the-user-id-column-from-chat-mentions
 + e0c0191bb...6751ac529 dev/dry-up-user-status-serializers -> origin/dev/dry-up-user-status-serializers  (forced update)
 * [new branch]          dev/experiment-theme-settings-free-ui -> origin/dev/experiment-theme-settings-free-ui
 * [new branch]          dev/groundwork-theme-settings-editor -> origin/dev/groundwork-theme-settings-editor
 * [new branch]          dev/make-admins-tl4-in-tests -> origin/dev/make-admins-tl4-in-tests
 * [new branch]          dev/promote-plugin-post-deploy-migrations -> origin/dev/promote-plugin-post-deploy-migrations
 * [new branch]          dev/proof-of-concept-rewriting-mentions-notification-logic -> origin/dev/proof-of-concept-rewriting-mentions-notification-logic
 * [new branch]          dev_category_order_database -> origin/dev_category_order_database
 * [new branch]          dev_update_category_position -> origin/dev_update_category_position
 * [new branch]          discover_enrollment     -> origin/discover_enrollment
 * [new branch]          extra-js-revert         -> origin/extra-js-revert
 * [new branch]          fast-edit-expand-regex  -> origin/fast-edit-expand-regex
   8780daeee..d92b6b19b  feature/bookmark-menu-and-modal-redesign -> origin/feature/bookmark-menu-and-modal-redesign
 * [new branch]          feature/custom-date-range -> origin/feature/custom-date-range
 * [new branch]          feature/paste-upload-on-first-upload-input-templates -> origin/feature/paste-upload-on-first-upload-input-templates
 * [new branch]          feature_async_load_colors -> origin/feature_async_load_colors
 * [new branch]          feature_track_mentions  -> origin/feature_track_mentions
 * [new branch]          fix_category_chooser    -> origin/fix_category_chooser
 * [new branch]          fix_in_any_groups       -> origin/fix_in_any_groups
 * [new branch]          fix_reviewable_translation -> origin/fix_reviewable_translation
   7f2bdca65..4db506cea  generic-import-latest   -> origin/generic-import-latest
 * [new branch]          hijacks                 -> origin/hijacks
 * [new branch]          issue/respect-homepage-admin-sidebar-back -> origin/issue/respect-homepage-admin-sidebar-back
 * [new branch]          issue/run-admin-sidebar-init-after-logs-notice -> origin/issue/run-admin-sidebar-init-after-logs-notice
 * [new branch]          loic-rubocop-plugin-rules -> origin/loic-rubocop-plugin-rules
 + 90a3535b7...54d0135bc migrations-convert      -> origin/migrations-convert  (forced update)
 * [new branch]          promote-migration-tweak -> origin/promote-migration-tweak
 * [new branch]          redesign-my-threads-index -> origin/redesign-my-threads-index
 * [new branch]          revert-25009-dev/convert-min-trust-to-allow-self-wiki-to-groups -> origin/revert-25009-dev/convert-min-trust-to-allow-self-wiki-to-groups
 * [new branch]          revert-bulk-topic-actions -> origin/revert-bulk-topic-actions
 * [new branch]          scrolling               -> origin/scrolling
 * [new branch]          sidebar-above-outlet    -> origin/sidebar-above-outlet
 * [new branch]          site-logo-height        -> origin/site-logo-height
   d7f7f79b6..92749b738  stable                  -> origin/stable
 * [new branch]          system-test-for-soft-load-search -> origin/system-test-for-soft-load-search
   6ab1a19e9..c30aeafd9  tests-passed            -> origin/tests-passed
 * [new branch]          tmp/move-notifiers-specs -> origin/tmp/move-notifiers-specs
 * [new branch]          unhide_lazy_load_categories -> origin/unhide_lazy_load_categories
 * [new branch]          universal-nonce         -> origin/universal-nonce
 * [new branch]          user-menu-full-name     -> origin/user-menu-full-name
 * [new branch]          ux-label-bold           -> origin/ux-label-bold
 * [new branch]          virtual-scroll-list     -> origin/virtual-scroll-list
 t [tag update]          beta                    -> beta
 t [tag update]          latest-release          -> latest-release
 * [new tag]             v3.1.4                  -> v3.1.4
 * [new tag]             v3.1.5                  -> v3.1.5
 * [new tag]             v3.2.0                  -> v3.2.0
 * [new tag]             v3.2.0.beta4            -> v3.2.0.beta4
 * [new tag]             v3.2.0.beta5            -> v3.2.0.beta5
I, [2024-02-15T10:33:21.020697 #1]  INFO -- : 
I, [2024-02-15T10:33:21.020828 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse bash -c '
  set -o errexit
  if [[ $(git symbolic-ref --short HEAD) == tests-passed ]] ; then
      git pull
  else
      git -c advice.detachedHead=false checkout tests-passed
  fi
'
Switched to a new branch 'tests-passed'
I, [2024-02-15T10:33:27.398231 #1]  INFO -- : Branch 'tests-passed' set up to track remote branch 'tests-passed' from 'origin'.

I, [2024-02-15T10:33:27.398367 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git config user.discourse-version tests-passed
I, [2024-02-15T10:33:27.409709 #1]  INFO -- : 
I, [2024-02-15T10:33:27.409806 #1]  INFO -- : > cd /var/www/discourse && mkdir -p tmp
I, [2024-02-15T10:33:27.414863 #1]  INFO -- : 
I, [2024-02-15T10:33:27.415285 #1]  INFO -- : > cd /var/www/discourse && chown discourse:www-data tmp
I, [2024-02-15T10:33:27.419956 #1]  INFO -- : 
I, [2024-02-15T10:33:27.420033 #1]  INFO -- : > cd /var/www/discourse && mkdir -p tmp/pids
I, [2024-02-15T10:33:27.423192 #1]  INFO -- : 
I, [2024-02-15T10:33:27.423258 #1]  INFO -- : > cd /var/www/discourse && mkdir -p tmp/sockets
I, [2024-02-15T10:33:27.426439 #1]  INFO -- : 
I, [2024-02-15T10:33:27.426506 #1]  INFO -- : > cd /var/www/discourse && touch tmp/.gitkeep
I, [2024-02-15T10:33:27.437513 #1]  INFO -- : 
I, [2024-02-15T10:33:27.437625 #1]  INFO -- : > cd /var/www/discourse && mkdir -p                    /shared/log/rails
I, [2024-02-15T10:33:27.443985 #1]  INFO -- : 
I, [2024-02-15T10:33:27.444072 #1]  INFO -- : > cd /var/www/discourse && bash -c "touch -a           /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log"
I, [2024-02-15T10:33:27.448513 #1]  INFO -- : 
I, [2024-02-15T10:33:27.448609 #1]  INFO -- : > cd /var/www/discourse && bash -c "ln    -s           /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log /var/www/discourse/log"
I, [2024-02-15T10:33:27.452785 #1]  INFO -- : 
I, [2024-02-15T10:33:27.452858 #1]  INFO -- : > cd /var/www/discourse && bash -c "mkdir -p           /shared/{uploads,backups}"
I, [2024-02-15T10:33:27.456712 #1]  INFO -- : 
I, [2024-02-15T10:33:27.456788 #1]  INFO -- : > cd /var/www/discourse && bash -c "ln    -s           /shared/{uploads,backups} /var/www/discourse/public"
I, [2024-02-15T10:33:27.460782 #1]  INFO -- : 
I, [2024-02-15T10:33:27.460919 #1]  INFO -- : > cd /var/www/discourse && bash -c "mkdir -p           /shared/tmp/{backups,restores}"
I, [2024-02-15T10:33:27.464865 #1]  INFO -- : 
I, [2024-02-15T10:33:27.465167 #1]  INFO -- : > cd /var/www/discourse && bash -c "ln    -s           /shared/tmp/{backups,restores} /var/www/discourse/tmp"
I, [2024-02-15T10:33:27.468963 #1]  INFO -- : 
I, [2024-02-15T10:33:27.469128 #1]  INFO -- : > cd /var/www/discourse && chown -R discourse:www-data /shared/log/rails /shared/uploads /shared/backups /shared/tmp
I, [2024-02-15T10:33:27.474059 #1]  INFO -- : 
I, [2024-02-15T10:33:27.474159 #1]  INFO -- : > cd /var/www/discourse && [ ! -d public/plugins ] || find public/plugins/ -maxdepth 1 -xtype l -delete
I, [2024-02-15T10:33:27.476508 #1]  INFO -- : 
I, [2024-02-15T10:33:27.476825 #1]  INFO -- : Replacing # redis with sv start redis || exit 1 in /etc/service/unicorn/run
I, [2024-02-15T10:33:27.477168 #1]  INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/docker_manager.git
Cloning into 'docker_manager'...
I, [2024-02-15T10:33:29.856181 #1]  INFO -- : 
I, [2024-02-15T10:33:29.856587 #1]  INFO -- : > cp /var/www/discourse/config/nginx.sample.conf /etc/nginx/conf.d/discourse.conf
I, [2024-02-15T10:33:29.860625 #1]  INFO -- : 
I, [2024-02-15T10:33:29.860903 #1]  INFO -- : > rm /etc/nginx/sites-enabled/default
I, [2024-02-15T10:33:29.864710 #1]  INFO -- : 
I, [2024-02-15T10:33:29.864911 #1]  INFO -- : > mkdir -p /var/nginx/cache
I, [2024-02-15T10:33:29.868193 #1]  INFO -- : 
I, [2024-02-15T10:33:29.868767 #1]  INFO -- : Replacing pid /run/nginx.pid; with daemon off; in /etc/nginx/nginx.conf
I, [2024-02-15T10:33:29.869091 #1]  INFO -- : Replacing (?m-ix:upstream[^\}]+\}) with upstream discourse { server 127.0.0.1:3000; } in /etc/nginx/conf.d/discourse.conf
I, [2024-02-15T10:33:29.869415 #1]  INFO -- : Replacing (?-mix:server_name.+$) with server_name _ ; in /etc/nginx/conf.d/discourse.conf
I, [2024-02-15T10:33:29.869918 #1]  INFO -- : Replacing (?-mix:client_max_body_size.+$) with client_max_body_size $upload_size ; in /etc/nginx/conf.d/discourse.conf
I, [2024-02-15T10:33:29.870365 #1]  INFO -- : Replacing (?-mix:worker_connections.+$) with worker_connections $nginx_worker_connections ; in /etc/nginx/nginx.conf
I, [2024-02-15T10:33:29.870684 #1]  INFO -- : > echo "done configuring web"
I, [2024-02-15T10:33:29.872978 #1]  INFO -- : done configuring web

I, [2024-02-15T10:33:29.873275 #1]  INFO -- : > cd /var/www/discourse && gem install bundler --conservative -v $(awk '/BUNDLED WITH/ { getline; gsub(/ /,""); print $0 }' Gemfile.lock)
I, [2024-02-15T10:33:30.711167 #1]  INFO -- : Successfully installed bundler-2.5.3
1 gem installed

I, [2024-02-15T10:33:30.711498 #1]  INFO -- : > cd /var/www/discourse && find /var/www/discourse ! -user discourse -exec chown discourse {} \+
I, [2024-02-15T10:33:34.307137 #1]  INFO -- : 
I, [2024-02-15T10:33:34.307628 #1]  INFO -- : > cd /var/www/discourse && if [ "tests-passed" != "tests-passed" ]; then
  rm -rf app/assets/javascripts/node_modules
fi
I, [2024-02-15T10:33:34.309730 #1]  INFO -- : 
I, [2024-02-15T10:33:34.309933 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'yarn install --frozen-lockfile && yarn cache clean'
warning "@discourse/lint-configs > eslint-plugin-ember > ember-eslint-parser@0.2.5" has unmet peer dependency "@typescript-eslint/parser@^6.15.0".
warning "@discourse/lint-configs > eslint-plugin-ember > ember-eslint-parser@0.2.5" has incorrect peer dependency "typescript@^5.3.3".
warning " > @glint/environment-ember-loose@1.3.0" has unmet peer dependency "@glimmer/component@^1.1.2".
warning Resolution field "unset-value@2.0.1" is incompatible with requested version "unset-value@^1.0.0"
warning Pattern ["wrap-ansi@^7.0.0"] is trying to unpack in the same destination "/home/discourse/.cache/yarn/v6/npm-wrap-ansi-cjs-7.0.0-67e145cff510a6a6984bdf1152911d69d2eb9e43-integrity/node_modules/wrap-ansi-cjs" as pattern ["wrap-ansi-cjs@npm:wrap-ansi@^7.0.0"]. This could result in non-deterministic behavior, skipping.
warning " > discourse-markdown-it@1.0.0" has unmet peer dependency "xss@*".
warning "workspace-aggregator-11c50905-365e-459d-8c6c-c8b5753a7862 > discourse > @uppy/aws-s3@3.0.6" has incorrect peer dependency "@uppy/core@^3.1.2".
warning "workspace-aggregator-11c50905-365e-459d-8c6c-c8b5753a7862 > discourse > @uppy/aws-s3-multipart@3.1.3" has incorrect peer dependency "@uppy/core@^3.1.2".
warning "workspace-aggregator-11c50905-365e-459d-8c6c-c8b5753a7862 > discourse > @uppy/xhr-upload@3.1.1" has incorrect peer dependency "@uppy/core@^3.1.2".
warning "workspace-aggregator-11c50905-365e-459d-8c6c-c8b5753a7862 > discourse > ember-router-service-refresh-polyfill@1.1.0" has incorrect peer dependency "ember-source@^3.4 || 4".
warning "workspace-aggregator-11c50905-365e-459d-8c6c-c8b5753a7862 > discourse-plugins > ember-this-fallback@0.4.0" has unmet peer dependency "ember-source@^3.28.11 || ^4.0.0".
warning "workspace-aggregator-11c50905-365e-459d-8c6c-c8b5753a7862 > admin > ember-source > router_js@8.0.3" has unmet peer dependency "rsvp@^4.8.5".
warning "workspace-aggregator-11c50905-365e-459d-8c6c-c8b5753a7862 > discourse > @uppy/aws-s3 > @uppy/xhr-upload@3.3.0" has incorrect peer dependency "@uppy/core@^3.2.1".
I, [2024-02-15T10:36:32.516222 #1]  INFO -- : yarn install v1.22.19
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
[4/5] Linking dependencies...
[5/5] Building fresh packages...
$ yarn --cwd app/assets/javascripts $(node -e 'const argv = JSON.parse(process.env.npm_config_argv).original; const passthrough = [`--frozen-lockfile`, `-s`].filter(arg => argv.includes(arg)); console.log(passthrough.join(` `));')
yarn install v1.22.19
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
$ ./run-patch-package
patch-package 8.0.0
Applying patches...
@ember/test-helpers@2.9.4 ✔
babel-plugin-debug-macros@0.3.4 ✔
content-tag@2.0.1 ✔
ember-cli@5.0.0 ✔
ember-this-fallback@0.4.0 (1 deprecation-name) ✔
ember-this-fallback@0.4.0 (2 themes) ✔
ember-this-fallback@0.4.0 (3 exclude-strict-mode) ✔
virtual-dom@2.1.1 ✔
Done in 74.39s.
Done in 171.35s.
yarn cache v1.22.19
success Cleared cache.
Done in 6.23s.

I, [2024-02-15T10:36:32.521200 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle config --local deployment true'
I, [2024-02-15T10:36:32.867913 #1]  INFO -- : 
I, [2024-02-15T10:36:32.868310 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle config --local without "development test"'
I, [2024-02-15T10:36:33.035500 #1]  INFO -- : You are replacing the current local value of without, which is currently "test:development"

I, [2024-02-15T10:36:33.035622 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle install --retry 3 --jobs 4'
I, [2024-02-15T10:36:58.988329 #1]  INFO -- : Fetching gem metadata from https://rubygems.org/.........
Fetching minitest 5.22.2
Fetching concurrent-ruby 1.2.3
Fetching bigdecimal 3.1.6
Fetching cbor 0.5.9.8
Installing concurrent-ruby 1.2.3
Installing minitest 5.22.2
Installing bigdecimal 3.1.6 with native extensions
Installing cbor 0.5.9.8 with native extensions
Fetching excon 0.109.0
Installing excon 0.109.0
Fetching fastimage 2.3.0
Fetching google-protobuf 3.25.2 (x86_64-linux)
Installing fastimage 2.3.0
Fetching highline 3.0.1
Installing highline 3.0.1
Installing google-protobuf 3.25.2 (x86_64-linux)
Fetching image_size 3.4.0
Installing image_size 3.4.0
Fetching regexp_parser 2.9.0
Fetching zeitwerk 2.6.13
Installing regexp_parser 2.9.0
Installing zeitwerk 2.6.13
Fetching logster 2.17.1
Fetching stackprof 0.2.26
Installing stackprof 0.2.26 with native extensions
Installing logster 2.17.1
Fetching nokogiri 1.16.2 (x86_64-linux)
Installing nokogiri 1.16.2 (x86_64-linux)
Fetching request_store 1.6.0
Installing request_store 1.6.0
Fetching rack-protection 3.2.0
Installing rack-protection 3.2.0
Fetching bootsnap 1.18.3
Installing bootsnap 1.18.3 with native extensions
Fetching net-http 0.4.1
Installing net-http 0.4.1
Fetching sass-embedded 1.70.0 (x86_64-linux-gnu)
Fetching puma 6.4.2
Installing sass-embedded 1.70.0 (x86_64-linux-gnu)
Installing puma 6.4.2 with native extensions
Fetching rbtrace 0.5.1
Installing rbtrace 0.5.1 with native extensions
Fetching net-imap 0.4.10
Installing net-imap 0.4.10
Fetching net-smtp 0.4.0.1
Installing net-smtp 0.4.0.1
Fetching activesupport 7.0.8
Installing activesupport 7.0.8
Fetching tzinfo-data 1.2024.1
Installing tzinfo-data 1.2024.1
Fetching faraday-net_http 3.1.0
Installing faraday-net_http 3.1.0
Fetching sassc-embedded 1.70.0
Installing sassc-embedded 1.70.0
Fetching activemodel 7.0.8
Installing activemodel 7.0.8
Fetching faraday 2.9.0
Installing faraday 2.9.0
Fetching actionview 7.0.8
Installing actionview 7.0.8
Fetching activejob 7.0.8
Installing activejob 7.0.8
Fetching activerecord 7.0.8
Installing activerecord 7.0.8
Fetching actionpack 7.0.8
Installing actionpack 7.0.8
Fetching actionmailer 7.0.8
Installing actionmailer 7.0.8
Fetching railties 7.0.8
Installing railties 7.0.8
Bundle complete! 138 Gemfile dependencies, 170 gems now installed.
Gems in the groups 'development' and 'test' were not installed.
Bundled gems are installed into `./vendor/bundle`

I, [2024-02-15T10:36:58.988791 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'LOAD_PLUGINS=0 bundle exec rake plugin:pull_compatible_all'
I, [2024-02-15T10:37:03.050745 #1]  INFO -- : docker_manager is already at latest compatible version

I, [2024-02-15T10:37:03.051134 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate'
I, [2024-02-15T10:37:09.441141 #1]  INFO -- : 
I, [2024-02-15T10:37:09.441453 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake themes:update assets:precompile'
Node.js heap_size_limit (958.25) is less than 2048MB. Setting --max-old-space-size=2048.
111:M 15 Feb 2024 10:38:06.041 * 100 changes in 300 seconds. Saving...
111:M 15 Feb 2024 10:38:06.057 * Background saving started by pid 3440
3440:C 15 Feb 2024 10:38:06.077 * DB saved on disk
3440:C 15 Feb 2024 10:38:06.077 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
111:M 15 Feb 2024 10:38:06.157 * Background saving terminated with success
2024-02-15 10:54:01.821 UTC [49] WARNING:  worker took too long to start; canceled
2024-02-15 10:55:16.957 UTC [3452] WARNING:  autovacuum worker started without a worker entry

my server has 2 logical core and 2 GB RAM, and i enabled 4GB swap.
it’s running ubuntu 20.04.
in previous build, i didn’t save the full log but i got this

WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.

so i did what it says, but it still can’t run.

this line is the last line i saw in ssh, and then the server freezes with 100% disk read. the two lines below is found in log files after i reboot.

Thanks for all the details. It’s good that you did this

but I think probably doing that will only last until the next reboot. After reboot you would need to do it again. See MKJ’s Opinionated Discourse Deployment Configuration for tips on making this permanent.

It feels possible that you have too little memory (by which I mean RAM+swap) and yet 2+4 should be enough. Please run the following quick diagnostics and post the results:

cat /etc/lsb-release
uptime
df -h /
free
swapon
vmstat 5 5
dmesg|egrep -i "memory|oom|kill"
ps auxrc

Please also share your app.yml file here - but not the passwords and secret tokens inside it!

If you are able to set up two ssh connections, you can use one to run an app rebuild and use the other to see what the machine is doing. I like to alternate

vmstat 5 5
ps auxrc

It’s possible you are swapping to a remote disk - a network-attached storage - and this is known to be a problem. It will be very slow. Perhaps it causes a timeout and this is the problem. Perhaps there’s a way to adjust the timeout.

I found this - maybe it helps?

(The default systemd timeout is 90 seconds, at least in some releases of systemd, so this fits quite nicely).

You could try to work around this by increasing TimeoutStartSec in postgresql’s systemd unit (or even globally), which perhaps only hides the problem until the next service suddenly doesn’t start anymore.

Edit: if so, then this advice might be good:

You can uncomment in /etc/systemd/system.conf the lines:

DefaultTimeoutStartSec=90s
DefaultTimeoutStopSec=90s

And change the value to what you consider appropriate.

1 Like

Thanks a lot for your help!

results:

root@iZj6cgi365ov99veqodfgnZ:/# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.6 LTS"
root@iZj6cgi365ov99veqodfgnZ:/# uptime
 09:24:47 up 12:57,  0 users,  load average: 0.04, 0.08, 0.07
root@iZj6cgi365ov99veqodfgnZ:/# df -h /
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        59G   14G   43G  24% /
root@iZj6cgi365ov99veqodfgnZ:/# free
              total        used        free      shared  buff/cache   available
Mem:        1912944      536564       85752        2120     1290628     1186896
Swap:       2097148        3584     2093564
root@iZj6cgi365ov99veqodfgnZ:/# swapon
NAME      TYPE SIZE USED PRIO
/swapfile file   2G 3.5M   -2
root@iZj6cgi365ov99veqodfgnZ:/# vmstat 5 5
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0   3868  98444 398504 870220    0    0    10     5   90  125  1  0 99  0  0
 0  0   3868 100300 398512 870424    0    0     1     3 1188 2171  1  1 98  0  0
 2  0   3868  99828 398520 870424    0    0     0     4 1117 2067  1  1 98  0  0
 0  0   3868  99324 398528 870428    0    0     0     5 1181 2186  1  0 98  0  0
 0  0   3868 104112 398584 870428    0    0     0    41 1144 2104  1  1 98  0  0
root@iZj6cgi365ov99veqodfgnZ:/# dmesg|egrep -i "memory|oom|kill"
[    0.005681] check: Scanning 1 areas for low memory corruption
[    0.005904] ACPI: Reserving FACP table memory at [mem 0x7a9e0bfe-0x7a9e0c71]
[    0.005905] ACPI: Reserving DSDT table memory at [mem 0x7a9e0040-0x7a9e0bfd]
[    0.005905] ACPI: Reserving FACS table memory at [mem 0x7a9e0000-0x7a9e003f]
[    0.005906] ACPI: Reserving SSDT table memory at [mem 0x7a9e0c72-0x7a9e18d7]
[    0.005906] ACPI: Reserving APIC table memory at [mem 0x7a9e18d8-0x7a9e1957]
[    0.005907] ACPI: Reserving SRAT table memory at [mem 0x7a9e1958-0x7a9e1a1f]
[    0.006464] Early memory node ranges
[    0.013922] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
[    0.013924] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
[    0.013925] PM: Registered nosave memory: [mem 0x000a0000-0x000effff]
[    0.013926] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff]
[    0.013928] PM: Registered nosave memory: [mem 0x14000000-0x1511ffff]
[    0.018845] Memory: 1851216K/1990960K available (14339K kernel code, 2395K rwdata, 9288K rodata, 2768K init, 17228K bss, 139744K reserved, 0K cma-reserved)
[    0.039645] Freeing SMP alternatives memory: 40K
[    0.042284] x86/mm: Memory block size: 128MB
[    0.275835] Freeing initrd memory: 53892K
[    0.276020] check: Scanning for low memory corruption every 60 seconds
[    0.658145] Freeing unused decrypted memory: 2040K
[    0.658779] Freeing unused kernel image memory: 2768K
[    0.659406] Freeing unused kernel image memory: 2036K
[    0.659689] Freeing unused kernel image memory: 952K
root@iZj6cgi365ov99veqodfgnZ:/# ps auxrc
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root        8756  0.1  3.0 684704 58528 ?        Rl   09:16   0:00 node
root        9225  0.0  0.1  11488  3248 pts/0    R+   09:26   0:00 ps

app.yml:

## this is the all-in-one, standalone Discourse Docker container template
##
## After making changes to this file, you MUST rebuild
## /var/discourse/launcher rebuild app
##
## BE *VERY* CAREFUL WHEN EDITING!
## YAML FILES ARE SUPER SUPER SENSITIVE TO MISTAKES IN WHITESPACE OR ALIGNMENT!
## visit http://www.yamllint.com/ to validate this file as needed

templates:
  - "templates/postgres.template.yml"
  - "templates/redis.template.yml"
  - "templates/web.template.yml"
  ## Uncomment the next line to enable the IPv6 listener
  #- "templates/web.ipv6.template.yml"
  - "templates/web.ratelimited.template.yml"
  ## Uncomment these two lines if you wish to add Lets Encrypt (https)
  - "templates/web.ssl.template.yml"
  - "templates/web.letsencrypt.ssl.template.yml"

## which TCP/IP ports should this container expose?
## If you want Discourse to share a port with another webserver like Apache or nginx,
## see https://meta.discourse.org/t/17247 for details
expose:
  - "80:80"   # http
  - "443:443" # https

params:
  db_default_text_search_config: "pg_catalog.english"

  ## Set db_shared_buffers to a max of 25% of the total memory.
  ## will be set automatically by bootstrap based on detected RAM, or you can override
  db_shared_buffers: "256MB"
  upload_size: 300m
  ## can improve sorting performance, but adds memory usage per-connection
  #db_work_mem: "40MB"

  ## Which Git revision should this container use? (default: tests-passed)
  #version: tests-passed

env:
  LC_ALL: en_US.UTF-8
  LANG: en_US.UTF-8
  LANGUAGE: en_US.UTF-8
  DISCOURSE_DEFAULT_LOCALE: zh_CN

  ## How many concurrent web requests are supported? Depends on memory and CPU cores.
  ## will be set automatically by bootstrap based on detected CPUs, or you can override
  UNICORN_WORKERS: 2

  ## TODO: The domain name this Discourse instance will respond to
  ## Required. Discourse will not work with a bare IP number.
  DISCOURSE_HOSTNAME: *

  ## Uncomment if you want the container to be started with the same
  ## hostname (-h option) as specified above (default "$hostname-$config")
  #DOCKER_USE_HOSTNAME: true

  ## TODO: List of comma delimited emails that will be made admin and developer
  ## on initial signup example 'user1@example.com,user2@example.com'
  DISCOURSE_DEVELOPER_EMAILS: *

  ## TODO: The SMTP mail server used to validate new accounts and send notifications
  # SMTP ADDRESS, username, and password are required
  # WARNING the char '#' in SMTP password can cause problems!
  DISCOURSE_SMTP_ADDRESS: *
  DISCOURSE_SMTP_PORT: *
  DISCOURSE_SMTP_USER_NAME: *
  DISCOURSE_SMTP_PASSWORD: *
  #DISCOURSE_SMTP_ENABLE_START_TLS: true           # (optional, default true)
  DISCOURSE_SMTP_DOMAIN: *
  DISCOURSE_NOTIFICATION_EMAIL: *

  ## If you added the Lets Encrypt template, uncomment below to get a free SSL certificate
  LETSENCRYPT_ACCOUNT_EMAIL: me@example.com

  ## The http or https CDN address for this Discourse instance (configured to pull)
  ## see https://meta.discourse.org/t/14857 for details
  #DISCOURSE_CDN_URL: https://discourse-cdn.example.com
  
  ## The maxmind geolocation IP address key for IP address lookup
  ## see https://meta.discourse.org/t/-/137387/23 for details
  #DISCOURSE_MAXMIND_LICENSE_KEY: 1234567890123456

## The Docker container is stateless; all data is stored in /shared
volumes:
  - volume:
      host: /var/discourse/shared/standalone
      guest: /shared
  - volume:
      host: /var/discourse/shared/standalone/log/var-log
      guest: /var/log

## Plugins go here
## see https://meta.discourse.org/t/19157 for details
hooks:
  after_code:
    - exec:
        cd: $home/plugins
        cmd:
          - git clone https://github.com/discourse/docker_manager.git

## Any custom commands to run after building
run:
  - exec: echo "Beginning of custom commands"
  ## If you want to set the 'From' email address for your first registration, uncomment and change:
  ## After getting the first signup email, re-comment the line. It only needs to run once.
  #- exec: rails r "SiteSetting.notification_email='info@unconfigured.discourse.org'"
  - exec: echo "End of custom commands"

as I mentioned earlier, I am using a cloud server. In fact, the cloud server dashboard provides real-time monitoring of server status, and I only became aware of the 100% disk read after the server froze based on the dashboard data. During regular usage, I usually open another terminal and run top. Before the crash, I could see cpu wa reaching 90 or above in top.

as for the disk… I’m not sure how the cloud service provider connects their disks. In the dashboard, I observed that the disk was performing at full load with a reading of approximately 2500 IOPS.

additionally, since all my services are running on the system disk, I don’t think the latency should be so high as to cause timeouts, I guess? After all, the system itself can’t tolerate extremely high latency.

I’m not very familiar with Linux, so the above are just my guesses. If you have any suggestions, please feel free to share at any time. I really appreciate any help I can get from the community.

(I’m using translation software, so please forgive me if there are any inaccuracies or awkward phrases in my language.)

I noticed you seem to be searching for OOM-related content in the logs? I’ve restarted since the last crash, so I’m not sure if there would be any records from the previous run in the logs?

actually, i didn’t run this command. Instead, I just edited sysctl.conf, and i think this may be permanent?

I changed the value to 600s and rebooted, then ran ‘./launch bootstrap app’ again. Unfortunately, the problem still persists. Here is the last ‘top’ output and the logs after my SSH connection was disconnected.

I will temporarily keep the server in its crashed state without rebooting, in case you need any debugging information.

The following are the monitoring data from the Aliyun dashboard.


After the crash, the ‘cpu_total’ data became discontinuous, only occasionally obtainable, hence the graph line in the chart appears as a distinct jagged line. However, the IOPS data remains relatively continuous.
image
At this point, RAM usage remains almost at 100%. However, doesn’t the official Discourse documentation state that 1GB is completely sufficient for a small community?

I’m completely baffled.

It is sufficient for communities, but you need at least 2 GB to rebuild. That should update because we have way too many questions regarding low memory.

But I don’t know if your issues are coming from that.

Haven’t heard that before, have been able to do successful console rebuild with a 1GB ram server, but some of the UX updater attempts have failed.

Adding swap for memory instead of ram could help.

The newer current version of discourse may need more memory than previous ones to work well, since they wrote there have been some/many recent significant changes to the core system.

Installing a lower version of Discourse seems feasible for me temporarily… but will this actually solve my current issue? Or, if I manage to install a lower version successfully, does it imply that 2GB of RAM is indeed insufficient for building the latest version of Discourse?

I’ve reached the maximum daily reply limit for new users… So, I can only rely on editing this reply to add more information. Apologies.

I’m using Aliyun’s Simple Application Server, and I can upgrade my configuration to 4GB of RAM. However, due to the package’s affordability, my network configuration would also be upgraded, which I don’t need, especially considering the nearly doubled price. Additionally, this server doesn’t support downgrading the configuration after backing up the data, so I might only be able to use a 2-core 2GB RAM configuration. Alternatively, I could temporarily open a high-configuration server on a non-monthly basis to complete the build, and then is there a way to upload the entire built app somewhere, and then download and run it directly on my server? Would this be feasible if I ensure the system software environment is identical? I don’t have much understanding of Docker and the build process, so if there’s a similar solution that works, I’d appreciate it if someone could clarify.

separate line…

I currently have a server with 2 cores, 2GB RAM, a 60GB ESSD system disk, 2TB traffic, and a peak bandwidth of 30Mbps, including a public IP. The monthly cost is 34 RMB (approximately $4.78 USD), which I think is very cheap.

Given its extremely low price, I have to tolerate the drawbacks of not being able to change one configuration item as a package and not being able to downgrade after upgrading :rofl:.

separation…

OK, I just restarted the server. Surprisingly, there’s this line at the end of the saved logs that caught my eye, I don’t think I’ve ever seen it before.

3440:C 16 Feb 2024 03:03:04.088 * DB saved on disk
3440:C 16 Feb 2024 03:03:04.088 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
111:M 16 Feb 2024 03:03:04.167 * Background saving terminated with success
111:M 16 Feb 2024 03:58:50.500 * Replication backlog freed after 3600 seconds without connected replicas.

Now I will increase the swap file size and try again, with vmstat running

Wait a moment… I suddenly realized that I’m using VSCode to connect to the server, and I’ve directly set the root directory ‘/’ as the workspace. VSCode has warned me that for large workspaces, it may not be able to track file changes, but I suppose it still consumes some IO and disk resources, right…? I will try building without opening the workspace, just connecting through the terminal. I won’t increase the swap for now to confirm if this is the issue.

1 Like

Don’t believe that is possible, unless anyone has saved copies of previous versions to use for new installs.

Am no expert myself, there are some experts here who may have the answers you seek.

If you are able to add ram temporarily for a rebuild then you may be able to downgrade to less ram after that to reduce server cost.

Edit: See you can’t post more replies today since you are a new member, ok can talk more tomorrow.

Here is price graph for digital ocean servers changes can be reversed for these unless there are changes made to the disk:

1 Like

Thanks for all the responses and extra detail.

Indeed, the search for OOM and similar would probably be useful before a reboot, not after. So yes please repeat, if you have not yet rebooted.

Running
vmstat 5
concurrently, in a second connection, as the rebuild hits the critical section, could be informative.

But really I have only two hypotheses

  • shortage of memory which might cause slow progress or might cause something to be terminated
  • running out of time because of slow progress

It is a pity your provider does not allow both increases and decreases of RAM.

It is true that initial installation, and subsequent updates which require rebuilds every month or two, require more memory than previously. But
memory=RAM+swap
so adding swap should be successful, provided there is no time pressure.

You have lots of disk. I recommend you add a second and a third swapfile. See the instructions here, modified to make new files

You already have /swapfile so run these commands with /swapfile.1 and /swapfile.2 instead

Afterwards
swapon
should show three files in use, or ready for use, and
free
should show about 2G RAM and about 6G of swap

Then try again!

Edit: I notice my 1G RAM instance has
db_shared_buffers: "128MB"
in the app.yml

Be very careful when modifying app.yml - do not add or remove spaces!

2 Likes

Thank you all so, so much! The issue has been resolved, and the bootstrap is successful! The problem was indeed with VSCode, perhaps because I directly set the root directory as the workspace, and VSCode was trying to track all file changes throughout the installation process? That’s my guess. Previously, I was also puzzled because during the crash, disk reads reached peak while writes were low. If it was due to heavy swap usage, writes should have been higher. Now it seems it was because VSCode was continuously reading file changes. Since I’ve made too many changes to the system, I’ve reset my cloud server and plan to start the installation from scratch. Thanks to everyone who helped me! It was all my fault.

Now I have successfully built the app, and the maximum memory usage during the build was 2GB RAM + approximately 2GB swap. There were some lags in the SSH connection during the build, but no disconnections, and overall it was relatively fast. During the peak swap usage, it did indeed reach the maximum read speed limit of my ESSD, but it was very brief and did not cause significant impact.

3 Likes

Congratulations, Tumbleweed @tumbleweed !! :sparkler: :sparkles: :tada: :rainbow:

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.