Hi everyone, My Discourse is currently offline :cry: :pensive: It failed after trying to upgrade this morning from/to v3.2.0.beta3 +468: [image] Comparing... A platform for community discussion. Free, open, simple. - Comparing aadc104817bb0306856b4718f5c50fb481e77c16...4949d85c15be99abb3515e659ef674d58cc2c4c4 · discourse/discourse On the command line, the error message is: And the full console log: root@greyarro:~# cd /var/discourse/ root@greyarro:/var/discourse# git pull Already up to date. root@greyarro:/var/discourse# ./launcher rebuild app x86_64 arch detected. WARNING: containers/app.yml file is world-readable. You can secure this file by running: chmod o-rwx containers/app.yml Ensuring launcher is up to date Fetching origin Launcher is up-to-date 2.0.20231023-1945: Pulling from discourse/base Digest: sha256:2b0eb484d20888cc2daadb690dcfa73522105650c1420212e99345a36a424d77 Status: Image is up to date for discourse/base:2.0.20231023-1945 /usr/local/lib/ruby/gems/3.2.0/gems/pups-1.2.1/lib/pups.rb /usr/local/bin/pups --stdin I, [2023-12-11T06:36:42.453494 #1] INFO -- : Reading from stdin I, [2023-12-11T06:36:42.461560 #1] INFO -- : > locale-gen $LANG && update-locale I, [2023-12-11T06:36:42.523575 #1] INFO -- : Generating locales (this might take a while)... Generation complete. I, [2023-12-11T06:36:42.524082 #1] INFO -- : > mkdir -p /shared/postgres_run I, [2023-12-11T06:36:42.529018 #1] INFO -- : I, [2023-12-11T06:36:42.529478 #1] INFO -- : > chown postgres:postgres /shared/postgres_run I, [2023-12-11T06:36:42.534085 #1] INFO -- : I, [2023-12-11T06:36:42.534610 #1] INFO -- : > chmod 775 /shared/postgres_run I, [2023-12-11T06:36:42.538665 #1] INFO -- : I, [2023-12-11T06:36:42.539119 #1] INFO -- : > rm -fr /var/run/postgresql I, [2023-12-11T06:36:42.542429 #1] INFO -- : I, [2023-12-11T06:36:42.542918 #1] INFO -- : > ln -s /shared/postgres_run /var/run/postgresql I, [2023-12-11T06:36:42.545917 #1] INFO -- : I, [2023-12-11T06:36:42.546272 #1] INFO -- : > socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1 2023/12/11 06:36:42 socat[17] E connect(6, AF=1 "/shared/postgres_run/.s.PGSQL.5432", 36): No such file or directory I, [2023-12-11T06:36:42.557289 #1] INFO -- : I, [2023-12-11T06:36:42.557926 #1] INFO -- : > rm -fr /shared/postgres_run/.s* I, [2023-12-11T06:36:42.562906 #1] INFO -- : I, [2023-12-11T06:36:42.563405 #1] INFO -- : > rm -fr /shared/postgres_run/*.pid I, [2023-12-11T06:36:42.568125 #1] INFO -- : I, [2023-12-11T06:36:42.568537 #1] INFO -- : > mkdir -p /shared/postgres_run/13-main.pg_stat_tmp I, [2023-12-11T06:36:42.571897 #1] INFO -- : I, [2023-12-11T06:36:42.572328 #1] INFO -- : > chown postgres:postgres /shared/postgres_run/13-main.pg_stat_tmp I, [2023-12-11T06:36:42.575429 #1] INFO -- : I, [2023-12-11T06:36:42.583069 #1] INFO -- : File > /etc/service/postgres/run chmod: +x chown: I, [2023-12-11T06:36:42.591097 #1] INFO -- : File > /etc/service/postgres/log/run chmod: +x chown: I, [2023-12-11T06:36:42.599557 #1] INFO -- : File > /etc/runit/3.d/99-postgres chmod: +x chown: I, [2023-12-11T06:36:42.607693 #1] INFO -- : File > /root/upgrade_postgres chmod: +x chown: I, [2023-12-11T06:36:42.608293 #1] INFO -- : > chown -R root /var/lib/postgresql/13/main I, [2023-12-11T06:36:43.061308 #1] INFO -- : I, [2023-12-11T06:36:43.061769 #1] INFO -- : > [ ! -e /shared/postgres_data ] && install -d -m 0755 -o postgres -g postgres /shared/postgres_data && sudo -E -u postgres /usr/lib/postgresql/13/bin/initdb -D /shared/postgres_data || exit 0 I, [2023-12-11T06:36:43.066134 #1] INFO -- : I, [2023-12-11T06:36:43.066571 #1] INFO -- : > chown -R postgres:postgres /shared/postgres_data I, [2023-12-11T06:36:43.105049 #1] INFO -- : I, [2023-12-11T06:36:43.105891 #1] INFO -- : > chown -R postgres:postgres /var/run/postgresql I, [2023-12-11T06:36:43.111420 #1] INFO -- : I, [2023-12-11T06:36:43.112072 #1] INFO -- : > /root/upgrade_postgres I, [2023-12-11T06:36:43.122177 #1] INFO -- : I, [2023-12-11T06:36:43.122713 #1] INFO -- : > rm /root/upgrade_postgres I, [2023-12-11T06:36:43.126702 #1] INFO -- : I, [2023-12-11T06:36:43.127981 #1] INFO -- : Replacing data_directory = '/var/lib/postgresql/13/main' with data_directory = '/shared/postgres_data' in /etc/postgresql/13/main/postgresql.conf I, [2023-12-11T06:36:43.128927 #1] INFO -- : Replacing (?-mix:#?listen_addresses *=.*) with listen_addresses = '*' in /etc/postgresql/13/main/postgresql.conf I, [2023-12-11T06:36:43.129583 #1] INFO -- : Replacing (?-mix:#?synchronous_commit *=.*) with synchronous_commit = $db_synchronous_commit in /etc/postgresql/13/main/postgresql.conf I, [2023-12-11T06:36:43.130214 #1] INFO -- : Replacing (?-mix:#?shared_buffers *=.*) with shared_buffers = $db_shared_buffers in /etc/postgresql/13/main/postgresql.conf I, [2023-12-11T06:36:43.130899 #1] INFO -- : Replacing (?-mix:#?work_mem *=.*) with work_mem = $db_work_mem in /etc/postgresql/13/main/postgresql.conf I, [2023-12-11T06:36:43.131457 #1] INFO -- : Replacing (?-mix:#?default_text_search_config *=.*) with default_text_search_config = '$db_default_text_search_config' in /etc/postgresql/13/main/postgresql.conf I, [2023-12-11T06:36:43.132096 #1] INFO -- : > install -d -m 0755 -o postgres -g postgres /shared/postgres_backup I, [2023-12-11T06:36:43.137173 #1] INFO -- : I, [2023-12-11T06:36:43.137747 #1] INFO -- : Replacing (?-mix:#?checkpoint_segments *=.*) with checkpoint_segments = $db_checkpoint_segments in /etc/postgresql/13/main/postgresql.conf I, [2023-12-11T06:36:43.138339 #1] INFO -- : Replacing (?-mix:#?logging_collector *=.*) with logging_collector = $db_logging_collector in /etc/postgresql/13/main/postgresql.conf I, [2023-12-11T06:36:43.139065 #1] INFO -- : Replacing (?-mix:#?log_min_duration_statement *=.*) with log_min_duration_statement = $db_log_min_duration_statement in /etc/postgresql/13/main/postgresql.conf I, [2023-12-11T06:36:43.140177 #1] INFO -- : Replacing (?-mix:^#local +replication +postgres +peer$) with local replication postgres peer in /etc/postgresql/13/main/pg_hba.conf I, [2023-12-11T06:36:43.140923 #1] INFO -- : Replacing (?-mix:^host.*all.*all.*127.*$) with host all all 0.0.0.0/0 md5 in /etc/postgresql/13/main/pg_hba.conf I, [2023-12-11T06:36:43.141403 #1] INFO -- : Replacing (?-mix:^host.*all.*all.*::1\/128.*$) with host all all ::/0 md5 in /etc/postgresql/13/main/pg_hba.conf I, [2023-12-11T06:36:43.141961 #1] INFO -- : > HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main I, [2023-12-11T06:36:43.144661 #1] INFO -- : > sleep 5 2023-12-11 06:36:43.213 UTC [40] LOG: starting PostgreSQL 13.12 (Debian 13.12-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit 2023-12-11 06:36:43.213 UTC [40] LOG: listening on IPv4 address "0.0.0.0", port 5432 2023-12-11 06:36:43.213 UTC [40] LOG: listening on IPv6 address "::", port 5432 2023-12-11 06:36:43.217 UTC [40] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" 2023-12-11 06:36:43.223 UTC [43] LOG: database system was shut down at 2023-12-11 06:33:31 UTC 2023-12-11 06:36:43.236 UTC [40] LOG: database system is ready to accept connections I, [2023-12-11T06:36:48.151212 #1] INFO -- : I, [2023-12-11T06:36:48.151469 #1] INFO -- : > su postgres -c 'createdb discourse' || true 2023-12-11 06:36:48.241 UTC [53] postgres@postgres ERROR: database "discourse" already exists 2023-12-11 06:36:48.241 UTC [53] postgres@postgres STATEMENT: CREATE DATABASE discourse; createdb: error: database creation failed: ERROR: database "discourse" already exists I, [2023-12-11T06:36:48.244084 #1] INFO -- : I, [2023-12-11T06:36:48.244295 #1] INFO -- : > su postgres -c 'psql discourse -c "create user discourse;"' || true 2023-12-11 06:36:48.310 UTC [57] postgres@discourse ERROR: role "discourse" already exists 2023-12-11 06:36:48.310 UTC [57] postgres@discourse STATEMENT: create user discourse; ERROR: role "discourse" already exists I, [2023-12-11T06:36:48.313540 #1] INFO -- : I, [2023-12-11T06:36:48.313932 #1] INFO -- : > su postgres -c 'psql discourse -c "grant all privileges on database discourse to discourse;"' || true I, [2023-12-11T06:36:48.372120 #1] INFO -- : GRANT I, [2023-12-11T06:36:48.372374 #1] INFO -- : > su postgres -c 'psql discourse -c "alter schema public owner to discourse;"' I, [2023-12-11T06:36:48.428450 #1] INFO -- : ALTER SCHEMA I, [2023-12-11T06:36:48.428697 #1] INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists hstore;"' NOTICE: extension "hstore" already exists, skipping I, [2023-12-11T06:36:48.509937 #1] INFO -- : CREATE EXTENSION I, [2023-12-11T06:36:48.510198 #1] INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists pg_trgm;"' NOTICE: extension "pg_trgm" already exists, skipping I, [2023-12-11T06:36:48.575901 #1] INFO -- : CREATE EXTENSION I, [2023-12-11T06:36:48.576560 #1] INFO -- : > su postgres -c 'psql template1 -c "create extension if not exists vector;"' NOTICE: extension "vector" already exists, skipping I, [2023-12-11T06:36:48.634184 #1] INFO -- : CREATE EXTENSION I, [2023-12-11T06:36:48.634407 #1] INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists hstore;"' NOTICE: extension "hstore" already exists, skipping I, [2023-12-11T06:36:48.691197 #1] INFO -- : CREATE EXTENSION I, [2023-12-11T06:36:48.691402 #1] INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists pg_trgm;"' NOTICE: extension "pg_trgm" already exists, skipping I, [2023-12-11T06:36:48.747583 #1] INFO -- : CREATE EXTENSION I, [2023-12-11T06:36:48.747801 #1] INFO -- : > su postgres -c 'psql discourse -c "create extension if not exists vector;"' NOTICE: extension "vector" already exists, skipping I, [2023-12-11T06:36:48.805422 #1] INFO -- : CREATE EXTENSION I, [2023-12-11T06:36:48.805817 #1] INFO -- : > sudo -u postgres psql discourse I, [2023-12-11T06:36:48.808688 #1] INFO -- : update pg_database set encoding = pg_char_to_encoding('UTF8') where datname = 'discourse' AND encoding = pg_char_to_encoding('SQL_ASCII'); I, [2023-12-11T06:36:48.891679 #1] INFO -- : File > /var/lib/postgresql/take-database-backup chmod: +x chown: postgres:postgres I, [2023-12-11T06:36:48.896830 #1] INFO -- : File > /var/spool/cron/crontabs/postgres chmod: chown: I, [2023-12-11T06:36:48.896998 #1] INFO -- : > echo postgres installed! I, [2023-12-11T06:36:48.899962 #1] INFO -- : postgres installed! I, [2023-12-11T06:36:48.907627 #1] INFO -- : File > /etc/service/redis/run chmod: +x chown: I, [2023-12-11T06:36:48.914411 #1] INFO -- : File > /etc/service/redis/log/run chmod: +x chown: I, [2023-12-11T06:36:48.920982 #1] INFO -- : File > /etc/runit/3.d/10-redis chmod: +x chown: I, [2023-12-11T06:36:48.922023 #1] INFO -- : Replacing daemonize yes with in /etc/redis/redis.conf I, [2023-12-11T06:36:48.922829 #1] INFO -- : Replacing (?-mix:^pidfile.*$) with in /etc/redis/redis.conf I, [2023-12-11T06:36:48.923923 #1] INFO -- : > install -d -m 0755 -o redis -g redis /shared/redis_data I, [2023-12-11T06:36:48.928742 #1] INFO -- : I, [2023-12-11T06:36:48.929799 #1] INFO -- : Replacing (?-mix:^logfile.*$) with logfile "" in /etc/redis/redis.conf I, [2023-12-11T06:36:48.930627 #1] INFO -- : Replacing (?-mix:^bind .*$) with in /etc/redis/redis.conf I, [2023-12-11T06:36:48.931845 #1] INFO -- : Replacing (?-mix:^dir .*$) with dir /shared/redis_data in /etc/redis/redis.conf I, [2023-12-11T06:36:48.933205 #1] INFO -- : Replacing (?-mix:^protected-mode yes) with protected-mode no in /etc/redis/redis.conf I, [2023-12-11T06:36:48.934373 #1] INFO -- : Replacing # io-threads 4 with io-threads $redis_io_threads in /etc/redis/redis.conf I, [2023-12-11T06:36:48.935362 #1] INFO -- : > echo redis installed I, [2023-12-11T06:36:48.939050 #1] INFO -- : redis installed I, [2023-12-11T06:36:48.939481 #1] INFO -- : > cat /etc/redis/redis.conf | grep logfile I, [2023-12-11T06:36:48.947715 #1] INFO -- : logfile "" I, [2023-12-11T06:36:48.948224 #1] INFO -- : > exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf I, [2023-12-11T06:36:48.950934 #1] INFO -- : > sleep 10 109:C 11 Dec 2023 06:36:48.962 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 109:C 11 Dec 2023 06:36:48.962 # Redis version=7.0.7, bits=64, commit=00000000, modified=0, pid=109, just started 109:C 11 Dec 2023 06:36:48.962 # Configuration loaded 109:M 11 Dec 2023 06:36:48.963 * monotonic clock: POSIX clock_gettime 109:M 11 Dec 2023 06:36:48.964 * Running mode=standalone, port=6379. 109:M 11 Dec 2023 06:36:48.964 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128. 109:M 11 Dec 2023 06:36:48.964 # Server initialized 109:M 11 Dec 2023 06:36:48.964 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. 109:M 11 Dec 2023 06:36:48.965 * Loading RDB produced by version 7.0.7 109:M 11 Dec 2023 06:36:48.965 * RDB age 196 seconds 109:M 11 Dec 2023 06:36:48.965 * RDB memory usage when created 238.89 Mb 109:M 11 Dec 2023 06:36:49.836 * Done loading RDB, keys loaded: 78979, keys expired: 55. 109:M 11 Dec 2023 06:36:49.836 * DB loaded from disk: 0.871 seconds 109:M 11 Dec 2023 06:36:49.836 * Ready to accept connections I, [2023-12-11T06:36:58.954574 #1] INFO -- : I, [2023-12-11T06:36:58.955135 #1] INFO -- : > thpoff echo "thpoff is installed!" I, [2023-12-11T06:36:58.960865 #1] INFO -- : thpoff is installed! I, [2023-12-11T06:36:58.961207 #1] INFO -- : > /usr/local/bin/ruby -e 'if ENV["DISCOURSE_SMTP_ADDRESS"] == "smtp.example.com"; puts "Aborting! Mail is not configured!"; exit 1; end' I, [2023-12-11T06:36:59.059159 #1] INFO -- : I, [2023-12-11T06:36:59.059736 #1] INFO -- : > /usr/local/bin/ruby -e 'if ENV["DISCOURSE_HOSTNAME"] == "discourse.example.com"; puts "Aborting! Domain is not configured!"; exit 1; end' I, [2023-12-11T06:36:59.148371 #1] INFO -- : I, [2023-12-11T06:36:59.149030 #1] INFO -- : > /usr/local/bin/ruby -e 'if (ENV["DISCOURSE_CDN_URL"] || "")[0..1] == "//"; puts "Aborting! CDN must have a protocol specified. Once fixed you should rebake your posts now to correct all posts."; exit 1; end' I, [2023-12-11T06:36:59.236730 #1] INFO -- : I, [2023-12-11T06:36:59.237387 #1] INFO -- : > rm -f /etc/cron.d/anacron I, [2023-12-11T06:36:59.242323 #1] INFO -- : I, [2023-12-11T06:36:59.247436 #1] INFO -- : File > /etc/cron.d/anacron chmod: chown: I, [2023-12-11T06:36:59.254458 #1] INFO -- : File > /etc/runit/1.d/copy-env chmod: +x chown: I, [2023-12-11T06:36:59.261375 #1] INFO -- : File > /etc/service/unicorn/run chmod: +x chown: I, [2023-12-11T06:36:59.268779 #1] INFO -- : File > /etc/service/nginx/run chmod: +x chown: I, [2023-12-11T06:36:59.276427 #1] INFO -- : File > /etc/runit/3.d/01-nginx chmod: +x chown: I, [2023-12-11T06:36:59.283178 #1] INFO -- : File > /etc/runit/3.d/02-unicorn chmod: +x chown: I, [2023-12-11T06:36:59.283492 #1] INFO -- : Replacing # postgres with sv start postgres || exit 1 in /etc/service/unicorn/run I, [2023-12-11T06:36:59.284057 #1] INFO -- : > exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf I, [2023-12-11T06:36:59.287039 #1] INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git reset --hard 137:C 11 Dec 2023 06:36:59.295 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 137:C 11 Dec 2023 06:36:59.295 # Redis version=7.0.7, bits=64, commit=00000000, modified=0, pid=137, just started 137:C 11 Dec 2023 06:36:59.295 # Configuration loaded 137:M 11 Dec 2023 06:36:59.295 * monotonic clock: POSIX clock_gettime 137:M 11 Dec 2023 06:36:59.295 # Warning: Could not create server TCP listening socket *:6379: bind: Address already in use 137:M 11 Dec 2023 06:36:59.295 # Failed listening on port 6379 (TCP), aborting. Updating files: 100% (33858/33858), done. I, [2023-12-11T06:37:03.209327 #1] INFO -- : HEAD is now at 043b4a418 FIX: replaces data-tooltip usage by (#24062) I, [2023-12-11T06:37:03.209496 #1] INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git clean -f I, [2023-12-11T06:37:03.416955 #1] INFO -- : I, [2023-12-11T06:37:03.417150 #1] INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse bash -c ' set -o errexit if [ $(git rev-parse --is-shallow-repository) == "true" ]; then git remote set-branches --add origin main git remote set-branches origin tests-passed git fetch --depth 1 origin tests-passed else git fetch --tags --prune-tags --prune --force origin fi ' From https://github.com/discourse/discourse - [deleted] (none) -> origin/0-flake-key - [deleted] (none) -> origin/0-grant-badge-fix - [deleted] (none) -> origin/0-obscuring - [deleted] (none) -> origin/0-router-lint - [deleted] (none) -> origin/0-router-service - [deleted] (none) -> origin/a-smokes-test - [deleted] (none) -> origin/autoimport-cleanup - [deleted] (none) -> origin/category-bullet-only - [deleted] (none) -> origin/dependabot/bundler/syntax_tree-6.2.0 - [deleted] (none) -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/aws-s3-3.4.0 - [deleted] (none) -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/aws-s3-multipart-3.8.0 - [deleted] (none) -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/core-3.6.0 - [deleted] (none) -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/utils-5.5.2 - [deleted] (none) -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/xhr-upload-3.4.2 - [deleted] (none) -> origin/dev-upgrade-reject-reason-reviewable-modal - [deleted] (none) -> origin/dev/ability-to-collect-stats-without-public-exposing - [deleted] (none) -> origin/dev/minio-s3-system-specs - [deleted] (none) -> origin/dev/re-enable-minio-specs - [deleted] (none) -> origin/dev/switch-over-category-settings - [deleted] (none) -> origin/dev/switch-over-more-category-settings - [deleted] (none) -> origin/discovery-named-outlets - [deleted] (none) -> origin/feature/theme-settings-migration - [deleted] (none) -> origin/fix/account-for-empty-string-site-settings - [deleted] (none) -> origin/footnote-plugin - [deleted] (none) -> origin/metadata-update - [deleted] (none) -> origin/modal-ux - [deleted] (none) -> origin/neil-2 - [deleted] (none) -> origin/perf_cache_async_search - [deleted] (none) -> origin/redo-post-focus-setting - [deleted] (none) -> origin/spoiler-plugin - [deleted] (none) -> origin/support-search-customizationz 043b4a418..4949d85c1 main -> origin/main * [new branch] 0-class-attribute -> origin/0-class-attribute * [new branch] 0-for-attributes -> origin/0-for-attributes * [new branch] 0-i18n-case -> origin/0-i18n-case * [new branch] add-custom-emoji-sanitization-clean -> origin/add-custom-emoji-sanitization-clean * [new branch] add-ds-store-to-gitignore -> origin/add-ds-store-to-gitignore + 64520a075...ff17131c9 add-new-chat-indicator-preference-only-mentions -> origin/add-new-chat-indicator-preference-only-mentions (forced update) * [new branch] api-topic-title-decorator -> origin/api-topic-title-decorator * [new branch] array-custom-field-order -> origin/array-custom-field-order * [new branch] boosted-searches -> origin/boosted-searches * [new branch] bulk-button-class -> origin/bulk-button-class * [new branch] category_definition -> origin/category_definition * [new branch] chat-github-onebox -> origin/chat-github-onebox * [new branch] chat-reaction-emoji-alignment -> origin/chat-reaction-emoji-alignment * [new branch] chat/direct-messages -> origin/chat/direct-messages * [new branch] confirm-session-only-after-5-mins -> origin/confirm-session-only-after-5-mins * [new branch] custom-field-store-cleanup -> origin/custom-field-store-cleanup * [new branch] delete-backups-based-on-time-window -> origin/delete-backups-based-on-time-window * [new branch] dependabot/github_actions/actions/labeler-5 -> origin/dependabot/github_actions/actions/labeler-5 * [new branch] dependabot/github_actions/actions/stale-9 -> origin/dependabot/github_actions/actions/stale-9 * [new branch] dependabot/npm_and_yarn/app/assets/javascripts/uppy/aws-s3-3.5.0 -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/aws-s3-3.5.0 * [new branch] dependabot/npm_and_yarn/app/assets/javascripts/uppy/aws-s3-multipart-3.9.0 -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/aws-s3-multipart-3.9.0 * [new branch] dependabot/npm_and_yarn/app/assets/javascripts/uppy/core-3.7.1 -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/core-3.7.1 * [new branch] dependabot/npm_and_yarn/app/assets/javascripts/uppy/utils-5.6.0 -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/utils-5.6.0 * [new branch] dependabot/npm_and_yarn/app/assets/javascripts/uppy/xhr-upload-3.5.0 -> origin/dependabot/npm_and_yarn/app/assets/javascripts/uppy/xhr-upload-3.5.0 * [new branch] desktop-notification-handlers -> origin/desktop-notification-handlers * [new branch] dev/add-a-reference-column-to-notifications -> origin/dev/add-a-reference-column-to-notifications + ee95e6359...a2b413786 dev/add-new-chat-metrics -> origin/dev/add-new-chat-metrics (forced update) * [new branch] dev/drop-notification_id-column-on-chat-mentions -> origin/dev/drop-notification_id-column-on-chat-mentions * [new branch] dev/dry-up-user-status-serializers -> origin/dev/dry-up-user-status-serializers * [new branch] dev/flaky-chat-details-collapser-spec -> origin/dev/flaky-chat-details-collapser-spec * [new branch] dev/no-need-to-include-user-status-twice -> origin/dev/no-need-to-include-user-status-twice * [new branch] dev/remove-legacy-hamburger-menu -> origin/dev/remove-legacy-hamburger-menu * [new branch] dev/rethink-mass-mentions -> origin/dev/rethink-mass-mentions * [new branch] dev/rethink-mass-mentions-polymorphic -> origin/dev/rethink-mass-mentions-polymorphic * [new branch] dev/revert-guardian-devex -> origin/dev/revert-guardian-devex + 54940ca5c...3eacfe150 dev_paginate_categories_json -> origin/dev_paginate_categories_json (forced update) * [new branch] dev_watched_words -> origin/dev_watched_words + 46fe4511f...a9d0e7069 ember-4 -> origin/ember-4 (forced update) * [new branch] ember-5-plugins -> origin/ember-5-plugins * [new branch] extract_stripper -> origin/extract_stripper * [new branch] feature/admin-sidebar-v0 -> origin/feature/admin-sidebar-v0 9a6c6fd5b..8780daeee feature/bookmark-menu-and-modal-redesign -> origin/feature/bookmark-menu-and-modal-redesign * [new branch] feature/migrations-schema-generator -> origin/feature/migrations-schema-generator * [new branch] feature_watched_words_validation -> origin/feature_watched_words_validation * [new branch] file-size-restriction -> origin/file-size-restriction 43df37774..4393dd37e filtered-events -> origin/filtered-events * [new branch] fix/count-mass-mentions-correctly -> origin/fix/count-mass-mentions-correctly * [new branch] full-page -> origin/full-page ce7fb6906..31e32f8a1 generic-import-latest -> origin/generic-import-latest * [new branch] generic-import-latest-snowflake -> origin/generic-import-latest-snowflake * [new branch] generic-import-update -> origin/generic-import-update * [new branch] image-assets -> origin/image-assets * [new branch] independent-theme-compiler -> origin/independent-theme-compiler * [new branch] input-text-field-component -> origin/input-text-field-component * [new branch] issue/fix-guardian-init-basic-user -> origin/issue/fix-guardian-init-basic-user * [new branch] keyword-site-setting -> origin/keyword-site-setting + bdcc24f96...92e545292 list-wrapper -> origin/list-wrapper (forced update) e3636078c..5def32699 mock-chat-index-redesign -> origin/mock-chat-index-redesign * [new branch] move_inflector -> origin/move_inflector * [new branch] multi-invite -> origin/multi-invite * [new branch] noti-glimmer -> origin/noti-glimmer * [new branch] oklab -> origin/oklab * [new branch] poc/auto-after-before-plugin-outlets -> origin/poc/auto-after-before-plugin-outlets * [new branch] poc/insert-before-after-wrapping-outlet -> origin/poc/insert-before-after-wrapping-outlet * [new branch] pretty-text-please -> origin/pretty-text-please * [new branch] reload-page-user-prefs-2fa -> origin/reload-page-user-prefs-2fa * [new branch] revert-maxmind -> origin/revert-maxmind ef3f41543..223626d7a stable -> origin/stable * [new branch] support-load-more-btn -> origin/support-load-more-btn * [new branch] test-issue-temporary -> origin/test-issue-temporary 043b4a418..4949d85c1 tests-passed -> origin/tests-passed * [new branch] tmp/reimplement-all-and-here-mentions -> origin/tmp/reimplement-all-and-here-mentions * [new branch] tobiaseigen-patch-3 -> origin/tobiaseigen-patch-3 * [new branch] transform-hooks -> origin/transform-hooks * [new branch] transform-post-fn-api -> origin/transform-post-fn-api * [new branch] tweaks-2-fr -> origin/tweaks-2-fr * [new branch] username_cache -> origin/username_cache * [new branch] ux-add-sorting-params-to-outlet -> origin/ux-add-sorting-params-to-outlet t [tag update] beta -> beta t [tag update] latest-release -> latest-release * [new tag] v3.1.3 -> v3.1.3 * [new tag] v3.2.0.beta3 -> v3.2.0.beta3 I, [2023-12-11T06:37:05.107220 #1] INFO -- : I, [2023-12-11T06:37:05.107337 #1] INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse bash -c ' set -o errexit if [[ $(git symbolic-ref --short HEAD) == tests-passed ]] ; then git pull else git -c advice.detachedHead=false checkout tests-passed fi ' Switched to a new branch 'tests-passed' I, [2023-12-11T06:37:10.039336 #1] INFO -- : Branch 'tests-passed' set up to track remote branch 'tests-passed' from 'origin'. I, [2023-12-11T06:37:10.039485 #1] INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git config user.discourse-version tests-passed I, [2023-12-11T06:37:10.055796 #1] INFO -- : I, [2023-12-11T06:37:10.055932 #1] INFO -- : > cd /var/www/discourse && mkdir -p tmp I, [2023-12-11T06:37:10.062383 #1] INFO -- : I, [2023-12-11T06:37:10.062911 #1] INFO -- : > cd /var/www/discourse && chown discourse:www-data tmp I, [2023-12-11T06:37:10.068906 #1] INFO -- : I, [2023-12-11T06:37:10.069051 #1] INFO -- : > cd /var/www/discourse && mkdir -p tmp/pids I, [2023-12-11T06:37:10.075337 #1] INFO -- : I, [2023-12-11T06:37:10.075478 #1] INFO -- : > cd /var/www/discourse && mkdir -p tmp/sockets I, [2023-12-11T06:37:10.081087 #1] INFO -- : I, [2023-12-11T06:37:10.081222 #1] INFO -- : > cd /var/www/discourse && touch tmp/.gitkeep I, [2023-12-11T06:37:10.087004 #1] INFO -- : I, [2023-12-11T06:37:10.087139 #1] INFO -- : > cd /var/www/discourse && mkdir -p /shared/log/rails I, [2023-12-11T06:37:10.093140 #1] INFO -- : I, [2023-12-11T06:37:10.093305 #1] INFO -- : > cd /var/www/discourse && bash -c "touch -a /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log" I, [2023-12-11T06:37:10.100413 #1] INFO -- : I, [2023-12-11T06:37:10.100570 #1] INFO -- : > cd /var/www/discourse && bash -c "ln -s /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log /var/www/discourse/log" I, [2023-12-11T06:37:10.108011 #1] INFO -- : I, [2023-12-11T06:37:10.108130 #1] INFO -- : > cd /var/www/discourse && bash -c "mkdir -p /shared/{uploads,backups}" I, [2023-12-11T06:37:10.116596 #1] INFO -- : I, [2023-12-11T06:37:10.116726 #1] INFO -- : > cd /var/www/discourse && bash -c "ln -s /shared/{uploads,backups} /var/www/discourse/public" I, [2023-12-11T06:37:10.124273 #1] INFO -- : I, [2023-12-11T06:37:10.124421 #1] INFO -- : > cd /var/www/discourse && bash -c "mkdir -p /shared/tmp/{backups,restores}" I, [2023-12-11T06:37:10.134061 #1] INFO -- : I, [2023-12-11T06:37:10.134228 #1] INFO -- : > cd /var/www/discourse && bash -c "ln -s /shared/tmp/{backups,restores} /var/www/discourse/tmp" I, [2023-12-11T06:37:10.141458 #1] INFO -- : I, [2023-12-11T06:37:10.141602 #1] INFO -- : > cd /var/www/discourse && chown -R discourse:www-data /shared/log/rails /shared/uploads /shared/backups /shared/tmp I, [2023-12-11T06:37:10.163707 #1] INFO -- : I, [2023-12-11T06:37:10.163854 #1] INFO -- : > cd /var/www/discourse && [ ! -d public/plugins ] || find public/plugins/ -maxdepth 1 -xtype l -delete I, [2023-12-11T06:37:10.169471 #1] INFO -- : I, [2023-12-11T06:37:10.170368 #1] INFO -- : Replacing # redis with sv start redis || exit 1 in /etc/service/unicorn/run I, [2023-12-11T06:37:10.171519 #1] INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/docker_manager.git Cloning into 'docker_manager'... I, [2023-12-11T06:37:11.142485 #1] INFO -- : I, [2023-12-11T06:37:11.142680 #1] INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/discourse-solved.git Cloning into 'discourse-solved'... I, [2023-12-11T06:37:11.877496 #1] INFO -- : I, [2023-12-11T06:37:11.878167 #1] INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/paviliondev/discourse-locations.git Cloning into 'discourse-locations'... I, [2023-12-11T06:37:12.713978 #1] INFO -- : I, [2023-12-11T06:37:12.714105 #1] INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/discourse-data-explorer.git Cloning into 'discourse-data-explorer'... I, [2023-12-11T06:37:13.532864 #1] INFO -- : I, [2023-12-11T06:37:13.533001 #1] INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/discourse-adplugin.git Cloning into 'discourse-adplugin'... I, [2023-12-11T06:37:14.305555 #1] INFO -- : I, [2023-12-11T06:37:14.305817 #1] INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/discourse-reactions.git Cloning into 'discourse-reactions'... I, [2023-12-11T06:37:15.045125 #1] INFO -- : I, [2023-12-11T06:37:15.045247 #1] INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/discourse-templates.git Cloning into 'discourse-templates'... I, [2023-12-11T06:37:15.809915 #1] INFO -- : I, [2023-12-11T06:37:15.810046 #1] INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/discourse-affiliate.git Cloning into 'discourse-affiliate'... I, [2023-12-11T06:37:16.433061 #1] INFO -- : I, [2023-12-11T06:37:16.434123 #1] INFO -- : > cp /var/www/discourse/config/nginx.sample.conf /etc/nginx/conf.d/discourse.conf I, [2023-12-11T06:37:16.442090 #1] INFO -- : I, [2023-12-11T06:37:16.442586 #1] INFO -- : > rm /etc/nginx/sites-enabled/default I, [2023-12-11T06:37:16.446759 #1] INFO -- : I, [2023-12-11T06:37:16.447263 #1] INFO -- : > mkdir -p /var/nginx/cache I, [2023-12-11T06:37:16.451474 #1] INFO -- : I, [2023-12-11T06:37:16.452405 #1] INFO -- : Replacing pid /run/nginx.pid; with daemon off; in /etc/nginx/nginx.conf I, [2023-12-11T06:37:16.453144 #1] INFO -- : Replacing (?m-ix:upstream[^\}]+\}) with upstream discourse { server 127.0.0.1:3000; } in /etc/nginx/conf.d/discourse.conf I, [2023-12-11T06:37:16.453873 #1] INFO -- : Replacing (?-mix:server_name.+$) with server_name _ ; in /etc/nginx/conf.d/discourse.conf I, [2023-12-11T06:37:16.454418 #1] INFO -- : Replacing (?-mix:client_max_body_size.+$) with client_max_body_size $upload_size ; in /etc/nginx/conf.d/discourse.conf I, [2023-12-11T06:37:16.454991 #1] INFO -- : Replacing (?-mix:worker_connections.+$) with worker_connections $nginx_worker_connections ; in /etc/nginx/nginx.conf I, [2023-12-11T06:37:16.455481 #1] INFO -- : > echo "done configuring web" I, [2023-12-11T06:37:16.459601 #1] INFO -- : done configuring web I, [2023-12-11T06:37:16.460326 #1] INFO -- : > cd /var/www/discourse && gem install bundler --conservative -v $(awk '/BUNDLED WITH/ { getline; gsub(/ /,""); print $0 }' Gemfile.lock) I, [2023-12-11T06:37:16.836806 #1] INFO -- : I, [2023-12-11T06:37:16.836974 #1] INFO -- : > cd /var/www/discourse && find /var/www/discourse ! -user discourse -exec chown discourse {} \+ I, [2023-12-11T06:37:19.438303 #1] INFO -- : I, [2023-12-11T06:37:19.438984 #1] INFO -- : > cd /var/www/discourse && if [ "tests-passed" != "tests-passed" ]; then rm -rf app/assets/javascripts/node_modules fi I, [2023-12-11T06:37:19.442692 #1] INFO -- : I, [2023-12-11T06:37:19.442789 #1] INFO -- : > cd /var/www/discourse && su discourse -c 'yarn install --frozen-lockfile && yarn cache clean' warning " > @glint/environment-ember-loose@1.1.0" has unmet peer dependency "@glimmer/component@^1.1.2". warning " > @glint/environment-ember-template-imports@1.1.0" has unmet peer dependency "ember-template-imports@^3.0.0". warning Resolution field "unset-value@2.0.1" is incompatible with requested version "unset-value@^1.0.0" warning Pattern ["wrap-ansi@^7.0.0"] is trying to unpack in the same destination "/home/discourse/.cache/yarn/v6/npm-wrap-ansi-cjs-7.0.0-67e145cff510a6a6984bdf1152911d69d2eb9e43-integrity/node_modules/wrap-ansi-cjs" as pattern ["wrap-ansi-cjs@npm:wrap-ansi@^7.0.0"]. This could result in non-deterministic behavior, skipping. warning " > discourse-markdown-it@1.0.0" has unmet peer dependency "xss@*". warning "workspace-aggregator-4a0ff1cd-82ba-483f-8207-6b2c0f0b31fb > discourse > @ember/legacy-built-in-components@0.5.0" has incorrect peer dependency "ember-source@>= 4.8". warning "workspace-aggregator-4a0ff1cd-82ba-483f-8207-6b2c0f0b31fb > discourse > @uppy/aws-s3@3.0.6" has incorrect peer dependency "@uppy/core@^3.1.2". warning "workspace-aggregator-4a0ff1cd-82ba-483f-8207-6b2c0f0b31fb > discourse > @uppy/aws-s3-multipart@3.1.3" has incorrect peer dependency "@uppy/core@^3.1.2". warning "workspace-aggregator-4a0ff1cd-82ba-483f-8207-6b2c0f0b31fb > discourse > @uppy/xhr-upload@3.1.1" has incorrect peer dependency "@uppy/core@^3.1.2". warning "workspace-aggregator-4a0ff1cd-82ba-483f-8207-6b2c0f0b31fb > discourse-plugins > ember-this-fallback@0.4.0" has unmet peer dependency "ember-source@^3.28.11 || ^4.0.0". warning "workspace-aggregator-4a0ff1cd-82ba-483f-8207-6b2c0f0b31fb > discourse > @uppy/aws-s3 > @uppy/xhr-upload@3.3.0" has incorrect peer dependency "@uppy/core@^3.2.1". I, [2023-12-11T06:39:22.730680 #1] INFO -- : yarn install v1.22.19 [1/5] Validating package.json... [2/5] Resolving packages... [3/5] Fetching packages... [4/5] Linking dependencies... [5/5] Building fresh packages... $ yarn --cwd app/assets/javascripts $(node -e 'const argv = JSON.parse(process.env.npm_config_argv).original; const passthrough = [`--frozen-lockfile`, `-s`].filter(arg => argv.includes(arg)); console.log(passthrough.join(` `));') yarn install v1.22.19 [1/4] Resolving packages... [2/4] Fetching packages... [3/4] Linking dependencies... [4/4] Building fresh packages... $ ./run-patch-package patch-package 8.0.0 Applying patches... babel-plugin-debug-macros@0.3.4 ✔ content-tag@1.1.2 ✔ ember-cli@5.0.0 ✔ ember-this-fallback@0.4.0 (1 deprecation-name) ✔ ember-this-fallback@0.4.0 (2 themes) ✔ virtual-dom@2.1.1 ✔ Done in 64.07s. Done in 115.18s. yarn cache v1.22.19 success Cleared cache. Done in 7.53s. I, [2023-12-11T06:39:22.731495 #1] INFO -- : > cd /var/www/discourse && su discourse -c 'bundle config --local deployment true' I, [2023-12-11T06:39:23.103216 #1] INFO -- : I, [2023-12-11T06:39:23.103337 #1] INFO -- : > cd /var/www/discourse && su discourse -c 'bundle config --local without "development test"' I, [2023-12-11T06:39:23.331037 #1] INFO -- : You are replacing the current local value of without, which is currently "test:development" I, [2023-12-11T06:39:23.331186 #1] INFO -- : > cd /var/www/discourse && su discourse -c 'bundle install --retry 3 --jobs 4' Gem::Ext::BuildError: ERROR: Failed to build gem native extension. current directory: /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rbtrace-0.5.0/ext /usr/local/bin/ruby extconf.rb -- tar zxvfo msgpack-1.1.0.tar.gz -- env CFLAGS=nil LDFLAGS=nil CC=nil -- ./configure --disable-dependency-tracking --disable-shared --with-pic --prefix=/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rbtrace-0.5.0/ext/dst/ --libdir=/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rbtrace-0.5.0/ext/dst/lib *** extconf.rb failed *** Could not create Makefile due to some reason, probably lack of necessary libraries and/or headers. Check the mkmf.log file for more details. You may need configuration options. Provided configuration options: --with-opt-dir --without-opt-dir --with-opt-include --without-opt-include=${opt-dir}/include --with-opt-lib --without-opt-lib=${opt-dir}/lib --with-make-prog --without-make-prog --srcdir=. --curdir --ruby=/usr/local/bin/$(RUBY_BASE_NAME) extconf.rb:6:in `sys': ./configure --disable-dependency-tracking --disable-shared --with-pic --prefix=/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rbtrace-0.5.0/ext/dst/ --libdir=/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rbtrace-0.5.0/ext/dst/lib failed, please report to https://github.com/tmm1/rbtrace/issues (RuntimeError) from extconf.rb:42:in `block (2 levels) in ' from extconf.rb:34:in `chdir' from extconf.rb:34:in `block in ' from extconf.rb:30:in `chdir' from extconf.rb:30:in ` ' To see why this extension failed to compile, please check the mkmf.log which can be found here: /var/www/discourse/vendor/bundle/ruby/3.2.0/extensions/x86_64-linux/3.2.0/rbtrace-0.5.0/mkmf.log extconf failed, exit code 1 Gem files will remain installed in /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rbtrace-0.5.0 for inspection. Results logged to /var/www/discourse/vendor/bundle/ruby/3.2.0/extensions/x86_64-linux/3.2.0/rbtrace-0.5.0/gem_make.out /usr/local/lib/ruby/site_ruby/3.2.0/rubygems/ext/builder.rb:120:in `run' /usr/local/lib/ruby/site_ruby/3.2.0/rubygems/ext/ext_conf_builder.rb:28:in `build' /usr/local/lib/ruby/site_ruby/3.2.0/rubygems/ext/builder.rb:188:in `build_extension' /usr/local/lib/ruby/site_ruby/3.2.0/rubygems/ext/builder.rb:222:in `block in build_extensions' /usr/local/lib/ruby/site_ruby/3.2.0/rubygems/ext/builder.rb:219:in `each' /usr/local/lib/ruby/site_ruby/3.2.0/rubygems/ext/builder.rb:219:in `build_extensions' /usr/local/lib/ruby/site_ruby/3.2.0/rubygems/installer.rb:844:in `build_extensions' /usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.4.13/lib/bundler/rubygems_gem_installer.rb:72:in `build_extensions' /usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.4.13/lib/bundler/rubygems_gem_installer.rb:28:in `install' /usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.4.13/lib/bundler/source/rubygems.rb:198:in `install' /usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.4.13/lib/bundler/installer/gem_installer.rb:54:in `install' /usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.4.13/lib/bundler/installer/gem_installer.rb:16:in `install_from_spec' /usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.4.13/lib/bundler/installer/parallel_installer.rb:156:in `do_install' /usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.4.13/lib/bundler/installer/parallel_installer.rb:147:in `block in worker_pool' /usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.4.13/lib/bundler/worker.rb:62:in `apply_func' /usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.4.13/lib/bundler/worker.rb:57:in `block in process_queue' /usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.4.13/lib/bundler/worker.rb:54:in `loop' /usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.4.13/lib/bundler/worker.rb:54:in `process_queue' /usr/local/lib/ruby/gems/3.2.0/gems/bundler-2.4.13/lib/bundler/worker.rb:90:in `block (2 levels) in create_threads' An error occurred while installing rbtrace (0.5.0), and Bundler cannot continue. In Gemfile: rbtrace I, [2023-12-11T06:40:09.131584 #1] INFO -- : Fetching gem metadata from https://rubygems.org/......... Fetching rake 13.1.0 Installing rake 13.1.0 Using concurrent-ruby 1.2.2 Using minitest 5.20.0 Using builder 3.2.4 Using erubi 1.12.0 Fetching racc 1.7.3 Using crass 1.0.6 Using rack 2.2.8 Using mini_mime 1.1.5 Fetching date 3.3.4 Fetching public_suffix 5.0.4 Fetching timeout 0.4.1 Installing timeout 0.4.1 Installing public_suffix 5.0.4 Installing date 3.3.4 with native extensions Installing racc 1.7.3 with native extensions Fetching aws-eventstream 1.3.0 Installing aws-eventstream 1.3.0 Using aws-partitions 1.583.0 Using jmespath 1.6.2 Fetching base64 0.2.0 Installing base64 0.2.0 Fetching bigdecimal 3.1.4 Using msgpack 1.7.2 Using bundler 2.4.13 Using byebug 11.1.3 Using cbor 0.5.9.6 Fetching cgi 0.4.1 Installing bigdecimal 3.1.4 with native extensions Installing cgi 0.4.1 with native extensions Using chunky_png 1.4.0 Using coderay 1.1.3 Using colored2 4.0.0 Using connection_pool 2.4.1 Using openssl 3.2.0 Using cppjieba_rb 0.4.2 Using diffy 3.4.2 Using digest 3.1.1 Using discourse-fonts 0.0.9 Using email_reply_trimmer 0.1.13 Fetching excon 0.105.0 Installing excon 0.105.0 Using execjs 2.9.1 Using exifr 1.4.0 Using faraday-net_http 3.0.2 Using ruby2_keywords 0.0.5 Using fast_blank 1.0.1 Using fast_xs 0.8.0 Using fastimage 2.2.7 Using ffi 1.16.3 Using fspath 3.1.2 Using gc_tracer 1.5.1 Fetching google-protobuf 3.25.1 (x86_64-linux) Installing google-protobuf 3.25.1 (x86_64-linux) Using guess_html_encoding 0.0.11 Using hana 1.3.7 Using hashie 5.0.0 Using highline 2.1.0 Using htmlentities 4.3.4 Using http_accept_language 2.1.1 Using image_size 3.3.0 Using in_threads 1.6.0 Using progress 3.6.0 Fetching json 2.7.1 Installing json 2.7.1 with native extensions Fetching regexp_parser 2.8.3 Installing regexp_parser 2.8.3 Fetching unf_ext 0.0.9.1 Installing unf_ext 0.0.9.1 with native extensions Using jwt 2.7.1 Using kgio 2.11.4 Using libv8-node 18.16.0.0 (x86_64-linux) Using method_source 1.0.0 Using thor 1.3.0 Using zeitwerk 2.6.12 Using logstash-event 1.2.02 Fetching logster 2.13.1 Installing logster 2.13.1 Using lru_redux 1.1.0 Using lz4-ruby 0.3.3 Using maxminddb 0.1.22 Using memory_profiler 1.0.1 Using redis 4.8.1 Using mini_sql 1.5.0 Using multi_json 1.15.0 Using multi_xml 0.6.0 Using mustache 1.1.1 Fetching uri 0.13.0 Installing uri 0.13.0 Fetching nio4r 2.7.0 Installing nio4r 2.7.0 with native extensions Using version_gem 1.1.3 Using optimist 3.1.0 Using pg 1.5.4 Using raindrops 0.20.1 Using rchardet 1.8.0 Using rexml 3.2.6 Using rinku 2.0.6 Using rotp 6.3.0 Using rqrcode_core 1.2.0 Using rubyzip 2.3.2 Using sshkey 3.0.0 Using stackprof 0.2.25 Using i18n 1.14.1 Using tzinfo 2.0.6 Using rack-test 2.1.0 Using request_store 1.5.1 Using message_bus 4.3.8 Fetching rack-mini-profiler 3.3.0 Installing rack-mini-profiler 3.3.0 Using rack-protection 3.1.0 Using sprockets 3.7.2 from https://github.com/rails/sprockets (at 3.x@f4d3dae) Fetching net-protocol 0.2.2 Installing net-protocol 0.2.2 Fetching addressable 2.8.6 Installing addressable 2.8.6 Using aws-sigv4 1.5.0 Fetching bootsnap 1.17.0 Installing bootsnap 1.17.0 with native extensions Using openssl-signature_algorithm 1.3.0 Using uglifier 4.2.0 Fetching faraday 2.7.12 Installing faraday 2.7.12 Using mini_suffix 0.3.3 Fetching nokogiri 1.15.5 (x86_64-linux) Installing nokogiri 1.15.5 (x86_64-linux) Using image_optim 0.31.3 Using omniauth 1.9.2 Fetching sass-embedded 1.69.5 (x86_64-linux-gnu) Installing sass-embedded 1.69.5 (x86_64-linux-gnu) Using unf 0.1.4 Fetching web-push 3.0.1 Installing web-push 3.0.1 Using logstash-logger 0.26.1 Using mini_racer 0.8.0 Using pry 0.14.2 Using sidekiq 6.5.12 Using redis-namespace 1.11.0 Fetching net-http 0.4.0 Installing net-http 0.4.0 Using oauth-tty 1.0.5 Using snaky_hash 2.0.1 Fetching rbtrace 0.5.0 Installing rbtrace 0.5.0 with native extensions Using activesupport 7.0.7 Using rqrcode 2.2.0 Using rss 0.3.0 Using tzinfo-data 1.2023.3 Using unicorn 6.1.0 Using net-pop 0.1.2 Using net-smtp 0.4.0 Using css_parser 1.16.0 Using aws-sdk-core 3.130.2 Using cose 1.3.0 Using faraday-retry 2.2.0 Using oauth2 1.4.11 Fetching loofah 2.22.0 Installing loofah 2.22.0 Using ruby-readability 0.7.0 Using sanitize 6.1.0 Fetching sassc-embedded 1.68.6 Installing sassc-embedded 1.68.6 Using simpleidn 0.2.1 Using rtlcss 0.2.1 Using pry-byebug 3.10.1 Using pry-rails 0.3.9 Using mini_scheduler 0.16.0 Using oauth 1.1.0 Using rails-dom-testing 2.2.0 Using globalid 1.2.1 Using activemodel 7.0.7 Using aws-sdk-kms 1.56.0 Using aws-sdk-sns 1.53.0 Using omniauth-oauth2 1.7.3 Using rails-html-sanitizer 1.6.0 Fetching json_schemer 2.1.1 Installing json_schemer 2.1.1 Using omniauth-oauth 1.2.0 Using actionview 7.0.7 Using activejob 7.0.7 Using active_model_serializers 0.8.4 Using activerecord 7.0.7 Using aws-sdk-s3 1.114.0 Using omniauth-facebook 9.0.0 Using omniauth-github 1.4.0 Using omniauth-google-oauth2 0.8.2 Using omniauth-twitter 1.4.0 Using actionpack 7.0.7 Using actionview_precompiler 0.3.0 Using discourse-seed-fu 2.3.12 Using railties 7.0.7 Using sprockets-rails 3.4.2 Using lograge 0.14.0 Using rails_failover 2.0.1 Using rails_multisite 5.0.0 Using puma 6.4.0 Fetching net-imap 0.4.7 Installing net-imap 0.4.7 Using mail 2.8.1 Using actionmailer 7.0.7 Fetching oj 3.16.2 Installing oj 3.16.2 with native extensions I, [2023-12-11T06:40:09.132711 #1] INFO -- : Terminating async processes I, [2023-12-11T06:40:09.133069 #1] INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main pid: 40 2023-12-11 06:40:09.133 UTC [40] LOG: received fast shutdown request I, [2023-12-11T06:40:09.133831 #1] INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 109 109:signal-handler (1702276809) Received SIGTERM scheduling shutdown... 2023-12-11 06:40:09.138 UTC [40] LOG: aborting any active transactions 2023-12-11 06:40:09.144 UTC [40] LOG: background worker "logical replication launcher" (PID 49) exited with exit code 1 2023-12-11 06:40:09.146 UTC [44] LOG: shutting down 2023-12-11 06:40:09.168 UTC [40] LOG: database system is shut down 109:M 11 Dec 2023 06:40:09.173 # User requested shutdown... 109:M 11 Dec 2023 06:40:09.173 * Saving the final RDB snapshot before exiting. 109:M 11 Dec 2023 06:40:10.723 * DB saved on disk 109:M 11 Dec 2023 06:40:10.723 # Redis is now ready to exit, bye bye... FAILED -------------------- Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle install --retry 3 --jobs 4' failed with return # Location of failure: /usr/local/lib/ruby/gems/3.2.0/gems/pups-1.2.1/lib/pups/exec_command.rb:132:in `spawn' exec failed with the params {"cd"=>"$home", "hook"=>"bundle_exec", "cmd"=>["su discourse -c 'bundle config --local deployment true'", "su discourse -c 'bundle config --local without \"development test\"'", "su discourse -c 'bundle install --retry 3 --jobs 4'"]} bootstrap failed with exit code 5 ** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one. ./discourse-doctor may help diagnose the problem. 9d6071df716185b1c31a8703028a03d6fb0198216704c41506a69b09c67d4519 root@greyarro:/var/discourse# I ran ./discourse-doctor which looked like it did lots of scary things that I didn’t understand but unfortunately it made no difference and it ended with what appears to be the same message: FAILED -------------------- Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle install --retry 3 --jobs 4' failed with return # Location of failure: /usr/local/lib/ruby/gems/3.2.0/gems/pups-1.2.1/lib/pups/exec_command.rb:132:in `spawn' exec failed with the params {"cd"=>"$home", "hook"=>"bundle_exec", "cmd"=>["su discourse -c 'bundle config --local deployment true'", "su discourse -c 'bundle config --local without \"development test\"'", "su discourse -c 'bundle install --retry 3 --jobs 4'"]} bootstrap failed with exit code 5 ** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one. ./discourse-doctor may help diagnose the problem. 9f72d4c67a46bda65d95233149ac8f661bdae02e70ae9a265666920d71de14e2 ==================== END REBUILD LOG ==================== Failed to rebuild app. Can anybody shed any light on the issue, or help me get us back online??

I’m going to have to assume the worst now, that my Discourse is dead. I’ve got a copy of my latest app.yml and I’ve just downloaded last nights database backup from S3. I think I’ll do-release-upgrade my Ubuntu from 18.04.6 LTS to latest, the I guess I’ll do a fresh installing following the guide at discourse/docs/INSTALL-cloud.md at main · discourse/discourse · GitHub Once installed, can I then upload my 1.5GB database backup via the admin panel? :thinking: I’ve been using S3 for uploads for years so I’m hoping the uploaded user content is all still accessible from there. Are there any gotchas? Is there anything that’s not going to be in the database? Any settings? Any files or other content? :flushed:

I’ve taken a (Digital Ocean) snapshot of the server in it’s current state, release upgrade now underway. Out of interest, I was upgrading from https://github.com/discourse/discourse/compare/aadc104817bb0306856b4718f5c50fb481e77c16…4949d85c15be99abb3515e659ef674d58cc2c4c4 and you’ll see I was only 128x commits / 11x days behind latest. Is there a way I can git pull that previous version back down, and rebuild using that? Just to get our community up and running again? :thinking:

Did you try ./launcher start app to try and get the pre-rebuild one back up?

Alas, in my blind panic at it all failing this morning, I removed all the containers which weren’t running :pensive:

I think as long as you have your app.yml info and your recent backup you should be back up in no time on a new droplet. :+1:

Whoa!! I think I may have gifted myself a lifeline :scream: While waiting for the distro to upgrade, it would appear I have rather cleverly enabled Digital Ocean backups :scream: [Screenshot 2023-12-11 at 09.42.14] Time to see if they’re any good :grimacing: Ok, Plan B then… Restore last weeks Digital Ocean snapshot backup from 2023-12-04 Upload my database backup from last night (2023-12-11 @ 2am) I think that might get us roughly back to were we were at 6am this morning when it all fell apart. I’ve not done a database restore before. Assuming the D.O. backup/restore works and gets me back to Monday last week, can I just upload the 1.5GB backup.tar.gz file directly via admin backup menus? do I need to extract it first or anything? Should I put the site in read-only mode or anything else, while doing so?

[image] Richie Rich: Ok, Plan B then… Restore last weeks Digital Ocean snapshot backup from 2023-12-04 Server up! (from last Monday’s snapshot) [image] Richie Rich: can I just upload the 1.5GB backup.tar.gz file directly via admin backup menus? Turns out you very clever people at Discourse built a “Restore” button directly in to the admin menu so I don’t even need to upload the backup :bowing_man: [Screenshot 2023-12-11 at 09.59.56] :crossed_fingers:

No, that didn’t work either. Error log: I’ve just stopped the app again. If I look at my sidekiq it has 12,000+ jobs in it. [Screenshot 2023-12-11 at 10.09.06] It looks like it’s trying to re-send out scheduled email announcements from last week again :man_facepalming: Is there a command line action I can take to kill all those off? I guess these queued items are stopping the restore.

What I would do is spin up a new droplet and follow Move a Discourse site to another VPS with rsync. But I guess you don’t have your /var/discourse anymore unless you made another backup before you restored the one you found? So spin up a new droplet, copy your app.yml and restore yesterday’s backup from S3. This is what to do. Also, what’s your docker version? I had a rebuild fail with an error that looks like this (I’ll track down that topic when I get to my laptop).

Discourse offline after failed upgrade "bootstrap failed with exit code 5"

Product Support Self-hosting

Richie (Richie Rich) December 11, 2023, 10:03am 13

Possibly because I put the site in read-only mode before trying the restore.

Turning off read only mode and trying to restore again…

Topic		Replies	Views
Unable to boostrap Discourse since last upgrade Self-hosting	4	879	January 22, 2023
Latest site upgrade not working Self-hosting	10	2001	February 22, 2022
Bootstrap failure during most recent upgrade Self-hosting	6	702	February 3, 2023
Rebuild failed - FAILED TO BOOTSTRAP Self-hosting	20	2351	December 28, 2023
I need help with install Self-hosting	9	925	November 7, 2024

Discourse offline after failed upgrade "bootstrap failed with exit code 5"

Related topics