Continuous Issues when Rebuilding Discourse

Hello there everyone,

A few days prior, I upgraded my Discourse Instance to the latest version. (I unfortunately don’t have the exact version it was upgraded from and to). The installation would fail, however, I was able to reset the upgrade and rebuild the Discourse app and the site would be working normally.

The next day, I decided to retry the upgrade, the upgrade failed again and this time I got one of those “Oops - Error 500” pages, I attempted to rebuild the app numerous times, however, I would get continuous rebuild errors, one of which I remember is something related to yarn timing out on the network. When I scrolled up the logs, it would state “There seems to be an issue with your network, retrying…” multiple times until giving up and moving on. I don’t have a photo of this, unfortunately, but I do have a photo of the failed message below.

After a bunch of rebuilds and looking up the errors I was getting in the log on this forum and coming to no real conclusion, I backed up my Discourse installation from the rails command line and made sure I also had a few earlier backups, then I deleted my virtual machine that was running Discourse. It is also important to note I was running Ubuntu 20.04 LTS.

I then recreated the virtual machine with Ubuntu 20.04 LTS and followed the steps for re-installing discourse for a standard install, however, the same issues would continue to arise. I would also sometimes get an error in the install logs saying that the Postgres SQL database was still starting up. Another error I noticed was that when the plugins were getting cloned, it would come to an access denied error.

I’ve even tried to recreate the virtual machine with Ubuntu 22.04.04 LTS to see if that would help but it did not rectify any issues.

For plugins, all my plugins that I am using are official from Discourse, I’ve even tried to remove all my plugins but that came to no avail.

This entire ordeal has been really frustrating for me since my forum has been offline for quite a while now, it is important to mention that I am not running the install in the cloud, but on a virtual machine on a dedicated server in my home. The resources allocated to the virtual machine meet Discourse’s system requirements, however, one important thing to note is that the server is running on a slow mechanical hard drive that is causing rebuilds to take approximately 20-30 minutes to rebuild, I’ve also read that the slow disk could also cause yarn to time out however I am not entirely sure that this is caused by the storage device. I plan on replacing the mechanical drive with a solid state soon as the HDD seems to be failing. Attached below is an image I took from CrystalDisk showing the status of the hard drive.


There had been no issues with the setup I was using to host my forum until this occurred, my network is properly configured with the virtual machine having its HTTP/HTTPS ports forwarded to the public internet and the virtual machine being allocated with resources that meet the requirements for Discourse, plus I am using an LTS version of Ubuntu. The virtual machine is running on Windows Server 2019 under Windows Hyper-V.

I’m beginning to believe there may be an issue that extends to my hardware or setup, however, I don’t know to how far of an extent this goes.

Here are the logs from the latest ./launcher rebuild app command:

x86_64 arch detected.
WARNING: containers/app.yml file is world-readable. You can secure this file by                                                                                         running: chmod o-rwx containers/app.yml
Ensuring launcher is up to date
Fetching origin
Launcher is up-to-date
2.0.20231218-0429: Pulling from discourse/base
Digest: sha256:468f70b9bb4c6d0c6c2bbb3efc1a5e12d145eae57bdb6946b7fe5558beb52dc1
Status: Image is up to date for discourse/base:2.0.20231218-0429
docker.io/discourse/base:2.0.20231218-0429
/usr/local/lib/ruby/gems/3.2.0/gems/pups-1.2.1/lib/pups.rb
/usr/local/bin/pups --stdin
I, [2024-03-19T16:32:41.438256 #1]  INFO -- : Reading from stdin
I, [2024-03-19T16:32:41.459943 #1]  INFO -- : > locale-gen $LANG && update-local                                                                                        e
I, [2024-03-19T16:32:41.567791 #1]  INFO -- : Generating locales (this might tak                                                                                        e a while)...
Generation complete.

I, [2024-03-19T16:32:41.568457 #1]  INFO -- : > mkdir -p /shared/postgres_run
I, [2024-03-19T16:32:41.574854 #1]  INFO -- :
I, [2024-03-19T16:32:41.575495 #1]  INFO -- : > chown postgres:postgres /shared/                                                                                        postgres_run
I, [2024-03-19T16:32:41.581326 #1]  INFO -- :
I, [2024-03-19T16:32:41.581893 #1]  INFO -- : > chmod 775 /shared/postgres_run
I, [2024-03-19T16:32:41.587104 #1]  INFO -- :
I, [2024-03-19T16:32:41.587652 #1]  INFO -- : > rm -fr /var/run/postgresql
I, [2024-03-19T16:32:41.593452 #1]  INFO -- :
I, [2024-03-19T16:32:41.594102 #1]  INFO -- : > ln -s /shared/postgres_run /var/                                                                                        run/postgresql
I, [2024-03-19T16:32:41.599363 #1]  INFO -- :
I, [2024-03-19T16:32:41.599920 #1]  INFO -- : > socat /dev/null UNIX-CONNECT:/sh                                                                                        ared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop                                                                                         container ; exit 1
2024/03/19 16:32:41 socat[20] E connect(6, AF=1 "/shared/postgres_run/.s.PGSQL.5                                                                                        432", 36): Connection refused
I, [2024-03-19T16:32:41.612696 #1]  INFO -- :
I, [2024-03-19T16:32:41.613175 #1]  INFO -- : > rm -fr /shared/postgres_run/.s*
I, [2024-03-19T16:32:41.620936 #1]  INFO -- :
I, [2024-03-19T16:32:41.621584 #1]  INFO -- : > rm -fr /shared/postgres_run/*.pi                                                                                        d
I, [2024-03-19T16:32:41.629005 #1]  INFO -- :
I, [2024-03-19T16:32:41.629459 #1]  INFO -- : > mkdir -p /shared/postgres_run/13                                                                                        -main.pg_stat_tmp
I, [2024-03-19T16:32:41.635403 #1]  INFO -- :
I, [2024-03-19T16:32:41.636075 #1]  INFO -- : > chown postgres:postgres /shared/                                                                                        postgres_run/13-main.pg_stat_tmp
I, [2024-03-19T16:32:41.642020 #1]  INFO -- :
I, [2024-03-19T16:32:41.654972 #1]  INFO -- : File > /etc/service/postgres/run                                                                                          chmod: +x  chown:
I, [2024-03-19T16:32:41.668688 #1]  INFO -- : File > /etc/service/postgres/log/r                                                                                        un  chmod: +x  chown:
I, [2024-03-19T16:32:41.680605 #1]  INFO -- : File > /etc/runit/3.d/99-postgres                                                                                          chmod: +x  chown:
I, [2024-03-19T16:32:41.692403 #1]  INFO -- : File > /root/upgrade_postgres  chm                                                                                        od: +x  chown:
I, [2024-03-19T16:32:41.693451 #1]  INFO -- : > chown -R root /var/lib/postgresq                                                                                        l/13/main
I, [2024-03-19T16:34:37.968585 #1]  INFO -- :
I, [2024-03-19T16:34:37.968992 #1]  INFO -- : > [ ! -e /shared/postgres_data ] &                                                                                        & install -d -m 0755 -o postgres -g postgres /shared/postgres_data && sudo -E -u                                                                                         postgres /usr/lib/postgresql/13/bin/initdb -D /shared/postgres_data || exit 0
I, [2024-03-19T16:34:37.975335 #1]  INFO -- :
I, [2024-03-19T16:34:37.975526 #1]  INFO -- : > chown -R postgres:postgres /shar                                                                                        ed/postgres_data
I, [2024-03-19T16:34:38.003767 #1]  INFO -- :
I, [2024-03-19T16:34:38.004482 #1]  INFO -- : > chown -R postgres:postgres /var/                                                                                        run/postgresql
I, [2024-03-19T16:34:38.012328 #1]  INFO -- :
I, [2024-03-19T16:34:38.013012 #1]  INFO -- : > /root/upgrade_postgres
I, [2024-03-19T16:34:38.027169 #1]  INFO -- :
I, [2024-03-19T16:34:38.027657 #1]  INFO -- : > rm /root/upgrade_postgres
I, [2024-03-19T16:34:38.032914 #1]  INFO -- :
I, [2024-03-19T16:34:38.033769 #1]  INFO -- : Replacing data_directory = '/var/l                                                                                        ib/postgresql/13/main' with data_directory = '/shared/postgres_data' in /etc/pos                                                                                        tgresql/13/main/postgresql.conf
I, [2024-03-19T16:34:38.035668 #1]  INFO -- : Replacing (?-mix:#?listen_addresse                                                                                        s *=.*) with listen_addresses = '*' in /etc/postgresql/13/main/postgresql.conf
I, [2024-03-19T16:34:38.109152 #1]  INFO -- : Replacing (?-mix:#?synchronous_com                                                                                        mit *=.*) with synchronous_commit = $db_synchronous_commit in /etc/postgresql/13                                                                                        /main/postgresql.conf
I, [2024-03-19T16:34:38.133759 #1]  INFO -- : Replacing (?-mix:#?shared_buffers                                                                                         *=.*) with shared_buffers = $db_shared_buffers in /etc/postgresql/13/main/postgr                                                                                        esql.conf
I, [2024-03-19T16:34:38.158429 #1]  INFO -- : Replacing (?-mix:#?work_mem *=.*)                                                                                         with work_mem = $db_work_mem in /etc/postgresql/13/main/postgresql.conf
I, [2024-03-19T16:34:38.183076 #1]  INFO -- : Replacing (?-mix:#?default_text_se                                                                                        arch_config *=.*) with default_text_search_config = '$db_default_text_search_con                                                                                        fig' in /etc/postgresql/13/main/postgresql.conf
I, [2024-03-19T16:34:38.207491 #1]  INFO -- : > install -d -m 0755 -o postgres -                                                                                        g postgres /shared/postgres_backup
I, [2024-03-19T16:34:38.214924 #1]  INFO -- :
I, [2024-03-19T16:34:38.216145 #1]  INFO -- : Replacing (?-mix:#?checkpoint_segm                                                                                        ents *=.*) with checkpoint_segments = $db_checkpoint_segments in /etc/postgresql                                                                                        /13/main/postgresql.conf
I, [2024-03-19T16:34:38.232187 #1]  INFO -- : Replacing (?-mix:#?logging_collect                                                                                        or *=.*) with logging_collector = $db_logging_collector in /etc/postgresql/13/ma                                                                                        in/postgresql.conf
I, [2024-03-19T16:34:38.256804 #1]  INFO -- : Replacing (?-mix:#?log_min_duratio                                                                                        n_statement *=.*) with log_min_duration_statement = $db_log_min_duration_stateme                                                                                        nt in /etc/postgresql/13/main/postgresql.conf
I, [2024-03-19T16:34:38.281452 #1]  INFO -- : Replacing (?-mix:^#local +replicat                                                                                        ion +postgres +peer$) with local replication postgres  peer in /etc/postgresql/1                                                                                        3/main/pg_hba.conf
I, [2024-03-19T16:34:38.282725 #1]  INFO -- : Replacing (?-mix:^host.*all.*all.*                                                                                        127.*$) with host all all 0.0.0.0/0 md5 in /etc/postgresql/13/main/pg_hba.conf
I, [2024-03-19T16:34:38.330730 #1]  INFO -- : Replacing (?-mix:^host.*all.*all.*                                                                                        ::1\/128.*$) with host all all ::/0 md5 in /etc/postgresql/13/main/pg_hba.conf
I, [2024-03-19T16:34:38.355097 #1]  INFO -- : > HOME=/var/lib/postgresql USER=po                                                                                        stgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /u                                                                                        sr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main
I, [2024-03-19T16:34:38.358895 #1]  INFO -- : > sleep 5
2024-03-19 16:34:38.803 UTC [43] LOG:  starting PostgreSQL 13.13 (Debian 13.13-1                                                                                        .pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 202                                                                                        10110, 64-bit
2024-03-19 16:34:38.804 UTC [43] LOG:  listening on IPv4 address "0.0.0.0", port                                                                                         5432
2024-03-19 16:34:38.804 UTC [43] LOG:  listening on IPv6 address "::", port 5432
2024-03-19 16:34:39.043 UTC [43] LOG:  listening on Unix socket "/var/run/postgr                                                                                        esql/.s.PGSQL.5432"
2024-03-19 16:34:39.808 UTC [46] LOG:  database system shutdown was interrupted;                                                                                         last known up at 2024-03-19 16:29:44 UTC
2024-03-19 16:34:40.550 UTC [46] LOG:  database system was not properly shut dow                                                                                        n; automatic recovery in progress
2024-03-19 16:34:40.687 UTC [46] LOG:  redo starts at 0/1BD48B8
2024-03-19 16:34:40.687 UTC [46] LOG:  invalid record length at 0/1BD49A0: wante                                                                                        d 24, got 0
2024-03-19 16:34:40.687 UTC [46] LOG:  redo done at 0/1BD4968
2024-03-19 16:34:41.469 UTC [43] LOG:  database system is ready to accept connec                                                                                        tions
I, [2024-03-19T16:34:43.364553 #1]  INFO -- :
I, [2024-03-19T16:34:43.365415 #1]  INFO -- : > su postgres -c 'createdb discour                                                                                        se' || true
2024-03-19 16:34:43.539 UTC [56] postgres@postgres ERROR:  database "discourse"                                                                                         already exists
2024-03-19 16:34:43.539 UTC [56] postgres@postgres STATEMENT:  CREATE DATABASE d                                                                                        iscourse;
createdb: error: database creation failed: ERROR:  database "discourse" already                                                                                         exists
I, [2024-03-19T16:34:43.543547 #1]  INFO -- :
I, [2024-03-19T16:34:43.544689 #1]  INFO -- : > su postgres -c 'psql discourse -                                                                                        c "create user discourse;"' || true
2024-03-19 16:34:43.722 UTC [60] postgres@discourse ERROR:  role "discourse" alr                                                                                        eady exists
2024-03-19 16:34:43.722 UTC [60] postgres@discourse STATEMENT:  create user disc                                                                                        ourse;
ERROR:  role "discourse" already exists
I, [2024-03-19T16:34:43.726332 #1]  INFO -- :
I, [2024-03-19T16:34:43.727467 #1]  INFO -- : > su postgres -c 'psql discourse -                                                                                        c "grant all privileges on database discourse to discourse;"' || true
I, [2024-03-19T16:34:43.889339 #1]  INFO -- : GRANT

I, [2024-03-19T16:34:43.890574 #1]  INFO -- : > su postgres -c 'psql discourse -                                                                                        c "alter schema public owner to discourse;"'
I, [2024-03-19T16:34:44.060633 #1]  INFO -- : ALTER SCHEMA

I, [2024-03-19T16:34:44.061745 #1]  INFO -- : > su postgres -c 'psql template1 -                                                                                        c "create extension if not exists hstore;"'
NOTICE:  extension "hstore" already exists, skipping
I, [2024-03-19T16:34:44.249298 #1]  INFO -- : CREATE EXTENSION

I, [2024-03-19T16:34:44.250601 #1]  INFO -- : > su postgres -c 'psql template1 -                                                                                        c "create extension if not exists pg_trgm;"'
NOTICE:  extension "pg_trgm" already exists, skipping
I, [2024-03-19T16:34:44.429609 #1]  INFO -- : CREATE EXTENSION

I, [2024-03-19T16:34:44.430432 #1]  INFO -- : > su postgres -c 'psql template1 -                                                                                        c "create extension if not exists vector;"'
NOTICE:  extension "vector" already exists, skipping
I, [2024-03-19T16:34:44.606159 #1]  INFO -- : CREATE EXTENSION

I, [2024-03-19T16:34:44.607078 #1]  INFO -- : > su postgres -c 'psql discourse -                                                                                        c "create extension if not exists hstore;"'
NOTICE:  extension "hstore" already exists, skipping
I, [2024-03-19T16:34:44.777476 #1]  INFO -- : CREATE EXTENSION

I, [2024-03-19T16:34:44.778760 #1]  INFO -- : > su postgres -c 'psql discourse -                                                                                        c "create extension if not exists pg_trgm;"'
NOTICE:  extension "pg_trgm" already exists, skipping
I, [2024-03-19T16:34:44.941936 #1]  INFO -- : CREATE EXTENSION

I, [2024-03-19T16:34:44.942985 #1]  INFO -- : > su postgres -c 'psql discourse -                                                                                        c "create extension if not exists vector;"'
NOTICE:  extension "vector" already exists, skipping
I, [2024-03-19T16:34:45.126152 #1]  INFO -- : CREATE EXTENSION

I, [2024-03-19T16:34:45.127264 #1]  INFO -- : > sudo -u postgres psql discourse
I, [2024-03-19T16:34:45.131449 #1]  INFO -- : update pg_database set encoding =                                                                                         pg_char_to_encoding('UTF8') where datname = 'discourse' AND encoding = pg_char_t                                                                                        o_encoding('SQL_ASCII');

I, [2024-03-19T16:34:45.319678 #1]  INFO -- : File > /var/lib/postgresql/take-da                                                                                        tabase-backup  chmod: +x  chown: postgres:postgres
I, [2024-03-19T16:34:45.327509 #1]  INFO -- : File > /var/spool/cron/crontabs/po                                                                                        stgres  chmod:   chown:
I, [2024-03-19T16:34:45.327975 #1]  INFO -- : > echo postgres installed!
I, [2024-03-19T16:34:45.337715 #1]  INFO -- : postgres installed!

I, [2024-03-19T16:34:45.350316 #1]  INFO -- : File > /etc/service/redis/run  chm                                                                                        od: +x  chown:
I, [2024-03-19T16:34:45.364575 #1]  INFO -- : File > /etc/service/redis/log/run                                                                                          chmod: +x  chown:
I, [2024-03-19T16:34:45.376683 #1]  INFO -- : File > /etc/runit/3.d/10-redis  ch                                                                                        mod: +x  chown:
I, [2024-03-19T16:34:45.377876 #1]  INFO -- : Replacing daemonize yes with  in /                                                                                        etc/redis/redis.conf
I, [2024-03-19T16:34:45.379812 #1]  INFO -- : Replacing (?-mix:^pidfile.*$) with                                                                                          in /etc/redis/redis.conf
I, [2024-03-19T16:34:45.522977 #1]  INFO -- : > install -d -m 0755 -o redis -g r                                                                                        edis /shared/redis_data
I, [2024-03-19T16:34:45.530519 #1]  INFO -- :
I, [2024-03-19T16:34:45.531688 #1]  INFO -- : Replacing (?-mix:^logfile.*$) with                                                                                         logfile "" in /etc/redis/redis.conf
I, [2024-03-19T16:34:45.605605 #1]  INFO -- : Replacing (?-mix:^bind .*$) with                                                                                          in /etc/redis/redis.conf
I, [2024-03-19T16:34:45.654747 #1]  INFO -- : Replacing (?-mix:^dir .*$) with di                                                                                        r /shared/redis_data in /etc/redis/redis.conf
I, [2024-03-19T16:34:45.704127 #1]  INFO -- : Replacing (?-mix:^protected-mode y                                                                                        es) with protected-mode no in /etc/redis/redis.conf
I, [2024-03-19T16:34:45.753115 #1]  INFO -- : Replacing # io-threads 4 with io-t                                                                                        hreads $redis_io_threads in /etc/redis/redis.conf
I, [2024-03-19T16:34:45.802297 #1]  INFO -- : > echo redis installed
I, [2024-03-19T16:34:45.808061 #1]  INFO -- : redis installed

I, [2024-03-19T16:34:45.808693 #1]  INFO -- : > cat /etc/redis/redis.conf | grep                                                                                         logfile
I, [2024-03-19T16:34:45.818814 #1]  INFO -- : logfile ""

I, [2024-03-19T16:34:45.819515 #1]  INFO -- : > exec chpst -u redis -U redis /us                                                                                        r/bin/redis-server /etc/redis/redis.conf
I, [2024-03-19T16:34:45.823505 #1]  INFO -- : > sleep 10
112:C 19 Mar 2024 16:34:45.842 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
112:C 19 Mar 2024 16:34:45.842 # Redis version=7.0.7, bits=64, commit=00000000,                                                                                         modified=0, pid=112, just started
112:C 19 Mar 2024 16:34:45.842 # Configuration loaded
112:M 19 Mar 2024 16:34:45.844 * monotonic clock: POSIX clock_gettime
112:M 19 Mar 2024 16:34:45.845 * Running mode=standalone, port=6379.
112:M 19 Mar 2024 16:34:45.846 # Server initialized
112:M 19 Mar 2024 16:34:45.846 # WARNING Memory overcommit must be enabled! With                                                                                        out it, a background save or replication may fail under low memory condition. Be                                                                                        ing disabled, it can can also cause failures without low memory condition, see h                                                                                        ttps://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overc                                                                                        ommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl                                                                                         vm.overcommit_memory=1' for this to take effect.
112:M 19 Mar 2024 16:34:45.847 * Loading RDB produced by version 7.0.7
112:M 19 Mar 2024 16:34:45.847 * RDB age 305 seconds
112:M 19 Mar 2024 16:34:45.847 * RDB memory usage when created 0.83 Mb
112:M 19 Mar 2024 16:34:45.847 * Done loading RDB, keys loaded: 11, keys expired                                                                                        : 0.
112:M 19 Mar 2024 16:34:45.848 * DB loaded from disk: 0.001 seconds
112:M 19 Mar 2024 16:34:45.848 * Ready to accept connections
I, [2024-03-19T16:34:55.828926 #1]  INFO -- :
I, [2024-03-19T16:34:55.829696 #1]  INFO -- : > thpoff echo "thpoff is installed                                                                                        !"
I, [2024-03-19T16:34:55.839100 #1]  INFO -- : thpoff is installed!

I, [2024-03-19T16:34:55.839934 #1]  INFO -- : > /usr/local/bin/ruby -e 'if ENV["                                                                                        DISCOURSE_SMTP_ADDRESS"] == "smtp.example.com"; puts "Aborting! Mail is not conf                                                                                        igured!"; exit 1; end'
I, [2024-03-19T16:34:56.090160 #1]  INFO -- :
I, [2024-03-19T16:34:56.090902 #1]  INFO -- : > /usr/local/bin/ruby -e 'if ENV["                                                                                        DISCOURSE_HOSTNAME"] == "discourse.example.com"; puts "Aborting! Domain is not c                                                                                        onfigured!"; exit 1; end'
I, [2024-03-19T16:34:56.340784 #1]  INFO -- :
I, [2024-03-19T16:34:56.341858 #1]  INFO -- : > /usr/local/bin/ruby -e 'if (ENV[                                                                                        "DISCOURSE_CDN_URL"] || "")[0..1] == "//"; puts "Aborting! CDN must have a proto                                                                                        col specified. Once fixed you should rebake your posts now to correct all posts.                                                                                        "; exit 1; end'
I, [2024-03-19T16:34:56.593304 #1]  INFO -- :
I, [2024-03-19T16:34:56.594095 #1]  INFO -- : > rm -f /etc/cron.d/anacron
I, [2024-03-19T16:34:57.388238 #1]  INFO -- :
I, [2024-03-19T16:34:57.396937 #1]  INFO -- : File > /etc/cron.d/anacron  chmod:                                                                                           chown:
I, [2024-03-19T16:34:57.418454 #1]  INFO -- : File > /etc/runit/1.d/copy-env  ch                                                                                        mod: +x  chown:
I, [2024-03-19T16:34:57.436263 #1]  INFO -- : File > /etc/service/unicorn/run  c                                                                                        hmod: +x  chown:
I, [2024-03-19T16:34:57.451285 #1]  INFO -- : File > /etc/service/nginx/run  chm                                                                                        od: +x  chown:
I, [2024-03-19T16:34:57.464828 #1]  INFO -- : File > /etc/runit/3.d/01-nginx  ch                                                                                        mod: +x  chown:
I, [2024-03-19T16:34:57.478016 #1]  INFO -- : File > /etc/runit/3.d/02-unicorn                                                                                          chmod: +x  chown:
I, [2024-03-19T16:34:57.478451 #1]  INFO -- : Replacing # postgres with sv start                                                                                         postgres || exit 1 in /etc/service/unicorn/run
I, [2024-03-19T16:34:57.479456 #1]  INFO -- : > exec chpst -u redis -U redis /us                                                                                        r/bin/redis-server /etc/redis/redis.conf
I, [2024-03-19T16:34:57.487402 #1]  INFO -- : > cd /var/www/discourse && sudo -H                                                                                         -E -u discourse git reset --hard
139:C 19 Mar 2024 16:34:57.509 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
139:C 19 Mar 2024 16:34:57.509 # Redis version=7.0.7, bits=64, commit=00000000,                                                                                         modified=0, pid=139, just started
139:C 19 Mar 2024 16:34:57.509 # Configuration loaded
139:M 19 Mar 2024 16:34:57.511 * monotonic clock: POSIX clock_gettime
139:M 19 Mar 2024 16:34:57.512 # Warning: Could not create server TCP listening                                                                                         socket *:6379: bind: Address already in use
139:M 19 Mar 2024 16:34:57.512 # Failed listening on port 6379 (TCP), aborting.
Updating files: 100% (33715/33715), done.
I, [2024-03-19T16:35:10.829468 #1]  INFO -- : HEAD is now at 6ab1a19e9 DEV: Conv                                                                                        ert min_trust_level_to_allow_invite to groups (#24893)

I, [2024-03-19T16:35:10.830148 #1]  INFO -- : > cd /var/www/discourse && sudo -H                                                                                         -E -u discourse git clean -f
I, [2024-03-19T16:35:11.274658 #1]  INFO -- :
I, [2024-03-19T16:35:11.275087 #1]  INFO -- : > cd /var/www/discourse && sudo -H                                                                                         -E -u discourse bash -c '
  set -o errexit
  if [ $(git rev-parse --is-shallow-repository) == "true" ]; then
      git remote set-branches --add origin main
      git remote set-branches origin tests-passed
      git fetch --depth 1 origin tests-passed
  else
      git fetch --tags --prune-tags --prune --force origin
  fi
'
From https://github.com/discourse/discourse
 - [deleted]             (none)     -> origin/0-assets-spec2
 - [deleted]             (none)     -> origin/0-drop-code
 - [deleted]             (none)     -> origin/0-followup
 - [deleted]             (none)     -> origin/0-gjs-iframed-html
 - [deleted]             (none)     -> origin/a11y-sidebar-contrast
 - [deleted]             (none)     -> origin/a11y–focus-tooltip
 - [deleted]             (none)     -> origin/add-ds-store-to-gitignore
 - [deleted]             (none)     -> origin/add-new-chat-indicator-preference-                                                                                        only-mentions
 - [deleted]             (none)     -> origin/add-topic-voting-webhook
 - [deleted]             (none)     -> origin/after-topic-list
 - [deleted]             (none)     -> origin/api-reduce-motion
 - [deleted]             (none)     -> origin/api-topic-title-decorator
 - [deleted]             (none)     -> origin/boosted-searches
 - [deleted]             (none)     -> origin/bulk-button-class
 - [deleted]             (none)     -> origin/card-container-plugin-outlet
 - [deleted]             (none)     -> origin/composer-warnings-use-notice-alt
 - [deleted]             (none)     -> origin/confirm-session-only-after-5-mins
 - [deleted]             (none)     -> origin/dependabot/bundler/excon-0.108.0
 - [deleted]             (none)     -> origin/dependabot/bundler/hashdiff-1.1.0
 - [deleted]             (none)     -> origin/dependabot/bundler/parallel-1.24.0
 - [deleted]             (none)     -> origin/dependabot/bundler/rubocop-discour                                                                                        se-3.6.0
 - [deleted]             (none)     -> origin/dependabot/npm_and_yarn/app/assets                                                                                        /javascripts/uppy/aws-s3-3.6.0
 - [deleted]             (none)     -> origin/dependabot/npm_and_yarn/app/assets                                                                                        /javascripts/uppy/aws-s3-multipart-3.10.0
 - [deleted]             (none)     -> origin/dependabot/npm_and_yarn/app/assets                                                                                        /javascripts/uppy/drop-target-2.0.2
 - [deleted]             (none)     -> origin/dependabot/npm_and_yarn/app/assets                                                                                        /javascripts/uppy/utils-5.7.0
 - [deleted]             (none)     -> origin/desktop-notification-handlers
 - [deleted]             (none)     -> origin/dev-rework-codeblock-fix
 - [deleted]             (none)     -> origin/dev/add-new-chat-metrics
 - [deleted]             (none)     -> origin/dev/allow-chat-mentions-to-have-se                                                                                        veral-notifications
 - [deleted]             (none)     -> origin/dev/convert-min-trust-level-to-cre                                                                                        ate-tag-to-group
 - [deleted]             (none)     -> origin/dev/convert-min-trust-level-to-ign                                                                                        ore-to-groups
 - [deleted]             (none)     -> origin/dev/dry-up-user-status-serializers
 - [deleted]             (none)     -> origin/dev/flaky-chat-details-collapser-s                                                                                        pec
 - [deleted]             (none)     -> origin/dev/no-need-to-include-user-status                                                                                        -twice
 - [deleted]             (none)     -> origin/dev/rafactor-the-export-csv-file-j                                                                                        ob
 - [deleted]             (none)     -> origin/dev/rethink-mass-mentions-polymorp                                                                                        hic
 - [deleted]             (none)     -> origin/dev/revert-guardian-devex
 - [deleted]             (none)     -> origin/dev/simplify-lib-test-js
 - [deleted]             (none)     -> origin/dev/test-status-tooltip-in-its-own                                                                                        -test-2
 - [deleted]             (none)     -> origin/dev/trust-level-value-object
 - [deleted]             (none)     -> origin/dev_categories
 - [deleted]             (none)     -> origin/dev_watched_words
 - [deleted]             (none)     -> origin/drop-kit
 - [deleted]             (none)     -> origin/experimantal/deleted-channel-and-m                                                                                        essages-export
 - [deleted]             (none)     -> origin/extract_stripper
 - [deleted]             (none)     -> origin/feat-render-images-in-local-onebox
 - [deleted]             (none)     -> origin/feature/admin-sidebar-v0
 - [deleted]             (none)     -> origin/feature/experimental-passkeys
 - [deleted]             (none)     -> origin/feature/migrations-schema-generato                                                                                        r
 - [deleted]             (none)     -> origin/feature/watched-word-groups
 - [deleted]             (none)     -> origin/feature_watched_words_validation
 - [deleted]             (none)     -> origin/filtered-events
 - [deleted]             (none)     -> origin/fix-client-tests-for-status
 - [deleted]             (none)     -> origin/fix-join-filter-upload
 - [deleted]             (none)     -> origin/fix-modal-flash-minor
 - [deleted]             (none)     -> origin/fix/count-mass-mentions-correctly
 - [deleted]             (none)     -> origin/fix/do-not-consider-codeblocks-whe                                                                                        n-parsing-mentions-alt
 - [deleted]             (none)     -> origin/fix/dont-fail-when-exporting-chat-                                                                                        messages-from-deleted-channels-or-of-deleted-users
 - [deleted]             (none)     -> origin/fix/multi-tag-search
 - [deleted]             (none)     -> origin/float-kit-3
 - [deleted]             (none)     -> origin/generic-import-latest-snowflake
 - [deleted]             (none)     -> origin/global-notice-reactive
 - [deleted]             (none)     -> origin/group-directory-cleanup
 - [deleted]             (none)     -> origin/issue/fix-guardian-init-basic-user
 - [deleted]             (none)     -> origin/keyword-site-setting
 - [deleted]             (none)     -> origin/labeler
 - [deleted]             (none)     -> origin/large-notification-table
 - [deleted]             (none)     -> origin/loic-use-ar-enums-posts
 - [deleted]             (none)     -> origin/low-ram-help
 - [deleted]             (none)     -> origin/mobile-categories-refactor
 - [deleted]             (none)     -> origin/mobile-view-param-key
 - [deleted]             (none)     -> origin/move-automation-to-core
 - [deleted]             (none)     -> origin/move-checkbox-public-up
 - [deleted]             (none)     -> origin/move_inflector
 - [deleted]             (none)     -> origin/multi-invite
 - [deleted]             (none)     -> origin/offline-indicator-by-default
 - [deleted]             (none)     -> origin/passkeys-part-3
 - [deleted]             (none)     -> origin/plugin-extra-js
 - [deleted]             (none)     -> origin/poc/auto-after-before-plugin-outle                                                                                        ts
 - [deleted]             (none)     -> origin/poc/insert-before-after-wrapping-o                                                                                        utlet
 - [deleted]             (none)     -> origin/rails-6.x-update
 - [deleted]             (none)     -> origin/rebake-quotes
 - [deleted]             (none)     -> origin/redirect-cat-trail
 - [deleted]             (none)     -> origin/reply-where-position
 - [deleted]             (none)     -> origin/revert-maxmind
 - [deleted]             (none)     -> origin/rotate-token-on-ip-change
 - [deleted]             (none)     -> origin/search_refactor
 - [deleted]             (none)     -> origin/security/bookmark-notification-unr                                                                                        ead-visibility
 - [deleted]             (none)     -> origin/support-load-more-btn
 - [deleted]             (none)     -> origin/table-builder-modal-modifier
 - [deleted]             (none)     -> origin/test-issue-temporary
 - [deleted]             (none)     -> origin/test-mentions-duplicates
 - [deleted]             (none)     -> origin/tests-passed-broken
 - [deleted]             (none)     -> origin/tmp/reimplement-all-and-here-menti                                                                                        ons
 - [deleted]             (none)     -> origin/topic-list-refactor
 - [deleted]             (none)     -> origin/transform-hooks
 - [deleted]             (none)     -> origin/transform-post-fn-api
 - [deleted]             (none)     -> origin/tweaks-2-fr
 - [deleted]             (none)     -> origin/update-login-modal-mark
 - [deleted]             (none)     -> origin/user-group-messages-after-latest
 - [deleted]             (none)     -> origin/username_cache
 - [deleted]             (none)     -> origin/ux-add-sorting-params-to-outlet
 - [deleted]             (none)     -> origin/watched-words-link
 - [deleted]             (none)     -> origin/webauthn-internal-refactor
 - [deleted]             (none)     -> origin/wip-close-on-click-outside-search
 - [deleted]             (none)     -> origin/wip_categories
 - [deleted]             (none)     -> origin/wizard-split
   6ab1a19e9..ce65a88d6  main                    -> origin/main
 * [new branch]          0-ember-bump            -> origin/0-ember-bump
 * [new branch]          0-extend-native         -> origin/0-extend-native
 * [new branch]          0-fix-perf-watched-words -> origin/0-fix-perf-watched-w                                                                                        ords
 * [new branch]          allow-staff-to-tag      -> origin/allow-staff-to-tag
 * [new branch]          asciinema-iframe        -> origin/asciinema-iframe
 * [new branch]          bulk-actions-plugin-api -> origin/bulk-actions-plugin-a                                                                                        pi
 * [new branch]          composer-button-dynamic -> origin/composer-button-dynam                                                                                        ic
 * [new branch]          d-object                -> origin/d-object
 * [new branch]          decorator-transforms    -> origin/decorator-transforms
 * [new branch]          dependabot/bundler/parallel_tests-4.4.0 -> origin/depen                                                                                        dabot/bundler/parallel_tests-4.4.0
 * [new branch]          dev/db-backed-admin-messages -> origin/dev/db-backed-ad                                                                                        min-messages
 * [new branch]          dev/deprecate-found-user-with-status-serializer -> orig                                                                                        in/dev/deprecate-found-user-with-status-serializer
 * [new branch]          dev/drop-found-user-with-status-serializer -> origin/de                                                                                        v/drop-found-user-with-status-serializer
 + 79ea38776...bd0faffec dev/drop-notification_id-column-on-chat-mentions -> ori                                                                                        gin/dev/drop-notification_id-column-on-chat-mentions  (forced update)
 * [new branch]          dev/drop-the-user-id-column-from-chat-mentions -> origi                                                                                        n/dev/drop-the-user-id-column-from-chat-mentions
 * [new branch]          dev/make-admins-tl4-in-tests -> origin/dev/make-admins-                                                                                        tl4-in-tests
 * [new branch]          dev/promote-block-checks-to-problem-check -> origin/dev                                                                                        /promote-block-checks-to-problem-check
 * [new branch]          dev/proof-of-concept-rewriting-mentions-notification-lo                                                                                        gic -> origin/dev/proof-of-concept-rewriting-mentions-notification-logic
 * [new branch]          dev/stylesheet_importer_categories_with_background_imag                                                                                        es-modifier -> origin/dev/stylesheet_importer_categories_with_background_images-                                                                                        modifier
 * [new branch]          dev/timeline-footer-controls-after-outlet--include-full                                                                                        screen -> origin/dev/timeline-footer-controls-after-outlet--include-fullscreen
 * [new branch]          dev_refactor_css_generators -> origin/dev_refactor_css_                                                                                        generators
 * [new branch]          dev_update_category_position -> origin/dev_update_categ                                                                                        ory_position
 * [new branch]          extra-js-revert         -> origin/extra-js-revert
   8780daeee..abc9cd9c5  feature/bookmark-menu-and-modal-redesign -> origin/feat                                                                                        ure/bookmark-menu-and-modal-redesign
 * [new branch]          feature/disable-failing-auth-methods -> origin/feature/                                                                                        disable-failing-auth-methods
 * [new branch]          feature/paste-upload-on-first-upload-input-templates ->                                                                                         origin/feature/paste-upload-on-first-upload-input-templates
 * [new branch]          feature_track_mentions  -> origin/feature_track_mention                                                                                        s
 * [new branch]          fix_default_value_validation_fail -> origin/fix_default                                                                                        _value_validation_fail
 * [new branch]          fix_reviewable_translation -> origin/fix_reviewable_tra                                                                                        nslation
 * [new branch]          generic-import-add-suspender -> origin/generic-import-a                                                                                        dd-suspender
   7f2bdca65..eb3016cad  generic-import-latest   -> origin/generic-import-latest
 * [new branch]          hide-chat-btn-if-chat-disabled -> origin/hide-chat-btn-                                                                                        if-chat-disabled
 * [new branch]          hijacks                 -> origin/hijacks
 * [new branch]          issue/fix-chat-admin-sidebar-drawer -> origin/issue/fix                                                                                        -chat-admin-sidebar-drawer
 * [new branch]          live-cache              -> origin/live-cache
 * [new branch]          loic-selenium-chromium  -> origin/loic-selenium-chromiu                                                                                        m
 * [new branch]          member-xp/mobile-modal  -> origin/member-xp/mobile-moda                                                                                        l
 + 90a3535b7...54d0135bc migrations-convert      -> origin/migrations-convert  (                                                                                        forced update)
 * [new branch]          morph-composer-preview  -> origin/morph-composer-previe                                                                                        w
 * [new branch]          omniauth-2              -> origin/omniauth-2
 * [new branch]          promote-migration-tweak -> origin/promote-migration-twe                                                                                        ak
 * [new branch]          reduce_filesize_of_emoji_group_sections_component -> or                                                                                        igin/reduce_filesize_of_emoji_group_sections_component
 * [new branch]          remove-bootbox          -> origin/remove-bootbox
 * [new branch]          see_if_yarn_cache_is_working -> origin/see_if_yarn_cach                                                                                        e_is_working
 * [new branch]          shared-draft-category   -> origin/shared-draft-category
 * [new branch]          simplify-sidebar        -> origin/simplify-sidebar
   d7f7f79b6..7d9c8993c  stable                  -> origin/stable
 * [new branch]          support-query-string    -> origin/support-query-string
   6ab1a19e9..ce65a88d6  tests-passed            -> origin/tests-passed
 * [new branch]          universal-nonce         -> origin/universal-nonce
 * [new branch]          update-add-icons-to-use-float-kit -> origin/update-add-                                                                                        icons-to-use-float-kit
 * [new branch]          user-menu-full-name     -> origin/user-menu-full-name
 * [new branch]          ux-input-sizing         -> origin/ux-input-sizing
 t [tag update]          beta                    -> beta
 t [tag update]          latest-release          -> latest-release
 * [new tag]             v3.1.4                  -> v3.1.4
 * [new tag]             v3.1.5                  -> v3.1.5
 * [new tag]             v3.2.0                  -> v3.2.0
 * [new tag]             v3.2.0.beta4            -> v3.2.0.beta4
 * [new tag]             v3.2.0.beta5            -> v3.2.0.beta5
 * [new tag]             v3.2.1                  -> v3.2.1
 * [new tag]             v3.3.0.beta1            -> v3.3.0.beta1
I, [2024-03-19T16:35:13.909906 #1]  INFO -- :
I, [2024-03-19T16:35:13.910340 #1]  INFO -- : > cd /var/www/discourse && sudo -H                                                                                         -E -u discourse bash -c '
  set -o errexit
  if [[ $(git symbolic-ref --short HEAD) == tests-passed ]] ; then
      git pull
  else
      git -c advice.detachedHead=false checkout tests-passed
  fi
'
Switched to a new branch 'tests-passed'
I, [2024-03-19T16:35:24.077857 #1]  INFO -- : Branch 'tests-passed' set up to tr                                                                                        ack remote branch 'tests-passed' from 'origin'.

I, [2024-03-19T16:35:24.079798 #1]  INFO -- : > cd /var/www/discourse && sudo -H                                                                                         -E -u discourse git config user.discourse-version tests-passed
I, [2024-03-19T16:35:24.227550 #1]  INFO -- :
I, [2024-03-19T16:35:24.228193 #1]  INFO -- : > cd /var/www/discourse && mkdir -                                                                                        p tmp
I, [2024-03-19T16:35:24.236342 #1]  INFO -- :
I, [2024-03-19T16:35:24.236676 #1]  INFO -- : > cd /var/www/discourse && chown d                                                                                        iscourse:www-data tmp
I, [2024-03-19T16:35:24.245352 #1]  INFO -- :
I, [2024-03-19T16:35:24.245660 #1]  INFO -- : > cd /var/www/discourse && mkdir -                                                                                        p tmp/pids
I, [2024-03-19T16:35:24.254265 #1]  INFO -- :
I, [2024-03-19T16:35:24.254685 #1]  INFO -- : > cd /var/www/discourse && mkdir -                                                                                        p tmp/sockets
I, [2024-03-19T16:35:24.262959 #1]  INFO -- :
I, [2024-03-19T16:35:24.263379 #1]  INFO -- : > cd /var/www/discourse && touch t                                                                                        mp/.gitkeep
I, [2024-03-19T16:35:24.271221 #1]  INFO -- :
I, [2024-03-19T16:35:24.271450 #1]  INFO -- : > cd /var/www/discourse && mkdir -                                                                                        p                    /shared/log/rails
I, [2024-03-19T16:35:24.279371 #1]  INFO -- :
I, [2024-03-19T16:35:24.279626 #1]  INFO -- : > cd /var/www/discourse && bash -c                                                                                         "touch -a           /shared/log/rails/{production,production_errors,unicorn.std                                                                                        out,unicorn.stderr,sidekiq}.log"
I, [2024-03-19T16:35:24.291996 #1]  INFO -- :
I, [2024-03-19T16:35:24.292367 #1]  INFO -- : > cd /var/www/discourse && bash -c                                                                                         "ln    -s           /shared/log/rails/{production,production_errors,unicorn.std                                                                                        out,unicorn.stderr,sidekiq}.log /var/www/discourse/log"
I, [2024-03-19T16:35:24.305093 #1]  INFO -- :
I, [2024-03-19T16:35:24.305386 #1]  INFO -- : > cd /var/www/discourse && bash -c                                                                                         "mkdir -p           /shared/{uploads,backups}"
I, [2024-03-19T16:35:24.319893 #1]  INFO -- :
I, [2024-03-19T16:35:24.320188 #1]  INFO -- : > cd /var/www/discourse && bash -c                                                                                         "ln    -s           /shared/{uploads,backups} /var/www/discourse/public"
I, [2024-03-19T16:35:24.332421 #1]  INFO -- :
I, [2024-03-19T16:35:24.332724 #1]  INFO -- : > cd /var/www/discourse && bash -c                                                                                         "mkdir -p           /shared/tmp/{backups,restores}"
I, [2024-03-19T16:35:24.345308 #1]  INFO -- :
I, [2024-03-19T16:35:24.345616 #1]  INFO -- : > cd /var/www/discourse && bash -c                                                                                         "ln    -s           /shared/tmp/{backups,restores} /var/www/discourse/tmp"
I, [2024-03-19T16:35:24.357207 #1]  INFO -- :
I, [2024-03-19T16:35:24.357719 #1]  INFO -- : > cd /var/www/discourse && chown -                                                                                        R discourse:www-data /shared/log/rails /shared/uploads /shared/backups /shared/t                                                                                        mp
I, [2024-03-19T16:35:24.366660 #1]  INFO -- :
I, [2024-03-19T16:35:24.366976 #1]  INFO -- : > cd /var/www/discourse && [ ! -d                                                                                         public/plugins ] || find public/plugins/ -maxdepth 1 -xtype l -delete
I, [2024-03-19T16:35:24.371930 #1]  INFO -- :
I, [2024-03-19T16:35:24.372597 #1]  INFO -- : Replacing # redis with sv start re                                                                                        dis || exit 1 in /etc/service/unicorn/run
I, [2024-03-19T16:35:24.375568 #1]  INFO -- : > cd /var/www/discourse/plugins &&                                                                                         git clone https://github.com/discourse/docker_manager.git
Cloning into 'docker_manager'...
I, [2024-03-19T16:35:26.673043 #1]  INFO -- :
I, [2024-03-19T16:35:26.673305 #1]  INFO -- : > cd /var/www/discourse/plugins &&                                                                                         sudo -E -u discourse git clone https://github.com/discourse/discourse-yearly-re                                                                                        view.git
Cloning into 'discourse-yearly-review'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2024-03-19T16:35:27.763857 #1]  INFO -- :
I, [2024-03-19T16:35:27.764363 #1]  INFO -- : > cd /var/www/discourse/plugins &&                                                                                         sudo -E -u discourse git clone https://github.com/discourse/discourse-reactions                                                                                        .git
Cloning into 'discourse-reactions'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2024-03-19T16:35:29.275124 #1]  INFO -- :
I, [2024-03-19T16:35:29.275598 #1]  INFO -- : > cd /var/www/discourse/plugins &&                                                                                         sudo -E -u discourse git clone https://github.com/discourse/discourse-user-note                                                                                        s.git
Cloning into 'discourse-user-notes'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2024-03-19T16:35:30.428436 #1]  INFO -- :
I, [2024-03-19T16:35:30.429173 #1]  INFO -- : > cd /var/www/discourse/plugins &&                                                                                         sudo -E -u discourse git clone https://github.com/discourse/discourse-gamificat                                                                                        ion.git
Cloning into 'discourse-gamification'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2024-03-19T16:35:31.788286 #1]  INFO -- :
I, [2024-03-19T16:35:31.789821 #1]  INFO -- : > cd /var/www/discourse/plugins &&                                                                                         sudo -E -u discourse git clone https://github.com/discourse/discourse-policy.gi                                                                                        t
Cloning into 'discourse-policy'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2024-03-19T16:35:34.719712 #1]  INFO -- :
I, [2024-03-19T16:35:34.720281 #1]  INFO -- : > cd /var/www/discourse/plugins &&                                                                                         sudo -E -u discourse git clone https://github.com/discourse/discourse-calendar.                                                                                        git
Cloning into 'discourse-calendar'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2024-03-19T16:35:39.165042 #1]  INFO -- :
I, [2024-03-19T16:35:39.165522 #1]  INFO -- : > cd /var/www/discourse/plugins &&                                                                                         sudo -E -u discourse git clone https://github.com/discourse/discourse-adplugin.                                                                                        git
Cloning into 'discourse-adplugin'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2024-03-19T16:35:45.006305 #1]  INFO -- :
I, [2024-03-19T16:35:45.006702 #1]  INFO -- : > cd /var/www/discourse/plugins &&                                                                                         sudo -E -u discourse git clone https://github.com/discourse/discourse-follow.gi                                                                                        t
Cloning into 'discourse-follow'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2024-03-19T16:35:50.572583 #1]  INFO -- :
I, [2024-03-19T16:35:50.573195 #1]  INFO -- : > cd /var/www/discourse/plugins &&                                                                                         sudo -E -u discourse git clone https://github.com/discourse/discourse-solved.gi                                                                                        t
Cloning into 'discourse-solved'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2024-03-19T16:35:52.922038 #1]  INFO -- :
I, [2024-03-19T16:35:52.922601 #1]  INFO -- : > cd /var/www/discourse/plugins &&                                                                                         sudo -E -u discourse git clone https://github.com/discourse/discourse-post-voti                                                                                        ng.git
Cloning into 'discourse-post-voting'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2024-03-19T16:35:54.380420 #1]  INFO -- :
I, [2024-03-19T16:35:54.380867 #1]  INFO -- : > cd /var/www/discourse/plugins &&                                                                                         sudo -E -u discourse git clone https://github.com/discourse/discourse-cakeday.g                                                                                        it
Cloning into 'discourse-cakeday'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2024-03-19T16:35:55.531642 #1]  INFO -- :
I, [2024-03-19T16:35:55.532304 #1]  INFO -- : > cd /var/www/discourse/plugins &&                                                                                         sudo -E -u discourse git clone https://github.com/discourse/discourse-subscript                                                                                        ions.git
Cloning into 'discourse-subscriptions'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2024-03-19T16:36:00.495608 #1]  INFO -- :
I, [2024-03-19T16:36:00.496142 #1]  INFO -- : > cd /var/www/discourse/plugins &&                                                                                         sudo -E -u discourse git clone https://github.com/discourse/discourse-whos-onli                                                                                        ne.git
Cloning into 'discourse-whos-online'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2024-03-19T16:36:01.759717 #1]  INFO -- :
I, [2024-03-19T16:36:01.760165 #1]  INFO -- : > cd /var/www/discourse/plugins &&                                                                                         sudo -E -u discourse git clone https://github.com/discourse/discourse-chat-inte                                                                                        gration.git
Cloning into 'discourse-chat-integration'...
warning: unable to access '/root/.config/git/attributes': Permission denied
I, [2024-03-19T16:36:03.246218 #1]  INFO -- :
I, [2024-03-19T16:36:03.247472 #1]  INFO -- : > cp /var/www/discourse/config/ngi                                                                                        nx.sample.conf /etc/nginx/conf.d/discourse.conf
I, [2024-03-19T16:36:03.255381 #1]  INFO -- :
I, [2024-03-19T16:36:03.255949 #1]  INFO -- : > rm /etc/nginx/sites-enabled/defa                                                                                        ult
I, [2024-03-19T16:36:03.261744 #1]  INFO -- :
I, [2024-03-19T16:36:03.262032 #1]  INFO -- : > mkdir -p /var/nginx/cache
I, [2024-03-19T16:36:03.268429 #1]  INFO -- :
I, [2024-03-19T16:36:03.269266 #1]  INFO -- : Replacing pid /run/nginx.pid; with                                                                                         daemon off; in /etc/nginx/nginx.conf
I, [2024-03-19T16:36:03.270349 #1]  INFO -- : Replacing (?m-ix:upstream[^\}]+\})                                                                                         with upstream discourse { server 127.0.0.1:3000; } in /etc/nginx/conf.d/discour                                                                                        se.conf
I, [2024-03-19T16:36:03.271428 #1]  INFO -- : Replacing (?-mix:server_name.+$) w                                                                                        ith server_name _ ; in /etc/nginx/conf.d/discourse.conf
I, [2024-03-19T16:36:03.321320 #1]  INFO -- : Replacing (?-mix:client_max_body_s                                                                                        ize.+$) with client_max_body_size $upload_size ; in /etc/nginx/conf.d/discourse.                                                                                        conf
I, [2024-03-19T16:36:03.370586 #1]  INFO -- : Replacing (?-mix:worker_connection                                                                                        s.+$) with worker_connections $nginx_worker_connections ; in /etc/nginx/nginx.co                                                                                        nf
I, [2024-03-19T16:36:03.371396 #1]  INFO -- : > echo "done configuring web"
I, [2024-03-19T16:36:03.375945 #1]  INFO -- : done configuring web

I, [2024-03-19T16:36:03.377150 #1]  INFO -- : > cd /var/www/discourse && gem ins                                                                                        tall bundler --conservative -v $(awk '/BUNDLED WITH/ { getline; gsub(/ /,""); pr                                                                                        int $0 }' Gemfile.lock)
I, [2024-03-19T16:36:05.075958 #1]  INFO -- : Successfully installed bundler-2.5                                                                                        .3
1 gem installed

I, [2024-03-19T16:36:05.076333 #1]  INFO -- : > cd /var/www/discourse && find /v                                                                                        ar/www/discourse ! -user discourse -exec chown discourse {} \+
I, [2024-03-19T16:36:09.016883 #1]  INFO -- :
I, [2024-03-19T16:36:09.017986 #1]  INFO -- : > cd /var/www/discourse && if [ "t                                                                                        ests-passed" != "tests-passed" ]; then
  rm -rf app/assets/javascripts/node_modules
fi
I, [2024-03-19T16:36:09.023415 #1]  INFO -- :
I, [2024-03-19T16:36:09.023782 #1]  INFO -- : > cd /var/www/discourse && su disc                                                                                        ourse -c 'yarn install --frozen-lockfile && yarn cache clean'
warning Resolution field "unset-value@2.0.1" is incompatible with requested vers                                                                                        ion "unset-value@^1.0.0"
warning Pattern ["wrap-ansi-cjs@npm:wrap-ansi@^7.0.0"] is trying to unpack in th                                                                                        e same destination "/home/discourse/.cache/yarn/v6/npm-wrap-ansi-cjs-7.0.0-67e14                                                                                        5cff510a6a6984bdf1152911d69d2eb9e43-integrity/node_modules/wrap-ansi-cjs" as pat                                                                                        tern ["wrap-ansi@^7.0.0"]. This could result in non-deterministic behavior, skip                                                                                        ping.
error An unexpected error occurred: "https://registry.yarnpkg.com/date-fns/-/date-fns-2.30.0.tgz: ESOCKETTIMEDOUT".
I, [2024-03-19T16:40:12.194716 #1]  INFO -- : yarn install v1.22.19
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
info If you think this is a bug, please open a bug report with the information provided in "/var/www/discourse/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

I, [2024-03-19T16:40:12.195985 #1]  INFO -- : Terminating async processes
112:signal-handler (1710866412) Received SIGTERM scheduling shutdown...
I, [2024-03-19T16:40:12.196356 #1]  INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main pid: 43
I, [2024-03-19T16:40:12.196559 #1]  INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 112
2024-03-19 16:40:12.196 UTC [43] LOG:  received fast shutdown request
112:M 19 Mar 2024 16:40:12.297 # User requested shutdown...
112:M 19 Mar 2024 16:40:12.297 * Saving the final RDB snapshot before exiting.
2024-03-19 16:40:12.630 UTC [43] LOG:  aborting any active transactions
2024-03-19 16:40:12.636 UTC [43] LOG:  background worker "logical replication launcher" (PID 52) exited with exit code 1
2024-03-19 16:40:12.640 UTC [47] LOG:  shutting down
112:M 19 Mar 2024 16:40:12.778 * DB saved on disk
112:M 19 Mar 2024 16:40:12.778 # Redis is now ready to exit, bye bye...
2024-03-19 16:40:13.564 UTC [43] LOG:  database system is shut down


FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && su discourse -c 'yarn install --frozen-lockfile && yarn cache clean' failed with return #<Process::Status: pid 457 exit 1>
Location of failure: /usr/local/lib/ruby/gems/3.2.0/gems/pups-1.2.1/lib/pups/exec_command.rb:132:in `spawn'
exec failed with the params {"cd"=>"$home", "cmd"=>["if [ \"$version\" != \"tests-passed\" ]; then\n  rm -rf app/assets/javascripts/node_modules\nfi", "su discourse -c 'yarn install --frozen-lockfile && yarn cache clean'"]}
bootstrap failed with exit code 1
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.
4207e08ef2f55ca92e6033e1cfa6585a54381e6f5947ad251e7eb5a4a21a37d1

If anyone has an idea of how to rectify this issue, please let me know. For now, I believe the best course of action is to switch out the failing hard drive for a better solid state as the failing drive might be writing garbled data and rebuilds are taking 20-30 minutes when they should take minutes at most, however, I am not entirely sure about this conclusion.

Since I cannot get a new drive at this time, I am most likely going to temporarily spin up an instance on DigitalOcean and run the site from there until I can get a drive replacement.

Please let me know about your thoughts, I would surely appreciate it.

I’ve only got two thoughts on this:

  1. The logs you shared are cut off on the right edge, so it’s hard to try to read them
  2. Looks like the fatal error might be a network one?
1 Like

Yeah, sorry for the issue with the logs, but yes I am running the commands as root with sudo.

I don’t believe it is a network issue as before the instance was able to communicate with the internet just fine. This post on GitHub also seems to mention Yarn timing out when being installed on a slow disk.

Then I’d say the disk running at the speed that it is is causing the primary failure point, not sure what caused the access denied chain though.

1 Like

Alright, thanks for your insight on this situation. I’ll spin up a DigitalOcean VM in the meantime until I get a drive replacement.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.