An upgrade knocked my site offline; how long until it's restored?

Installation
1

I tried to upgrade Discourse yesterday for https://creativecommunity.us. The upgrade was unsuccessful and knocked our site oflline. The following message is now displayed:
"Oops

The software powering this discussion forum encountered an unexpected problem. We apologize for the inconvenience.

Detailed information about the error was logged, and an automatic notification generated. We’ll take a look at it.

No further action is necessary. However, if the error condition persists, you can provide additional detail, including steps to reproduce the error, by posting a discussion topic in the site’s feedback category."

THis message says that “We’ll take a look at it.” How long does that typically take?

Thanks ~

1 Like
2

This message is displayed to users. I think the “we” here refers to “you”. So as an administrator, just waiting won’t solve the problem. :cry:

You can try to rebuild. If the error persists, paste the detailed error on meta to ask for help. :slightly_smiling_face:

6 Likes
3

Hello and welcome @Scott-CC :slight_smile:

As Linca said, a second rebuild can often give it a kick and resolve some things. :crossed_fingers:

Though as it’s an ‘oops’ page you may also find more detail in your /logs if there’s a conflict somewhere.

Safe mode is also worth trying as well. This can help narrow it down to a plugin or theme component which you can then disable while you work out what the exact issue is.

2 Likes
4

Thanks!

5

I’ll give it a shot. Thanks!

1 Like
6

Unfortunately, safe mode took me right back to the same error message. Any suggestions for a next step?

I am complete noob (just helping out a friend) with Discourse so huge thanks for any and all help.

7

This should be your next step:

If even this doesn’t work, you can watch the logs via e.g.:

docker exec app ls -l /shared/log/rails/
docker exec -it app tail -f /shared/log/rails/production.log
1 Like
8

Thanks. My apologies, as I am a complete noob at this, but to access the logs I’d have to login to the host, yes? (Digital Ocean)

And I have no clue what “docker exec app” is. Sorry.

9

Only if the /logs path doesn’t work on the site (you need to be logged in as admin).

1 Like
10

The /logs path leads to a white screen :frowning:

11

Can you share which plugins the site has installed? We may be able to spot one that has some known issues.

Without being able to get any extra details from logs or safe mode I think I would try a rebuild to see if that shakes something loose. You could also remove all the non-official plugins from the app.yml as well if you have any (and then rebuild), as they can often be the likely culprit if safe mode isn’t helping.

Though there is also the admin login safe mode route, just in case something about the site being login required is adding an obstacle to the regular safe mode path. If you add /u/admin-login to your site address and then enter an admin email as well as ticking the safe mode option that will send you a login link that may slip passed whatever’s going on. Long shot, but possibly still a shot. :slight_smile:

1 Like
12

Unfortunately, the admin login didn’t work.

I can access the back end at the host, Digital Ocean, but it’s terminal line Ubuntu and I have no idea what to do there.

I’m helping a friend and she has no idea what plugins are installed. I recall there weren’t many; chat and maybe one or two others.

Thanks for your help.

1 Like
13

EDIT: Looks like I used to manage your site. I’m rebuilding it now. You also needed to do the PostgreSQL 15 update; I did that too.

cd /var/discourse
./launcher rebuild app

That should fix it. It’ll take about 10 minutes. If not, see if the disk is full or you need swap.

df -h
free -h

There’s a chance that you need to add swap.

If that fails you want to solve the problem with money, contact me and I’ll get you back up and running.

5 Likes
14

You saved our bacon! Thank you!!

1 Like
15

I’m having the same issue as the OP – I tried to upgrade via the UI, but it failed and knocked the site offline.

I’ve been trying to update via the terminal (AWS), but the /launcher rebuild app hangs after about a half hour or so and simply won’t progress any further.

I’ve let it run overnight, but no joy. In fact, it seems to lock the instance and I have to restart it through the AWS console to even be able to connect again.

I thought maybe I didn’t have enough room to rebuild, but doubling the the EBS didn’t seem to help.

Here’s where I get to:

Summary 
Welcome to Ubuntu 24.04 LTS (GNU/Linux 6.8.0-1031-aws x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/pro

 System information as of Thu Jul 10 14:26:49 UTC 2025

  System load:  0.13               Processes:             111
  Usage of /:   56.9% of 47.39GB   Users logged in:       0
  Memory usage: 24%                IPv4 address for enX0: 172.31.60.182
  Swap usage:   0%


Expanded Security Maintenance for Applications is not enabled.

173 updates can be applied immediately.
To see these additional updates run: apt list --upgradable

Enable ESM Apps to receive additional future security updates.
See https://ubuntu.com/esm or run: sudo pro status


Last login: Thu Jul 10 12:14:09 2025 from 18.206.107.28
ubuntu@ip-172-31-60-182:~$ cd /var/discourse/
ubuntu@ip-172-31-60-182:/var/discourse$ launcher stop app
launcher: command not found
ubuntu@ip-172-31-60-182:/var/discourse$ /launcher stop app
-bash: /launcher: No such file or directory
ubuntu@ip-172-31-60-182:/var/discourse$ ./launcher stop app
Cannot connect to the docker daemon - verify it is running and you have access
ubuntu@ip-172-31-60-182:/var/discourse$ docker status
docker: 'status' is not a docker command.
See 'docker --help'
ubuntu@ip-172-31-60-182:/var/discourse$ ./launcher update app
Cannot connect to the docker daemon - verify it is running and you have access
ubuntu@ip-172-31-60-182:/var/discourse$ ./launcher rebuild app
Cannot connect to the docker daemon - verify it is running and you have access
ubuntu@ip-172-31-60-182:/var/discourse$ docker version
Client: Docker Engine - Community
 Version:           27.1.2
 API version:       1.46
 Go version:        go1.21.13
 Git commit:        d01f264
 Built:             Mon Aug 12 11:50:54 2024
 OS/Arch:           linux/amd64
 Context:           default
permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.46/version": dial unix /var/run/docker.sock: connect: permission denied
ubuntu@ip-172-31-60-182:/var/discourse$ sudo docker version
Client: Docker Engine - Community
 Version:           27.1.2
 API version:       1.46
 Go version:        go1.21.13
 Git commit:        d01f264
 Built:             Mon Aug 12 11:50:54 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          27.1.2
  API version:      1.46 (minimum version 1.24)
  Go version:       go1.21.13
  Git commit:       f9522e5
  Built:            Mon Aug 12 11:50:54 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.20
  GitCommit:        8fc6bcff51318944179630522a095cc9dbf9f353
 runc:
  Version:          1.1.13
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
ubuntu@ip-172-31-60-182:/var/discourse$ sudo su -
root@ip-172-31-60-182:~# cd /var/discourse
root@ip-172-31-60-182:/var/discourse# ./launcher update app
x86_64 arch detected.
Usage: launcher COMMAND CONFIG [--skip-prereqs] [--docker-args STRING]
Commands:
    start:       Start/initialize a container
    stop:        Stop a running container
    restart:     Restart a container
    destroy:     Stop and remove a container
    enter:       Open a shell to run commands inside the container
    logs:        View the Docker logs for a container
    bootstrap:   Bootstrap a container for the config based on a template
    run:         Run the given command with the config in the context of the last bootstrapped image
    rebuild:     Rebuild a container (destroy old, bootstrap, start new)
    cleanup:     Remove all containers that have stopped for > 24 hours
    start-cmd:   Generate docker command used to start container

Options:
    --skip-prereqs             Don't check launcher prerequisites
    --docker-args              Extra arguments to pass when running docker
    --skip-mac-address         Don't assign a mac address
    --run-image                Override the image used for running the container
root@ip-172-31-60-182:/var/discourse# ./launcher run app
x86_64 arch detected.
Unable to find image 'local_discourse/app:latest' locally
docker: Error response from daemon: pull access denied for local_discourse/app, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.
root@ip-172-31-60-182:/var/discourse# sudo ./launcher run app
x86_64 arch detected.
Unable to find image 'local_discourse/app:latest' locally
docker: Error response from daemon: pull access denied for local_discourse/app, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.
root@ip-172-31-60-182:/var/discourse# ./launcher rebuild app
x86_64 arch detected.
Ensuring launcher is up to date
Launcher is up-to-date
2.0.20250710-0019: Pulling from discourse/base
Digest: sha256:9f9e69653b83259d951b2bc63393a9200a6a260db0a0b55bf85ab567f596c9a3
Status: Image is up to date for discourse/base:2.0.20250710-0019
docker.io/discourse/base:2.0.20250710-0019
/usr/local/lib/ruby/gems/3.3.0/gems/pups-1.3.0/lib/pups.rb
/usr/local/bin/pups --stdin
I, [2025-07-10T14:34:06.749840 #1]  INFO -- : Reading from stdin
I, [2025-07-10T14:34:06.783263 #1]  INFO -- : File > /etc/service/postgres/run  chmod: +x  chown: 
I, [2025-07-10T14:34:06.787190 #1]  INFO -- : File > /etc/service/postgres/log/run  chmod: +x  chown: 
I, [2025-07-10T14:34:06.794555 #1]  INFO -- : File > /etc/runit/3.d/99-postgres  chmod: +x  chown: 
I, [2025-07-10T14:34:06.800229 #1]  INFO -- : File > /root/install_postgres  chmod: +x  chown: 
I, [2025-07-10T14:34:06.804200 #1]  INFO -- : File > /root/upgrade_postgres  chmod: +x  chown: 
I, [2025-07-10T14:34:06.808529 #1]  INFO -- : Replacing data_directory = '/var/lib/postgresql/15/main' with data_directory = '/shared/postgres_data' in /etc/postgresql/15/main/postgresql.conf
I, [2025-07-10T14:34:06.816873 #1]  INFO -- : Replacing (?-mix:#?listen_addresses *=.*) with listen_addresses = '*' in /etc/postgresql/15/main/postgresql.conf
I, [2025-07-10T14:34:06.817572 #1]  INFO -- : Replacing (?-mix:#?synchronous_commit *=.*) with synchronous_commit = $db_synchronous_commit in /etc/postgresql/15/main/postgresql.conf
I, [2025-07-10T14:34:06.821784 #1]  INFO -- : Replacing (?-mix:#?shared_buffers *=.*) with shared_buffers = $db_shared_buffers in /etc/postgresql/15/main/postgresql.conf
I, [2025-07-10T14:34:06.822781 #1]  INFO -- : Replacing (?-mix:#?work_mem *=.*) with work_mem = $db_work_mem in /etc/postgresql/15/main/postgresql.conf
I, [2025-07-10T14:34:06.824081 #1]  INFO -- : Replacing (?-mix:#?default_text_search_config *=.*) with default_text_search_config = '$db_default_text_search_config' in /etc/postgresql/15/main/postgresql.conf
I, [2025-07-10T14:34:06.825023 #1]  INFO -- : Replacing (?-mix:#?checkpoint_segments *=.*) with checkpoint_segments = $db_checkpoint_segments in /etc/postgresql/15/main/postgresql.conf
I, [2025-07-10T14:34:06.826225 #1]  INFO -- : Replacing (?-mix:#?logging_collector *=.*) with logging_collector = $db_logging_collector in /etc/postgresql/15/main/postgresql.conf
I, [2025-07-10T14:34:06.827143 #1]  INFO -- : Replacing (?-mix:#?log_min_duration_statement *=.*) with log_min_duration_statement = $db_log_min_duration_statement in /etc/postgresql/15/main/postgresql.conf
I, [2025-07-10T14:34:06.829174 #1]  INFO -- : Replacing (?-mix:^#local +replication +postgres +peer$) with local replication postgres  peer in /etc/postgresql/15/main/pg_hba.conf
I, [2025-07-10T14:34:06.831036 #1]  INFO -- : Replacing (?-mix:^host.*all.*all.*127.*$) with host all all 0.0.0.0/0 md5 in /etc/postgresql/15/main/pg_hba.conf
I, [2025-07-10T14:34:06.831299 #1]  INFO -- : Replacing (?-mix:^host.*all.*all.*::1\/128.*$) with host all all ::/0 md5 in /etc/postgresql/15/main/pg_hba.conf
I, [2025-07-10T14:34:06.832342 #1]  INFO -- : > if [ -f /root/install_postgres ]; then
  /root/install_postgres && rm -f /root/install_postgres
elif [ -e /shared/postgres_run/.s.PGSQL.5432 ]; then
  socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1
fi

2025/07/10 14:34:09 socat[33] E connect(, AF=1 "/shared/postgres_run/.s.PGSQL.5432", 36): Connection refused
I, [2025-07-10T14:34:09.325619 #1]  INFO -- : Generating locales (this might take a while)...
  en_US.UTF-8... done
Generation complete.

I, [2025-07-10T14:34:09.326466 #1]  INFO -- : > HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/15/bin/postmaster -D /etc/postgresql/15/main
I, [2025-07-10T14:34:09.336782 #1]  INFO -- : File > /usr/local/bin/create_db  chmod: +x  chown: 
I, [2025-07-10T14:34:09.356777 #1]  INFO -- : File > /var/lib/postgresql/take-database-backup  chmod: +x  chown: postgres:postgres
I, [2025-07-10T14:34:09.369678 #1]  INFO -- : File > /var/spool/cron/crontabs/postgres  chmod:   chown: 
I, [2025-07-10T14:34:09.370276 #1]  INFO -- : > sleep 5
2025-07-10 14:34:09.616 UTC [38] LOG:  starting PostgreSQL 15.13 (Debian 15.13-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
2025-07-10 14:34:09.618 UTC [38] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2025-07-10 14:34:09.618 UTC [38] LOG:  listening on IPv6 address "::", port 5432
2025-07-10 14:34:09.624 UTC [38] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2025-07-10 14:34:09.651 UTC [49] LOG:  database system was interrupted; last known up at 2025-07-10 12:26:43 UTC
2025-07-10 14:34:09.723 UTC [49] LOG:  database system was not properly shut down; automatic recovery in progress
2025-07-10 14:34:09.732 UTC [49] LOG:  redo starts at B/E8A9AB18
2025-07-10 14:34:09.732 UTC [49] LOG:  invalid record length at B/E8A9AC00: wanted 24, got 0
2025-07-10 14:34:09.732 UTC [49] LOG:  redo done at B/E8A9ABC8 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
2025-07-10 14:34:09.741 UTC [47] LOG:  checkpoint starting: end-of-recovery immediate wait
2025-07-10 14:34:09.758 UTC [47] LOG:  checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.005 s, sync=0.002 s, total=0.020 s; sync files=2, longest=0.001 s, average=0.001 s; distance=0 kB, estimate=0 kB
2025-07-10 14:34:09.780 UTC [38] LOG:  database system is ready to accept connections
I, [2025-07-10T14:34:14.374060 #1]  INFO -- : 
I, [2025-07-10T14:34:14.374950 #1]  INFO -- : > /usr/local/bin/create_db
2025-07-10 14:34:14.556 UTC [56] postgres@postgres ERROR:  database "discourse" already exists
2025-07-10 14:34:14.556 UTC [56] postgres@postgres STATEMENT:  CREATE DATABASE discourse;
createdb: error: database creation failed: ERROR:  database "discourse" already exists
2025-07-10 14:34:14.653 UTC [59] postgres@discourse ERROR:  role "discourse" already exists
2025-07-10 14:34:14.653 UTC [59] postgres@discourse STATEMENT:  create user discourse;
ERROR:  role "discourse" already exists
NOTICE:  extension "hstore" already exists, skipping
NOTICE:  extension "pg_trgm" already exists, skipping
NOTICE:  extension "vector" already exists, skipping
NOTICE:  version "0.8.0" of extension "vector" is already installed
NOTICE:  extension "hstore" already exists, skipping
NOTICE:  extension "pg_trgm" already exists, skipping
NOTICE:  extension "vector" already exists, skipping
NOTICE:  version "0.8.0" of extension "vector" is already installed
I, [2025-07-10T14:34:15.375245 #1]  INFO -- : GRANT
ALTER SCHEMA
CREATE EXTENSION
CREATE EXTENSION
CREATE EXTENSION
ALTER EXTENSION
CREATE EXTENSION
CREATE EXTENSION
CREATE EXTENSION
ALTER EXTENSION
UPDATE 0

I, [2025-07-10T14:34:15.375613 #1]  INFO -- : > echo postgres installed!
I, [2025-07-10T14:34:15.380090 #1]  INFO -- : postgres installed!

I, [2025-07-10T14:34:15.384491 #1]  INFO -- : File > /etc/service/redis/run  chmod: +x  chown: 
I, [2025-07-10T14:34:15.388524 #1]  INFO -- : File > /etc/service/redis/log/run  chmod: +x  chown: 
I, [2025-07-10T14:34:15.392347 #1]  INFO -- : File > /etc/runit/3.d/10-redis  chmod: +x  chown: 
I, [2025-07-10T14:34:15.394875 #1]  INFO -- : Replacing daemonize yes with  in /etc/redis/redis.conf
I, [2025-07-10T14:34:15.400010 #1]  INFO -- : Replacing (?-mix:^pidfile.*$) with  in /etc/redis/redis.conf
I, [2025-07-10T14:34:15.400706 #1]  INFO -- : > install -d -m 0755 -o redis -g redis /shared/redis_data
I, [2025-07-10T14:34:15.407363 #1]  INFO -- : 
I, [2025-07-10T14:34:15.407683 #1]  INFO -- : Replacing (?-mix:^logfile.*$) with logfile "" in /etc/redis/redis.conf
I, [2025-07-10T14:34:15.408567 #1]  INFO -- : Replacing (?-mix:^bind .*$) with  in /etc/redis/redis.conf
I, [2025-07-10T14:34:15.409953 #1]  INFO -- : Replacing (?-mix:^dir .*$) with dir /shared/redis_data in /etc/redis/redis.conf
I, [2025-07-10T14:34:15.411380 #1]  INFO -- : Replacing (?-mix:^protected-mode yes) with protected-mode no in /etc/redis/redis.conf
I, [2025-07-10T14:34:15.412629 #1]  INFO -- : Replacing # io-threads 4 with io-threads $redis_io_threads in /etc/redis/redis.conf
I, [2025-07-10T14:34:15.413828 #1]  INFO -- : > echo redis installed
I, [2025-07-10T14:34:15.415894 #1]  INFO -- : redis installed

I, [2025-07-10T14:34:15.416407 #1]  INFO -- : > cat /etc/redis/redis.conf | grep logfile
I, [2025-07-10T14:34:15.422899 #1]  INFO -- : logfile ""

I, [2025-07-10T14:34:15.423444 #1]  INFO -- : > exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf
I, [2025-07-10T14:34:15.426194 #1]  INFO -- : > sleep 10
105:C 10 Jul 2025 14:34:15.460 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
105:C 10 Jul 2025 14:34:15.460 # Redis version=7.0.15, bits=64, commit=00000000, modified=0, pid=105, just started
105:C 10 Jul 2025 14:34:15.460 # Configuration loaded
105:M 10 Jul 2025 14:34:15.460 * monotonic clock: POSIX clock_gettime
105:M 10 Jul 2025 14:34:15.464 * Running mode=standalone, port=6379.
105:M 10 Jul 2025 14:34:15.465 # Server initialized
105:M 10 Jul 2025 14:34:15.465 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
105:M 10 Jul 2025 14:34:15.465 # WARNING Your system is configured to use the 'xen' clocksource which might lead to degraded performance. Check the result of the [slow-clocksource] system check: run 'redis-server --check-system' to check if the system's clocksource isn't degrading performance.
105:M 10 Jul 2025 14:34:15.471 * Loading RDB produced by version 7.0.15
105:M 10 Jul 2025 14:34:15.471 * RDB age 7659 seconds
105:M 10 Jul 2025 14:34:15.471 * RDB memory usage when created 7.58 Mb
105:M 10 Jul 2025 14:34:15.505 * Done loading RDB, keys loaded: 1240, keys expired: 18.
105:M 10 Jul 2025 14:34:15.505 * DB loaded from disk: 0.036 seconds
105:M 10 Jul 2025 14:34:15.505 * Ready to accept connections
I, [2025-07-10T14:34:25.428346 #1]  INFO -- : 
I, [2025-07-10T14:34:25.428571 #1]  INFO -- : > thpoff echo "thpoff is installed!"
I, [2025-07-10T14:34:25.433266 #1]  INFO -- : thpoff is installed!

I, [2025-07-10T14:34:25.434051 #1]  INFO -- : > /usr/local/bin/ruby -e 'if ENV["DISCOURSE_SMTP_ADDRESS"] == "smtp.example.com"; puts "Aborting! Mail is not configured!"; exit 1; end'
I, [2025-07-10T14:34:25.623161 #1]  INFO -- : 
I, [2025-07-10T14:34:25.623817 #1]  INFO -- : > /usr/local/bin/ruby -e 'if ENV["DISCOURSE_HOSTNAME"] == "discourse.example.com"; puts "Aborting! Domain is not configured!"; exit 1; end'
I, [2025-07-10T14:34:25.707792 #1]  INFO -- : 
I, [2025-07-10T14:34:25.708514 #1]  INFO -- : > /usr/local/bin/ruby -e 'if (ENV["DISCOURSE_CDN_URL"] || "")[0..1] == "//"; puts "Aborting! CDN must have a protocol specified. Once fixed you should rebake your posts now to correct all posts."; exit 1; end'
I, [2025-07-10T14:34:25.792429 #1]  INFO -- : 
I, [2025-07-10T14:34:25.793174 #1]  INFO -- : > rm -f /etc/cron.d/anacron
I, [2025-07-10T14:34:25.798896 #1]  INFO -- : 
I, [2025-07-10T14:34:25.801342 #1]  INFO -- : File > /etc/cron.d/anacron  chmod:   chown: 
I, [2025-07-10T14:34:25.808492 #1]  INFO -- : File > /etc/runit/1.d/copy-env  chmod: +x  chown: 
I, [2025-07-10T14:34:25.812525 #1]  INFO -- : File > /etc/service/unicorn/run  chmod: +x  chown: 
I, [2025-07-10T14:34:25.816576 #1]  INFO -- : File > /etc/service/nginx/run  chmod: +x  chown: 
I, [2025-07-10T14:34:25.820635 #1]  INFO -- : File > /etc/runit/3.d/01-nginx  chmod: +x  chown: 
I, [2025-07-10T14:34:25.824561 #1]  INFO -- : File > /etc/runit/3.d/02-unicorn  chmod: +x  chown: 
I, [2025-07-10T14:34:25.825004 #1]  INFO -- : Replacing # postgres with if [ -f /root/install_postgres ]; then
  /root/install_postgres
  rm /root/install_postgres
fi
sv start postgres || exit 1
 in /etc/service/unicorn/run
I, [2025-07-10T14:34:25.826554 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git clean -f
I, [2025-07-10T14:34:26.964358 #1]  INFO -- : 
I, [2025-07-10T14:34:26.964998 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse bash -c '
  set -o errexit
  if [ $(git rev-parse --is-shallow-repository) == "true" ]; then
      git remote set-branches --add origin main
      git remote set-branches origin tests-passed
      git fetch --depth 1 origin tests-passed
  else
      git fetch --tags --prune-tags --prune --force origin
  fi
'
From https://github.com/discourse/discourse
 - [deleted]             (none)     -> origin/0-a-reactions
 - [deleted]             (none)     -> origin/dev/gravatar-toggle-setting
 - [deleted]             (none)     -> origin/dev/load-admin-bundles-for-staff-only
 - [deleted]             (none)     -> origin/fix-composer-dmenu-tablets
 - [deleted]             (none)     -> origin/fix-readonly-take-2
 - [deleted]             (none)     -> origin/fix/mt/upload-optimization
   63df79592..2e11eb20b  main                   -> origin/main
 + 37e2eed7b...e8ef2f73b dev-floatkit-autocomplete-d-editor -> origin/dev-floatkit-autocomplete-d-editor  (forced update)
   a42149b9f..5f1148cc8  dev-lock-alias         -> origin/dev-lock-alias
 * [new branch]          dev/remove-automatic-backups-enabled -> origin/dev/remove-automatic-backups-enabled
 * [new branch]          edit-category-description-layout-fix -> origin/edit-category-description-layout-fix
   4ebd596a2..75d289b97  feature/composer-toolbar-heading-list -> origin/feature/composer-toolbar-heading-list
   44fb1a213..f07b9cbab  feature/theme-site-settings -> origin/feature/theme-site-settings
   7faaab65c..6043ede0a  load-post-draft-from-reply-btn -> origin/load-post-draft-from-reply-btn
 + dcaa1a5cf...cae266b62 loic-upgrade-rails-8.0 -> origin/loic-upgrade-rails-8.0  (forced update)
   b1e0591e3..e8ef2f73b  sandbox/dev-xp-1       -> origin/sandbox/dev-xp-1
   876dbe662..530fbe111  sandbox/dev-xp-2       -> origin/sandbox/dev-xp-2
   daabd040e..75d289b97  sandbox/member-xp-4    -> origin/sandbox/member-xp-4
 * [new branch]          send-email-when-signing-up-via-external-auth -> origin/send-email-when-signing-up-via-external-auth
 * [new branch]          skipped-address-filter -> origin/skipped-address-filter
   f70640f77..a83bd0f67  stable                 -> origin/stable
   6e03440b0..2e11eb20b  tests-passed           -> origin/tests-passed
 * [new branch]          translations-from-crowdin-main -> origin/translations-from-crowdin-main
 * [new branch]          translations-from-crowdin-stable -> origin/translations-from-crowdin-stable
I, [2025-07-10T14:34:28.010715 #1]  INFO -- : 
I, [2025-07-10T14:34:28.011023 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse bash -c '
  set -o errexit
  if [[ $(git symbolic-ref --short HEAD) == tests-passed ]] ; then
      git pull
  else
      git -c advice.detachedHead=false checkout tests-passed
  fi
'
Switched to a new branch 'tests-passed'
I, [2025-07-10T14:34:40.617744 #1]  INFO -- : branch 'tests-passed' set up to track 'origin/tests-passed'.

I, [2025-07-10T14:34:40.618255 #1]  INFO -- : > cd /var/www/discourse && sudo -H -E -u discourse git config user.discourse-version tests-passed
I, [2025-07-10T14:34:40.629799 #1]  INFO -- : 
I, [2025-07-10T14:34:40.630226 #1]  INFO -- : > cd /var/www/discourse && mkdir -p tmp
I, [2025-07-10T14:34:40.633195 #1]  INFO -- : 
I, [2025-07-10T14:34:40.633585 #1]  INFO -- : > cd /var/www/discourse && chown discourse:www-data tmp
I, [2025-07-10T14:34:40.636417 #1]  INFO -- : 
I, [2025-07-10T14:34:40.636731 #1]  INFO -- : > cd /var/www/discourse && mkdir -p tmp/pids
I, [2025-07-10T14:34:40.639770 #1]  INFO -- : 
I, [2025-07-10T14:34:40.640244 #1]  INFO -- : > cd /var/www/discourse && mkdir -p tmp/sockets
I, [2025-07-10T14:34:40.643048 #1]  INFO -- : 
I, [2025-07-10T14:34:40.643395 #1]  INFO -- : > cd /var/www/discourse && touch tmp/.gitkeep
I, [2025-07-10T14:34:40.648782 #1]  INFO -- : 
I, [2025-07-10T14:34:40.649284 #1]  INFO -- : > cd /var/www/discourse && mkdir -p                    /shared/log/rails
I, [2025-07-10T14:34:40.653705 #1]  INFO -- : 
I, [2025-07-10T14:34:40.654203 #1]  INFO -- : > cd /var/www/discourse && bash -c "touch -a           /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log"
I, [2025-07-10T14:34:40.660408 #1]  INFO -- : 
I, [2025-07-10T14:34:40.660856 #1]  INFO -- : > cd /var/www/discourse && bash -c "ln    -s           /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log /var/www/discourse/log"
I, [2025-07-10T14:34:40.669796 #1]  INFO -- : 
I, [2025-07-10T14:34:40.670284 #1]  INFO -- : > cd /var/www/discourse && bash -c "mkdir -p           /shared/{uploads,backups}"
I, [2025-07-10T14:34:40.674466 #1]  INFO -- : 
I, [2025-07-10T14:34:40.674928 #1]  INFO -- : > cd /var/www/discourse && bash -c "ln    -s           /shared/{uploads,backups} /var/www/discourse/public"
I, [2025-07-10T14:34:40.680307 #1]  INFO -- : 
I, [2025-07-10T14:34:40.680774 #1]  INFO -- : > cd /var/www/discourse && bash -c "mkdir -p           /shared/tmp/{backups,restores}"
I, [2025-07-10T14:34:40.686442 #1]  INFO -- : 
I, [2025-07-10T14:34:40.686862 #1]  INFO -- : > cd /var/www/discourse && bash -c "ln    -s           /shared/tmp/{backups,restores} /var/www/discourse/tmp"
I, [2025-07-10T14:34:40.691052 #1]  INFO -- : 
I, [2025-07-10T14:34:40.691529 #1]  INFO -- : > cd /var/www/discourse && chown -R discourse:www-data /shared/log/rails /shared/uploads /shared/backups /shared/tmp
I, [2025-07-10T14:34:40.943384 #1]  INFO -- : 
I, [2025-07-10T14:34:40.943501 #1]  INFO -- : > cd /var/www/discourse && [ ! -d public/plugins ] || find public/plugins/ -maxdepth 1 -xtype l -delete
I, [2025-07-10T14:34:40.946212 #1]  INFO -- : 
I, [2025-07-10T14:34:40.946735 #1]  INFO -- : Replacing # redis with if [ ! -d /shared/redis_data ]; then
  install -d -m 0755 -o redis -g redis /shared/redis_data
fi
sv start redis || exit 1 in /etc/service/unicorn/run
I, [2025-07-10T14:34:40.947772 #1]  INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/docker_manager.git
Cloning into 'docker_manager'...
I, [2025-07-10T14:34:41.920020 #1]  INFO -- : 
I, [2025-07-10T14:34:41.920542 #1]  INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/discourse-yearly-review
Cloning into 'discourse-yearly-review'...
I, [2025-07-10T14:34:42.189827 #1]  INFO -- : 
I, [2025-07-10T14:34:42.190570 #1]  INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/discourse-cakeday
Cloning into 'discourse-cakeday'...
I, [2025-07-10T14:34:42.489411 #1]  INFO -- : 
I, [2025-07-10T14:34:42.489888 #1]  INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/angusmcleod/discourse-ratings.git
Cloning into 'discourse-ratings'...
I, [2025-07-10T14:34:42.792033 #1]  INFO -- : 
I, [2025-07-10T14:34:42.792683 #1]  INFO -- : > cd /var/www/discourse/plugins && git clone https://github.com/discourse/discourse-calendar
Cloning into 'discourse-calendar'...
I, [2025-07-10T14:34:43.546091 #1]  INFO -- : 
I, [2025-07-10T14:34:43.547502 #1]  INFO -- : > cp /var/www/discourse/config/nginx.sample.conf /etc/nginx/conf.d/discourse.conf
I, [2025-07-10T14:34:43.561409 #1]  INFO -- : 
I, [2025-07-10T14:34:43.561632 #1]  INFO -- : > rm /etc/nginx/sites-enabled/default
I, [2025-07-10T14:34:43.566037 #1]  INFO -- : 
I, [2025-07-10T14:34:43.566244 #1]  INFO -- : > mkdir -p /var/nginx/cache
I, [2025-07-10T14:34:43.568477 #1]  INFO -- : 
I, [2025-07-10T14:34:43.569068 #1]  INFO -- : > grep -q 'outlets/before-server' /etc/nginx/conf.d/discourse.conf || ( >&2 echo 'The "before-server" Nginx outlet is missing. This version of discourse_docker is not compatible with the chosen Discourse version.' ; exit 1 )
I, [2025-07-10T14:34:43.571902 #1]  INFO -- : 
I, [2025-07-10T14:34:43.572250 #1]  INFO -- : > grep -q 'outlets/server' /etc/nginx/conf.d/discourse.conf || ( >&2 echo 'The "server" Nginx outlet is missing. This version of discourse_docker is not compatible with the chosen Discourse version.' ; exit 1 )
I, [2025-07-10T14:34:43.575152 #1]  INFO -- : 
I, [2025-07-10T14:34:43.575592 #1]  INFO -- : > grep -q 'outlets/discourse' /etc/nginx/conf.d/discourse.conf || ( >&2 echo 'The "discourse" Nginx outlet is missing. This version of discourse_docker is not compatible with the chosen Discourse version.' ; exit 1 )
I, [2025-07-10T14:34:43.578518 #1]  INFO -- : 
I, [2025-07-10T14:34:43.578955 #1]  INFO -- : > mkdir -p /etc/nginx/conf.d/outlets/before-server
I, [2025-07-10T14:34:43.581226 #1]  INFO -- : 
I, [2025-07-10T14:34:43.581753 #1]  INFO -- : > touch /etc/nginx/conf.d/outlets/before-server/20-redirect-http-to-https.conf
I, [2025-07-10T14:34:43.583621 #1]  INFO -- : 
I, [2025-07-10T14:34:43.583900 #1]  INFO -- : > touch /etc/nginx/conf.d/outlets/before-server/30-ratelimited.conf
I, [2025-07-10T14:34:43.586013 #1]  INFO -- : 
I, [2025-07-10T14:34:43.586181 #1]  INFO -- : > mkdir -p /etc/nginx/conf.d/outlets/server
I, [2025-07-10T14:34:43.588522 #1]  INFO -- : 
I, [2025-07-10T14:34:43.589108 #1]  INFO -- : > touch /etc/nginx/conf.d/outlets/server/10-http.conf
I, [2025-07-10T14:34:43.591027 #1]  INFO -- : 
I, [2025-07-10T14:34:43.591377 #1]  INFO -- : > touch /etc/nginx/conf.d/outlets/server/20-https.conf
I, [2025-07-10T14:34:43.593154 #1]  INFO -- : 
I, [2025-07-10T14:34:43.593653 #1]  INFO -- : > touch /etc/nginx/conf.d/outlets/server/30-offline-page.conf
I, [2025-07-10T14:34:43.595616 #1]  INFO -- : 
I, [2025-07-10T14:34:43.595991 #1]  INFO -- : > mkdir -p /etc/nginx/conf.d/outlets/discourse
I, [2025-07-10T14:34:43.598072 #1]  INFO -- : 
I, [2025-07-10T14:34:43.598471 #1]  INFO -- : > touch /etc/nginx/conf.d/outlets/discourse/20-https.conf
I, [2025-07-10T14:34:43.600421 #1]  INFO -- : 
I, [2025-07-10T14:34:43.601002 #1]  INFO -- : > touch /etc/nginx/conf.d/outlets/discourse/30-ratelimited.conf
I, [2025-07-10T14:34:43.602883 #1]  INFO -- : 
I, [2025-07-10T14:34:43.604077 #1]  INFO -- : Replacing pid /run/nginx.pid; with daemon off; in /etc/nginx/nginx.conf
I, [2025-07-10T14:34:43.606485 #1]  INFO -- : Replacing (?-mix:worker_connections.+$) with worker_connections $nginx_worker_connections; in /etc/nginx/nginx.conf
I, [2025-07-10T14:34:43.606808 #1]  INFO -- : Replacing (?-mix:client_max_body_size.+$) with client_max_body_size $upload_size; in /etc/nginx/conf.d/discourse.conf
I, [2025-07-10T14:34:43.607501 #1]  INFO -- : > sed -i 's#listen 80;##g' /etc/nginx/conf.d/discourse.conf
I, [2025-07-10T14:34:43.611686 #1]  INFO -- : 
I, [2025-07-10T14:34:43.612254 #1]  INFO -- : > if [ -f "/proc/net/if_inet6" ]; then
  echo "listen 80;\nlisten [::]:80;" > /etc/nginx/conf.d/outlets/server/10-http.conf
else
  echo "listen 80;" > /etc/nginx/conf.d/outlets/server/10-http.conf
fi
I, [2025-07-10T14:34:43.614205 #1]  INFO -- : 
I, [2025-07-10T14:34:43.614712 #1]  INFO -- : > echo "done configuring web"
I, [2025-07-10T14:34:43.616568 #1]  INFO -- : done configuring web

I, [2025-07-10T14:34:43.617121 #1]  INFO -- : > cd /var/www/discourse && gem install bundler --conservative -v $(awk '/BUNDLED WITH/ { getline; gsub(/ /,""); print $0 }' Gemfile.lock)
I, [2025-07-10T14:34:44.144672 #1]  INFO -- : 
I, [2025-07-10T14:34:44.145379 #1]  INFO -- : > cd /var/www/discourse && find /var/www/discourse ! -user discourse -exec chown discourse {} \+
I, [2025-07-10T14:34:58.288175 #1]  INFO -- : 
I, [2025-07-10T14:34:58.298580 #1]  INFO -- : > cd /var/www/discourse && if [ -f yarn.lock ]; then
  if [ -d node_modules/.pnpm ]; then
    echo "This version of Discourse uses yarn, but pnpm node_modules are preset. Cleaning up..."
    find ./node_modules ./app/assets/javascripts/*/node_modules -mindepth 1 -maxdepth 1 -exec rm -rf {} +
  fi
  su discourse -c 'yarn install --frozen-lockfile && yarn cache clean'
else
  su discourse -c 'CI=1 pnpm install --frozen-lockfile && pnpm prune'
fi
I, [2025-07-10T14:35:07.928627 #1]  INFO -- : Scope: all 16 workspace projects
Lockfile is up to date, resolution step is skipped
Already up to date

Done in 5.4s using pnpm v9.15.9
Lockfile is up to date, resolution step is skipped
Already up to date


I, [2025-07-10T14:35:07.934991 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle install --jobs $(($(nproc) - 1)) --retry 3'
I, [2025-07-10T14:35:09.407430 #1]  INFO -- : Bundle complete! 143 Gemfile dependencies, 190 gems now installed.
Gems in the groups 'test' and 'development' were not installed.
Bundled gems are installed into `./vendor/bundle`
3 installed gems you directly depend on are looking for funding.
  Run `bundle fund` for details

I, [2025-07-10T14:35:09.407945 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle clean'
I, [2025-07-10T14:35:09.843516 #1]  INFO -- : 
I, [2025-07-10T14:35:09.844596 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'find /var/www/discourse/vendor/bundle -name cache -not -path "*/gems/*" -type d -exec rm -rf {} +'
I, [2025-07-10T14:35:13.000166 #1]  INFO -- : 
I, [2025-07-10T14:35:13.000275 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'find /var/www/discourse/vendor/bundle -name tmp -type d -exec rm -rf {} +'
I, [2025-07-10T14:35:13.145199 #1]  INFO -- : 
I, [2025-07-10T14:35:13.150515 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'LOAD_PLUGINS=0 bundle exec rake plugin:pull_compatible_all'
I, [2025-07-10T14:35:19.066578 #1]  INFO -- : discourse-cakeday is already at latest compatible version
discourse-calendar is already at latest compatible version
discourse-ratings is already at latest compatible version
discourse-yearly-review is already at latest compatible version
docker_manager is already at latest compatible version

I, [2025-07-10T14:35:19.068741 #1]  INFO -- : > exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf
I, [2025-07-10T14:35:19.083032 #1]  INFO -- : > sleep 10
399:C 10 Jul 2025 14:35:19.103 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
399:C 10 Jul 2025 14:35:19.103 # Redis version=7.0.15, bits=64, commit=00000000, modified=0, pid=399, just started
399:C 10 Jul 2025 14:35:19.104 # Configuration loaded
399:M 10 Jul 2025 14:35:19.105 * monotonic clock: POSIX clock_gettime
399:M 10 Jul 2025 14:35:19.105 # Warning: Could not create server TCP listening socket *:6379: bind: Address already in use
399:M 10 Jul 2025 14:35:19.105 # Failed listening on port 6379 (TCP), aborting.
I, [2025-07-10T14:35:29.088163 #1]  INFO -- : 
I, [2025-07-10T14:35:29.089838 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate'
I, [2025-07-10T14:35:58.643957 #1]  INFO -- : 
I, [2025-07-10T14:35:58.671217 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake assets:precompile:build'
Node.js heap_size_limit is less than 2048MB. Setting --max-old-space-size=2048 and CHEAP_SOURCE_MAPS=1
No existing build info file found.
Running full core build...
...[ConfigLoader]
...[Babel: @embroider/macros > applyPatches]
...[Babel: pretty-text > applyPatches]
...[Babel: @ember/legacy-built-in-components > applyPatches]
...[Babel: ember-source > applyPatches]
[BABEL] Note: The code generator has deoptimised the styling of /var/www/discourse/app/assets/javascripts/discourse/ember/ember-template-compiler.js as it exceeds the max of 500KB.
[BABEL] Note: The code generator has deoptimised the styling of /var/www/discourse/app/assets/javascripts/discourse/ember/ember.js as it exceeds the max of 500KB.
...[Babel: @glimmer/component > applyPatches]
...[Babel: @ember/render-modifiers > applyPatches]
...[Babel: dialog-holder > applyPatches]
...[Babel: ember-this-fallback > applyPatches]
...[Babel: ember-cached-decorator-polyfill > applyPatches]
...[Babel: float-kit > applyPatches]
...[Babel: select-kit > applyPatches]
...[@embroider/compat/app]
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
2025-07-10 14:39:10.537 UTC [47] LOG:  checkpoint starting: time
2025-07-10 14:39:15.479 UTC [47] LOG:  checkpoint complete: wrote 32 buffers (0.2%); 0 WAL file(s) added, 0 removed, 0 recycled; write=4.708 s, sync=0.009 s, total=5.522 s; sync files=23, longest=0.007 s, average=0.001 s; distance=134 kB, estimate=134 kB
105:M 10 Jul 2025 14:39:16.016 * 100 changes in 300 seconds. Saving...
105:M 10 Jul 2025 14:39:16.063 * Background saving started by pid 676
676:C 10 Jul 2025 14:39:17.341 * DB saved on disk
676:C 10 Jul 2025 14:39:17.341 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB
105:M 10 Jul 2025 14:39:17.368 * Background saving terminated with success
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
...[@embroider/webpack]
...[SourceMapConcat]
[WARN] (broccoli-terser-sourcemap) Minifying "assets/admin.js" took: 331711ms (more than 20,000ms)
...[Babel: discourse > applyPatches]
...[SourceMapConcat]
...[Babel: discourse-plugins > applyPatches]
...[SourceMapConcat]
16

Small update – I increased the swap space and that seemed to allow the update to complete. Sadly, even though the app seems to be running in docker, my site is still unresponsive.