Stuck and lost updating forum, problems with PG migration

Hey posting here after trying everything and got stuck with my upgrade.

  • I was updating the forum after many months. I ran a backup as usual and downloaded it.
  • Apparently, in this update process, there is a migration from PG 13 to 15.
    Everything was running with no issues until the update failed I was interrupted due to lack of disk space.
  • I went to my Digitalocean droplet, stoped it, expanded the disk, and turn it on again
  • I ran some regular commants to release disk space (removing old logs, ./launcher cleanup, etc)

So I started the rebuild, and then then hit the first wall:

root@discourse-ubuntu-s-2vcpu-4gb-fra1-01:/var/discourse# ./launcher rebuild app
WARNING: Docker version 20.10.1 deprecated, recommend upgrade to 24.0.7 or newer.
x86_64 arch detected.
WARNING: containers/app.yml file is world-readable. You can secure this file by running: chmod o-rwx containers/app.yml
Ensuring launcher is up to date
Fetching origin
Launcher is up-to-date
2.0.20250226-0128: Pulling from discourse/base
Digest: sha256:6f18aa2cd22bba0deb91d69194e577d4f96130ad555ae8ec646a8792cbfe37db
Status: Image is up to date for discourse/base:2.0.20250226-0128
docker.io/discourse/base:2.0.20250226-0128
/usr/local/lib/ruby/gems/3.3.0/gems/pups-1.2.1/lib/pups.rb
/usr/local/bin/pups --stdin
I, [2025-06-11T08:57:06.040908 #1]  INFO -- : Reading from stdin
I, [2025-06-11T08:57:06.055466 #1]  INFO -- : File > /etc/service/postgres/run  chmod: +x  chown:
I, [2025-06-11T08:57:06.059785 #1]  INFO -- : File > /etc/service/postgres/log/run  chmod: +x  chown:
I, [2025-06-11T08:57:06.064063 #1]  INFO -- : File > /etc/runit/3.d/99-postgres  chmod: +x  chown:
I, [2025-06-11T08:57:06.068330 #1]  INFO -- : File > /root/install_postgres  chmod: +x  chown:
I, [2025-06-11T08:57:06.072788 #1]  INFO -- : File > /root/upgrade_postgres  chmod: +x  chown:
I, [2025-06-11T08:57:06.073111 #1]  INFO -- : Replacing data_directory = '/var/lib/postgresql/15/main' with data_directory = '/shared/postgres_data' in /etc/postgresql/15/main/postgresql.conf
I, [2025-06-11T08:57:06.074045 #1]  INFO -- : Replacing (?-mix:#?listen_addresses *=.*) with listen_addresses = '*' in /etc/postgresql/15/main/postgresql.conf
I, [2025-06-11T08:57:06.074682 #1]  INFO -- : Replacing (?-mix:#?synchronous_commit *=.*) with synchronous_commit = $db_synchronous_commit in /etc/postgresql/15/main/postgresql.conf
I, [2025-06-11T08:57:06.075269 #1]  INFO -- : Replacing (?-mix:#?shared_buffers *=.*) with shared_buffers = $db_shared_buffers in /etc/postgresql/15/main/postgresql.conf
I, [2025-06-11T08:57:06.080274 #1]  INFO -- : Replacing (?-mix:#?work_mem *=.*) with work_mem = $db_work_mem in /etc/postgresql/15/main/postgresql.conf
I, [2025-06-11T08:57:06.081190 #1]  INFO -- : Replacing (?-mix:#?default_text_search_config *=.*) with default_text_search_config = '$db_default_text_search_config' in /etc/postgresql/15/main/postgresql.conf
I, [2025-06-11T08:57:06.081687 #1]  INFO -- : Replacing (?-mix:#?checkpoint_segments *=.*) with checkpoint_segments = $db_checkpoint_segments in /etc/postgresql/15/main/postgresql.conf
I, [2025-06-11T08:57:06.082397 #1]  INFO -- : Replacing (?-mix:#?logging_collector *=.*) with logging_collector = $db_logging_collector in /etc/postgresql/15/main/postgresql.conf
I, [2025-06-11T08:57:06.083025 #1]  INFO -- : Replacing (?-mix:#?log_min_duration_statement *=.*) with log_min_duration_statement = $db_log_min_duration_statement in /etc/postgresql/15/main/postgresql.conf
I, [2025-06-11T08:57:06.083657 #1]  INFO -- : Replacing (?-mix:^#local +replication +postgres +peer$) with local replication postgres  peer in /etc/postgresql/15/main/pg_hba.conf
I, [2025-06-11T08:57:06.083968 #1]  INFO -- : Replacing (?-mix:^host.*all.*all.*127.*$) with host all all 0.0.0.0/0 md5 in /etc/postgresql/15/main/pg_hba.conf
I, [2025-06-11T08:57:06.084391 #1]  INFO -- : Replacing (?-mix:^host.*all.*all.*::1\/128.*$) with host all all ::/0 md5 in /etc/postgresql/15/main/pg_hba.conf
I, [2025-06-11T08:57:06.084877 #1]  INFO -- : > if [ -f /root/install_postgres ]; then
  /root/install_postgres && rm -f /root/install_postgres
elif [ -e /shared/postgres_run/.s.PGSQL.5432 ]; then
  socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1
fi

failed to set locale!
[error] character map file `UTF-8' not found: No such file or directory
failed to set locale!
[error] default character map file `ANSI_X3.4-1968' not found: No such file or directory
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
	LANGUAGE = "en_US.UTF-8",
	LC_ALL = "en_US.UTF-8",
	LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
popen failure: Cannot allocate memory
initdb: error: program "postgres" is needed by initdb but was not found in the same directory as "/usr/lib/postgresql/15/bin/initdb"
I, [2025-06-11T08:57:07.265267 #1]  INFO -- : Generating locales (this might take a while)...
  en_US.UTF-8... done
Generation complete.
Upgrading PostgreSQL from version 13 to 15

I, [2025-06-11T08:57:07.265539 #1]  INFO -- : > HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/15/bin/postmaster -D /etc/postgresql/15/main
I, [2025-06-11T08:57:07.267705 #1]  INFO -- : Terminating async processes
2025-06-11 08:57:07.283 UTC [50] FATAL:  database files are incompatible with server
2025-06-11 08:57:07.283 UTC [50] DETAIL:  The data directory was initialized by PostgreSQL version 13, which is not compatible with this version 15.12 (Debian 15.12-1.pgdg120+1).

Searching on Google and a bit of chatGPT to understand the log, I decided to go back to PG 13 and focus on restore the forum, editing the template in the yml to force “templates/postgres.13.template.yml”

And the hit the second wall:


root@discourse-ubuntu-s-2vcpu-4gb-fra1-01:/var/discourse# ./launcher rebuild app
WARNING: Docker version 20.10.1 deprecated, recommend upgrade to 24.0.7 or newer.
x86_64 arch detected.
WARNING: containers/app.yml file is world-readable. You can secure this file by running: chmod o-rwx containers/app.yml
Ensuring launcher is up to date
Fetching origin
Launcher is up-to-date
2.0.20250226-0128: Pulling from discourse/base
Digest: sha256:6f18aa2cd22bba0deb91d69194e577d4f96130ad555ae8ec646a8792cbfe37db
Status: Image is up to date for discourse/base:2.0.20250226-0128
docker.io/discourse/base:2.0.20250226-0128
/usr/local/lib/ruby/gems/3.3.0/gems/pups-1.2.1/lib/pups.rb
/usr/local/bin/pups --stdin
I, [2025-06-11T09:07:40.418731 #1]  INFO -- : Reading from stdin
I, [2025-06-11T09:07:40.428644 #1]  INFO -- : > DEBIAN_FRONTEND=noninteractive apt-get purge -y postgresql-15 postgresql-client-15 postgresql-contrib-15 postgresql-15-pgvector
E: Problem executing scripts DPkg::Post-Invoke 'rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true'
E: Sub-process returned an error code
I, [2025-06-11T09:07:41.892381 #1]  INFO -- : Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
  libcommon-sense-perl libio-pty-perl libipc-run-perl libjson-perl
  libjson-xs-perl libllvm19 libtypes-serialiser-perl libz3-4
  postgresql-client-common postgresql-common postgresql-common-dev ssl-cert
Use 'apt autoremove' to remove them.
The following packages will be REMOVED:
  postgresql-15* postgresql-15-pgvector* postgresql-client-15*
0 upgraded, 0 newly installed, 3 to remove and 0 not upgraded.
After this operation, 60.9 MB disk space will be freed.
(Reading database ... 33951 files and directories currently installed.)
Removing postgresql-15-pgvector (0.8.0-1.pgdg120+1) ...
Removing postgresql-15 (15.12-1.pgdg120+1) ...
invoke-rc.d: could not determine current runlevel
update-alternatives: warning: forcing reinstallation of alternative /usr/share/postgresql/15/man/man1/psql.1.gz because link group psql.1.gz is broken
Removing postgresql-client-15 (15.12-1.pgdg120+1) ...
Processing triggers for postgresql-common (274.pgdg120+1) ...
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
Removing obsolete dictionary files:
(Reading database ... 31913 files and directories currently installed.)
Purging configuration files for postgresql-15 (15.12-1.pgdg120+1) ...
Dropping cluster main...



FAILED
--------------------
Pups::ExecError: DEBIAN_FRONTEND=noninteractive apt-get purge -y postgresql-15 postgresql-client-15 postgresql-contrib-15 postgresql-15-pgvector failed with return #<Process::Status: pid 7 exit 100>
Location of failure: /usr/local/lib/ruby/gems/3.3.0/gems/pups-1.2.1/lib/pups/exec_command.rb:132:in `spawn'
exec failed with the params "DEBIAN_FRONTEND=noninteractive apt-get purge -y postgresql-15 postgresql-client-15 postgresql-contrib-15 postgresql-15-pgvector"
bootstrap failed with exit code 100
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.
59b3543b15880ef0391688f71c9cb827ef791dbc37f1e4f896d9d7a31841c83b

So now I’m stuck and not sure how to process, any recomendation would be great, it is a huge forum and I have just a few hours to put it back online before getting fire :frowning:

Update:
I still trying things. At this point, I just want to go back to a buld that supports PG13 so I can restore my backup, but I still blocked trying that.

The commit hash before the upgrade was v3.3.0.beta6 +294, im trying to re-build that one vie YML but still saying this when building up:

root@discourse-ubuntu-s-2vcpu-4gb-fra1-01:/var/discourse# ./launcher rebuild app
WARNING: Docker version 20.10.1 deprecated, recommend upgrade to 24.0.7 or newer.
x86_64 arch detected.

WARNING: We are about to start downloading the Discourse base image
This process may take anywhere between a few minutes to an hour, depending on your network speed

Please be patient

2.0.20250226-0128: Pulling from discourse/base
af302e5c37e9: Pull complete
ed49ab5c933d: Pull complete
4d13d450f300: Pull complete
a67afd5cca97: Pull complete
d22a5b6769a6: Pull complete
83c133b53b22: Pull complete
fec147508f0c: Pull complete
21850d8b3ba1: Pull complete
02e250b6bcc4: Pull complete
71ba5ea7da00: Pull complete
ddcfbf6d2647: Pull complete
9c3e54d34536: Pull complete
29ce98d2d62d: Pull complete
4c9694455fd5: Pull complete
58a8ebcc6563: Pull complete
9c8966f1f999: Pull complete
b3dde96c4217: Pull complete
f934de9e2e8d: Pull complete
834e25f513a1: Pull complete
85d3cf606a95: Pull complete
94331a6982ee: Pull complete
14936beb0e9f: Pull complete
3ef05b122d47: Pull complete
6c2dff13a1e7: Pull complete
7682543ad38b: Pull complete
f16f8d5c5a04: Pull complete
06d7037d56ef: Pull complete
bc6c17ae1969: Pull complete
d19f089ead0e: Pull complete
Digest: sha256:6f18aa2cd22bba0deb91d69194e577d4f96130ad555ae8ec646a8792cbfe37db
Status: Downloaded newer image for discourse/base:2.0.20250226-0128
docker.io/discourse/base:2.0.20250226-0128
WARNING: containers/app.yml file is world-readable. You can secure this file by running: chmod o-rwx containers/app.yml
Ensuring launcher is up to date
Fetching origin
Launcher is up-to-date
2.0.20250226-0128: Pulling from discourse/base
Digest: sha256:6f18aa2cd22bba0deb91d69194e577d4f96130ad555ae8ec646a8792cbfe37db
Status: Image is up to date for discourse/base:2.0.20250226-0128
docker.io/discourse/base:2.0.20250226-0128
/usr/local/lib/ruby/gems/3.3.0/gems/pups-1.2.1/lib/pups.rb
/usr/local/bin/pups --stdin
I, [2025-06-11T09:54:52.350929 #1]  INFO -- : Reading from stdin
I, [2025-06-11T09:54:52.357807 #1]  INFO -- : > DEBIAN_FRONTEND=noninteractive apt-get purge -y postgresql-15 postgresql-client-15 postgresql-contrib-15 postgresql-15-pgvector
E: Problem executing scripts DPkg::Post-Invoke 'rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true'
E: Sub-process returned an error code
I, [2025-06-11T09:54:53.837808 #1]  INFO -- : Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
  libcommon-sense-perl libio-pty-perl libipc-run-perl libjson-perl
  libjson-xs-perl libllvm19 libtypes-serialiser-perl libz3-4
  postgresql-client-common postgresql-common postgresql-common-dev ssl-cert
Use 'apt autoremove' to remove them.
The following packages will be REMOVED:
  postgresql-15* postgresql-15-pgvector* postgresql-client-15*
0 upgraded, 0 newly installed, 3 to remove and 0 not upgraded.
After this operation, 60.9 MB disk space will be freed.
(Reading database ... 33951 files and directories currently installed.)
Removing postgresql-15-pgvector (0.8.0-1.pgdg120+1) ...
Removing postgresql-15 (15.12-1.pgdg120+1) ...
invoke-rc.d: could not determine current runlevel
update-alternatives: warning: forcing reinstallation of alternative /usr/share/postgresql/15/man/man1/psql.1.gz because link group psql.1.gz is broken
Removing postgresql-client-15 (15.12-1.pgdg120+1) ...
Processing triggers for postgresql-common (274.pgdg120+1) ...
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
Removing obsolete dictionary files:
(Reading database ... 31913 files and directories currently installed.)
Purging configuration files for postgresql-15 (15.12-1.pgdg120+1) ...
Dropping cluster main...



FAILED
--------------------
Pups::ExecError: DEBIAN_FRONTEND=noninteractive apt-get purge -y postgresql-15 postgresql-client-15 postgresql-contrib-15 postgresql-15-pgvector failed with return #<Process::Status: pid 7 exit 100>
Location of failure: /usr/local/lib/ruby/gems/3.3.0/gems/pups-1.2.1/lib/pups/exec_command.rb:132:in `spawn'
exec failed with the params "DEBIAN_FRONTEND=noninteractive apt-get purge -y postgresql-15 postgresql-client-15 postgresql-contrib-15 postgresql-15-pgvector"
bootstrap failed with exit code 100
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.
a609309cf458b146e5650d21103bf084e9bf66843e46e627827f50b6f8c03376

If you made a backup like you said you did, the easiest thing is to spin up a new droplet, follow Move a Discourse site to another VPS with rsync (but do not copy the database files), and then restore your backup.

3 Likes

Hy

Hi Jay! thank you so much for responding. Just started doing that, the droplet is setting up right now.

Quick question: I downloaded my backup file before starting the update, this is the file: forumname-2025-06-11-060249-v20240807024301.tar.gz (4,6GB compressed) Forum version was v3.3.0.beta6 +294 with PG13.

I was planning to initialize the new droplet forcing this version in the app.yml, like:

## this is the all-in-one, standalone Discourse Docker container template
##
## After making changes to this file, you MUST rebuild
## /var/discourse/launcher rebuild app
##
## BE *VERY* CAREFUL WHEN EDITING!
## YAML FILES ARE SUPER SUPER SENSITIVE TO MISTAKES IN WHITESPACE OR ALIGNMENT!
## visit http://www.yamllint.com/ to validate this file as needed

templates:
  - "templates/postgres.template.yml"
  - "templates/redis.template.yml"
  - "templates/web.template.yml"
  - "templates/web.ratelimited.template.yml"
## Uncomment these two lines if you wish to add Lets Encrypt (https)
  - "templates/web.ssl.template.yml"
  - "templates/web.letsencrypt.ssl.template.yml"

## which TCP/IP ports should this container expose?
## If you want Discourse to share a port with another webserver like Apache or nginx,
## see https://meta.discourse.org/t/17247 for details
expose:
  - "80:80"   # http
  - "443:443" # https

params:
  db_default_text_search_config: "pg_catalog.english"

  ## Set db_shared_buffers to a max of 25% of the total memory.
  ## will be set automatically by bootstrap based on detected RAM, or you can override
  db_shared_buffers: "1024MB"

  ## can improve sorting performance, but adds memory usage per-connection
  #db_work_mem: "40MB"

  ## Which Git revision should this container use? (default: tests-passed)
  version: v3.3.0.beta6

And then, with the new forum up and running, import the backup from there.

Does this makes sense? or I am missing some important param or step?

1 Like

That’s it.

See also Restore a backup from the command line

3 Likes

Thanks again, Jay, for responding. I really appreciate your time helping me through this nightmare.

I’m trying to do it that way (forcing the version in the YML) but apparently discourse_docker is not happy (yet) with that change:

FAILED
--------------------
Pups::ExecError: grep -q 'outlets/before-server' /etc/nginx/conf.d/discourse.conf || ( >&2 echo 'The "before-server" Nginx outlet is missing. This version of discourse_docker is not compatible with the chosen Discourse version.' ; exit 1 ) failed with return #<Process::Status: pid 2788 exit 1>
Location of failure: /usr/local/lib/ruby/gems/3.3.0/gems/pups-1.2.1/lib/pups/exec_command.rb:132:in spawn'
exec failed with the params {"cmd"=>["cp $home/config/nginx.sample.conf /etc/nginx/conf.d/discourse.conf", "rm /etc/nginx/sites-enabled/default", "mkdir -p /var/nginx/cache", "grep -q 'outlets/before-server' /etc/nginx/conf.d/discourse.conf || ( >&2 echo 'The \"before-server\" Nginx outlet is missing. This version of discourse_docker is not compatible with the chosen Discourse version.' ; exit 1 )", "grep -q 'outlets/server' /etc/nginx/conf.d/discourse.conf || ( >&2 echo 'The \"server\" Nginx outlet is missing. This version of discourse_docker is not compatible with the chosen Discourse version.' ; exit 1 )", "grep -q 'outlets/discourse' /etc/nginx/conf.d/discourse.conf || ( >&2 echo 'The \"discourse\" Nginx outlet is missing. This version of discourse_docker is not compatible with the chosen Discourse version.' ; exit 1 )", "mkdir -p /etc/nginx/conf.d/outlets/before-server", "touch /etc/nginx/conf.d/outlets/before-server/20-redirect-http-to-https.conf", "touch /etc/nginx/conf.d/outlets/before-server/30-ratelimited.conf", "mkdir -p /etc/nginx/conf.d/outlets/server", "touch /etc/nginx/conf.d/outlets/server/10-http.conf", "touch /etc/nginx/conf.d/outlets/server/20-https.conf", "touch /etc/nginx/conf.d/outlets/server/30-offline-page.conf", "mkdir -p /etc/nginx/conf.d/outlets/discourse", "touch /etc/nginx/conf.d/outlets/discourse/20-https.conf", "touch /etc/nginx/conf.d/outlets/discourse/30-ratelimited.conf"]}
bootstrap failed with exit code 1
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.
d511b0ec31ac912c99c682611c35bbb446dde1488e1762aa07ccd24690c63164



2/2


Before updating I took an screenshot to the current version just in case, is there any way to force discourse_docker to a compatible older version?

I’m trying searching on google and here in meta, but nothing yet about how to do it.

Don’t do that.

Just do a normal install of the latest version of everything and restore the database.

1 Like

Understood, I will try right now.

However, quick question: If the backup is from an old version of Discourse that were using PG13, will I have problems importing the backup into the latest version of Discourse with PG15?

1 Like

Sorry @pfaffman I wrote a previous message thinking the problem was gone but then a plot twist happened.

After a long time importing, I saw how the forum was online again listing all the posts and so on.

In the terminal, the log was:

[...]

-- execute("DELETE FROM site_settings WHERE name='warn_reviving_old_topic_age'")
   -> 0.0002s
== 20250609115711 RemoveComposerTipsSiteSettings: migrated (0.0012s) ==========

Reconnecting to the database...
Reloading site settings...
Disabling outgoing emails for non-staff users...
Disabling readonly mode...
Clearing category cache...
Reloading translations...
Remapping uploads...
Restoring uploads, this may take a while...
Migrating uploads to S3 for 'default'...
Uploading files to S3...
 - Listing local files
 => 5 files
 - Listing S3 files
............................................................................................................................................................................ => 171490 files
 - Syncing files to S3
.....
Updating the URLs in the database...

I was there for a while, and I though it was just my term connection. However, few minutes later, it continued to end failing:

490 files
 - Syncing files to S3
.....
Updating the URLs in the database...
Removing old optimized images...
Flagging all posts containing lightboxes for rebake...
76198 posts were flagged for a rebake
EXCEPTION: 903 posts are not remapped to new S3 upload URL. S3 migration failed for db 'default'.
/var/www/discourse/lib/file_store/to_s3_migration.rb:132:in `raise_or_log'
/var/www/discourse/lib/file_store/to_s3_migration.rb:81:in `migration_successful?'
/var/www/discourse/lib/file_store/to_s3_migration.rb:383:in `migrate_to_s3'
/var/www/discourse/lib/file_store/to_s3_migration.rb:59:in `migrate'
/var/www/discourse/lib/file_store/s3_store.rb:354:in `copy_from'
/var/www/discourse/lib/backup_restore/uploads_restorer.rb:69:in `restore_uploads'
/var/www/discourse/lib/backup_restore/uploads_restorer.rb:49:in `restore'
/var/www/discourse/lib/backup_restore/restorer.rb:167:in `restore_uploads'
/var/www/discourse/lib/backup_restore/restorer.rb:71:in `run'
script/discourse:191:in `restore'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/thor-1.3.2/lib/thor/command.rb:28:in `run'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/thor-1.3.2/lib/thor/invocation.rb:127:in `invoke_command'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/thor-1.3.2/lib/thor.rb:538:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/thor-1.3.2/lib/thor/base.rb:584:in `start'
script/discourse:345:in `<top (required)>'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.6.4/lib/bundler/cli/exec.rb:59:in `load'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.6.4/lib/bundler/cli/exec.rb:59:in `kernel_load'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.6.4/lib/bundler/cli/exec.rb:23:in `run'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.6.4/lib/bundler/cli.rb:452:in `exec'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.6.4/lib/bundler/vendor/thor/lib/thor/command.rb:28:in `run'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.6.4/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.6.4/lib/bundler/vendor/thor/lib/thor.rb:538:in `dispatch'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.6.4/lib/bundler/cli.rb:35:in `dispatch'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.6.4/lib/bundler/vendor/thor/lib/thor/base.rb:584:in `start'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.6.4/lib/bundler/cli.rb:29:in `start'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.6.4/exe/bundle:28:in `block in <top (required)>'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.6.4/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.6.4/exe/bundle:20:in `<top (required)>'
/usr/local/bin/bundle:25:in `load'
/usr/local/bin/bundle:25:in `<main>'
Trying to rollback...
Rolling back...
Cleaning stuff up...
Dropping functions from the discourse_functions schema...
Removing tmp '/var/www/discourse/tmp/restores/default/2025-06-11-121115' directory...
Unpausing sidekiq...
Marking restore as finished...
Notifying 'system' of the end of the restore...
Finished!
[FAILED]
Restore done.

Suddenly, the forum that was “restored” online, all the post vanished, leaving just a couple of topics alive.

Since the log says “76198 posts were flagged for a rebake”, I thought it was just that now they have to be rebaked, so I went to Sidekiq but there is nothing pending to process :frowning:

Update:

Logged into the rails console, and apparently the are only 10 posts, not all the ones to be imported :thinking:

/var/www/discourse# rails c
Loading production environment (Rails 7.2.2.1)
discourse(prod)> Post.count
=> 10
discourse(prod)> Post.where(“baked_version IS NULL”).count
=> 0

1 Like

Sigh. Are you using the same hostname? (Maybe so?)

That’s your problem now.

Did you configure S3 in the database or in the yml file? I guess it’s the former. The easiest thing would be to restore a database-only, but that’s hard to explain (extract the sql file and then name it with the same stuff as the tar.gz file)

You can try this

discourse restore --pause <backup_filename>

That will pause before it tries to restore the S3 stuff. Then, if you knew what to do, you could fix the issue with the errant uploads.

See Issues with restoring backup to discourse - #16 by gerhard

Another thing would be to set DISCOURSE_USE_S3 to false in your app.yml. You might need to set the other S3 stuff too like the ones that say to use S3 uploads and/or the S3 buckets.

1 Like

Yes, but now that you mention it.

  • The forum is 10 years old now.
  • I set S3 back in the day for a while, but then remove it, so those posts with S3 images are quite old.

Is there a way to identify them and ignore/remove them, so the import can work with no friction?

Another thing would be to set DISCOURSE_USE_S3 to false in your app.yml . You might need to set the other S3 stuff too like the ones that say to use S3 uploads and/or the S3 buckets.

I will try this too, because at the end, we don’t care about those S3 files or posts, it is just I don’t know yet how to handle it to ignore them

If you really don’t care about them you could do the pause thing and when it pauses, open another terminal, go into rails and do something like

bad=Upload.where("url like '%the-s3-url%'")
bad.destroy_all

BTW, if you’re still stuck and desperate and have a budget, you can contact me via email or PM or ask in Marketplace.

You should have told me from the beginning! I would have gladly paid in the first place! hehe

It is up and running now.

After trying tons of things, I ended up doing some counterintuitive but that it is working now:

  • Setup S3 in the new instance properly (via admin panel)
  • Ran the import with the –pause thing but in the second step, just stopped the process. I was expecting the forum to no work or having issues with the images, but it is running and old images are loading with no issue.

I still testing things, maybe I find something broken later, but by now, after 12h of troubleshooting, I’ll call it a win for now.

1 Like

That’s not proper. See Configure an S3 compatible object storage provider for uploads. You want all those settings in your yml files.