Hi! Each time I launch /launcher rebuild app I get the following message (even if I launch it again) : UPGRADE OF POSTGRES COMPLETE Old 9.5 database is stored at /shared/postgres_data_old To complete the upgrade, rebuild again using: ./launcher rebuild app What information do you need to help me? PS: The first time I ran it, the command stoped because I needed more space

Temporarily, I used the templates/postgres.9.5.template.yml template to avoid downtime. The app is running again, but I’d like to finish the update. When I do a cleanup I get the following message : Old PostgreSQL backup data cluster detected taking up 1.2G detected. Would you like to remove it? (Y/n):

Sounds like a question for @tgxworld, but I am guessing that it failed due to lack of space but somehow didn’t catch that the upgrade had failed. How much space do you have? And how big is your database?

I’ve been having the same problem for many months now and I can’t upgrade my installation. It loops and never succeeds. One problem during the rebuild app is this: mv: cannot move '/shared/postgres_data' to '/shared/postgres_data_old': Device or resource busy Entering in the Docker the postgres version I have is: /usr/lib/postgresql/10/bin/postgres --version postgres (PostgreSQL) 10.14 (Debian 10.14-1.pgdg100+1) These are the errors I get: FAILED -------------------- Pups::ExecError: /root/upgrade_postgres failed with return # Location of failure: /pups/lib/pups/exec_command.rb:112:in `spawn' exec failed with the params "/root/upgrade_postgres" 0c74c9de4d4315b63c0ef9055631f38c0cf4b3dd0be6500fd83ca0a5b13e0d9d ** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one. ./discourse-doctor may help diagnose the problem. It looks like there’s a problem in the /pups/lib/pups/exec_command.rb script executing spawn. I can start the app though ./laucher start app These are the files I have under /shared total 48 drwxr-xr-x 12 root root 4096 Jan 12 13:12 . drwxr-xr-x 57 root root 4096 Dec 15 09:09 .. drwxr-xr-x 3 discourse www-data 4096 Aug 20 2019 backups drwxr-xr-x 4 root root 4096 Aug 19 2019 log drwxr-xr-x 2 postgres postgres 4096 Aug 19 2019 postgres_backup drwx------ 20 postgres postgres 4096 Jan 12 13:14 postgres_data drwx------ 19 postgres postgres 4096 Jan 12 13:12 postgres_data_new drwxrwxr-x 5 postgres postgres 4096 Jan 12 13:14 postgres_run drwxr-xr-x 2 redis redis 4096 Jan 12 13:07 redis_data drwxr-xr-x 4 root root 4096 Aug 19 2019 state drwxr-xr-x 4 discourse www-data 4096 Jan 12 13:14 tmp drwxr-xr-x 4 discourse www-data 4096 Sep 8 2019 uploads What can I do?

Is your shared directory some sort of network mount?

No, but I’ve managed to do it finally! The two mv commands in templates/postgres.template.yml are not checked for goodness. The script always says UPGRADE OF POSTGRES COMPLETE which is not true when the move is in error. I removed the commands, then moved the directories by hand after entering in the docker, and the next rebuild app is currently working as before! I’m so happy :smiley: Thank you.

I hit this problem during the latest upgrade from PotsgreSQL 13 to 15. In our deployment, our/shared/postgres_data directory is mounted on a faster NVMe storage device, so moving it failed with a Device or resource busy error message. We fixed this by patching the postgres template as such: diff --git a/templates/postgres.template.yml b/templates/postgres.template.yml index c24bfe6..03813c4 100644 --- a/templates/postgres.template.yml +++ b/templates/postgres.template.yml @@ -139,8 +139,10 @@ run: exit 1 fi - mv /shared/postgres_data /shared/postgres_data_old - mv /shared/postgres_data_new /shared/postgres_data + mkdir /shared/postgres_data_old + mv /shared/postgres_data/* /shared/postgres_data_old + mv /shared/postgres_data_new/* /shared/postgres_data + rmdir /shared/postgres_data_new Instead of attempting to manipulate the directory itself, just move the contents around. Please consider integrating this change to improve the reliability of database upgrades.

I think this might be related: FIX: improve postgres upgrade reliability by jcharaoui · Pull Request #989 · discourse/discourse_docker · GitHub

[image] pfaffman: FIX: improve postgres upgrade reliability by jcharaoui · Pull Request #989 · discourse/discourse_docker · GitHub The PR has been merged :+1:

Infinite loop during postgres upgrade

Product Support Self-hosting

tgxworld (Alan Tan) July 9, 2018, 1:35am 4

Can you provide me with the full log generated during the rebuild? Thank you!

Topic		Replies	Views
Issue Rebuilding App Failing on Postgres Upgrade Self-hosting	13	2633	October 18, 2020
Postgres upgrade success loop due to prior postgres 8 to 10 migration Self-hosting	3	1317	May 26, 2020
Trouble with update 2.5.0.beta4 to 2.6.0.beta2 (probably Postgres) Self-hosting	4	697	September 1, 2020
Stuck in an update loop after PostgreSQL 13 update Self-hosting	14	2519	October 19, 2021
Cannot rebuild app because UPGRADE OF POSTGRES FAILED Self-hosting	7	392	February 12, 2025

Infinite loop during postgres upgrade

Related topics