# Infinite loop during postgres upgrade

**URL:** https://meta.discourse.org/t/infinite-loop-during-postgres-upgrade/91767
**Category:** Self-hosting
**Created:** [July 7, 2018, 10:39am UTC](https://meta.discourse.org/t/infinite-loop-during-postgres-upgrade/91767 "2018-07-07T10:39:33Z")
**Posts on this page:** 11
**Page:** 1

<div class="post-metadata">

### Author: ![CamilleRoux](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/camilleroux/32/119575_2.png) [@CamilleRoux](https://meta.discourse.org/u/CamilleRoux)
#### Post date: [July 7, 2018, 10:39am UTC](https://meta.discourse.org/t/infinite-loop-during-postgres-upgrade/91767/1 "2018-07-07T10:39:33Z")

</div>

Hi! Each time I launch `/launcher rebuild app` I get the following message (even if I launch it again) :

```
UPGRADE OF POSTGRES COMPLETE
Old 9.5 database is stored at /shared/postgres_data_old
To complete the upgrade, rebuild again using:
./launcher rebuild app

```

What information do you need to help me?

PS: The first time I ran it, the command stoped because I needed more space

---

<div class="post-metadata">

### Author: ![CamilleRoux](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/camilleroux/32/119575_2.png) [@CamilleRoux](https://meta.discourse.org/u/CamilleRoux)
#### Post date: [July 7, 2018, 10:49am UTC](https://meta.discourse.org/t/infinite-loop-during-postgres-upgrade/91767/2 "2018-07-07T10:49:10Z")

</div>

Temporarily, I used the templates/postgres.9.5.template.yml template to avoid downtime. The app is running again, but I’d like to finish the update.

When I do a cleanup I get the following message : `Old PostgreSQL backup data cluster detected taking up 1.2G detected. Would you like to remove it? (Y/n):`

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [July 8, 2018, 7:55am UTC](https://meta.discourse.org/t/infinite-loop-during-postgres-upgrade/91767/3 "2018-07-08T07:55:09Z")

</div>

Sounds like a question for @tgxworld, but I am guessing that it failed due to lack of space but somehow didn’t catch that the upgrade had failed.

How much space do you have? And how big is your database?

---

<div class="post-metadata">

### Author: ![tgxworld](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/tgxworld/32/106117_2.png) [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [July 9, 2018, 1:35am UTC](https://meta.discourse.org/t/infinite-loop-during-postgres-upgrade/91767/4 "2018-07-09T01:35:25Z")

</div>

Can you provide me with the full log generated during the rebuild? Thank you!

---

<div class="post-metadata">

### Author: ![vulkanino](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/vulkanino/32/119654_2.png) [@vulkanino](https://meta.discourse.org/u/vulkanino)
#### Post date: [January 12, 2021, 1:21pm UTC](https://meta.discourse.org/t/infinite-loop-during-postgres-upgrade/91767/5 "2021-01-12T13:21:11Z")

</div>

I’ve been having the same problem for many months now and I can’t upgrade my installation.

It loops and never succeeds. One problem during the `rebuild app` is this:  
`mv: cannot move '/shared/postgres_data' to '/shared/postgres_data_old': Device or resource busy`

Entering in the Docker the postgres version I have is:

```
/usr/lib/postgresql/10/bin/postgres --version
postgres (PostgreSQL) 10.14 (Debian 10.14-1.pgdg100+1)

```

These are the errors I get:

```
FAILED

--------------------

Pups::ExecError: /root/upgrade_postgres failed with return #<Process::Status: pid 47 exit 1>

Location of failure: /pups/lib/pups/exec_command.rb:112:in `spawn'

exec failed with the params "/root/upgrade_postgres"

0c74c9de4d4315b63c0ef9055631f38c0cf4b3dd0be6500fd83ca0a5b13e0d9d

**FAILED TO BOOTSTRAP** please scroll up and look for earlier error messages, there may be more than one.

./discourse-doctor may help diagnose the problem.

```

It looks like there’s a problem in the `/pups/lib/pups/exec_command.rb` script executing `spawn`.

I can start the app though  
./laucher start app

These are the files I have under `/shared`

```
total 48
drwxr-xr-x 12 root root 4096 Jan 12 13:12 .
drwxr-xr-x 57 root root 4096 Dec 15 09:09 ..
drwxr-xr-x 3 discourse www-data 4096 Aug 20 2019 backups
drwxr-xr-x 4 root root 4096 Aug 19 2019 log
drwxr-xr-x 2 postgres postgres 4096 Aug 19 2019 postgres_backup
drwx------ 20 postgres postgres 4096 Jan 12 13:14 postgres_data
drwx------ 19 postgres postgres 4096 Jan 12 13:12 postgres_data_new
drwxrwxr-x 5 postgres postgres 4096 Jan 12 13:14 postgres_run
drwxr-xr-x 2 redis redis 4096 Jan 12 13:07 redis_data
drwxr-xr-x 4 root root 4096 Aug 19 2019 state
drwxr-xr-x 4 discourse www-data 4096 Jan 12 13:14 tmp
drwxr-xr-x 4 discourse www-data 4096 Sep 8 2019 uploads

```

What can I do?

---

<div class="post-metadata">

### Author: ![Falco](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/falco/32/179432_2.png) [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [January 12, 2021, 1:42pm UTC](https://meta.discourse.org/t/infinite-loop-during-postgres-upgrade/91767/6 "2021-01-12T13:42:37Z")

</div>

Is your shared directory some sort of network mount?

---

<div class="post-metadata">

### Author: ![vulkanino](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/vulkanino/32/119654_2.png) [@vulkanino](https://meta.discourse.org/u/vulkanino)
#### Post date: [January 12, 2021, 1:52pm UTC](https://meta.discourse.org/t/infinite-loop-during-postgres-upgrade/91767/7 "2021-01-12T13:52:24Z")

</div>

No, but I’ve managed to do it finally!

The two `mv` commands in templates/postgres.template.yml are not checked for goodness. The script always says `UPGRADE OF POSTGRES COMPLETE` which is not true when the move is in error.

I removed the commands, then moved the directories by hand after entering in the docker, and the next `rebuild app` is currently working as before! I’m so happy 😃  
Thank you.

---

<div class="post-metadata">

### Author: ![lavamind](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/lavamind/32/237909_2.png) [@lavamind](https://meta.discourse.org/u/lavamind)
#### Post date: [July 30, 2025, 3:16pm UTC](https://meta.discourse.org/t/infinite-loop-during-postgres-upgrade/91767/8 "2025-07-30T15:16:13Z")

</div>

I hit this problem during the latest upgrade from PotsgreSQL 13 to 15.

In our deployment, our`/shared/postgres_data` directory is mounted on a faster NVMe storage device, so moving it failed with a `Device or resource busy` error message.

We fixed this by patching the postgres template as such:

```plaintext
diff --git a/templates/postgres.template.yml b/templates/postgres.template.yml
index c24bfe6..03813c4 100644
--- a/templates/postgres.template.yml
+++ b/templates/postgres.template.yml
@@ -139,8 +139,10 @@ run:
            exit 1
          fi
 
- mv /shared/postgres_data /shared/postgres_data_old
- mv /shared/postgres_data_new /shared/postgres_data
+ mkdir /shared/postgres_data_old
+ mv /shared/postgres_data/* /shared/postgres_data_old
+ mv /shared/postgres_data_new/* /shared/postgres_data
+ rmdir /shared/postgres_data_new

```

Instead of attempting to manipulate the directory itself, just move the contents around.

Please consider integrating this change to improve the reliability of database upgrades.

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [July 30, 2025, 10:35pm UTC](https://meta.discourse.org/t/infinite-loop-during-postgres-upgrade/91767/9 "2025-07-30T22:35:34Z")

</div>

I think this might be related: [FIX: improve postgres upgrade reliability by jcharaoui · Pull Request #989 · discourse/discourse\_docker · GitHub](https://github.com/discourse/discourse_docker/pull/989)

---

<div class="post-metadata">

### Author: ![tgxworld](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/tgxworld/32/106117_2.png) [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [August 11, 2025, 6:55am UTC](https://meta.discourse.org/t/infinite-loop-during-postgres-upgrade/91767/10 "2025-08-11T06:55:22Z")

</div>

> [@pfaffman](#):
>
> FIX: improve postgres upgrade reliability by jcharaoui · Pull Request #989 · discourse/discourse\_docker · GitHub

The PR has been merged 👍

---

<div class="post-metadata">

### Author: ![tgxworld](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/tgxworld/32/106117_2.png) [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [August 18, 2025, 12:00am UTC](https://meta.discourse.org/t/infinite-loop-during-postgres-upgrade/91767/11 "2025-08-18T00:00:23Z")

</div>

This topic was automatically closed after 6 days. New replies are no longer allowed.
