Upgrade fail unsupported Docker Version

Ubuntu 16
Clicked the web upgrade like we have many times. failed now the app won’t rebuild unsupported docker version.

Any ideas? I could try upgrading the OS etc but that’s a lot of time I didn’t have today.

Anyway to go back?

I tried setting the version to the old hash but I immediately get the unsupported docker version when I try to rebuild

You can’t upgrade.

I would move to a new vm and restore the database there. It’s easier, had little down time, and if something goes wrong, you just keep using the working server.

Oh. Sorry. I didn’t read carefully enough.

No. You can’t go back.

You might be able to upgrade docker if you use docker’s install.

I got OS upgraded etc and it went through postgress 15 upgrade but now when I do rebuild app I get

2025-05-18 14:58:28.208 UTC [1455] discourse@discourse ERROR:  column "require_message" does not exist at character 72
2025-05-18 14:58:28.208 UTC [1455] discourse@discourse STATEMENT:  SELECT id, name, name_key, description, notify_type, auto_action_type, require_message, applies_to, position, enabled, score_type FROM "flags" ORDER BY "flags"."position" ASC
** PLUGIN FAILURE **

You are unable to start Discourse due to this error during plugin
initialization:

PG::UndefinedColumn: ERROR:  column "require_message" does not exist
LINE 1: ..._key, description, notify_type, auto_action_type, require_me...
                                                             ^

After I ran rebuild (post postgress upgrade)

Performing Upgrade
------------------
Analyzing all rows in the new cluster                       ok
Freezing all rows in the new cluster                        ok
Deleting files from new pg_xact                             ok
Copying old pg_xact to new server                           ok
Setting oldest XID for new cluster                          ok
Setting next transaction ID and epoch for new cluster       ok
Deleting files from new pg_multixact/offsets                ok
Copying old pg_multixact/offsets to new server              ok
Deleting files from new pg_multixact/members                ok
Copying old pg_multixact/members to new server              ok
Setting next multixact ID and offset for new cluster        ok
Resetting WAL archives                                      ok
Setting frozenxid and minmxid counters in new cluster       ok
Restoring global objects in the new cluster                 ok
Restoring database schemas in the new cluster               ok
Copying user relation files                                 ok
Setting next OID for new cluster                            ok
Sync data directory to disk                                 ok
Creating script to delete old cluster                       ok
Checking for extension updates                              notice

Your installation contains extensions that should be updated
with the ALTER EXTENSION command.  The file
    update_extensions.sql
when executed by psql by the database superuser will update
these extensions.


Upgrade Complete
----------------
Optimizer statistics are not transferred by pg_upgrade.
Once you start the new server, consider running:
    /usr/lib/postgresql/15/bin/vacuumdb --all --analyze-in-stages

Running this script will delete the old cluster's data files:
    ./delete_old_cluster.sh
-------------------------------------------------------------------------------------
UPGRADE OF POSTGRES COMPLETE

Old 13 database is stored at /shared/postgres_data_old

To complete the upgrade, rebuild again using:

./launcher rebuild app

Anything I can do? I’m not sure how it is missing a column.

Yeah. There are some topics about similar problems. A migration for reverted, I think, so you’re left in a limbo state.

Maybe add the column by hand.

I’m not sure how to do that, more specifically to which table etc. Is there some information on that that I can find. I did find that post you mention but there were no specifics.

THank you!

I can’t even get the DB mounted enough to add the column. I’m going to try disabling all plugins and see if I can at least get the countainer started.

Ok here’s how we got it fixed (It is not for the faint of heart)

  1. Upgrade Ubuntu to a version that supports Docker 20+
    We were in Ubuntu 16 I had to get to at least Ubuntu 20 so we did this upgrade twice
RELEASE_UPGRADER_ALLOW_THIRD_PARTY=1 do-release-upgrade
  1. Once we got to Ubuntu 20 we had to Update Docker which Ubuntu refused to do on its own.
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
  1. That got us to docker 28
  2. Then run
./launcher rebuild app

5.This will upgrade your postgress from 13 to 15 see:

6. Once the Db upgraded successfully ran

./launcher rebuild app
  1. This yielded an error about missing column
2025-05-18 14:58:28.208 UTC [1455] discourse@discourse ERROR:  column "require_message" does not exist at character 72
2025-05-18 14:58:28.208 UTC [1455] discourse@discourse STATEMENT:  SELECT id, name, name_key, description, notify_type, auto_action_type, require_message, applies_to, position, enabled, score_type FROM "flags" ORDER BY "flags"."position" ASC
** PLUGIN FAILURE **

You are unable to start Discourse due to this error during plugin
initialization:

PG::UndefinedColumn: ERROR:  column "require_message" does not exist
LINE 1: ..._key, description, notify_type, auto_action_type, require_me...
                                                             ^
  1. At this point there is no way to get the container started, so I went to app.yml and disabled all plugins then ran rebuild again
  2. The app finally rebuilt, then I was able to enter the rails console to add the above column manually.
./launcher enter app
rails db
ActiveRecord::Base.connection.execute("ALTER TABLE flags ADD COLUMN require_message BOOLEAN DEFAULT FALSE;")
exit
exit 
  1. Once the column was present went back i to app.yml and enabled all plugins and did a rebuild again
  2. Rebuild succeeded … and we are up!

Thank you @pfaffman for your prompt response on the weekend no less. We are going to make a new updated droplet and move over. This was suposed to be a quick inplace upgrade from the web interface. But I guess the script doesn’t test docker compatibility. When it upgraded discourse docker then it threw incompatible docker error.

This was fully our fault for saying in such an old version of Ubunutu 16, one of the good and bad things about a stable system is that it tends to linger.

1 Like

Glad you got it working! That’s quite impressive!

You don’t get to do quick updates when your OS is 4 years beyond EOL. :wink:

1 Like

Yuup, Lesson Learned thanks again!