PostgreSQL 18 update for self-hosters

The Literate Computing dashboard has successfully upgraded two standalone and one two-container sites without incident. Mostly all that was required was changing the target Postgres version in a variable, so as advertised, the process is the same as the last several.

It’s actually pretty supported, it’s much safer than upgrading a database major version. If something goes wrong, you just don’t switch over to the new server. If you are close to needing an OS upgrade and/or want minimal downtime it’s a good way to go. You have read-only while you build the new server and then switch to it. The easy way you have just downtime for the final rebuild, or zero downtime if you copy the certs from the old server).

If everything goes wrong with a DB upgrade, building a new server and restoring the backup is the easy solution.

Do make sure to make a backup before starting. I do a database-only backup since you’re not going to lose your uploads.

5 Likes

NOTE: Not sure the reason, but the PostgreSQL 18 upgrade disables the native data checksums. PostgreSQL data checksums a default feature and it seems very unusual to disable them.

PR to not disable data checkums (enabled by default): Do not disable PostgreSQL 18 data checksums - Pull Request #1105 - discourse/discourse_docker - GitHub

And it’s better than having to have 20GB free for the 1 hour you need it in one year lol

Perhaps this should be the recommended approach in order to save trees and water. :sweat_smile:

Just to confirm, I’m not using PostgreSQL provided by Discourse and Discourse itself does not require PG18 yet, right? So I am not (yet) required to upgrade to PG18.

Good point. But the other one is that LTS releases come every couple of years, so it’s not a bad idea to go ahead and do that at the same time.

You definitely have a while. They pushed for . . . uh, some release pretty quickly after the update because of some feature that was required, but you can likely wait as long as a year. I watch the discourse_docker repo. At some point they’ll start talking about removing support for PG15; it’s not especially noisy and is an easy way to keep up with updates to interal stuff.

2 Likes

Worked fine on my Pi 5 install, two rebuilds and done.

6 Likes

Do we have any benchmarks btw?

This might encourage others to follow in our intrepid footsteps sooner.

My free web search AI is telling me:

For a typical Rails app, moving from PostgreSQL 15 → 18 can yield ~10–25% faster query performance with no code changes, and up to 40% in specific query patterns if you take advantage of new indexing and planner features.

If true, that’s quite an upgrade! :tada:

7 Likes

Just to confirm that everything went flawlessly in our self-hosted instance. Thanks for this update and keep us notified.

4 Likes

True, but we’re aiming for a simple transparent upgrade mechanism. There are merits both ways.

I’d say do whatever you’re comfortable with and don’t be afraid to customise the templates in discourse_docker as your needs require. Obviously the key is to test first and have a rollback plan.

On our hosted platform we intend to do more testing and benchmarking before switching these on, and so I disabled them in discourse_docker as well to align the config. It’s not strictly necessary to disable them though (we only use the web container portion of discourse_docker internally) and I wouldn’t be against enabling them.

One potential gotcha is that pg_upgrade will not work if the old and new data directories have different checksum settings. The process would have to be to shut down the PG15 server, run pg_upgrade to convert it to PG18 without checksums, run pg_checksums to enable checksums and then start up PG18. This isn’t a problem when doing a dump and restore (as with this upgrade) but is something to watch out for.

Note that data checksums have been available since Postgres 9.3 but until now have been disabled by default. Postgres 19 will also include the ability to enable/disable them online.

Not at this time, no. Most of the Discourse functionality uses the Rails PostgreSQL adapter to talk to the DB, but backup/restore uses pg_dump and psql in the web container. Right now we’re installing both PG15 and PG18 clients to enable backup/restore using both of these versions, but at some point in the future we’ll remove PG15.

The driving factor was switching to the new builtin locale provider. We are working towards an OS upgrade on our hosted platform and want to break the coupling with glibc.

1 Like

I manage my postgres independently from the container. What’s the recommended git hash I should switch to pg18 for?

1 Like

e7f1201 added the PG18 client to the web image for backup compatibility. If you are not using any of the Postgres server components of discourse_docker then that is the earliest revision you should use for PG18.

1 Like

I was talking about an upgrade or two ago.

Agreed. My point was just that it’s no less supported to be able to restore an older postgres backup to a newer one. The upgrade in place mechanism works amazingly well, for the vast majority of people, but when something goes wrong, it’s hard to know what to do (largely because it happens so rarely).

1 Like

If it helps anyone, for the moments where the SSH console remains static and those moments of cold panic sweat start to kick in… :sweat_smile:

I ran this in another terminal to keep an eye on progress:

watch -n 10 'df -h /; echo; du -sh /var/discourse/shared/standalone/postgres_data* 2>/dev/null'

Which updates you every 10s and keeps you sane :sweat_smile:

My 35GB migration took around 10 minutes.

1 Like

The update went through without any issues using the standard installation. Of course, I made a backup and downloaded it beforehand in case anything went wrong :smiley:

2 Likes