PostgreSQL 18 update for self-hosters

NOTE: Not sure the reason, but the PostgreSQL 18 upgrade disables the native data checksums. PostgreSQL data checksums a default feature and it seems very unusual to disable them.

PR to not disable data checkums (enabled by default): Do not disable PostgreSQL 18 data checksums - Pull Request #1105 - discourse/discourse_docker - GitHub

And it’s better than having to have 20GB free for the 1 hour you need it in one year lol

Perhaps this should be the recommended approach in order to save trees and water. :sweat_smile:

Just to confirm, I’m not using PostgreSQL provided by Discourse and Discourse itself does not require PG18 yet, right? So I am not (yet) required to upgrade to PG18.

Good point. But the other one is that LTS releases come every couple of years, so it’s not a bad idea to go ahead and do that at the same time.

You definitely have a while. They pushed for . . . uh, some release pretty quickly after the update because of some feature that was required, but you can likely wait as long as a year. I watch the discourse_docker repo. At some point they’ll start talking about removing support for PG15; it’s not especially noisy and is an easy way to keep up with updates to interal stuff.

2 Likes

Worked fine on my Pi 5 install, two rebuilds and done.

7 Likes

Do we have any benchmarks btw?

This might encourage others to follow in our intrepid footsteps sooner.

My free web search AI is telling me:

For a typical Rails app, moving from PostgreSQL 15 → 18 can yield ~10–25% faster query performance with no code changes, and up to 40% in specific query patterns if you take advantage of new indexing and planner features.

If true, that’s quite an upgrade! :tada:

7 Likes

Just to confirm that everything went flawlessly in our self-hosted instance. Thanks for this update and keep us notified.

5 Likes

True, but we’re aiming for a simple transparent upgrade mechanism. There are merits both ways.

I’d say do whatever you’re comfortable with and don’t be afraid to customise the templates in discourse_docker as your needs require. Obviously the key is to test first and have a rollback plan.

On our hosted platform we intend to do more testing and benchmarking before switching these on, and so I disabled them in discourse_docker as well to align the config. It’s not strictly necessary to disable them though (we only use the web container portion of discourse_docker internally) and I wouldn’t be against enabling them.

One potential gotcha is that pg_upgrade will not work if the old and new data directories have different checksum settings. The process would have to be to shut down the PG15 server, run pg_upgrade to convert it to PG18 without checksums, run pg_checksums to enable checksums and then start up PG18. This isn’t a problem when doing a dump and restore (as with this upgrade) but is something to watch out for.

Note that data checksums have been available since Postgres 9.3 but until now have been disabled by default. Postgres 19 will also include the ability to enable/disable them online.

Not at this time, no. Most of the Discourse functionality uses the Rails PostgreSQL adapter to talk to the DB, but backup/restore uses pg_dump and psql in the web container. Right now we’re installing both PG15 and PG18 clients to enable backup/restore using both of these versions, but at some point in the future we’ll remove PG15.

The driving factor was switching to the new builtin locale provider. We are working towards an OS upgrade on our hosted platform and want to break the coupling with glibc.

3 Likes

I manage my postgres independently from the container. What’s the recommended git hash I should switch to pg18 for?

1 Like

e7f1201 added the PG18 client to the web image for backup compatibility. If you are not using any of the Postgres server components of discourse_docker then that is the earliest revision you should use for PG18.

1 Like

I was talking about an upgrade or two ago.

Agreed. My point was just that it’s no less supported to be able to restore an older postgres backup to a newer one. The upgrade in place mechanism works amazingly well, for the vast majority of people, but when something goes wrong, it’s hard to know what to do (largely because it happens so rarely).

2 Likes

If it helps anyone, for the moments where the SSH console remains static and those moments when the cold panic sweats start to kick in… :sweat_smile:

I ran this in another terminal to keep an eye on progress:

watch -n 10 'df -h /; echo; du -sh /var/discourse/shared/standalone/postgres_data* 2>/dev/null'

Which updates you every 10s and keeps you sane :sweat_smile:

My 35GB migration took around 10 minutes.

8 Likes

The update went through without any issues using the standard installation. Of course, I made a backup and downloaded it beforehand in case anything went wrong :smiley:

3 Likes

Nice. I’d add a free -h to that - memory starvation is a common enough problem.

The only thing about a ‘watch’ or indeed a ‘top’ is that it keeps refreshing so you might miss something. If you run out of something and the update fails, a few seconds later you’ve lost the record. So I tend to run something more like a while - maybe like this:

while true; do date; echo; free -h; echo; df -h /; echo; sh -c 'du -sh /var/discourse/shared/standalone/postgres_data* 2>/dev/null'; sleep 10; echo; done
4 Likes

I can report success . . . on one half of my multisite setup. The default site migrated as expected, but the secondary site looked like a fresh install. Fortunately it’s not hard to restore a backup. I have another server that I use for clients and I’m thinking of spinning up a new Droplet and restoring the sites from backups to do this update. This is one of the risks of using a non-standard install, I suppose.

1 Like

Was your data container a standard data container? I’ve been wondering if it would move just the one database or all of them in the cluster. It sounds like you’ve answered my question!

What I’ve been doing is hand moving each database from the old cluster to the new one, then hand editing discourse.conf to point to the new database and then doing a rebuild to point the whole multisite container to the new cluster (on a different machine or port).

1 Like

Yep. I can’t say I haven’t screwed up anything in my setup, of course. :wink:

I had been planning to rsync my postgres data to run qn upgrade there to see for sure if the process moved just the discourse database or the whole cluster, but it sounds like you’ve answered my question and that will be clear if I look at the code.

2 Likes

Got two questions here:

  1. We’ve mounted an additional storage block on the test server. But the postgresql upgrade fails due to the space check is only checking the main disk. Is there a way to bypass the space check?
  2. For our production site, we use Google Cloud SQL. Is there anything we need to know before the upgrade through google cloud?