It looks like 2+2 may not be enough any more… I’m managing a fairly unassuming (no big/fancy plugins, etc) Discourse instance that, as of today, is failing to bootstrap because ember is chewing all the RAM it can find, and all the swap, and grinding the machine into unresponsiveness. Adding another 2GB of swap allowed the bootstrap to complete, with a peak swap usage of around 2.5GB.

Yikes, this is on @david’s list to investigate.

@david has been investigating, we do confirm that as it stands 2gb is enough for docker rebuilds, but not enough for the web upgrader to work. One idea I have tossed around is just shutting down all ruby processes during the web upgrader to save an extra 300-500mb which would leave enough for asset precompilation. A long term approach we are likely going to need to go down for self hosters is shipping bootstrapped containers which is a pandoras box cause how would a web upgrader be able to pull that one off. We don’t want to mount docker sockets … it sure is a pickle.

[image] sam: it stands 2gb is enough for docker rebuilds Well, it wasn’t for me. Is this compared between basic pure install against real world situations?

Indeed, it’s not perfectly consistent. Even with everything else shut down, it can still fail. Unfortunately we’re fighting a losing battle against modern JS build tooling here. It’s all designed to run on 8GB+ of Ram on modern developer machines, not 1GB VPSs :cry: We do have some solutions in mind. For example: providing pre-built assets which can be automatically downloaded. The big challenge we have is plugins, because they vary on everyone’s site, and right now they’re integrated tightly into the core build. But for now, doing a full CLI rebuild should have a higher success rate than a web-ui update.

[image] sam: we do confirm that as it stands 2gb is enough for docker rebuilds, Like Jagster, 2gb RAM + 2gb swap is not, in fact, enough for my CLI-driven docker rebuild. Checking further, the only plugins on this install are docker_manager and discourse-prometheus – neither of which would, I expect, put unexpected load onto ember. If the minimum specs have to change, that would suck, but it would be a lot better than the current situation, where machines unexpectedly grind to death on every upgrade.

[image] david: Indeed, it’s not perfectly consistent. Even with everything else shut down, it can still fail. If that’s the case, I think it would still be better to increase the recommended specs a bit. Personally, I don’t really mind adding 2 (or even 4) more GB of swap if it makes rebuilds more reliable - at least as long as daily operations are still fine with 2-4 GB of RAM (for small to medium-sized communities).

[image]Create a swapfile for your Linux server [image] sam: we do confirm that as it stands 2gb is enough for docker rebuilds, Like Jagster, 2gb RAM + 2gb swap is not, in fact, enough for my CLI-driven docker rebuild. Checking further, the only plugins on this install are docker_manager Indeed initial install failed in my recent install in 4c 4g instance. Ed suggested creating a swap file. Found the topic to create a swap and created a 4g swap. Now everything working as expected in web or cli update/upgrade Imho we may just need to accept discourse requires more ram than it used to.

We just landed this commit which should hopefully improve the situation. Please let us know how you get on! (it’ll hit tests-passed in the next ~30 mins) When testing with a memory-constrained docker container locally, I can now get a successful build with -m 1600m. Before this change, the minimum I could achieve was -m 3000m.

I did a test rebuild (fresh install), which went through without issue. Now that machine has 4GiB of RAM (Hetzner CAX11) and a swap file of the same size, so it’s certainly less constraint than the 2+2 GB setup mentioned above. However, swap use was minimal during the whole build and the maximum RAM usage I saw was ~3.1GB. And most of the time, it stayed around ~2GB, so it doesn’t look too bad (build time was more or less unchanged, i.e. about 8 minutes). [Maximum RAM usage during build (htop)]

High rebuild memory requirements: April 2025 edition

Product Support Self-hosting

Jagster (Jakke Flemming) April 1, 2025, 4:44am 4

Well, it wasn’t for me.

Is this compared between basic pure install against real world situations?

Topic		Replies	Views
Precompiling assets takes 20 minutes Self-hosting server-resources	18	1788	January 31, 2024
Failed to upgrade discourse instance to Feb 15 2022 Self-hosting server-resources	29	3377	February 16, 2022
Suddenly boot strapping is failing Self-hosting two-container	25	1540	December 30, 2022
Upgrade from 3.2.0.beta3-dev to 3.2.0.beta3 failed due to out of memory Self-hosting server-resources	18	1866	February 6, 2024
Failed to bootstrap due to out of memory killer Bug	35	3130	April 29, 2021

High rebuild memory requirements: April 2025 edition

Related topics