Configure ruby's memory limit?

Contribute Feature

AI-generated summary

The discussion revolves around configuring Ruby’s memory limit in Discourse. Ed_S initially questioned whether the memory limit could be made a site configuration option, citing concerns about a 1GB node experiencing memory issues. However, it was discovered that the current limit is set to 20MB, not 2G as initially thought.

Falco explained that Discourse has predictable memory usage, and the key to managing memory is the number of Unicorn workers. During upgrades, specific environment variables are set to prevent memory pressure on low-spec machines.

Ed_S suggested making the memory limit configurable, but Falco stated that the current settings have been extensively tested and should work fine for small communities on a 1GB VPS. Falco also mentioned that forking the plugin and adjusting the settings is an option if needed.

The conversation also touched on the topic of an older thread on tuning Ruby and Rails for Discourse, which is now outdated. Sam noted that the latest Ruby has multiple malloc limits, and while the topic has historical value, it would need to be clearly marked as out-of-date.

1

Could it be made a site configuration option as to how much memory can be used? Presently the tuning is set to 2G (Edit: oops, I am wrong, see below) - previously it was 4G - and yet the recommended hardware configuration is 1G + swap. It seems to me that a 1G node might well get into a bit of a paging frenzy if ruby’s usage really did get up to 2G.

From an upgrade log:

Migrating default
Seeding default
*** Bundling assets. This will take a while *** 
$ RUBY_GC_MALLOC_LIMIT_MAX=20971520 RUBY_GC_OLDMALLOC_LIMIT_MAX=20971520 RUBY_GC_HEAP_GROWTH_MAX_SLOTS=50000 RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR=0.9  bundle exec rake assets:precompile
Purging temp files
Bundling assets
1 Like
2

Not sure how relevant these settings are in modern Ruby but @sam would know best.

1 Like
3

Ahem, excuse me, that’s not 2G, that’s 20M. So nothing like the situation that I thought it was. (Edit: it’s worse! See below)

1 Like
4

Try 20G. :wink: :slightly_smiling_face:

1 Like
5

Discourse have a predictable memory usage, so the knob you want is the number of Unicorn workers.

If you have a 1GB server we will default to 2 workers, which should operate well under this low amount of memory.

2 Likes
6

Indeed, while running as a forum. But this is not that!

During upgrades, some tasks specify those envvars for ruby.

1 Like
7

Yes, during updates we set those vars so the upgrade doesn’t fail because of memory pressure on very low spec machines.

2 Likes
8

Right, and my observation is that the limits might be set too high for some cases, so I’m wondering if it can be made configurable.

2 Likes
9

We tested this procedure a lot over the years and can ensure that it will work for small communities using a 1GB VPS just fine. It even terminates additional Unicorn workers to reclaim memory during the upgrade to help.

You can read the code that came from this commit:

https://github.com/discourse/docker_manager/commit/2bc201509216bb9181419e0a87a5bde3866087c7

If you can reproduce it failing on a Digital Ocean droplet we are happy to take a look into it.

If that is a must you can fork the plugin and adjust to your needs.

4 Likes
10

Thanks, I’m getting a picture now. (Two very important points: these GC settings are adjusted specifically for upgrades, and the units are bytes, so the limit is set presently to 20MB. There are useful and interesting details in this earlier thread [archive link because it’s unavailable to me now].)

2 Likes
11

Should we undelete that topic @sam? Is it of historical value or obsolete?

1 Like
12

It is certainly out-of-date, latest ruby has multiple malloc limits. It is an interesting topic with some historical value but would need a big “Out-of-date” sign on top.

Our upgrade is already running as tight as it can, best we could do on top of this is decrease availability. If memory is super constrained then you are going to have to eat outages when you upgrade.

5 Likes