Configure ruby's memory limit?

feature
1

Could it be made a site configuration option as to how much memory can be used? Presently the tuning is set to 2G (Edit: oops, I am wrong, see below) - previously it was 4G - and yet the recommended hardware configuration is 1G + swap. It seems to me that a 1G node might well get into a bit of a paging frenzy if ruby’s usage really did get up to 2G.

From an upgrade log:

Migrating default
Seeding default
*** Bundling assets. This will take a while *** 
$ RUBY_GC_MALLOC_LIMIT_MAX=20971520 RUBY_GC_OLDMALLOC_LIMIT_MAX=20971520 RUBY_GC_HEAP_GROWTH_MAX_SLOTS=50000 RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR=0.9  bundle exec rake assets:precompile
Purging temp files
Bundling assets
1 Like
2

Not sure how relevant these settings are in modern Ruby but @sam would know best.

1 Like
3

Ahem, excuse me, that’s not 2G, that’s 20M. So nothing like the situation that I thought it was. (Edit: it’s worse! See below)

1 Like
4

Try 20G. :wink: :slightly_smiling_face:

1 Like
5

Discourse have a predictable memory usage, so the knob you want is the number of Unicorn workers.

https://github.com/discourse/discourse_docker/blob/master/discourse-setup#L247

If you have a 1GB server we will default to 2 workers, which should operate well under this low amount of memory.

2 Likes
6

Indeed, while running as a forum. But this is not that!

During upgrades, some tasks specify those envvars for ruby.

1 Like
7

Yes, during updates we set those vars so the upgrade doesn’t fail because of memory pressure on very low spec machines.

2 Likes
8

Right, and my observation is that the limits might be set too high for some cases, so I’m wondering if it can be made configurable.

2 Likes
9

We tested this procedure a lot over the years and can ensure that it will work for small communities using a 1GB VPS just fine. It even terminates additional Unicorn workers to reclaim memory during the upgrade to help.

You can read the code that came from this commit:

https://github.com/discourse/docker_manager/commit/2bc201509216bb9181419e0a87a5bde3866087c7

If you can reproduce it failing on a Digital Ocean droplet we are happy to take a look into it.

If that is a must you can fork the plugin and adjust to your needs.

4 Likes
10

Thanks, I’m getting a picture now. (Two very important points: these GC settings are adjusted specifically for upgrades, and the units are bytes, so the limit is set presently to 20MB. There are useful and interesting details in this earlier thread [archive link because it’s unavailable to me now].)

2 Likes
11

Should we undelete that topic @sam? Is it of historical value or obsolete?

1 Like
12

It is certainly out-of-date, latest ruby has multiple malloc limits. It is an interesting topic with some historical value but would need a big “Out-of-date” sign on top.

Our upgrade is already running as tight as it can, best we could do on top of this is decrease availability. If memory is super constrained then you are going to have to eat outages when you upgrade.

5 Likes