Worker process is more than defined?

AI-generated summary

User Vencent noticed that their Discourse container has more worker processes than expected, with 4 workers shown in top despite setting UNICORN_WORKERS to 2. Falco explained that this is because other processes like Sidekiq and Nginx are running in the container.

Vencent then mentioned that their low-traffic site had OOM errors with a 1GB RAM limit. Schleifer recommended a minimum configuration of 2GB RAM + 2GB swap, but Vencent was looking for RAM optimization ideas to avoid upgrading their server.

RGJ suggested reducing the number of Unicorn and Sidekiq workers, but Vencent was concerned that this could cause their site to block if a single request is doing I/O. RGJ reassured that it won’t be that bad, and Vencent clarified that they were asking out of curiosity.

The discussion also highlighted the importance of having swap space, with Falco stating that disabling swap is an unsupported configuration, and RGJ and Corsterix advising against disabling swap.

I use discourse_docker to generate Docker image, then deploy to kubernetes. In my container/web_only.yml:

env:
  UNICORN_WORKERS: 2

However, if I execute top in the container, I can see 4 workers:

Is it out of control, or did I missed something?

That controle the number of unicorn workers but that isn’t the only thing in the container. We also run sidekiq, nginx, etc. Sidekiq also shows as Ruby in top until you press c

Thank you. I see. So is this the minimum configuration for a low-traffic site? I set a maximum memory usage of 1GB to the container and it got OOM a few times.

Our recommended configuration is at least 2G of RAM + 2G of swap.

Yeah, I understand. The “recommended” configuration, I guess, is for a real “community”. My instance, however, is only a support forum for my personal project with really low traffic (really). I know bigger RAM can always solve the problem, but it would be great if I can get some ideas on RAM optimization so I don’t have to upgrade my server with actually no need:)

The minimum is 1GB Ram with 2GB Swap. And if you follow the official install guide we automatically set swap for you. Did you get OOM with swap?

You could probably get away with one unicorn (you have two now) and one sidekiq.

That is what I’m thinking. Does that mean my whole site will block if a single request is doing I/O (i.e. database read)? I’m not really familiar with the tech details in Discourse but it seems likely to happen.

No, I disabled swap because my cloud service provider suggests me to. I also heard it’s better to disable swap on a Kubernetes node. Maybe I should try to loose the RAM limit for Discourse and see what happens.

Well, so that is an unsupported configuration. Good luck with the experiments, but that is not a setup we support.

Yes, but that’s in the same league as “it’s better to have more RAM” .
And if you don’t have more RAM then you should have swap as a fallback.

No, it won’t be THAT bad.
But you said you had really low traffic…

I’m not a Ruby developer but this is what seems to happen when there is only one thread in one process and the code has no async I/O support.

Yeah, I do have really low traffic. Just asking for curiosity :grin:

I’m not sure about Unicorn, I know thin has an eventmachine that deals with this.

Disabling swap is an extremely bad idea, I suggest you ignore your cloud service provider and get yourself some swap.