Redis used_memory limited to 1024mb?

Product Support

AI-generated summary

User markersocial reported concerns that Redis usage appeared capped at 1024MB, noting that exceeding this threshold in the Sidekiq dashboard triggered severe network latency (RTT) warnings. Despite setting UNICORN_SIDEKIQ_MAX_RSS to 8000 in app.yml, the redis-cli info memory output showed used_memory_human at 1023.14M while maxmemory remained at 0. The user questioned if the configuration variable was intended to increase Redis RAM limits, given that Sidekiq RSS usage reached 4,553 MB. In response, sam clarified that Redis only consumes the memory it requires and that the RTT errors were likely unrelated to memory limits, suggesting instead that the issues stemmed from the installation’s location or insufficient machine resources. markersocial subsequently acknowledged that the RTT problems were likely caused by Transparent Huge Pages (THP) remaining enabled, referencing a related deployment configuration thread.

1

It seems like there is some limit preventing Redis from being able to use more than 1024mb?

I have noticed as soon as that amount is exceeded in the Sidekiq dashboard (MB RSS) is when I start getting logs like:

Your Redis network connection is performing extremely poorly. Last RTT readings were [96585, 101554, 97189, 99769, 94618], ideally these should be < 1000. Ensure Redis is running in the same AZ or dat

When I check using this:
./launcher enter app
redis-cli info memory

This is the result:

# Memory
used_memory:1072837696
used_memory_human:1023.14M
used_memory_rss:484577280
used_memory_rss_human:462.13M
..
maxmemory:0
maxmemory_human:0B

I set this in the app.yml:
UNICORN_SIDEKIQ_MAX_RSS: 8000

In the Sidekiq dashboard, there is 4,553 MB RSS being used and utilisation is 100%. This is with 2 UNICORN_SIDEKIQS with DISCOURSE_SIDEKIQ_WORKERS not being set (so the default of 5).

I was under the impression UNICORN_SIDEKIQ_MAX_RSS would allow redis to use more ram? I’m a bit lost, any guidance would be appreciated! :love_letter:

This is with Discourse 3.3.2 Stable.

5

Redis will only uses the memory it needs, the RTT errors are unrelated, they are more likely coming from an install being far away or a machine being out of juice to run Redis.

6

Thanks! I believe the RTT issues were because THP was still enabled, when I had thought otherwise: