Sidekiq is consuming too much memory even with UNICORN_SIDEKIQ_MAX_RSS adjusted

Was there any resolution to this, as I see in my logs:

Sidekiq is consuming too much memory (using: 501.37M) , restarting

UNICORN_SIDEKIQ_MAX_RSS: 1000

and the site is not a very busy one

What else should I try?