Closing the loop here, over the past 2 years we had one large change that reduced sidekiq memory. We made a change to the hosted v8 instance to ensure it cleans up used memory and runs a gc. This resulted in a 10-20% memory reduction in our hosting.
Except for that, yes, on mega busy multisites you need to adjust UNICORN_SIDEKIQ_MAX_RSS