Redis error : 'Loading redis is loading dataset in memory'


(Loganathan Shanmugam) #1

We are running discourse docker on amazon ec2 t2.xlarge instance (32GB RAM in which 24GB is allocated to discourse dcoker app,8 cpu’s) and postgres database host for discourse is running seperately on AWS RDS db.m4.2xlarge instance.
At present this is our dataset size:
2.1 Million users,
5 million topics,
6 million posts.

Now a days,We are encountering 502 bad gateway error frequently and our production error log says:
“Loading redis is loading dataset in memory”

Whenever ,
1.we are trying to load admin dashboard after bulk topic creation or
2.whenever we are trying to post bulk replies to topics the above one is being logged into production.log file.
3.whenever we are trying to load latest topics by category (present category count 100)

This is our unicorn error log:

_> Failed to report error: LOADING Redis is loading the dataset in memory 3 Failed to warm up pretty text: LOADING Redis is loading the dataset in memory _
_> I, [2018-02-21T14:06:33.008665 #14913]  INFO -- : worker=0 ready_
_> Failed to report error: LOADING Redis is loading the dataset in memory 2 LOADING Redis is loading the dataset in memory subscribe failed, reconnecting in 1 second. Call stack ["/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.5/lib/redis/client.rb:121:in `call'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.5/lib/redis.rb:862:in `block in get'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.5/lib/redis.rb:58:in `block in synchronize'", "/usr/local/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.5/lib/redis.rb:58:in `synchronize'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.5/lib/redis.rb:861:in `get'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/message_bus-2.1.2/lib/message_bus/backends/redis.rb:284:in `process_global_backlog'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/message_bus-2.1.2/lib/message_bus/backends/redis.rb:320:in `block in global_subscribe'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/message_bus-2.1.2/lib/message_bus/backends/redis.rb:333:in `global_subscribe'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/message_bus-2.1.2/lib/message_bus.rb:525:in `global_subscribe_thread'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/message_bus-2.1.2/lib/message_bus.rb:473:in `block in new_subscriber_thread'"] _
_> Failed to report error: LOADING Redis is loading the dataset in memory 3 Failed to warm up pretty text: LOADING Redis is loading the dataset in memory_ 

I am new to discourse and Rails and Redis.Any help would be appreciated.


(Matt Palmer) #2

Redis doesn’t serve requests until it has loaded all its data into memory. So you need to wait until that’s done before anything will work. Since you’re already using RDS, why don’t you use Redis ElastiCache as well?


(Loganathan Shanmugam) #3

Ok thanks for the reponse.Is there any documentation or tutorial available to do it.