Why did Discourse start running their own Redis instead of using Elasticache?

Support Self-hosting

AI-generated summary

The discussion revolves around why Discourse switched from using Amazon’s Elasticache to running their own Redis. According to Falco, the main reasons were two pain points with Elasticache:

  1. Instance types: Elasticache required a minimum instance type of M4.large to run HA Redis without Cluster Mode, which was a limitation for smaller sites.
  2. Read-only mode: Elasticache made it difficult to provide endpoints for replicas in the main cluster, which was necessary for Discourse to keep reading when masters went down.

However, Falco notes that since 2019, AWS has addressed these issues, allowing smaller instance types and providing replica endpoints.

Additionally, sam points out that running their own Redis also offers cost savings, as they no longer pay the Elasticache service tax and can optimize instance utilization.

1

Continuing the discussion from Meta is moving to the Cloud :cloud_with_lightning::

This sounds like a really interesting story. I’d love to hear it.

Have you told it already anywhere?

2

Our main pain points were two.

1. Instance types

Elasticache didn’t allow you to run HA Redis without Cluster Mode on instance types below M4.large.

Support for cluster mode in Ruby Redis libraries is still being ironed out. MessageBus also does atomic operations on keys using LUA scripts and since that will span multiple servers it’s not allowed in cluster mode. We do host some BIG instances, so we may revisit using distributed writes for databases (Rails 6 is coming with that for PostgreSQL) but we aren’t needing this yet.

We do host sites which need instance types way larger than that. But also host a lot where a t3.small is more than enough.

Moving to running our own Redis allows us to pick and play with any instance types available in the target region.

2. Read only mode

Discourse can keep a connection open with read only nodes with both Discourse and Redis.

That allows people to keep reading on the site when masters go down.

Elasticache wasn’t very straightforward in providing endpoints for the replicas in the main cluster, that updated to current replicas in failover events.

3

Also … not to forget. This offers us significant cost savings.

When you use ElasticCache you pay for the instances plus a tax for the ElasticCache service. Additionally we get much better control on utilisation since we can run multiple redises if we wish on a single instance.

4

Hi Falco, after reading your points, it seems like the free Elasticache do not support the requirement of Redis for Discourse. Therefore if I want to discouple my discourse site, I must find a Redis server in some other place. Is my understanding correct?

5

That’s not true. Elasticache works just fine for Discourse, and since 2019 AWS made good progress to fix both of our pain points that were described above. AWS lets you pick smaller instance types and provides replica endpoints now.

6

Thanks Falco, I will have a try!