How do you scale discourse horizontally?

Installation
1

Hi there,
I’m confused by the installation instructions. Is there a way to scale discourse horizontally?
For ex: having 2+ docker containers on different servers each running discourse?

In the instructions from GitHub - discourse/discourse_docker: A Docker image for Discourse

The multiple container configuration setup is far more flexible and robust, however it is also more complicated to set up. A multiple container setup allows you to:

  • Minimize downtime when upgrading to new versions of Discourse. You can bootstrap new web processes while your site is running and only after it is built, switch the new image in.
  • Scale your forum to multiple servers.
  • Add servers for redundancy.
  • Have some required services (e.g. the database) run on beefier hardware.

If you want a multiple container setup, see the data.yml and web_only.yml templates in the samples directory. To ease this process, launcher will inject an env var called DISCOURSE_HOST_IP which will be available inside the image.

Is there a place where I can find more info on how I can run the containers on different servers, and not many containers on the same server?

Thanks

2

There is a lot of related discussion on Can Discourse ship frequent Docker images that do not need to be bootstrapped?.

The gist of it is:

  1. You provide external services for PostgreSQL, Redis and Load Balancer and configure the app.yml to use those, like for example Running Discourse with a separate PostgreSQL server.

  2. Run ./launcher bootstrap app somewhere. This will result in a container image being created in the local machine. Upload this image to your container registry service.

  3. Now you can spin as many instances as you want, telling all to run the image exported in the previous step.

4 Likes
3

You can do something like Running Discourse docker alongside Apache and create two web containers with something being a reverse proxy in front.

You can use multisite to have a single container serving multiple sites (using the same plugins and Discourse version). This describes how to do that without an external reverse proxy. If you try that, please leave a comment there and let me know whether it worked. I intend to post it here when I am fairly certain it works.

Running containers on multiple servers is the same as any of the above. You’d have multiple containers sharing the same Redis and Postgres running behind a load balancer.

Mostly doing anything other than the Discourse official Standard Installation is unsupported because there are infinite numbers of ways to do those things.

4

I have two questions:

  • First one for Sidekiq in the case of running multiple web_only.yml docker each in different VM: So in this first case, different Sidekiq instance sharing same Redis server may run same job twice, is Discourse robust for this kind of error? Is it more reliable to use one Sidekiq in a dedicated VM (or Docker), so how to separate it from web_only Discourse?
  • About your last remarks for supported installation: I can understand that people can have infinity ways to make multiple container configuration but, in my opinion, we need a supported way to run multiple instance for horizontal scaling or/and High Availability in two cases: based on VM (different Discourse web_only docker in different VMs and Redis/Postgres are installed in other separate VMs directly without using docker) and Based on containers (everything is based on container).