How to move from standalone container to separate web and data containers

[jomaxro (Joshua Rosenfeld)]

In other words: what’s the catch?

It’s more complicated to set up? More has to be done during setup? More potential points of failure? Harder to debug if you don’t understand Docker?

[pfaffman (Jay Pfaffman)]

In other words: what’s the catch?

For people like you and me, there is none.

When the 30 minute install was conceived, it involved editing app.yml with a tool like nano (I’m an Emacs user and even I prefer vi to nano). Having people edit copy and edit two files and bootstrap and start both of them in the right order is on the order of 10 times more complicated. Now that ./discourse-setup is how most people configure Discourse, the setup part could be exactly the same for a two-container setup. I’ve looked in to doing just that & it wouldn’t be very hard.

But even still, with two containers, there would then be a bunch of problems with the data container wasn’t running and then no one would say which way their site was configured and that would be a lot more complicated to help out. Most of the time the web-based upgrade works just fine, and so unless you’re changing your plugin config, there’s not that much of a win for the two-container setup.

I think soon that I’ll start offering a two-container setup along side of my $99 install, but I’ve not gotten around to it.

[tophee (Christoph)]

For people like you and me, there is none

So is everyone here just pretending that they’re running one container but privately they’re running two?

Well, I guess, maybe even for “people like you and me” it is more convenient with one container, given that you don’t change plugins so often.

On the other hand, on standard troubleshooting advice that keeps coming up is obviously “disable all apps and re-enable them onr by one” and unless you do that by just disabling them under settings, this will give you plenty of downtime with one container…

And when I see that people are talking about 10,000 visits per day, that is quite a few annoyed users, even for half an hour downtime.

Anyway, thanks for explaining. And, yes, you should offer the 2-container install, if only to make it better known

[pfaffman (Jay Pfaffman)]

So is everyone here just pretending that they’re running one container but privately they’re running two?

No. Usually if someone posts a problem and they’re running multiple containers, they’ll mention that (probably because it’s a problem specifically with multiple containers), but mostly, if you know how to have multiple containers, it won’t make any difference that you do.

FWIW, it took me nearly 2 years to (bother to) figure it out. And six months of that time I was earning all of my income from Discourse consulting (not to say that the income I earned was a living all of that time).

I’d guess that the vast majority of people running Discourse have a single container. I’d guess that the vast majority of people* who earn some of their income from managing or hosting it* and/or would identify as a “system administrator” run two.

[net_deamon]

In other words: what’s the catch?

It is not that hard too. You have app.yml file which contains both the properties of datasource and web related(which port discourse should run eg 9000, or the plugins config and the custom commands)

So you just divide the app.yml into data.yml and web.yml.
Data.yml will contain the datasources part from app.yml
While the web.yml will contain rest of config.

I usually use nginx webserver infront of discourse.
So I can rebuild another web contianer at say 9001, and reverse proxy to it from nginx.
Then I safely stop the previous web container running at 9000.
This swapping is done in few seconds… So there is no downtime.

[Berto]

Could use some help here:

Start with data.yml and set a password for the database. Then:

This is confusing. It states to do a step (set a password) but doesn’t state how to do that aforementioned step… and immediately says “then” do some other stuff. Are we missing instructions on setting this password here?

So I didn’t change any password because I don’t know how or what OP is talking about, but did run ./launcher boostrap data and got the following response:

[...bootstrap command running...]
Successfully bootstrapped, to startup use ./launcher start data

prompt$ ./launcher enter data
Error: No such container: data

Note that I didn’t rename anything, only copied the files. I simply have data.yml and web_only.yml in my /var/discourse/containers directory.

Thanks!

[5minpause]

I wrote this “guide” in May 2015. I do not use Discourse any more (stopped soon after). I do not know if any of these instructions still work or how things are done nowadays.

[Berto]

Thanks, people are still linking to it, going to just hire some help. Cheers!

[codinghorror (Jeff Atwood)]

Thanks for getting it started, we will take it from here!

[TheBestPessimist]

Is this still the good tutorial?

Or should follow

Running Discourse with a separate PostgreSQL server

Let’s say you’re using AWS RDS, or an existing PostgreSQL server managed elsewhere. It’s fairly straightforward to get Discourse to use such a setup, without needing to grant superuser privileges to anything that Discourse itself runs. Here’s how. 0. Prerequisites Discourse requires PostgreSQL 10 or later; if you try to run on an older version, it might work, but we don’t guarantee it, and if something breaks in the future, you get to keep both pieces. We also require some extensions that ar…

or

Multisite configuration with Docker

You may wish to host multiple domains on a singled Docker setup. To do so follow these instructions: We strongly recommend you run separate web and data containers The standalone container is extremely easy to configure, however has a couple of drawbacks You will need to stop your site when bootstrapping a new image You run through the db bootstrap only when needed Multisite is a fairly advanced topic, learn about hooks before attempting this. Understand hooks Discourse templates use pups, …

Right now both of those tutorials still leave me with questions :-/.

[Falco (Rafael dos Santos Silva)]

The 3 tutorials apply to 3 different situations, so pick the one that applies to what you want.

Running Discourse with a separate PostgreSQL server is for when you have an external PostgreSQL running somewhere else, like AWS RDS.

Multisite configuration with Docker is about running multiple Discourse instances inside the same container.

And this topic is about using different containers for data and web.

The three guides are for advanced users, and we recommend sticking to defaults for people who aren’t familiar with Discourse, containers and the whole sysadmin lingo.

[TheBestPessimist]

well, what i want is to be able to host 3 discourse forums on my own VM.

From that i understand that i need to

1. Separate the data and web containers (this also brings speedup when rebuilding the app)
2. Configure 2 other discourse instances (somehow?) for my 2 other forums.

So this is why i don’t know exactly how to approach this situation.

[fefrei (Felix Freiberger)]

Separate the data and web containers (this also brings speedup when rebuilding the app)

You may want to do that (mainly to reduce rebuild times), but this is not required, and doesn’t really have anything to do with running multiple sites.

To run three sites, you can either bootstrap them separately (which is rather easy, but triples resource requirements), or use these instructions for setting up multisite:

Multisite configuration with Docker

You may wish to host multiple domains on a singled Docker setup. To do so follow these instructions: We strongly recommend you run separate web and data containers The standalone container is extremely easy to configure, however has a couple of drawbacks You will need to stop your site when bootstrapping a new image You run through the db bootstrap only when needed Multisite is a fairly advanced topic, learn about hooks before attempting this. Understand hooks Discourse templates use pups, …

I’m running a setup like this (i.e. multisite, but without separating data and web containers or any other fanciness), and this works fine, but setup is indeed a bit tricky.

[TheBestPessimist]

Well I want ALL the speed, so i will have to do both of these.

[fefrei (Felix Freiberger)]

You can sure do both

But just to dampen expectations: The multi-container setup reduces downtime when you rebuild, but will not improve performance during normal operation.

[joebuhlig]

I made a couple minor edits above to include starting the containers and not just bootstrap them. I only noticed that when going through this process myself just now. Other than that component, it all went as expected.

[BobbyZopfan (Bobby)]

This guide is excellent. But does leave a few doubts, which I’d like to list (although I seem to have completed all the steps in the guide above, successfully, I’m not sure, don’t know a way to tell, have I succeeded or not, although my website didn’t stop working):

1. In the op above, its been told to create 1 web server and 1 data server. And when needed, we have to rebuild both. Then how can we save the downtime? Won’t 2 files take as much time in rebuilding as 1 combined app.yml? Or do we need to build total 3 containers: 2 web containers (+1 data container) so that when we stop/rebuild the one web container, the other keeps running? If yes, then the method has been missed in the tutorial guide. How do we do that?

Found answer: There will remain 2 containers always: webonly and data. When next time you bootstrap your webonly container, your site won’t stop. It’ll only stop for a few seconds, a minute in rare cases, when your new image is ready and then when you destroy your old container and start the new one.
2. Could something go wrong if we don’t change the names of web_only.yml and data.yml?
Answer found: No, doesn’t matter. I’d recommend no change initially, till you are very conversant.
3. Start with data.yml and set a password for the database: Is this line asking to replace the wording SOME_SECRET in the last lines of data.yml? What if we skip this step (i.e. what if don’t setup any pw for the db)?
Found answer: Yes, it says that only. No need to change it while practicing. Change later on when you’re an expert.

6. No guideline has been truly segregated for the persons running nginx inside the container and for those running the outside container. Are all steps exactly same?
   Answer: No big deal, apply your mind and you’ll find that both cases are almost same.
7. And, after we’ve done all the steps, how can be sure that which web container are we using now? Old app.yml or new web_only.yml? In future, how can we save time, by running what?
   Answer found: Its clear in your running containers list. You can stop the ones you don’t need.

[dfriestedt (Drew Friestedt)]

I’m trying to implement the separate containers, but with a remote database. I’ve followed these instructions above and the howto for setting up a remote postgres DB. The setup works, but I’m wondering why there are two identical references (under web_only and data) to the same database. It makes me believe I’m doing something wrong and the web_only container is not even using the data container.

Am I doing this correctly?

Here is my setup.

Under the web_only.yml I added:

  DISCOURSE_DB_SOCKET: ''
  DISCOURSE_DB_USERNAME: REMOVE
  DISCOURSE_DB_PASSWORD: REMOVE
  DISCOURSE_DB_HOST: xxx.ondigitalocean.com
  DISCOURSE_DB_NAME: REMOVE
  DISCOURSE_DB_PORT: 25060
  DISCOURSE_DB_BACKUP_PORT: 25060
  DISCOURSE_REDIS_HOST: data

under data.yml

I removed postgres.template.yml

templates:
#  - "templates/postgres.template.yml"
  - "templates/redis.template.yml"

I also added the following:

env:
  # ensure locale exists in container, you may need to install it
  LANG: en_US.UTF-8
  DISCOURSE_DB_USERNAME: REMOVE
  DISCOURSE_DB_PASSWORD: REMOVE
  DISCOURSE_DB_HOST: REMOVE.ondigitalocean.com
  DISCOURSE_DB_NAME: REMOVE
  DISCOURSE_DB_PORT: 25060
  DISCOURSE_DB_BACKUP_PORT: 25060

[pfaffman (Jay Pfaffman)]

If you’re using a remote database, you don’t need to create the data container which has a database in it. Note that you need both postgres and redis (so you might need the data container for that).

[dfriestedt (Drew Friestedt)]

If I were to run postgres and redis remotely, should I just continue to use app.yml?

Or is there an advantage or difference in running discourse with web_only.yml while using the same postgres and redis remotely?

I’m trying to setup discourse so I can update it per your post below. If I continue to use app.yml would the update command be:

./launcher bootstrap app && ./launcher destroy app && ./launcher start app

Does web based updates replace rebuilding the container?

For a description see: How to move from standalone container to separate web and data containers. For a normal rebuild/upgrade It’s pretty much the same as a single container install. Rather than ./launcher rebuild app you do ./launcher bootstrap web_only && ./launcher destroy web_only && ./launcher start web_only

thx for help!