How to move from standalone container to separate web and data containers

It’s more complicated to set up? More has to be done during setup? More potential points of failure? Harder to debug if you don’t understand Docker?

6 Likes

For people like you and me, there is none.

When the 30 minute install was conceived, it involved editing app.yml with a tool like nano (I’m an Emacs user and even I prefer vi to nano). Having people edit copy and edit two files and bootstrap and start both of them in the right order is on the order of 10 times more complicated. Now that ./discourse-setup is how most people configure Discourse, the setup part could be exactly the same for a two-container setup. I’ve looked in to doing just that & it wouldn’t be very hard.

But even still, with two containers, there would then be a bunch of problems with the data container wasn’t running and then no one would say which way their site was configured and that would be a lot more complicated to help out. Most of the time the web-based upgrade works just fine, and so unless you’re changing your plugin config, there’s not that much of a win for the two-container setup.

I think soon that I’ll start offering a two-container setup along side of my $99 install, but I’ve not gotten around to it.

4 Likes

So is everyone here just pretending that they’re running one container but privately they’re running two?

Well, I guess, maybe even for “people like you and me” it is more convenient with one container, given that you don’t change plugins so often.

On the other hand, on standard troubleshooting advice that keeps coming up is obviously “disable all apps and re-enable them onr by one” and unless you do that by just disabling them under settings, this will give you plenty of downtime with one container…

And when I see that people are talking about 10,000 visits per day, that is quite a few annoyed users, even for half an hour downtime.

Anyway, thanks for explaining. And, yes, you should offer the 2-container install, if only to make it better known :wink:

2 Likes

No. Usually if someone posts a problem and they’re running multiple containers, they’ll mention that (probably because it’s a problem specifically with multiple containers), but mostly, if you know how to have multiple containers, it won’t make any difference that you do.

FWIW, it took me nearly 2 years to (bother to) figure it out. And six months of that time I was earning all of my income from Discourse consulting (not to say that the income I earned was a living all of that time).

I’d guess that the vast majority of people running Discourse have a single container. I’d guess that the vast majority of people* who earn some of their income from managing or hosting it* and/or would identify as a “system administrator” run two.

4 Likes

It is not that hard too. You have app.yml file which contains both the properties of datasource and web related(which port discourse should run eg 9000, or the plugins config and the custom commands)

So you just divide the app.yml into data.yml and web.yml.
Data.yml will contain the datasources part from app.yml
While the web.yml will contain rest of config.

I usually use nginx webserver infront of discourse.
So I can rebuild another web contianer at say 9001, and reverse proxy to it from nginx.
Then I safely stop the previous web container running at 9000.
This swapping is done in few seconds… So there is no downtime.

4 Likes

Could use some help here:

This is confusing. It states to do a step (set a password) but doesn’t state how to do that aforementioned step… and immediately says “then” do some other stuff. Are we missing instructions on setting this password here?

So I didn’t change any password because I don’t know how or what OP is talking about, but did run ./launcher boostrap data and got the following response:

[...bootstrap command running...]
Successfully bootstrapped, to startup use ./launcher start data

prompt$ ./launcher enter data
Error: No such container: data

Note that I didn’t rename anything, only copied the files. I simply have data.yml and web_only.yml in my /var/discourse/containers directory.

Thanks!

1 Like

I wrote this “guide” in May 2015. I do not use Discourse any more (stopped soon after). I do not know if any of these instructions still work or how things are done nowadays.

2 Likes

Thanks, people are still linking to it, going to just hire some help. Cheers!

Thanks for getting it started, we will take it from here!

4 Likes

Is this still the good tutorial?

Or should follow

or

Right now both of those tutorials still leave me with questions :-/.

1 Like

The 3 tutorials apply to 3 different situations, so pick the one that applies to what you want.

Running Discourse with a separate PostgreSQL server is for when you have an external PostgreSQL running somewhere else, like AWS RDS.

Multisite configuration with Docker is about running multiple Discourse instances inside the same container.

And this topic is about using different containers for data and web.

The three guides are for advanced users, and we recommend sticking to defaults for people who aren’t familiar with Discourse, containers and the whole sysadmin lingo.

6 Likes

well, what i want is to be able to host 3 discourse forums on my own VM.

From that i understand that i need to

  1. Separate the data and web containers (this also brings speedup when rebuilding the app)
  2. Configure 2 other discourse instances (somehow?) for my 2 other forums.

So this is why i don’t know exactly how to approach this situation.

1 Like

You may want to do that (mainly to reduce rebuild times), but this is not required, and doesn’t really have anything to do with running multiple sites.

To run three sites, you can either bootstrap them separately (which is rather easy, but triples resource requirements), or use these instructions for setting up multisite:

I’m running a setup like this (i.e. multisite, but without separating data and web containers or any other fanciness), and this works fine, but setup is indeed a bit tricky.

3 Likes

Well I want ALL the speed, so i will have to do both of these. :smiley:

You can sure do both :slight_smile:

But just to dampen expectations: The multi-container setup reduces downtime when you rebuild, but will not improve performance during normal operation.

9 Likes

I made a couple minor edits above to include starting the containers and not just bootstrap them. I only noticed that when going through this process myself just now. Other than that component, it all went as expected. :+1:

5 Likes

This guide is excellent. But does leave a few doubts, which I’d like to list (although I seem to have completed all the steps in the guide above, successfully, I’m not sure, don’t know a way to tell, have I succeeded or not, although my website didn’t stop working):

  1. In the op above, its been told to create 1 web server and 1 data server. And when needed, we have to rebuild both. Then how can we save the downtime? Won’t 2 files take as much time in rebuilding as 1 combined app.yml? Or do we need to build total 3 containers: 2 web containers (+1 data container) so that when we stop/rebuild the one web container, the other keeps running? If yes, then the method has been missed in the tutorial guide. How do we do that?

Found answer: There will remain 2 containers always: webonly and data. When next time you bootstrap your webonly container, your site won’t stop. It’ll only stop for a few seconds, a minute in rare cases, when your new image is ready and then when you destroy your old container and start the new one.
2. Could something go wrong if we don’t change the names of web_only.yml and data.yml?
Answer found: No, doesn’t matter. I’d recommend no change initially, till you are very conversant.
3. Start with data.yml and set a password for the database: Is this line asking to replace the wording SOME_SECRET in the last lines of data.yml? What if we skip this step (i.e. what if don’t setup any pw for the db)?
Found answer: Yes, it says that only. No need to change it while practicing. Change later on when you’re an expert.

  1. No guideline has been truly segregated for the persons running nginx inside the container and for those running the outside container. Are all steps exactly same?
    Answer: No big deal, apply your mind and you’ll find that both cases are almost same.
  2. And, after we’ve done all the steps, how can be sure that which web container are we using now? Old app.yml or new web_only.yml? In future, how can we save time, by running what?
    Answer found: Its clear in your running containers list. You can stop the ones you don’t need.

I’m trying to implement the separate containers, but with a remote database. I’ve followed these instructions above and the howto for setting up a remote postgres DB. The setup works, but I’m wondering why there are two identical references (under web_only and data) to the same database. It makes me believe I’m doing something wrong and the web_only container is not even using the data container.

Am I doing this correctly?

Here is my setup.

Under the web_only.yml I added:

  DISCOURSE_DB_SOCKET: ''
  DISCOURSE_DB_USERNAME: REMOVE
  DISCOURSE_DB_PASSWORD: REMOVE
  DISCOURSE_DB_HOST: xxx.ondigitalocean.com
  DISCOURSE_DB_NAME: REMOVE
  DISCOURSE_DB_PORT: 25060
  DISCOURSE_DB_BACKUP_PORT: 25060
  DISCOURSE_REDIS_HOST: data

under data.yml

I removed postgres.template.yml

templates:
#  - "templates/postgres.template.yml"
  - "templates/redis.template.yml"

I also added the following:

env:
  # ensure locale exists in container, you may need to install it
  LANG: en_US.UTF-8
  DISCOURSE_DB_USERNAME: REMOVE
  DISCOURSE_DB_PASSWORD: REMOVE
  DISCOURSE_DB_HOST: REMOVE.ondigitalocean.com
  DISCOURSE_DB_NAME: REMOVE
  DISCOURSE_DB_PORT: 25060
  DISCOURSE_DB_BACKUP_PORT: 25060

If you’re using a remote database, you don’t need to create the data container which has a database in it. Note that you need both postgres and redis (so you might need the data container for that).

If I were to run postgres and redis remotely, should I just continue to use app.yml?

Or is there an advantage or difference in running discourse with web_only.yml while using the same postgres and redis remotely?

I’m trying to setup discourse so I can update it per your post below. If I continue to use app.yml would the update command be:

./launcher bootstrap app && ./launcher destroy app && ./launcher start app

thx for help!