Should we switch to Docker?

(Logan Rosen) #1

I saw that the Ubuntu installation/upgrade guide on git was deprecated, which made me concerned about future support for our installation. Should we consider switching to Docker, or can we stay where we’re at and expect that upgrades will work properly?

We’re running DB and redis separately, and it looks like Docker packages them all into one container. If we were to split them out, would that break Docker upgrades?

(Régis Hanol) #2

This is not mandantory, just highly recommended.

Only if you use the single container :wink:

We already support multiple containers. If you look in the samples directory, you’ll find templates for the database, redis and NGINX/rails application. It’s a little more complicated than the standalone but we can assure you that they work, because that’s what we’re using.

(Jeff Atwood) #3

If you would like to help us flesh out our Docker recipe for multiple containers, with more specific steps in the guide, please do!

@sam I wonder if we could have a demo of a multiple container guide on a much larger Digital Ocean instance like the $40 per month plan which is 4gb, or the $80 per month plan which is 4 cores and 8gb.

(stellarhopper) #4

I’m in a similar position as OP. Is there a docker migration guide yet? I couldn’t spot one in docs/ …

(Sam Saffron) #5

we have a guide now:

(Lee_Ars) #6

I just feel curmudgeonly about this, maybe because it’s Friday and it’s been a long-ass week. It feels like forcing abstraction into a place where none is truly needed, and adding what looks like a ludicrous amount of containerized complexity.

Am I reading the docs wrong, or do I really need to run entirely separate instances of postgresql, nginx, and redis inside the docker container(s)? That seems literally insane.

Sorry for being that guy who posts the big crappy image macro. I want to be a good beta user and do things the right way. But if that means calving off Discourse and just duplicating and containerizing a ton of infrastructure applications that are already perfectly capable of sane, traditional multitenancy, then my interest level is way, way, low.

(Sam Saffron) #7

No, you do not need to run everything in one container. For example, we don’t.

We have sample containers for redis / postgres and web.

You can run the web in docker and postgres / redis on the main box if you wish.

NGINX is in the container for a very good reason, we have this file discourse/nginx.sample.conf at master · discourse/discourse · GitHub we occasionally update. When using the docker setup I can amend our recommended setup and have confidence people will be running it next time they bootstrap. It’s not an imagineered problem, we hit it multiple times last year.

If you insist on running NGINX out of your container you can amend your container definition and just delete /etc/service/nginx

I find “complexity” an “interesting” argument here. I don’t long for the days I had to follow a 100 step setup guide to get a server configured.

The abstraction is there to ensure consistency, it completely eliminates a huge class of support issues, freeing us up to work on features :slight_smile:

(Jacob) #8

I think this would make my life easier with the https situation. Could you explain how to go about this in a little more detail? Thank you!

(Lee_Ars) #9

It’s a matter of where you want to spend your resources, and how you approach the administration of a system. This is philosophical as much as technical.

Containerized schemes like this absolutely have their place, and I’d be crazy to say that this isn’t a good move for Discourse as a deployable application! But this is not wholly a good move for monolithic servers hosting multiple applications.

When an application brings its own web stack with it and that app includes a web server that expects to be on port 80, that’s more than problematic—it’s presumptive. With the exception of your map variable, everything you’re doing within your nginx sample config is located in the server-context—and that’s what vhosts are for. Bringing along a whole web server just to get a few config stanzas right isn’t just inelegant—it’s downright ugly.

I know, I know—minor complaints, just use nginx outside of the container if you’re confident in your ability to make it work. Which I’d have to do, because otherwise my discourse traffic would be reverse proxied through varnish to a reverse-proxy that would reverse-proxy it to a containerized reverse proxy that would reverse-proxy it to containerized thin instances and cue the Inception music.

It just feels like this is a developer-centric move, made from a developer-centric mindset. You can’t assume that the installation environment or the installer have everything you need, so you pack in module after module to make the app as self-sufficient as possible. Every prereq and dependency you can think of gets stuffed into the container, and the app is as immune as possible to the environment in which it sits.

But it’s just graceless. Applications should be tenants—they should live in the house, not bring a whole other house along with them to set up inside of the first hosue. The developer shouldn’t presume that the environment is known-hostile or known-broken. The developer’s chocolate should stay entirely out of the sysadmin’s peanut butter. Plus, reading your blog post about docker, the ownership and mounting issues are gag-me ugly (though I recognize that’s not entirely docker’s fault), and are exactly the kind of thing you don’t want to expose an inexperienced admin to.

Shit, man. I know you guys have worked hard on this, and I’m sorry. Just feeling grumpy. Maybe I’m like the gearhead bemoaning the fact that under the hood of my car there’s nothing but a big bunch of plastic covers. But, seriously, for the love of all that’s good and holy in this world, please realize that some of us know how to run a server, and duplication of services and extra abstraction are unwanted complexity.

Okay. Okay, I’m good now. I’m ready to leave this week behind and be good. Promise :smiley:

(Jeff Atwood) #10

Longer term the resource issues do not matter. Being able to package and deploy Linux apps in standardized shipping containers facilitates a massive improvement in, well, everything:

There is always the manual install, we just can’t directly support it.

(Sam Saffron) #11

That is completely not the case, from: discourse_docker/web_only.yml at master · discourse/discourse_docker · GitHub change to:

  - "8089:80"
  - "2222:22"

Now the web is on port 8089, problem solved.

The sad reality about Ruby is that it is not graceless, its moving fast, apt-get install ruby on precise gives you drumroll Ruby 1.8.7 which is no longer maintained. Getting a good install of Ruby on a box is a non trivial problem.

I totally understand Docker may not be for everyone, if its not for you … fine, stay with you current setup. However, you are likely to miss out on a pile of optimisations I make like jemalloc which reduces memory usage, proper gc tuning, out-of-band gc and more. Heck you run on apache which means you don’t get proper long polling, which is a huge problem in itself.

This is actually very much an OPs move on our part, ask @supermathie docker allows us to isolate customers and have extreme control over environments and configuration. It makes deploying easier as well.

(Ben T) #12

To me, containerization connects directly with the way hosting has become less about the physical box and more about allocating resources to hosted apps. I might not have a lot of past experience… but the move back to virtualization and the cloud give sysadmins the chance of set the environment per application and not per physical box. Instead of “the production SQL server” I hear “application 1’s 2nd SQL server on boxA”

I live in an apartment complex with some questionable internet and would likely need to co-locate a server if I wanted to get things done. Instead, I can spread some cheap VPSes that have varying goals and prices per month and pool individual resources and know exactly where my security issues arise. I can also pay for the resources I think I’ll need… and other cloud buzz words.

I point everything to a front-facing virtual server that’s locked down; which proxies out requests to various other virtual servers. Then, various applications do their things in various locations; all tied together. Security flaw in PHP? I only have to push updates to the server with php on it. Something broken beyond repair? Rollback the entire server without fear of crippling other apps. Someone crack your password? Everything’s isolated (and hopefully using different passwords!)

Of course there are drawbacks in that set up (network outages, updates, ), but overall it solves a lot of poking around when the SQL server crashes and it takes out all the apps.

… getting back to containers:

Containerization is just a more scalable and sensible way to get it done. I view it as one in the same as separating out hosted applications. You can still go under the hood and tinker with things; but when that part goes haywire it does not have to take apart everything to fix it. Having an easy to manage list of separate instances is a win in my book… given it’s easy to set up an isolated application.

(Lee_Ars) #13

I know, I know. You guys do excellent work—i’m just venting. I’ll likely get changed over at some point in the next month or so. And you’ve made provisions for folks like me who are going to want to break at least some of the container walls—or at least rearrange the furniture.

Scandal, sir. My stack is varnish + nginx + passenger! Apache does not sully my servers. I’ve had that long-polling box checked for quite some time now :smile:

(Erlend Sogge Heggen) #14

I think a “How to run Discourse (on Docker) alongside a LAMP stack” tutorial is in order. We recently had this question come up on our forum as well.

(Lee_Ars) #15

It’s been a few weeks since I switched over to Docker, and I’m happy to say that my reservations were unfounded and my bitching was unwarranted. Miniprofiler-reported perf numbers have dropped significantly from my original much-battered non-Docker install, and I can attack the forum with Siege and it’ll support 1000 simultaneous connections without any problem (which is kinda silly, since my poor 10Mbps uplink can’t sustain more than about 50 simultaneous connections).

The single-button-click update process is wonderful and is worth any amount of adaptation pain—and the two-button backup & restore process meant that there just wasn’t that much adaptation pain in the first place.

You guys are awesome. Carry on.

(Dave McClure) #16

First off, glad to hear that you’re liking the docker install now and why.

But I just have to jump in here and say that I thoroughly enjoyed reading your earlier rants… some of the most thoughtful well-written ranting I’ve ever read. Don’t let this experience stop you from being the guy to post the big crappy image macro when you just gotta do what you gotta do.

(Gastón Sánchez) #17

Heck, I enjoyed your rants too. It covers my current concerns with Docker. But since it’s working for you I guess I’ll give it another shot…

(Jeff Atwood) #18