Mainly because I’ve been asked to by our team lead. His argument is when testing … he doesn’t want to rely on sysadmins, or devs knowing how to set up discourse other than doing docker-compose up -d.
I agree … I followed the instructions and it didn’t take a lot of time to set up. The main issue I will have is selling the idea of running the scripts
I am still not following at all, maybe ask your dev manager to post here explaining why our officially supported docker dev setup is a problem.
I asked if Beginners Guide to Install Discourse for Development using Docker would suffice.
The answer was yes.
I am not associated with the OP here, but I can tell you why I want a docker compose file rather than a shell script:
- Shell scripts tend to not work on Windows.
- The shell script requires git, my docker hosts only have a bare OS and docker installed. They are, by design, incredibly lean and bare.
- Docker compose files can, with a little work, be converted to docker swarm files which can be run on my infrastructure and managed with docker swarm management tooling.
- Every other piece of my infrastructure is managed via docker-swarm tooling. Having a single piece of infrastructure that is an outlier adds a large amount of cognitive overhead that I need to store somewhere and will certainly be lost/forgotten when it comes time to upgrade (every other piece of our infrastructure gets upgrades by just updating image version in docker swarm file).
What I would like to see is a template docker swarm or docker compose file that I can use, which references an image published to Docker Hub, and I can configure by just editing the various swarm file settings.
I haven’t yet tried running the scripts locally to see if I can extract the compose file, but it sounds like that is my best bet to not have Discourse be some oddball outlier in my system. I would prefer if I didn’t need to go run a script just to generate a sample docker-compose/swarm file and if instead I could just see one in docs or a gist or something.
I want to be able to do all docker building on a separate build server to where I run in production. Why use Docker if it is a bunch of scripts that are required on the host?
On my production host I want Docker installed in swarm mode and the ability to Docker login to my registry to pull what is needed. I also want to be able to run e.g 3 instances of any container over 3 nodes (so any volumes / bind mounts to the host should be easily manageable).
Totally happy to build and do exactly the supported discourse way on staging server.
The prod docker swarm I am talking about will include many other products on the same cluster.
I plan to docker inspect all the containers on my Discourse officially supported staging server, push all pre-built docker images to a registry and try deploy them to production swarm cluster.
If I get there I will share here.
I would want to split out into multiple containers. From what I have seen, the officially supported Discourse treats Docker like VM and does everything in one container (how do you scale only one component if necessary?).
Maybe the open-sourced supported version is purposefully not orchestration manager friendly so that they can get contracts to manage deploys that require massive scale. That model sits fine by me. They should have every right to be able to make money installing Discourse to large already established production grade Swarm/Kubernetes systems.
Not saying that is how some of their income is earned, just speculating. Not important whether they are or not, just pointing out that they are in no way obligated to support anything other than their currently supported mechanisms. It must just be acknowledged that the supported method is not friendly towards a production grade container orchestration cluster.
My understanding is it for ease of support - that CDKC uses the open sourced version - they do make their income from hosting and that pays for the free support they give to the official supported version.
There is a guide to splitting the containers here Multisite configuration with Docker
We did manage to get a instance running under docker-compose so what you’re aiming for is certainly doable.
Did you open-source it? Would be great to see some examples.
We used an older version of libre.sh / compose / discourse · GitLab as a guide. That should get you going.
It’s May 2020 and this is still a completely contentious area of usage.
I’ve read every thread I can find on current Discourse support for docker and I’ve gone through the 5 stages of grief on this - I’m now at resignation.
I have a docker swarm that runs everything for our sites and businesses. I don’t want to buy/maintain additional VPS just to host a forum.
I just want to deploy Discourse image on it with individual containers so I can scale things differently as needed. So far I have failed. It should not be this hard.
I have done this with far more complex but comparable codebases like Gitlab and Sentry in 1-2 hours flat. This is my 3rd week of trying to get a working solution with Discourse and still no reliable installation.
Seems to me…
- Discourse is feature rich and been around a while and has passionate and skilled contributors.
- Discourse was an early adopter of containerisation and did what it did using both the tools and the knowledge around at the time.
- Criticism of how it does what it does is often met with defensiveness.
- Clear cut guidance is missing on this exact problem that many people over the last few years have asked and many more will ask in the future re: cluster orchestration.
- Launcher etc is fine for a single docker machine, it is a beautiful script that suits a common use case. But my experience of using it create a portable image is failure.
Things that are silly…
- the passive aggressive responses to legitimate concerns raised by people just trying to get a tiny part of their work done.
- insisting that launcher is the ONLY way it should work.
- referring to ‘documentation’ in threads that are out of date and expecting people to spend ages trying to read up such content to simply install something.
We can do better than this.
The ask / offer
I am willing to spend my time with someone who knows how launcher works and to create a step by step quide for creating repeatable separate container installs - with a video.
Who is up for this?
Hey Mike. Basically, what you do is use
./launcher bootstrap to build the image, push it to a repo, and then use
./launcher start-cmd to see what ENV needs to get passed to Docker to crank things up.
The reason that such a guide doesn’t exist is that people who don’t have the most basic understanding of any of the underlying pieces work will find it and not be able to make sense of it and then expect a tutorial on how ingress works on whatever system that the think is the only answer.
Thanks Jay - I figured that much out. I finally made it work.
- Will create a video and guide for exactly what I did, so that the next me doesn’t spend their time trying to figure this out.
I guess part of the issue is that it goes counter the grain of all the other docker images out there. It isn’t as plug and play. It is more fiddle, plug and then play.
As I said, the launcher is a beautiful and reliable thing for what it does.
On a really positive note, the web only build works a treat, its super fast on my swarm, updateable because I have the build server.
When I have time I will explore a build server on my swarm that allows a web based build of a docker image.
Things I tried that left much to be desired…
- bitnami docker image - ran like a dog with no legs.
- indiehosts dockerfile - didn’t get as far as a build - seemed determined to make everything, I think it was compiling linux. Again there has to an easier way!
Ok - thanks.
we got that First Mover Disadvantage
The launcher script was created before docker-compose, docker-swarm, kubernetes, and all that existed, and… it still works. There’s not really any compelling reason for us to switch to something else. The “advantages” of these systems aren’t advantages for the #1 consumer of Discourse self-hosting, which is budget installations on cloud VPSes.
Oh! I always wondered. And last I looked, it was an extra step to install docker-compose on Ubuntu. It’s like they don’t want people to use it, unless those people use windows (or Mac?)
A bit sad to hear this as official response from the development team. You basically saying that if you’re not good enough to figure out this undocumented process of running Discourse in production then maybe you shouldn’t be running Discourse in production.
Well, not every one of us has extensive devops experience, sadly.
@Mike_Sutton I am on the same page with you. Have been reading the forums for the last few weeks trying to crack this nut. Did you manage to make a video on how you solved the problem?
Would that work? What about DB migrations?
In this scenario, you’d have bootstrap do the migrations as it builds the new container image.
Yes, you can push Discourse docker images to a repo and archive the rest of
/var/discourse like in the discussion below but it is not an efficient way to do things and is not officially supported. I recently fully tested this:
But in this case the bootstrap step must be done in the production machine, this kinda defeats the purpose of pushing a base image to a container registry, if I’m not mistaken (because this would be done with the intention of not having to bootstrap an image in production, except maybe to run a db migration step or something like that).
This is what I do in non discourse sites I host, I normally use flyway to run migrations in the db, along with some other stuff like emptying the redis cache, but without running other intensive processes, especially ones that depends on external services, like updating https certificates (I would do in a cronjob, except the 1st time, which is fine), or changing the codebase (this would be static in the image, like packages, libraries, and other stuff). It seems discourse uses rake which is fine, but it does other stuff like installing gems, generating assets, updating maxmind db and maybe other services (which would break the rebuild step if, for example, the service is down, or if they change the API, which is rare but could happen).
I’m not saying that discourse should do that way, of course, but generating an image in a CI environment and just running a db migration step in staging/production servers, along with defining the environment variables, is what I would expect, and is easily achievable with other softwares like Wordpress, Mediawiki, Rocketchat, etc… Discourse is the best software for forums IMO, and they might have good reasons to do it the way they do, but for now I would only use the standard install (or multicontainer install) to avoid headaches and just hope that nothing goes wrong with rebuilds.
It seems like the bootstrap should still be done in a production environment, and not in a CI environment to generate a base image to use in both a staging and production environment. Furthermore, not being a standard install is a huge red flag, that can became a headache in the future as the software receives updates.