Sure. I think you’ll have to maintain two paths, and make it clear the relative merits of either. I’d definitely be interested in your feedback on the GitHub project I linked above; I think I’ve successfully pulled together an image that reflects the web container, with Unicorn runner, etc., which is at least a proof of concept.
1 Like
I really like what you did, I think it is a very good first effort.
Some points of feedback:
- Missing log rotation is quite huge, this should be added,
- You should define volumes for uploads and backups
- You should define a volume for logs
- Booting into runsvdir is a problem, boot into https://github.com/discourse/discourse_docker/blob/master/image/base/boot instead, it gives you a cleaner shutdown and bootup process, the signal transalation there is critical, stuff like
docker stopwill not work right if you boot directly into runsvdir
I keep wishing for a shared base, so you don’t have to keep reaching to the official repo to update stuff, maybe do a PR to https://github.com/discourse/discourse_docker/ and keep your stuff in images/contrib symlinking to the original files.
2 Likes
Thanks for the feedback. Right now I’m not really making any money with Discourse so it’s fitting in where I can, but I will definitely package it up as a PR sometime soon and we can work it there. What are your thoughts on tracking the various small changes that are made right now in the template files? There are a few sed-type translations, file loads, etc. that happen there?
You could invoke pups from the Dockerfile, that way you could reuse the actual templates.
1 Like
Is there any particular reason you are not piping logs to STDOUT/STDERR? Preferably, the logging handler of the host should handle all log stuff incl. rotation. If you want to keep some logging view inside Discourse, maybe you can find some middle ground here?
1 Like
A couple spring to mind:
- Rails’ logging is bad enough already, interleaving log entries from different requests without indicating association; mixing that in with everything else the system chatters about would make an ugly situation even worse.
- Docker’s default logging configuration isn’t disk-space-conservation friendly, nor does the default log driver have any time-based retention options. At least when we control log rotation within the container we can do something useful to keep disk usage under control in a semi-intelligent way.
I also wouldn’t be surprised if our process management framework might complicate the “just stuff everything down stdout/stderr” plan, although I’m sure something could be glued together if necessary – but to me, the above two reasons are enough of a show-stopper to not even bother peering into the innards of runit to see what horrors lurk within.
6 Likes
@bradj things went so well, then I got this after docker-compose-up
Step 8 : RUN /tmp/install-imagemagick
—> Running in 21f1d17e6b93
e[91m/bin/sh: 1: /tmp/install-imagemagick: Permission denied
e[0me[31mERRORe[0m: Service ‘app’ failed to build: The command ‘/bin/sh -c /tmp/install-imagemagick’ returned a non-zero code: 126
Anything I do wrong or something that can be fixed?
(Edit - pfff, I finally understand things a little, more had to comment out all COPY/RUN image related stuff and it went on, now I’m stuck at can’t read /etc/nginx/nginx.conf: No such file or directory
e[0me[31mERRORe[0m: Service ‘app’ failed to build: The command ‘/bin/sh -c sed -i “s/pid /run/nginx.pid;/daemon off;/” /etc/nginx/nginx.conf’ returned a non-zero code: 2
I guess this is a permission thing?
)
(Edit 2 - if I comment out the following, the rest continues, but as these are required, it doesn’t work, nearly there 
RUN sed -i “s/pid /run/nginx.pid;/daemon off;/” /etc/nginx/nginx.conf
RUN rm /etc/nginx/sites-enabled/default
&& mkdir -p /var/nginx/cache
)
(I’m trying to use this on Windows Pro -> Docker using Hyper-V.)
@DoMiN8ToR I haven’t worked on Discourse for Docker in a few months so I apologize my memory on particular items like this is hazy… I will say though that the “right” approach for this is likely what Sam suggests above, building the image using the same tools as Discourse is using now for their novel approach, just without the need for an external bootstrapper. I don’t have a paying client needing Discourse at this moment so it’s on the back burner, but that’s the idea and goal.
Thanks for getting back to me anyway @bradj. In the end I was able to build the Dockerfile but running it was something different. This appears to be due to the bootstrapping files that need to be executed. Anyway, not giving up yet 
That’s exactly what I was thinking while reading the code. It’s basically a big hack that somehow manages to use docker under the hood.
This could be so much easier by using docker-compose, adding some checks to the app container and using a env file for configuration.
The only thing that won’t be possible is to configure the postgres container with environment variables, because the official image isn’t capable of doing that. But, in my opinion, that’s a thing for the docs anyway.
3 Likes
Several people have said that. Nobody has, as yet, demonstrated the simplicity of that configuration by actually delivering a working setup using docker-compose. If you’d care to be the exception, please have at it.
6 Likes
Or perhaps an example of another open source project (containing DB+Web) that is shipped like Discourse that uses this pattern successfully.
Or, in reality, welcome to every Discourse instance out there never upgrading major versions of postgres anymore.
Keep in mind, our hosted customers (including you 
) are all running on the same Docker image self installers are. We keep postgres and redis in separate containers on separate machines. The web runs on multiple hosts and all launcher really does is converts a “conf file” to “command line params”.
3 Likes
Nobody has, as yet, demonstrated the simplicity of that configuration by
actually /delivering/ a working setup using docker-compose. If you’d
care to be the exception, please have at it.
I use PGP to protect our privacy, if you want to know more, you can
follow this
If you have further questions, please do not hesitate to ask.
You can verify my public key here: pierreozoux (Pierre Ozoux) | Keybase
3 Likes
Yep, this. It’s far from perfect, but a solid foundation for a new implementation. If there is a real interest to change things, I’m willing to help.
1 Like
Take a look at Sentry: https://github.com/docker-library/docs/tree/master/sentry
They provide the base image that’s capable of running the webapp (Django) and the worker processes (celery, comparable to sidekiq).
I don’t know if you are aware of this, but there’s now even a docker store that lets you ship a “officially endorsed” image and make money from it.
2 Likes
Hey @pierreozoux nice work.
This way, when bumping a major version of postgresql the user would need to backup and restore?
Yes, that’s the safest path, although there’s a POC using pg_upgrade.
1 Like
Yes it is not perfect, but I use it in prod for some users (5 different instances) and it is doing the job so far.
I know there is a lot to improve, but you are welcome to come and PR.
About upgrade, I never had to do it yet. But I do it for the end user.
If you want, we can even have our own “organization”. It was in a separate orga to collaborate, but people never did. So it is under my namespace. If there are some PRs and people want to have push rights, we can discuss.
Cheers!
6 Likes
Keep in mind there are two very strong and opinionated camps here clashing.
- Discourse self installers
vs
- Docker “best practices” crowd
As it stands our self installers can install latest and update to latest by running one command. This compose based installation has no bootstrapping phase so
- To update Discourse a new image needs to be published on docker hub or locally
- There is no mechanism for installing plugins short of rolling out your own image
On top of that as an admin now you need to worry about:
- Troubleshooting multi container issues
- Dealing with log rotation of these giant Rails logs and NGINX logs
Additionally you lose out on:
./launcher enter app which now becomes docker exec -it rails /bin/bash
and
./launcher cleanup which has no analog until the next major version of Docker is released.
and
./launcher rebuild app becomes docker pull xyx/discourse && docker compose down && docker compose up
From a performance perspective this proposed setup decouple NGINX from unicorn, something I strongly disagree with and means that stuff like XSendfile no longer works and all static files go through the unicorn process at least once. If you start sharing directories between the NGINX and unicorn containers then you start violating the Docker purity rules so… “best practice” ends up being slower.
Additionally, we take lots of care in our base image to pick the exact version of image magick and a bunch of other libraries that seem missing from your image.
I appreciate the effort here but am uneasy giving this extra work to all self installers and worry about the huge additional burden we will have supporting such a setup, it would become:
- Install docker
- Install docker compose
- Edit file
- Setup log rotation
- Learn how to use compose
- docker compose up
I just don’t see this as an improvement over what we have now. Regardless this kind of setup is doable today using the official image, just bootstrap it… then tag it an publish to docker hub.
5 Likes
The idea is to ship a bash script that runs docker-compose under the hood and is doing the install/upgrade/whatnot.
That’s simply not true. You can share volumes just fine.
Because you live and breathe the code you wrote there.
Let’s put this in another perspective: Say I’ve just found out about ruby and want to write my own website. Instead of using RoR I’m writing my own thing that implements a WSGI subset and speaks directly to nginx. It works perfect, is extremely performant and easy to use for end users, but nobody ever understands what I wrote there.
It’s the same thing here. Instead of using something like docker-compose you wrote your own thing. That’s totally fine and was probably the right thing to do at this time. But really, only the team actively working on it understands what’s going on.
It basically comes down to: Do you want to take the longer road that people understand by using established tools, or do you prefer to use well working shortcuts nobody is able to follow (except the team) by writing your own thing?
Again, I totally understand why you did it this way. If you ever prefer to go down the docker-compose route, @ping me at GitHub and I’ll make sure to help you out with all roadblocks you might encounter.
3 Likes