# Can Discourse ship frequent Docker images that do not need to be bootstrapped? **URL:** \ **Category:** Development\ **Created:** [9월 14, 2015, 7:01오전 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205 "2015-09-14T07:01:52Z")\ **Posts on this page:** 20\ **Page:** 4
**Author:** ![bradj](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/bradj/32/121570_2.png) [@bradj](https://meta.discourse.org/u/bradj)\ **Post date:** [6월 10, 2016, 1:20오전 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/61 "2016-06-10T01:20:19Z")
> [@Falco](#): > > but we can’t give away the nice things the today setup gives: easy plugin install and easy updates Sure. I think you’ll have to maintain two paths, and make it clear the relative merits of either. I’d definitely be interested in your feedback on the GitHub project I linked above; I think I’ve successfully pulled together an image that reflects the web container, with Unicorn runner, etc., which is at least a proof of concept. ---
**Author:** ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)\ **Post date:** [6월 10, 2016, 1:27오전 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/62 "2016-06-10T01:27:06Z")
I really like what you did, I think it is a very good first effort. Some points of feedback: - Missing log rotation is quite huge, this should be added, - You should define volumes for uploads and backups - You should define a volume for logs - Booting into runsvdir is a problem, boot into [discourse\_docker/image/base/boot at master · discourse/discourse\_docker · GitHub](https://github.com/discourse/discourse_docker/blob/master/image/base/boot) instead, it gives you a cleaner shutdown and bootup process, the signal transalation there is critical, stuff like `docker stop` will not work right if you boot directly into runsvdir I keep wishing for a shared base, so you don’t have to keep reaching to the official repo to update stuff, maybe do a PR to [GitHub - discourse/discourse\_docker: A Docker image for Discourse · GitHub](https://github.com/discourse/discourse_docker/) and keep your stuff in `images/contrib` symlinking to the original files. ---
**Author:** ![bradj](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/bradj/32/121570_2.png) [@bradj](https://meta.discourse.org/u/bradj)\ **Post date:** [6월 10, 2016, 1:30오전 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/63 "2016-06-10T01:30:16Z")
> [@sam](#): > > keep your stuff in images/contrib symlinking to the original files Thanks for the feedback. Right now I’m not really making any money with Discourse so it’s fitting in where I can, but I will definitely package it up as a PR sometime soon and we can work it there. What are your thoughts on tracking the various small changes that are made right now in the template files? There are a few sed-type translations, file loads, etc. that happen there? ---
**Author:** ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)\ **Post date:** [6월 10, 2016, 1:31오전 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/64 "2016-06-10T01:31:21Z")
> [@bradj](#): > > What are your thoughts on tracking the various small changes that are made right now in the template files? There are a few sed-type translations, file loads, etc. that happen there? You could invoke pups from the Dockerfile, that way you could reuse the actual templates. ---
**Author:** ![fbender](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/fbender/32/116736_2.png) [@fbender](https://meta.discourse.org/u/fbender)\ **Post date:** [6월 19, 2016, 3:48오후 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/65 "2016-06-19T15:48:30Z")
> [@sam](#): > > Missing log rotation is quite huge, this should be added, Is there any particular reason you are not piping logs to STDOUT/STDERR? Preferably, the logging handler of the host should handle all log stuff incl. rotation. If you want to keep some logging view inside Discourse, maybe you can find some middle ground here? ---
**Author:** ![mpalmer](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/mpalmer/32/45740_2.png) [@mpalmer](https://meta.discourse.org/u/mpalmer)\ **Post date:** [6월 20, 2016, 1:07오전 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/66 "2016-06-20T01:07:36Z")
> [@fbender](#): > > Is there any particular reason you are not piping logs to STDOUT/STDERR? A couple spring to mind: 1. Rails’ logging is bad enough already, interleaving log entries from different requests without indicating association; mixing that in with everything else the system chatters about would make an ugly situation even worse. 2. Docker’s default logging configuration isn’t disk-space-conservation friendly, nor does the default log driver have any time-based retention options. At least when we control log rotation within the container we can do something useful to keep disk usage under control in a semi-intelligent way. I also wouldn’t be surprised if our process management framework might complicate the “just stuff everything down stdout/stderr” plan, although I’m sure something could be glued together if necessary – but to me, the above two reasons are enough of a show-stopper to not even bother peering into the innards of runit to see what horrors lurk within. ---
**Author:** ![DoMiN8ToR](https://avatars.discourse-cdn.com/v4/letter/d/b2d939/32.png) [@DoMiN8ToR](https://meta.discourse.org/u/DoMiN8ToR)\ **Post date:** [8월 29, 2016, 2:48오후 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/67 "2016-08-29T14:48:43Z")
@bradj things went so well, then I got this after docker-compose-up Step 8 : RUN /tmp/install-imagemagick —\> Running in 21f1d17e6b93 e[91m/bin/sh: 1: /tmp/install-imagemagick: Permission denied e[0me[31mERRORe[0m: Service ‘app’ failed to build: The command ‘/bin/sh -c /tmp/install-imagemagick’ returned a non-zero code: 126 Anything I do wrong or something that can be fixed? (Edit - pfff, I finally understand things a little, more had to comment out all COPY/RUN image related stuff and it went on, now I’m stuck at can’t read /etc/nginx/nginx.conf: No such file or directory e[0me[31mERRORe[0m: Service ‘app’ failed to build: The command ‘/bin/sh -c sed -i “s/pid /run/nginx.pid;/daemon off;/” /etc/nginx/nginx.conf’ returned a non-zero code: 2 I guess this is a permission thing? ) (Edit 2 - if I comment out the following, the rest continues, but as these are required, it doesn’t work, nearly there ☹ RUN sed -i “s/pid /run/nginx.pid;/daemon off;/” /etc/nginx/nginx.conf RUN rm /etc/nginx/sites-enabled/default && mkdir -p /var/nginx/cache ) (I’m trying to use this on Windows Pro → Docker using Hyper-V.) ---
**Author:** ![bradj](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/bradj/32/121570_2.png) [@bradj](https://meta.discourse.org/u/bradj)\ **Post date:** [9월 1, 2016, 7:20오후 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/68 "2016-09-01T19:20:20Z")
@DoMiN8ToR I haven’t worked on Discourse for Docker in a few months so I apologize my memory on particular items like this is hazy… I will say though that the “right” approach for this is likely what Sam suggests above, building the image using the same tools as Discourse is using now for their novel approach, just without the need for an external bootstrapper. I don’t have a paying client needing Discourse at this moment so it’s on the back burner, but that’s the idea and goal. ---
**Author:** ![DoMiN8ToR](https://avatars.discourse-cdn.com/v4/letter/d/b2d939/32.png) [@DoMiN8ToR](https://meta.discourse.org/u/DoMiN8ToR)\ **Post date:** [9월 3, 2016, 3:55오후 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/69 "2016-09-03T15:55:38Z")
Thanks for getting back to me anyway @bradj. In the end I was able to build the Dockerfile but running it was something different. This appears to be due to the bootstrapping files that need to be executed. Anyway, not giving up yet 😉 ---
**Author:** ![jayfk](https://avatars.discourse-cdn.com/v4/letter/j/2bfe46/32.png) [@jayfk](https://meta.discourse.org/u/jayfk)\ **Post date:** [10월 8, 2016, 2:46오후 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/70 "2016-10-08T14:46:42Z")
> [@Redundancy](#): > > I think what worries me most is that there are assumptions, expectations, tooling, best practices and principles that generally apply to how people work with docker images, and you manage to break almost all of them. The system you have currently is surprising, which is rarely a good thing, and has added to the issue that I can’t assume anything you’re doing follows convention. That’s exactly what I was thinking while reading the code. It’s basically a big hack that somehow manages to use docker under the hood. This could be so much easier by using docker-compose, adding some checks to the app container and using a env file for configuration. The only thing that won’t be possible is to configure the postgres container with environment variables, because the official image isn’t capable of doing that. But, in my opinion, that’s a thing for the docs anyway. ---
**Author:** ![mpalmer](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/mpalmer/32/45740_2.png) [@mpalmer](https://meta.discourse.org/u/mpalmer)\ **Post date:** [10월 10, 2016, 12:25오전 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/71 "2016-10-10T00:25:25Z")
> [@jayfk](#): > > This could be so much easier by using docker-compose, adding some checks to the app container and using a env file for configuration. Several people have said that. Nobody has, as yet, demonstrated the simplicity of that configuration by actually _delivering_ a working setup using docker-compose. If you’d care to be the exception, please have at it. ---
**Author:** ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)\ **Post date:** [10월 10, 2016, 2:20오전 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/72 "2016-10-10T02:20:00Z")
> [@mpalmer](#): > > Several people have said that. Nobody has, as yet, demonstrated the simplicity of that configuration by actually delivering a working setup using docker-compose. If you’d care to be the exception, please have at it. Or perhaps an example of another open source project (containing DB+Web) that is shipped like Discourse that uses this pattern successfully. > [@jayfk](#): > > But, in my opinion, that’s a thing for the docs anyway. Or, in reality, welcome to every Discourse instance out there never upgrading major versions of postgres anymore. Keep in mind, our hosted customers (including you 🙂 ) are all running on the same Docker image self installers are. We keep postgres and redis in separate containers on separate machines. The web runs on multiple hosts and all launcher really does is converts a “conf file” to “command line params”. ---
**Author:** ![pierreozoux](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pierreozoux/32/296611_2.png) [@pierreozoux](https://meta.discourse.org/u/pierreozoux)\ **Post date:** [10월 10, 2016, 9:02오전 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/73 "2016-10-10T09:02:09Z")
> Nobody has, as yet, demonstrated the simplicity of that configuration by > actually /delivering/ a working setup using docker-compose. If you’d > care to be the exception, please have at it. [https://github.com/indiehosters/discourse](https://github.com/indiehosters/discourse) I use PGP to protect our privacy, if you want to know more, you can follow this > **[Email Self-Defense - a guide to fighting surveillance with GnuPG encryption](https://emailselfdefense.fsf.org/en/)** > > Email surveillance violates our fundamental > rights and makes free speech risky. This guide will teach you email > self-defense in 40 minutes with GnuPG. If you have further questions, please do not hesitate to ask. You can verify my public key here: [pierreozoux (Pierre Ozoux) | Keybase](https://keybase.io/pierreozoux) ---
**Author:** ![jayfk](https://avatars.discourse-cdn.com/v4/letter/j/2bfe46/32.png) [@jayfk](https://meta.discourse.org/u/jayfk)\ **Post date:** [10월 10, 2016, 1:03오후 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/74 "2016-10-10T13:03:10Z")
> [@pierreozoux](#): > > Nobody has, as yet, demonstrated the simplicity of that configuration by > actually /delivering/ a working setup using docker-compose. If you’d > care to be the exception, please have at it. > > [GitHub - libresh/compose-discourse: Moved to https://lab.libreho.st/libre.sh/compose/discourse · GitHub](https://github.com/indiehosters/discourse) Yep, this. It’s far from perfect, but a solid foundation for a new implementation. If there is a real interest to change things, I’m willing to help. ---
**Author:** ![jayfk](https://avatars.discourse-cdn.com/v4/letter/j/2bfe46/32.png) [@jayfk](https://meta.discourse.org/u/jayfk)\ **Post date:** [10월 10, 2016, 1:14오후 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/75 "2016-10-10T13:14:38Z")
> [@sam](#): > > Or perhaps an example of another open source project (containing DB+Web) that is shipped like Discourse that uses this pattern successfully. Take a look at Sentry: [https://github.com/docker-library/docs/tree/master/sentry](https://github.com/docker-library/docs/tree/master/sentry) They provide the base image that’s capable of running the webapp (Django) and the worker processes (celery, comparable to sidekiq). I don’t know if you are aware of this, but there’s now even a [docker store](https://store.docker.com/) that lets you ship a “officially endorsed” image and make money from it. ---
**Author:** ![Falco](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/falco/32/179432_2.png) [@Falco](https://meta.discourse.org/u/Falco)\ **Post date:** [10월 10, 2016, 1:41오후 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/76 "2016-10-10T13:41:20Z")
Hey @pierreozoux nice work. This way, when bumping a major version of postgresql the user would need to backup and restore? ---
**Author:** ![jayfk](https://avatars.discourse-cdn.com/v4/letter/j/2bfe46/32.png) [@jayfk](https://meta.discourse.org/u/jayfk)\ **Post date:** [10월 10, 2016, 1:50오후 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/77 "2016-10-10T13:50:47Z")
Yes, that’s the safest path, although there’s a [POC](https://github.com/tianon/docker-postgres-upgrade) using `pg_upgrade`. ---
**Author:** ![pierreozoux](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pierreozoux/32/296611_2.png) [@pierreozoux](https://meta.discourse.org/u/pierreozoux)\ **Post date:** [10월 10, 2016, 3:49오후 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/78 "2016-10-10T15:49:37Z")
@Falco @jayfk Yes it is not perfect, but I use it in prod for some users (5 different instances) and it is doing the job so far. I know there is a lot to improve, but you are welcome to come and PR. About upgrade, I never had to do it yet. But I do it for the end user. If you want, we can even have our own “organization”. It was in a separate orga to collaborate, but people never did. So it is under my namespace. If there are some PRs and people want to have push rights, we can discuss. Cheers! ---
**Author:** ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)\ **Post date:** [10월 10, 2016, 10:54오후 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/79 "2016-10-10T22:54:08Z")
Keep in mind there are two very strong and opinionated camps here clashing. - Discourse self installers vs - Docker “best practices” crowd As it stands our self installers can install latest and update to latest by running one command. This compose based installation has no bootstrapping phase so - To update Discourse a new image needs to be published on docker hub or locally - There is no mechanism for installing plugins short of rolling out your own image On top of that as an admin now you need to worry about: - Troubleshooting multi container issues - Dealing with log rotation of these giant Rails logs and NGINX logs Additionally you lose out on: `./launcher enter app` which now becomes `docker exec -it rails /bin/bash` and `./launcher cleanup` which has no analog until the next major version of Docker is released. and `./launcher rebuild app` becomes `docker pull xyx/discourse && docker compose down && docker compose up` From a performance perspective this proposed setup decouple NGINX from unicorn, something I strongly disagree with and means that stuff like [XSendfile](https://www.nginx.com/resources/wiki/start/topics/examples/xsendfile/) no longer works and all static files go through the unicorn process at least once. If you start sharing directories between the NGINX and unicorn containers then you start violating the Docker purity rules so… “best practice” ends up being slower. Additionally, we take lots of care in our base image to pick the exact version of image magick and a bunch of other libraries that seem missing from your image. I appreciate the effort here but am uneasy giving this extra work to all self installers and worry about the huge additional burden we will have supporting such a setup, it would become: - Install docker - Install docker compose - Edit file - Setup log rotation - Learn how to use compose - docker compose up I just don’t see this as an improvement over what we have now. Regardless this kind of setup is doable today using the official image, just bootstrap it… then tag it an publish to docker hub. ---
**Author:** ![jayfk](https://avatars.discourse-cdn.com/v4/letter/j/2bfe46/32.png) [@jayfk](https://meta.discourse.org/u/jayfk)\ **Post date:** [10월 11, 2016, 8:12오전 UTC](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205/80 "2016-10-11T08:12:32Z")
> [@sam](#): > > As it stands our self installers can install latest and update to latest by running one command. This compose based installation has no bootstrapping phase so > > [snip] The idea is to ship a bash script that runs docker-compose under the hood and is doing the install/upgrade/whatnot. > [@sam](#): > > If you start sharing directories between the NGINX and unicorn containers then you start violating the Docker purity rules so… “best practice” ends up being slower. That’s simply not true. You can share volumes just fine. > [@sam](#): > > I just don’t see this as an improvement over what we have now. Because you live and breathe the code you wrote there. Let’s put this in another perspective: Say I’ve just found out about ruby and want to write my own website. Instead of using RoR I’m writing my own thing that implements a WSGI subset and speaks directly to nginx. It works perfect, is extremely performant and easy to use for end users, but nobody ever understands what I wrote there. It’s the same thing here. Instead of using something like docker-compose you wrote your own thing. That’s totally fine and was probably the right thing to do at this time. But really, only the team actively working on it understands what’s going on. It basically comes down to: Do you want to take the longer road that people understand by using established tools, or do you prefer to use well working shortcuts nobody is able to follow (except the team) by writing your own thing? Again, I totally understand why you did it this way. If you ever prefer to go down the docker-compose route, @ping me at GitHub and I’ll make sure to help you out with all roadblocks you might encounter. [이전 페이지](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205.md?page=3) [다음 페이지](https://meta.discourse.org/t/can-discourse-ship-frequent-docker-images-that-do-not-need-to-be-bootstrapped/33205.md?page=5)