Gold words.
Yes, but its possible not to defend at all. People can tell what they want, and its ok to reply “ok, you are right”. Its not necessary to do what they tell.
1 Like
I just wish the energy spent here would be spent making an letsencrypt ssl template instead. With all of this energy and passion we could have had a working, free, and trivial to install SSL option for all our self hosters.
Want SSL? add this line.
templates:
- "templates/web.ssl.letsencrypt.yml"
just sayin…
11 Likes
If anyone want to manage this before me, you can have a look at an automation someone already did.
2 Likes
There’s letsencrypt.sh, which does not rely on external software like Python.
But for the Let’s Encrypt discussion, see https://meta.discourse.org/t/support-for-lets-encrypt/22308/
3 Likes
@sam Hi! Today I played with rebuild command and something interested me.
Two most time consuming commands are:
- bundle install
- rake assets:precompile
If we add docker volume with bundler cache, it will dramatically decrease execution time for bundler install. Is it possible?
What info rake assets:precompile use from DB? If I run default template version, will it change comparing to another default template version? Is it possible to speed up this process for default installations?
1 Like
Looks like this will get a lot faster soon, with work done by Sam on mini_racer.
1 Like
Not really, the Ruby gem used to uglify is arbitrarily 2x slower than running the js uglifier via node, so this is not getting any faster, we use uglifier via node
6 Likes
First off, in reading this thread, the first thing that strikes me is the patience and helpfulness of @sam and his team. You all have built some amazing software here and we all owe you a big thank you. Double-thanks for making the project open-source.
Sam et. al. definitely have the right to build their project out as they see fit; I can also understand the perspective of those (myself included) who see the Docker image boostrapping used by Discourse to be not consistent with many similar software packages in the container ecosystem. In particular, using a launcher script from the command line is not how most/all orchestration tools work, at least in my limited expeirence.
I think the answer here is for the broader user community to accept the architectural decision made by the maintainers, but work to generate an “alternative” base image that captures common use cases. As Matt points out, there’s no way you can do everything with an omnibus image, but we probably can get to a point where those of us who prefer a “single” base image can keep up with the mainline offering.
I’ve taken the initiative to follow Sam’s recommendation to pull the “knowledge” from the base image and have created a fork of the project @pierreozoux mentions above to make a slightly leaner base image. I’m actively committing to https://github.com/BradJonesLLC/docker-discourse so take a look and let me know what you think.
12 Likes
I appreciate the sentiment here a lot, and totally understand where you are coming from.
Regarding this effort I would strongly recommend coupling NGINX, I get the desire to keep stuff separate but NGINX+discourse are a unified piece. There is a lot of smarts in our NGINX config around integrating the two. try_files and X-Send-File and so on, https://github.com/discourse/discourse/blob/master/config/nginx.sample.conf by decoupling a bunch of this logic becomes impossibly complicated.
Furthermore, these days our base image is already compiling NGINX.
I am open to making a leaner discourse/base image that strips out some of the components and to layer them in later for the “everything container”, would much prefer if there was a single shared base to having a fork.
8 Likes
Thanks! In large part due to your encouraging response, I’ve continued to work on this and think I have a (mostly?) working container that mirrors the bootstrapping that occurs in the standard web template. I put most of my applications behind a standard Nginx TLS-terminating reverse proxy and think that’s pretty common, so didn’t worry about the SSL stuff yet.
I’m not sure the best way to move forward on a “base image” like we’re discussing but hopefully this is a decent proof of concept.
3 Likes
If it is just for try_files and X-Send-File, you can mount volume from the app container in the web container. I do it like this on my wordpress setup:
https://github.com/indiehosters/wordpress/blob/master/docker-compose.yml#L21-L22
@bradj To keep you updated, I have a setup that I use in production for 3 instances, and would be happy to collaborate on it.
You can find it here: https://github.com/indiehosters/discourse
@sam I still think that discourse should be listed here also: https://hub.docker.com/explore/
I already helped Piwik, ownCloud, Rocket.Chat and Known to be listed there, so I could help you also if you want.
But the first step would be to have a decoupled app image, with a single process living in. And it would be better if based on the current rails image, or you’d need strong arguments to convince docker official repo (I’m sure you do 
)
Let me know if you need!
2 Likes
Thanks, Pierre, and you’ll see while I didn’t fork your project directly in the GitHub UI, I actually started with your repo! The difference in our approaches, though, is that I’ve tried to bring in the configuration as it’s done in the Discourse image bootstrapper. I was able to fire up an application with your IndieHosters config but it runs directly with rails, not Unicorn, etc.
While I’m approaching my limit on the heavy-lifting work I can justify on this at the moment, I think the next steps here would be for a project-maintainer to decide what, if any, approach they would want to take for an “omnibus” (to share GitLab’s term) image and we can go from there.
1 Like
@bradj and @pierreozoux,
Would this work help on running Discourse on Container Services, like Google Container Engine and Amazon EC2 Container Service?
I think that would be a really great addition.
1 Like
@Falco That’s the idea. You’d likely need linked/networked containers for Redis, PgSQL, but yeah.
2 Likes
That means you need to start storing precompiled assets in a volume, which in turn complicates deployment over multiple hosts. We serve all our static and precompiled assets from inside the container, not a volume.
Don’t get me started on this
So … single process you say, fine.
-
Are we allowed to use postgres? it forks off a worker per connection? It is clearly violating single process per container?
-
What about NGINX? also forking workers? also not allowed?
-
Are we allowed to boot with tini https://github.com/krallin/tini , I thought it was made for Docker?
-
What about the app server? We are using which forks for unicorn workers? Is that allowed. But… we also fork off sidekiq workers, perhaps that violates the “multiple processes only allowed if they have the same name Docker rule”
-
Ok, so we allow unicorn and allow forking off of sidekiq, violating the “multiple processes only allowed if they have the same name docker rule”, What about the microcontroller that takes care of seamless web-ui based upgrades? Is that allowed? Do we give up on “web UI” based upgrades for Docker purity?
Wordpress is somehow allowed to ship apache… I see no reason why we are banned from shipping NGINX and our container boot process.
Similarly official redmine is allowed to ship NGINX I see no reason why we should be banned.
I can accept linking to PG and Redis, it does come with some serious caveats because when 9.6 is released I am not sure how people will upgrade.
I am fine for people to BYO NGINX, just set our NGINX as the upstream. For me the minimal 3 pieces are
- Web
- Postgres
- Redis
Breaking it down any further than this is not something I can support.
I am open to getting us an official bootstrapped web image, upgrades get a bit tricky and plugins + additional precompiles that are required after plugin install are very complicated. I do welcome the effort, but don’t want forks, ideally I want one repo that is official, takes care of all the concerns
9 Likes
One way would be obligatory backup restore when PostgreSQL updates.
1 Like
Thing is … as it stands there is no clean documented way to upgrade the official PG docker image.
Visit: Docker Hub , search for “upgrade” nothing is there.
If you start with docker run postgres:9.5.3 nuke the container and then do docker run postgres:9.6 you are out of luck. Stuff simply will be broken and container will not launch:
That is because both 9.5.3 and 9.6 have this exact same line in the Dockerfile
VOLUME /var/lib/postgresql/data
So, you would have to:
- Backup Discourse using web UI
- Destroy your PG container
- Destroy the Docker data volume for the PG container (good luck teaching people how to do that, also you probably want to backup first)
- Start a new blank PG in running new version
- Get the username/password working so Discourse can auth
- Get Discourse to somehow run migrations on this database
- Launch web UI
- Restore the database
Considering how much struggle the zero effort current upgrade process was I am not optimistic the general public can handle that.
5 Likes
I think this is a central point; an omnibus base Docker image isn’t suited for the “general public,” the audience is users who have a compelling technical motivation to run Discourse without the need for a bootstrap script. If anything, the “base” image could/would be exactly that, a base upon which a developer could build a derivative image with plugins, etc.
If you’re looking for one-click everything, then the bootstrap script is for you. If you can understand the technical overhead of managing your own database, keeping the schema in line with the code, etc. etc., then you’ll find use in the image we’re discussing.
One point of clarification, though - I don’t think the VOLUME line in the Dockerfile is to blame for any particular upgrade pains re: Postgres. That declaration just establishes the mount point for the db data. It’s broader than that and there’s a long-running issue in the repo on this point:
https://github.com/docker-library/postgres/issues/37
But again - out-of-scope for a Discourse app-only container, I think.
2 Likes
That’s true. I wasn’t advocating for the single process mantra just saying how we upgrade PG at work, where we don’t pg_upgrade our way, and backup/restore every time.
I think a bootstrapped image would do wonders, by making people use containers hosts, but we can’t give away the nice things the today setup gives: easy plugin install and easy updates.
1 Like