Community supported official docker image

dev
21

Awesome! I’ve bookmarked this post, and if anybody asks how to run Discourse in Kubernetes or Swarm, I’ll be sure to point them at your images :+1:. Advantage of being not-an-employee: my word doesn’t have to carry the weight of being Official™.

They, on the other hand, benefit immensely from having One Official™️ Way of deploying Discourse. CDCK is not going to take on the load of maintaining a deployment system that they themselves don’t use and is massive overkill for most self-hosters. And if they don’t maintain it, they ain’t endorsing it. Brand protection demands it.

11 Likes
22

Thanks for creating this thread @pierreozoux!

I recently reached critical appreciation for discourse and got interested in hosting a deployment. I’m currently hosting JupyterHub, GitLab and Mattermost - everything through Helm charts and would very much like to do the same with Discourse.

Some background about Helm / Kubernetes:
A Helm chart is a set of configurable Kubernetes resources, and Kubernetes resources are for example a Deployment Kubernetes resource that makes a given docker image run at all time. Installing something can end up becoming a single line command + some configuration.

I would be happy to at least review code and make some PRs to fix various things in a Helm chart for discourse if it would be developed. I’m currently a maintainer of JupyterHub’s Helm chart and have various contributions to other charts.

3 Likes
23

I think it’s actually possible / feasible to break up Discourse into a set of kube pods, but have fun sizing the resource requirements for the web and sidekiq runners :slight_smile:

If insane levels of infrastructure failure tolerance is not a business requirement for you, just use the singleton fat container, it’s going to be order of magnitude cheaper & easier for the same level of steady state service.

7 Likes
24

Hello guys!

I’m from a quite large Italian community that uses Discourse as the tool for their forum.
While we love Discourse, we’ve also observed some difficulties deploying it on Kubernetes, through some kind of “standard” procedures.

I feel the pain of the maintainers :slight_smile: I’ve been working at Open Networking Foundation for years, and one of the biggest challenges has been moving CORD -one of our biggest platforms- away from scripts-based installations to Dockerfiles/docker-compose and Docker images hosted on DockerHub + helm-charts. While there’s a reasonable fear in doing so, and if a solution that fits all the cases never really exists, I can tell we absolutely saw the benefits of it since the very first deployments.

Usually, the biggest step to take is to enter in the mindset to leave existing tools do whatever they can do best :slight_smile: Docker has installation procedures, that are standard, easy and already well-documented. Scripts are something custom, that often needs customizations and maintenance.
Also, there’s really no reason why you shouldn’t be able to use the original Docker images / helm-charts as Discourse dependencies (i.e. unicorn, postgres, redis, …).

I’d suggest to tackle the basics first (i.e. having a Dockerfile, docker-compose, and a dockerhub automated build for each component). Once this is done (with the proper software and docs modifications), the helm-charts development is the easiest thing.

I’d be very happy to help, even having a call together to discuss pros and cons of the solutions proposed above.

Along my brief research, I’ve also noticed that Bitnami (super famous in the Docker space) has come up with some images (https://github.com/bitnami/bitnami-docker-discourse#how-to-use-this-image). I was wondering if this was some separate work they’ve been doing alone, or something coordinated with you. It may be a very good starting point. If there was no partnership with them, I’d suggest it as a possible way to go. Not sure if they would do it for free, but I may be a possibility.
FYI I’ve opened an issue on their repo to know more about the work that has been done (https://github.com/bitnami/bitnami-docker-discourse/issues/132).

Please, let me know your thoughts and if I can help anyhow.

1 Like
26

Hi @Luca_Prete, did you check @pierreozoux’s work above and the team’s response?

Most of the Docker packaging work would be to match what pups is doing in the background for launcher, including some custom configuration setup for Postgres, Sidekiq, etc. @unteem scratched an itch and went through following what pups did for earlier versions of Discourse. Keeping the docker image up-to-date with the official release is challenging. It would be interesting to unfold the whole process and walk that path in a standard docker approach. If there’s a workable path towards using a standard setup I guess many people here would be super happy.

2 Likes
27

@hellekin thanks.

I have not. I’m sure there’s a lot there, but I usually tend to stay away -at least for work- from single-user (as opposed to community-supported) jobs, for the simple fact that it may become hard to maintain them later on.

The specific path really depends on some details of the platform.

What I see as a possible solution in your case would be start understanding how standard images (Postgres, Redis, …) would work with Discourse without specific customizations.
The reason why I’m considering this important is that it gives you the ability to rely -anywhere you install Discourse- on external, standard services (that could be installed on physical hardware, on a VM, on some containers, on k8s in form of chart deps, …).
Each of these services typically allows the use of some startup scripts, to create a db and so on…shouldn’t be so difficult.

Then I would create a proper Dockerfile (which automatically also becomes your quick start guide for users that wish to install Discourse without Docker).

Then comes the docker-compose.yaml (this is pretty much the same of what bitnami exposes on their github)

At this point you should be able to bring up Discourse on your laptop in form of "micro"services, using standard dep images, in few seconds, in one command, without any custom script.

Last but not least comes Kubernetes fun (few yaml files, possibly packaged in form of a helm-charts), to be published on the official, stable repo, or alternatively -at least at the beginning of the process- on your self-hosted repo.

28

@unteem is not alone :slight_smile: We work together.
We do this, because we host several discourse instances.
We started on docker, and now we run on kubernetes.

We moved our work under https://lab.libreho.st/ which is a community effort (@hellekin is also part of it ) . We want to advertise more our work in the coming weeks/months.

It is a real pain to maintain this… I literally spent hours, if not days for this commit that fixed my builds:

Any way. We are actually working on kubernetes operators. We start on Nextcloud, then RocketChat, and the next will probably be Discourse.

In the mean time, you can find the code for the docker images we use here:


The images themself are here:
https://hub.docker.com/r/libresh/discourse
https://hub.docker.com/r/libresh/discourse-web

As you can see we have been putting some time on it lately. So we have tags and pipelines. We need to add automation to have weekly builds.

We have an helm chart there:


But as you can see, it is not really maintained.

What I can say is that, it works for us :slight_smile: If you want to share the ride and you feel adventurous, feel free to join us :slight_smile: We have fun :slight_smile:

We don’t really provide support, not much time for it, but if you PR, it will be welcomed. I really wish we could do this work under official discourse umbrella, this would be so much easier.
But at the end of the day, I start to understand discourse team. They have only one tool that they support for the community, and it works well. They provide good support for not so technical users, and this is really nice. If there is a problem, git pull && rebuild, it solves 99% of the problems :slight_smile: I understand that supporting another tool is a big risk, and if not supported or not well done, it will harm the project in some ways. Again, thanks a lot discourse team for the hard work :slight_smile:
My only problem is that we are probably many to develop our own solution, and the only way to collaborate is to collaborate here upstream.

Actually one way to do it, would be to have another image in discourse_docker that would be called super_base? without runit/anacron/nginx/postgres and the base would be based on super_base, and we could reuse it for our k8s deployment? I think this would work :slight_smile:

What do you think?

Is discourse using kubernetes? I’m curious :slight_smile:

6 Likes
29

@pierreozoux thanks for the reply!

I think you guys did a great job and I’d be happy to contribute as I can :slight_smile:

I also had more time to better look at the work that the Bitnami guys did. The good news is that they’ve already separated the containers. Did you have any chance to look at it? What do you think?

Here are the Docker file used both for the web and the sidekiq containers: Docker Hub

I can’t paste the link to the docker-compose, but you can easily find it following the link I added in my previous post in the thread.

In the same repo there’s also a kubernetes yaml file: https://github.com/bitnami/bitnami-docker-discourse/blob/master/test.yaml

Starting with the Docker image, do you see any issue? Is the software “updated” enough?

What seems missing is the helm-chart, that should be done following the best-practice adopted for the stable charts.

What do you guys think? Could be worth to join the efforts and integrate your files into their work?

30

That is exact the same change made by https://github.com/discourse/discourse_docker/commit/c87937dc40abe2791a128b4463a6a5ceba181be4. If you are forking the image you may want to look into upstreaming changes from that repo concurrently.

This is pretty much our motivation for the status quo.

Adding anything that we don’t use internally just means it will break and be deprecated shortly afterwards, causing more pain to everyone who depended on it. We had multiple examples of this in the project.

If that is something the community wants to work on that is okay.

No, we don’t use k8s.

6 Likes
31

It’s just that there are a bunch of dependencies to keep up with.

People fairly regularly post here that they aren’t working in one way or another.

What I’ve been doing is using launcher to build images and push those to a repo that I then deploy with kubernetes, and I do have zero-downtime upgrades scripted. It’s huge overkill for the clients I’ve set it up for. I’m hosting 3M pageviews/month on commodity hardware with a (mostly) standard setup (just has traefik reverse-proxying multiple sites).

I have considered publishing a set of images with, say, various plugin sets, but the issue is that if there are any problems with them, they can’t get support here.

5 Likes
32

Any changes on the topic? Is there, or will there be, an official Discourse image that can be easily used for Kubernetes deployments?

33

There will not. The solution that I use is to bootstrap the new image, push it to a repo, then launch it, then do post upgrade migrations after the new pods crank up.

5 Likes
34

Any updates? Still no official Helm Chart to install Discourse to Kubernetes?

1 Like
35

Nope, and that isn’t on our roadmap either.

See Can Discourse ship frequent Docker images that do not need to be bootstrapped? for context.

36

I’ve been considering making such an offering (and it wouldn’t be “official”), but would need to make sure it didn’t make for extra support issues here and paid me for the time I spent maintaining and supporting it. I have little idea how to solve either of those problems, but if you have a budget, feel free to contact me.

1 Like
37

We are successfully hosting discourse on helm with our own docker image since some months now.
(before we were using our own image as well, with compose)

And it is with s3/minio support.

(but it definitely lack documentation, and “openess” to open accounts and contribute.
If there is demand, here or in pm, we can open :slight_smile: )

@pfaffman if you find happy people that are willing to support your time, we’d be happy to collaborate!

We are hosting discourse for some small communities, mainly in France, part of the common movement.
(Like https://forum.chatons.org/ is the biggest one we host pro bono)

6 Likes