Can Discourse ship frequent Docker images that do not need to be bootstrapped?

You’re not wrong, but ideally, the reason that I’m here talking about it is that its obviously not something that is working in my use case in its current state. If I personally had the ability to do all of the things that are being done using your software I would be doing that instead of whining about it to you people.

I assumed that if I came up with a reasonably well thought through response explaining how the other technology could potentially alleviate many of these issues could be beneficial to this project.

Truthfully I would love to do this myself and then submit a solution but the reason I’m even looking at Discourse isn’t so that I can start working for Discourse. I’m looking at Discourse because I want a discussion board that will suit my communities needs better than XenForo.

We don’t sell support, we sell hosting. So there’s that.

Switching to docker-compose has real costs (in engineering time), for very little benefit. Every time someone comes along and says “you should use docker-compose!”, we reply “show us how!” and then things go… eerily quiet. It’s almost like launcher does things that can’t be done with docker-compose… the best “well thought through response” would be a working demonstration of a launcher-less alternative.

If you have problems with any of the documentation here, please tell us about it in the relevant topic(s) so it can be improved. Tearing down everything and starting again just because a howto needs a tweak seems like an incredibly poor use of everyone’s time.

8 Likes

I do not know how any reasonable person could read this topic and feel further “discussion” was necessary.

If you would like to prove us wrong then by all means, do the work and make it whatever you want it to be. Otherwise, with all due respect, you are wasting our time.

Go ahead. Show us how wrong we are. Really rub our noses in it with your implementation. I’ll be waiting to see your results.

1 Like

I see their responses as professional. The response you got was clear and concise. It leaves no ambiguity or wriggle room which is precisely what I’d expect of a professional response.

Perhaps you read that response as heavily loaded against you personally or too dismissive of your argument. I read it as a simple statement. That is why I think you should have left out the second paragraph. It reeks of pique and wastes an “ultimately” on a chain of less than conclusive probabilities:

if I am able …
maybe, one day, I might …
if this is …
if I were …

P.S. I wrote this before seeing codinghorror’s response and edit.

1 Like

I wasn’t intending to sound irritated but it is rather frustrating that I have asked several times for clarification throughout the past year on different occasions but have received no clarification. I just think that there can be many ways to simplify the whole process which would only increase adoption unless that’s not the goals here.

@codinghorror believe me I would love to. I’m not trying to waste your time but I’ve asked on several occasions for assistance or clarifications on documentation on different problems that were created by the launcher.

Also, I have zero interest in “rubbing your noses in anything” all that I was suggesting was that there may be a vastly simpler methodology that would solve certain ambiguities that seem to pop up frequently. If anything if I had the time available, which I am fully aware that you are all busy also, I would gladly contribute. Since I don’t have that time nor do I have time to pick apart the entire application itself to understand all of the moving parts which all seem like they could be reasonably easily converted to the industry standard so it would be easier to deploy to things like Kubernetes or RancherOS or other misc docker solutions without needing to employ the use of a private registry just to ship the container in a pre-made fashion.

I really don’t see how it’s unreasonable to re-hash old ideas especially after many new developments have arisen.

1 Like

I fail to see how docker-compose would solve that problem – it won’t magically allow two processes to bind to the same port. Either way, you will need to set up a proxy that can route incoming requests.

The #howto could certainly benefit from a section on how to proceed if Discourse is not already set up, but I currently don’t have the time to describe that properly. A short version boils down to this:

  • Follow normal instructions, stopping just before executing discourse-setup.
  • Copy /var/discourse/samples/standalone.yml to /var/discourse/containers/app.yml.
  • Fill in the necessary settings in /var/discourse/containers/app.yml. Be careful, you cannot test this yet!
  • Then continue with the howto, beginning with installing Nginx.
2 Likes

What I find particularly striking about this entire discussion is that you appear to have never read through discourse_docker.

Looking at this file, is it really that hard to figure out how to change exposed ports? In fact our line there looks exactly the same as the ports line in compose.

The whole narrative here is, “I want to do super advanced things with docker, I already know how to cope with docker compose, don’t make me learn something new”

99% of our free installers just spin up a digital ocean instance and run the setup. They do not need to learn anything.

They do not need to learn about docker compose or figure out how to install it.

They don’t even need to figure out how to install docker.

So you are here, making a lot of noise about how we are not catering right for the 1% of free installers who both want to do something advanced and do not want to learn anything new. So, I am not sure I want to be in the business of helping these people.

As I said many times, Discourse docker can deploy fine in a kubernates or what have you. You just need to understand.

  • We ship a “naked fat image”

  • We have a custom process that runs docker build on that image adding in specific information that is only relevant to you, like the specific plugins you run.

  • The same step compiles our assets (cause plugins may introduce new ones) and migrates the database

  • At the end of this bootstrap you have an real docker image you can deploy anywhere that is capable of running Discourse.

The whole pain point I am trying to address here has almost nothing to do with particular tech.

Pain points I am interested in solving long term are

  • Faster bootstrap
  • Easier and more frequent base image updates
  • More consistent update process from web upgrader vs a full rebuild
  • Probably move default setup to 2 containers vs 1 container
8 Likes

./discourse-setup is for people who don’t know what nano is (and would never be able to figure out how to quit vim). It seems that frequently people who don’t need ./discourse-setup fail to realize that it’s not magic.

Perhaps install-CLOUD should include a section like:

For advanced system administrators:

If you want to take matters into your own hands rather than trust a script with limited flexibility, you might do something like this:

cp samples/standalone.yml containers/app.yml
[nano | vim ] containers/app.yml
# edit the file as indicated by the comments in the file
./launcher bootstrap app

Would that help, @chamunks ?

8 Likes

Thank you for your thoughtful response I appreciate your analysis on the matter.

Yes, I think you’re on exactly the right track. The fact that you say, “fail to realize” in your reply in regards to a tutorial for helping people “Set up Discourse in the cloud in under 30 minutes with zero knowledge of Rails or Linux shell.” which would be the logical baseline to start poking around, is concerning to me for people looking to adopt the technology.

I’m just concerned about the jump between the 30-minute Beginner Install Guide and the Advanced Docker install guide there’s also no clear disclaimer anywhere that explains what it’s essentially doing for you. The fact that I’m even here in the first place should be evidence that it’s not quite there yet.

@fefrei I also appreciate your response greatly thank you for your helpful input as well.

The docker-compose situation is not supposed to offer a solution to the port binding issues I can appreciate that the discourse-setup is intended for helping people who don’t care about learning Linux and just want to magic into place a discourse install.

Ultimately my concerns are just for documentation and knowledge assumption.

It would be ideal, yes, that the howto would include some clarification or even more beneficial a step by step to get you running on a recommended basic working setup that people can fork off from and customize once they get the base concepts down on what will and won’t work.

I did actually end up just changing the file itself to skip those ports only to have it fail for a completely different reason which I’m probably going to need to open a discussion about.

My point was to either suggest that maybe a docker-compose file might alleviate some of the responsibility from your dev team in the future. The other side of it was to see if someone could suggest some clarity in the documentation or scripts themselves.

Maybe the 1% would be a larger percent if it was clearer your mentality on this situation you suggested is appalling. Making assumptions that just because of it currently being a small demographic, that it could never be a larger one and that it would be a waste of time. To even attempt to open your platform up to other people you would never have expected to be your demographic. Sometimes the biggest innovations and coolest implementations of your technologies come from the unlikeliest of sources.

Goodness, if you look at contemporary artists most of these people aren’t all experts on the technologies or mediums that they use in their art and that’s in part how they make something more amazing than anyone else has thought of before. I apologize if I sound like I’m nitpicking here but it’s just frustrating to me when people think this way and I’ll keep my opinions of this to myself from now forward.

If or when I end up with this thing installed live I’ll see what types of suggestions I could potentially make for a code to make it go. Also, I’m not trying to walk in here guns a waving saying change all the things! I’m just saying that it might not be the worst thing in the world to try switching things to Env vars or even just have a script that bootstraps inside the discourse official images that insert the Env vars.

I doubt it. It’s crazy difficult to install docker-compose. Though it gets installed if you install Docker on Mac or Windows, for some reason, though they have a repo that will install Docker, if you want docker-compose, you have to download a shell script on your own. It’s like they don’t want you to use it in Linux. That fact aside, it’s not clear that people necessarily understand docker-compose. I’ve tried several times, and still find it no less obfuscated than the yml file that configures Discourse.

But, really, if there had been a little text that suggested you edit the yml file by hand, would you have been able to move on and follow the Discourse-with-other-web-server instructions?

1 Like

I was trying to fuddle along that way before I came in here in the first place asking questions and trying to make helpful suggestions. The last time I posted about this was some time early last year I believe and basically, the whole problem fell on deaf ears and I got no response at all.

I remember before there was the discourse-setup script there was documentation on how to do it just using the launcher if I recall correctly, forgive me if my recollection is a bit off its been a while. I managed to easily get a multi-container setup running using the launcher. I ended up having to write my own wrapper script to automate some of it so an upgrade would be easier and things like that ( a bit meta I know ) but it all came together rather nicely.

The reason I was making the suggestions was that if this is still a problem one year later it might be ideal to offload some of the documentation work.

There should maybe be a from Zero to Hero kind of guide written into the Advanced guide or something along those lines which might help to get a working base that isn’t the standalone model only and also one that is.

1 Like

@mpalmer, if suitable, please merge the following thread here. I was thinking of posting here in the first place, but I thought that this thread is already loaded with problem descriptions and justifications, so a potential solution perhaps would be better suited in a separate thread.

How does any of this address:

I would add to this:

  1. cd /var/discourse && ./launcher rebuild app is a scourge for our free installers hate. We want a web UI for that.

How does any of the multi FROM image build fanciness move the needle for 2 or 3 or 4?

It simply does not.

It allows us to cut out a few shell scripts, possibly.

If anyone really wants to help here, there are specifics that can be worked on.

A non-brainer that keeps popping up OVER AND OVER AND OVER again is :rage: yaml the worst config format in the world, hated by humans ever since it was invented. Dear docker compose fans, docker compose is yaml

Want to make make a difference that will help lots of people, add optional toml support to launcher/pups. That would be huge. ENORMOUS. Imagine if all the effort that went into this topic went to building a toml patch? We would have toml today.

Second area that would be interesting is experimentation with a “admin/upgrade web container” that is separate to Discourse. That way we throw away the existing web updater and replace it with something that talks directly to docker. Lots and lots of details here but it is 100% clear that it is better to have 1 way to upgrade Discourse.

5 Likes

If you don’t already have something in mind, I’ve seen a few solutions to having an admin docker container, that mounts the docker socket to interact with. Specifically it was for service discovery registration through docker, but I don’t see why it wouldn’t work here either.
https://gliderlabs.com/registrator/latest/

I proposed a solution over here. (Just make discourse-setup or discourse-multi-setup do that.).

It’s not what you’re talking about, but I’m getting close to having a Discourse installer web app that, given Digital Ocean and Mailgun API keys, configures everything, installs Discourse, and tells you what DNS records to set. Once it’s deployed, I’ll start adding features like rebuilds, plugin management, and so on.

Yeah, we’ve got a bunch of stuff in our infrastructure that talks directly to docker. We even ran registrator for a while, until consul turned out to be a dog’s breakfast.

It’s a big pile of work, though, to build something that operates as a one-stop frontend proxy to everything Discourse-related (because in order to have a web-based control panel that you can access via the same address, you need to have one place listening which then proxies elsewhere) and then passes requests to the “manager” container, Discourse itself, etc etc etc as required. Big job. Yuuuuuge.

4 Likes

I figured you guys had something planned – I’m definitely looking forward to seeing what you guys come up with. Obviously balancing security + accessibility on things like this will be an immense undertaking.

Disheartened to hear that consul+registrator didn’t work out for you guys though; from the onset, it looked fairly elegant as a docker solution. I’m fond of consul, but I’m also at a place with fairly static servers, and that’s less docker-centric, so I’m not actually fighting with it all the time.

(Possibly veering a little off the topic here…)

Yeah, I was heartbroken when consul turned out to be a network pig at scale; it seemed like such a good solution. Registrator was always a bit of a love-hate thing; it really did not want to support IPv6, and occasionally decided it would duplicate registrations under different names and then not clean up after itself. Nothing significant, but enough to make me grumble.

5 Likes

In case this has been missed, the new docker version includes a multi-stage feature, specifically built to handle build steps in a better way than before. I’ll (have to) leave it to the experts to assess whether this helps in any way to solve (at least part of) the issue at hand. See the release announcement blog post linked below:

5 Likes