Deployment isn't very sysadmin-friendly

(Drewcrawford) #1

(copied from GitHub

Hi. I want to rain on your deployment parade for a minute. Sorry, I’m not trying to be rude, but I think this is really important to get right, and I had to do a redeploy today, it took awhile, there was a lot of fighting with your launcher script. I won, but somewhat unsatisfied with the parley.

So as far as making deployment good for newbies–which is a fine goal for an early-stage product–you’ve basically succeeded, or at least as close as you can without offering a hosted solution. You tell the user what OS to run, what directories things should be stored in, and everything’s fine.

The trouble is that as the product matures, more and more people with actual sysadmin/devops skills will be maintaining it, and the deployment really isn’t at all intuitive to that group.

For example, the way Docker is supposed to work is you are supposed to be able to run “docker build .” at a command line and something sensible will happen. You can’t do this with Discourse.

Now that’s not really your fault, since the Dockerfile format is ridiculously underpowered and cannot possibly do the kind of configuration processing you want to do. But in this case the answer is to get involved on issues like here here and here to get the format to a place where it can start handling these cases.

Or alternatively, the answer is to write a standard tool that could build docker images with sufficient configuration power. And then that tool could get re-used among various projects that want to package complicated things in Docker. You’re early in on the scene, I think people would listen to you.

But the answer definitely isn’t for every vendor to write their own shell script installer with their own syntax to install every package. That is so 90s.

Here is a laundry list of similar complaints:

  • Discourse assumes things about the file structure of my host system, for example that I’ll have a /var/docker/shared directory. If only there was some kind of lightweight container system, that would let me organize my host FS how I want and let each guest lay out the filesystem how it likes… Oh wait that’s how Docker is supposed to work.
  • Discourse makes me reach into some vendor-specific yml file to edit what ports will be forwarded on the host system. If only there was some vendor-independent syntax, like docker run -p, that could specify ports for any applications I run…
  • Discourse has ssh enabled by default. That’s not how Docker is supposed to work. Puzzlingly, Discourse also installs nsenter.
  • Discourse requires git to be installed on the host. If only there was some vendor-independent syntax, like docker pull, to move software across a network

I could go on, but this is getting long.

Really what I’m saying is this: Docker isn’t a private interface that you can hide behind a shell script. It’s a public interface, and one that you potentially share with other tenant applications on the machine. My job as a sysadmin is to expose ports, start and stop containers, mount volumes, and I am tired of wrestling with a shell script that thinks it knows how to be a sysadmin better than me.

(Sam Saffron) #2

This is not correct: you can place files wherever, ./launcher deals with relative paths.

Additionally nothing about the container format is locking you into location for shared stuff:

Totally open to amending ./launcher to allow for that if needed, the key is that we need to support forwarding to 80 cause that is the general config all our users use. If you want to provision a swarm on docker assigned ports, so be it, open to extending the format.

SSH support was there way before nsenter, we may deprecate the ssh template, nothing is really forcing you to mix it in.

Nothing is stopping you bootstrapping an image and then deploying it in the way you see fit. I would love to work with the Docker team to get Docker file to support mixins, replace syntax, hooks and so on to make it suitable for us, but it is not there yet, which is why we use GitHub - discourse/pups: Simple yaml based bootstrapper for Linux machines which is trivial to learn and can be reused by others.

(Drewcrawford) #3

This is still an assumption. The assumption is that it is okay to write to files on my host system. Docker supports other scenarios, for example I might want all volatile data on a volume container.

I find this hard to believe, since nsenter predates all of docker. But in any event priority does not establish correctness.

“launcher bootstrap” creates or generates an ssh key and puts it into the container; not only is this a default but it’s not even user-configurable.

One thing that is stopping me is this sentence:

I interpret this sentence to mean “The only officially supported install is to use our launcher script for build/start/stop”. That may or may not be what the author of that page intended, but I think it’s a reasonable interpolation of the statement.

It follows from that policy that whatever sort of docker munging I do is nonstandard and may break in a future release.

I quite agree, but I don’t think it would be an unreasonable addition to have some kind of “./launcher preflight” command which would (possibly leveraging pups) do all the mixins and heavy lifting and then allow the user to run docker build . in the standard way. Such a scheme is going to be a lot more compatible with standard orchestration tools that are currently being written.

(system) #4

(Jeff Atwood) #5

I am curious @drewcrawford, four years later, do you think the Dockerfile format has evolved enough and is now sufficiently powerful to do this?

(Drewcrawford) #6

No, I don’t believe so.

(Sam Saffron) #7

I think this a bit too wide, as long as you bootstrap the image using launcher, feel free to start it however you wish and tag it however you wish. That is the spirit here.

The trouble is that it can get very hairy to do this right and get all the ENV perfect, you are dealing with a sharp object. But as long as you are fine with knives … sure that is the path of least resistance to getting Discourse deployed on kubernates and so on.