Running Discourse on Docker for Mac


#1

Hi!

I’m trying to install Discourse locally using the relatively new Docker for Mac.

This is the error I’m getting while trying to rebuild for the first time:

/launcher rebuild app
Ensuring launcher is up to date
Fetching origin
Launcher is up-to-date
cd /pups && git pull && /pups/bin/pups --stdin
/usr/local/bin/docker: Error response from daemon: Mounts denied: er.com/docker-for-mac/osxfs/#namespaces for more info.
.
r/discourse/shared/standalone/log/var-log
are not shared from OS X and are not known to Docker.
You can configure shared paths from Docker -> Preferences... -> File Sharing.
See https://docs.dock.
888ba058ad2b2c2e797e1d298183472aa7c75f29deb2344a0ffec9e7d987927f
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one

Maybe I need to share some folder first? Is anyone trying to run it like this?

tks!


(Sam Saffron) #2

If you want to run it locally for dev, look at the /bin/docker dir, it contains some very alpha stuff I built for developing with docker images


(Jay Pfaffman) #3

If you want to try out discourse, for a few dimes a day you can run it on digital ocean. Otherwise,you are probably better off following the development guide for Mac.


(Jared Reisinger) #4

I’m trying out the alpha stuff… @sam, is this something you think you’d like to move towards (and away from Vagrant)? I haven’t gotten everything up and running yet (the images on Docker hub seem to have an out-of-date version of Ruby that doesn’t understand the <<~ syntax), but when I figure things out I can create a PR.


(Sam Saffron) #5

Yes, long term this is a far better approach

Will see if I can update the image


(Jared Reisinger) #6

Woo! Got it working enough that it’s now just (from my local Discourse source root directory):

./bin/docker/boot_dev
./bin/docker/bundle install
./bin/docker/rake db:migrate
./bin/docker/rake admin:create
./bin/docker/rails s

… and I’m running Discourse at localhost:3000! I don’t need ruby, postgress, or redis installed… it’s all coming from the container.

In order to make this work, I did make a couple of changes in discourse_docker:

  1. Moved the discourse user creation and /var/www directory creation from the discourse Dockerfile into base, for reasons that will become clean in #3.

  2. Updated the ImageMagic dependency from 6.9.5-8 to 6.9.5-9. (The -8 version has disappeared from imagemagick.org… perhaps this is something that needs to be vendored?)

  3. Changed the discourse_dev Dockerfile to come from base, rather than discourse, so that it doesn’t carry the weight of a full Discourse installation only to replace it with local sources. (I bypassed discourse_fast_switch simply because I don’t quite understand what it’s doing. Discourse appears to need ruby 2.3.0, and that seemed to deal with 2.0.0/2.2.0.)

  4. Gave the discourse user NOPASSWD sudoer’s permissions in the discourse_dev image. (I was running into problems with ./bin/docker/bundle install, and this was an expedient fix… and for a development container seems not-so-risky.)

  5. Under the assumption that a development environment should be fairly self-contained, removed the step that moved /shared/postgres_data out of the way. Postgres had permissions/db-creation issues when I tried mounting a local data directory into place. The downside is that the postgres data now lives completely inside the container, and ./bin/docker/shutdown_dev will cause you to lose everything. I’m personally okay with this for now, but will keep playing with ways to make this work better (e.g. give you the option to mount a volume and persist data across invocations).

  6. Replaced discourse_dev’s postgres.template.yml and redis.template.yml with the ones from the main templates directory (the ones that the “standlaone” app description use), changing the postgres DB name from discourse to discourse_development. (Ideally, I’d like to see that as a programmatic step, rather than a manual copy, so that the Docker images are auto-updated with each build.) This ensured that the database was set up like Discourse expected.

Right now, the Docker image build is handled by build.rb, but I’m really, really tempted to drive this via Makefile… but partly that’s just because I understand Makefiles better than I understand ruby. :smile:

Also… when using this on my machine, the ember-data-source pulled in by ./bin/docker/bundle install was 2.3.0.beta.5, which seems to be missing its dist directory. Updating Discourse’s ruby dependencies seeemed well beyond my pay grade (and experience), so I just manually patched my running discourse_dev container with locally-built dist files. It worked, but that’s clearly not a tenable long-term solution. Sadly the “small” step from 2.3.0.beta.5 to 2.3.0 proper forces ember-source minimum dependency to go from 1.8 to 2.0… and I kept getting an “empty” home page after that. :sadpanda:


(Jared Reisinger) #7

The first part of these changes (the discourse_docker repo ones) are now available as a PR: Improve image building (primarily for discourse_dev) by JaredReisinger · Pull Request #292 · discourse/discourse_docker · GitHub. Any feedback/discussion is greatly appreciated.


(Jared Reisinger) #8

And the second part (the discourse proper, bin/docker command-line tools) are also available as a PR: Improve the "develop inside Docker" experience by JaredReisinger · Pull Request #4450 · discourse/discourse · GitHub.

My favorite part about all of this is how easy it made development (from the README):

Step-by-step

It should be as easy as (from your source root):

./bin/docker/boot_dev --init
    # wait while:
    #   - dependencies are installed,
    #   - the database is migrated, and
    #   - an admin user is created (you'll need to interact with this)
./bin/docker/rails s

Yup… two commands: one to start the container, and one to kick off Discourse. (Although full disclosure… there’s still that caveat about the ember-data-source gem, but even that has a boot_dev command-line helper.)


(Sam Saffron) #9

Very nice work, give me a bit to review it but generally looks real good.


(Sam Saffron) #10

Hi @JaredReisinger we are all merged and up to date.

I wonder if someone could try your instructions out to see how everything is working.


(Chromium) #12

Hi guys (@sam & @JaredReisinger)! I’m trying to get dev workflow with docker.

I built discourse_dev image:

cd discourse_docker
cp templates/redis.template.yml image/discourse_dev/
cp templates/postgres.template.yml image/discourse_dev/
cd image/discourse_dev/
docker build -t discourse_dev .

run bootstrap:

cd discourse
./bin/docker/boot_dev -i

and stuck with Postgresql access rights:

Couldn't create database for {"prepared_statements"=>false, "adapter"=>"postgresql", "database"=>"discourse_development", "min_messages"=>"warning", "pool"=>5, "timeout"=>5000, "host_names"=>["localhost"]}

So I suspect that I’m doing something wrong from the beginning.

Any simple quick setup steps that we can convert later to DEVELOPMENT-OSX-DOCKER.md ?


(Joel Pickup) #13

I am trying to use this approach, but when running ./bin/docker/boot_dev --init I get the error

Unable to find image 'discourse/discourse_dev:latest' locally
Pulling repository docker.io/discourse/discourse_dev
docker: Error: image discourse/discourse_dev:latest not found.

Any help?
Thanks


(Rafael dos Santos Silva) #14

I’ll push a new dev image today, cooking it right now.


(Andrew Waugh) #15

I’m getting the same error message with docker for windows (both the main and the toolbox version of docker for windows). Perhaps the issue is with non Linux docker engines?


(Twlz0ne) #16

Try this:
http://stackoverflow.com/a/41381813/2639425

It works fine for me,fowllowing is my configuration:

⋊> grep -A 6 '^volumes:' /var/discourse/containers/app.yml
volumes:
  - volume:
      host: /Applications/DockerShareFolder/discourse/shared/standalone
      guest: /shared
  - volume:
      host: /Applications/DockerShareFolder/discourse/shared/standalone/log/var-log
      guest: /var/log

(Sam Saffron) #17

Following:

Doing dev work in docker on a mac may be workable!

Rails Boot Slowdown
Host 5.64
Virtual Box 6.15 9%
Virtual Box + Docker 8.78 55%
Docker Mac (cached) 11.2 98%
Docker Mac (uncached) 15.7 178%

Note: for Docker setup on virtual box has gems are in AUFS and working directory in a mounted volume. On mac the working directory is a cache mounted volume. It is likely performance would improve if overlay was used or gems are mounted on the host to avoid virtualized file system.

Docker for Mac … uncached:

sams-MBP:discourse sam$ time d/rails r 'puts'


real	0m19.035s
user	0m0.016s
sys	0m0.014s
sams-MBP:discourse sam$ time d/rails r 'puts'


real	0m15.702s
user	0m0.017s
sys	0m0.014s
sams-MBP:discourse sam$ time d/rails r 'puts'


real	0m17.290s
user	0m0.016s
sys	0m0.011s

Docker for mac … cached

sams-MBP:discourse sam$ time d/rails r 'puts'


real	0m11.430s
user	0m0.018s
sys	0m0.012s
sams-MBP:discourse sam$ time d/rails r 'puts'


real	0m11.293s
user	0m0.019s
sys	0m0.012s
sams-MBP:discourse sam$ time d/rails r 'puts'


real	0m11.995s
user	0m0.017s
sys	0m0.012s

Ruby installed on host

sams-MBP:discourse sam$ time bin/rails r 'puts'


real	0m7.204s
user	0m4.541s
sys	0m1.780s
sams-MBP:discourse sam$ time bin/rails r 'puts'


real	0m5.644s
user	0m4.319s
sys	0m1.254s
sams-MBP:discourse sam$ time bin/rails r 'puts'


real	0m5.646s
user	0m4.329s
sys	0m1.256s

VirtualBox + docker on the same host

sam@ubuntu:~/Source/discourse$ time d/rails r 'puts'


real	0m12.519s
user	0m0.016s
sys	0m0.012s
sam@ubuntu:~/Source/discourse$ time d/rails r 'puts'


real	0m8.786s
user	0m0.016s
sys	0m0.000s
sam@ubuntu:~/Source/discourse$ time d/rails r 'puts'


real	0m8.733s
user	0m0.008s
sys	0m0.004s

VirtualBox, no docker

real	0m9.386s
user	0m5.332s
sys	0m1.692s
sam@ubuntu:~/Source/discourse$ time bin/rails r 'puts'


real	0m6.032s
user	0m4.796s
sys	0m1.116s
sam@ubuntu:~/Source/discourse$ time bin/rails r 'puts'


real	0m6.155s
user	0m4.924s
sys	0m1.104s
sam@ubuntu:~/Source/discourse$ time bin/rails r 'puts'


real	0m6.112s
user	0m4.804s
sys	0m1.180s


(Sam Saffron) #18

This is looking like the most promising solution for Mac/windows dev

Going to test it out

http://docker-sync.io


Docker setup for development
(DiscourseMetrics.com) #19

Did docker-sync improve performance @sam? (or anyone else?) :slight_smile:


(Sam Saffron) #20

docker sync should improve perf significantly, I have not measured it cause I use Linux.


(DiscourseMetrics.com) #21

I’m struggling to understand how to set up docker-sync to develop on a mac.

I’ve followed the setup guide at discourse/bin/docker at master · discourse/discourse · GitHub to set up docker - which works fine, just veeery slowly.

A couple of questions:

  1. To setup docker-sync, I have to create at least one docker-sync.yml file

    In this file I have to select which directories should sync - but I’m not sure which?

  2. To start docker-sync, I run docker-sync start, but do I run that as well as ./bin/docker/rails s?

Appreciate tips if anyone out there has this set up already :slight_smile: