Install Discourse for development using Docker

Developing using Docker

Since Discourse runs in Docker, you should be able to run Discourse directly from your source directory using a Discourse development container.

:white_check_mark: Pros: No need to install any system dependencies, no configuration needed at all for setting up a development environment quickly.

:x: Cons: Will be slightly slower than the native dev environment on Ubuntu, and much slower than a native install on MacOS.

Step 1: Install Docker

Ubuntu

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get install -y docker-ce

19.10

sudo apt-get install docker.io
sudo usermod -a -G docker $USER
sudo reboot

Windows WSL: Ubuntu

You can run the above commands within WSL but you must have all the files inside WSL file system. E.g. it works just fine if it’s inside ~/discourse but it fails if it’s placed inside /mnt/c/discourse.

MacOS

:warning: The Discourse development docker image is only available for x86_64 architectures. M1 Macs are capable of starting the image using architecture emulation, but Discourse is unlikely to boot due to the lack of inotify support in QEMU.

Instead, you should use Install Discourse on macOS for development

Option 1: Download a packaged .dmg from the Docker store
Option 2: brew install docker

Step 2: Start Container

Clone Discourse repository to your local device.

git clone https://github.com/discourse/discourse.git
cd discourse

(from your source root)

d/boot_dev --init
    # wait while:
    #   - dependencies are installed,
    #   - the database is migrated, and
    #   - an admin user is created (you'll need to interact with this)

# In one terminal:
d/rails s

# And in a separate terminal
d/ember-cli

… then open a browser on http://localhost:4200 and voila!, you should see Discourse.

Plugin Symlinks

The Docker development flow supports symlinks under the plugins/ directory, with the following caveat:

Whenever a new plugin symlink is created, the Docker container must be restarted with:

d/shutdown_dev; d/boot_dev

Notes:

  • To test emails, run MailHog :

    d/mailhog
    
  • If there are missing gems, run:

    d/bundle install
    
  • If a db migration is needed:

    d/rake db:migrate RAILS_ENV=development
    
  • When you’re done, you can choose to kill the Docker container with:

    d/shutdown_dev
    
  • Data is persisted between invocations of the container in your source root tmp/postgres directory. If for any reason you want to reset your database run:

    sudo rm -fr data
    
  • If you see errors like “permission denied while trying to connect to Docker”, Run:

    run `sudo usermod -aG docker ${USER}` 
    sudo service docker restart
    
  • If you wish to globally expose the ports from the container to the network (default off) use:

    d/boot_dev -p
    
  • The Dockerfile comes from discourse/discourse_docker on GitHub, in particular image/discourse_dev.

Running Tests

d/rake autospec

To run specific plugin tests, you can also do something like this:

d/rake plugin:spec["discourse-follow"]

Or even something like this to be even more specific:

my-machine:~/discourse$ d/shell
discourse@discourse:/src$ LOAD_PLUGINS=1 RAILS_ENV=test /src/bin/rspec plugins/discourse-follow/spec/lib/updater_spec.rb:37

This document is version controlled - suggest changes on github.

71 Likes

This is really easy to get set up! Thanks.

I’m surprised, however, by how slow the local installation is. Is there a way to adjust the container specs somehow? I’d expect a slowdown, but maybe not one so dramatic…

Thanks again.

7 Likes

I’m trying to use Docker on Windows 10, and I have a question about the guide. I have Docker installed and it can run an ubuntu container (with basically nothing installed in the container). This docker guide just says to clone the repo next, but I can’t do that since the container doesn’t have git installed.

At this point, should I just follow the Ubuntu Guide and install all of the prerequisites in the container? Or should I be using the discourse_dev image referenced on Github? It seems like this guide is skipping over a lot of steps, but I’ve never used docker before so maybe I’m missing something.

4 Likes

I think what is tripping you is that you are trying to clone the repo within the container, but the git clone should be done on your machine. So if you already have git installed locally, you are close to get it working.

3 Likes

I actually did that first, which is good. So I already have a copy of the repo on my local machine.

Do I automatically have access have my host file system from the docker container? I think I was partly confused by d/boot_dev --init because I didn’t know what the d/ part was (I don’t see a folder called d for example in the repo, even though I do see boot_dev).

3 Likes

The d is on the repo (scroll further down) — it is not a folder but a symlink to bin/docker. You should get it if you do the git clone.

I haven’t used this setup extensively myself, but my understanding is that you edit the source files directly in your host filesystem (that you have just checked out) and the docker containers mount that and run from it. The d/* scripts will to most of the docker setup work for you.

6 Likes

Thanks for that. I didn’t even think to look past the folders. I think I have enough information to make another pass at this.

3 Likes

I’m still getting something wrong. Between this topic, here, and here, nowhere just states what docker command to run. I settled on docker run -ti -v LocalRepoPath:/src discourse/discourse_dev, which initialized a whole bunch of stuff and seemingly mounted the repo to /src. But I can’t run d/boot_dev --init for some reason. Even just running ls in the repo’s top-level directory gives me:

ls: cannot access 'adminjs': Input/output error
ls: cannot access 'd': Input/output error
ls: cannot access 'jsapp': Input/output error

Does anyone know what’s going on here?

3 Likes

You might be overcomplicating this: the tutorial on the first post already has everything you need to get started.

The d/* commands will do all the docker command magic so that you don’t have to. Once you have cloned the repo, the d/boot_dev --init and the d/unicorn commands (that you do on the host machine) should be all it takes to get http://localhost:9292 working. Those scripts will call docker to generate the container and start it for you. The d/shutdown_dev will stop it.

If you are getting those errors while browsing the repo files on the host, then something else seems amiss with your filesystem. Did you run out of disk space perhaps?

4 Likes

sigh, I see. I thought those had to be run in docker. That said, d/boot_dev --init apparently doesn’t work on Windows in Powershell (which I guess is why this docker guide only mentions Mac and Ubuntu hosts). I tried to run the command in my linux subsystem (v1), but of course that fails at the docker step since I installed docker on Windows, not on the linux subsystem.


Next I tried to run it in git bash for Windows. I couldn’t run d/boot_dev --init (error “d/boot_dev: Is a directory”), but I was able to run bin/docker/boot_dev --init. This got a lot further, doing a bunch of pulls, but then it failed with

C:\Program Files\Docker\Docker\resources\bin\docker.exe: Error response from daemon: Mount denied:
The source path “PathToRepo/data/postgres;C”
doesn’t exist and is not known to Docker.

I was able to get past that (I think) by escaping the mount path, per:
Docker mounted volume adds ;C to end of windows path when translating from linux style path - Stack Overflow

After that I got this error:

C:\Program Files\Docker\Docker\resources\bin\docker.exe: Error response from daemon: OCI runtime create failed: container_linux.go:346: starting container process caused “exec: "C:/Program Files/Git/sbin/boot": stat C:/Program Files/Git/sbin/boot: no such file or directory”: unknown.

C:/Program Files/Git/sbin/boot doesn’t exist on my file system, so I guess the error’s not surprising. The only thing mentioning sbin/boot in the boot_dev file is this part of the docker command: discourse/discourse_dev:release /sbin/boot.

And that’s pretty much where I’m at right now… I don’t really know what to try at this point.

3 Likes

I’m afraid that’s how far I’m able to help you as I’m not familiar with windows.

Perhaps @rishabh is able to confirm whether this is supposed to work on windows too and update any missing steps for that (or add a note if it doesn’t).

3 Likes

@sam Is what I’m trying to do even a remotely supported way of running Discourse for development? In the Windows development thread you mentioned trying this, but it hasn’t been going great so far. Basic (non-Discourse) Ubuntu containers technically work, but not the specific commands in this thread’s OP.

You originally mentioned this to me:

So unless you have a suggestion to get me through the issues I’m having with Docker on Windows, it’s probably best for me to cut my losses and run it in an Ubuntu VM using something like Virtual Box.

2 Likes

So many of us are on the slow ring with zero issues. Games work, steam works everything works.

I do get the fear here, but it really is not a giant deal. Anyway if this is just too scary for you, I recommend a traditional VM like Hyper-V or VirtualBox. Install Docker in there and use our docker dev setup.

6 Likes

If this wasn’t my primary (and only) computer I would join you on the slow ring, but I just can’t risk it. So I’ll go with a traditional VM with docker. Thanks again for the advice. I hope to have something up and running soon.

What would be sufficient for disk space and RAM to allocate to the VM? I was thinking maybe 16GB of disk space and 4GB of RAM, but I wanted to check first.

1 Like

That would probably be enough, best way to find the ideal recipe is to experiment

6 Likes

Alright, I’m close to getting this working for real. I have a local instance running using docker in an ubuntu VM. I assume that I should only run d/boot_dev --init the very first time, and after that I’d just run d/boot_dev. Is that accurate? The reason I ask is that d/unicorn works when I include --init, but doesn’t work when I omit the --init from d/boot_dev. When it fails to start I get the error

Could not find aws-partitions-1.270.0 in any of the sources.
Run bundle install to install missing gems.

But then running bundle install fails with

actionview_precompiler-0.2.2 requires ruby version >= 2.6, which is incompatible with the current version, ruby 2.5.1p57

Is there a specific version of ruby I should update to? Or is there something else going on?


My VM is Ubuntu 18.04 (LTS) and this time I was able to directly follow the instructions in the OP, so I’m not sure where I might have gone wrong (if something is wrong).

2 Likes

You would not use bundle install you would use d/bundle install you want to install the bundle inside the container, the VM does not need Ruby.

4 Likes

Yeah, I had a feeling that was the case, just didn’t know what the right way of running it in the container was. Thanks again for the help!

2 Likes

I pulled the latest version of the repo and as usual d/unicorn tells me “Run bundle install to install missing gems.”. However, this time when I run d/bundle install I get the following:

Fetching source index from https://rubygems.org/

Retrying fetcher due to error (2/4): Bundler::HTTPError Could not fetch specs from https://rubygems.org/

Retrying fetcher due to error (3/4): Bundler::HTTPError Could not fetch specs from https://rubygems.org/

Retrying fetcher due to error (4/4): Bundler::HTTPError Could not fetch specs from https://rubygems.org/

Could not fetch specs from https://rubygems.org/

So I can’t actually run Discourse at the moment. I have no idea if this is related to the pull (directly or indirectly) or if it’s something completely unrelated. Googling for the error hasn’t been particularly helpful, so I don’t know how to proceed.

1 Like

We you able to reach rubygems.org?

1 Like