Windows / Vagrant and (Docker) for development


(lid) #1

So here are my random thoughts, please feel free to let me know why this is a bad idea.

##Why don’t we develop on docker?

  1. Docker is the closest environment to production, well it is the production, and apprently the only form of installation supported by discourse. What is better then giving the developers a
    development environment that mimics production. Also an environment that is kept up to date and will not require additional resources for maintenance at least not as much.

####What do we have now?
We have vagrant and virtualbox combination.

####What can we have
We can still have vagrant set the virtualbox machine, but this time with docker support.

All we need is the ability to push code changes from host machine to the container.
We can use git to handle that for us.

So for source code changes, I think this configuration could work. In addition we don’t have to deal with docker container commits as the code will be actually pulled from the host (using git) . I am against docker commits because of the file system layer complexity and possible limits.


I have already started working on a solution that will bridge guest to host code sync using the git protocol.
I started this for a different reason issue, but I think that method could be used on this configuration as well.

##What are the complexities?

  1. We are basically setting up a virtual env, and then set another layer of container nesting with docker.
    I believe that will create another level of network setup complexity in the virtual machine that will need to be mitigated.

  2. We are creating 2 levels of ssh, one to the virtual machine, and 2nd to the container if needed.

3.The initial setup environment will be slightly more bloated. base image for the virtual machine, then the docker base image.


####If you think this is a good idea, let me know if you wish to join the effort of creating this beast.


Vagrant and Virtualbox = slow on Windows (update)


Building the development VM
Building the development VM
(Sam Saffron) #2

I definitely want to move to a docker based dev environment, in fact I started building bits here:

With boot2docker it would also be quite easy getting going.


(lid) #3

So some good news:

I managed to get the discourse/docker running on the Boot2Docker image ( without the boot2docker-cli at this time)

The bootstrapping failed on me a few time in the process.

First failure, due to a network issue (fail on fetching a gem).
Which I found unpleasant because, it got that far and it required a restart of the bootstrap. which I believe repeated some of the step previously applied that were time consuming, for example setting recursive permissions on the /var/www/discourse and I am pretty sure it did a full git pull on the source.

Network error while fetching
https://rubygems.org/quick/Marshal.4.8/railties-4.1.4.gemspec.rz

Failure #2
Second attempt to bootstrap, I got another error this time related to db:migrate
At this point I think it was my fault, I believe the process just ran out of memory. The virtual machine only had 512mb of ram allocated, I have restarted the machine with 2gb

/pups/lib/pups/exec_command.rb:85:in `spawn': cd /var/www/discourse && sudo -E -u discourse bundle exec rake db:migrate failed with return #<Process::Status: pid 293 exit 1> (RuntimeError)

3rd attempt = Success
After allocating more memory to the machine and most network activity completed from previous step I have got a nice little message
Successfully bootstrapped, to startup use ./launcher start app

My next step is to restart the machine and see if everything is working.


(lid) #4

How to install Discourse on Boot2Docker.


(Sam Saffron) #5

The issue though is how to adapt this so it works in dev mode and is good for development.

We need to address

  1. How do you change files, how do you create PRs?
  2. It should run in RAILS_ENV=development as opposed to production
  3. A guide on how to restart stuff when needed sv restart unicorn for example.

(lid) #6

I currently trying to integrate the source from a repository on the host machine
and having the docker container pull the code over git:// ( concept I already got working with the vagrant setup)
This way development is done on the host machine. we just need a way to make the container pull our changes.
see bullet point 3 how I plan to allow this.

  1. Is this something we need at the bootstrap phase? or is that something that can be done after the bootstrap completed?
    Do we need something like?
    $> ./launcher bootstrap-dev app

3 . I am thinking of exposing an API to help with the development workflow.

http://docker:81/api/pull
Will be invoked on commit on host (git hooks) and trigger the guest to pull changes(from host repository).
the api can be extended to control various services, restarting unicorns, pipeline rebuilds and so


(lid) #7

I think it is time to give this beast another shot.

###Whats new?

  1. I have created a prototype Sinatra server that manage the git daemon and repository on the host.

2.A Directory watcher, to automate pushing changes to guest. Whenever the developer save a change to a file. the host will commit the changes as an “autosave” commit. and trigger a pull by the guest.

3.Auto commit “autosave” commits will mess the commit log. but fear not. “Git WIP collapser” will take care of that ( side project)
Which will turn this

cf344de9 "final commit"
 |_aa33cb3 "autosave"      <-- will be collapesed into final commit
 |_ddb0042 "autosave"      <-- will be collapesed into final commit
aa1100d Initial Commit

into

ba4ddc1 "final commit"
aa1100d Initial Commit

###What stops me now?

Before I move forward and try to integrate the solution into a docker container. I need to understand better
what is expected from a development instance of discourse. and solve so issues on the way.

#1:
Can production and development environments co-exists inside a single container. where all that is needed is to change the RAILS_ENV variable?

I have tried to setup the env with development flag
bundle exec rake db:create RAILS_ENV=development
bundle exec rake db:migrate RAILS_ENV=development
and I encountered many difficulties and errors. related to permissions. and what not.

#2:
Would it be possible to run tests / specs inside the development container?
bundle exec rake test RAILS_ENV=development

#3
what is the advantage of using RAILS_ENV=development.