Create docker image for a customized discourse


(Prasoon Shukla) #1

Hello discourse devs

I’ve been working on a project for which I had to make a few changes to the development version of discourse. Now, I need to package it to a docker image. How do I make this custom version work with docker? From what little documentation I’ve read, I suspect it’ll be a matter of changing a few Dockerfiles to point at correct github URLs but well, I don’t really know.

Please let me know how to do this. Thanks.

@techAPJ, @codinghorror: I hope you guys can help :smiley:


How can I keep my database on separate machine using docker installer?
(Arpit Jalan) #2

@sam: I think you will be able to answer this better.


(Sam Saffron) #3

You would add an after code hook that nukes /var/www/discourse and places your custom discourse there.


(Kane York) #4

Pretty sure you can just checkout a different branch in git rather than destroying the directory.


(Sam Saffron) #5

btw, you also have to create the pid and tmp directory if you nuke.


(Prasoon Shukla) #6

@sam: Can’t I just change the pull directive in the Dockerfile (in /image/discourse) and pull the custom version of Discourse using git? I think I could just create a new container - like app.yaml already is - and the next time I bootstrap and run the (new) container, it would just pull the (custom) discourse version from my github repository.

Also, if I can’t do that, then I’m at a loss. I barely know what docker is - where do I add the hook? In the launcher? Or in the Dockerfile? (I’m assuming it’s the latter since the actual docker folder is a convoluted path containing the cid of the container.)

You must forgive me but again, how? Using the launcher? The Dockerfile?

I’ve never used Docker - it’s like learning git all over again - so you’ll need to be more specific.

Anyway, I think that the solution I mentioned might work. Let me know if it does. If not, please can you be more specific in you instructions as to how I destroy the directory and pull my custom version in /var/www/discourse? And how do I create the pid and tmp folder afterwards?

Sorry for the bother and thanks for all the help. :smile:


(Prasoon Shukla) #7

Uh, guys? Any answers?


(Sam Saffron) #8

You would need to create a brand new base image for that and then specify it, its more work.

I would just go the “editing yaml” file route as its simpler.


(Prasoon Shukla) #9

Hi again

So, I decided to edit the Dockerfiles. I replaced the default cloning git repos with my own repository. That didn’t work. When I run the launcher to bootstrap a container, the log messages show that it is still cloning from the official discourse repo.

So, I tried something else. Once the container was bootstrapped, I did an ssh using the launcher into the container, then moved into /var/www/discourse, added a git remote to my repository and did a fetch and checkout to the branch in which I’ve done my work. Then, I restarted nginx.

Result: 502, Bad Gateway. :cry:

I even restarted the container, still no luck.

Any idea why this might be happening? I’m getting desperate here. I’ve hardly made any real changes - just a few added CSS and one JS file + a few lines in the default layout. That’s it.

Anyway, here’s the appropriate part of the unicorn logs. Looks like a problem with, image_sorcery, though my version of discourse isn’t using it.

E, [2014-06-24T18:37:47.021280 #58] ERROR -- : reaped #<Process::Status: pid 8169 exit 1> worker=unknown
Detected dead worker 8169, restarting...
Loading Sidekiq in process id 8186
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.1/lib/active_support/dependencies.rb:247:in `require': No such file to load -- image_sorcery (LoadError)
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.1/lib/active_support/dependencies.rb:247:in `block in require'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.1/lib/active_support/dependencies.rb:232:in `load_dependency'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.1/lib/active_support/dependencies.rb:247:in `require'
        from /var/www/discourse/app/jobs/regular/generate_avatars.rb:1:in `<top (required)>'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.1/lib/active_support/dependencies.rb:247:in `require'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.1/lib/active_support/dependencies.rb:247:in `block in require'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.1/lib/active_support/dependencies.rb:232:in `load_dependency'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.1/lib/active_support/dependencies.rb:247:in `require'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.1/lib/active_support/dependencies.rb:348:in `require_or_load'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.1/lib/active_support/dependencies.rb:307:in `depend_on'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.1.1/lib/active_support/dependencies.rb:225:in `require_dependency'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/railties-4.1.1/lib/rails/engine.rb:468:in `block (2 levels) in eager_load!'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/railties-4.1.1/lib/rails/engine.rb:467:in `each'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/railties-4.1.1/lib/rails/engine.rb:467:in `block in eager_load!'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/railties-4.1.1/lib/rails/engine.rb:465:in `each'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/railties-4.1.1/lib/rails/engine.rb:465:in `eager_load!'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/sidekiq-3.1.3/lib/sidekiq/cli.rb:212:in `boot_system'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/sidekiq-3.1.3/lib/sidekiq/cli.rb:49:in `run'
        from /var/www/discourse/lib/demon/sidekiq.rb:31:in `after_fork'
        from /var/www/discourse/lib/demon/base.rb:93:in `run'
        from /var/www/discourse/lib/demon/base.rb:82:in `start'
        from /var/www/discourse/lib/demon/base.rb:68:in `ensure_running'
        from /var/www/discourse/lib/demon/base.rb:30:in `block in ensure_running'
        from /var/www/discourse/lib/demon/base.rb:29:in `each'
        from /var/www/discourse/lib/demon/base.rb:29:in `ensure_running'
        from config/unicorn.conf.rb:100:in `master_sleep'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/unicorn-4.8.2/lib/unicorn/http_server.rb:291:in `join'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/unicorn-4.8.2/bin/unicorn:126:in `<top (required)>'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/bin/unicorn:23:in `load'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/bin/unicorn:23:in `<main>'

Here are the nginx logs:

2014/06/24 18:43:22 [error] 48#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 42.104.24.157, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "54.187.40.150"
2014/06/24 18:43:25 [error] 48#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 42.104.24.157, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "54.187.40.150"
2014/06/24 18:43:27 [error] 48#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 42.104.24.157, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "54.187.40.150"
2014/06/24 18:43:51 [error] 48#0: *10 connect() failed (111: Connection refused) while connecting to upstream, client: 172.17.42.1, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "localhost"
2014/06/24 18:45:53 [error] 48#0: *14 connect() failed (111: Connection refused) while connecting to upstream, client: 42.104.24.157, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "54.187.40.150"
2014/06/24 18:54:50 [error] 48#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.17.42.1, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "localhost"
2014/06/24 18:54:53 [error] 48#0: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 172.17.42.1, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "localhost"

Here, 172.17.42.1 is a private IP address from which I tried accessing the site. The site itself is currently hosted at 54.187.40.150

Any help is much appreciated!


(Kane York) #10

You need to edit the pups templates, not the Dockerfiles. Try making new files in the /var/docker/templates folder (note that you will not receive template updates if you do this).


(Pahlevi Fikri Auliya) #11

I think you can mimic the following logic inside templates/web.template.tml

  - exec:
      cd: $home
      hook: code
      cmd:
        - git reset --hard
        - git clean -f
        - git remote set-branches --add origin master
        - git pull
        - git fetch origin $version
        - git checkout $version
        - mkdir -p tmp/pids
        - mkdir -p tmp/sockets
        - touch tmp/.gitkeep
        - mkdir -p                    /shared/log/rails
        - bash -c "touch -a           /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr}.log"
        - bash -c "ln    -s           /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr}.log $home/log"
        - bash -c "mkdir -p           /shared/{uploads,backups}"
        - bash -c "ln    -s           /shared/{uploads,backups} $home/public"
        - chown -R discourse:www-data /shared/log/rails /shared/uploads /shared/backups

(Pahlevi Fikri Auliya) #12

I manage to do it by making a copy of templates/web.template.yml and modify the clone to:

  1. Add new git remote pointing to my repo
  2. Pull from that remote instead of the default repo.

Here are the modification:

root@discourse:/var/discourse# diff templates/fikri.web.template.yml templates/web.template.yml
18c18
<   version: my-version
---
>   version: tests-passed
61d60
<         - git remote add origin-fikri https://github.com/fikriauliya/discourse.git
64c63
<         - git remote set-branches --add origin-fikri master
---
>         - git remote set-branches --add origin master
66c65
<         - git fetch origin-fikri $version
---
>         - git fetch origin $version

Also change containers/app.yml to point to the new template:

root@discourse:/var/discourse# diff containers/app.yml containers/app_backup.yml
17c17
<   - "templates/fikri.web.template.yml"
---
>   - "templates/web.template.yml"

(Love Chopra ) #13

But when I try to go to cd /var/www/discourse directory then getting error “-bash: cd: /var/www/discourse: No such file or directory”

Please let me know how to access /var/www/discourse directory ?


(Love Chopra ) #14

@sam request you to please share a sample code snipped for this purpose which we can use and add to our app.yml file. Thanks


(Love Chopra ) #15

@Pahlevi_Fikri_Auliya I tried cloning and modifying the web.template.yml file as per your guidlines and placed my custom repo path but while bootstrapping it giving me error

FAILED
--------------------
RuntimeError: cd /var/www/discourse && git fetch origin-mcg tests-passed failed with return #<Process::Status: pid 272 exit 128>
Location of failure: /pups/lib/pups/exec_command.rb:105:in `spawn'
exec failed with the params {"cd"=>"$home", "hook"=>"code", "cmd"=>["git reset --hard", "git clean -f", "git remote add origin-mcg https://github.com/lovechopra1/discourse.git", "git remote set-branches --add origin-mcg master", "git pull", "git fetch origin-mcg $version", "git checkout $version", "mkdir -p tmp/pids", "mkdir -p tmp/sockets", "touch tmp/.gitkeep", "mkdir -p                    /shared/log/rails", "bash -c \"touch -a           /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr}.log\"", "bash -c \"ln    -s           /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr}.log $home/log\"", "bash -c \"mkdir -p           /shared/{uploads,backups}\"", "bash -c \"ln    -s           /shared/{uploads,backups} $home/public\"", "chown -R discourse:www-data /shared/log/rails /shared/uploads /shared/backups"]}
25802827196a41a4fc30fcbf20dfe32115204f2c8393d321d0af007453a604ae
FAILED TO BOOTSTRAP

Please help what wrong I am doing here


(Nilesh Trivedi) #16

Can you post your this part of web.template.yml where all these git commands are written?


(Love Chopra ) #17

Thanks @nileshtrivedi
Now I have manage to run it fine, there was an issue the way I wrote my commands


(Ben Maher) #18
https://meta.discourse.org/t/how-to-install-my-customized-repo-of-discourse-on-production-server/23364/5?u=mindriot&source_topic_id=16683

(m0g) #19

Hi @lovechopra1

I have exactly the same error as you. Could you maybe explain what was the issue in the commands you wrote?

Much obliged


(Nikita Zonov) #20

In my mind all I have to do it’s change git reference in /var/www/discourse as already discussed.
And as I’ve found this reference is comited in the docker image samsaffron/discourse:1.0.8 which hardcoded in the Launcher on the 15 string.
image=samsaffron/discourse:1.0.8
So I’ve create my own docker image with my Repo in the /var/www/discourse from samsaffron/discourse:1.0.8. And set my image in the launcher.
image=myaccount/myimage:tag
also i’ve added one command to the web.template.yml before 84 string

  • mkdir -p $home/log
    otherwise there was an error during bootstrap