Iterating over multisite.yml in the data container

(eriko) #1

So I have modified the pups files for web_only.yml to create and share multisite.yml in /var/shared so that i can access it in the container created by data.yml. My idea is to have the bootstrap in data.yml create missing databases in in multisite.yml to make adding more sites in the container in created by web_only.yml.

So my question is it possible to do this with pups or should I add a small script to do this work?

(Sam Saffron) #2

You are going to need a small script, pups is purposely limited to simple replaces and execs, though you could specify the file in the template and then execute it

(eriko) #3

Ok thanks. That was the feeling I got reading the examples and code. In some ways that will make it cleaner. If I come up with something clean I will post it.


(Kane York) #4

Try a file: with chmod: +x that contains your script, followed by an exec: block.

(eriko) #5

Ok here is what I have so far. I am using data.yml and web_only.yml as skeletons. I am assuming that the stock versions were used to set things up and now I want to convert to multi site.

In data.yml I have replaced the hooks section with following. It creates a script called multisite (written in ruby) and makes it executable. This script reads a ln -s version of multisite.yml that gets created in web_only.yml. This link is manual at this point as I am not sure how to link in to another containers shared area. The script is then executed on the multisite.yml files and reads the config for each site. It then tries to create each db and if it does runs the ownership and extension stuff. It also handles different users owning each db (though I do not know if discourse handles this)

  - file:
     path: /shared/multisite
     chmod: "+x"
     contents: |
       #!/usr/bin/env ruby
       require 'psych'
       puts "in multisite"
       multisite = "/shared/multisite.yml"
       sites = Psych.load_file(multisite)
       sites.each do |site,config|
         if system("createdb #{config['database']}")
           if system("createuser #{config['username']}")
             #If the user is newly created set the password otherwise do not mess with it.
             system("/bin/bash -c 'psql #{config['database']} <<< \"alter user #{config['username']} with password '#{config['password']}';\"'")
             system("/bin/bash -c 'psql #{config['database']} <<< \"grant all privileges on database #{config['database']} to #{config['username']};\"'")
           rescue StandardError => err
             $stderr.puts "multisite failed to grant ownership on #{config['database']} to #{config['username']} so stopping: #{err.message}"
           system("/bin/bash -c ' psql #{config['database']} <<< \"alter schema public owner to #{config['username']};\"'")
           system("/bin/bash -c ' psql #{config['database']} <<< \"create extension if not exists hstore;\"' ")
           system("/bin/bash -c ' psql #{config['database']} <<< \"create extension if not exists pg_trgm;\"'")

  - exec: /bin/bash -c 'sudo -u postgres /shared/multisite'

After doing the multsite changes from Multisite configuration with Docker in web_only.yml I added to hooks -> before_bundle _exec -> exec section code to create the multisite.yml as a link into the shared area. This gets manually linked over to the data containers shared area to be read by the above multisite script.

      - mkdir -p                    /shared/config/rails
      - bash -c "touch -a           /shared/config/rails/multisite.yml"
      - bash -c "ln    -s           /shared/config/rails/multisite.yml $home/config/"

It seems like I need to stop that data container to run the bootstrap to create the new database and possibly the db owner. The web_only container does not need to be restarted as it seems to re read the multisite.yml file when either new site is accessed or the file gets changed.

I and thinking about writing a script on the outside of the data container to ssh in and run multisite as shutting down the container to do the deed does not make much sense.

Any thoughts?

(Sam Saffron) #6

This is kind of cool, the way we avoid needing to re-bootstrap is by having an extra daemon run inside the container that takes care of enforcing config and issuing sv restart unicorn as needed.

(eriko) #7

Ok so totally skip the bootstrap and have a daemon watch the file and update the db as needed. This also explains why I did not need to restart web_only. That makes way more sense.

Any thoughts on how to more cleanly crosslink multisite.yml between the containers with out doing it manually?


(Sam Saffron) #8

We drive the configuration from a central location, pull data as needed and rebuild multisite. Plenty of options for central store, postgres, etcd etc.

(eriko) #9

I am not, thankfully, looking to run anything on your scale. Basically a couple containers running one main site and an easy way to add extras as needed.

We started a year long pilot before docker became the production install route and we are getting ready to bless it as production so I cleaning up. Moving from bare metal on Oracle RHEL6 to docker and I would like to make it reproducible.

Thanks for the pointing out the daemon idea. That will make it much cleaner.

(Kane York) #10

Not totally relevant but, I think that the canonical location for scripts such as this is /tmp.

(eriko) #11

Hmm it seems I was mistaken about not needing to bootstrap web_only. The links multisite.yml in the web_only container went missing and the main site was just responding to all hostnames. Still This gets me closer. I can run the monit in both to watch for changes in multisite.yml. Create the db in data and once the db is created then run rake multisite:migrate to finish up.

Yes /tmp would be a better location for the website script. I first wrote it outside the container and ran it manually inside so /shared was easier.