How to install Discourse on an isolated CentOS 7 server


#1

Here is what worked for me to install Discourse on an isolated CentOS 7 server. In a nutshell, on a machine with internet access I prepared:

  1. the Git repos that Discourse setup needs: SamSaffron/pups, discourse/discourse, discourse/discourse_docker and discourse/docker_manager.
  2. the two Docker images: discourse/base and samsaffron/docker-gc
  3. the complete Ruby Gem cache that bundle install downloads during bootstrap.

I copied these to the isolated server, and used them to install offline; below are the full details.I hope this helps.

1. Preparation on a machine with internet access

With Docker-CE and Git installed, run these commands:

mkdir -p ~/local/github.com
mkdir -p ~/local/rubygems.org
mkdir -p ~/local/docker-images

# 1
git clone --bare https://github.com/SamSaffron/pups.git            ~/local/github.com/SamSaffron/pups.git
git clone --bare https://github.com/discourse/discourse.git        ~/local/github.com/discourse/discourse.git
git clone --bare https://github.com/discourse/discourse_docker.git ~/local/github.com/discourse/discourse_docker.git
git clone --bare https://github.com/discourse/docker_manager.git   ~/local/github.com/discourse/docker_manager.git

#2
sudo mkdir -p /var/discourse
sudo git clone https://github.com/discourse/discourse_docker.git   /var/discourse

Step #1 clones the required Discourse GitHub repositories locally.

Step #2 prepares for the local installation of Discourse, which must run once, to extract Ruby GEMs from the Discourse image (step #4 below).

Search /var/discourse/launcher to find the line starting with image=discourse/base e.g.:

image=discourse/base:2.0.20171008

Two Docker images are needed: discourse/base with the version above, and docker-gc. To get them locally, run:

#3
docker pull discourse/base:2.0.20171008
docker pull samsaffron/docker-gc
docker save -o ~/local/docker-images/discourse_base   discourse/base:2.0.20171008
docker save -o ~/local/docker-images/docker-gc        samsaffron/docker-gc

Step #3 downloads the images and exports them to files under ~/local/docker-images/.

The ~/local folder now has local copies of the Discourse code, and Docker images required for installation.

However, local copies of the Ruby GEMs that Discourse downloads as part of its bootstrap process are still needed. IMHO it would be much easier if they were just included in the discourse/base Docker image.

Discourse needs to be bootstrapped to get these files. Follow the official instructions, but stop at the Start Discourse step. That is, configure and bootstrap Discourse, up to the point where the Docker container local_discourse/app is created.

Then run the following commands:

# 4
docker run -it -v ~/local/rubygems.org:/local-rubygems local_discourse/app /bin/bash
# the commands below run inside the Discourse 'app' container
cp -r /var/www/discourse/vendor/bundle/ruby /local-rubygems 
exit

Step #4 copies the full Ruby GEM cache out of the Discourse image into the ~/local/rubygems.org folder.

Now everything required to set up Discourse offline is available in the ~/local folder. Copy this folder to the target machine.

2. Install Discourse on the target machine, with no internet access

Copy the ~/local folder prepared above to the target machine that has no internet access, e.g. to /root/local.

You will need to be root through the rest of the setup and bootstrap process:

sudo -s

Import the two Docker images:

docker load -i /root/local/docker-images/discourse_base
docker load -i /root/local/docker-images/docker-gc

Configure Git to use the /root/local/github.com/ folder instead of going out to https://github.com/:

git config --global url."file:///root/local/github.com/".insteadOf https://github.com/

Create the Discourse folder and clone discourse_docker.git there:

mkdir -p /var/discourse
git clone https://github.com/discourse/discourse_docker.git /var/discourse/

Run ./discourse-setup to generate your Discourse configuration. At the end of the configuration process, when you see the message Updates successful. Rebuilding in 5 seconds., press Ctrl+C.

cd /var/discourse/
./discourse-setup

Edit the /var/discourse/launcher file and disable updating pups (the version bundled with Discourse will be used instead), changing from:

  if [[ ! "false" =  $update_pups ]]; then
    run_command="$run_command git pull &&"
  fi

to:

#  if [[ ! "false" =  $update_pups ]]; then
#    run_command="$run_command git pull &&"
#  fi

Edit the /var/discourse/containers/app.yml file and update the volumes section as follows:

volumes:
  # these two volumes are already defined
  - volume:
      host: /var/discourse/shared/standalone
      guest: /shared
  - volume:
      host: /var/discourse/shared/standalone/log/var-log
      guest: /var/log
  # ADD THE TWO VOLUMES BELOW, making the local copies of github.com repos and Ruby GEM cache
  # available inside the Discourse container when it is built
  - volume:
      host: /root/local/github.com
      guest: /local-github.com
  - volume:
      host: /root/local/rubygems.org
      guest: /local-rubygems.org

This makes the local GitHub repo clones, and the Ruby GEM cache, available to the Discourse container.

Edit the /var/discourse/templates/web.template.yml file, and make the following changes.

(1) Disable updating Bundler (using the version bundled with Discourse instead) by commenting-out the gem update bundler line, from:

  - exec:
      cd: $home
      hook: web
      cmd:
        # ensure we are on latest bundler
        - gem update bundler
        - chown -R discourse $home

to:

  - exec:
      cd: $home
      hook: web
      cmd:
        # ensure we are on latest bundler
        #- gem update bundler
        - git config --global url."file:///local-github.com/".insteadOf https://github.com/
        - chown -R discourse $home

Instead of updating bundler, Git (the one inside the container) is configured to use the local folder instead of going out to https://github.com/.

(2) Configure bundle install to use the local Ruby GEM cache and not connect to rubygems.org by changing these lines from:

  - exec:
      cd: $home
      hook: bundle_exec
      cmd:
        - su discourse -c 'bundle install --deployment --verbose --without test --without development'
        - su discourse -c 'bundle exec rake db:migrate'
        - su discourse -c 'bundle exec rake assets:precompile'

to:

  - exec:
      cd: $home
      hook: bundle_exec
      cmd:
        # copy the locally-cached Ruby GEMS to /var/www/discourse/vendor/...
        - cp -rv /local-rubygems.org/ $home/vendor/bundle/ruby/
        # install GEMs from local cache only, using `--local` (see http://bundler.io/v1.15/bundle_install.html)
        - su discourse -c 'bundle install --local --deployment --verbose --without test --without development'
        - su discourse -c 'bundle exec rake db:migrate'
        - su discourse -c 'bundle exec rake assets:precompile'

And that’s it. Run ./launcher bootstrap app and, hopefully, enjoy your offline Discourse!


Discourse in a closed intranet
Active Directory SSO aka Integrated Windows Login
Proxy during discourse bootstrap
Discobot not responding to bookmark?
(Felix Freiberger) #2

This may be a very naive question, but… couldn’t you just move the bootstrapped container to the offline server instead of capturing all the Gems necessary to bootstrap there?


#3

Unfortunately that did not work for me. The container Discourse runs in is not isolated from its host; the database, uploads, backups, etc. are stored in volumes mapped to the host file system. You need to copy those as well, and when I copied the bootstrapped container from another machine, I could not line up the file permissions between the users inside the container and those on the host.

See also the comments in the related thread Discourse in a closed intranet

Another issue is that the launcher script does upgrades (Postgress DB migrations, for example) when time comes to update the Discourse version.

With this approach, I can simply update the local copies of Git repos and base Docker images, then run ./launcher rebuild app and let it handle the upgrade correctly.


(Kane York) #4

Be aware that you will have to manually check for security updates; because the code cannot reach discourse.org for version checks it cannot email you when updates are available.

There’s several other features that assume full internet connectivity, but I expect you’re prepared to deal with not having those.


#5

Thanks @riking, that’s right. Fortunately upgrading seems pretty straightforward, unless I am missing something.

On the connected PC, run git fetch --all on the Git repos, rebuild Discourse, extract Ruby Gem cache, and if needed export newer versions of the Docker images. Then copy these over and run ./launcher rebuild app.


#6

This doesn’t work for me, there is an earlier step in this template where git tries to connect to the internet and fails

the way to do it seems to be to change this earlier block from:

  - exec:
      cd: $home
      hook: code
      cmd:
        - git reset --hard
        - git clean -f
        - git remote set-branches --add origin master

to:

  - exec:
      cd: $home
      hook: code
      cmd:
        - git reset --hard
        - git clean -f
        - git config --global url."file:///local-github.com/".insteadOf https://github.com/
        - git remote set-branches --add origin master

and I had to modify the path when copying the ruby GEMs, like this:

      # copy the locally-cached Ruby GEMS to /var/www/discourse/vendor/...
        - cp -rv /local-rubygems.org/ruby/* $home/vendor/bundle/ruby/

Otherwise they would get copied too deep and the bootstrap fails:

'/local-rubygems.org/ruby/2.4.0' -> '/var/www/discourse/vendor/bundle/ruby/local-rubygems.org/ruby/2.4.0'`
etc
I, [2018-05-29T19:33:47.678827 #5]  INFO -- : Running `bundle install --deployment --local --verbose --without "development"` with bundler 1.16.1
Frozen, using resolution from the lockfile
The definition is missing ["fastimage-2.1.1", "libv8-6.3.292.48.1", "mini_racer-0.1.15"]
Could not find fastimage-2.1.1 in any of the sources

It also seems like upgrading an existing installation from postgres 9.5 to 10 runs into trouble when the rebuild script tries to apt-get the postgres 9.5 packages. I worked around it by backing up the previous version, moving /var/discourse/shared out of the way and redeploying a blank discourse to restore to. It seems like as long as you do a restore from backup it doesn’t have to download any ubuntu packages to convert the database.

Maybe there’s a way to point it to cached local .deb files for the various postgres versions but in our case clearing the old DB and restoring from backup was simpler.


#7

@ssvenn you are spot on all the changes, particularly the command to copy the Ruby gems, thanks! :heart:

It wasn’t clear to me which Git command was giving you trouble (where you had to move git config to execute earlier on). The only thing that I ran into, at the time, was the attempt to git pull to update pups, but that is in the launcher script. Anyway, happy to hear you found a way around it!

I tried to edit the OP with your suggestions but it looks like I don’t have permission, unfortunately.

When I upgraded Postgres to v10, I had to use some ugly tricks; downloading apt packages offline for both v9.5 and v10, and drop them in the apt-cache folder just before the commands to install them would run :face_with_head_bandage:

Glad I got past that hurdle, and happy to see you’ve found (and shared!) your solution!