Thanks @riking, that’s right. Fortunately upgrading seems pretty straightforward, unless I am missing something.
On the connected PC, run git fetch --all on the Git repos, rebuild Discourse, extract Ruby Gem cache, and if needed export newer versions of the Docker images. Then copy these over and run ./launcher rebuild app.
This doesn’t work for me, there is an earlier step in this template where git tries to connect to the internet and fails
the way to do it seems to be to change this earlier block from:
- exec:
cd: $home
hook: code
cmd:
- git reset --hard
- git clean -f
- git remote set-branches --add origin master
to:
- exec:
cd: $home
hook: code
cmd:
- git reset --hard
- git clean -f
- git config --global url."file:///local-github.com/".insteadOf https://github.com/
- git remote set-branches --add origin master
and I had to modify the path when copying the ruby GEMs, like this:
# copy the locally-cached Ruby GEMS to /var/www/discourse/vendor/...
- cp -rv /local-rubygems.org/ruby/* $home/vendor/bundle/ruby/
Otherwise they would get copied too deep and the bootstrap fails:
'/local-rubygems.org/ruby/2.4.0' -> '/var/www/discourse/vendor/bundle/ruby/local-rubygems.org/ruby/2.4.0'`
etc
I, [2018-05-29T19:33:47.678827 #5] INFO -- : Running `bundle install --deployment --local --verbose --without "development"` with bundler 1.16.1
Frozen, using resolution from the lockfile
The definition is missing ["fastimage-2.1.1", "libv8-6.3.292.48.1", "mini_racer-0.1.15"]
Could not find fastimage-2.1.1 in any of the sources
It also seems like upgrading an existing installation from postgres 9.5 to 10 runs into trouble when the rebuild script tries to apt-get the postgres 9.5 packages. I worked around it by backing up the previous version, moving /var/discourse/shared out of the way and redeploying a blank discourse to restore to. It seems like as long as you do a restore from backup it doesn’t have to download any ubuntu packages to convert the database.
Maybe there’s a way to point it to cached local .deb files for the various postgres versions but in our case clearing the old DB and restoring from backup was simpler.
@ssvenn you are spot on all the changes, particularly the command to copy the Ruby gems, thanks! 
It wasn’t clear to me which Git command was giving you trouble (where you had to move git config to execute earlier on). The only thing that I ran into, at the time, was the attempt to git pull to update pups, but that is in the launcher script. Anyway, happy to hear you found a way around it!
I tried to edit the OP with your suggestions but it looks like I don’t have permission, unfortunately.
When I upgraded Postgres to v10, I had to use some ugly tricks; downloading apt packages offline for both v9.5 and v10, and drop them in the apt-cache folder just before the commands to install them would run 
Glad I got past that hurdle, and happy to see you’ve found (and shared!) your solution!
Hi,
This topic is the one and only reason I was able to deploy discourse in my environment. I am very grateful for the info provided here.
However I tried updating to the latest version and it seems a new dependency was added and it tries to pull from geolite.maxmind.com and this causes the rebuild to fail (using HEAD – If I try to use tests-passed I get another error that might not be related.)
If anyone has any idea on how to fix this, i’d love to be able to continue using discourse.
On the other hand, I think it’d be great if the discourse team could provide us with some form of offline installer for the major versions, it doesn’t seem it would be too complicated to create a pipeline to bundle all the assets and adapt the install script for offline install.
Thanks anyway to everyone involved in this project.
Yes, we should provide some workaround. This is for the geolocation of IP addresses to show detailed login history to all users, and proactively alert if admins are logging in from suspicious locations. Any ideas @sam or @nbianca ?
some sort of official --create-cache and --use-cache flags in the launcher script would be great, but I think isolated networks is kind of an edge case.
we can probably work around this issue by messing with /etc/hosts in the container and point geolite.maxmind.com to the IP of an internal webserver, I’ll play around with it in my lab environment.
Thought about this and realized it would make handling SSL verification very complex, but I don’t know much about SSL so I might be wrong on this one 
I decided to circumvent the problem by hosting the two required files GeoLite2-City and GeoLite2-ASN on our internal software repository and modifying lib/discourse_ip_info.rb line 27 (discourse.git)
uri = URI("https://geolite.maxmind.com/download/geoip/database/#{name}.tar.gz")
Changed it to our internal software repository URL and it worked fine (if you are using HTTPS you will either have to inject your certificate chain and use update-ca-trust OR create an unsecured SSL context)
Creating a separate post to discuss this issue (maybe this should be split in another topic entirely for clarity?)
100 % this.
I would tend to disagree with this. Lots of industries have isolated networks for security reasons, be it banking, energy, health & medicine…
Is it an edge case for the current population of Discourse users ? yes, definitely
But maybe it’s an untapped market for Discourse that could further expand the userbase and bring feedback & improvements (I know my Discourse instance is likely to receive a stringent security audit in the near future, I will report everything that can be safely disclosed).
I hope we can hear from @codinghorror and the rest of the team on this issue.
Again, I would like to thank everyone involved and I am very glad I was able to bring back my discourse online (rollback to previous app container was impossible in my situation due to my own stupidity).
Yeah I agree there’s definitely an untapped market for enterprises wanting to use it for collaboration on firewalled corporate networks. It is fairly easy to get really slick integration with Active Directory and internal Exchange email in a Windows domain.
https://meta.discourse.org/t/active-directory-sso-aka-integrated-windows-login/26548?u=ssvenn
As a full-time Discourse consultant, I am always on the lookout for “untapped markets.” I have a bit for work for corporate environments, but in my experience, people who have firewalled corporate networks won’t let anyone on their firewalled corporate network to help them do the work.
I have suffered through some fairly painful screen shares. . . and a fair amount of "OK, now what do you see after you typed ./launcher rebuild web_only"
If anyone is interested, I can help create Ansible playbooks to deploy Discourse, which might allow for testing on non-firewalled networks that could then be deployed internally. I keep thinking that I’ll have a Kubernetes installation available as a service Real Soon Now.
@malec today I ran into the same issue, and managed to find a way around it, but as usual it wasn’t pretty.
It looks like the rake task added by @nbianca in PR #7340 checks the modification date on the GeoLite files included in the Discourse base image, and if they’re older than 2 days, will attempt to download them again during bootstrap; see the relevant code here.
The added task could arguably handle failure more gracefully, e.g. by printing a warning and failing back to using the already-present files, but right now it just fails bootstrap.
So here is what I did, hoping it helps you and anyone else stuck with this:
Download the GeoLite2-ASN.mmdb and GeoLite2-City.mmdb files and store them on your Discourse host; I put them under /root/local/maxmind.com.
Map this folder as a Docker volume in your app.yml file, e.g via:
- volume:
host: /root/local/maxmind.com
guest: /local-maxmind.com
Add a bundle_exec hook to your app.yml file, to be run as early as possible:
run:
- exec:
cd: $home
hook: bundle_exec
cmd:
- mkdir -p $home/vendor/data
- ls -la $home/vendor/data
- cp -rv /local-maxmind.com/*.mmdb $home/vendor/data/
- ls -la $home/vendor/data
This copies the GeoLite files from your host, overwriting the ones already present in the Discourse base image.
You can verify in the bootstrap log, using the output of the two ls -la statements, whether the files have been overwritten. The first ls -la output will look something like:
I, [2019-05-11T11:06:39.528299 #6] INFO -- : > cd /var/www/discourse && ls -la /var/www/discourse/vendor/data
I, [2019-05-11T11:06:39.533286 #6] INFO -- : total 65948
-rw-r--r-- 1 root root 6473392 Apr 29 10:37 GeoLite2-ASN.mmdb
-rw-r--r-- 1 root root 61106856 Apr 29 10:37 GeoLite2-City.mmdb
Note that the modification time is Apr 29, i.e. older than 2 days. The rake task would attempt to download these files again.
After overwriting the files, the second ls -la output will instead look something like this:
I, [2019-05-11T11:06:39.533336 #6] INFO -- : > cd /var/www/discourse && cp -rv /local-maxmind.com/*.mmdb /var/www/discourse/vendor/data/
I, [2019-05-11T11:06:39.604593 #6] INFO -- : '/local-maxmind.com/GeoLite2-ASN.mmdb' -> '/var/www/discourse/vendor/data/GeoLite2-ASN.mmdb'
'/local-maxmind.com/GeoLite2-City.mmdb' -> '/var/www/discourse/vendor/data/GeoLite2-City.mmdb'
I, [2019-05-11T11:06:39.604695 #6] INFO -- : > cd /var/www/discourse && ls -la /var/www/discourse/vendor/data
I, [2019-05-11T11:06:39.608645 #6] INFO -- : total 66028
-rw-r--r-- 1 2000 2000 6492486 May 11 10:52 GeoLite2-ASN.mmdb
-rw-r--r-- 1 2000 2000 61030828 May 11 10:52 GeoLite2-City.mmdb
The mmdb files are now up-to-date as far as the rake tasks are concerned. This will allow bootstrap to carry on without trying to download them.
Additionally, I also added an environment variable in my app.yml file:
env:
DISCOURSE_REFRESH_MAXMIND_DB_DURING_PRECOMPILE_DAYS: 3650
I was trying to override the default setting of 2 days for GeoLite checks, but this appeared to not have any effect in my testing. Your mileage may vary.
/cc @sam @nbianca @codinghorror – a better (or at least more graceful) way to handle GeoLite updates, or the ability to disable this during bootstrap would be greatly appreciated. Both GeoLite files are already present in the Discourse base image, and for some use cases (mine is on-premise use in an isolated network environment), fresh copies are really not essential.
It would be really good to have an opt-in bootstrap option to just use the files embedded in the base image, without downloading. Or at least to not fail bootstrap when the GeoLite download fails; printing a warning and continuing with the files in the base image would be, in my mind, better than the current experience.
If the only thing being checked is the modification time couldn’t you just bump that using touch?
From the site_settings.yml:
refresh_maxmind_db_during_precompile_days:
min: 0
max: 120
default: 2
hidden: true
So:
bundle exec rails c
SiteSetting.refresh_maxmind_db_during_precompile_days = 120
You can’t set it beyond the limit defined in the file you linked, but you can at least push it to four months.
@Stephen you’re absolutely right on both accounts, thanks a lot! This is a way cleaner solution – and much appreciated!
Using touch to update the GeoLite file mtime on every bootstrap works, of course.
I also set DISCOURSE_REFRESH_MAXMIND_DB_DURING_PRECOMPILE_DAYS to 120 just in case the logic changes. (I don’t know how I missed the max value earlier!)
Yeah, we were missing a shadow by global option there, fixed now, you can just use that simple env change instead of the giant hack here.
https://github.com/discourse/discourse/commit/4e1f25197da2b6e39e6f003f1c1229037486462c
It seems like some plugins at some point started requiring more workarounds to be installable on the isolated server, when trying to build with - git clone https://github.com/discourse/discourse-prometheus.git or - git clone https://github.com/discourse/discourse-calendar.git i get
I, [2019-08-01T13:23:05.482497 #6] INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate'
ERROR: While executing gem ... (TypeError)
no implicit conversion of nil into String
I, [2019-08-01T13:24:10.583829 #6] INFO -- : gem install prometheus_exporter -v 0.4.13 -i /var/www/discourse/plugins/discourse-prometheus/gems/2.6.3 --no-document --ignore-dependencies --no-user-install
You are specifying the gem prometheus_exporter in /var/www/discourse/plugins/discourse-prometheus/plugin.rb, however it does not exist!
Looked for: /var/www/discourse/plugins/discourse-prometheus/gems/2.6.3/specifications/prometheus_exporter-0.4.13.gemspec
and
I, [2019-08-01T13:15:56.607026 #6] INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate'
ERROR: While executing gem ... (TypeError)
no implicit conversion of nil into String
I, [2019-08-01T13:17:01.618404 #6] INFO -- : gem install holidays -v 7.1.0 -i /var/www/discourse/plugins/discourse-calendar/gems/2.6.3 --no-document --ignore-dependencies --no-user-install
You are specifying the gem holidays in /var/www/discourse/plugins/discourse-calendar/plugin.rb, however it does not exist!
Looked for: /var/www/discourse/plugins/discourse-calendar/gems/2.6.3/specifications/holidays-7.1.0.gemspec
I haven’t found a way to get around this yet, for now I’ve just disabled the ones that cause rebuild errors.
Most plugins carry all the code they need and have a simple install. Those two you tried to use need to fetch gems (libraries) from the internet to able to function.
do you know where in the code those ruby gem fetches for plugins happen? I tried looking for it but haven’t found the right place yet.
I’m thinking maybe instead of using cached gems maybe it would be more elegant to run a rubygems mirror on the isolated host, i found a tutorial here and it looks like you can configure extra system-wide gem sources
Our internal Discourse has happily been running the same old version for a good long while now, but we are finally getting rid of IE9 so it’s way overdue for an upgrade.
I can report that this guide is still good with the latest versions of Discourse. Even with RHEL8 instead of 7 
While experimenting I figured out a way to get those discourse-prometheus and discourse-calendar plugins working offline too even with the extra dependencies, the trick was to fish out the ruby gems from the plugin directory of the build server as well as the ones in /var/www/discourse/vendor/bundle/ruby
docker run -it -v ~/local/rubygems.org:/local-rubygems local_discourse/app /bin/bash -c "cp -rv /var/www/discourse/vendor/bundle/ruby /local-rubygems"
docker run -it -v ~/local/rubygems.org/plugin-gems/discourse-calendar:/local-rubygems local_discourse/app /bin/bash -c "cp -rv /var/www/discourse/plugins/discourse-calendar/gems /local-rubygems"
docker run -it -v ~/local/rubygems.org/plugin-gems/discourse-prometheus:/local-rubygems local_discourse/app /bin/bash -c "cp -rv /var/www/discourse/plugins/discourse-prometheus/gems /local-rubygems"
and then in /templates/web.template.yml :
- exec:
cd: $home
hook: bundle_exec
cmd:
# copy local ruby cache
- cp -rv /local-rubygems.org/ruby/* $home/vendor/bundle/ruby/
- cp -rv /local-rubygems.org/plugin-gems/* $home/plugins/
- su discourse -c 'bundle install --local --deployment --retry 3 --jobs 4 --verbose --without test development'
By the way it looks like the topic for Active Directory IIS SSO got lost at some point, but the code is still available at GitHub - laktak/discourse-sso: Single Sign On for Discourse with Active Directory and still works with the now renamed DiscourseConnect SSO
I tried another upgrade and it looks like at some point a new “yarn install” section was added to web.template.yml which breaks inside the isolated environment.
- exec:
cd: $home
cmd:
- "[ ! -d 'node_modules' ] || su discourse -c 'yarn install --production && yarn cache list'"
When comparing the contents of the old and new container it looks like there’s a bunch of cached yarn packages on the new one in /usr/local/share/.cache/yarn/v6 but nothing in the old one, I guess all the required node.js used to be included in the base image but now they get updated during a rebuild?
I will experiment with copying out the yarn cache the same way as the ruby cache and see if i can get yarn to use the cached packages from the build box.