Install discourse with internet access only via proxy


(James) #1

Hi All,

(Note that I have deliberately broken all domain names by inserting whitespace because they are getting detected as links, and as a noob I cannot post >2 links in a post)

I am trying to install discourse on a redhat VM at my work.
I have the following constraints:

  • Host has no direct access to internet. Access is only via http/https proxy
  • Host cannot resolve external DNS names (this is handled via the proxy)

yum, git, docker on the host are all configured to use the proxy.

I have discovered that I can pass the installer docker arguments, but these do not seem to get used in all locations.
I hope that someone on the forum can give me a pointer to progress?

I have observed that the installed needs to be able to connect to github .com and to rubygems .org.
Based on this I am using the following command to bootstrap with static host mappings for github and rubygems, and passing https proxy env variable:

./launcher bootstrap app --docker-args "--add-host github.com:192.30.253.113 --add-host rubygems.org:151.101.2.2 --add-host myproxy:10.10.10.10 -e HTTPS_PROXY=https://myproxy:8080"

The problem is that only github can be contacted. The installer cannot reach rubygems, and I get a failures like this:

Fetching source index from https://rubygems.org/
Need to query more than 500 gems. Downloading full index instead...
HTTP GET https://rubygems.org/quick/Marshal.4.8/libv8-5.7.492.65.1-x86_64-linux.gemspec.rz
Errno::EHOSTUNREACH: Failed to open TCP connection to rubygems.org:443 (No route to host - connect(2) for "rubygems.org" port 443)
/usr/local/lib/ruby/2.3.0/net/http.rb:882:in `rescue in block in connect'
...
...
...    
Network error while fetching
https://rubygems.org/quick/Marshal.4.8/libv8-5.7.492.65.1-x86_64-linux.gemspec.rz
(Failed to open TCP connection to rubygems.org:443 (No route to host -
connect(2) for "rubygems.org" port 443))

I, [2017-07-18T15:24:15.899372 #13]  INFO -- : Terminating async processes
I, [2017-07-18T15:24:15.899452 #13]  INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/9.5/bin/postmaster -D /etc/postgresql/9.5/main pid: 42
I, [2017-07-18T15:24:15.899557 #13]  INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 155
2017-07-18 15:24:15 UTC [42-2] LOG:  received fast shutdown request
2017-07-18 15:24:15 UTC [42-3] LOG:  aborting any active transactions
2017-07-18 15:24:15 UTC [49-2] LOG:  autovacuum launcher shutting down
155:signal-handler (1500391455) Received SIGTERM scheduling shutdown...
2017-07-18 15:24:15 UTC [46-1] LOG:  shutting down
155:M 18 Jul 15:24:15.905 # User requested shutdown...
155:M 18 Jul 15:24:15.905 * Saving the final RDB snapshot before exiting.
155:M 18 Jul 15:24:15.908 * DB saved on disk
155:M 18 Jul 15:24:15.908 # Redis is now ready to exit, bye bye...
2017-07-18 15:24:15 UTC [46-2] LOG:  database system is shut down


FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle install --deployment --verbose --without test --without development' failed with return #<Process::Status: pid 277 exit 17>
Location of failure: /pups/lib/pups/exec_command.rb:108:in `spawn'
exec failed with the params {"cd"=>"$home", "hook"=>"bundle_exec", "cmd"=>["su discourse -c 'bundle install --deployment --verbose --without test --without development'", "su discourse -c 'bundle exec rake db:migrate'", "su discourse -c 'bundle exec rake assets:precompile'"]}
6e41afbf21a5aa0eb0e98e620801f06ae8bffb69f64c3c059475c00f66f17261
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one

Seems that whatever is connecting to rubygems is not respecting the https proxy environment variable passed to docker, hence I am getting the:
Errno::EHOSTUNREACH: Failed to open TCP connection to rubygems .org:443 (No route to host - connect(2) for “rubygems .org” port 443)

Any ideas how I can progress this?

BTW for anyone considering installing behind a proxy, think long and hard - it’s gruelling :frowning:


Proxy during discourse bootstrap
(Jay Pfaffman) #2

That does sound grueling.

It looks like someone managed to do it; perhaps their notes (with a problem you’ll have down the road) will help.


(James) #3

Looks like gem does not respect the https proxy env var.
I can see here that the proxy needs to be passed as a switch on the gem command or set in .gemrc:

Depending on your gem version, one of these should work:

gem install --http-proxy http://[user]:[password]@[server]:[port]
gem install -p http://[user]:[password]@[server]:[port]
If that’s your main net access, then you can set it to be used by default. Create .gemrc file in your home dir (user folder in windows or linux) and set the proxy settings there:

http_proxy: http://[user]:[password]@[server]:[port]

AFAIK neither is possible via the discourse installer… I beg to be contradicted :slight_smile:


(James) #4

Hi Jay

Thanks for the pointer. I’ve been searching around on here. but most mentions of proxy are regarding configuring a working discourse to be behind a reverse proxy,

The page you linked conformed my hypothesis regarding gem not respecting env var for proxy.

Unfortunately this is for a working env. I cant get to that point :frowning:


(Rafael dos Santos Silva) #5

This is possible, you need:

  • Export http and https ENV variables in your current shell

  • Add the same variables to the env block inside the containers/app.yml file

  • Add the same variables to the docker system.d service config file (read system.d docs on this, you will need to create a new service file somewhere).

After this, our official guide will work just fine.


(James) #6

Thanks!
Just need to do the middle bullet.
I already updated the docker system.d


(James) #7

Looking good.
Added HTTP_PROXY and HTTPS_PROXY env variables and its going well so far…


(Matt Palmer) #8

Traditionally, the environment variable names for proxies are lower case. Perhaps -e https_proxy=<...> might have worked?


(James) #9

Good point.
Seems that the upperCASE env vars work sometimes.
I noticed that nginx log had errors connecting to avatars.discourse.org, and pretty sure its due to only having HTTP_PROXY

I changed to lowercase.

Thanks!


(Rafael dos Santos Silva) #10

So did the install work for you?


(James) #11

Difficult to say right now.
The install completes :+1:
I cannot access the web UI :-1:

Trying to debug atm.
Portainer running on the same docker works fine (port 9000), but neither 80 or 443 is responding for discourse.

Errors in logs:
nginx log I am seeing this every second:

2017/07/19 14:22:19 [emerg] 1318#1318: host not found in upstream “avatars.discourse.org” in /etc/nginx/conf.d/discourse.conf:217

In production.log I am seeing the following… seems serious:

Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED) subscribe failed, reconnecting in 1 second. Call stack ["/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis/client.rb:345:in rescue in establish_connection'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis/client.rb:331:inestablish_connection’", “/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis/client.rb:101:in block in connect'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis/client.rb:293:inwith_reconnect’”, “/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis/client.rb:100:in connect'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis/client.rb:364:inensure_connected’”, “/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis/client.rb:221:in block in process'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis/client.rb:306:inlogging’”, “/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis/client.rb:220:in process'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis/client.rb:134:inblock in call_loop’”, “/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis/client.rb:280:in with_socket_timeout'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis/client.rb:133:incall_loop’”, “/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis/subscribe.rb:43:in subscription'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis/subscribe.rb:12:insubscribe’”, “/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis.rb:2765:in _subscription'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis.rb:2143:inblock in subscribe’”, “/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis.rb:58:in block in synchronize'", "/usr/local/lib/ruby/2.4.0/monitor.rb:214:inmon_synchronize’”, “/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis.rb:58:in synchronize'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-3.3.3/lib/redis.rb:2142:insubscribe’”, “/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/message_bus-2.0.2/lib/message_bus/backends/redis.rb:304:in global_subscribe'", "/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/message_bus-2.0.2/lib/message_bus.rb:513:inglobal_subscribe_thread’”, “/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/message_bus-2.0.2/lib/message_bus.rb:461:in `block in new_subscriber_thread’”]
Job exception: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)

I kept a close eye on installer stdout and didnt see anything fatal looking…
Right now I am doing a lot of googling :slight_smile:
But v grateful for the responsive community - thanks!


(Rafael dos Santos Silva) #12

You can enter the container with ./launcher enter app and try to debug WTF is redis doing. (try a redis-cli for starters).


(James) #13

Thanks.
Actually the redis error was/maybe a red herring.
I can now access the welcome splash once I fixed up access to “avatars.discourse.org
The problem for me is that my corporate DNS servers resolve only internal hosts.
nginx expects to resolve the above domain, but cannot - hence ngnix crashes :frowning:
This was the key - Restarting container when internet is down causes discourse to fail
I progress by passing docker args to set /etc/hosts. Its not a production solution though…


(Rafael dos Santos Silva) #14

Well if you have an intranet, you will not be using that anyway.

You can either use a different source of default avatars using the site setting default avatars, pass all the needed avatars trough SSO or even run a instance of avatars.discourse.org that we open source.


(James) #15

Thanks for the inputs.
Is there a guide for intranet install that I can refer to?


(Rafael dos Santos Silva) #16

Nope, I just had some experience running a big instance on Intranet in the same conditions you have :smile:.

Do your company intranet already have a reusable avatar service?


(James) #17

Frankly I have no idea. My guess would be no, but I’ll be having a chat with some colleagues to find out.


(James) #18

For the interim I disabled external avatars.
I noticed you guys kindly provide a docker image for discourse/letter-avatars, so I might go with that at some point, but I’m thinking to keep it simple (stupid) for now :D.
But thanks for the help!