Discourse doesn't deliver webpages, fresh install on Linode Ubuntu 14.04

installation
1

Hi there. I’m new to Discourse and new to Docker.

I’ve got a Linode running Ubuntu 14.04 LTS. It functions as a basic webserver with Apache listening on port 80 and serving a handful of PHP+MySQL websites.

I have an existing site host.com and I’d like to get Discourse running at subdomain.host.com. Since Apache is listening on host.com:80 I’ve tried setting Discourse to run at subdomain.host.com:8080

Unfortunately when I point my browser at subdomain.host.com:8080 it produces a “connection refused” error.

DNS isn’t an issue, subdomain.host.com:80 delivers Apache’s default webpage.

Installation and config

To install Docker then Discourse I followed the installation instructions on GitHub.

I edited the file containers/app.yml thusly so traffic incoming on host port 8080 is redirected to container port 80:

templates:
  - "templates/cron.template.yml"
  - "templates/postgres.template.yml"
  - "templates/redis.template.yml"
  - "templates/sshd.template.yml"
  - "templates/web.template.yml"
  - "templates/web.ratelimited.template.yml"
  - "templates/web.socketed.template.yml"

## which TCP/IP ports should this container expose?
expose:
  - "8080:80"
# Removed:  - "2222:22" # fwd host port 2222 to container port 22 (ssh)
# Removed:  - "80:80"   # fwd host port 80   to container port 80 (http)

Running ./launcher bootstrap app or ./launcher rebuild app plus ./launcher start app complete without errors, and I confirm all looks okay with docker ps (trimmed):

root@www:/var/discourse# docker ps
CONTAINER ID        IMAGE                 COMMAND        ...    PORTS                  NAMES
ab********00        local_discourse/app   "/sbin/boot"   ...    0.0.0.0:8080->80/tcp   app

However, when I point Chrome at subdomain.host.com:8080 it gives me ERR_CONNECTION_REFUSED.

Confirmed with telnet subdomain.host.com 8080 from my Windows desktop:

C:\dir\dir>telnet subdomain.host.com 8080
Connecting To subdomain.host.com...Could not open connection to the host,
on port 8080: Connect failed

Discourse is failing to listen on port 80, but succeeds on port 22 after I re-enabled in app.yml and rebuild:

me@my-linode:/$ ./launcher enter app
root@www-app:/# apt-get-install telnet
...
root@www-app:/# telnet localhost 80
Trying ::1...
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
root@www-app:/# telnet localhost 22
Trying ::1...
Connected to localhost.
Escape character is '^]'.
SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2
^]
telnet>

Log files

Looking through docker logs app there are a few warnings, and some port numbers that don’t look right to me:

Started runsvdir, PID is 29
ok: run: redis: (pid 41) 0s
41:M 30 Jun 12:16:57.444 * Redis 3.0.1 (00000000/0) 64 bit, standalone mode, port 6379, pid 41 ready to start.
41:M 30 Jun 12:16:57.445 # Server started, Redis version 3.0.1
41:M 30 Jun 12:16:57.445 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
41:M 30 Jun 12:16:57.446 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
ok: run: postgres: (pid 40) 0s
41:M 30 Jun 12:16:57.452 * DB loaded from disk: 0.006 seconds
41:M 30 Jun 12:16:57.453 * The server is now ready to accept connections on port 6379
Server listening on 0.0.0.0 port 22.
Server listening on :: port 22.
rsyslogd: invalid or yet-unknown config file command 'KLogPermitNonKernelFacility' - have you forgotten to load a module? [try http://www.rsyslog.com/e/3003 ]
rsyslogd: warning: ~ action is deprecated, consider using the 'stop' statement instead [try http://www.rsyslog.com/e/2307 ]
rsyslogd: Could no open output pipe '/dev/xconsole': No such file or directory [try http://www.rsyslog.com/e/2039 ]
2015-06-30 12:16:57 UTC [61-1] LOG:  database system was shut down at 2015-06-30 12:16:35 UTC
2015-06-30 12:16:57 UTC [40-1] LOG:  database system is ready to accept connections
2015-06-30 12:16:57 UTC [65-1] LOG:  autovacuum launcher started
supervisor pid: 43 unicorn pid: 67
41:M 30 Jun 12:21:58.015 * 10 changes in 300 seconds. Saving...
41:M 30 Jun 12:21:58.017 * Background saving started by pid 460
460:C 30 Jun 12:21:58.023 * DB saved on disk
460:C 30 Jun 12:21:58.024 * RDB: 0 MB of memory used by copy-on-write
41:M 30 Jun 12:21:58.117 * Background saving terminated with success
41:M 30 Jun 12:26:59.016 * 10 changes in 300 seconds. Saving...
41:M 30 Jun 12:26:59.018 * Background saving started by pid 786
786:C 30 Jun 12:26:59.026 * DB saved on disk

According to /bin/ps Docker appears to be proxying host port 8080 to container port 80:

root@www:/var/discourse# ps ax | grep -i dock
10647 ?        Sl     0:00 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 8080 -container-ip 172.17.0.79 -container-port 80

I confess ignorance of iptables, but hunting through similar threads I’ve seen clues emerge from there, so here is output from iptables -L

root@www:/var/discourse# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
fail2ban-ssh  tcp  --  anywhere             anywhere             multiport dports ssh
ACCEPT     all  --  anywhere             anywhere
REJECT     all  --  anywhere             127.0.0.0/8          reject-with icmp-port-unreachable
ACCEPT     all  --  anywhere             anywhere             state RELATED,ESTABLISHED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:http
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:https
ACCEPT     tcp  --  anywhere             anywhere             state NEW tcp dpt:ssh
ACCEPT     icmp --  anywhere             anywhere
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:3000
LOG        all  --  anywhere             anywhere             limit: avg 5/min burst 5 LOG level debug prefix "iptables denied: "
DROP       all  --  anywhere             anywhere

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
DOCKER     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere

Chain DOCKER (1 references)
target     prot opt source               destination
ACCEPT     tcp  --  anywhere             172.17.0.79          tcp dpt:http

Chain fail2ban-ssh (1 references)
target     prot opt source               destination
REJECT     all  --  106.243.125.195      anywhere             reject-with icmp-port-unreachable
REJECT     all  --  43.255.189.23        anywhere             reject-with icmp-port-unreachable
RETURN     all  --  anywhere             anywhere

…and iptables -L -t nat

root@www:/var/discourse# iptables -L -t nat
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
DOCKER     all  --  anywhere             anywhere             ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
DOCKER     all  --  anywhere            !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
MASQUERADE  all  --  172.17.0.0/16        anywhere
MASQUERADE  tcp  --  172.17.0.79          172.17.0.79          tcp dpt:http

Chain DOCKER (2 references)
target     prot opt source               destination
DNAT       tcp  --  anywhere             anywhere             tcp dpt:http-alt to:172.17.0.79:80

Should I really be running Nginx or HAProxy as suggested in other threads? I have a feeling this should be working without that sort of thing.

Many thanks in advance.

2

Okay, this seems partially fixed. It was largely an oversight on my behalf, and I’ll post it here to save anyone else tearing their hair out.

Short version:

Edit containers/app.yml to forward host port whatever (8080 in my case) to container port 3000, which is where Discourse is communicating web traffic:

## which TCP/IP ports should this container expose?
expose:
  - "8080:3000"

Followed of course by ./launcher rebuild app

HOWEVER… while I can now browse to subdomain.host.com:8080 the resulting Discourse pages appear completely blank in Chrome. If I view-source there’s plenty of HTML, so I’m unsure why the browser isn’t rendering it. More on that later, hopefully.

Long version:

I trawled the log files:

./launcher enter app
me@my-linode:/$ ./launcher enter app
root@www-app:/# ls -l /var/log
total 176
-rw-r--r-- 1 root   root        606 Jun 30 17:00 alternatives.log
drwxr-xr-x 2 root   root       4096 Jun 30 15:46 apt
-rw-r----- 1 syslog adm         272 Jun 30 14:41 auth.log
-rw-r--r-- 1 root   root       1218 Jun 30 17:00 dpkg.log
drwxr-xr-x 2 root   www-data   4096 Jun 30 10:12 nginx
-rw-r----- 1 syslog adm      154031 Jun 30 17:29 syslog

I looked through the syslog but didn’t see much to enlighten me in there. So I looked at the Nginx logs:

root@www-app:/# tail /var/log/nginx/error.log
2015/06/30 17:05:57 [emerg] 2026#0: bind() to unix:/shared/nginx.http.sock failed (98: Address already in use)
2015/06/30 17:05:57 [emerg] 2026#0: bind() to unix:/shared/nginx.http.sock failed (98: Address already in use)
2015/06/30 17:05:57 [emerg] 2026#0: bind() to unix:/shared/nginx.http.sock failed (98: Address already in use)
2015/06/30 17:05:57 [emerg] 2026#0: bind() to unix:/shared/nginx.http.sock failed (98: Address already in use)
2015/06/30 17:05:57 [emerg] 2026#0: bind() to unix:/shared/nginx.http.sock failed (98: Address already in use)
2015/06/30 17:05:57 [emerg] 2026#0: still could not bind()
2015/06/30 17:05:59 [emerg] 2030#0: bind() to unix:/shared/nginx.http.sock failed (98: Address already in use)
2015/06/30 17:05:59 [emerg] 2030#0: bind() to unix:/shared/nginx.http.sock failed (98: Address already in use)
2015/06/30 17:05:59 [emerg] 2030#0: bind() to unix:/shared/nginx.http.sock failed (98: Address already in use)
2015/06/30 17:05:59 [emerg] 2030#0: bind() to unix:/shared/nginx.http.sock failed (98: Address already in use)

That didn’t look good. Back on the host file system I deleted /var/discourse/shared/standalone/nginx.http.sock as advised in several other threads, and restarted Discourse.

Then I decided to investigate Nginx a little closer.

root@www-app:/# less /etc/nginx/conf.d/discourse.conf
...
upstream discourse { server 127.0.0.1:3000; }

That looked like a clue. So I updated the host’s app.yml and rebuilt.

Now to uncover why the webpages that Discourse is serving are failing to render in the browser. Curiouser and curiouser.

1 Like
3

we need to add a host_run command to web.socketed. template.yml to delete that file on rebuild, this is about the third time or something

2 Likes
4

Yes can you take that @riking? I agree it is happening too much.

1 Like
5

Thanks for that, great to see you guys responding to issues like this.

I’ve raised the blank pages issue separately, for anyone who wants to follow it:
https://meta.discourse.org/t/discourse-delivering-blank-web-pages-on-ubuntu-14-on-linode/30665

Happy for this thread to be closed.

1 Like
6

Not really the correct solution. On stage one is should delete any existing socket file, on stage 3 it should also do that.

We support boot stages like init.d does in our docker image.