Lots of problems installing Discourse under CentOS with NGINX forwarding

john3 · September 1, 2016, 8:08am

Hi,

I have installed discourse multiple times but the issue remains the same, even a fresh discourse installation did not help. If i upload an image, i get the link and it works, but the default discourse links such as upload icon, default discourse logo etc is broken as shown in the images below.

Logo:

Emojis:

URL: (“https://meta.domain.org/images/emoji/emoji_one/kissing_smiling_eyes.png?v=3”) gives 404 error, doesn’t exist or is private.

root@meta:/var/www/discourse/public/images# ls
avatar.png  default-apple-touch-icon.png  default-favicon.ico  default-favicon.png  d-logo-    sketch.png  d-logo-sketch-small.png  emoji  welcome
root@meta:/var/www/discourse/public/images# pwd
/var/www/discourse/public/images 

root@meta:/var/www/discourse/images# ls
discourse.png
root@meta:/var/www/discourse/images# pwd
/var/www/discourse/images  

root@meta:/var/www/discourse/app/assets/images# ls
chosen-sprite.png  favicons      logo.png             logo-single.png  select2-spinner.gif  spinner_96.gif
favicon.ico        logo-dev.png  logo-single-dev.png  select2.png      select2x2.png
root@meta:/var/www/discourse/app/assets/images# pwd
/var/www/discourse/app/assets/images

I am using nginx proxy and SSL for discourse. url to my app.yml file is given below:

http://pastebin.com/QjwdWj2W

I have also tried several ways already mentioned in other topics, but they don’t resemble my problem, hence I am creating a new topic.

mpalmer · September 1, 2016, 9:08am

There’s an emoji entry in /var/www/discourse/public/images, what’s in there?

john3 · September 1, 2016, 9:19am

Here is the output.

[root@www discourse]# sudo ./launcher enter app
which: no docker.io in (/sbin:/bin:/usr/sbin:/usr/bin)
root@meta:/var/www/discourse# cd /var/www/discourse/public/images
root@meta:/var/www/discourse/public/images# cd emoji
root@meta:/var/www/discourse/public/images/emoji# ls
apple  emoji_one  google  twitter  win10
root@meta:/var/www/discourse/public/images/emoji# cd emoji_one
root@meta:/var/www/discourse/public/images/emoji/emoji_one# ls
100.png                              fist.png                            pineapple.png    
1234.png                             five.png                            ping_pong.png
-1.png                               flag_black.png                      pisces.png
+1.png                               flag_cn.png                         pizza.png
......   
......

mpalmer · September 1, 2016, 9:21am

Is kissing_smiling_eyes.png in there, given that’s the 404 you quoted in your original message?

john3 · September 1, 2016, 9:23am

Yes, it exists. that emoji is just an example, even the discourse logo’s url is broken, however the image exists as shown log.

root@meta:/var/www/discourse/public/images/emoji/emoji_one# ls -la | grep kissing_smiling_eyes.png
-rw-r--r-- 1 discourse root        646 Aug 31 00:24 kissing_smiling_eyes.png

mpalmer · September 1, 2016, 9:27am

What does /var/discourse/shared/standalone/log/var-log/nginx/error.log say about the missing files?

john3 · September 1, 2016, 6:54pm

`/var/log/nginx/error.log

2016/09/01 07:59:31 [error] 15308#0: *1176 access forbidden by rule, client: 1.1.1.1, server:     meta.domain.org, request: "GET /docker-manager-    aff8eaea0445c0488c19f8cfd14faa8c2b278924438f19048eacc175d7d134e4.png HTTP/1.1", host:     "meta.domain.org" 
2016/09/01 07:53:59 [error] 15308#0: *1008 open() "/usr/share/images/d-logo-sketch.png"     failed (2: No such file or directory), client: 1.1.1.1, server:
meta.domain.org, request: "GET /images/d-logo-sketch.png HTTP/1.1", host: "meta.domain.org", referrer: "https://meta.domain.org/images/emoji/emoji_one/kissing_smiling_eyes.png?v=3"
2016/09/01 07:54:00 [error] 15308#0: *822 open() "/usr/share/images/emoji/emoji_one/stuck_out_tongue_winking_eye.png" failed (2: No such file or directory), client: 1.1.1.1, server: meta.domain.org, request: "GET /images/emoji/emoji_one/stuck_out_tongue_winking_eye.png?v=3 HTTP/1.1", host: "meta.domain.org", referrer: "https://meta.domain.org/"
2016/09/01 07:54:00 [error] 15308#0: *1103 open() "/usr/share/images/emoji/emoji_one/stuck_out_tongue.png" failed (2: No such file or directory), client: 1.1.1.1, server: meta.domain.org, request: "GET /images/emoji/emoji_one/stuck_out_tongue.png?v=3 HTTP/1.1", host: "meta.domain.org", referrer: "https://meta.domain.org/"
2016/09/01 07:54:00 [error] 15308#0: *1105 open() "/usr/share/images/emoji/emoji_one/astonished.png" failed (2: No such file or directory), client: 1.1.1.1, server: meta.domain.org, request: "GET /images/emoji/emoji_one/astonished.png?v=3 HTTP/1.1", host: "meta.domain.org", referrer: "https://meta.domain.org/"    
2016/09/01 07:54:00 [error] 15308#0: *1108 open() "/usr/share/images/emoji/emoji_one/flushed.png" failed (2: No such file or directory), client: 1.1.1.1, server: meta.domain.org, request: "GET /images/emoji/emoji_one/flushed.png?v=3 HTTP/1.1", host: "meta.domain.org", referrer: "https://meta.domain.org/"
2016/09/01 07:54:03 [error] 15308#0: *1026 open() "/usr/share/images/emoji/emoji_one/grimacing.png" failed (2: No such file or directory), client: 1.1.1.1, server: meta.domain.org, request: "GET /images/emoji/emoji_one/grimacing.png?v=3 HTTP/1.1", host: "meta.domain.org", referrer: "https://meta.domain.org/"

/var/discourse/shared/standalone/log/var-log/nginx/error.log

2016/09/01 07:55:14 [error] 56#56: *1107 limiting requests, excess: 12.172 by zone "flood", client: 0.0.0.0, server: _, request: "POST /mini-profiler-resources/results HTTP/1.0", host: "meta.domain.org:25654", referrer: "https://meta.domain.org/"
2016/09/01 07:55:14 [error] 56#56: *1109 limiting requests, excess: 12.172 by zone "flood", client: 0.0.0.0, server: _, request: "POST /mini-profiler-resources/results HTTP/1.0", host: "meta.domain.org:25654", referrer: "https://meta.domain.org/"
2016/09/01 07:55:14 [error] 56#56: *1110 limiting requests, excess: 12.172 by zone "flood", client: 0.0.0.0, server: _, request: "POST /mini-profiler-resources/results HTTP/1.0", host: "meta.domain.org:25654", referrer: "https://meta.domain.org/"
2016/09/01 07:55:14 [error] 56#56: *1111 limiting requests, excess: 12.100 by zone "flood", client: 0.0.0.0, server: _, request: "POST /mini-profiler-resources/results HTTP/1.0", host: "meta.domain.org:25654", referrer: "https://meta.domain.org/"
2016/09/01 07:55:14 [error] 56#56: *1112 limiting requests, excess: 12.028 by zone "flood", client: 0.0.0.0, server: _, request: "POST /mini-profiler-resources/results HTTP/1.0", host: "meta.domain.org:25654", referrer: "https://meta.domain.org/"

1.1.1.1 = my IP
0.0.0.0 = server IP

riking · September 1, 2016, 7:08pm

Uhhhhhhhhhhhh

why is nginx looking in /usr/share
and that’s the outside nginx, too

You clearly don’t have the forwarding set up right

john3 · September 1, 2016, 7:49pm

Hi,
my nginx configuration file is:

http://pastebin.com/V5cyq6Bj

in the admin settings, default logo points to “/images/…” not /usr/share/…

But still nginx points to /usr/share/…

riking · September 1, 2016, 7:51pm

… Delete everything from line 38 to the last closing brace.

john3 · September 1, 2016, 7:54pm

I removed everything below line 38, and now images work fine.

P.S: Why was "location ~* .php$ { … } " interrupting images?

@riking now, I have another problem, when I “signup” a new account, it doesn’t respond once I click “Create new account”, and it points to “http://meta.domain.org:25654/…” , as port 25654 is local on docker, I am using nginx proxy to divert port 25654 traffic to 443/80, looks like for registration purpose it does not forward properly.

error log:

2016/09/01 19:51:24 [error] 56#56: *1392 limiting requests, excess: 12.088 by zone "flood", client: 0.0.0.0, server: _, request: "POST /mini-profiler-resources/results HTTP/1.0", host: "meta.domain.org:25654", referrer: "https://meta.domain.org/"

mpalmer · September 2, 2016, 12:20am

You have got so many broken things here… my suggestion would be to burn it all down and start again, this time following the relevant howto topics for your situation.

john3 · September 2, 2016, 12:35am

Hi @mpalmer

I have already solved all the issues I have had been facing so far. All the relative topics I followed, some of them were not solving my issues, because I am using CentOS 7, and most of them are/were for Ubuntu. I am thinking to create a topic for centOS, a fully detailed topic so that the problems, I faced other may not face.

However, the last problem I encounter is port 25654 specifically during registration process! i have posted my app.yml, and nginx.conf file here, please review it.

To mention once again, whenever I create a new account, on clicking “Create new account” it redirects to “$domain:25654/users/account-created” instead of “$domain/users/account-created”, as 25654 is is acting as docker proxy, hence it ends up getting me a “not responding” error. During registration process its not forwarding to local nginx, hence end up on url ("$domain:25654/users/account-created").

mpalmer · September 2, 2016, 1:11am

You’re also not forwarding the original request IP, which is why you’re hitting rate limits.

john3 · September 2, 2016, 6:41am

@mpalmer I implemented following iptable rules by following Digital Ocean tutorial.

[root@host etc]# sudo iptables -A FORWARD -i docker0 -o eth0 -p tcp --syn --dport 25654 -m conntrack --ctstate NEW -j ACCEPT
[root@host etc]# sudo iptables -A FORWARD -i docker0 -o eth0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
[root@host etc]# sudo iptables -A FORWARD -i eth0 -o docker0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
[root@host etc]# sudo iptables -P FORWARD DROP
[root@host etc]# sudo iptables -t nat -A PREROUTING -i docker0 -p tcp --dport 25654 -j DNAT --to-destination SERVER.IP
[root@host etc]# sudo iptables -t nat -A POSTROUTING -o eth0 -p tcp --dport 25654 -d SERVER.IP -j SNAT --to-source DOCKER.IP

Not sure if I did it in the correct way, as you said but it didn’t help.

mpalmer · September 2, 2016, 6:50am

Firewall rules are a whole different thing to forwarding origin IP information through a HTTP proxy. The howtos on setting up Discourse with a reverse proxy in front (for hosting multiple websites) contains details on how to do this properly.

john3 · September 2, 2016, 9:22am

Everything works now. Finally. added 3 more rules to nginx configuration file.

my nginx.conf file now looks like:

    server {

    listen   80;
    listen [::]:80;
    server_name forum.domain.org;
    return 301 https://forum.domain.org$request_uri;
    }
    server {
    listen   443 ssl spdy;
    listen [::]:443 ssl spdy;
    server_name forum.domain.org;
    keepalive_timeout    70;
    gzip    on;
    ssl_certificate   /etc/ssl/domain.crt;
    ssl_certificate_key   /etc/ssl/domain.key;
    ssl_protocols        TLSv1.1 TLSv1.2;
    ssl_ciphers     EECDH+AESGCM:EDH+AESGCM:ECDHE-RSA-AES128-GCM-SHA256:AES256+EECDH:DHE-RSA-AES128-GCM-SHA256:AES256+EDH:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4;
    ssl_dhparam     /etc/nginx/conf.d/dhparam.pem;
    ssl_prefer_server_ciphers   on;
    ssl_session_tickets     off;
    ssl_session_cache       shared:SSL:10m;

    add_header   Strict-Transport-Security "max-age=63072000; includeSubdomains; ";
    add_header   X-Frame-Options "DENY";
    spdy_headers_comp 6;
    spdy_keepalive_timeout 300;

    access_log  /var/log/nginx/localhost.access.log;

    location / {
            rewrite         /(.*) /$1  break;
            proxy_pass      http://forum.domain.org:25654/;
            proxy_read_timeout      90;
            proxy_redirect  http://forum.domain.org:25654/ https://forum.domain.org/;

      #Adding below rules solved the problem
            proxy_set_header Host $http_host;
            proxy_http_version 1.1;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    }

I hope it can help others as well. Thank you @mpalmer @riking

P.S: I am not using any iptable rules to forward the traffic. The rules I implemented above, I have removed them.

Topic		Replies	Views
Discourse not serving pages installation	25	2406	August 4, 2020
Our discourse Images are not lightboxed support	34	3377	July 28, 2020
Discourse working with jwilder /nginx proxy & acme-companion installation	7	4239	February 9, 2022
Discourse shows server IP/localhost as user's IP installation unsupported-install	19	1558	July 12, 2022
Installing Discourse With Plesk Onyx (Ubuntu 14.04) installation unsupported-install	47	12095	October 10, 2019

Lots of problems installing Discourse under CentOS with NGINX forwarding

Related Topics