One of my forums went down yesterday, first with a blank page then displaying 429’s. Sam thinks it may be possible that my proxy is not configured correctly - anyone know what the best way is to ascertain whether IP addresses are being handed over correctly to Discourse? I’m pretty sure it is set up correctly (as I have option forwardfor set in HAProxy and users’ IP addresses correctly show up in the admin CP) but I’d like to check Discourse’s end just in case.

Yay!! It’s done! IP’s are now showing correctly in the access log :smiley: For some reason the spacing messed up in the snippet above, here it is for anyone who might need it in future: run: - replace: filename: /etc/nginx/conf.d/discourse.conf from: "types {" to: | set_real_ip_from 172.17.0.0/24; real_ip_header X-Forwarded-For; real_ip_recursive on; types { Thank you @pfaffman and @sam and @mpalmer for your help :blue_heart:

[image] AstonJ: users’ IP addresses correctly show up in the admin CP That’s the important bit. If the Discourse admin panel is listing users’ IP addresses correctly, then the IP addresses are getting into Discourse.

Thanks Matt. I wonder what caused the issues yesterday then - reading this it appears that rate limits are applied per user or per IP and so it was odd that the site was inaccessible to everyone (I also tried using various IPs). The forum has been getting busier lately - we’re now serving around 600K pages a month, but I wouldn’t have thought that would trigger this in itself. Would the rate limiter only show a 429 to the IPs affected, or everyone? Any other ideas what it might have been or how to troubleshoot this?

If nginx’s set_real_ip_from is misconfigured, though, won’t Discourse not be able to get the real IP, either?

It actually will, cause as long as Discourse sees the HTTP X-Forwarded-For header it is fine to set the IP. So you can have a state where IP looks good in Discourse, but NGINX is not “setting real IP” for rate limiting purposes.

Thanks Sam. So do I basically need to do what’s in this post? Or is there a simpler way? If following that post, I guess for me that file would read something like the following? run: - replace: filename: "/etc/nginx/conf.d/discourse.conf" from: /^add_header Strict-Transport-Security 'max-age=31536000';$/ to: | add_header Strict-Transport-Security 'max-age=31536000'; # Cloudflare set_real_ip_from my.server.ip; real_ip_header CF-Connecting-IP; (It’s a dedicated server running a single IP address.)

If nginx isn’t stripping untrusted XFF, and Discourse is seeing a request from 127.0.0.1 and saying “I trust that IP to give me legit XFF headers”, doesn’t that imply that source IP can be spoofed?

Possibly, we should test, maybe our nginx template is lacking

i actually just got this error. working through it now… we’re back up. that seemed totally random.

Coming back to this Topic as the other one is slightly different. Do we still need to be doing something like in the post above? Looking through the /etc/nginx/conf.d/discourse.conf file there seems to be no mention of set_real_ip_from (searching the Discourse repo for the same yields no results either). From the discourse.conf file these seem most relevant: (mentions of IP) # This big block is needed so we can selectively enable # acceleration for backups and avatars # see note about repetition above location ~ ^/(letter_avatar/|user_avatar|highlight-js|stylesheets|favicon/proxied|service-worker) { proxy_set_header Host $http_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Request-Start "t=${msec}"; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $thescheme; # we need buffering off for message bus location /message-bus/ { proxy_set_header X-Request-Start "t=${msec}"; proxy_set_header Host $http_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $thescheme; proxy_http_version 1.1; proxy_buffering off; proxy_pass http://discourse; break; } # auth_basic on; # auth_basic_user_file /etc/nginx/htpasswd; location ~* (assets|plugins|uploads)/.*\.(eot|ttf|woff|woff2|ico)$ { expires 1y; add_header Cache-Control public,immutable; add_header Access-Control-Allow-Origin *; } location = /srv/status { access_log off; log_not_found off; proxy_set_header Host $http_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Request-Start "t=${msec}"; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $thescheme; proxy_pass http://discourse; break; } Which appear to be setting the correct remote address :confused: For completeness I received a couple of 429s after a peak in traffic a couple of days ago, and looking at my app.yml I found that the set_real_ip_from my.server.ip; and real_ip_header CF-Connecting-IP; lines were commented out… but it doesn’t look like they do anything anyway? Can you remember what the outcome of your investigations were after? Currently I am recommending adding those lines to the app.yml in the following guide: Set up Discourse on a server with existing Apache sites (and so may be giving bad advice to others?)

Troubleshooting a 429 (rate limit)

Product Support Self-hosting

sam (Sam Saffron) February 20, 2018, 11:46pm 4

Some of the throttling happens in NGINX though not in Discourse. So you need NGINX to have a set_real_ip_from going.

Topic		Replies	Views
Problem Error 429 with Reverse proxy Self-hosting	4	1278	April 30, 2024
Rate Limiting w/ Reverse Proxy Self-hosting	3	819	November 16, 2021
Discourse shows server IP/localhost as user's IP Self-hosting unsupported-install	18	2552	July 12, 2022
All IPs recorded as 127.0.0.1 Self-hosting	4	2890	June 8, 2017
429 Too Many Requests - IP logs disabled - discourse can see 127.0.0.1 only Product Support	1	691	March 9, 2022

Troubleshooting a 429 (rate limit)

Related topics