Discourse rate-limiting behind a reverse proxy: my best guess (tested once)
I think I figured that out, and I deployed this and it seems to work, but I’ve learned half of this in the last 10 minutes and tested it on one instance, so beware. It also seems that moving the rate-limiting in the reverse-proxy could be more efficient, but that’s not a change I’d want to do live and I’m not sure that matters.
The change to the template (/var/discourse/templates/web.ratelimited.template.yml) would be
-limit_req_zone $binary_remote_addr zone=flood:10m rate=$reqs_per_secondr/s;
-limit_req_zone $binary_remote_addr zone=bot:10m rate=$reqs_per_minuter/m;
+limit_req_zone $http_x_forwarded_for zone=flood:100m rate=$reqs_per_secondr/s;
+limit_req_zone $http_x_forwarded_for zone=bot:100m rate=$reqs_per_minuter/m;
With this change, the rate limiting uses
X-Forwarded-For as key in the rate limiting hashtable, rather than the IP addresses of the direct HTTP client in binary form. References: rate limiting docs, http_* variable docs, your reverse proxy configuration, and docs for
$proxy_add_x_forwarded_for, since this is used to set
Note I increased the size of the hash table out of caution (probably overly much), because the keys are bigger (a list of IP addresses in text form, rather than one binary one) and the docs mention the key size matters (Module ngx_http_limit_req_module). By doing the change on the reverse proxy you could avoid that.
Rebuilding the instance after you start getting a usage burst is not perfect, so I also did the change live. I don’t recommend this unless you know what you’re doing, and I’m not sure I knew it myself. In any case, also modify the template, if you don’t want to destroy the change next time you rebuild.
(In particular, I needed this after telling ~400 people in person to register — all students of a 1st year CS course).
Since when you’re doing this you might not be able to afford rebuilding the instance, I also did the change live on the instance.
# docker exec -it info1-discourse env TERM=xterm /bin/bash
# vi /etc/nginx/conf.d/discourse.conf
limit_req_zone $binary_remote_addr zone=flood:10m rate=12r/s;
limit_req_zone $binary_remote_addr zone=bot:10m rate=200r/m;
limit_req_zone $http_x_forwarded_for zone=flood:100m rate=12r/s;
limit_req_zone $http_x_forwarded_for zone=bot:100m rate=200r/m;
reload nginx with
nginx -s reload
and retest your website. EDIT: fixed typo in code, sorry!