April 10, 2023, 4:16pm
I occasionally see forums that I run show a splash screen that never stops loading.
It seems to happen after rebuilds or if I haven’t visited the forum for a long time. It doesn’t seem associated with different browsers or forum plugins. And it happens to some people but not others.
A hard refresh (Ctrl-F5 on Windows or Cmd-Shift-R on Mac) seems to fix it every time. My guess is that this is a caching issue with the CDN, but I’m not sure. Anyone else experiencing something like this?
Console when page isn’t loading
Console after a hard refresh
Every single one of these errors is due to the request getting rate limited. Discourse can’t run if its assets can’t load.
You should check:
is the correct end user IP address being passed to the backend?
is rate limiting set up correctly? (in general)
what entity is performing the rate limiting here?
April 10, 2023, 4:56pm
For this server in question, there is no reverse proxy that would be hiding all user traffic behind one IP. The rate limits set inside Discourse are the defaults.
I suppose the CDN could be doing some proxying? Is there a recommended way to confirm that end user IPs are being passed to the backend? I don’t see anything about too many requests in the logs.
This is literally the job of the CDN, so yes, you need to make sure that the correct end user IP is maintained through the request chain.
I’m guessing you haven’t done that, and all of the requests from the CDN are being tallied against the request rate limit of the CDN POPs rather than the end users.
You can see the CDN is returning the 429 here, but you’ll need to investigate your specific setup to determine who is making the decision to return that error (i.e. the proxy or the real server).
April 14, 2023, 9:53pm
KeyCDN is the CDN in question. I tried enabling
OriginShield and adding a
template.yml (in the style of
cloudflare.template.yml) to my
app.yml, but I was still getting 429s.
Rather than keep futzing with it, I’ve switched to BunnyCDN and that seems to be working better.
I’ve put the template file below in case it helps anyone else.
# Add list of keycdn ips
curl -s 'https://www.keycdn.com/shield-prefixes.json' | \
python3 -c "import sys, json; print('\n'.join(json.load(sys.stdin)['prefixes']))" > /tmp/keycdn-ips
# Make into nginx commands and escape for inclusion into sed append command
CONTENTS=$(</tmp/keycdn-ips sed 's/^/set_real_ip_from /' | sed 's/$/;/' | tr '\n' '\\' | sed 's/\\/\\n/g')
echo keycdn IPs:
echo $(echo | sed "/^/a $CONTENTS")
# Insert into discourse.conf
sed -i "/sendfile on;/a $CONTENTS\nreal_ip_header X-Forwarded-For;\nreal_ip_recursive on;" /etc/nginx/conf.d/discourse.conf
# Clean up
- exec: "/tmp/add-keycdn-ips"
- exec: "rm /tmp/add-keycdn-ips"
May 14, 2023, 9:53pm
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.