Splash screen stuck until hard refresh

[yanokwa (Yaw Anokwa)]

I occasionally see forums that I run show a splash screen that never stops loading.

It seems to happen after rebuilds or if I haven’t visited the forum for a long time. It doesn’t seem associated with different browsers or forum plugins. And it happens to some people but not others.

A hard refresh (Ctrl-F5 on Windows or Cmd-Shift-R on Mac) seems to fix it every time. My guess is that this is a caching issue with the CDN, but I’m not sure. Anyone else experiencing something like this?

Console when page isn’t loading

Console after a hard refresh

[supermathie (Michael Brown)]

Every single one of these errors is due to the request getting rate limited. Discourse can’t run if its assets can’t load.

You should check:

• is the correct end user IP address being passed to the backend?
• is rate limiting set up correctly? (in general)
• what entity is performing the rate limiting here?

[yanokwa (Yaw Anokwa)]

For this server in question, there is no reverse proxy that would be hiding all user traffic behind one IP. The rate limits set inside Discourse are the defaults.

I suppose the CDN could be doing some proxying? Is there a recommended way to confirm that end user IPs are being passed to the backend? I don’t see anything about too many requests in the logs.

[supermathie (Michael Brown)]

@yanokwa

For this server in question, there is no reverse proxy

@yanokwa

I suppose the CDN could be doing some proxying?

This is literally the job of the CDN, so yes, you need to make sure that the correct end user IP is maintained through the request chain.

I’m guessing you haven’t done that, and all of the requests from the CDN are being tallied against the request rate limit of the CDN POPs rather than the end users.

You can see the CDN is returning the 429 here, but you’ll need to investigate your specific setup to determine who is making the decision to return that error (i.e. the proxy or the real server).

[yanokwa (Yaw Anokwa)]

KeyCDN is the CDN in question. I tried enabling OriginShield and adding a template.yml (in the style of cloudflare.template.yml) to my app.yml, but I was still getting 429s.

Rather than keep futzing with it, I’ve switched to BunnyCDN and that seems to be working better.

I’ve put the template file below in case it helps anyone else.

keycdn.template.yml

run:
  - file:
      path: /tmp/add-keycdn-ips
      chmod: +x
      contents: |
        #!/bin/bash -e
        # Add list of keycdn ips
        curl -s 'https://www.keycdn.com/shield-prefixes.json' | \
          python3 -c "import sys, json; print('\n'.join(json.load(sys.stdin)['prefixes']))" > /tmp/keycdn-ips

        # Make into nginx commands and escape for inclusion into sed append command
        CONTENTS=$(</tmp/keycdn-ips sed 's/^/set_real_ip_from /' | sed 's/$/;/' | tr '\n' '\\' | sed 's/\\/\\n/g')
        
        echo keycdn IPs:
        echo $(echo | sed "/^/a $CONTENTS")
        # Insert into discourse.conf
        sed -i "/sendfile on;/a $CONTENTS\nreal_ip_header X-Forwarded-For;\nreal_ip_recursive on;" /etc/nginx/conf.d/discourse.conf
        # Clean up
        rm /tmp/keycdn-ips

  - exec: "/tmp/add-keycdn-ips"
  - exec: "rm /tmp/add-keycdn-ips"