429 Too Many Requests on /admin/upgrade page

So I’ve been seeing this for a few weeks now, but thought I’d see what configuration I may need to tweak to get this working again.

Here is the admin/upgrade page

It always hangs on docker_manager with the following error

Failed to load resource: the server responded with a status of 429 (OK)
docker-manager-vendor-e62406a2f5676e280b40622751069e76.js:11 Error: OK
    at o (docker-manager-vendor-e62406a2f5676e280b40622751069e76.js:11)
    at Object.e.default.trigger (docker-manager-vendor-e62406a2f5676e280b40622751069e76.js:16)
    at h._onerror (docker-manager-vendor-e62406a2f5676e280b40622751069e76.js:16)
    at d (docker-manager-vendor-e62406a2f5676e280b40622751069e76.js:16)
    at docker-manager-vendor-e62406a2f5676e280b40622751069e76.js:11
    at r.invoke (docker-manager-vendor-e62406a2f5676e280b40622751069e76.js:4)
    at Object.r.flush (docker-manager-vendor-e62406a2f5676e280b40622751069e76.js:4)
    at Object.n.flush (docker-manager-vendor-e62406a2f5676e280b40622751069e76.js:4)
    at Object.a.end (docker-manager-vendor-e62406a2f5676e280b40622751069e76.js:4)
    at Object.a.run (docker-manager-vendor-e62406a2f5676e280b40622751069e76.js:4)

The 429 refers to https://sandbox.cpradio.org/admin/docker/latest?path=%2Fvar%2Fwww%2Fdiscourse%2Fplugins%2Fdocker_manager&version=549b1d8&branch=master, which works fine if you go to it directly… :confused:

I’m not using Cloudflare, just pure Discourse. No Nginx setup either. So I’m not sure why the 429 status is occuring…

Could be standard rate limiting stuff. In the network tab, you should be able to see more details about that 429 since we send the description of the exception back to the client.

The response is empty.

The nginx server must be in reference to the docker instance, as I know I’m not running it outside of the docker container.

So with it possibly being a ratelimit, should that be excluded for Admins, especially on the Upgrade page? Or provide a way to have it redo the check for a given package/component?

As the way it sits, I can’t do an upgrade via the web anymore (which isn’t critical to me as it is a ‘sandbox’ instance).

2 Likes

Right. We add these rate limits to NGINX when it’s installed using the discourse_docker image

https://github.com/discourse/discourse_docker/blob/master/templates/web.ratelimited.template.yml

So I guess you can’t have more than 11 plugins installed :wink:

1 Like

Fair enough. Awesome. That gives me a place to look for now. So I’m hitting an edge case (that’s good enough for me). :slight_smile:

1 Like

FYI, I updated the requests per second and that solved my issue of using the web interface for upgrades. Since it is a sandbox, I don’t mind doing that, realizing that may not be recommended for typical production installs.

1 Like

Hm, I don’t understand how this is the solution other than it identifies the cause of the problem. Could you explain how I can have more than 11 plugins installed?

(Strangely, the rate limit doesn’t always seem to trigger. Perhaps because update requests are not re-sent within a certain time period so that when I try again after a short while, the remaining requests can get sent without triggering the rate limit?)

1 Like

The correct solution would be to have the page make fewer requests - if that’s impossible (I forget how the plugin has been built) then it should be throttled so that only a few plugins are checked simultaneously.

3 Likes

So you’re saying the docker-manager plugin needs to be fixed and there’s nothing I can do as a user? I’m sure I’m not the only one with more than 11 plugins installed and even for me it has been working most of the time. I don’t quite understand why I sometimes get the 429 error and sometimes I don’t. And when I get it, it sometimes works when I just re-try immediately. Sometimes it works after re-trying after a while and sometimes I just can’t get it to work. (And by “work” I mean get all plugins checked so that the “Update to latest version” button turns blue and becomes clickable.)

There is no workaround right now unfortunately :frowning:

The reason you’re seeing it happen sometimes and not others has to do with the speed at which the server and your internet connection can respond, which is variable. If it happens too fast, you will be throttled.

2 Likes

So my server/internet connection is too fast? Well, at least that’s good news :wink:

I’ve updated docker_manager to now only check for plugins sequentially. It’s slightly slower but it will prevent this bug which I consider quite severe:

https://github.com/discourse/docker_manager/commit/146e2de6622d794e70461c7e21e842a6fb66e5e2

9 Likes