Rate limit errors although IP is whitelisted

Support Self-hosting

AI-generated summary

Users are experiencing rate limit errors despite having their IP whitelisted. sithmein has configured Nginx with a whitelisted IP, but still receives 429 errors when making requests from that IP. MrBuBBLs points out that Discourse has its own rate limiting at the application level, and MrBuBBLs discovers that DISCOURSE_MAX_REQS_PER_IP_EXCEPTIONS might be the solution. However, Alex_Bourge has tried various configuration settings, including DISCOURSE_MAX_REQS_PER_IP_EXCEPTIONS, but still encounters rate limit errors, indicating that the issue is with the Discourse RateLimiter.

1

We are using Discourse in a Docker container. We have rate limits set up with a whitelisted IP. However, we are still getting rate limit errors when making requests from that IP. I’m pretty sure the nginx configuration is correct, here is how it looks:

geo $limit {
    default 1;
    1.1.1.1 0; # not the real IP
}

map $limit $limit_key {
    0 "";
    1 $binary_remote_addr;
}

limit_req_zone $limit_key zone=flood:10m rate=12r/s;
limit_req_zone $limit_key zone=bot:10m rate=200r/m;
limit_req_status 429;
limit_conn_zone $limit_key zone=connperip:10m;
limit_conn_status 429;

...
  location @discourse {
    limit_conn connperip 20;
    limit_req zone=flood burst=12 nodelay;
    limit_req zone=bot burst=100 nodelay;
   ...
  }

nginx is configured to log an error in case a rate limit has been reached and we are indeed seeing some log message - but none from the whitelisted IP. Still we are getting tons of 429 when making requests from the whitelisted IP. The request URLs are user profiles (e.g. /users/foo.json). Is there some kind of rate limit in Discourse itself?

2

Hi there, Nginx is indeed managing rate limiting as you stated but Discourse also has it’s own way to manage rate limiting at application level. Sam has an interesting piece on this :

The only thing I’m wondering is if we can indeed rate limit with exceptions (i.e. whitelist IPs through those rules). I’m still searching for a way to do this… :sweat_smile:

3

Okay I think I’m onto something :

Maybe DISCOURSE_MAX_REQS_PER_IP_EXCEPTIONS might be just what we needed :grin:

4

I, too, am hitting a rate limit of 60 calls to the API per minute which I cannot seem to alleviate. I’ve set all of these:

      - DISCOURSE_MAX_REQS_PER_IP_MODE=none
      - DISCOURSE_MAX_USER_API_REQS_PER_MINUTE=20000
      - DISCOURSE_MAX_USER_API_REQS_PER_DAY=30000
      - DISCOURSE_MAX_ADMIN_API_REQS_PER_MINUTE=20000
      - DISCOURSE_MAX_REQS_PER_IP_PER_MINUTE=20000
      - DISCOURSE_MAX_REQS_PER_IP_PER_10_SECONDS=2000
      - DISCOURSE_MAX_ASSET_REQS_PER_IP_PER_10_SECONDS=2000
      - DISCOURSE_SKIP_PER_IP_RATE_LIMIT_TRUST_LEVEL=0
      - DISCOURSE_MAX_ADMIN_API_REQS_PER_KEY_PER_MINUTE=20000
      - DISCOURSE_MESSAGE_BUS_MAX_BACKLOG_SIZE=1000
      - DISCOURSE_MAX_REQS_PER_IP_EXCEPTIONS=....

I know it’s not nginx because I’m getting the " You’ve performed this action too many times" message, which means the Discourse RateLimiter is handling it.

Did you find a solution?