Rate limit errors although IP is whitelisted

Installation
1

We are using Discourse in a Docker container. We have rate limits set up with a whitelisted IP. However, we are still getting rate limit errors when making requests from that IP. I’m pretty sure the nginx configuration is correct, here is how it looks:

geo $limit {
    default 1;
    1.1.1.1 0; # not the real IP
}

map $limit $limit_key {
    0 "";
    1 $binary_remote_addr;
}

limit_req_zone $limit_key zone=flood:10m rate=12r/s;
limit_req_zone $limit_key zone=bot:10m rate=200r/m;
limit_req_status 429;
limit_conn_zone $limit_key zone=connperip:10m;
limit_conn_status 429;

...
  location @discourse {
    limit_conn connperip 20;
    limit_req zone=flood burst=12 nodelay;
    limit_req zone=bot burst=100 nodelay;
   ...
  }

nginx is configured to log an error in case a rate limit has been reached and we are indeed seeing some log message - but none from the whitelisted IP. Still we are getting tons of 429 when making requests from the whitelisted IP. The request URLs are user profiles (e.g. /users/foo.json). Is there some kind of rate limit in Discourse itself?

2 Likes
2

Hi there, Nginx is indeed managing rate limiting as you stated but Discourse also has it’s own way to manage rate limiting at application level. Sam has an interesting piece on this :

The only thing I’m wondering is if we can indeed rate limit with exceptions (i.e. whitelist IPs through those rules). I’m still searching for a way to do this… :sweat_smile:

3

Okay I think I’m onto something :

Maybe DISCOURSE_MAX_REQS_PER_IP_EXCEPTIONS might be just what we needed :grin:

1 Like
4

I, too, am hitting a rate limit of 60 calls to the API per minute which I cannot seem to alleviate. I’ve set all of these:

      - DISCOURSE_MAX_REQS_PER_IP_MODE=none
      - DISCOURSE_MAX_USER_API_REQS_PER_MINUTE=20000
      - DISCOURSE_MAX_USER_API_REQS_PER_DAY=30000
      - DISCOURSE_MAX_ADMIN_API_REQS_PER_MINUTE=20000
      - DISCOURSE_MAX_REQS_PER_IP_PER_MINUTE=20000
      - DISCOURSE_MAX_REQS_PER_IP_PER_10_SECONDS=2000
      - DISCOURSE_MAX_ASSET_REQS_PER_IP_PER_10_SECONDS=2000
      - DISCOURSE_SKIP_PER_IP_RATE_LIMIT_TRUST_LEVEL=0
      - DISCOURSE_MAX_ADMIN_API_REQS_PER_KEY_PER_MINUTE=20000
      - DISCOURSE_MESSAGE_BUS_MAX_BACKLOG_SIZE=1000
      - DISCOURSE_MAX_REQS_PER_IP_EXCEPTIONS=....

I know it’s not nginx because I’m getting the " You’ve performed this action too many times" message, which means the Discourse RateLimiter is handling it.

Did you find a solution?

2 Likes