Available settings for global rate limits and throttling

[Discourse]

Discourse ships with 3 different global rate limits that can be configured by site admins.

Global per-ip rate limits

These limits apply to every unique IP address that hits the Discourse application. (files that are served directly from the filesystem or the CDN are excluded)

By default this rate limit is enabled, you may disable it or set it to a reporting mode.

DISCOURSE_MAX_REQS_PER_IP_MODE : default block, this rate limit applies out of the box. (other options are warn, warn+block, and none)

DISCOURSE_MAX_REQS_PER_IP_PER_MINUTE: number of requests per IP per minute (default is 200)

DISCOURSE_MAX_REQS_PER_IP_PER_10_SECONDS: number of requests per IP per 10 seconds (default is 50)

DISCOURSE_MAX_ASSET_REQS_PER_IP_PER_10_SECONDS: number of asset (avatars/css) requests per IP per 10 seconds (default is 200)

DISCOURSE_MAX_REQS_RATE_LIMIT_ON_PRIVATE: should the rate limit apply to private IPs accessing Discourse? default is false.

DISCOURSE_SKIP_PER_IP_RATE_LIMIT_TRUST_LEVEL: use per user rate limits vs IP rate limits for users with this trust level or more (default 1)

User API rate limits

The mobile applications acquire a user API key per device to access Discourse on behalf of a user (using an open protocol). These API keys are very tightly limited.

DISCOURSE_MAX_USER_API_REQS_PER_MINUTE: default 20
DISCOURSE_MAX_USER_API_REQS_PER_DAY: default 2880

Admin API rate limits

The administrative API keys can be generated via the yoursite.com/admin/api/keys page. These keys can operate on behalf of users, but require administrative privileges to generate. There is a limit of 60 requests per minute, shared between all keys.

Self-hosted users can change this in their app.yml file. Hosted customers will need to contact their hosting provider.

DISCOURSE_MAX_ADMIN_API_REQS_PER_MINUTE : 60

Data Explorer Plugin API rate limits

DISCOURSE_MAX_DATA_EXPLORER_API_REQ_MODE: default warn , this rate limit applies out of the box. (other options are block , warn+block , and none )

DISCOURSE_MAX_DATA_EXPLORER_API_REQS_PER_10_SECONDS: 2

Note: The requests made via the Data Explorer UI do not count towards the rate limit.

What should I do if I hit a rate limit and get throttled?

If you are consuming the API programmatically and receive back a 429 status code throttle reply you should respect it and slow down.

As an end user you should not really experience rate limits if you do, slow down. You could trigger it by opening 50 tabs real quick or doing something like that.

Firewall and proxy warning!

If you are running a reverse proxy which is mis-configured Discourse may think all the requests are coming from a single IP address, it is very likely you will hit rate limits early. Be sure to configure your reverse proxy to forward the IP correctly.

How do I amend these limits?

To amend the limits add the desired change into your app.yml file in the env section.

If you are hosted by Discourse, and on an Enterprise plan, contact team@discourse.org it you need to adjust any of these limits.

Global Rate Limits are not adjustable on starter, pro, or business plans.

[pfaffman (Jay Pfaffman)]

It looks to me like if you have the web.ratelimited.template.yml installed, then these don’t matter since stuff gets rate limited by NGINX before it gets to Discourse, right?

That’s how it seems from my nginx logs.

My short term solution is to add my IP address to the local IP list, so it gets past NGINX. I guess the thing to do is remove the ratelimited template to make these have any meaning?

[guidoleenders]

Regarding the following text:

Firewall and proxy warning!

If you are running a reverse proxy which is mis-configured Discourse may think all the requests are coming from a single IP address, it is very likely you will hit rate limits early. Be sure to configure your reverse proxy to forward the IP correctly.

In our setup we tunnel all API calls through a proxy. This proxy handles authentication and a lot of other stuff before sometimes querying Discourse.

What is the recommended way (specific header?) to forward the IP adress of the original requester?

Related

Rate Limiting w/ Reverse Proxy

Had a question on what header is being used for rate-limiting. For context we have an nginx.conf w/ set_real_ip and we’re using a provider that sends us traffic with the client’s actual IP in Some-Client-IP header. sendfile on; set_real_ip_from ...; set_real_ip_from ...; set_real_ip_from ...; set_real_ip_from ...; real_ip_header Some-Client-IP; real_ip_recursive on; Some-Client-IP is what we get from upstream w/ the actual user’s IP. In the current discourse.conf in conf.d for nginx. We ha…

[Richie (Richie Rich)]

Could someone help me understand this config setting please?

@Discourse

DISCOURSE_SKIP_PER_IP_RATE_LIMIT_TRUST_LEVEL: use per user rate limits vs IP rate limits for users with this trust level or more (default 1)

Am I right in thinking that this is already set to 1 and that the rate limits are by default already set to per-user rather than per-IP-address (if the user is TL1 or higher) ?

[Ethsim2 (Ethan)]

@j.jaffeux

where is the 429?

it actually didn’t exist to cause me to specify one or more of these settings.

my script was always respecting the Retry-After header, as we can see from original line 101;

retry_after = r.headers.get("Retry-After")

however this header was becoming unnecessarily large as i schedule different service units for this same script to run at different times on a day.

Therefore, multiples of some of these settings from the default; improved email deliverability via mail-receiver which remains important.