# How to bypass rate limiter when using an API key?
**URL:** https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990
**Category:** Self-hosting
**Created:** [March 5, 2015, 12:01am UTC](https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990 "2015-03-05T00:01:14Z")
**Posts on this page:** 17
**Page:** 1
### Author:  [@rcfox](https://meta.discourse.org/u/rcfox)
#### Post date: [March 5, 2015, 12:01am UTC](https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990/1 "2015-03-05T00:01:14Z")
I’ve noticed that even when using an API key, I run into both the 12 requests per second limit in nginx and the per-user post limits. It makes scripting against Discourse very difficult. Is there a way that this can be disabled for requests that include a valid API key?
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [March 5, 2015, 1:20am UTC](https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990/2 "2015-03-05T01:20:48Z")
Not really, no, since it implies nginx (the thing enforcing the limits) would be able to test the validity of the API key when it is far upstream of all that..
> [@rcfox](#):
>
> It makes scripting against Discourse very difficult
Also that’s by design, wouldn’t you say 😉
---
### Author:  [@rcfox](https://meta.discourse.org/u/rcfox)
#### Post date: [March 5, 2015, 1:52am UTC](https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990/3 "2015-03-05T01:52:36Z")
So, just spit-balling here: I don’t know very much about configuring nginx, so maybe this isn’t possible.
How about a /api/ location that would require HTTP auth using the API key, and then rewrites the URL to the normal URL, and doesn’t have a rate limit?
You could perhaps write to the auth config file when API keys are generated or revoked.
---
### Author:  [@riking](https://meta.discourse.org/u/riking)
#### Post date: [March 5, 2015, 3:47am UTC](https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990/4 "2015-03-05T03:47:32Z")
The per-user post rate limits will be disabled if the acting user is an admin.
As for requests per second, what kind of scripting are you doing?
---
### Author:  [@rcfox](https://meta.discourse.org/u/rcfox)
#### Post date: [March 5, 2015, 4:07am UTC](https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990/5 "2015-03-05T04:07:31Z")
Automated posting as users, grabbing a bunch of topics to get stats, migrating data from older systems. Stuff like that.
---
### Author:  [@riking](https://meta.discourse.org/u/riking)
#### Post date: [March 5, 2015, 4:15am UTC](https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990/6 "2015-03-05T04:15:56Z")
> [@rcfox](#):
>
> migrating data from older systems
Have you looked at the import scripts? They run as Ruby code, much more suited to one-shot creating a lot of posts.
For grabbing stats from a bunch of topics, I suggest either funneling the requests through a queue or grabbing “request tokens” from a emitter going every 1/10th second, depending on what programming language you’re using.
---
### Author:  [@dylanh724](https://meta.discourse.org/u/dylanh724)
#### Post date: [September 18, 2017, 2:30pm UTC](https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990/7 "2017-09-18T14:30:51Z")
> [@riking](#):
>
> The per-user post rate limits will be disabled if the acting user is an admin.
This is old, but … it’s pretty relevant. I was looking for the same thing. I’m using the API to report bad players for a game. This shouldn’t be limited. However, our BOT account doesn’t need to access any admin settings, just simply post. I read that you need to be an ADMIN to do this:
This is pretty nasty … why would I have to make the bot account an _admin_ to bypass rate limitations just for posting? In 2017, is there a better way to do this?
---
### Author:  [@riking](https://meta.discourse.org/u/riking)
#### Post date: [September 18, 2017, 4:04pm UTC](https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990/8 "2017-09-18T16:04:03Z")
You could simply increase the problematic rate limits in the site settings.
This isn’t really a problem that has come up in the past - usually, bots that needed to bypass rate limits also were doing something else that required either admin or acting as arbitrary users.
---
### Author:  [@dylanh724](https://meta.discourse.org/u/dylanh724)
#### Post date: [September 18, 2017, 4:06pm UTC](https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990/9 "2017-09-18T16:06:56Z")
> [@riking](#):
>
> You could simply increase the problematic rate limits in the site settings.
Actually, I did try adding my bot admin to test it – I am still running into rate limitations, every now and then. Do you happen to know anything about this? While debugging, I’m testing once every few mins and my admin account is still getting rate-limited 422. So strange ..
EDIT: Strange, the rate limiters seem to default to 5~15 **SECONDS** (I figured minutes). Something is wrong
Either way, shouldn’t an admin bypass these?
---
### Author:  [@riking](https://meta.discourse.org/u/riking)
#### Post date: [September 18, 2017, 4:08pm UTC](https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990/10 "2017-09-18T16:08:53Z")
It’s probably the nginx web server rate limits - it has additional short-term request-counted rate limits. The Discourse rate limits are there to protect against bad behavior and generally expire per day. The nginx ratelimits are there to protect against flooding.
As you just noticed in your edit, the nginx ratelimits are on the order of seconds and minutes.
---
### Author:  [@dylanh724](https://meta.discourse.org/u/dylanh724)
#### Post date: [September 18, 2017, 4:09pm UTC](https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990/11 "2017-09-18T16:09:55Z")
Ahh that makes more sense.
I wonder if there’s a way to delay requests instead of just 422 them
---
### Author:  [@riking](https://meta.discourse.org/u/riking)
#### Post date: [September 18, 2017, 4:11pm UTC](https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990/12 "2017-09-18T16:11:30Z")
Yes, there is: have a model of the rate limits in your client and force it to sleep if you are about to violate them! 🙂
Also, check the code for whether or not mods bypass the rate limits because I don’t actually remember.
Also, I don’t believe there is a once/x ratelimit on editing your post - maybe you could make the bot edit posts if data comes in too fast?
---
### Author:  [@rcfox](https://meta.discourse.org/u/rcfox)
#### Post date: [September 18, 2017, 5:01pm UTC](https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990/13 "2017-09-18T17:01:10Z")
It has been a long while, but I think I ended up turning off nginx’s rate limiting, and enabled rate limiting in haproxy (which I already had running in front of Discourse.) In haproxy, I was able to whitelist the IP for my bot.
---
### Author:  [@DeanMarkTaylor](https://meta.discourse.org/u/DeanMarkTaylor)
#### Post date: [September 18, 2017, 5:06pm UTC](https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990/14 "2017-09-18T17:06:54Z")
> [@riking](#):
>
> have a model of the rate limits in your client and force it to sleep if you are about to violate them!
All HTTP clients should really use “Truncated Exponential Backoff” or something similar:
> [Exponential Backoff](http://en.wikipedia.org/wiki/Exponential_backoff) is an algorithm that retries requests to the server based on certain status codes in the server response. The retries exponentially increase the waiting time up to a certain threshold. The idea is that if the server is down temporarily, it is not overwhelmed with requests hitting at the same time when it comes back up.
It’s quite common to find an open source implementation in all of Google’s API libraries.
Some example implementations listed here for Google Storage calls:
> **[Retry strategy | Cloud Storage | Google Cloud Documentation](https://docs.cloud.google.com/storage/docs/retry-strategy)**
---
### Author:  [@system](https://meta.discourse.org/u/system)
#### Post date: [October 4, 2019, 4:54pm UTC](https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990/15 "2019-10-04T16:54:19Z")
---
### Author:  [@system](https://meta.discourse.org/u/system)
#### Post date: [June 4, 2023, 8:01pm UTC](https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990/16 "2023-06-04T20:01:35Z")
---
### Author:  [@JammyDodger](https://meta.discourse.org/u/JammyDodger)
#### Post date: [June 4, 2023, 8:02pm UTC](https://meta.discourse.org/t/how-to-bypass-rate-limiter-when-using-an-api-key/25990/17 "2023-06-04T20:02:24Z")