So here is my issue. I have a requirement to essentially seed a Discourse production instance with around 200 posts at a time from a plugin admin action. These posts will be ‘created by’ 1 of 10 different regular users. The reason it’s a plugin action is because the users of this particular instance have a team of moderators they want to train up, and wanted some test posts to train them on, and the ability to seed more when they need them. I got this working fine by passing skip_validations: true to PostCreator.new, however now there’s a requirement that some of the created posts are also flagged. I’m using PostActionCreator.create to flag some of these posts, but now I’m getting hit by the rate limiter here: discourse/lib/post_action_creator.rb at ad7a13921f2af8c792530c84386b64911c8e7ea2 · discourse/discourse · GitHub I first attempted to disable the RateLimiter but that was causing my action to crash the server process eventually, possibly when I was trying to turn it back on, and then I realised it probably wasn’t a good idea anyway. So my question is, is there a better way to bypass the rate limiter when running some arbitrary code i.e. something like: RateLimiter.bypass do # run some code not affected by the rate limiter end Or do I need to basically just need to copy most of what the PostActionCreator is doing but leave out the troublesome line? Any help would be greatly appreciated! I’m still absorbing a lot of the Discourse code so I’m aware I’ve probably missed something that makes this a lot easier!

What I’d likely do is run a script in the Rails console. Have you checked out: Available settings for global rate limits and throttling? It looks like you can change those rate limits.

Did you look at the link about changing the rate limits?

Did you figure this out? Modfying the yaml file is not ideal as it requires a rebuild. I’m not familiar enough with rails/discourse to figure out how to disable it temporarily in the console.

what is your use case? What problem are you trying to solve that rate limits are standing in the way of?

Trying to import 60k users via API as fast as possible without doing a rebuild. I also tried on a test installation to raise the ADMIN API limit via YAML, but it didn’t seem to work but maybe I did something wrong. (I’m running imports on the container so the http throttling shouldn’t apply).

This isn’t very helpful, but what I’d do is create the users in rails rather than via the API. discourse/script/import_scripts/csv_importer.rb at main · discourse/discourse · GitHub could be a start. But you can go into the container and edit /var/www/discourse/config/discourse.conf and set those variables there and then do an sv restart unicorn (or maybe sv reload unicorn). You’ll probably want to do something like apt-get update; apt-get install -y vim to have an editor.

Strange. When I view the conf file the new limit is already set. So the YAML update worked: max_admin_api_reqs_per_key_per_minute = '6000' But it doesn’t seem to work. There’s still a 60 per minute limitation. When I throttle requests to 60 per minute, most go through but due to jitter a few still trigger the rate limiter: {'success': True, 'active': True, 'message': 'Your account is activated and ready to use.', 'user_id': 3596} {'success': False, 'message': ' New registrations are not allowed from your IP address (maximum limit reached). Contact a staff member.', 'errors': {'ip_address': ['New registrations are not allowed from your IP address (maximum limit reached). Contact a staff member.']}, 'values': {'name': None, 'username': 'asdfd', 'email': 'user@domain.com'}, 'is_developer': False} {'success': True, 'active': True, 'message': 'Your account is activated and ready to use.', 'user_id': 3597}

I wonder if it’s an issue with the quotes? You might check SiteSettings.max_admin_api_reqs_per_key_per_minute and see if it’s an integer

in default_current_user_provider.rb (the only file I found that seemed to reference this value, it has: limit = GlobalSetting.max_admin_api_reqs_per_minute.to_i so it does seem to convert it. I even tried replacing with a number in the code.

Any way to turn off RateLimiter temporarily for bulk creation by admin?

Development

jamesmarkcook (James Cook) January 16, 2020, 5:46pm 3

This isn’t really a path I want to go down as I want it to initiated by users rather than them relying on me to run a script in the console.

Topic		Replies	Views
How do I avoid rate limiting when importing a bunch of data? Development rest-api , rails-console	13	1349	July 6, 2023
Unable to disable the rate limit Self-hosting	2	715	August 2, 2022
Rate limits for API users Development rest-api	11	3393	March 17, 2020
How to bypass rate limiter when using an API key? Self-hosting	13	6558	September 18, 2017
Changing/removing API rate limit with category creation Development rest-api	16	4364	July 31, 2019

Any way to turn off RateLimiter temporarily for bulk creation by admin?

Related topics