有办法让管理员暂时关闭 RateLimiter 以进行批量创建吗?

So here is my issue. I have a requirement to essentially seed a Discourse production instance with around 200 posts at a time from a plugin admin action. These posts will be ‘created by’ 1 of 10 different regular users. The reason it’s a plugin action is because the users of this particular instance have a team of moderators they want to train up, and wanted some test posts to train them on, and the ability to seed more when they need them.

I got this working fine by passing skip_validations: true to PostCreator.new, however now there’s a requirement that some of the created posts are also flagged.

I’m using PostActionCreator.create to flag some of these posts, but now I’m getting hit by the rate limiter here: https://github.com/discourse/discourse/blob/ad7a13921f2af8c792530c84386b64911c8e7ea2/lib/post_action_creator.rb#L69

I first attempted to disable the RateLimiter but that was causing my action to crash the server process eventually, possibly when I was trying to turn it back on, and then I realised it probably wasn’t a good idea anyway.

So my question is, is there a better way to bypass the rate limiter when running some arbitrary code i.e. something like:

RateLimiter.bypass do
# run some code not affected by the rate limiter
end

Or do I need to basically just need to copy most of what the PostActionCreator is doing but leave out the troublesome line?

Any help would be greatly appreciated! I’m still absorbing a lot of the Discourse code so I’m aware I’ve probably missed something that makes this a lot easier!

1 个赞

What I’d likely do is run a script in the Rails console. Have you checked out: Global rate limits and throttling in Discourse? It looks like you can change those rate limits.

2 个赞

This isn’t really a path I want to go down as I want it to initiated by users rather than them relying on me to run a script in the console.

3 个赞

Did you look at the link about changing the rate limits?

你弄明白了吗?修改 yaml 文件并不理想,因为它需要重新构建。我对 rails/discourse 不够熟悉,无法弄清楚如何在控制台中暂时禁用它。

您的用例是什么?您试图解决的、受到速率限制阻碍的问题是什么?

尝试通过 API 尽快导入 60k 用户,而无需重建。我也在测试安装上尝试通过 YAML 提高 ADMIN API 限制,但似乎不起作用,但也许我做错了什么。(我正在容器上运行导入,因此不应应用 HTTP 限制)。

但这并没有太大帮助,但我会这样做:在 Rails 中创建用户,而不是通过 API。discourse/script/import_scripts/csv_importer.rb at main · discourse/discourse · GitHub 可以作为起点。

但你可以进入容器并编辑 /var/www/discourse/config/discourse.conf 并在此处设置这些变量,然后执行 sv restart unicorn(或者可能是 sv reload unicorn)。你可能需要执行类似 apt-get update; apt-get install -y vim 的操作来安装编辑器。

2 个赞

奇怪。当我查看 conf 文件时,新限制已设置。因此,YAML 更新有效:

max_admin_api_reqs_per_key_per_minute = '6000'

但它似乎不起作用。每分钟仍然有 60 的限制。当我将请求限制为每分钟 60 个时,大多数请求都能通过,但由于抖动,少数请求仍然触发了速率限制器:

{'success': True, 'active': True, 'message': 'Your account is activated and ready to use.', 'user_id': 3596}
{'success': False, 'message': ' New registrations are not allowed from your IP address (maximum limit reached). Contact a staff member.', 'errors': {'ip_address': ['New registrations are not allowed from your IP address (maximum limit reached). Contact a staff member.']}, 'values': {'name': None, 'username': 'asdfd', 'email': 'user@domain.com'}, 'is_developer': False}
{'success': True, 'active': True, 'message': 'Your account is activated and ready to use.', 'user_id': 3597}

我想知道这是否是引号的问题?

您可以检查 SiteSettings.max_admin_api_reqs_per_key_per_minute 并查看它是否为整数。

1 个赞

in default_current_user_provider.rb(我找到的唯一引用此值的文件的内容是:

limit = GlobalSetting.max_admin_api_reqs_per_minute.to_i

所以它确实会进行转换。我甚至尝试在代码中将其替换为一个数字。)

1 个赞

糟糕。我早就料到了。

1 个赞

我认为这与另一个限制有关:我认为它达到了 IP 地址的最大注册数限制,但不明白为什么达到此限制后不会永久阻止。这可能是此垃圾邮件限制中的一个错误?

如果那些用户都有相同的IP地址,那么我打赌你是对的。我认为解决方案是更改该设置或为人们提供虚假的IP地址,例如 127.0.0.x(或者使用IPv6使其更容易?)

1 个赞

我认为它正在检测请求的来源(主机计算机),但我可以通过在同一 IP 地址上添加 TL2 用户(或调整 IP 限制)来解决它。

user add API 速度太慢了。

这就是为什么我建议用rails来做。它大概每分钟能处理500个。

1 个赞

是的,我会尝试一下。但我不太清楚该如何运行脚本,应该把它复制到哪里,以及(一旦我把 CSV 文件放到正确的位置)如何执行它?

您可以查看其他“操作方法”主题以了解其他内容。它们大多都以相同的方式工作。

1 个赞

鉴于理解脚本和 Ruby 等的努力以及可能的性能,我选择了另一条路线,通过直接使用 SQL 写入数据库来绕过所有限制。这样,我每秒可以插入数千条记录。

有一个关于需要哪些表的帖子,现在已关闭,所以我在这里回复,我更改了以下表来插入用户:

user_avatars
user_emails
user_profiles
user_search_data
user_stats
users
1 个赞