Hello. I wanted to update my discourse for new commits, it did fail. Then I rebuild app, it also failed with the following error. Any Idea? FAILED -------------------- Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate' failed with return # Location of failure: /pups/lib/pups/exec_command.rb:108:in `spawn' exec failed with the params {"cd"=>"$home", "hook"=>"bundle_exec", "cmd"=>["su discourse -c 'bundle install --deployment --verbose --without test --without development'", "su discourse -c 'bundle exec rake db:migrate'", "su discourse -c 'bundle exec rake assets:precompile'"]} 1202bf591b3bb66f4bb14db9ffd2761171eaaece447b8e3c00051fd4866d511c ** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one

[image] hnaseri: I removed the backup to dropbox plugin and added all others and now it rebuilded without any error. In that case, @Falco might want to have a look at the dropbox plugin.

If you get rate-limited by rubygems, you need to talk to rubygems. They explain how in their rate-limiting documentation. We can do absolutely nothing to help with rate-limiting problems. (This doesn’t just apply to the OP; everyone giving advice on this problem any time it occurs needs to refer people to rubygems, loudly, explicitly, and using words of no more than two syllables.) I can’t see any specific error messages relating to the dropbox plugin in this thread. Without a complete error message and backtrace (if displayed in the logs), @Falco isn’t going to be able to do anything to fix whatever error might be going wrong.

It doesn’t make sense to me why in such situations we need to talk to rubygems. We are installing/Updating something that passes their thresholds. So I think their response would simply be for us to check the plugins we are trying to install. Am I right?

[image] mpalmer: If you get rate-limited by rubygems, you need to talk to rubygems. They explain how in their rate-limiting documentation. We can do absolutely nothing to help with rate-limiting problems. That’s what I think too, but this has been coming up lately, and I’m using a host I’ve used for a while and have not had problems previously. Their site says: Is it possible that the bootstrap script is doing that as a matter of course? Here’s an excerpt from a build just now: HTTP GET https://index.rubygems.org/info/aws-sdk-cloudwatchlogs HTTP 200 OK https://index.rubygems.org/info/aws-sdk-cloudsearchdomain HTTP 200 OK https://index.rubygems.org/info/aws-sdk-cloudwatchevents HTTP 429 Too Many Requests https://index.rubygems.org/info/xattr HTTP GET https://index.rubygems.org/info/aws-sdk-codecommit HTTP 200 OK https://index.rubygems.org/info/aws-sdk-cloudsearch HTTP 200 OK https://index.rubygems.org/info/aws-sdk-applicationdiscoveryservice HTTP 200 OK https://index.rubygems.org/info/aws-sdk-cloudhsm HTTP GET https://index.rubygems.org/info/aws-sdk-codedeploy HTTP GET https://index.rubygems.org/info/aws-sdk-codepipeline Bundler::HTTPError: Net::HTTPTooManyRequests: 429 Too Many Requests 429 Too Many Requests nginx /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/fetcher/downloader.rb:36:in `fetch' /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/fetcher/compact_index.rb:116:in `call' /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/compact_index_client/updater.rb:43:in `block in update' /usr/local/lib/ruby/2.4.0/tmpdir.rb:89:in `mktmpdir' /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/compact_index_client/updater.rb:30:in `update' /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/compact_index_client.rb:81:in `update' /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/compact_index_client.rb:97:in `update_info' /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/compact_index_client.rb:54:in `block in dependencies' /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/fetcher/compact_index.rb:87:in `block (3 levels) in compact_index_client' /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/worker.rb:63:in `apply_func' /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/worker.rb:58:in `block in process_queue' /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/worker.rb:55:in `loop' /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/worker.rb:55:in `process_queue' /usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/worker.rb:89:in `block (2 levels) in create_threads' HTTP GET https://index.rubygems.org/info/aws-sdk-cognitoidentity HTTP GET https://index.rubygems.org/info/aws-sdk-cognitoidentityprovider HTTP GET https://index.rubygems.org/info/aws-sdk-cognitosync HTTP GET https://index.rubygems.org/info/aws-sdk-configservice HTTP GET https://index.rubygems.org/info/aws-sdk-databasemigrationservice HTTP 200 OK https://index.rubygems.org/info/aws-sdk-configservice HTTP GET https://index.rubygems.org/info/aws-sdk-datapipeline HTTP 200 OK https://index.rubygems.org/info/aws-sdk-cloudwatchlogs HTTP GET https://index.rubygems.org/info/aws-sdk-devicefarm And more disturbingly, at the end of that run: Successfully bootstrapped, to startup use ./launcher start webonly-multi So does that mean that I’m missing that gem and won’t find out until something tries to call them? I don’t use AWS, so I guess I don’t care? I’m confused. I’ve read where you (or someone on team) said that you guys aren’t using any kind of proxy and rebuild all the time, so I can’t make sense of what I’m seeing.

[image] hnaseri: It doesn’t make sense to me why in such situations we need to talk to rubygems. Because they’re the ones rate-limiting your requests. [image] hnaseri: We are installing/Updating something that passes their thresholds. So I think their response would simply be for us to check the plugins we are trying to install. I have no idea what their response would be. I don’t presume to speak for rubygems, which is why I suggest talking to them, to find out what they would say, and save all this guesswork. [image] pfaffman: Their site says: API and website: 10 requests per second It also says Some endpoints may be cached by our CDN at times and therefore may allow higher request rates. and The RubyGems.org team may occasionally blackhole user IP addresses for extreme cases to protect the platform. Which means that the correct description of the rate limit is “10 requests per second, unless it isn’t”. Which is less than entirely helpful. [image] pfaffman: Is it possible that the bootstrap script is doing that as a matter of course? Here’s an excerpt from a build just now: If there were timestamps on that log, it would help to identify how quickly requests are actually being made. As it is, all we can see is “requests were made, and some got 429’d”. Also, according to that exerpt, as is, you got rate-limited after two successful requests, so either you’re hitting a blacklist (it’s hard to see how making three requests could get you to hit a 10 req/sec limit) or there’s other requests you didn’t show. Either way, though, without timestamps nobody can tell whether you might be hitting a limit or not. [image] pfaffman: I’ve read where you (or someone on team) said that you guys aren’t using any kind of proxy and rebuild all the time, so I can’t make sense of what I’m seeing. Exactly. We’ve got customers with some pretty monstrous plugin lists, and their rebuilds never get rate-limited that I’ve seen. Perhaps we do builds often enough that the CDN node we hit is kept constantly fresh with all the packages we need, which is… nice, I guess, but not helpful to anyone else. Which is why, if you are seeing this problem, you should talk to rubygems so you can make sense of what you’re seeing, because our pontificating on meta about what may or may not be going on isn’t going to get the problem solved. If rubygems comes back with some actionable advice as to how Discourse and its environs can be modified to work better with rubygems rate-limits, great. Let us know, and we’ll look into it. However, having someone from Discourse directly talk to rubygems isn’t going to help, because we’re not seeing the problem. We can’t provide any useful diagnostic data (such as source IPs, timestamps, or anything) to rubygems, nor can we describe how to reliably (or otherwise) reproduce the problem. So… once again, if you are seeing rate-limiting problems with rubygems, talk to rubygems. If you’re not willing to do that, then please don’t complain about being rate-limited here on meta, because all you’re doing is adding to the noise, not the signal, about this problem.

I’ve got an idea now. I’ll see if I can figure out how to ask ruby gems about the problem. If I find out anything useful, I’ll let you know. :wink:

[image] pfaffman: I’ve got an idea now. I’ll see if I can figure out how to ask ruby gems about the problem. If I find out anything useful, I’ll let you know. :wink: @pfaffman – Why not work with @hnaseri – I recall @11145 having an issue as well – maybe the three of you gather: outputs, source ips and such and see if there is a common denominator (privately!)..

Is there any update on this issue @pfaffman?

I’ve not heard back from them.

Has anybody solved the problem? I have the same issue: every attempt to run ./launcher rebuild fails with the «429 Too Many Requests» from rubygems.org: [01] Here is the only solution I have found: The only solution I have found to workaround «429 Too Many Requests» failure from rubygems.org

Update failing due to rate limiting by RubyGems

Product Support

pfaffman (Jay Pfaffman) June 12, 2018, 12:37am 39

It sounds like this might be a solution

Topic		Replies	Views
Unable to start discourse due to rubygems rate limiting Product Support	43	3945	June 14, 2018
Rubygems: Too Many Requests problem returns! Product Support	16	3913	February 20, 2019
The only solution I have found to workaround «429 Too Many Requests» failure from rubygems.org Product Support	4	2650	October 27, 2017
Rebuild Failed, please help! Self-hosting	7	1197	September 28, 2022
Discourse won't rebuild Self-hosting	3	1191	August 7, 2021

Update failing due to rate limiting by RubyGems

Related topics