Update failing due to rate limiting by RubyGems

(Hosein Naseri) #1

Hello. I wanted to update my discourse for new commits, it did fail. Then I rebuild app, it also failed with the following error. Any Idea?

Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate' failed with return #<Process::Status: pid 13406 exit 1>
Location of failure: /pups/lib/pups/exec_command.rb:108:in `spawn'
exec failed with the params {"cd"=>"$home", "hook"=>"bundle_exec", "cmd"=>["su discourse -c 'bundle install --deployment --verbose --without test --without development'", "su discourse -c 'bundle exec rake db:migrate'", "su discourse -c 'bundle exec rake assets:precompile'"]}
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one

The only solution I have found to workaround «429 Too Many Requests» failure from rubygems.org
Constantly hitting rubygems rate limits
Unable to start discourse due to rubygems rate limiting
Bootstrap fails because of the wrong IP Address
(Matt Palmer) #2

What’s the actual error? “please scroll up and look for earlier error messages”

(Hosein Naseri) #3

I dont know its a lot of text. I searched for “error” and these are the lines I got:

HTTP 429 Too Many Requests https://index.rubygems.org/info/sprockets-rails
Bundler::HTTPError: Net::HTTPTooManyRequests: <html>
<head><title>429 Too Many Requests</title></head>

I, [2017-06-16T21:36:48.032740 #14]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate'
rake aborted!
Gem::MissingSpecError: Could not find 'http-cookie' (~> 1.0) among 248 total gem(s)
Checked in 'GEM_PATH=/var/www/discourse/vendor/bundle/ruby/2.4.0', execute `gem env` for more information

(Robby O'Connor) #4

You’re getting throttled by rubygems. Are you running on AWS? What does your pups template look like?

Read more here: RubyGems.org API - RubyGems Guides

(Jay Pfaffman) #5

I got this rebuilding last week. I commented out a plugin. I built again. It worked. Then I couldn’t figure out why that plugin could have caused a problem, put it back in, rebuilt, and it worked. I had prepared a bug report plea for help, but since the problem went away, I never posted it.

There have been a handful of these “too many requests” errors from rubygems lately. I can’t make sense of them.

(Hosein Naseri) #6

No. I’m not on AWS. I have a local server. [quote=“robbyoconnor, post:4, topic:64636”]
What does your pups template look like?

I don’t know what is pups template. could you explain?

So I commented out two plugins but it didn’t help. now I commented even more. Hope it would work this time.

(Hosein Naseri) #7

Now I’m getting this error :confused:

Removing old container
+ /usr/bin/docker rm app
Error response from daemon: Driver overlay failed to remove root filesystem abd13a925ecc025fb672e6ee0e594758f846f3ed393a8f647bb6cf1cf970fa6b: remove /var/lib/docker/overlay/d6866837ea0650d91bb09d012158ca82276cecc6d2c1c332d31e18c0651845dc/merged: device or resource busy

starting up existing container
+ /usr/bin/docker start app
Error response from daemon: Container is marked for removal and cannot be started.
Error: failed to start containers: app

what is this?

(Hosein Naseri) #8

I rebooted my server as the last resort :joy: and rebuild app and now it was rebuilded without error.

Then I added the plugins I’ve removed, However I got the same error I reported earlier again. So could it be one of the plugins that causes error?

(Robby O'Connor) #9

I’m not sure why it would…somehow you’re hitting the rubygems rate limit threshold.

(Hosein Naseri) #10

Is there any thing I could do to find the culprit?

(Robby O'Connor) #11

I’m not entirely sure – but what are you hosted?

(Jay Pfaffman) #12

Digital Ocean. At about the same time, I did similar rebuilds at Digital Ocean (and I’m pretty sure, the same data center).

(Robby O'Connor) #13

I’m not entirely sure – A lot of people have had this issue – I’ve never encountered this issue…

Odd…what plugin caused it? Look at its Gemfile maybe?

(Hosein Naseri) #14

The plugins I removed that got it fixed with a server reboot was these:

      - git clone http://github.com/discoursehosting/discourse-sitemap.git
      - git clone https://github.com/xfalcox/discourse-backups-to-dropbox.git

I added them back and it failed again.

(Jay Pfaffman) #15

Of those discourse-sitemap is the more likely culprit.

(Hosein Naseri) #16

Ok so I remove this and add all others to see what happens. However I dont know why it is the culprit? It has not been updated for a long time.

(Jay Pfaffman) #17

Plugins get broken if Discourse gets updated in a way that breaks the plugin. The other plugin you listed is maintained by a team member, so it’s somewhat more likely he’ll notice something’s wrong before other people.

(Hosein Naseri) #18

Nope. It failed again. So I’m removing backup to dropbox this time.

(Hosein Naseri) #19

But even if it is a broken plugin, why it would trigger rubbygem threshold? Its not making sense.

(Jay Pfaffman) #20

I couldn’t figure that out either. I suspect that it’s unrelated.

Are you still getting the rubygem error?