Users are experiencing issues starting or rebuilding their Discourse instances due to rate limiting from rubygems.org, which returns HTTP 429 Too Many Requests errors during the rebuild process. shaneoh first reported the problem after running standard update commands, with logs showing over 32K lines ending in a PostgreSQL connection failure and rubygems rate limit error. zogstrip identified the root cause as hitting rubygems.org’s rate limits, which can occur especially on VPS environments or after multiple rebuilds in quick succession. pfaffman and others confirmed similar experiences, noting inconsistent behavior and lack of response from rubygems.org support. Suggestions included waiting before retrying, changing IP addresses, or using ./launcher bootstrap app to rebuild without destroying the existing container. Falco later announced a new base image update (including Rails 5.2) designed to reduce rubygems requests, which Matt_1 confirmed resolved the issue after deployment. Some users, including codinghorror, reported never encountering the issue, speculating that shared IPs (e.g., on AWS or Vultr) may be more prone to triggering rate limits. The consensus is that while the problem is intermittent and poorly documented, the new base image significantly mitigates it. Users are advised to avoid rapid rebuilds and consider the bootstrap method for minimal downtime.
There shouldn’t be 32K lines in the rebuild log. What Régis is looking for the the console after from after you run ./launcher rebuild app. Share everything that appears in the console after that command.
Thanks for clarifying, but that’s actually what I am talking about. I run the rebuild command, and after that there are 32K lines that appear in the console, right up to the failure notification.
Bundler::HTTPError: Net::HTTPTooManyRequests: <html>
<head><title>429 Too Many Requests</title></head>
<body bgcolor="white">
<center><h1>429 Too Many Requests</h1></center>
<hr><center>nginx</center>
</body>
</html>
Thanks for spotting that. I have no idea what I can do about that though - I’m just running the commands that are meant to work with Discourse. I can’t even find any results for rubygem rate limits.
Every time we have had to do anything that involved restarting this server, Discourse breaks and I have to spend hours or days trying to fix it. It doesn’t seem to really work.
Do you mean that the rebuild could work if I try it later on? It seems to me that the problem is that the rebuild job itself involves too many requests and is therefore impossible. I can see others have stumbled across this recently too.
Pretty sure it will unless your IP has been blacklisted. The only way to know for certain is to contact rubygems.org.
From what I’ve seen, it only happens when using a VPS (where you don’t know what the other/previous tenants are doing) and/or when you rebuild your Discourse several times in a row in a short period of time.
FYI: we’ve never been rate limited, even when we deploy all our customers.
I can’t make rhyme or reason of the cause when I’ve been rate limited. It also doesn’t make sense that you guys have never been rate limited when I have on several occasions. Stranger still, sometimes I see the rate limit errors, but the build appears to complete successfully.
Yeah, general feedback online seems the same, nobody gets a response.
I’ve just restarted the server and made sure it was assigned a new IP address. On the first attempt I got the same issue. This rate limit thing doesn’t seem to be very logical. As it’s a brand new attempt, I still have to suspect something about the way discourse is attempting to do this is breaking things, but I don’t know enough about how it works to know for sure.