Rubygem timeouts during launcher rebuild app

Hi all,

When attempting an upgrade from 2.5.0beta2 to 2.5.0.beta3 via ./launcher rebuild app on the command line I keep running into rubygem timeouts at

cd /var/www/discourse && gem update bundler

that cause the rebuild to fail… has anyone else encountered the same? Sometimes it times out on the rubygems.org/specs.4.8.gz file and sometimes it has timeouts on specific packages. Previous upgrades used to work seamlessly on this particular instance. Any ideas on how to fix this? I’m ready to try a fresh install off a backup otherwise…

I, [2020-04-26T00:37:59.817638 #1]  INFO -- : > cd /var/www/discourse && gem update bundler
ERROR:  While executing gem ... (Gem::RemoteFetcher::UnknownHostError)
    timed out (https://rubygems.org/specs.4.8.gz)
167:signal-handler (1587861540) Received SIGTERM scheduling shutdown...
I, [2020-04-26T00:39:00.334060 #1]  INFO -- : 
I, [2020-04-26T00:39:00.334828 #1]  INFO -- : Terminating async processes
I, [2020-04-26T00:39:00.335004 #1]  INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/10/bin/postmaster -D /etc/postgresql/10/
main pid: 50
I, [2020-04-26T00:39:00.335154 #1]  INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 167
2020-04-26 00:39:00.335 UTC [50] LOG:  received fast shutdown request
167:M 26 Apr 2020 00:39:00.351 # User requested shutdown...
167:M 26 Apr 2020 00:39:00.351 * Saving the final RDB snapshot before exiting.
2020-04-26 00:39:00.360 UTC [50] LOG:  aborting any active transactions
2020-04-26 00:39:00.362 UTC [50] LOG:  worker process: logical replication launcher (PID 59) exited with exit code 1
2020-04-26 00:39:00.365 UTC [54] LOG:  shutting down
167:M 26 Apr 2020 00:39:00.616 * DB saved on disk
167:M 26 Apr 2020 00:39:00.616 # Redis is now ready to exit, bye bye...
2020-04-26 00:39:00.747 UTC [50] LOG:  database system is shut down


FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && gem update bundler failed with return #<Process::Status: pid 307 exit 1>
Location of failure: /pups/lib/pups/exec_command.rb:112:in `spawn'
exec failed with the params {"cd"=>"$home", "hook"=>"web", "cmd"=>["gem update bundler", "find $home ! -user discourse -exec chown discourse {} \\+"]}
7b53e40ff232069b5b0ac1355c706d777536284eece872cb7cfe46647561aa9b
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.

Rubygem occasionally have glitches especially on some AWS instances where they are being hammered from or China.

Is the issue persisting?

4 Likes

Yep, I’m still encountering this (on an instance running on Compute Canada’s OpenStack cloud infrastructure). If no one else is encountering issues it seems more likely that it’s an issue with our local environment, I’ll try to investigate further…

UPDATE nevermind, it just completed after trying it one more time! It complained about timeouts but continued using resolution from the lockfile.

I, [2020-04-27T07:06:48.945052 #1]  INFO -- : Running `bundle install --deployment --jobs 4 --retry 3 --verbose --without "test" "development"` with bundler 2.1.4
Frozen, using resolution from the lockfile
The definition is missing ["tzinfo-1.2.7", "zeitwerk-2.3.0", "activesupport-6.0.2.2", "nokogiri-1.10.9", "loofah-2.5.0", "actionview-6.0.2.2", "actionpack-6.0.2.2", "activejob-6.0.2.2", "actionmailer-6.0.2.2", "activemodel-6.0.2.2", "activerecord-6.0.2.2", "public_suffix-4.0.4", "aws-eventstream-1.1.0", "aws-partitions-1.298.0", "aws-sigv4-1.1.2", "aws-sdk-core-3.94.0", "aws-sdk-kms-1.30.0", "aws-sdk-s3-1.62.0", "aws-sdk-sns-1.22.0", "bootsnap-1.4.6", "openssl-signature_algorithm-0.4.0", "cose-1.0.0", "railties-6.0.2.2", "json-2.3.0", "logster-2.8.0", "message_bus-3.0.0", "mini_racer-0.2.10", "sidekiq-6.0.7", "mini_sql-0.2.5", "oauth2-1.4.4", "oj-3.10.6", "omniauth-1.9.1", "onebox-1.9.27.1", "optimist-3.0.1", "pg-1.2.3", "puma-4.3.3", "rack-mini-profiler-2.0.1", "rails_multisite-2.1.1", "rbtrace-0.4.12", "rubyzip-2.3.0", "unf_ext-0.0.7.7", "unicorn-5.5.4"]
HTTP GET https://index.rubygems.org/versions
HTTP 206 Partial Content https://index.rubygems.org/versions
Fetching gem metadata from https://rubygems.org/
...