Failed To Bootstrap Pups::ExecError 'bundle exec rake db:migrate' failed with return exec failed with the params 2.9.0.beta4 to 2.9.0.beta5

Installation
1

I upgrade my discourse two months ago to 2.9.0.beta4 34fba417f1. Unfortunately, I forgot to recheck my Upgrade page for ./launcher rebuild app. When I try to rebuild in last two days ago, my disc space is insufficient. My backups is 130 MB.

So, upgrade my droplet and resize into 50 GB from 25 GB.

This error occur when I tried to rebuild the app.


102:M 11 Jun 2022 13:40:32.373 * DB saved on disk

102:M 11 Jun 2022 13:40:32.374 # Redis is now ready to exit, bye bye...

FAILED

--------------------

Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate' failed with return #<Process::Status: pid 1605 exit 1>

Location of failure: /usr/local/lib/ruby/gems/2.7.0/gems/pups-1.1.1/lib/pups/exec_command.rb:117:in `spawn'

exec failed with the params {"cd"=>"$home", "hook"=>"db_migrate", "cmd"=>["su discourse -c 'bundle exec rake db:migrate'"]}

bootstrap failed with exit code 1

** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.

./discourse-doctor may help diagnose the problem.

9a7eb814ecd8b017057c4f6b9c54fc1c65520c7b5e9427984ee6fc326246b170

Things that I have tried and ended with failed:

  1. ./discourse-doctor: I still no idea what I should do next.
  2. Move my instance into new server and restore the backup. I think, it is failed because my discourse is not the latest version. Current metadata version is 20220526203356, mine 20220407195246 . I have the error logs.

Some ideas that I want to do:

  1. Install a same discourse commit (34fba417f1) in new digitalocean server and restore my backup. But, I cannot find any clue how to install the same version. I have tried some clue but failed. Like this one.
  2. Use a same discourse version and restore my backup but I do not how to make it.

My plugins:

  1. discourse-canned-replies
  2. discourse-push-notifications

Ubuntu version 20.04.4 LTS.

thank you

2

Please supply the 60 lines which precede that snippet of your log file.

Please also show the output of
free
as typed on the command line.

3 Likes
3

Thank you for your kindness and response Mr Ed.

This error still occur when I try to upgrade from 2.9.0.beta4 to 2.9.0.beta5.

Here is 60 lines precede my snippet above to the end:

Caused by:
PG::InvalidTextRepresentation: ERROR:  invalid input syntax for type integer: "[]"
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-3.0.0/lib/patches/db/pg.rb:110:in `exec'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-3.0.0/lib/patches/db/pg.rb:110:in `async_exec'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:48:in `block (2 levels) in execute'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/dependencies/interlock.rb:41:in `permit_concurrent_loads'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:47:in `block in execute'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract_adapter.rb:765:in `block in log'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract_adapter.rb:756:in `log'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:46:in `execute'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:932:in `block in method_missing'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:900:in `block in say_with_time'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:900:in `say_with_time'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:921:in `method_missing'
/var/www/discourse/db/migrate/20220330160747_copy_site_settings_uploads_to_upload_references.rb:5:in `up'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:873:in `public_send'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:873:in `exec_migration'
/var/www/discourse/lib/freedom_patches/schema_migration_details.rb:9:in `block in exec_migration'
/var/www/discourse/lib/freedom_patches/schema_migration_details.rb:8:in `exec_migration'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:854:in `block (2 levels) in migrate'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:853:in `block in migrate'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:215:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:852:in `migrate'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:1046:in `migrate'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:1360:in `block in execute_migration_in_transaction'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/transaction.rb:319:in `block in within_new_transaction'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/transaction.rb:317:in `within_new_transaction'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/database_statements.rb:316:in `transaction'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/transactions.rb:209:in `transaction'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:1411:in `ddl_transaction'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:1359:in `execute_migration_in_transaction'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:1333:in `each'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:1333:in `migrate_without_lock'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:1282:in `block in migrate'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:1432:in `block in with_advisory_lock'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:215:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:1447:in `with_advisory_lock_connection'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:1428:in `with_advisory_lock'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:1282:in `migrate'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:1117:in `up'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/migration.rb:1092:in `migrate'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/tasks/database_tasks.rb:262:in `migrate'
/var/www/discourse/lib/tasks/db.rake:231:in `block (2 levels) in <main>'
/var/www/discourse/lib/distributed_mutex.rb:33:in `block in synchronize'
/var/www/discourse/lib/distributed_mutex.rb:29:in `synchronize'
/var/www/discourse/lib/distributed_mutex.rb:29:in `synchronize'
/var/www/discourse/lib/distributed_mutex.rb:14:in `synchronize'
/var/www/discourse/lib/tasks/db.rake:214:in `block in <main>'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/usr/local/bin/bundle:25:in `load'
/usr/local/bin/bundle:25:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
I, [2022-06-19T10:38:18.212808 #1]  INFO -- : == 20220330160747 CopySiteSettingsUploadsToUploadReferences: migrating ========
-- execute("WITH site_settings_uploads AS (\n  SELECT id, unnest(string_to_array(value, '|'))::integer upload_id\n  FROM site_settings\n  WHERE data_type = 17\n  UNION\n  SELECT id, value::integer\n  FROM site_settings\n  WHERE data_type = 18 AND value != ''\n)\nINSERT INTO upload_references(upload_id, target_type, target_id, created_at, updated_at)\nSELECT site_settings_uploads.upload_id, 'SiteSetting', site_settings_uploads.id, uploads.created_at, uploads.updated_at\nFROM site_settings_uploads\nJOIN uploads ON uploads.id = site_settings_uploads.upload_id\nON CONFLICT DO NOTHING\n")

I, [2022-06-19T10:38:18.213900 #1]  INFO -- : Terminating async processes
I, [2022-06-19T10:38:18.214413 #1]  INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main pid: 41
I, [2022-06-19T10:38:18.214817 #1]  INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 102
2022-06-19 10:38:18.215 UTC [41] LOG:  received fast shutdown request
102:signal-handler (1655635098) Received SIGTERM scheduling shutdown...
2022-06-19 10:38:18.225 UTC [41] LOG:  aborting any active transactions
2022-06-19 10:38:18.235 UTC [41] LOG:  background worker "logical replication launcher" (PID 50) exited with exit code 1
2022-06-19 10:38:18.236 UTC [45] LOG:  shutting down
102:M 19 Jun 2022 10:38:18.286 # User requested shutdown...
102:M 19 Jun 2022 10:38:18.286 * Saving the final RDB snapshot before exiting.
2022-06-19 10:38:18.301 UTC [41] LOG:  database system is shut down
102:M 19 Jun 2022 10:38:18.515 * DB saved on disk
102:M 19 Jun 2022 10:38:18.516 # Redis is now ready to exit, bye bye...


FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate' failed with return #<Process::Status: pid 1602 exit 1>
Location of failure: /usr/local/lib/ruby/gems/2.7.0/gems/pups-1.1.1/lib/pups/exec_command.rb:117:in `spawn'
exec failed with the params {"cd"=>"$home", "hook"=>"db_migrate", "cmd"=>["su discourse -c 'bundle exec rake db:migrate'"]}
bootstrap failed with exit code 1
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.
732ef5a2f11cd751fa276bc9936bfa82c95292dc11894b0dba69fac64d6b0848

I think, there are more than one error. I try to take the main error message before 60 lines above:

First

I, [2022-06-19T10:38:11.832691 #1]  INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate'
2022-06-19 10:38:18.154 UTC [1616] discourse@discourse ERROR:  invalid input syntax for type integer: "[]"
2022-06-19 10:38:18.154 UTC [1616] discourse@discourse STATEMENT:  WITH site_settings_uploads AS (
          SELECT id, unnest(string_to_array(value, '|'))::integer upload_id
          FROM site_settings
          WHERE data_type = 17
          UNION
          SELECT id, value::integer
          FROM site_settings
          WHERE data_type = 18 AND value != ''
        )
        INSERT INTO upload_references(upload_id, target_type, target_id, created_at, updated_at)
        SELECT site_settings_uploads.upload_id, 'SiteSetting', site_settings_uploads.id, uploads.created_at, uploads.updated_at
        FROM site_settings_uploads
        JOIN uploads ON uploads.id = site_settings_uploads.upload_id
        ON CONFLICT DO NOTHING

rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

PG::InvalidTextRepresentation: ERROR:  invalid input syntax for type integer: "[]"
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-3.0.0/lib/patches/db/pg.rb:110:in `exec'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-3.0.0/lib/patches/db/pg.rb:110:in `async_exec'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:48:in `block (2 levels) in execute'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'

Second

Caused by:
ActiveRecord::StatementInvalid: PG::InvalidTextRepresentation: ERROR:  invalid input syntax for type integer: "[]"
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-3.0.0/lib/patches/db/pg.rb:110:in `exec'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-3.0.0/lib/patches/db/pg.rb:110:in `async_exec'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:48:in `block (2 levels) in execute'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/dependencies/interlock.rb:41:in `permit_concurrent_loads'

thank you

1 Like
4

Someone had a similar error recently too - Unable to restore/update discourse (PG::InvalidTextRepresentation: ERROR: invalid input syntax for type integer: "[]"), which was due to some stray data in the selectable_avatar site setting field blocking its migration. There is a manual workaround using the rails console, however you do need to be comfortable using the rails console.

If the rails console is out of your comfort zone, @david did mention creating an automatic tweak if there were any more reports. :crossed_fingers:

5 Likes
5

For the record: that should not be a reason for the restore to fail. If the instance you’re restoring into is newer than your backup, you should be good.

This would have happened to you as well in case you were upgrading on the same host.

2 Likes
6

I think this should fix the issue as soon as it’s merged: :slightly_smiling_face: :+1:

6 Likes
7

Thanks God, it is finish. My Discourse is ready to dance again. :dancer:
Even my Discourse now is in 10 USD per month (50 GB). :sweat_smile:

I fix it with Jammy’s hint above by entering the rails console with:


cd /var/discourse
./launcher enter app
rails c
SiteSetting.selectable_avatars = ""

I use exit twice to exit from rails console.

Then, rebuild the app as usual.

I got same message when ./launcher enter app or rails c. I forget which one.
But, at least, my Discourse is back. :partying_face:

Thank you for your kindness, response, and sharing Ed_s, Jammy, and Richard. May all goodness always comes to you all.

Anyway, I also grateful with Resize and Snapshot VPS feature and I will try to always remember to always open Upgrade page after successful rebuild the app.

Thank you…

5 Likes
8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.