Falha na migração multi-instância ao adicionar uma nova instância

[Padoup-Padoup]

Hello,

We have been running a multi-instance (single container) Discourse server for about 2 years, and host ~20 Discourse instances. We performed regular updates with success. For the last two weeks, we experience a blocking issue:

• We can still upgrade Discourse using ./launcher rebuild app without issues ;
• But we cannot add any extra instance. Adding an extra instance in the .yml file and running ./launcher rebuild app ends up with the following error (in theme-transpiler.js):

Error message

[...]
I, [2023-10-26T14:20:43.219318 #1]  INFO -- : > cd /var/www/discourse && sudo -E -u discourse bundle exec rake multisite:migrate

--------------------------------------------------------------------------------
1 migrations failed!

Failed to migrate amapyvette
#<Errno::ENOENT: No such file or directory @ rb_sysopen - tmp/theme-transpiler.js>
/var/www/discourse/lib/discourse_js_processor.rb:139:in `read'
/var/www/discourse/lib/discourse_js_processor.rb:139:in `create_new_context'
/var/www/discourse/lib/discourse_js_processor.rb:155:in `block in v8'
/var/www/discourse/lib/discourse_js_processor.rb:153:in `synchronize'
/var/www/discourse/lib/discourse_js_processor.rb:153:in `v8'
/var/www/discourse/lib/discourse_js_processor.rb:168:in `block in v8_call'
/var/www/discourse/lib/discourse_js_processor.rb:167:in `synchronize'
/var/www/discourse/lib/discourse_js_processor.rb:167:in `v8_call'
/var/www/discourse/lib/discourse_js_processor.rb:192:in `perform'
/var/www/discourse/lib/pretty_text.rb:54:in `apply_es6_file'
/var/www/discourse/lib/pretty_text.rb:67:in `block in ctx_load_directory'
/var/www/discourse/lib/pretty_text.rb:66:in `each'
/var/www/discourse/lib/pretty_text.rb:66:in `ctx_load_directory'
/var/www/discourse/lib/pretty_text.rb:99:in `create_es6_context'
/var/www/discourse/lib/pretty_text.rb:138:in `block in v8'
/var/www/discourse/lib/pretty_text.rb:136:in `synchronize'
/var/www/discourse/lib/pretty_text.rb:136:in `v8'
/var/www/discourse/lib/pretty_text.rb:290:in `block in escape_emoji'
/var/www/discourse/lib/pretty_text.rb:665:in `block in protect'
/var/www/discourse/lib/pretty_text.rb:665:in `synchronize'
/var/www/discourse/lib/pretty_text.rb:665:in `protect'
/var/www/discourse/lib/pretty_text.rb:290:in `escape_emoji'
/var/www/discourse/app/models/emoji.rb:266:in `unicode_unescape'
/var/www/discourse/app/models/topic.rb:503:in `fancy_title'
/var/www/discourse/app/models/topic.rb:381:in `block in <class:Topic>'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:445:in `instance_exec'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:445:in `block in make_lambda'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:199:in `block (2 levels) in halting'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:687:in `block (2 levels) in default_terminator'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:686:in `catch'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:686:in `block in default_terminator'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:200:in `block in halting'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:595:in `block in invoke_before'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:595:in `each'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:595:in `invoke_before'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:116:in `block in run_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/autosave_association.rb:370:in `around_save_collection_association'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:127:in `block in run_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:138:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:929:in `_run_save_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/callbacks.rb:455:in `create_or_update'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/timestamp.rb:126:in `create_or_update'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/persistence.rb:648:in `save!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/validations.rb:53:in `save!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:302:in `block in save!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:354:in `block in with_transaction_returning_status'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/database_statements.rb:314:in `transaction'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:350:in `with_transaction_returning_status'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:302:in `save!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/suppressor.rb:54:in `save!'
/var/www/discourse/app/models/category.rb:420:in `block in create_category_definition'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/database_statements.rb:314:in `transaction'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:209:in `transaction'
/var/www/discourse/app/models/category.rb:409:in `create_category_definition'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:400:in `block in make_lambda'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:236:in `block in halting_and_conditional'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:599:in `block in invoke_after'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:599:in `each'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:599:in `invoke_after'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:108:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:929:in `_run_create_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/callbacks.rb:459:in `_create_record'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/timestamp.rb:108:in `_create_record'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/persistence.rb:1069:in `create_or_update'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/callbacks.rb:455:in `block in create_or_update'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/autosave_association.rb:370:in `around_save_collection_association'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:127:in `block in run_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:138:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:929:in `_run_save_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/callbacks.rb:455:in `create_or_update'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/timestamp.rb:126:in `create_or_update'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/persistence.rb:648:in `save!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/validations.rb:53:in `save!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:302:in `block in save!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:354:in `block in with_transaction_returning_status'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/database_statements.rb:314:in `transaction'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:350:in `with_transaction_returning_status'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:302:in `save!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/suppressor.rb:54:in `save!'
/var/www/discourse/lib/seed_data/categories.rb:136:in `create_category'
/var/www/discourse/lib/seed_data/categories.rb:15:in `block (2 levels) in create'
/var/www/discourse/lib/seed_data/categories.rb:15:in `each'
/var/www/discourse/lib/seed_data/categories.rb:15:in `block in create'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/i18n-1.14.1/lib/i18n.rb:322:in `with_locale'
/var/www/discourse/lib/seed_data/categories.rb:14:in `create'
(eval):5:in `block (2 levels) in run_file'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:46:in `eval'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:46:in `block (2 levels) in run_file'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:58:in `block in open'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:57:in `open'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:57:in `open'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:36:in `block in run_file'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/transaction.rb:319:in `block in within_new_transaction'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/transaction.rb:317:in `within_new_transaction'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/database_statements.rb:316:in `transaction'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:209:in `transaction'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:35:in `run_file'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:26:in `block in run'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:25:in `each'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:25:in `run'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/discourse-seed-fu.rb:29:in `seed'
/var/www/discourse/lib/tasks/db.rake:205:in `block (3 levels) in <main>'
/var/www/discourse/lib/tasks/db.rake:159:in `block (3 levels) in execute_concurrently'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rails_multisite-5.0.0/lib/rails_multisite/connection_management.rb:280:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rails_multisite-5.0.0/lib/rails_multisite/connection_management.rb:79:in `with_connection'
/var/www/discourse/lib/tasks/db.rake:157:in `block (2 levels) in execute_concurrently'

[...]

FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && sudo -E -u discourse bundle exec rake multisite:migrate failed with return #<Process::Status: pid 903 exit 1>
Location of failure: /usr/local/lib/ruby/gems/3.2.0/gems/pups-1.2.1/lib/pups/exec_command.rb:132:in `spawn'
exec failed with the params "cd /var/www/discourse && sudo -E -u discourse bundle exec rake multisite:migrate"
bootstrap failed with exit code 1
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.

Extra context:

• The only difference with previous addition seems to be that we migrated an instance following this guide
• Following this post, I tried removing all plugins (by commenting them out), same issue
• (mentioning for completenesse), we increased a couple of weeks ago the number of concurrent connexions to postgreSQL to 200.

Do you have any idea about how to fix it?

[twofoursixeight]

Não tenho a melhor ideia sobre este assunto, mas posso tentar ajudar.
Este tópico está relacionado:

Migration failed upgrading to latest from ~3 weeks ago

1 migrations failed! Failed to migrate default #<StandardError: An error has occurred, all later migrations canceled: PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "index_user_emails_on_email" DETAIL: Key (lower(email::text))=(talk@remobjects.com<mailto:talk@remobjects.com) already exists. > /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-2.3.3/lib/patches/db/pg.rb:110:in `exec' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profil…

[sam (Sam Saffron)]

Avisei as pessoas que trabalharam nesta alteração original. Devemos ter uma resposta para você nos próximos dias.

Nós implantamos vários sites com frequência, então existe uma solução alternativa.

[david (David Taylor)]

Obrigado pelo relatório @Padoup-Padoup, e desculpe pela interrupção!

Acabei de mesclar uma correção no core do Discourse, que deve estar no tests-passed nos próximos 30 minutos.

Depois disso, sua reconstrução deverá ser bem-sucedida. Por favor, informe-nos se você ainda estiver vendo algum problema.

[Padoup-Padoup]

Uau, estou impressionado! Obrigado pela sua reatividade !

De fato, a compilação foi bem-sucedida na minha máquina de teste. Farei a migração na máquina de produção e avisarei

[Padoup-Padoup]

Confirmo que a compilação foi bem-sucedida e que consegui adicionar uma nova instância. Muito obrigado