Сбой миграции нескольких экземпляров при добавлении нового экземпляра

[Padoup-Padoup]

Здравствуйте,

Мы уже около двух лет работаем с сервером Discourse в многоэкземплярном режиме (один контейнер) и хостим примерно 20 экземпляров Discourse. Мы успешно выполняли регулярные обновления. Однако последние две недели мы сталкиваемся с блокирующей проблемой:

• Обновление Discourse через команду ./launcher rebuild app по-прежнему проходит без ошибок;
• Но добавить новый экземпляр не удаётся. При добавлении нового экземпляра в файл .yml и запуске ./launcher rebuild app возникает следующая ошибка (в файле theme-transpiler.js):

Сообщение об ошибке

[...]
I, [2023-10-26T14:20:43.219318 #1]  INFO -- : > cd /var/www/discourse && sudo -E -u discourse bundle exec rake multisite:migrate

--------------------------------------------------------------------------------
1 миграция не удалась!

Не удалось выполнить миграцию для amapyvette
#<Errno::ENOENT: No such file or directory @ rb_sysopen - tmp/theme-transpiler.js>
/var/www/discourse/lib/discourse_js_processor.rb:139:in `read'
/var/www/discourse/lib/discourse_js_processor.rb:139:in `create_new_context'
/var/www/discourse/lib/discourse_js_processor.rb:155:in `block in v8'
/var/www/discourse/lib/discourse_js_processor.rb:153:in `synchronize'
/var/www/discourse/lib/discourse_js_processor.rb:153:in `v8'
/var/www/discourse/lib/discourse_js_processor.rb:168:in `block in v8_call'
/var/www/discourse/lib/discourse_js_processor.rb:167:in `synchronize'
/var/www/discourse/lib/discourse_js_processor.rb:167:in `v8_call'
/var/www/discourse/lib/discourse_js_processor.rb:192:in `perform'
/var/www/discourse/lib/pretty_text.rb:54:in `apply_es6_file'
/var/www/discourse/lib/pretty_text.rb:67:in `block in ctx_load_directory'
/var/www/discourse/lib/pretty_text.rb:66:in `each'
/var/www/discourse/lib/pretty_text.rb:66:in `ctx_load_directory'
/var/www/discourse/lib/pretty_text.rb:99:in `create_es6_context'
/var/www/discourse/lib/pretty_text.rb:138:in `block in v8'
/var/www/discourse/lib/pretty_text.rb:136:in `synchronize'
/var/www/discourse/lib/pretty_text.rb:136:in `v8'
/var/www/discourse/lib/pretty_text.rb:290:in `block in escape_emoji'
/var/www/discourse/lib/pretty_text.rb:665:in `block in protect'
/var/www/discourse/lib/pretty_text.rb:665:in `synchronize'
/var/www/discourse/lib/pretty_text.rb:665:in `protect'
/var/www/discourse/lib/pretty_text.rb:290:in `escape_emoji'
/var/www/discourse/app/models/emoji.rb:266:in `unicode_unescape'
/var/www/discourse/app/models/topic.rb:503:in `fancy_title'
/var/www/discourse/app/models/topic.rb:381:in `block in <class:Topic>'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:445:in `instance_exec'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:445:in `block in make_lambda'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:199:in `block (2 levels) in halting'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:687:in `block (2 levels) in default_terminator'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:686:in `catch'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:686:in `block in default_terminator'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:200:in `block in halting'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:595:in `block in invoke_before'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:595:in `each'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:595:in `invoke_before'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:116:in `block in run_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/autosave_association.rb:370:in `around_save_collection_association'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:127:in `block in run_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:138:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:929:in `_run_save_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/callbacks.rb:455:in `create_or_update'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/timestamp.rb:126:in `create_or_update'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/persistence.rb:648:in `save!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/validations.rb:53:in `save!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:302:in `block in save!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:354:in `block in with_transaction_returning_status'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/database_statements.rb:314:in `transaction'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:350:in `with_transaction_returning_status'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:302:in `save!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/suppressor.rb:54:in `save!'
/var/www/discourse/app/models/category.rb:420:in `block in create_category_definition'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/database_statements.rb:314:in `transaction'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:209:in `transaction'
/var/www/discourse/app/models/category.rb:409:in `create_category_definition'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:400:in `block in make_lambda'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:236:in `block in halting_and_conditional'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:599:in `block in invoke_after'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:599:in `each'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:599:in `invoke_after'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:108:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:929:in `_run_create_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/callbacks.rb:459:in `_create_record'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/timestamp.rb:108:in `_create_record'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/persistence.rb:1069:in `create_or_update'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/callbacks.rb:455:in `block in create_or_update'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/autosave_association.rb:370:in `around_save_collection_association'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:127:in `block in run_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:138:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/callbacks.rb:929:in `_run_save_callbacks'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/callbacks.rb:455:in `create_or_update'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/timestamp.rb:126:in `create_or_update'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/persistence.rb:648:in `save!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/validations.rb:53:in `save!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:302:in `block in save!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:354:in `block in with_transaction_returning_status'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/database_statements.rb:314:in `transaction'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:350:in `with_transaction_returning_status'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:302:in `save!'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/suppressor.rb:54:in `save!'
/var/www/discourse/lib/seed_data/categories.rb:136:in `create_category'
/var/www/discourse/lib/seed_data/categories.rb:15:in `block (2 levels) in create'
/var/www/discourse/lib/seed_data/categories.rb:15:in `each'
/var/www/discourse/lib/seed_data/categories.rb:15:in `block in create'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/i18n-1.14.1/lib/i18n.rb:322:in `with_locale'
/var/www/discourse/lib/seed_data/categories.rb:14:in `create'
(eval):5:in `block (2 levels) in run_file'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:46:in `eval'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:46:in `block (2 levels) in run_file'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:58:in `block in open'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:57:in `open'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:57:in `open'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:36:in `block in run_file'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/transaction.rb:319:in `block in within_new_transaction'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activesupport-7.0.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/transaction.rb:317:in `within_new_transaction'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/database_statements.rb:316:in `transaction'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/transactions.rb:209:in `transaction'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:35:in `run_file'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:26:in `block in run'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:25:in `each'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/seed-fu/runner.rb:25:in `run'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/discourse-seed-fu-2.3.12/lib/discourse-seed-fu.rb:29:in `seed'
/var/www/discourse/lib/tasks/db.rake:205:in `block (3 levels) in <main>'
/var/www/discourse/lib/tasks/db.rake:159:in `block (3 levels) in execute_concurrently'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rails_multisite-5.0.0/lib/rails_multisite/connection_management.rb:280:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rails_multisite-5.0.0/lib/rails_multisite/connection_management.rb:79:in `with_connection'
/var/www/discourse/lib/tasks/db.rake:157:in `block (2 levels) in execute_concurrently'

[...]

FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && sudo -E -u discourse bundle exec rake multisite:migrate завершилась ошибкой с кодом возврата #<Process::Status: pid 903 exit 1>
Место возникновения ошибки: /usr/local/lib/ruby/gems/3.2.0/gems/pups-1.2.1/lib/pups/exec_command.rb:132:in `spawn'
Ошибка выполнения с параметрами "cd /var/www/discourse && sudo -E -u discourse bundle exec rake multisite:migrate"
Загрузка не удалась с кодом выхода 1
** НЕ УДАЛОСЬ ЗАГРУЗИТЬ ** пожалуйста, прокрутите вверх и поищите более ранние сообщения об ошибках, их может быть несколько.
Утилита ./discourse-doctor может помочь в диагностике проблемы.

Дополнительный контекст:

• Единственное отличие от предыдущих добавлений заключается в том, что мы перенесли один экземпляр следуя этому руководству
• Следуя этому посту, я попытался удалить все плагины (закомментировав их), но проблема осталась
• (для полноты картины), мы увеличили несколько недель назад количество одновременных подключений к PostgreSQL до 200.

Есть ли у вас какие-либо идеи, как это исправить?

[twofoursixeight]

У меня нет чёткого представления по этой теме, но я могу попробовать помочь.
Эта тема имеет отношение:

Migration failed upgrading to latest from ~3 weeks ago

1 migrations failed! Failed to migrate default #<StandardError: An error has occurred, all later migrations canceled: PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "index_user_emails_on_email" DETAIL: Key (lower(email::text))=(talk@remobjects.com<mailto:talk@remobjects.com) already exists. > /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-2.3.3/lib/patches/db/pg.rb:110:in `exec' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profil…

[sam (Sam Saffron)]

Я уведомил людей, работавших над этим первоначальным изменением. Мы свяжемся с вами в ближайшие несколько дней.

Мы часто разворачиваем мультисайты, поэтому есть обходной путь.

[david (David Taylor)]

Спасибо за отчет, @Padoup-Padoup, и извините за неудобства!

Я только что объединил исправление в основную ветку Discourse, и оно должно пройти тесты в течение следующих 30 минут.

После этого ваша пересборка должна завершиться успешно. Пожалуйста, сообщите нам, если вы всё ещё столкнётесь с какими-либо проблемами.

[Padoup-Padoup]

Ого, я впечатлён! Спасибо за оперативность !

Действительно, сборка успешно прошла на моём тестовом компьютере. Я выполню миграцию на производственном сервере и сообщу вам

[Padoup-Padoup]

Подтверждаю, что сборка прошла успешно и я смог добавить новый экземпляр. Большое спасибо