Problem with restore Discourse from backup (3.4, quite large DB)

Support
1

Hi there,
We are one of your early bird users, we are using Discourse since 2015.
It works just fine until latest update with Postgres update included (possibly 3.4?). We have one backup points before this update. We decide to make clean install then restore from this backup.

OS: Ubuntu 24.04.1 LTS (latest updates) + docker 26.1.3
Trying to restore large discourse backup (44GB, single site) to a blank container.

Automated restore fails on DB migration with following error:
PG::DuplicateColumn: ERROR: column “notification_level” of relation “user_chat_channel_memberships” already exists (PG::DuplicateColumn)

logs:
Specifically, the migration stops after deploying the SQL archive because a column in the database is not unique.
column "notification_level" of relation "user_chat_channel_memberships" already exists

Summary

ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
Migrating the database…
EXCEPTION: /var/www/discourse/lib/discourse.rb:139:in `exec’: Failed to migrate database.
rake aborted!
StandardError: An error has occurred, all later migrations canceled: (StandardError)

PG::DuplicateColumn: ERROR: column “notification_level” of relation “user_chat_channel_memberships” already exists
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rack-mini-profiler-3.3.1/lib/patches/db/pg.rb:110:in exec' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rack-mini-profiler-3.3.1/lib/patches/db/pg.rb:110:in async_exec’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:56:in block (2 levels) in raw_execute' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:1004:in block in with_raw_connection’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activesupport-7.2.2.1/lib/active_support/concurrency/null_lock.rb:9:in synchronize' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:976:in with_raw_connection’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:55:in block in raw_execute' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activesupport-7.2.2.1/lib/active_support/notifications/instrumenter.rb:58:in instrument’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:1119:in log' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:54:in raw_execute’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:538:in internal_execute' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:137:in execute’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/query_cache.rb:27:in execute' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:48:in execute’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/schema_statements.rb:629:in add_column' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql/schema_statements.rb:418:in add_column’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration/default_strategy.rb:10:in method_missing' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1059:in block in method_missing’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1025:in block in say_with_time' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/benchmark-0.4.0/lib/benchmark.rb:304:in measure’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1025:in say_with_time' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1048:in method_missing’
/var/www/discourse/plugins/chat/db/migrate/20241003122030_add_notification_level_to_user_chat_channel_memberships.rb:10:in up' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:997:in public_send’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:997:in exec_migration' /var/www/discourse/lib/freedom_patches/schema_migration_details.rb:8:in block in exec_migration’

/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/benchmark-0.4.0/lib/benchmark.rb:304:in measure' /var/www/discourse/lib/freedom_patches/schema_migration_details.rb:8:in exec_migration’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:978:in block (2 levels) in migrate' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/benchmark-0.4.0/lib/benchmark.rb:304:in measure’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:977:in block in migrate' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:415:in with_connection’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:976:in migrate' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1190:in migrate’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1538:in block in execute_migration_in_transaction' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1591:in ddl_transaction’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1537:in execute_migration_in_transaction' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1512:in each’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1512:in migrate_without_lock' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1457:in block in migrate’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1609:in with_advisory_lock' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1457:in migrate’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1264:in up' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1239:in migrate’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/tasks/database_tasks.rb:262:in migrate' /var/www/discourse/lib/tasks/db.rake:259:in block (2 levels) in ’
/var/www/discourse/lib/distributed_mutex.rb:53:in block in synchronize' /var/www/discourse/lib/distributed_mutex.rb:49:in synchronize’
/var/www/discourse/lib/distributed_mutex.rb:49:in synchronize' /var/www/discourse/lib/distributed_mutex.rb:34:in synchronize’
/var/www/discourse/lib/tasks/db.rake:234:in block in <main>' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/exe/rake:27:in <top (required)>’

Caused by:
ActiveRecord::StatementInvalid: PG::DuplicateColumn: ERROR: column “notification_level” of relation “user_chat_channel_memberships” already exists (ActiveRecord::StatementInvalid)
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rack-mini-profiler-3.3.1/lib/patches/db/pg.rb:110:in exec' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rack-mini-profiler-3.3.1/lib/patches/db/pg.rb:110:in async_exec’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:56:in block (2 levels) in raw_execute' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:1004:in block in with_raw_connection’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activesupport-7.2.2.1/lib/active_support/concurrency/null_lock.rb:9:in synchronize' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:976:in with_raw_connection’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:55:in block in raw_execute' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activesupport-7.2.2.1/lib/active_support/notifications/instrumenter.rb:58:in instrument’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:1119:in log' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:54:in raw_execute’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:538:in internal_execute' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:137:in execute’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/query_cache.rb:27:in execute' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:48:in execute’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/schema_statements.rb:629:in add_column' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql/schema_statements.rb:418:in add_column’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration/default_strategy.rb:10:in method_missing' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1059:in block in method_missing’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1025:in block in say_with_time' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/benchmark-0.4.0/lib/benchmark.rb:304:in measure’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1025:in say_with_time' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1048:in method_missing’
/var/www/discourse/plugins/chat/db/migrate/20241003122030_add_notification_level_to_user_chat_channel_memberships.rb:10:in up' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:997:in public_send’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:997:in exec_migration' /var/www/discourse/lib/freedom_patches/schema_migration_details.rb:8:in block in exec_migration’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/benchmark-0.4.0/lib/benchmark.rb:304:in measure' /var/www/discourse/lib/freedom_patches/schema_migration_details.rb:8:in exec_migration’

/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:978:in block (2 levels) in migrate' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/benchmark-0.4.0/lib/benchmark.rb:304:in measure’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:977:in block in migrate' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:415:in with_connection’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:976:in migrate' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1190:in migrate’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1538:in block in execute_migration_in_transaction' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1591:in ddl_transaction’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1537:in execute_migration_in_transaction' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1512:in each’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1512:in migrate_without_lock' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1457:in block in migrate’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1609:in with_advisory_lock' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1457:in migrate’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1264:in up' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1239:in migrate’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/tasks/database_tasks.rb:262:in migrate' /var/www/discourse/lib/tasks/db.rake:259:in block (2 levels) in ’
/var/www/discourse/lib/distributed_mutex.rb:53:in block in synchronize' /var/www/discourse/lib/distributed_mutex.rb:49:in synchronize’
/var/www/discourse/lib/distributed_mutex.rb:49:in synchronize' /var/www/discourse/lib/distributed_mutex.rb:34:in synchronize’
/var/www/discourse/lib/tasks/db.rake:234:in block in <main>' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/exe/rake:27:in <top (required)>’

Caused by:
PG::DuplicateColumn: ERROR: column “notification_level” of relation “user_chat_channel_memberships” already exists (PG::DuplicateColumn)
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rack-mini-profiler-3.3.1/lib/patches/db/pg.rb:110:in exec' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rack-mini-profiler-3.3.1/lib/patches/db/pg.rb:110:in async_exec’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:56:in block (2 levels) in raw_execute' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:1004:in block in with_raw_connection’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activesupport-7.2.2.1/lib/active_support/concurrency/null_lock.rb:9:in synchronize' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:976:in with_raw_connection’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:55:in block in raw_execute' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activesupport-7.2.2.1/lib/active_support/notifications/instrumenter.rb:58:in instrument’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:1119:in log' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:54:in raw_execute’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:538:in internal_execute' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:137:in execute’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/query_cache.rb:27:in execute' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:48:in execute’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/schema_statements.rb:629:in add_column' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/postgresql/schema_statements.rb:418:in add_column’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration/default_strategy.rb:10:in method_missing' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1059:in block in method_missing’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1025:in block in say_with_time' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/benchmark-0.4.0/lib/benchmark.rb:304:in measure’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1025:in say_with_time' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1048:in method_missing’
/var/www/discourse/plugins/chat/db/migrate/20241003122030_add_notification_level_to_user_chat_channel_memberships.rb:10:in up' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:997:in public_send’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:997:in exec_migration' /var/www/discourse/lib/freedom_patches/schema_migration_details.rb:8:in block in exec_migration’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/benchmark-0.4.0/lib/benchmark.rb:304:in measure' /var/www/discourse/lib/freedom_patches/schema_migration_details.rb:8:in exec_migration’

/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:978:in block (2 levels) in migrate' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/benchmark-0.4.0/lib/benchmark.rb:304:in measure’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:977:in block in migrate' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:415:in with_connection’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:976:in migrate' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1190:in migrate’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1538:in block in execute_migration_in_transaction' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1591:in ddl_transaction’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1537:in execute_migration_in_transaction' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1512:in each’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1512:in migrate_without_lock' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1457:in block in migrate’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1609:in with_advisory_lock' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1457:in migrate’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1264:in up' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/migration.rb:1239:in migrate’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/activerecord-7.2.2.1/lib/active_record/tasks/database_tasks.rb:262:in migrate' /var/www/discourse/lib/tasks/db.rake:259:in block (2 levels) in ’
/var/www/discourse/lib/distributed_mutex.rb:53:in block in synchronize' /var/www/discourse/lib/distributed_mutex.rb:49:in synchronize’
/var/www/discourse/lib/distributed_mutex.rb:49:in synchronize' /var/www/discourse/lib/distributed_mutex.rb:34:in synchronize’
/var/www/discourse/lib/tasks/db.rake:234:in block in <main>' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/exe/rake:27:in <top (required)>’
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
/var/www/discourse/lib/discourse.rb:173:in execute_command' /var/www/discourse/lib/discourse.rb:139:in exec’
/var/www/discourse/lib/discourse.rb:35:in execute_command' /var/www/discourse/lib/backup_restore/database_restorer.rb:153:in migrate_database’
/var/www/discourse/lib/backup_restore/database_restorer.rb:28:in restore' /var/www/discourse/lib/backup_restore/restorer.rb:61:in run’
script/discourse:191:in restore' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/thor-1.3.2/lib/thor/command.rb:28:in run’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/thor-1.3.2/lib/thor/invocation.rb:127:in invoke_command' /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/thor-1.3.2/lib/thor.rb:538:in dispatch’
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/thor-1.3.2/lib/thor/base.rb:584:in start' script/discourse:345:in <top (required)>’
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/cli/exec.rb:58:in load' /usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/cli/exec.rb:58:in kernel_load’
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/cli/exec.rb:23:in run' /usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/cli.rb:455:in exec’
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/vendor/thor/lib/thor/command.rb:28:in `run’

/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in invoke_command' /usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/vendor/thor/lib/thor.rb:527:in dispatch’
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/cli.rb:35:in dispatch' /usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/vendor/thor/lib/thor/base.rb:584:in start’
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/cli.rb:29:in start' /usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/exe/bundle:28:in block in <top (required)>’
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/friendly_errors.rb:117:in with_friendly_errors' /usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/exe/bundle:20:in <top (required)>’
/usr/local/bin/bundle:25:in load' /usr/local/bin/bundle:25:in ’
Trying to rollback…
Rolling back…
Cleaning stuff up…
Dropping functions from the discourse_functions schema…
Removing tmp ‘/var/www/discourse/tmp/restores/default/2025-01-09-180715’ directory…
Unpausing sidekiq…
Marking restore as finished…
Notifying ‘system’ of the end of the restore…
Finished!
[FAILED]

3 Likes
2

How to fix an SQL file so that db:migrate runs successfully?

1 Like
3

What commit is the old site on?

There have been a couple of regressions where a commit included a migration that it should not have. My guess is that this is one of those cases.

2 Likes
4

Do you have the exact same set of plugins on both instances?
On which commit is your existing Discourse, and on which commit of the chat plugin is it?

1 Like
5

Appreciate your kind involvement. SQL dump is 45GB, takes forever to work on.
Tried running GUI update on Dec’31 that made DB migration hung. Old site is gone since 06-Jan, all I have is SQL dump and media uploads saved. Trying to fire up fresh site and restore my DB there. Running only a few plugins in my YAML:

GitHub - jannolii/discourse-topic-trade-buttons"

1 Like
6

What is the file name of your backup? (It contains information about the last migration that has been run)

1 Like
7

sitename-2025-01-06-033732-v20240912212253.sql.gz

When doing manual restore, following issues reported (might mean something too):
psql:/shared/backups/default/da-stereo-2025-01-06-033732-v20240912212253.sql:842596361: ERROR: could not create unique index “category_featured_topics_pkey”
DETAIL: Key (id)=(14) is duplicated.
psql:/shared/backups/default/da-stereo-2025-01-06-033732-v20240912212253.sql:842596928: ERROR: could not create unique index “incoming_links_pkey”
DETAIL: Key (id)=(1) is duplicated.
psql:/shared/backups/default/da-stereo-2025-01-06-033732-v20240912212253.sql:842596937: ERROR: could not create unique index “incoming_referers_pkey”
DETAIL: Key (id)=(1) is duplicated.
psql:/shared/backups/default/da-stereo-2025-01-06-033732-v20240912212253.sql:842597117: ERROR: could not create unique index “post_hotlinked_media_pkey”
DETAIL: Key (id)=(1) is duplicated.
psql:/shared/backups/default/da-stereo-2025-01-06-033732-v20240912212253.sql:842597378: ERROR: could not create unique index “sitemaps_pkey”
DETAIL: Key (id)=(2) is duplicated.
psql:/shared/backups/default/da-stereo-2025-01-06-033732-v20240912212253.sql:842597693: ERROR: could not create unique index “upload_references_pkey”
DETAIL: Key (id)=(8) is duplicated.

8

So that’s interesting.

  • those conflicts are on fairly trivial tables and the conflicting id’s are low
  • your backup is from September and the column "notification_level" of relation "user_chat_channel_memberships" already exists migration is from October, so it has not been ran on your previous instance, and it is one of the first migrations that is done after the restore.

This sounds to me like you have a different problem. Is this a standard install? And was your previous instance a standard install? I’ve got a hunch that the backup is not restoring in the right schema and the migrations are running against a blank instance.

2 Likes
9

Yes, it is a standard docker install and my original site been created using this manual back in 2015.

Anything I can pull from postgres or rails console to shed some light on what’s going on?

10

Thanks,
Any hints, tips, or direction will be greatly appreciated.

11

Do you mean a sql-only Discourse backup or some SQL dump that you made some other way?

12

Discourse only backup.

At home, I set up PostgreSQL, and our data backup was restored almost without any issues.
It seems that when restoring the file in PostgreSQL (which is inside the Discourse container), the old database is not deleted, and this causes a conflict, preventing the backup from being deployed properly.

13

That sounds like what Richard was saying.

There were some commits around postgres recently, though I’m not sure they’ve been accepted. You might try rebuilding before you try to restore again.

1 Like
14

When restoring sql.gz archive I can see following:
…
Copying archive to tmp directory…
Extracting dump file…
Validating metadata…
Current version: 20241224191732
Restored version: 20240912212253

and obviously db:migration fails with the reported error in post #1

I can see that postgres DB “discourse” restored with all required tables/data in it:
psql (13.18 (Debian 13.18-1.pgdg120+1))
Type “help” for help.

discourse=> \dn+
                                  List of schemas
        Name         |   Owner   |   Access privileges    |      Description
---------------------+-----------+------------------------+------------------------
 backup              | discourse |                        |
 discourse_functions | discourse |                        |
 public              | discourse | discourse=UC/discourse+| standard public schema
                     |           | =UC/discourse          |
(3 rows)

discourse=> discourse=> \l
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges
-----------+----------+----------+-------------+-------------+------------------------
 discourse | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres          +
           |          |          |             |             | postgres=CTc/postgres +
           |          |          |             |             | discourse=CTc/postgres
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres           +
           |          |          |             |             | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres           +
           |          |          |             |             | postgres=CTc/postgres
(4 rows)

but rake repoting that is using wrong DB:

rake db:version

database: discourse
Current version: 20241224191732

How do I point rake towards 20240912212253 ?

Thanks

15

This is what happens when there is a permissions problem. Discourse juggles with schemas during restore and when it does not have permissions to enumerate the tables in the database, things go south.

Although I have never experienced that, this might also be an issue with available disk space?

1 Like
16

Just a gut feeling: could there be a timeout during table movements after a restore?

17

Thanks for hint.

Disk space is not an issue; I have 6-7 times more space than the DB size. I can try a manual import before running the migration and fix the permissions. The question is, what exactly must be granted?

18

I think you need to update manual script in this way:

DROP SCHEMA public CASCADE;
CREATE SCHEMA public;
ALTER SCHEMA public OWNER TO discourse;
CREATE EXTENSION IF NOT EXISTS hstore;
CREATE EXTENSION IF NOT EXISTS pg_trgm;
END;

There is no need to CREATE SCHEMA public (and set owner), cause newer Pg creates schema from SQL backup dump.
However table permissions must be fixed to discourse user, by default they are postgres.

Now it works! Thank you all for ideas and support.

3 Likes