Today I did a test of creating a full backup of a Teams site and restoring it to a self-hosted site set up using discourse/INSTALL-cloud.md at master · discourse/discourse · GitHub . This worked. S3 was already disabled and everything worked as expected. It was a small test site without much content.
One step to keep in mind for a seamless transition is to make sure you have all the plugins installed that were enabled on the Teams site. To find this list, you want to go to /admin/plugins on the Teams site, and then look here on meta for the links to the official plugins on github. Then add those plugins to your app.yml file on your new self-hosted site.
@biyaniyash I think the problem you ran into, which we talked about by email, is that your backups did not contain the uploads. You now have the correct backup file with the uploads to use to restore to your self-hosted site. Let us know how you get on and if you learn anything in the process.
2 Likes
Hi Community,
Unfortunately still running into errors upon restore in the self-hosted site. I also upgraded it to the latest release but to no avail. The only thing left is installing the plugins (right now I have not installed any custom plugins on the self-hosted site)
I have requested Tobias and the discourse team to share the app.yml file from my teams site or the code that I could copy paste and they are currently helping with that. But would love for any ideas on how to get the site up and running, even if the plugins come in later.
The restore has failed.
Here’s the log:
[2021-05-11 17:11:08] [STARTED]
[2021-05-11 17:11:08] 'biyaniyash' has started the restore!
[2021-05-11 17:11:08] Marking restore as running...
[2021-05-11 17:11:08] Making sure /var/www/discourse/tmp/restores/default/2021-05-11-171108 exists...
[2021-05-11 17:11:08] Copying archive to tmp directory...
[2021-05-11 17:11:08] Unzipping archive, this may take a while...
[2021-05-11 17:11:10] Extracting dump file...
[2021-05-11 17:11:10] Validating metadata...
[2021-05-11 17:11:10] Current version: 20210420015635
[2021-05-11 17:11:10] Restored version: 20210429154322
[2021-05-11 17:11:10] EXCEPTION: You're trying to restore a more recent version of the schema. You should migrate first!
[2021-05-11 17:11:10] /var/www/discourse/lib/backup_restore/meta_data_handler.rb:31:in `validate'
/var/www/discourse/lib/backup_restore/restorer.rb:113:in `validate_backup_metadata'
/var/www/discourse/lib/backup_restore/restorer.rb:43:in `run'
/var/www/discourse/script/spawn_backup_restore.rb:23:in `restore'
/var/www/discourse/script/spawn_backup_restore.rb:36:in `block in <main>'
/var/www/discourse/script/spawn_backup_restore.rb:4:in `fork'
/var/www/discourse/script/spawn_backup_restore.rb:4:in `<main>'
[2021-05-11 17:11:10] Trying to rollback...
[2021-05-11 17:11:10] There was no need to rollback
[2021-05-11 17:11:10] Cleaning stuff up...
[2021-05-11 17:11:10] Removing tmp '/var/www/discourse/tmp/restores/default/2021-05-11-171108' directory...
[2021-05-11 17:11:10] Marking restore as finished...
[2021-05-11 17:11:10] Notifying 'biyaniyash' of the end of the restore...
3 Likes
This line in your log seems to be the clue:
[2021-05-11 17:11:10] EXCEPTION: You’re trying to restore a more recent version of the schema. You should migrate first!
I am not a huge expert at this but I think you need to make sure the discourse version is the same as the backup.
Just to confirm - when you updated discourse you did it from the command line, right? With the ./launcher rebuild app command.
2 Likes
Looks like there is a plugin migration script that requires that you have at least the assign, policy and solved plugins installed.
Here’s the full list of plugins we include in Teams at the moment - this is subject to change in future. You can just copy/paste this list into the app.yml under the line referencing the docker manager. Be sure to match the indentation exactly! These are all official plugins so are safe to install even if you decide not to use them. There are some additional plugins not included here that provide the Teams sidebar and features specific to Teams hosting.
- git clone https://github.com/discourse/discourse-assign.git
- git clone https://github.com/discourse/discourse-cakeday.git
- git clone https://github.com/discourse/discourse-calendar.git
- git clone https://github.com/discourse/discourse-chat-integration.git
- git clone https://github.com/discourse/discourse-checklist.git
- git clone https://github.com/discourse/discourse-code-review.git
- git clone https://github.com/discourse/discourse-docs.git
- git clone https://github.com/discourse/discourse-encrypt.git
- git clone https://github.com/discourse/discourse-footnote.git
- git clone https://github.com/discourse/discourse-github.git
- git clone https://github.com/discourse/discourse-graphviz.git
- git clone https://github.com/discourse/discourse-math.git
- git clone https://github.com/discourse/discourse-openid-connect.git
- git clone https://github.com/discourse/discourse-policy.git
- git clone https://github.com/discourse/discourse-shared-edits.git
- git clone https://github.com/discourse/discourse-solved.git
- git clone https://github.com/discourse/discourse-spoiler-alert.git
3 Likes
Hey Tobias,
Whoops, I upgraded via the admin panel.
Aarrrgh.
2 Likes
Inching ever closer!
Can you look at /logs and tell me if you see any errors? If you do, paste the error here.
2 Likes
OK, I have now also finished a backup/restore from a Teams site to self-hosted, with uploads and all plugins in app.yml, and am getting the same 500 error when looking at latest. I am able to access the admin section.
I see an error in the log that looks relevant. It seems to be about s3 but in the admin settings, s3 is clearly disabled on my self-hosted site.
See log:
/var/www/discourse/app/models/global_setting.rb:107:in `s3_bucket_name'
/var/www/discourse/app/models/site_setting.rb:157:in `absolute_base_url'
/var/www/discourse/app/models/upload.rb:186:in `secure_media_url_from_upload_url'
/var/www/discourse/lib/url_helper.rb:60:in `secure_proxy_without_cdn'
/var/www/discourse/lib/url_helper.rb:83:in `cook_url'
/var/www/discourse/app/models/topic_link.rb:274:in `ensure_entry_for'
/var/www/discourse/app/models/topic_link.rb:128:in `block (2 levels) in extract_from'
activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:320:in `block in transaction'
activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/transaction.rb:310:in `block in within_new_transaction'
activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize'
activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/transaction.rb:308:in `within_new_transaction'
activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:320:in `transaction'
activerecord-6.1.3.2/lib/active_record/transactions.rb:209:in `transaction'
/var/www/discourse/app/models/topic_link.rb:126:in `block in extract_from'
/var/www/discourse/app/models/topic_link.rb:124:in `each'
/var/www/discourse/app/models/topic_link.rb:124:in `extract_from'
/var/www/discourse/app/models/post.rb:691:in `rebake!'
/var/www/discourse/app/models/post.rb:645:in `block in rebake_old'
/var/www/discourse/app/models/post.rb:639:in `each'
/var/www/discourse/app/models/post.rb:639:in `rebake_old'
/var/www/discourse/app/jobs/scheduled/periodical_updates.rb:29:in `execute'
/var/www/discourse/app/jobs/base.rb:232:in `block (2 levels) in perform'
rails_multisite-3.0.0/lib/rails_multisite/connection_management.rb:80:in `with_connection'
/var/www/discourse/app/jobs/base.rb:221:in `block in perform'
/var/www/discourse/app/jobs/base.rb:217:in `each'
/var/www/discourse/app/jobs/base.rb:217:in `perform'
/var/www/discourse/app/jobs/base.rb:279:in `perform'
mini_scheduler-0.13.0/lib/mini_scheduler/manager.rb:93:in `process_queue'
mini_scheduler-0.13.0/lib/mini_scheduler/manager.rb:37:in `block (2 levels) in initialize'
And also, I attempted a rebake of posts, and saw the same error there. Here’s the full log:
Summary
downcase' for nil:NilClass /var/www/discourse/app/models/global_setting.rb:107:in s3_bucket_name’absolute_base_url' /var/www/discourse/app/models/upload.rb:186:in secure_media_url_from_upload_url’secure_proxy_without_cdn' /var/www/discourse/lib/url_helper.rb:83:in cook_url’ensure_entry_for' /var/www/discourse/app/models/topic_link.rb:128:in block (2 levels) in extract_from’block in transaction' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/transaction.rb:310:in block in within_new_transaction’block (2 levels) in synchronize' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in handle_interrupt’block in synchronize' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in handle_interrupt’synchronize' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/transaction.rb:308:in within_new_transaction’transaction' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/transactions.rb:209:in transaction’block in extract_from' /var/www/discourse/app/models/topic_link.rb:124:in each’extract_from' /var/www/discourse/app/models/post.rb:691:in rebake!’rebake_post' /var/www/discourse/lib/tasks/posts.rake:125:in block (2 levels) in rebake_posts’each' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/relation/delegation.rb:88:in each’block in rebake_posts' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/core_ext/range/each.rb:14:in step’step' /var/www/discourse/lib/tasks/posts.rake:123:in rebake_posts’block in rebake_posts_all_sites' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rails_multisite-3.0.0/lib/rails_multisite/connection_management.rb:80:in with_connection’each_connection' /var/www/discourse/lib/tasks/posts.rake:106:in rebake_posts_all_sites’block in <main>' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/task.rb:281:in block in execute’each' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/task.rb:281:in execute’block in invoke_with_call_chain' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/task.rb:199:in synchronize’invoke_with_call_chain' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/task.rb:188:in invoke’invoke_task' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:116:in block (2 levels) in top_level’each' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:116:in block in top_level’run_with_threads' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:110:in top_level’block in run' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:186:in standard_exception_handling’run' bin/rake:13:in <top (required)>’load' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/cli/exec.rb:63:in kernel_load’run' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/cli.rb:494:in exec’run' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in invoke_command’dispatch' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/cli.rb:30:in dispatch’start' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/cli.rb:24:in start’block in <top (required)>' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/friendly_errors.rb:130:in with_friendly_errors’<top (required)>' /usr/local/bin/bundle:23:in load’<main>' 148 / 2590 ( 5.7%) Failed to rebake (topic_id: 6527, post_id: 7981) undefined method downcase’ for nil:NilClasss3_bucket_name' /var/www/discourse/app/models/site_setting.rb:157:in absolute_base_url’secure_media_url_from_upload_url' /var/www/discourse/lib/url_helper.rb:60:in secure_proxy_without_cdn’cook_url' /var/www/discourse/app/models/topic_link.rb:274:in ensure_entry_for’block (2 levels) in extract_from' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:320:in block in transaction’block in within_new_transaction' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in block (2 levels) in synchronize’handle_interrupt' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in block in synchronize’handle_interrupt' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in synchronize’within_new_transaction' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:320:in transaction’transaction' /var/www/discourse/app/models/topic_link.rb:126:in block in extract_from’each' /var/www/discourse/app/models/topic_link.rb:124:in extract_from’rebake!' /var/www/discourse/lib/tasks/posts.rake:140:in rebake_post’block (2 levels) in rebake_posts' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/relation/delegation.rb:88:in each’each' /var/www/discourse/lib/tasks/posts.rake:124:in block in rebake_posts’step' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/core_ext/range/each.rb:14:in step’rebake_posts' /var/www/discourse/lib/tasks/posts.rake:107:in block in rebake_posts_all_sites’with_connection' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rails_multisite-3.0.0/lib/rails_multisite/connection_management.rb:90:in each_connection’rebake_posts_all_sites' /var/www/discourse/lib/tasks/posts.rake:7:in block in ’block in execute' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/task.rb:281:in each’execute' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/task.rb:219:in block in invoke_with_call_chain’synchronize' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/task.rb:199:in invoke_with_call_chain’invoke' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:160:in invoke_task’block (2 levels) in top_level' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:116:in each’block in top_level' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:125:in run_with_threads’top_level' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:83:in block in run’standard_exception_handling' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:80:in run’<top (required)>' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/cli/exec.rb:63:in load’kernel_load' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/cli/exec.rb:28:in run’exec' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor/command.rb:27:in run’invoke_command' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor.rb:392:in dispatch’dispatch' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor/base.rb:485:in start’start' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/exe/bundle:49:in block in <top (required)>’with_friendly_errors' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/exe/bundle:37:in <top (required)>’load' /usr/local/bin/bundle:23:in ’downcase' for nil:NilClass /var/www/discourse/app/models/global_setting.rb:107:in s3_bucket_name’absolute_base_url' /var/www/discourse/app/models/upload.rb:186:in secure_media_url_from_upload_url’secure_proxy_without_cdn' /var/www/discourse/lib/url_helper.rb:83:in cook_url’ensure_entry_for' /var/www/discourse/app/models/topic_link.rb:128:in block (2 levels) in extract_from’block in transaction' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/transaction.rb:310:in block in within_new_transaction’block (2 levels) in synchronize' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in handle_interrupt’block in synchronize' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in handle_interrupt’synchronize' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/transaction.rb:308:in within_new_transaction’transaction' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/transactions.rb:209:in transaction’block in extract_from' /var/www/discourse/app/models/topic_link.rb:124:in each’extract_from' /var/www/discourse/app/models/post.rb:691:in rebake!’rebake_post' /var/www/discourse/lib/tasks/posts.rake:125:in block (2 levels) in rebake_posts’each' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/relation/delegation.rb:88:in each’block in rebake_posts' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/core_ext/range/each.rb:14:in step’step' /var/www/discourse/lib/tasks/posts.rake:123:in rebake_posts’block in rebake_posts_all_sites' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rails_multisite-3.0.0/lib/rails_multisite/connection_management.rb:80:in with_connection’each_connection' /var/www/discourse/lib/tasks/posts.rake:106:in rebake_posts_all_sites’block in <main>' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/task.rb:281:in block in execute’each' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/task.rb:281:in execute’block in invoke_with_call_chain' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/task.rb:199:in synchronize’invoke_with_call_chain' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/task.rb:188:in invoke’invoke_task' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:116:in block (2 levels) in top_level’each' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:116:in block in top_level’run_with_threads' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:110:in top_level’block in run' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:186:in standard_exception_handling’run' bin/rake:13:in <top (required)>’load' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/cli/exec.rb:63:in kernel_load’run' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/cli.rb:494:in exec’run' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in invoke_command’dispatch' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/cli.rb:30:in dispatch’start' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/cli.rb:24:in start’block in <top (required)>' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/friendly_errors.rb:130:in with_friendly_errors’<top (required)>' /usr/local/bin/bundle:23:in load’<main>' 523 / 2590 ( 20.2%) Failed to rebake (topic_id: 6448, post_id: 7559) undefined method downcase’ for nil:NilClasss3_bucket_name' /var/www/discourse/app/models/site_setting.rb:157:in absolute_base_url’secure_media_url_from_upload_url' /var/www/discourse/lib/url_helper.rb:60:in secure_proxy_without_cdn’cook_url' /var/www/discourse/app/models/topic_link.rb:274:in ensure_entry_for’block (2 levels) in extract_from' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:320:in block in transaction’block in within_new_transaction' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in block (2 levels) in synchronize’handle_interrupt' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in block in synchronize’handle_interrupt' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in synchronize’within_new_transaction' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:320:in transaction’transaction' /var/www/discourse/app/models/topic_link.rb:126:in block in extract_from’each' /var/www/discourse/app/models/topic_link.rb:124:in extract_from’rebake!' /var/www/discourse/lib/tasks/posts.rake:140:in rebake_post’block (2 levels) in rebake_posts' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/relation/delegation.rb:88:in each’each' /var/www/discourse/lib/tasks/posts.rake:124:in block in rebake_posts’step' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/core_ext/range/each.rb:14:in step’rebake_posts' /var/www/discourse/lib/tasks/posts.rake:107:in block in rebake_posts_all_sites’with_connection' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rails_multisite-3.0.0/lib/rails_multisite/connection_management.rb:90:in each_connection’rebake_posts_all_sites' /var/www/discourse/lib/tasks/posts.rake:7:in block in ’block in execute' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/task.rb:281:in each’execute' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/task.rb:219:in block in invoke_with_call_chain’synchronize' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/task.rb:199:in invoke_with_call_chain’invoke' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:160:in invoke_task’block (2 levels) in top_level' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:116:in each’block in top_level' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:125:in run_with_threads’top_level' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:83:in block in run’standard_exception_handling' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:80:in run’<top (required)>' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/cli/exec.rb:63:in load’kernel_load' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/cli/exec.rb:28:in run’exec' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor/command.rb:27:in run’invoke_command' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor.rb:392:in dispatch’dispatch' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor/base.rb:485:in start’start' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/exe/bundle:49:in block in <top (required)>’with_friendly_errors' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/exe/bundle:37:in <top (required)>’load' /usr/local/bin/bundle:23:in ’downcase' for nil:NilClass /var/www/discourse/app/models/global_setting.rb:107:in s3_bucket_name’absolute_base_url' /var/www/discourse/app/models/upload.rb:186:in secure_media_url_from_upload_url’secure_proxy_without_cdn' /var/www/discourse/lib/url_helper.rb:83:in cook_url’ensure_entry_for' /var/www/discourse/app/models/topic_link.rb:128:in block (2 levels) in extract_from’block in transaction' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/transaction.rb:310:in block in within_new_transaction’block (2 levels) in synchronize' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in handle_interrupt’block in synchronize' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in handle_interrupt’synchronize' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/transaction.rb:308:in within_new_transaction’transaction' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/transactions.rb:209:in transaction’block in extract_from' /var/www/discourse/app/models/topic_link.rb:124:in each’extract_from' /var/www/discourse/app/models/post.rb:691:in rebake!’rebake_post' /var/www/discourse/lib/tasks/posts.rake:125:in block (2 levels) in rebake_posts’each' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/relation/delegation.rb:88:in each’block in rebake_posts' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/core_ext/range/each.rb:14:in step’step' /var/www/discourse/lib/tasks/posts.rake:123:in rebake_posts’block in rebake_posts_all_sites' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rails_multisite-3.0.0/lib/rails_multisite/connection_management.rb:80:in with_connection’each_connection' /var/www/discourse/lib/tasks/posts.rake:106:in rebake_posts_all_sites’block in <main>' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/task.rb:281:in block in execute’each' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/task.rb:281:in execute’block in invoke_with_call_chain' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/task.rb:199:in synchronize’invoke_with_call_chain' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/task.rb:188:in invoke’invoke_task' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:116:in block (2 levels) in top_level’each' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:116:in block in top_level’run_with_threads' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:110:in top_level’block in run' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.3/lib/rake/application.rb:186:in standard_exception_handling’run' bin/rake:13:in <top (required)>’load' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/cli/exec.rb:63:in kernel_load’run' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/cli.rb:494:in exec’run' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in invoke_command’dispatch' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/cli.rb:30:in dispatch’start' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/cli.rb:24:in start’block in <top (required)>' /usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.16/lib/bundler/friendly_errors.rb:130:in with_friendly_errors’<top (required)>' /usr/local/bin/bundle:23:in load’
I had a look on meta for some of these errors and it seems to have come up before, but I did not see any obvious resolutions. I’m sure the solution will be obvious to someone!
3 Likes
Any solutions or thoughts around how we could tackle this? Is there a different way of backup and restore that could be used?
1 Like
Hi Yash! Did you see this request from me above?
It would help if you could confirm that the errors you are getting are the same as the ones I am getting, or if your problem is different. It does appear to me that there is a bug with migrating from Teams to self-hosted.
This is going to take a little bit of time to sort through, sorry! Hope you’re not too inconvenienced, but hope also the learning process will be a positive one as you embark on your self-hosting journey. There is quite a bit to learn at first but once you know it you will be empowered.
If being in this limbo is a problem for your team, you could always return to Discourse for Teams hosting! marketplace
1 Like
Oops, sorry.
Any ETA by when the discourse team would be able to look into this and debug/solve the issue? That would help decide on the alternatives.
1 Like
Cool! It helps to know that the issue you are having is the same one that I am able to replicate on my end.
We’ve extended your Discourse for Teams hosting for a little while longer, so you can keep using it for free until we find and squash the bug. At that point you can cancel hosting and get the latest backup and then restore that to your self-hosted site. Hope that works for you.
Thanks for being a patient and willing guinea pig through this process!
3 Likes
Thank you Tobias! Appreciate the gesture!
2 Likes
@neil found the fix. Good sleuthing!
From the command line:
cd /var/discourse
./launcher enter app
rails c
Upload.update_all(secure: false)
This brings the site back up for me. @biyaniyash please give it a shot and let us know how it goes.
The explanation for this is that, on Discourse for Teams, Secure Media Uploads is enabled which requires S3. On the newly restored self-hosted site, secure media is disabled and uploads are stored locally on the server. But any existing uploads from the backup were already marked as secure, resulting in errors.
3 Likes
Trying this out. Am i doing something wrong here?
What do I need to type after the last command you have given?
1 Like
Nope! You’re doing nothing wrong and you’re done.
The => 270 indicates how many posts were processed. You can now type in exit to exit rails, and then exit again to exit the docker container.
Now go to your forum homepage and see what you see, and report back here!
2 Likes
We’re BACK UP! Wohoo! Thank you @neil and @tobiaseigen and to everyone else who’s been a part of our little discussion!
For all you folks here’s the solution summary:
Ensure the teams backup has sql + file uploads.
Launch self-hosted site on DO using these instructions discourse/INSTALL-cloud.md at master · discourse/discourse · GitHub [Do note, haven’t tested it with one-click DO Marketplace droplet]
Go to /admin/site_settings/category/all_results?filter=s3 on your new site and disable everything related to s3.
Update discourse to latest version. [Recommended: use ./launcher rebuild app command.]
Install all plugins (do check with the discourse team on the latest plugins/check your teams plugin page)
- git clone https://github.com/discourse/discourse-assign.git
- git clone https://github.com/discourse/discourse-cakeday.git
- git clone https://github.com/discourse/discourse-calendar.git
- git clone https://github.com/discourse/discourse-chat-integration.git
- git clone https://github.com/discourse/discourse-checklist.git
- git clone https://github.com/discourse/discourse-code-review.git
- git clone https://github.com/discourse/discourse-docs.git
- git clone https://github.com/discourse/discourse-encrypt.git
- git clone https://github.com/discourse/discourse-footnote.git
- git clone https://github.com/discourse/discourse-github.git
- git clone https://github.com/discourse/discourse-graphviz.git
- git clone https://github.com/discourse/discourse-math.git
- git clone https://github.com/discourse/discourse-openid-connect.git
- git clone https://github.com/discourse/discourse-policy.git
- git clone https://github.com/discourse/discourse-shared-edits.git
- git clone https://github.com/discourse/discourse-solved.git
- git clone https://github.com/discourse/discourse-spoiler-alert.git
Restore your backup.
From the command line:
cd /var/discourse
./launcher enter app
rails c
Upload.update_all(secure: false)
And you’re done! Everything should be in order!
3 Likes
Tobias, just FYI - tried to take another backup on the teams site. The backup issue is still not fixed. You might want to patch that bug that is not taking the file uploads in the backup.
1 Like
Nice! Thanks for the summary of steps, Yash! Would you like to post this in a new topic for folks seeking to switch from Teams to self-hosted? If not, I will do it.
We do not at all support any install method except the official instructions at discourse/docs/INSTALL-cloud.md at main · discourse/discourse · GitHub . The reason for this is that we can’t know for sure if the other methods work and troubleshooting becomes too difficult or even impossible.
This step should no longer be required in future, once this fix @neil made is merged. Might take a few weeks.
discourse:master ← discourse:neil-12
opened 04:14PM - 12 May 21 UTC
After restoring a database from a site that had secure media configured and disa… bling the secure_media setting, any page with an upload will raise errors. This change will allow the uploads to be loaded from the local files. I'm not sure if this is the correct fix, so someone who knows more about secure media should review.
Actually, this functionality is not broken and is working as designed. As explained on the page, you need to ask us for full backups. We don’t advertise this page on Discourse for Teams because there is no need for customers to keep their own backups - they are kept automatically every 12 hours for all hosting customers and stores offsite. In the event you cancel your hosting we provide you with a full backup with uploads that you can download so you can set up shop elsewhere according to your needs.
2 Likes
Hey Tobias,
Thank you for the clarification!
I’m still figuring my way around this community. Should I post this in Hosting? I can’t seem to post in the How-To category. I’m assuming that’s maintained as an official wiki-esque resource. If it’s easier, you could post it.
Just my 2 cents on the backups.
I feel if you explain it clearly on the page, it would definitely help. From what I can tell discourse is built on transparency and authenticity, and this would sit perfectly under “full disclosure” segment.
But anyway since you have disabled the backups altogether on teams from customer admin page, they will ultimately reach out to you and that’s when you can explain the fine print.
2 Likes
I will go ahead and post the howto topic - you are right, it takes privileges™ to post there. When I do I will share the link here and I’d appreciate a review.
I will also think about the backups page to see how it can be improved so it is more transparent what is going on, especially for hosting customers. For self-hosted sites the backup/restore functionality is fairly straightforward. You just have to watch out for yourself that you are not keeping so many backups locally that you run out of storage!
5 Likes
Calling out to discourse community!
Or you can pay for your Discourse for Teams subscription for just a little bit longer instead of cancelling it right away, and then do the migration later when we have ironed out the kinks - let me know. You could also ask for paid help from the community in 
