Rake uploads:migrate_to_s3 error

Installation
1

2.4.0.beta9 discourse latest-release +35

> rake uploads:migrate_to_s3
rake aborted!
NoMethodError: undefined method `name' for :vapid_public_key:Symbol
/var/www/discourse/plugins/procourse-custom-homepage/lib/dl_custom_homepage/engine.rb:42:in `block in <top (required)>'
/var/www/discourse/lib/discourse_event.rb:14:in `block in trigger'
/var/www/discourse/lib/discourse_event.rb:13:in `trigger'
/var/www/discourse/lib/site_setting_extension.rb:355:in `add_override!'
/var/www/discourse/lib/site_setting_extension.rb:508:in `block in setup_methods'
/var/www/discourse/config/initializers/100-push-notifications.rb:18:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.1/lib/active_support/dependencies.rb:319:in `load'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.1/lib/active_support/dependencies.rb:319:in `block in load'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.1/lib/active_support/dependencies.rb:291:in `load_dependency'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.1/lib/active_support/dependencies.rb:319:in `load'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/engine.rb:667:in `block in load_config_initializer'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.1/lib/active_support/notifications.rb:182:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/engine.rb:666:in `load_config_initializer'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/engine.rb:624:in `block (2 levels) in <class:Engine>'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/engine.rb:623:in `each'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/engine.rb:623:in `block in <class:Engine>'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/initializable.rb:32:in `instance_exec'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/initializable.rb:32:in `run'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/initializable.rb:61:in `block in run_initializers'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/initializable.rb:50:in `each'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/initializable.rb:50:in `tsort_each_child'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/initializable.rb:60:in `run_initializers'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/application.rb:363:in `initialize!'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/railtie.rb:190:in `public_send'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/railtie.rb:190:in `method_missing'
/var/www/discourse/config/environment.rb:7:in `<top (required)>'
2

still when i git pull source code and rebuild app to (f7a7263164)

rake uploads:migrate_to_s3
Checking if default already migrated...
2 of 300 uploads are not migrated to S3. S3 migration failed for db 'default'.
Looking for missing uploads on: default

0 post uploads are missing.

No posts require rebaking
Migrating uploads to S3 for 'default'...
Uploading files to S3...
 - Listing local files => 284 files
 - Listing S3 files. => 285 files
 - Syncing files to S3
............................................................................................................................................................................................................................................................................................
Updating the URLs in the database...
Removing old optimized images...
Flagging all posts containing lightboxes for rebake...
3 posts were flagged for a rebake
rake aborted!
2 of 300 uploads are not migrated to S3. S3 migration failed for db 'default'.
/var/www/discourse/lib/tasks/uploads.rake:238:in `migration_successful?'
/var/www/discourse/lib/tasks/uploads.rake:551:in `migrate_to_s3'
/var/www/discourse/lib/tasks/uploads.rake:216:in `block in migrate_to_s3_all_sites'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rails_multisite-2.0.7/lib/rails_multisite/connection_management.rb:63:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rails_multisite-2.0.7/lib/rails_multisite/connection_management.rb:73:in `each_connection'
/var/www/discourse/lib/tasks/uploads.rake:214:in `migrate_to_s3_all_sites'
/var/www/discourse/lib/tasks/uploads.rake:210:in `block in <top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'
Tasks: TOP => uploads:migrate_to_s3
(See full trace by running task with --trace)
3
  1. rake posts:rebake is ok
    even tried` DISCOURSE_S3_BUCKET=“antivteassets/uploads” ’

DISCOURSE_S3_BUCKET="antivteassets"
2.

rake uploads:migrate_to_s3 --trace
** Invoke uploads:migrate_to_s3 (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute uploads:migrate_to_s3
Checking if default already migrated...
2 of 301 uploads are not migrated to S3. S3 migration failed for db 'default'.
** Invoke posts:missing_uploads (first_time)
** Invoke environment 
** Execute posts:missing_uploads
Looking for missing uploads on: default

0 post uploads are missing.

No posts require rebaking
Migrating uploads to S3 for 'default'...
Uploading files to S3...
 - Listing local files => 284 files
 - Listing S3 files. => 286 files
 - Syncing files to S3
............................................................................................................................................................................................................................................................................................
Updating the URLs in the database...
Removing old optimized images...
Flagging all posts containing lightboxes for rebake...
69 posts were flagged for a rebake
rake aborted!
2 of 301 uploads are not migrated to S3. S3 migration failed for db 'default'.
/var/www/discourse/lib/tasks/uploads.rake:238:in `migration_successful?'
/var/www/discourse/lib/tasks/uploads.rake:551:in `migrate_to_s3'
/var/www/discourse/lib/tasks/uploads.rake:216:in `block in migrate_to_s3_all_sites'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rails_multisite-2.0.7/lib/rails_multisite/connection_management.rb:63:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rails_multisite-2.0.7/lib/rails_multisite/connection_management.rb:73:in `each_connection'
/var/www/discourse/lib/tasks/uploads.rake:214:in `migrate_to_s3_all_sites'
/var/www/discourse/lib/tasks/uploads.rake:210:in `block in <top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:281:in `block in execute'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:281:in `each'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:281:in `execute'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/usr/local/lib/ruby/2.6.0/monitor.rb:235:in `mon_synchronize'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:199:in `invoke_with_call_chain'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:188:in `invoke'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:160:in `invoke_task'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:116:in `each'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:116:in `block in top_level'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:125:in `run_with_threads'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:110:in `top_level'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:83:in `block in run'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:186:in `standard_exception_handling'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:80:in `run'
bin/rake:13:in `<top (required)>'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli/exec.rb:63:in `load'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli/exec.rb:63:in `kernel_load'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli/exec.rb:28:in `run'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli.rb:476:in `exec'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/vendor/thor/lib/thor.rb:399:in `dispatch'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli.rb:30:in `dispatch'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/vendor/thor/lib/thor/base.rb:476:in `start'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli.rb:24:in `start'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/exe/bundle:46:in `block in <top (required)>'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/friendly_errors.rb:123:in `with_friendly_errors'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/exe/bundle:34:in `<top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'
Tasks: TOP => uploads:migrate_to_s3
4

Looks like this plugin is the root of the error.

3 Likes
5

actually

ls: cannot access '/var/www/discourse/plugins/procourse-custom-homepage': No such file or directory

all images files have beed transfed to s3 already

6

Remove that plugin and rebuild

7

i dont have this one whether ls directory or on site setting admin-cumstmize page

8

weired .with a rebuild old app,and rerun migrate

here test cdn upload - 血栓之家

** Invoke uploads:migrate_to_s3 (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute uploads:migrate_to_s3
Checking if default already migrated...
2 of 301 uploads are not migrated to S3. S3 migration failed for db 'default'.
** Invoke posts:missing_uploads (first_time)
** Invoke environment 
** Execute posts:missing_uploads
Looking for missing uploads on: default

0 post uploads are missing.

54 posts still require rebaking and will be rebaked during regular job
Migrating uploads to S3 for 'default'...
Uploading files to S3...
 - Listing local files => 286 files
 - Listing S3 files. => 294 files
 - Syncing files to S3
..............................................................................................................................................................................................................................................................................................
Updating the URLs in the database...
Removing old optimized images...
Flagging all posts containing lightboxes for rebake...
69 posts were flagged for a rebake
rake aborted!
2 of 301 uploads are not migrated to S3. S3 migration failed for db 'default'.
/var/www/discourse/lib/tasks/uploads.rake:238:in `migration_successful?'
/var/www/discourse/lib/tasks/uploads.rake:551:in `migrate_to_s3'
/var/www/discourse/lib/tasks/uploads.rake:216:in `block in migrate_to_s3_all_sites'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rails_multisite-2.0.7/lib/rails_multisite/connection_management.rb:63:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rails_multisite-2.0.7/lib/rails_multisite/connection_management.rb:73:in `each_connection'
/var/www/discourse/lib/tasks/uploads.rake:214:in `migrate_to_s3_all_sites'
/var/www/discourse/lib/tasks/uploads.rake:210:in `block in <top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:281:in `block in execute'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:281:in `each'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:281:in `execute'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/usr/local/lib/ruby/2.6.0/monitor.rb:235:in `mon_synchronize'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:199:in `invoke_with_call_chain'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:188:in `invoke'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:160:in `invoke_task'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:116:in `each'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:116:in `block in top_level'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:125:in `run_with_threads'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:110:in `top_level'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:83:in `block in run'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:186:in `standard_exception_handling'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:80:in `run'
bin/rake:13:in `<top (required)>'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli/exec.rb:63:in `load'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli/exec.rb:63:in `kernel_load'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli/exec.rb:28:in `run'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli.rb:476:in `exec'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/vendor/thor/lib/thor.rb:399:in `dispatch'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli.rb:30:in `dispatch'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/vendor/thor/lib/thor/base.rb:476:in `start'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli.rb:24:in `start'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/exe/bundle:46:in `block in <top (required)>'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/friendly_errors.rb:123:in `with_friendly_errors'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/exe/bundle:34:in `<top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'
Tasks: TOP => uploads:migrate_to_s3

make a fresh installation of discourse to test,with the same s3 setting ,successfully run rebake and migrate command ,when images upload done ,url is still not the cdn url + image filename
https://cp.antivte.com/t/topic/14

9

it seems i use Fortify in cloudflare for
Nine out of ten doctors agree that [Content-Security-Policy](https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP), [HSTS](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Strict-Transport-Security), and [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) Headers are part of a well-balanced breakfast.
and just add s3 to header like this ,woola

Access-Control-Allow-Origin: https://bbs.antivte.com
Content-Security-Policy: default-src 'self' *.amazonaws.com google-analytics.com