Rake uploads:migrate_to_s3 Fehler

wanghaisheng · 5. Januar 2020 um 11:08

2.4.0.beta9 discourse latest-release +35

> rake uploads:migrate_to_s3
rake aborted!
NoMethodError: undefined method `name' for :vapid_public_key:Symbol
/var/www/discourse/plugins/procourse-custom-homepage/lib/dl_custom_homepage/engine.rb:42:in `block in <top (required)>'
/var/www/discourse/lib/discourse_event.rb:14:in `block in trigger'
/var/www/discourse/lib/discourse_event.rb:13:in `trigger'
/var/www/discourse/lib/site_setting_extension.rb:355:in `add_override!'
/var/www/discourse/lib/site_setting_extension.rb:508:in `block in setup_methods'
/var/www/discourse/config/initializers/100-push-notifications.rb:18:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.1/lib/active_support/dependencies.rb:319:in `load'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.1/lib/active_support/dependencies.rb:319:in `block in load'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.1/lib/active_support/dependencies.rb:291:in `load_dependency'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.1/lib/active_support/dependencies.rb:319:in `load'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/engine.rb:667:in `block in load_config_initializer'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.1/lib/active_support/notifications.rb:182:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/engine.rb:666:in `load_config_initializer'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/engine.rb:624:in `block (2 levels) in <class:Engine>'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/engine.rb:623:in `each'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/engine.rb:623:in `block in <class:Engine>'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/initializable.rb:32:in `instance_exec'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/initializable.rb:32:in `run'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/initializable.rb:61:in `block in run_initializers'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/initializable.rb:50:in `each'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/initializable.rb:50:in `tsort_each_child'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/initializable.rb:60:in `run_initializers'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/application.rb:363:in `initialize!'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/railtie.rb:190:in `public_send'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/railtie.rb:190:in `method_missing'
/var/www/discourse/config/environment.rb:7:in `<top (required)>'

wanghaisheng · 5. Januar 2020 um 12:31

Immer noch, wenn ich den Quellcode per git pull hole und die App auf (f7a7263164) neu erstelle:

rake uploads:migrate_to_s3
Prüfe, ob die Standardmigration bereits durchgeführt wurde...
2 von 300 Uploads wurden nicht zu S3 migriert. Die S3-Migration für die Datenbank 'default' ist fehlgeschlagen.
Suche nach fehlenden Uploads auf: default

Keine Post-Uploads fehlen.

Keine Beiträge müssen neu aufbereitet werden.
Migriere Uploads zu S3 für 'default'...
Lade Dateien zu S3 hoch...
 - Auflisten lokaler Dateien => 284 Dateien
 - Auflisten von S3-Dateien => 285 Dateien
 - Synchronisieren der Dateien mit S3
............................................................................................................................................................................................................................................................................................
Aktualisiere die URLs in der Datenbank...
Entferne alte optimierte Bilder...
Markiere alle Beiträge mit Lightboxes zur Neubearbeitung...
3 Beiträge wurden zur Neubearbeitung markiert.
rake abgebrochen!
2 von 300 Uploads wurden nicht zu S3 migriert. Die S3-Migration für die Datenbank 'default' ist fehlgeschlagen.
/var/www/discourse/lib/tasks/uploads.rake:238:in `migration_successful?'
/var/www/discourse/lib/tasks/uploads.rake:551:in `migrate_to_s3'
/var/www/discourse/lib/tasks/uploads.rake:216:in `block in migrate_to_s3_all_sites'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rails_multisite-2.0.7/lib/rails_multisite/connection_management.rb:63:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rails_multisite-2.0.7/lib/rails_multisite/connection_management.rb:73:in `each_connection'
/var/www/discourse/lib/tasks/uploads.rake:214:in `migrate_to_s3_all_sites'
/var/www/discourse/lib/tasks/uploads.rake:210:in `block in <top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'
Tasks: TOP => uploads:migrate_to_s3
(Vollständigen Trace durch Ausführen der Aufgabe mit --trace anzeigen)

wanghaisheng · 5. Januar 2020 um 13:30

rake posts:rebake funktioniert
auch versucht DISCOURSE_S3_BUCKET="antivteassets/uploads"

DISCOURSE_S3_BUCKET="antivteassets"
2.

rake uploads:migrate_to_s3 --trace
** Invoke uploads:migrate_to_s3 (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute uploads:migrate_to_s3
Checking if default already migrated...
2 of 301 uploads are not migrated to S3. S3 migration failed for db 'default'.
** Invoke posts:missing_uploads (first_time)
** Invoke environment 
** Execute posts:missing_uploads
Looking for missing uploads on: default

0 post uploads are missing.

No posts require rebaking
Migrating uploads to S3 for 'default'...
Uploading files to S3...
 - Listing local files => 284 files
 - Listing S3 files. => 286 files
 - Syncing files to S3
............................................................................................................................................................................................................................................................................................
Updating the URLs in the database...
Removing old optimized images...
Flagging all posts containing lightboxes for rebake...
69 posts were flagged for a rebake
rake aborted!
2 of 301 uploads are not migrated to S3. S3 migration failed for db 'default'.
/var/www/discourse/lib/tasks/uploads.rake:238:in `migration_successful?'
/var/www/discourse/lib/tasks/uploads.rake:551:in `migrate_to_s3'
/var/www/discourse/lib/tasks/uploads.rake:216:in `block in migrate_to_s3_all_sites'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rails_multisite-2.0.7/lib/rails_multisite/connection_management.rb:63:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rails_multisite-2.0.7/lib/rails_multisite/connection_management.rb:73:in `each_connection'
/var/www/discourse/lib/tasks/uploads.rake:214:in `migrate_to_s3_all_sites'
/var/www/discourse/lib/tasks/uploads.rake:210:in `block in <top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:281:in `block in execute'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:281:in `each'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:281:in `execute'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/usr/local/lib/ruby/2.6.0/monitor.rb:235:in `mon_synchronize'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:199:in `invoke_with_call_chain'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:188:in `invoke'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:160:in `invoke_task'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:116:in `each'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:116:in `block in top_level'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:125:in `run_with_threads'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:110:in `top_level'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:83:in `block in run'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:186:in `standard_exception_handling'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:80:in `run'
bin/rake:13:in `<top (required)>'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli/exec.rb:63:in `load'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli/exec.rb:63:in `kernel_load'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli/exec.rb:28:in `run'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli.rb:476:in `exec'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/vendor/thor/lib/thor.rb:399:in `dispatch'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli.rb:30:in `dispatch'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/vendor/thor/lib/thor/base.rb:476:in `start'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli.rb:24:in `start'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/exe/bundle:46:in `block in <top (required)>'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/friendly_errors.rb:123:in `with_friendly_errors'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/exe/bundle:34:in `<top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'
Tasks: TOP => uploads:migrate_to_s3

Falco · 5. Januar 2020 um 17:21

Anscheinend ist dieses Plugin die Ursache des Fehlers.

wanghaisheng · 6. Januar 2020 um 03:12

Tatsächlich:

ls: Zugriff auf '/var/www/discourse/plugins/procourse-custom-homepage' nicht möglich: Datei oder Verzeichnis nicht gefunden

Alle Bilddateien wurden bereits auf S3 übertragen.

codinghorror · 6. Januar 2020 um 04:15

Entferne dieses Plugin und erstelle es neu.

wanghaisheng · 6. Januar 2020 um 04:17

Ich habe das nicht, weder im ls-Verzeichnis noch auf der Admin-Anpassungsseite der Site-Einstellungen.

wanghaisheng · 6. Januar 2020 um 04:30

Seltsam. Mit einer neu aufgebauten alten App und erneutem Ausführen von migrate

Hier: https://bbs.antivte.com/t/topic/194

** Invoke uploads:migrate_to_s3 (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute uploads:migrate_to_s3
Prüfe, ob die Standardmigration bereits durchgeführt wurde...
2 von 301 Uploads wurden nicht nach S3 migriert. Die S3-Migration für die Datenbank 'default' ist fehlgeschlagen.
** Invoke posts:missing_uploads (first_time)
** Invoke environment 
** Execute posts:missing_uploads
Suche nach fehlenden Uploads in: default

0 Post-Uploads fehlen.

54 Posts müssen noch neu gebacken werden und werden während der regulären Jobs neu gebacken.
Migriere Uploads nach S3 für 'default'...
Lade Dateien nach S3 hoch...
 - Auflisten lokaler Dateien => 286 Dateien
 - Auflisten von S3-Dateien => 294 Dateien
 - Synchronisieren der Dateien nach S3
..............................................................................................................................................................................................................................................................................................
Aktualisiere die URLs in der Datenbank...
Entferne alte optimierte Bilder...
Markiere alle Posts mit Lightboxes zum Neubebacken...
69 Posts wurden zum Neubebacken markiert
rake abgebrochen!
2 von 301 Uploads wurden nicht nach S3 migriert. Die S3-Migration für die Datenbank 'default' ist fehlgeschlagen.
/var/www/discourse/lib/tasks/uploads.rake:238:in `migration_successful?'
/var/www/discourse/lib/tasks/uploads.rake:551:in `migrate_to_s3'
/var/www/discourse/lib/tasks/uploads.rake:216:in `block in migrate_to_s3_all_sites'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rails_multisite-2.0.7/lib/rails_multisite/connection_management.rb:63:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rails_multisite-2.0.7/lib/rails_multisite/connection_management.rb:73:in `each_connection'
/var/www/discourse/lib/tasks/uploads.rake:214:in `migrate_to_s3_all_sites'
/var/www/discourse/lib/tasks/uploads.rake:210:in `block in <top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:281:in `block in execute'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:281:in `each'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:281:in `execute'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/usr/local/lib/ruby/2.6.0/monitor.rb:235:in `mon_synchronize'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:199:in `invoke_with_call_chain'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/task.rb:188:in `invoke'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:160:in `invoke_task'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:116:in `each'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:116:in `block in top_level'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:125:in `run_with_threads'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:110:in `top_level'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:83:in `block in run'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:186:in `standard_exception_handling'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rake-13.0.1/lib/rake/application.rb:80:in `run'
bin/rake:13:in `<top (required)>'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli/exec.rb:63:in `load'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli/exec.rb:63:in `kernel_load'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli/exec.rb:28:in `run'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli.rb:476:in `exec'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/vendor/thor/lib/thor.rb:399:in `dispatch'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli.rb:30:in `dispatch'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/vendor/thor/lib/thor/base.rb:476:in `start'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/cli.rb:24:in `start'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/exe/bundle:46:in `block in <top (required)>'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/lib/bundler/friendly_errors.rb:123:in `with_friendly_errors'
/usr/local/lib/ruby/gems/2.6.0/gems/bundler-2.1.1/exe/bundle:34:in `<top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'
Tasks: TOP => uploads:migrate_to_s3

Eine frische Installation von Discourse zum Testen erstellt, mit denselben S3-Einstellungen. Der Befehl zum Neubebacken und Migrieren wurde erfolgreich ausgeführt. Nach dem Hochladen der Bilder ist die URL jedoch immer noch nicht die CDN-URL + Dateiname.
https://cp.antivte.com/t/topic/14

wanghaisheng · 6. Januar 2020 um 09:06

Es scheint, dass ich Fortify in Cloudflare für
Neun von zehn Ärzten sind sich einig, dass [Content-Security-Policy](https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP), [HSTS](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Strict-Transport-Security) und [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS)-Header Teil eines ausgewogenen Frühstücks sind.
verwende und einfach S3 wie folgt zum Header hinzufüge, wolla

Access-Control-Allow-Origin: https://bbs.antivte.com
Content-Security-Policy: default-src 'self' *.amazonaws.com google-analytics.com

Thema		Antworten	Aufrufe
Rake uploads:migrate_from_s3 fails Self-hosting s3	31	3067	24. April 2021
1 of 16551 uploads are not migrated to S3. S3 migration failed for db 'default' Self-hosting s3 , cdn	22	1799	10. August 2023
Cannot execute the rake uploads:migrate_from_s3 Bug	7	2223	28. Januar 2018
Stop using Amazon s3 for uploads Support	52	5030	22. August 2021
Uploads migration to S3 fails Support	26	3212	7. Mai 2021

Rake uploads:migrate_to_s3 Fehler

Verwandte Themen