Миграция не-изображений из S3/spaces в локальное хранилище

mcdanlj · 20.Июнь.2020 13:00:43

Множество дополнительных связанных исправлений, теперь подтверждённых реальным процессом миграции, в новом PR

github.com/discourse/discourse

FIX: Make migrations from S3 more robust; fix bare URL migration (#10093)

master ← johnsonm:mkj-fix-more-urls

closed 09:33PM - 17 Jan 21 UTC

johnsonm

+333 -44

Fix handling of bare URLs to (more) correctly process URLs present in s3 or a c…lone when migrating to local, including changing to an `upload:` psuedo-protocol URL, tagging audio and video files, using the `![name](url)` syntax, and in the process not creating non-URLs that point to nothing. Fix `batch_migrate_from_s3` to take both a max number of posts to actually modify in a migration (the original but failed intent of limit) and a limit of total posts to consider for migration (for cheaper database queries). Because a primary goal of the `max` limit is debugging, also use it to cause verbose logging output. Print diagnostic messages for conditions that might require manual intervention, such as file too large (if upload sizes were reduced in site settings) or failure to download (which can happen on an intermittent basis and might require re-running the migration). Finally, in the case of a truly unexpected error, print the error and stop processing, so that error cascades do not become hidden failures stopping a successful migration, but can instead be resolved.

Тема		Ответов	Просм.
Migrating S3 images to local; or from one S3 bucket to another Support	3	1178	29.06.2017
Migrate_to_s3 fails because of logo images Self-hosting s3	1	493	20.04.2023
Migrate_from_s3 problems Bug	19	3469	17.01.2021
Pictures from s3 to local machine Support	2	803	23.01.2018
Migrating old uploads to S3 Self-hosting s3	8	1717	11.05.2021

Миграция не-изображений из S3/spaces в локальное хранилище

Связанные темы