# Migrando uploads não-imagens do S3/spaces para local
**URL:** https://meta.discourse.org/t/migrating-s3-spaces-non-image-uploads-to-local/132484
**Category:** Support
**Created:** [3 Novembro , 2019 12:02 UTC](https://meta.discourse.org/t/migrating-s3-spaces-non-image-uploads-to-local/132484 "2019-11-03T12:02:22Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@mcdanlj](https://meta.discourse.org/u/mcdanlj)
#### Post date: [3 Novembro , 2019 12:02 UTC](https://meta.discourse.org/t/migrating-s3-spaces-non-image-uploads-to-local/132484/1 "2019-11-03T12:02:23Z")
Li a seguinte página:
> [@Migrating S3 images to local; or from one S3 bucket to another](https://meta.discourse.org/t/migrating-s3-images-to-local-or-from-one-s3-bucket-to-another/65313):
>
> We have an old Discourse instance (running latest version but initially installed a while back). It’s running under HTTP with all its images stored on an AWS S3 bucket with . (dots) in the name. We want to move to HTTPS – configuring for Lets Encrypt works a treat but all the images are broken as S3 buckets with (dots) don’t appear to do SSL. I’m thinking the simplest option would be to migrate all the images to the local instance – as its not an image heavy forum. But I can’t seem to work o…
Então, examinei `lib/tasks/uploads.rake:migrate_from_s3` e encontrei:
```plaintext
.where("raw LIKE '%.s3%.amazonaws.com/%' OR raw LIKE '%(upload://%'")
```
No entanto, notei que os uploads de vídeo não recebem o pseudo-protocolo `upload://` literal, mas sim acabam como links literais para o provedor de armazenamento (no meu caso, o DigitalOcean Spaces).
Parece óbvio que terei que modificar essa tarefa para que ela funcione.
Faria mais sentido verificar `SiteSetting.s3_endpoint` e `SiteSetting.s3_upload_bucket` em vez de, ou além da, referência literal à Amazon?
---
### Author:  [@mcdanlj](https://meta.discourse.org/u/mcdanlj)
#### Post date: [19 Novembro , 2019 02:36 UTC](https://meta.discourse.org/t/migrating-s3-spaces-non-image-uploads-to-local/132484/2 "2019-11-19T02:36:51Z")
Existem testes para as tarefas? Não vejo nenhum. Tenho o que pode ser algo como a correção óbvia, mas sem como aprimorar os testes existentes e sem uma maneira fácil de testar de forma não destrutiva. Isso me deixa preocupado…
```diff
index 0761c4712a..63f49155f3 100644
--- a/lib/tasks/uploads.rake
+++ b/lib/tasks/uploads.rake
@@ -129,12 +129,12 @@ def migrate_from_s3
Post
.where("user_id > 0")
- .where("raw LIKE '%.s3%.amazonaws.com/%' OR raw LIKE '%(upload://%'")
+ .where("raw LIKE '%.s3%.amazonaws.com/%' OR raw LIKE '%#{SiteSetting.Upload.absolute_base_url}%' OR raw LIKE '%(upload://%'")
.find_each do |post|
begin
updated = false
- post.raw.gsub!(/(\/\/[\w.-]+amazonaws\.com\/(original|optimized)\/([a-z0-9]+\/)+\h{40}([\w.-]+)?)/i) do |url|
+ post.raw.gsub!(/(\/\/[\w.-]+(amazonaws\.com|#{Regexp.quote(SiteSetting.s3_endpoint)})\/(original|optimized)\/([a-z0-9]+\/)+\h{40}([\w.-]+)?)/i) do |url|
begin
if filename = guess_filename(url, post.raw)
file = FileHelper.download("http:#{url}", max_file_size: max_file_size, tmp_file_name: "from_s3", follow_redirect: true)
```
Além disso, pela experiência, espero que, mesmo que todas essas imagens já tenham sido otimizadas, o sistema decida que precisa gastar 10 dias reotimizando todos os 50 GB de imagens (96 GB de arquivos no total, originais + otimizadas) enquanto as move, desativando todas as notificações por e-mail do nosso site inteiro durante o processo. Como não tenho uma boa maneira de testar, achei por bem perguntar se é esse o caso; se for, gostaria de saber se há uma maneira de contornar isso, ou seja, apenas copiar as imagens já otimizadas.
Posso copiar facilmente todos os arquivos para o sistema local usando o MinIO Client. Estou curioso sobre o quão difícil seria simplesmente colocar os arquivos no lugar certo e modificar o banco de dados para apontar para o novo local, sem reotimizar todas essas imagens…
---
### Author:  [@mcdanlj](https://meta.discourse.org/u/mcdanlj)
#### Post date: [17 Maio , 2020 23:06 UTC](https://meta.discourse.org/t/migrating-s3-spaces-non-image-uploads-to-local/132484/3 "2020-05-17T23:06:19Z")
>
Ainda não testei, mas pelo menos foi compartilhado como um PR em vez de apenas um post no meta.
---
### Author:  [@mcdanlj](https://meta.discourse.org/u/mcdanlj)
#### Post date: [20 Junho , 2020 13:00 UTC](https://meta.discourse.org/t/migrating-s3-spaces-non-image-uploads-to-local/132484/4 "2020-06-20T13:00:43Z")
Muitas outras correções relacionadas, agora validadas pelo processo real de migração, em um novo PR
>
>
> Fix handling of bare URLs to (more) correctly process URLs present
> in s3 or a c…lone when migrating to local, including changing to an
> \`upload:\` psuedo-protocol URL, tagging audio and video files, using
> the \`!\[name\](url)\` syntax, and in the process not creating non-URLs
> that point to nothing.
>
> Fix \`batch\_migrate\_from\_s3\` to take both a max number of posts to
> actually modify in a migration (the original but failed intent of
> limit) and a limit of total posts to consider for migration (for
> cheaper database queries). Because a primary goal of the \`max\`
> limit is debugging, also use it to cause verbose logging output.
>
> Print diagnostic messages for conditions that might require manual
> intervention, such as file too large (if upload sizes were reduced
> in site settings) or failure to download (which can happen on an
> intermittent basis and might require re-running the migration).
>
> Finally, in the case of a truly unexpected error, print the error
> and stop processing, so that error cascades do not become hidden
> failures stopping a successful migration, but can instead be resolved.