# Problemi con le migrazioni S3 da/verso Minio
**URL:** https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333
**Category:** Bug
**Created:** [17 Settembre 2018, 9:13am UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333 "2018-09-17T09:13:48Z")
**Posts on this page:** 20
**Page:** 2
### Author:  [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [20 Novembre 2018, 3:47am UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/22 "2018-11-20T03:47:20Z")
What I meant is that the URL for your upload records are incorrect as
```plaintext
url: "/original/1X/dd758eae1e79c7a2b844d571da9c67ca1df02d39.mp3"
```
should be
```plaintext
url: "/uploads/default/original/1X/dd758eae1e79c7a2b844d571da9c67ca1df02d39.mp3",
```
My suggestion for now is to restore from a recent backup and try the migrate\_to\_s3 script again. If it doesn’t work, avoid running the `migrate_from_s3` script so that we can help you debug what is happening.
---
### Author:  [@hosna](https://meta.discourse.org/u/hosna)
#### Post date: [10 Dicembre 2018, 9:57pm UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/23 "2018-12-10T21:57:37Z")
I put a new pic and a new video in [a topic](https://test.talk.zabanshenas.com/t/topic/483) and did the migration process.
According to the logs, the rake task successfully migrated both. However, links aren’t updated in discourse so now they are broken.
```
`Migrating /uploads/default/original/1X/c42728dcfbe57731a145e0eea56644326090753c.mp4 --> //m2.res.zabanshenas.com/talktest/original/1X/c42728dcfbe57731a145e0eea56644326090753c.mp4 took 1.6314957910217345 seconds`
```
---
### Author:  [@hosna](https://meta.discourse.org/u/hosna)
#### Post date: [11 Dicembre 2018, 10:55pm UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/24 "2018-12-11T22:55:29Z")
Is it safe to manually transfer everything inside upload folder to minio and then change all upload links from local to that of minio by a rake task?
---
### Author:  [@parisa](https://meta.discourse.org/u/parisa)
#### Post date: [18 Dicembre 2018, 11:21am UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/25 "2018-12-18T11:21:01Z")
Did you find a way to fix this issue? I also want to try minio.
---
### Author:  [@hosna](https://meta.discourse.org/u/hosna)
#### Post date: [28 Dicembre 2018, 10:04pm UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/26 "2018-12-28T22:04:00Z")
Unfortunately no. My advice is that if you have already a live discourse forum, you shouldn’t migrate it to minio. Cause you will face many problems. I didn’t test it with a completely new instance. But I guess its ok.
---
### Author:  [@hosna](https://meta.discourse.org/u/hosna)
#### Post date: [5 Gennaio 2019, 8:52pm UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/27 "2019-01-05T20:52:02Z")
I’ve seen some [recent commits related to s3](https://github.com/discourse/discourse/search?o=desc&q=s3&s=committer-date&type=Commits) in discourse repository. Is there any chance problems reported here are solved by those comments? 🤔
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [6 Gennaio 2019, 3:47am UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/28 "2019-01-06T03:47:32Z")
Not sure, try it out and let us know!
---
### Author:  [@hosna](https://meta.discourse.org/u/hosna)
#### Post date: [7 Gennaio 2019, 7:09pm UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/29 "2019-01-07T19:09:03Z")
This time, I’m getting the following error:
```plaintext
root@test-app:/var/www/discourse# rake uploads:migrate_to_s3
Migrating uploads to S3 for 'default'...
Some uploads were not migrated to the new scheme. Please run these commands in the rails console
SiteSetting.migrate_to_new_scheme = true
Jobs::MigrateUploadScheme.new.execute(nil)
root@test-app:/var/www/discourse# SiteSetting.migrate_to_new_scheme = true
bash: SiteSetting.migrate_to_new_scheme: command not found
```
I wanted to manually move all of the files to minio then do the migration. Now with this error, it seems I am not able to do that too.
---
### Author:  [@itsbhanusharma](https://meta.discourse.org/u/itsbhanusharma)
#### Post date: [8 Gennaio 2019, 2:28pm UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/30 "2019-01-08T14:28:10Z")
Before running those commands you need to run `rails c` to enter rails console.
---
### Author:  [@hosna](https://meta.discourse.org/u/hosna)
#### Post date: [8 Gennaio 2019, 2:44pm UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/31 "2019-01-08T14:44:01Z")
thank you very much 😊
```plaintext
root@test-app:/var/www/discourse# rails c
SiteSetting.migrate_to_new_scheme = true
Jobs::MigrateUploadScheme.new.execute(nil)
[1] pry(main)> SiteSetting.migrate_to_new_scheme = true
=> true
[2] pry(main)> Jobs::MigrateUploadScheme.new.execute(nil)
=> []
[3] pry(main)>
```
now what should I do? I should exit? how?
---
### Author:  [@itsbhanusharma](https://meta.discourse.org/u/itsbhanusharma)
#### Post date: [8 Gennaio 2019, 2:44pm UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/32 "2019-01-08T14:44:52Z")
That seems like the most valid action to do 😉
`exit`
---
### Author:  [@hosna](https://meta.discourse.org/u/hosna)
#### Post date: [8 Gennaio 2019, 2:46pm UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/33 "2019-01-08T14:46:06Z")
😅
ok but still I get the same error message
```plaintext
root@test-app:/var/www/discourse# rails c
SiteSetting.migrate_to_new_scheme = true
Jobs::MigrateUploadScheme.new.execute(nil)
[1] pry(main)> SiteSetting.migrate_to_new_scheme = true
=> true
[2] pry(main)> Jobs::MigrateUploadScheme.new.execute(nil)
=> []
[3] pry(main)> exit
root@test-app:/var/www/discourse# rake uploads:migrate_to_s3
Migrating uploads to S3 for 'default'...
Some uploads were not migrated to the new scheme. Please run these commands in the rails console
SiteSetting.migrate_to_new_scheme = true
Jobs::MigrateUploadScheme.new.execute(nil)
root@test-app:/var/www/discourse#
```
---
### Author:  [@itsbhanusharma](https://meta.discourse.org/u/itsbhanusharma)
#### Post date: [8 Gennaio 2019, 2:47pm UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/34 "2019-01-08T14:47:54Z")
that’s beyond my understanding at this point so it’s probably not working yet?
---
### Author:  [@hosna](https://meta.discourse.org/u/hosna)
#### Post date: [8 Gennaio 2019, 3:12pm UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/35 "2019-01-08T15:12:02Z")
Idk perhaps @rishabh, @tgxworld, or @sam, can help me. Cause as far as I see, the newest changes to s3 task are done by them 😕
---
### Author:  [@hosna](https://meta.discourse.org/u/hosna)
#### Post date: [9 Gennaio 2019, 12:32pm UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/38 "2019-01-09T12:32:33Z")
What happens if I manually migrate all of the files to minio, and then turn on s3 option without doing the migration rake task? Is it ok?"
---
### Author:  [@hosna](https://meta.discourse.org/u/hosna)
#### Post date: [11 Gennaio 2019, 12:08am UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/39 "2019-01-11T00:08:12Z")
I tried to test migrate to s3 for another time to test if [this commit](https://github.com/discourse/discourse/commit/f181e9cc08a5d3c31598587957f8999b7e62ac50) which was done by @rishabh, fixes some of the problems above, However, I got the following error
```plaintext
rake uploads:migrate_to_s3
rake aborted!
NoMethodError: undefined method `include?' for nil:NilClass
/var/www/discourse/lib/tasks/uploads.rake:217:in `migrate_to_s3'
/var/www/discourse/lib/tasks/uploads.rake:210:in `block in migrate_to_s3_all_sites'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rails_multisite-2.0.4/lib/rails_multisite/connection_management.rb:63:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rails_multisite-2.0.4/lib/rails_multisite/connection_management.rb:73:in `each_connection'
/var/www/discourse/lib/tasks/uploads.rake:210:in `migrate_to_s3_all_sites'
/var/www/discourse/lib/tasks/uploads.rake:206:in `block in '
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `'
Tasks: TOP => uploads:migrate_to_s3
(See full trace by running task with --trace)
```
>
---
### Author:  [@hosna](https://meta.discourse.org/u/hosna)
#### Post date: [16 Gennaio 2019, 11:17am UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/40 "2019-01-16T11:17:15Z")
> [@hosna](#):
>
> ok but still I get the same error message
@zogstrip can you take a look at this? I get this message
```plaintext
Some uploads were not migrated to the new scheme. Please run these commands in the rails console
SiteSetting.migrate_to_new_scheme = true
Jobs::MigrateUploadScheme.new.execute(nil)
```
again and again even after I apply those settings, and I think its because of [this commit](https://github.com/discourse/discourse/commit/5381096bfd4e208657f928601d3792a5f0d1d0c8).
---
### Author:  [@rishabh](https://meta.discourse.org/u/rishabh)
#### Post date: [16 Gennaio 2019, 11:47am UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/41 "2019-01-16T11:47:29Z")
That error is now fixed in [FIX: Use GlobalSetting values instead of ENV variables in migrate\_to\_s3 · discourse/discourse@60790eb · GitHub](https://github.com/discourse/discourse/commit/60790eb006eeaf288740bbfc3cc6eaa602179c13)
* * *
Unfortunately, the Minio setup is currently broken, there are plans in the next few weeks to pull the SiteSetting `s3_force_path_style` that was introduced to support the Minio style urls.
Minio now supports the standard url structure that we use for S3. After the Minio-specific site setting is removed, the migration tasks will be tested and hopefully will work 🤞
The `migrate_to_s3` task will not work properly for Minio urls till this work is done.
---
### Author:  [@hosna](https://meta.discourse.org/u/hosna)
#### Post date: [16 Gennaio 2019, 11:54am UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/42 "2019-01-16T11:54:50Z")
> [@rishabh](#):
>
> That error is now fixed in [FIX: Use GlobalSetting values instead of ENV variables in migrate\_to\_s3 · discourse/discourse@60790eb · GitHub](https://github.com/discourse/discourse/commit/60790eb006eeaf288740bbfc3cc6eaa602179c13)
Yes. I noticed that. Thank you.
So I keep an eye on recent S3 related commit hopefully to see this problem would be fixed soon, before I run out of space in my discourse instance 😅
---
### Author:  [@hosna](https://meta.discourse.org/u/hosna)
#### Post date: [18 Marzo 2019, 6:03pm UTC](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333/43 "2019-03-18T18:03:32Z")
> [@Migrate to minio](https://meta.discourse.org/t/migrate-to-minio/111830/9):
>
> Can you update to latest and run the rake task again? I [fixed the rake task](https://github.com/discourse/discourse/commit/9dbca41152e6f32aa45539ec50f499885b78edd8) to only check for uploads made by users otherwise it wouldn’t run.
I updated to the latest. however still migration to s3 doesn’t work. cc : @zogstrip
```plaintext
Some uploads were not migrated to the new scheme. Please run these commands in the rails console
SiteSetting.migrate_to_new_scheme = true
Jobs::MigrateUploadScheme.new.execute(nil)
```
[Pagina precedente](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333.md?page=1)
[Pagina seguente](https://meta.discourse.org/t/s3-migrations-from-to-minio-problems/97333.md?page=3)