Migrate to minio

[zh99998]

I configured minio as image upload storage, it works well for new uploads, but I can’t migrate olds into that.

root@9c9267547230:/var/www/discourse# rake uploads:migrate_to_s3
Migrating uploads to S3 for 'default'...
Some uploads were not migrated to the new scheme. Please run these commands in the rails console

SiteSetting.migrate_to_new_scheme = true
Jobs::MigrateUploadScheme.new.execute(nil)

when I execute Jobs::MigrateUploadScheme.new.execute(nil) first time, it takes several hours and returned some exception, but seems finally success.

I execute it second time, it returns an empty array [].

and then I try to rake uploads:migrate_to_s3 again, it still says Some uploads were not migrated to the new scheme.

[zh99998]

they’re site icons.

[hosna]

This is a known issue, already reported in the following topic:

S3 migrations from/to minio problems

I recently tried to configure my discourse to use minio according to this topic: https://meta.discourse.org/t/setting-up-backup-and-image-uploads-to-minio-s3/95123 lets say my discourse domain is example.com and my minio endpoint is minio.example.com and my bucket name is test. I faced the following problems 1- Migration from local to minio. I did this cd /var/discourse ./launcher enter app rake uploads:migrate_to_s3 rake posts:rebake It seems all files are migrated to minio however all li…

It’s been a long time since it was promised to be fixed. However unfortunately it’s not fixed yet.

[zh99998]

is there any known work-around solution?

[pfaffman (Jay Pfaffman)]

I’m currently having a similar problem with Spaces.

[hosna]

I think It should be possible to manually transfer upload files from local to minio by minio client (mc). Then update links by a rake task. However I didn’t try it cause I cant risk it on my community.

In your case if its only a handful of site icons, maybe just upload them manually and update links in your dashboard.

[codinghorror (Jeff Atwood)]

So this is ultimately CORS related as well, @pfaffman?

[pfaffman (Jay Pfaffman)]

The problem that I think he was referring to is that migrate_to_s3 won’t run unless everything has been changed to the new scheme but the default/stock images are in the old scheme. And the process that updates them didn’t touch the stock images. (and if you run an update they get recreated)

[zogstrip]

Can you update to latest and run the rake task again?

I fixed the rake task to only check for uploads made by users otherwise it wouldn’t run.

[pfaffman (Jay Pfaffman)]

Oooh! . by_users! I didn’t know about that one. I’ll try to have a look, though it seems pretty clear they should fix it.

I suspect my task is still running.

[zogstrip]

@pfaffman

I didn’t know about that one.

That’s because it’s new-ish. Was added in

https://github.com/discourse/discourse/commit/b0c8fdd7da20fc4eeb30575050360fbf47ec4c8f

[zh99998]

ooh sorry I have already migrated it manually and can’t do this again.