Extend S3 configuration for other S3 API compatible services

feature
#39

Yes, it works with uploads! :+1:

No, previous uploads would be not be affected. They don’t transfer automatically and only the future uploads will be uploaded to the new provider when you switch. I will check if it’s possible/feasible to migrate from one method to another.

1 Like
#40

awesome, so how do we setup digitalocean spaces any tutorials out there?

#41

I’ve almost finished writing the tutorial, it’ll be up on Meta very soon!

Edit: For everyone following this thread, here’s a link to the tutorial: :tada:

8 Likes
#42

I am going to try to migrate my existing images on S3 to Digital Ocean Spaces. An obvious path is to use the rake tasks to move the images back to the DO droplet and then run a eventually-available rake task to move the images from the DO droplet to DO Spaces. Since my droplet doesn’t have enough space for all the S3 images, I’ve come up a potential idea that lets me bypass my droplet’s space constraint. Can anyone comment whether it would work, or share if you see obvious issues?

The idea:

  • Download all S3 images into my own hard drive, then upload them all into Digital Ocean spaces, maintaining all folder structure.
  • Use a SQL query to change the ‘url’ of any file in the ‘uploads’ table from the S3 domain to the DO Space domain

That seems to be a faster way to do things, since the migration rake tasks had been very slow for me in the past, like 1mb / second.

I can try with one file, but want to ask anyone sees any obvious issues before I try it.

Thanks!

3 Likes
#43

You could create additional space with digital Ocean whatever they call the bigger space you can attach. That’d be much simpler.

1 Like
#44

The only issue is the upsizing is permanent. And if I want to scale down the size afterwards, I have to migrate the forum to another droplet. I am not sure how involved that is.

#45

No. You can add just extra space in a separate partition. It’s on the same menu where you’d add a droplet.

1 Like
#46

Oh, yes. I forgot to say that my droplet is in a region where there is no block storage available. To do that I have to set up a new droplet and migrate the droplet over.

1 Like
#47

Oh. Bummer. Sorry, I forgot that some regions don’t have block storage (as you might guess since I can’t even remember what they call it!). If I understand your situation, though, I’d be inclined to move to one that does have block storage.

4 Likes
#48
7 Likes
#49

can we use minio instead of digital ocean?

1 Like
#50

Yes this has been tested @rishabh can you update details both here and at: Minio instead of S3?

3 Likes
#51

I need one more day to finish it, it’s a bit longer than the s3 guide. I’ll update the details in both places :+1:

4 Likes
#52

Can we also migrate previous data from our server to minio?

#53

Migrate from s3 or from local?

#54

migrate from local to minio

#55

@hosna This should help you set up Minio uploads/backups:
https://meta.discourse.org/t/setting-up-backup-and-image-uploads-to-minio-s3/95123

Note: I’ll be adding a section for enabling HTTPS on a Minio Server very soon!

6 Likes
#56

Does it as a plugin form better than build-in? More explain about the upload or some new API expose make it flex.

#57

A guide and a list of compatible services lives at Using Object Storage for Uploads (S3 Clones).

2 Likes
#58