The discussion revolves around moving from AWS S3 to DigitalOcean Spaces for backup and uploads. itsbhanusharma is seeking a low-risk strategy and proposes a step-by-step approach. merefield asks about the driver for moving, and itsbhanusharma responds that it’s primarily for experimentation and cost-effectiveness.
Falco suggests using rclone to migrate data from S3 to Spaces, but itsbhanusharma is more concerned about rewrites, specifically with the plan to switch from CloudFront to Google Cloud CDN or Fastly.
During the migration process, itsbhanusharma encounters issues with assets being served locally and decides to wait before setting up DO Spaces. However, pfaffman raises concerns about DigitalOcean’s ability to run Ceph, citing issues with backups and latency. itsbhanusharma agrees, recommending S3 over Spaces due to latency issues and preferring the S3+CloudFront setup.
I’m trying to figure out a Low-Risk strategy to move from AWS S3 to DO Spaces for Backup and uploads, Initially I am thinking of doing it the ‘long’ way i.e.:
Migrating uploads to local
Removing old S3 Config
Adding new DO Spaces Config
Migrating to Spaces
Alternative path can be to spin up an alternative container and setting it up to use DO Spaces but I think that importing a backup into it will simply override the settings and connect it to S3 once again.
But I’m curious to know if there is a short cut way to do the same using some tool like S3CMD to move assets from S3 to DO and simply changing the settings to link up to DO?
I have in past migrated data from S3 to Spaces using S3CMD and I assume it will go through just fine with rclone as well! What I’m more concerned about is rewrites here! Specifically because currently I’m using S3 in addition with CloudFront but I plan to switching to Google Cloud CDN or fastly once my assets have been migrated.
Update: I Moved uploads to local today and the first thing that I’ve come across is that assets (despite being migrated from s3 to local) are taking a long time to be served locally. I don’t want to risk breaking things so presumably I’ll have to wait a few hours until all the uploads are being served from my forum then only I can proceed to setting up DO Spaces!
It doesn’t look like digitalocean is very good at running ceph (the software that they use for spaces). Their ams3 data center’s cluster is pretty broken now. Backups fairly regularly time out. I was very hopeful about this product but am not recommending it at right now.
Agreed, I’d recommend S3 over Spaces for sanity reasons too. I’ve had latency issues with DO Spaces which isn’t the case where I use the S3+CloudFront set up that powers our forum.