Separation of S3 backups provider from primary S3 provider

I think it would be a nice addition to have the flexibility to specify a different S3 provider for backups, rather than needing to use the same provider as the main S3 bucket provider.

Primary reasons:

  • Requirements for a S3 bucket storing static assets will often differ from one storing backups. E.g. Strong network performance is important for a bucket storing static assets, but not so much for a backups bucket (beyond being reliable). Security may be a higher concern for a backups bucket.

  • For a backups bucket, the ideal provider has competitive pricing per GB storage, with egress pricing not being of much concern.

  • Easier to change providers to solve issues with specific providers. E.g. Scaleway S3 works perfectly as a primary S3 bucket for me, however with backups there are issues because they only support 1,000 part multipart uploads, where as the AWS s3 SDK uses 10,000 as the maximum. I’ve solved this using pups replacing the maximum parts in the S3 gem, but it’s quite makeshift and I need to check if the path for the file has changed on each update/rebuild. For me to solve this problem, I’d need to migrate the primary bucket in order to switch the backups bucket to a different compatible provider.

  • Better security easily, if the backup bucket provider or account is compromised, the primary bucket is elsewhere. It wouldn’t help as much in the reverse situation, if the main bucket provider was compromised and wiped for example (seeing S3 static assets aren’t included in the backups) - but at least the malicious party wouldn’t get access to the backups data.

  • May be easier to provide access to employees or contractors that is limited to either the main bucket or backup backet if the provider doesn’t have advanced access/role management per api key.

Anyhow, just a suggestion - cheers!

2 Likes