Digitalocean block storage VS amazon S3

I had got a warning in my command line about disc space before rebuilding my container while installing a Discourse plugin. I had to run some commands to remove some older versions of linux on my digitalocean droplet, but I’m still over 50% ssd capacity.

As my Discourse website has grown, I started looking at storage solutions. I’m currently on a 5$ digitalocean droplet and have read around about digitalocean block storage and amazon s3. I see the S3 settings built into the Discourse settings, so I’m guessing its easier to setup.

What do you use for your website? Which has better loading speeds, s3 vs digiocean blocks? What option is cheaper? Wouldn’t the free tier of s3 make it cheaper than digiocean blocks? Do you store your images AND backups? Or do you recommend another option outside of digiocean and amazon?

1 Like

Hi,

There are a lot of Providers that offer S3 compatible storage solutions. You can use either of those. I’ve used a few and can recommend that Digital Ocean Spaces works great.

If You’re already hosting on DO, I’d recommend going with spaces as it’s fast and reliable.

However, there are some concerns relating to their CDN so you may have to consider a 3rd party CDN.

Check this topic for more information:

1 Like

S3 is obviously the more mature and more tested solution, but price wise DO offering may make sense for your community.

No matter what you pick, you must put a CDN in front of the bucket, so don’t forget that.

3 Likes

Would the free tier of Cloudflare work as the CDN in front of the bucket? I already use them with my domain settings for the free DDOS protection.

I noticed DigitalOcean advertises their Spaces service is cheaper than the other well known providers. So that’s definitely a plus.

Because the CDN that comes with DigitalOcean Spaces doesn’t work correctly with Discourse, I’m thinking I should just go with one of the other providers on your list. That way I don’t have to pay for DigitalOcean Spaces and a separate CDN service.

I’ll post this here, because the Using Object Storage for Uploads (S3 Clones) is set to delete all replies after a month.

A missing content-encoding in the DO CDN implementation definitely is a bummer, but it only happens when setting all the S3 parameters inside app.yml. If those parameters are set in the site settings via web admin console, the DO only serves the S3 uploads via CDN, and the site assets are still served from the source.

It seems that this is by design, because the DISCOURSE_S3_CDN_URL env variable, if set in the app.yml, overrides the CDN setting for assets too, while this same setting, when declared in site settings only, doesn’t?

This is a bit inconsistent, but it allows the DO CDN to be used for S3 uploads only, without breaking the site:

There are two ways to do this:

  • declare all S3 settings in site settings only
rails c
SiteSetting.s3_upload_bucket="<bucket_name>/<uploads_folder>"
SiteSetting.s3_backup_bucket="<bucket_name>/<backups_folder>"
SiteSetting.enable_s3_uploads=true
SiteSetting.s3_access_key_id="<key>"
SiteSetting.s3_secret_access_key="<secret_key>"
SiteSetting.s3_endpoint="https://<sfo2>.digitaloceanspaces.com"
SiteSetting.s3_cdn_url="https://<bucket_name>.<sfo2>.cdn.digitaloceanspaces.com/<uploads_folder>"
SiteSetting.backup_location="s3"
  • shadow all S3 settings except DISCOURSE_S3_CDN_URL in the app.yml, and declare DO CDN in SiteSetting.s3_cdn_url
  DISCOURSE_USE_S3: true
  DISCOURSE_S3_REGION: <sfo2>
  DISCOURSE_S3_ENDPOINT: https://<sfo2>.digitaloceanspaces.com
  DISCOURSE_S3_ACCESS_KEY_ID: <key>
  DISCOURSE_S3_SECRET_ACCESS_KEY: <secret_key>
#  DISCOURSE_S3_CDN_URL: https://<bucket_name>.<sfo2>.cdn.digitaloceanspaces.com/<uploads_folder>
  DISCOURSE_S3_BUCKET: <bucket_name>/<uploads_folder>
  DISCOURSE_S3_BACKUP_BUCKET: <bucket_name>/<backup_folder>
  DISCOURSE_BACKUP_LOCATION: s3
rails c
SiteSetting.s3_cdn_url="https://<bucket_name>.<sfo2>.cdn.digitaloceanspaces.com/<uploads_folder>"

@falco if you agree with this I can update the original wiki topic

I don’t think that is a good idea.

  1. All other providers, even AWS, require a CDN. DO offering a free one, that is broken, is just a a quirk we can ignore it and treat it like a standard Object Storage service.

  2. I’m not sure that the implementation detail of setting S3 CDN URL in the yml and on the site setting resulting in different behaviors is something we want to promote, as it can be fixed anytime.

I have not tested it, but I can assure it works with real CDN services, like KeyCDN, Cloudfront, StackPath, etc.

2 Likes