Digitalocean block storage VS amazon S3

[md-misko]

@Bill

Because the CDN that comes with DigitalOcean Spaces doesn’t work correctly with Discourse, I’m thinking I should just go with one of the other providers on your list. That way I don’t have to pay for DigitalOcean Spaces and a separate CDN service.

I’ll post this here, because the Using Object Storage for Uploads (S3 Clones) is set to delete all replies after a month.

@Falco

@md-misko

@Falco can you share any specifics on this CDN brokenness?

It’s what is documented here:

Warning

Due to a known issue, file metadata headers like Content-Encoding are not passed through the CDN. Metadata headers are correctly set when fetching content directly from the origin.

A missing content-encoding means all our brotli encoded assets are broken.

That means you can use their Object Storage offering, but must shop for a caching CDN in another provider.

A missing content-encoding in the DO CDN implementation definitely is a bummer, but it only happens when setting all the S3 parameters inside app.yml. If those parameters are set in the site settings via web admin console, the DO only serves the S3 uploads via CDN, and the site assets are still served from the source.

It seems that this is by design, because the DISCOURSE_S3_CDN_URL env variable, if set in the app.yml, overrides the CDN setting for assets too, while this same setting, when declared in site settings only, doesn’t?

This is a bit inconsistent, but it allows the DO CDN to be used for S3 uploads only, without breaking the site:

There are two ways to do this:

• declare all S3 settings in site settings only

rails c
SiteSetting.s3_upload_bucket="<bucket_name>/<uploads_folder>"
SiteSetting.s3_backup_bucket="<bucket_name>/<backups_folder>"
SiteSetting.enable_s3_uploads=true
SiteSetting.s3_access_key_id="<key>"
SiteSetting.s3_secret_access_key="<secret_key>"
SiteSetting.s3_endpoint="https://<sfo2>.digitaloceanspaces.com"
SiteSetting.s3_cdn_url="https://<bucket_name>.<sfo2>.cdn.digitaloceanspaces.com/<uploads_folder>"
SiteSetting.backup_location="s3"

• shadow all S3 settings except DISCOURSE_S3_CDN_URL in the app.yml, and declare DO CDN in SiteSetting.s3_cdn_url

  DISCOURSE_USE_S3: true
  DISCOURSE_S3_REGION: <sfo2>
  DISCOURSE_S3_ENDPOINT: https://<sfo2>.digitaloceanspaces.com
  DISCOURSE_S3_ACCESS_KEY_ID: <key>
  DISCOURSE_S3_SECRET_ACCESS_KEY: <secret_key>
#  DISCOURSE_S3_CDN_URL: https://<bucket_name>.<sfo2>.cdn.digitaloceanspaces.com/<uploads_folder>
  DISCOURSE_S3_BUCKET: <bucket_name>/<uploads_folder>
  DISCOURSE_S3_BACKUP_BUCKET: <bucket_name>/<backup_folder>
  DISCOURSE_BACKUP_LOCATION: s3

rails c
SiteSetting.s3_cdn_url="https://<bucket_name>.<sfo2>.cdn.digitaloceanspaces.com/<uploads_folder>"

@falco if you agree with this I can update the original wiki topic