Digitalocean block storage مقابل amazon S3

[Bill]

لقد تلقيت تحذيرًا في سطر الأوامر بشأن مساحة القرص قبل إعادة بناء الحاوية أثناء تثبيت إضافة Discourse. اضطرتني إلى تشغيل بعض الأوامر لإزالة بعض إصدارات لينكس الأقدم من قطرة DigitalOcean الخاصة بي، لكنني لا أزال أستهلك أكثر من 50% من سعة القرص الصلب من نوع SSD.

مع نمو موقعي الإلكتروني على Discourse، بدأت أفكر في حلول التخزين. أنا حالياً أستخدم قطرة DigitalOcean بسعر 5 دولارات، وقد قرأت بعض المعلومات حول تخزين الكتل في DigitalOcean وAmazon S3. أرى أن إعدادات S3 مدمجة في إعدادات Discourse، لذا أفترض أنها أسهل في الإعداد.

ما الذي تستخدمونه لمواقعكم الإلكترونية؟ أيهما يوفر سرعة تحميل أفضل: S3 أم تخزين الكتل في DigitalOcean؟ وأي خيار أرخص؟ ألا يجعل المستوى المجاني من S3 الخيار أرخص من تخزين الكتل في DigitalOcean؟ هل تخزنون الصور والنسخ الاحتياطية معًا؟ أم أنكم توصون بخيار آخر خارج نطاق DigitalOcean وAmazon؟

[itsbhanusharma (Bhanu Sharma)]

Hi,

There are a lot of Providers that offer S3 compatible storage solutions. You can use either of those. I’ve used a few and can recommend that Digital Ocean Spaces works great.

If You’re already hosting on DO, I’d recommend going with spaces as it’s fast and reliable.

However, there are some concerns relating to their CDN so you may have to consider a 3rd party CDN.

Check this topic for more information:

Configure an S3 compatible object storage provider for uploads

This topic covers how to configure some common S3 compatible Object Storage providers (S3 clones). See Set up file and image uploads to S3 for more details about Amazon AWS S3 configuration, which is officially supported and used internally by Discourse for our hosting services. Provider Service Name Works with Discourse? Amazon AWS S3 Yes Digital Ocean Spaces Yes Linode Object Storage Yes Google Cloud Storage Yes Scaleway Object Storage Yes Vultr Obj…

[Falco]

S3 is obviously the more mature and more tested solution, but price wise DO offering may make sense for your community.

No matter what you pick, you must put a CDN in front of the bucket, so don’t forget that.

[Bill]

Would the free tier of Cloudflare work as the CDN in front of the bucket? I already use them with my domain settings for the free DDOS protection.

I noticed DigitalOcean advertises their Spaces service is cheaper than the other well known providers. So that’s definitely a plus.

Because the CDN that comes with DigitalOcean Spaces doesn’t work correctly with Discourse, I’m thinking I should just go with one of the other providers on your list. That way I don’t have to pay for DigitalOcean Spaces and a separate CDN service.

[md-misko]

@Bill

Because the CDN that comes with DigitalOcean Spaces doesn’t work correctly with Discourse, I’m thinking I should just go with one of the other providers on your list. That way I don’t have to pay for DigitalOcean Spaces and a separate CDN service.

I’ll post this here, because the Configure an S3 compatible object storage provider for uploads is set to delete all replies after a month.

@Falco

@md-misko

@Falco can you share any specifics on this CDN brokenness?

It’s what is documented here:

Warning

Due to a known issue, file metadata headers like Content-Encoding are not passed through the CDN. Metadata headers are correctly set when fetching content directly from the origin.

A missing content-encoding means all our brotli encoded assets are broken.

That means you can use their Object Storage offering, but must shop for a caching CDN in another provider.

A missing content-encoding in the DO CDN implementation definitely is a bummer, but it only happens when setting all the S3 parameters inside app.yml. If those parameters are set in the site settings via web admin console, the DO only serves the S3 uploads via CDN, and the site assets are still served from the source.

It seems that this is by design, because the DISCOURSE_S3_CDN_URL env variable, if set in the app.yml, overrides the CDN setting for assets too, while this same setting, when declared in site settings only, doesn’t?

This is a bit inconsistent, but it allows the DO CDN to be used for S3 uploads only, without breaking the site:

There are two ways to do this:

• declare all S3 settings in site settings only

rails c
SiteSetting.s3_upload_bucket="<bucket_name>/<uploads_folder>"
SiteSetting.s3_backup_bucket="<bucket_name>/<backups_folder>"
SiteSetting.enable_s3_uploads=true
SiteSetting.s3_access_key_id="<key>"
SiteSetting.s3_secret_access_key="<secret_key>"
SiteSetting.s3_endpoint="https://<sfo2>.digitaloceanspaces.com"
SiteSetting.s3_cdn_url="https://<bucket_name>.<sfo2>.cdn.digitaloceanspaces.com/<uploads_folder>"
SiteSetting.backup_location="s3"

• shadow all S3 settings except DISCOURSE_S3_CDN_URL in the app.yml, and declare DO CDN in SiteSetting.s3_cdn_url

  DISCOURSE_USE_S3: true
  DISCOURSE_S3_REGION: <sfo2>
  DISCOURSE_S3_ENDPOINT: https://<sfo2>.digitaloceanspaces.com
  DISCOURSE_S3_ACCESS_KEY_ID: <key>
  DISCOURSE_S3_SECRET_ACCESS_KEY: <secret_key>
#  DISCOURSE_S3_CDN_URL: https://<bucket_name>.<sfo2>.cdn.digitaloceanspaces.com/<uploads_folder>
  DISCOURSE_S3_BUCKET: <bucket_name>/<uploads_folder>
  DISCOURSE_S3_BACKUP_BUCKET: <bucket_name>/<backup_folder>
  DISCOURSE_BACKUP_LOCATION: s3

rails c
SiteSetting.s3_cdn_url="https://<bucket_name>.<sfo2>.cdn.digitaloceanspaces.com/<uploads_folder>"

@falco if you agree with this I can update the original wiki topic

[Falco]

@md-misko

@falco if you agree with this I can update the original wiki topic

I don’t think that is a good idea.

1. All other providers, even AWS, require a CDN. DO offering a free one, that is broken, is just a a quirk we can ignore it and treat it like a standard Object Storage service.

2. I’m not sure that the implementation detail of setting S3 CDN URL in the yml and on the site setting resulting in different behaviors is something we want to promote, as it can be fixed anytime.

@Bill

Would the free tier of Cloudflare work as the CDN in front of the bucket?

I have not tested it, but I can assure it works with real CDN services, like KeyCDN, Cloudfront, StackPath, etc.