S3 image bandwidth costs are getting annoying

I have noticed our site’s S3 image service bandwidth (transfer) costs keep going up and up. The pricing is $.09/GB and that can add up pretty fast… I assume I am not the only site with this issue? I am paying Amazon $70/month for transfer alone now, that’s more than my VM costs. The S3 storage cost is only $5/month, the whole problem is transfer charges.

I don’t have any CDN or caching set up, just a totally generic Discourse running on a Linode VM using the S3 image buckets. What are bigger sites with lots of images doing? CDN (I assume that will cut costs)? Using Digital Ocean Spaces or Minio? Something else I am missing??

7 Likes

Using a CDN for S3 is the right approach. That will drive the transfer costs way down.

If you are in a tight budget, you may be able to use a cloudflare subdomain for that.

10 Likes

Hmm… looking at the bandwidth costs on Fastly and the Amazon CloudFront (and others) they are pretty much the same as the .09/GB I am paying now :frowning:

Cloudflare seems to not charge for bandwidth?? Maybe that is the thing to try then. Is there a post on how to hook up Cloudflare for S3 images CDN on Discourse? I see the s3_cdn_url but can’t find instructions on how to use it (other than I need to rebake once I set it up). Looking at Cloudflare it seems like you need to have your own domain since the DNS needs to be changed… so I would have to make a CNAME something like s3-cloudflare.mydomain.com which would point to my S3 bucket IP, and tell Cloudflare to do it’s thing on that URL, and also set s3_cdn_url to that, and then rebake?

Here is a support page I found at Cloudflare which seems to indicate this might work:

Anybody doing this?? I am a bit surprised there are not many sites with this issue as it is more than doubling the monthly cost for me at least.

5 Likes

I’m using Cloudflare on my Discourse site and it works fine. You basically route all your traffic through Cloudflare and tell it what to process or cache and that’s it. There are a few things to keep in mind though:

  1. Configure Cloudflare to NOT cache anything (set up exceptions using page rules, see below).
  2. Don’t let Cloudflare touch your Javascript - specifically, turn off Rocket Loader.
  3. I’m not sure about Brotli compression yet, I disabled it.
  4. Set up page rules to cache the avatar and upload directories as below (ignore rule #1, it’s site specific).

Cloudflare is currently caching a little over 3TB of data per month for me and it’s serving large graphics pretty fast.

If you’re more interested in a CDN I can also recommend https://www.belugacdn.com/ (I’m not affiliated with them). I use them on a large WordPress platform and performance is good, at a fraction of the cost of some other CDN platforms I looked at (those might be even more performant, but Beluga meets my needs).

17 Likes

Did you ever move away from Amazon CloudFront in the end @scottfsmith ?

I’m finding myself in a similar boat in terms of rising expenditure.

CloudFront is quite new to me. I’ve been running only an S3 bucket for the last few years but based on recommendations here I setup CloudFront last month.

All this has done is move the cost away from S3 data transfer and over to CloudFront data transfer instead, the total monthly bill is still the same.

I find myself approaching an unsustainable monthly bill for data transfer.

2 Likes

I’m in the same boat. I also moved to CloudFront not too long ago and the costs are looking similar. I had thought there was some sort of savings due to the local caching, i.e. the bandwidth would be less if the same image repeatedly loaded from the same geographical location, but so far it looks like not.

It seems like there should be some other CDN that should work in place of CloudFront, all Discourse is doing is rewriting image URLS so it is pretty generic (it seems). Maybe Cloudflare could be used for the images only.

3 Likes

That was exactly my understanding too.

+98% of our traffic is UK based, and naturally 98% of our S3 egress via CloudFront is also to the UK.

I’m seeing zero benefit of having a CDN thus far. If our traffic was worldwide, perhaps I might be seeing something different.

I’ll have a read up about Cloudflare (how did these two get away with such similar names!) in the coming days and see if I can find any benefit.

Thanks :+1:t2:

4 Likes

Please follow up here if you find anything. I have been meaning to look into this myself.

2 Likes

@Richie & @scottfsmith,

There are some fixed cost CDNs like https://www.stackpath.com/ for $10 that I used to do all the testing for Using Object Storage for Uploads (S3 & Clones). They may be a better fit for your community.

6 Likes

Stackpath CDN is $20/month with a cap at 1TB and in many months I am over that. Then it bumps to $200/month for the 20TB cap which is more than the $100/month or so Amazon is charging me.

Maybe there is another one in that list which is better though, thanks for the reminder about the list.

2 Likes

imgur is free for all , but no plugin for this

Tobey Maguire Reaction GIF

4 Likes

What would such a plugin do? Would it just not download IMGUR plugins, or would it work as the current system does and upload all images to IMGUR through some API interface that presumably exists?

What would it be worth to you? I doubt that it’s something that makes sense for the CDCK business model, as their paid hosting is what makes Discourse possible for all of us, and their hosting works just fine without an IMGUG plugin.

6 Likes

Can someone please confirm if this config works well in 2022 or not? :slight_smile:

1 Like

We’re still using the Cloudflare setup yes.

2 Likes

Thanks for confirming.

The setup you’re using is not full-page caching, right?

Or is there any special benefit of using Cloudflare CDN over others?

1 Like

No, we only cache the uploads and avatars directories; see our rules above. If you try to do anything else you’re likely to break Discourse.

It’s free :slight_smile:

4 Likes

What about static assets like JS and CSS?

If we use a CDN using this guide then it will deliver static assets as well along with uploads and avatars (if I know correctly).

Then why would Discourse break if we deliver all the static assets through Cloudflare?

1 Like

I can’t recall, it’s been a few years. Why not try it and report back about the results?

1 Like

While this thread is active again I should follow up … it seems that Amazon CloudFront significantly lowered their pricing. I have no idea what they did but my average AWS costs went from around $80 a month to $15 a month. This is for both S3 and CloudFront for images (only) on a site with a whole lot of images.

2 Likes

Cloudflare is fine - just disable their Rocket Loader feature.

With no configuration effort, it saves half the bandwidth back to my server.

It might be that years ago Cloudflare didn’t work well with Discourse, but it’s not true today. It would be nice if we didn’t keep seeing doubt about it. If anyone has a specific problem, they should start a support thread here describing what’s not working.

5 Likes