Discourse Meta is now running S3 for uploads

In the following post I described the weaknesses in our S3 implementation:

https://meta.discourse.org/t/amazon-s3-is-not-deprecated-in-1-3-for-image-attachment-storage/26594/108?u=sam

Since then @zogstrip and to a minor extent me, have been working on fixing the s3 issues.

In particular:

  1. We now upload to s3 in the background (without blocking a web worker)
  2. We stopped using the fog gem.
  3. We moved to an s3 “friendly” upload scheme for avatars
  4. We support an additional CDN in front of S3 content
  5. We changed the URL scheme on s3 to match the local scheme (which makes it way easier to move to a from s3)

I went ahead and enabled this here, we will be refining this and even migrating all the existing images and avatars on meta to s3

13 Likes

fastly complains…

avatars are broken atm due to some cdn changes, working on a fix

2 Likes

Fixed, if anyone is having issues, please clear browser cache.

HUGE fan of decoupling uploads from app server. Thanks!

This is fantastic news, thank you @sam . I can now upgrade our site, I’ve been delaying it until I had time to migrate off of S3!

Is there anything we need to run to move current avatars/uploads that are already on S3 to the new scheme?

J