Discourse Meta is now running S3 for uploads


(Sam Saffron) #1

In the following post I described the weaknesses in our S3 implementation:

https://meta.discourse.org/t/amazon-s3-is-not-deprecated-in-1-3-for-image-attachment-storage/26594/108?u=sam&source_topic_id=29241

Since then @zogstrip and to a minor extent me, have been working on fixing the s3 issues.

In particular:

  1. We now upload to s3 in the background (without blocking a web worker)
  2. We stopped using the fog gem.
  3. We moved to an s3 “friendly” upload scheme for avatars
  4. We support an additional CDN in front of S3 content
  5. We changed the URL scheme on s3 to match the local scheme (which makes it way easier to move to a from s3)

I went ahead and enabled this here, we will be refining this and even migrating all the existing images and avatars on meta to s3


(Erick Guan) #2

fastly complains…


(Sam Saffron) #3

avatars are broken atm due to some cdn changes, working on a fix


(Sam Saffron) #5

Fixed, if anyone is having issues, please clear browser cache.


(Jeff Widman) #6

HUGE fan of decoupling uploads from app server. Thanks!


(James Hayward) #7

This is fantastic news, thank you @sam . I can now upgrade our site, I’ve been delaying it until I had time to migrate off of S3!

Is there anything we need to run to move current avatars/uploads that are already on S3 to the new scheme?

J