Migrating uploads to s3 does not work with private bucket


(Ash Belmokadem) #1

Hi all,

We want to use a CDN with Discourse using Cloudfront and S3. The S3 bucket is not public and can only be read by and written to using the Discourse IAM role or Cloudfront. However, if we migrate the existing uploads or try to upload new files we get Access Denied. This is because the bucket does not allow public-read files. I believe that the following piece of code prevents us from using the bucket.

If I remove the piece of code acl: "public-read", then the task rake uploads:migrate_to_s3 works fine. :slight_smile:


(Andrew Schleifer) #2

IAM roles and CloudFront are what we call “Advanced Clouds and Discourses”. The majority of installers aren’t going to be able to figure it out. It’s a reasonable setup, but out-of-scope for the intended purpose of that rake task. So I’d call this a feature request, not a bug.

If the bucket is supposed to be public, then changing this will copy over a bunch of private objects and the posts will show broken links.


(Ash Belmokadem) #3

Hi @schleifer, I thought about it after finishing the post and indeed it would be a nice feature if this was configurable. The bucket we have is only accessible by Cloudfront and the CDN that I wanted to use is the Cloudfront endpoint. Thus the links should work fine if we use the Cloudfront endpoint.

Would it be preferable to have this as another option to configure with public-read as default? I wouldn’t mind spending some time to whip up a pull request. :slight_smile:


(Andrew Schleifer) #4

Yeah, an argument to specify the acl – so you could call rake uploads:migrate_to_s3[private] – would work nicely and could keep the default as public-read.


(Ash Belmokadem) #5

@schleifer this should also work when users upload files, so I decided to implement it as a site setting :slight_smile:

edit: by “should”, I mean I tested both scenarios + rake task on my s3 bucket and it works


(Sam Saffron) #6

I am against a site setting here, an extensibility point so you can create a plugin to inject this is OK. But I do not want to complicate our already extremely complicated S3 site settings with yet another very extremely specific mega enterprisey kind of setting that even I am struggling really hard to get my head around.


(Ash Belmokadem) #7

I haven’t had a look at developing a Discourse plugin yet, but I can work on a plugin that uses Signed Cookies to ensure that content can only be viewed by users of the forum. I’ll see what I can do.