We want to use a CDN with Discourse using Cloudfront and S3. The S3 bucket is not public and can only be read by and written to using the Discourse IAM role or Cloudfront. However, if we migrate the existing uploads or try to upload new files we get Access Denied. This is because the bucket does not allow public-read files. I believe that the following piece of code prevents us from using the bucket.
IAM roles and CloudFront are what we call “Advanced Clouds and Discourses”. The majority of installers aren’t going to be able to figure it out. It’s a reasonable setup, but out-of-scope for the intended purpose of that rake task. So I’d call this a feature request, not a bug.
If the bucket is supposed to be public, then changing this will copy over a bunch of private objects and the posts will show broken links.
Hi @schleifer, I thought about it after finishing the post and indeed it would be a nice feature if this was configurable. The bucket we have is only accessible by Cloudfront and the CDN that I wanted to use is the Cloudfront endpoint. Thus the links should work fine if we use the Cloudfront endpoint.
Would it be preferable to have this as another option to configure with public-read as default? I wouldn’t mind spending some time to whip up a pull request.
Yeah, an argument to specify the acl – so you could call rake uploads:migrate_to_s3[private] – would work nicely and could keep the default as public-read.
I am against a site setting here, an extensibility point so you can create a plugin to inject this is OK. But I do not want to complicate our already extremely complicated S3 site settings with yet another very extremely specific mega enterprisey kind of setting that even I am struggling really hard to get my head around.
I haven’t had a look at developing a Discourse plugin yet, but I can work on a plugin that uses Signed Cookies to ensure that content can only be viewed by users of the forum. I’ll see what I can do.
A fairly standard (and recommended by AWS) configuration is to never have a public s3 bucket/objects, and if you need to serve objects publically, do it through a Cloudfront distribution.
The current s3 implementation forces public s3 bucket access (through this issue, as well as S3 CDN URL ignored when uploading into posts), which is recommended against - for security and costs.
I am grateful that this thread remains open, as historical context over time is important. (Often I find that relevant older posts are locked by moderators )
Can Discourse support private S3 buckets? Feature request please
)