Users are discussing the inability to migrate or upload files to a private S3 bucket in Discourse when using CloudFront, due to the hardcoded acl: "public-read" setting in the S3 store code ash-kadem. While removing this setting allows migration to work, it’s not a safe default since it could break links if the bucket is meant to be public schleifer. A feature request is proposed to make the ACL configurable — either via rake task argument schleifer or a site setting ash-kadem. However, sam opposes adding another complex setting and suggests using a plugin instead. ash-kadem agrees to explore a plugin using Signed Cookies for secure access. Others timkelty, mahcr, and catchthewavecoke echo the need for private S3 support, citing AWS best practices against public buckets. mpalmer confirms Discourse can support private buckets — implying it’s a matter of configuration or extension. The consensus leans toward a plugin-based solution rather than core changes to avoid overcomplicating S3 settings.
We want to use a CDN with Discourse using Cloudfront and S3. The S3 bucket is not public and can only be read by and written to using the Discourse IAM role or Cloudfront. However, if we migrate the existing uploads or try to upload new files we get Access Denied. This is because the bucket does not allow public-read files. I believe that the following piece of code prevents us from using the bucket.
If I remove the piece of code acl: "public-read", then the task rake uploads:migrate_to_s3 works fine.
IAM roles and CloudFront are what we call “Advanced Clouds and Discourses”. The majority of installers aren’t going to be able to figure it out. It’s a reasonable setup, but out-of-scope for the intended purpose of that rake task. So I’d call this a feature request, not a bug.
If the bucket is supposed to be public, then changing this will copy over a bunch of private objects and the posts will show broken links.
Hi @schleifer, I thought about it after finishing the post and indeed it would be a nice feature if this was configurable. The bucket we have is only accessible by Cloudfront and the CDN that I wanted to use is the Cloudfront endpoint. Thus the links should work fine if we use the Cloudfront endpoint.
Would it be preferable to have this as another option to configure with public-read as default? I wouldn’t mind spending some time to whip up a pull request.
Yeah, an argument to specify the acl – so you could call rake uploads:migrate_to_s3[private] – would work nicely and could keep the default as public-read.
I am against a site setting here, an extensibility point so you can create a plugin to inject this is OK. But I do not want to complicate our already extremely complicated S3 site settings with yet another very extremely specific mega enterprisey kind of setting that even I am struggling really hard to get my head around.
I haven’t had a look at developing a Discourse plugin yet, but I can work on a plugin that uses Signed Cookies to ensure that content can only be viewed by users of the forum. I’ll see what I can do.
A fairly standard (and recommended by AWS) configuration is to never have a public s3 bucket/objects, and if you need to serve objects publically, do it through a Cloudfront distribution.
The current s3 implementation forces public s3 bucket access (through this issue, as well as S3 CDN URL ignored when uploading into posts), which is recommended against - for security and costs.
I am grateful that this thread remains open, as historical context over time is important. (Often I find that relevant older posts are locked by moderators )
Can Discourse support private S3 buckets? Feature request please
