S3 and Cloudfront - Setup Not Working

Product Support Self-hosting

AI-generated summary

User akhil is troubleshooting an S3 and CloudFront setup for Discourse, following the official guide but encountering issues where uploads fail to serve from S3. The problem stems from misconfigured environment variables — supermathie clarifies that S3 settings must be set via environment variables (not site settings) when running rake tasks like uploads:migrate_to_s3. akhil attempts to add variables to app.yml, but still faces permission errors. supermathie identifies the likely issue: anonymous users or CDN can’t read from the bucket. After reviewing IAM policies, buckets, and CloudFront distributions, akhil shares detailed setup steps. A breakthrough comes from pfaffman, who suggests using www in the host address — akhil confirms switching from non-www to www resolves the issue, possibly due to SSL or Cloudflare-related conflicts. The setup now works with both S3 and CloudFront serving assets correctly. pfaffman expresses surprise at the fix but is glad it’s resolved. akhil speculates the prior outage may have been tied to Cloudflare’s service disruption.

1

Hello all!

I followed this guide to setup s3.

The only option I couldn’t find is " make sure the “Programmatic access ” checkbox is checked"

I couldn’t find any option like that.

But S3 is not working.

image
image2305×1013 205 KB

image
image1663×1620 283 KB

image
image1589×1160 120 KB

Notes: CDN is via CloudFront (s3 is the source of CloudFront)

Thanks

2

The rake task you’re running insists the specified values are in environment variables, it will not use the ones in the site settings.

3

Thanks for the response.

I used rake, thinking that image urls would be replaced with CDN link.

But the S3 setting itself is not working.

Did I miss anything in the S3 configuration?

4

Yes, you need to set the environment variables to run the rake task to migrate to S3.

5

I’m not getting. Hope you reviewed my discourse settings.

Any extra steps needed apart from this guide? Set up file and image uploads to S3

6

I believe you can get an overview of what environment variables you need here:
Configure an S3 compatible object storage provider for uploads

EDIT: In the context of a rake command, see the answer below :point_down:

7

or from the error:

image
image986×143 23.7 KB

They must be set in the environment, not in site settings.

You can provide them at the time the rake task is run by doing e.g.:

DISCOURSE_S3_BUCKET=foo DISCOURSE_S3_REGION=us-east-1 DISCOURSE_S3_USE_IAM_PROFILE=true rake uploads:migrate_to_s3
8

oh okay!

So, I’ll add this to app.yml

Example:

DISCOURSE_USE_S3: true
  DISCOURSE_S3_REGION: us-west-1
  DISCOURSE_S3_ACCESS_KEY_ID: myaccesskey
  DISCOURSE_S3_SECRET_ACCESS_KEY: mysecretkey
  DISCOURSE_S3_CDN_URL: https://falcoland-files-cdn.falco.dev
  DISCOURSE_S3_BUCKET: falcoland-files
  DISCOURSE_S3_BACKUP_BUCKET: falcoland-files/backups
  DISCOURSE_BACKUP_LOCATION: s3
9

Please read Michael’s answer instead!

10

Since my installation is fresh!

My only concern is new images. Rake, I did because the article suggested.

I thought just adding S3 settings via discource admin was enough to serve new images from S3!

11

See Configure an S3 compatible object storage provider for uploads

12

Yeah, since my setup is S3 and Cloudfront, I added the S3 code as following.

image
image1457×414 46.1 KB

Now rebuilding. I hope S3 will work!

13

Didn’t work! I think something is missing related to S3 permissions!

image
image2797×1560 279 KB

image
image2338×1004 244 KB

image
image1588×1767 167 KB

14

Any suggestions?

15

There’s 3 parts here:

  • can Discourse read/write/etc. on the bucket?
  • can anonymous users read from the bucket?
  • can the CDN read from the bucket?

Looks like it’s the second part not working?

17

Please review my setup and kindly let me know if I missed anything: S3 and CloudFront CDN

Based on this Guide

Created two buckets:

One for CDN (Enabled ACL also)
One for Backups
Created policy by adding two buckets, attached with IAM user, and generated the access keys.

Created two CloudFront distributions:

  1. cdn.mysite.com (origin is the cdn bucket) by configuring CNAME.

  2. assets.mysite.com (origin is mysite.com) (Community is on the main domain, not sub-domain) by configuring CNAME.

Based on this guide:

Added the AWS S3 code, and configuration to app.yml as follows:

image
image1650×648 46.1 KB

image
image1511×678 26.1 KB

Thank You.

19

Thanks a lot to @pfaffman

Who gave me a clue about using www in the host address. My forum was using a non-www version!

I changed to www version, and rebuilt it!

Everything now works fine including cdn and assets!

Thank You.

20

I can’t quite see how that fixed it, but glad you got it sorted!

21

As I explained here, after I did this, the site was offline. So, I was unable to see the outcome of that. But when I tried the same setup on a test sub-domain, all was working fine. Therefore, I suspect there is something wrong with the SSL.

Based on your clue of www, I changed the host address, and the issue was fixed immediately.

I also suspect that the site may have gone offline because of Cloudflare’s recent service outage.

Anyway, it got fixed.

Thank you!