S3 CDN URL ignored when uploading into posts

[DanielMarquard]

All of our traffic goes through CloudFront, and static content is fetched by CloudFront from one of several S3 buckets.

With uploads being sent to S3, Discourse embeds the content with a generic bucket URL despite our having s3 cdn url set. Once submitted, posts appear to use the correct URL set in s3 cdn url, although this may also be because we have DISCOURSE_CDN_URL set as well (both are set to the same URL).

The post preview window references the standard S3 URL seen in the unbaked post, though. This is a problem because we use a private ACL on files uploaded to our Discourse uploads bucket, as content on this bucket should never be accessed outside of CloudFront. The result is a broken image in the preview window.

[timkelty]

Disappointing this has been reported for so many years with no fix.

@DanielMarquard Did you ever figure out a workaround, or were you forced to make your s3 ACL public?

[Stephen]

I’m not sure it’s fair to suggest this is broken, as much as the more complicated implementation isn’t fully supported.

S3 uploads as documented work and behave correctly.

[timkelty]

@Stephen Certainly up for interpretation…

To me it seems “broken” that it uses the CDN url in one instance and not the other, and it forces an S3 configuration that AWS actually recommends against (public ACL for an s3 bucket).

I’m new to the Discourse project in general, but can certainly work on a PR.

[riking (Kane York)]

This may be worth fixing now - AWS is getting more serious about informing people of public read ACLs.

[sam (Sam Saffron)]

Given our current implementation no longer has an IMG tag eg:

![image|690x298,50%](upload://eGAOR4Xy4aRM2lwPIuTokBf60jS.png)

It should be quite easy to apply a CDN to the above and it saves traffic.

@vinothkannans can you fix this up so preview uses the CDN url

Instead of preview showing:

<img src="//assets-meta-cdck-prod-meta.s3.dualstack.us-west-1.amazonaws.com/original/3X/6/6/66ef078b3ef9da9aa1b3356cb21fe22a0d25eaf0.png" alt="image" class="resizable" width="345" height="149">

We should show the CDN URL which is this:

<img src="https://d11a6trkgmumsb.cloudfront.net/original/3X/6/6/66ef078b3ef9da9aa1b3356cb21fe22a0d25eaf0.png" alt="image" class="resizable" width="345" height="149">

[vinothkannans (Vinoth Kannan)]

Now it’s fixed as per the below commit

https://github.com/discourse/discourse/commit/e4fe864c0b7d4d5d1f0cd604829eca81b71bb4ec

[RGJ (Richard - Communiteq)]

Continuing the discussion from S3 CDN URL ignored when uploading into posts:

It seems like this issue has regressed, I can repro here on meta.

[codinghorror (Jeff Atwood)]

Can we check this @vinothkannans or @falco?

[vinothkannans (Vinoth Kannan)]

As per the recent change, we’re now getting the uploaded file’s URL from the initial upload request itself. Previously, we received it from the /uploads/lookup-urls.json endpoint. This commit should fix the issue.

https://github.com/discourse/discourse/commit/f11363d446c4c6a01a59d192f5f078c813ed1919

[Falco]

Looks like this fix doesn’t cover custom emoji in the preview, for example .

[applebee1558 (Alex Liu)]

This is also missing attachments like data exports or just any file attachment. Those still come from s3.

[nbianca (Bianca)]

I can fix that!

https://github.com/discourse/discourse/pull/11401