S3 CDN URL ignored when uploading into posts

Bug
1

All of our traffic goes through CloudFront, and static content is fetched by CloudFront from one of several S3 buckets.

With uploads being sent to S3, Discourse embeds the content with a generic bucket URL despite our having s3 cdn url set. Once submitted, posts appear to use the correct URL set in s3 cdn url, although this may also be because we have DISCOURSE_CDN_URL set as well (both are set to the same URL).

The post preview window references the standard S3 URL seen in the unbaked post, though. This is a problem because we use a private ACL on files uploaded to our Discourse uploads bucket, as content on this bucket should never be accessed outside of CloudFront. The result is a broken image in the preview window.

Pasted image1053Ă—119 12 KB

2 Likes
2

Disappointing this has been reported for so many years with no fix.

@DanielMarquard Did you ever figure out a workaround, or were you forced to make your s3 ACL public?

3

I’m not sure it’s fair to suggest this is broken, as much as the more complicated implementation isn’t fully supported.

S3 uploads as documented work and behave correctly.

4

@Stephen Certainly up for interpretation…

To me it seems “broken” that it uses the CDN url in one instance and not the other, and it forces an S3 configuration that AWS actually recommends against (public ACL for an s3 bucket).

I’m new to the Discourse project in general, but can certainly work on a PR.

5

This may be worth fixing now - AWS is getting more serious about informing people of public read ACLs.

4 Likes
6

Given our current implementation no longer has an IMG tag eg:

image
image.png908Ă—393 55 KB

![image|690x298,50%](upload://eGAOR4Xy4aRM2lwPIuTokBf60jS.png)

It should be quite easy to apply a CDN to the above and it saves traffic.

@vinothkannans can you fix this up so preview uses the CDN url

Instead of preview showing:

<img src="//assets-meta-cdck-prod-meta.s3.dualstack.us-west-1.amazonaws.com/original/3X/6/6/66ef078b3ef9da9aa1b3356cb21fe22a0d25eaf0.png" alt="image" class="resizable" width="345" height="149">

We should show the CDN URL which is this:

<img src="https://d11a6trkgmumsb.cloudfront.net/original/3X/6/6/66ef078b3ef9da9aa1b3356cb21fe22a0d25eaf0.png" alt="image" class="resizable" width="345" height="149">
6 Likes
8

Now it’s fixed as per the below commit

https://github.com/discourse/discourse/commit/e4fe864c0b7d4d5d1f0cd604829eca81b71bb4ec

6 Likes
9

This topic was automatically closed after 32 hours. New replies are no longer allowed.

11

Continuing the discussion from S3 CDN URL ignored when uploading into posts:

It seems like this issue has regressed, I can repro here on meta.

image
image1222Ă—505 78.4 KB

3 Likes
12

Can we check this @vinothkannans or @falco?

3 Likes
14

As per the recent change, we’re now getting the uploaded file’s URL from the initial upload request itself. Previously, we received it from the /uploads/lookup-urls.json endpoint. This commit should fix the issue.

https://github.com/discourse/discourse/commit/f11363d446c4c6a01a59d192f5f078c813ed1919

5 Likes
15

Looks like this fix doesn’t cover custom emoji in the preview, for example :allthethings:.

4 Likes
16

This is also missing attachments like data exports or just any file attachment. Those still come from s3.

18

I can fix that!

https://github.com/discourse/discourse/pull/11401

3 Likes