Assets not serving from CDN in latest update

Hello!

I couldn’t figure out why in the latest update our installation stopped serving assets from the CDN. We are using Cloudfront and it was working good to serve both uploads and assets, but it looks like in the latest update assets are serving from the server instead.

Another issue we are having is if we pass the variable DISCOURSE_CDN_URL, Discourse starts requesting assets that are not being generated or uploaded by the s3:upload_assets task (stylesheets for example). This breaks completely the site with a white page. This post mentions something similar.

Any help would be greatly appreciated.

Thanks!

1 Like

Is this a new bug @falco? Did we break something here?

2 Likes

@eatcodetravel we need a lot more detail in order to help you in any way.

What are the exact values of the settings and environment variables related to S3 and CDN?

1 Like

@Falco sure what information do you need? Our setup is open source, and here are the settings we are running.

I removed the DISCOURSE_CDN_URL which is was set to the same value as DISCOURSE_S3_CDN_URL because that broke the site (it was trying to pull assets that didn’t exist on S3).

Looking back at the site that I was having trouble with, it appears that I’d left out the https:// on the CDN URL I was using. I’m not clear how it was that it had ever worked, or, if it hadn’t ever worked, how I hadn’t noticed that when I added the broken value.

@eatcodetravel, you of course need to redact SMTP passwords and S3 keys, but I think you’ll need to share at least the CDN values for anyone to be able to make any guesses what the problem might be.

Sure let me write these here,I think just the CDN_URL since the others are validated via the aws sdk

DISCOURSE_S3_CDN_URL: 'https://community-cdn-prod.debtcollective.org'
DISCOURSE_S3_REGION: 'us-west-1'

DISCOURSE_CDN_URL was set to ‘https://community-cdn-prod.debtcollective.org’ too before I removed it

Nothing mayor changed in our setup, just migrated to a bigger ec2 machine.

Also, let me know if you need more info @Falco, happy to answer your questions

So you are currently using a S3 CDN but not the normal CDN. I’m not sure that is a supported setup…

Other 3 combinations work fine for sure:

  1. No CDN
  2. Just DISCOURSE_CDN_URL
  3. Both DISCOURSE_CDN_URL and DISCOURSE_S3_CDN_URL.
2 Likes

I think that might explain some other CDN issues that I’ve had in the past.

1 Like

Yeah, I had to remove DISCOURSE_CDN_URL, it started asking for assets that are not being uploaded to S3 by the rake s3:upload_assets task and this breaks the site completely. Maybe this was the part that changed and we need to update rake s3:upload_assets to work again correctly

Yeah, using the same CDN URL for both DISCOURSE_CDN_URL and DISCOURSE_S3_CDN_URL can be done, but it requires a lot of login in the CDN to use the correct source for each asset based on the URL. For example, CSS comes from the app, JS from S3 and this is just one case, we have dozens.

So the expectation is that you set both, where the DISCOURSE_CDN_URL is a “proxy” to your web app, and DISCOURSE_S3_CDN_URL is a proxy to your object storage bucket.

2 Likes

Oh ok, is this something that changed recently? I’m not sure if Cloudfront supports being a reserve proxy to our app instead of an S3 bucket.

I see you are using Cloudfront in meta, is there anything special you do to run it or Discourse itself takes care of everything?

If you inspect this page you will see that we we have two different Cloudfront URLs, so we use what I said above, two distributions, one for the normal CDN and another for S3.

The problem from the title of the OP is this: CSS come from the app and JS from S3. You need a CDN that is aware of where every single asset is coming from and can reach the necessary place or two CDNs. That is why we have two different settings after all.

1 Like

Though this totally makes sense now that I understand it, I’m not sure that this is documented. And you’re likely to get in trouble (at least I did!), as the move-uploads-to-s3 rake task requires that you have an s3_cdn. This explains why several times I’ve tried to move stuff to S3 and ended up giving up.

I think it’d be nice if there were some “Dude, you can’t have an S3 CDN without an asset CDN warning somewhere.”

Ok now I understand what you are saying now, didn’t know this was required. May I ask why CSS needs to come from the app? Why we can’t upload it via the rake s3:upload_assets task?

I’ll make some research on how to achieve this on Cloudfront and put my findings/blockers here.

Thanks @Falco, this was really helpful

Well, I may be explaining this badly. You can. But it is fiddly as your topics and this one on it shows.

The site from @eatcodetravel is running right now with just S3_CDN set, but then you get into a weird state where CSS (which blocks rendering) is served from your server while images in posts aren’t, which makes no sense.

Because Admin > Customize is a thing. People can change CSS on the admin panel. Custom JS snippets living in themes are also served from the app AFAIK.

3 Likes

Can we somehow document this better @falco? Maybe even in a “here be :dragon:s” way?

5 Likes

I ended up fixing this by providing a second Cloudfront distribution that points to the web server, and keep the other one just for S3. Thank you @Falco for the feedback, but I agree this needs to be documented a bit better.

7 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.