How to load gzipped assets from cdn?


(Dmitry Skavish) #1

hey there,

we are setting up our forum at forum.animatron.com and found some problems we could not find any answers to. if you guys can answer it that would be awesome, thanks!

  • we enabled cdn for our forum following this article: Enable a CDN for your Discourse
    the problem is that it loads static assets (.js etc) in uncompressed state which results in huge downloads (around 6mb).
    the funny things is that gzipped assets are already there, if you add .gz to the end of any of those static assets (i.e. blablabla.js.gz) it will be loaded. the question is how to force discourse to request those gzipped assets, not uncompressed ones?

  • we installed our SSO using this plugin plugin.rb · GitHub
    the problem is that it created some nasty looking usernames which we cannot change. attempt to change any of those usernames either shows an error (error changing username) or just does nothing.

any insights into how to fix it would be much appreciated, thanks!


Cloudfront Compress Objects Automatically
(Sam Saffron) #2

What CDN are you using, in general CDNs are well behaved and respect Accept-Encoding: compress, gzip … but I dunno maybe you have a rogue CDN at play.

Are you sure its not compressed, what makes you say it is not?


(Dmitry Skavish) #3

we are using cloudfront.

here is screenshot of chrome network tab, you can see those files http://glui.me/v1/?i=kw2vacdp70ct0jl/2015-06-09_at_6.41_PM_2x.png/

let’s take this one for example: https://forum-cdn.animatron.com/assets/vendor-4cda468a208905f3a293a1e9b1c1791c.js
copying the request as curl and executing it shows that the resource is clearly uncompressed:

curl 'https://forum-cdn.animatron.com/assets/vendor-4cda468a208905f3a293a1e9b1c1791c.js' -H 'Accept-Encoding: gzip, deflate, sdch' -H 'Accept-Language: en-US,en;q=0.8,ru;q=0.6' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.81 Safari/537.36' -H 'Accept: */*' -H 'Referer: https://forum.animatron.com/' -H 'Connection: keep-alive'

also it’s clear from those links
gtmetrix.com/reports/forum.animatron.com/TSssE1GE
developers.google.com/speed/pagespeed/insights/?url=https%3A%2F%2Fforum.animatron.com%2F&tab=desktop


(Dmitry Skavish) #4

I just realized that I need to pass through Accept-Encoding header to the origin. I am going to try that.


(Jeff Atwood) #5

What version of Discourse are you on? It should be 1.4 beta or 1.3.1.

I also followed that document to set up CDN on my site at http://discourse.codinghorror.com (1.4 beta 1) and here is what you will see in view source on the homepage there:

<script src="//discourse-cdn.codinghorror.com/assets/preload_store-762f2e128a64e19a8cf3d8eda0f6f587.js"></script>
<script src="//discourse-cdn.codinghorror.com/assets/locales/en-08254d43550caf6369afde3a00b4dbb4.js"></script>
<script src="//discourse-cdn.codinghorror.com/assets/vendor-4cda468a208905f3a293a1e9b1c1791c.js"></script>
<script src="//discourse-cdn.codinghorror.com/assets/application-9a229ea5a220db15c341cfdeef33f00e.js"></script>
<script src="//discourse-cdn.codinghorror.com/assets/admin-f3c40c88c39abdf90c0d44ea1a647392.js"></script>

Checking each file with HTTP Compression Test / WhatsMyIP.org I see:

discourse-cdn.codinghorror.com/assets/preload_store-762f2e128a64e19a8cf3d8eda0f6f587.js is Compressed
Uncompressed Page Size: 2.4 KB
Compressed Page Size: 0.9 KB
Savings: 63.2%

Uncompressed Page Size: 0.5 KB
Compressed Page Size: 0.3 KB
Savings: 43.6%

and

discourse-cdn.codinghorror.com/assets/locales/en-08254d43550caf6369afde3a00b4dbb4.js is Compressed
Uncompressed Page Size: 207.7 KB
Compressed Page Size: 52.9 KB
Savings: 74.5%

Uncompressed Page Size: 126.6 KB
Compressed Page Size: 39.8 KB
Savings: 68.5%

and

discourse-cdn.codinghorror.com/assets/vendor-4cda468a208905f3a293a1e9b1c1791c.js is Compressed
Uncompressed Page Size: 2531.2 KB
Compressed Page Size: 572.3 KB
Savings: 77.4%

Uncompressed Page Size: 836.4 KB
Compressed Page Size: 245.6 KB
Savings: 70.6%

and

discourse-cdn.codinghorror.com/assets/application-9a229ea5a220db15c341cfdeef33f00e.js is Compressed
Uncompressed Page Size: 3181.4 KB
Compressed Page Size: 407.8 KB
Savings: 87.2%

Uncompressed Page Size: 1604.2 KB
Compressed Page Size: 257.2 KB
Savings: 84%

and

discourse-cdn.codinghorror.com/assets/admin-f3c40c88c39abdf90c0d44ea1a647392.js is Compressed
Uncompressed Page Size: 1560.2 KB
Compressed Page Size: 133.2 KB
Savings: 91.5%

Uncompressed Page Size: 807 KB
Compressed Page Size: 87.2 KB
Savings: 89.2%

So this is not a problem with Discourse, I suggest this is a problem with your installation or your CDN…

edit: we had a pretty massive bug as you can see, minification was basically not happening. If you compare the numbers below with the numbers above … thanks @sam for noticing this and fixing it.


(Sam Saffron) #6

Or a problem with the test methodology :slight_smile:


(Dmitry Skavish) #7

ok, we finally fixed it. the problem is combination of nginx and cloudfront, here is some explanation: Cloudfront, nginx and gzip: not that simple - CDN Planet

on our case the following nginx directives helped:

gzip  on;
gzip_http_version 1.0;
gzip_vary on;
gzip_proxied any;
gzip_comp_level 9;
gzip_types
    application/atom+xml
    application/javascript
    application/json
    application/rss+xml
    application/vnd.ms-fontobject
    application/x-font-ttf
    application/x-web-app-manifest+json
    application/xhtml+xml
    application/xml
    font/opentype
    image/svg+xml
    image/x-icon
    text/css
    text/plain
    text/x-component;
gzip_disable "msie6";

(Jeff Atwood) #8

I am guessing you are not using the recommended Docker install?

I can’t see how this would be a problem on our standard Docker install…


(Dmitry Skavish) #9

we are using docker image, but I am not sure about nginx, my admin installed it.


(Jeff Atwood) #10

OK is there some weird proxy in front? Unless you can repro any of this “not compressed” stuff happening on a 100% standard docker install such as the one at http://discourse.codinghorror.com


(Dmitry Skavish) #11

I am not blaming discourse, I am just sharing our findings.
discourse works as it should, thanks for that!


(Jeff Atwood) #12

Of course, I am just not clear how this can happen if someone uses our standard Docker install… that’s the part I don’t get.


(Kane York) #13

A nginx in front of Docker, without proper compression settings set up, would be doing this:

Discourse --(compressed)--> nginx --(uncompressed)--> CloudFront --(uncompressed)--> User

(徐徐浩) #14

Hello,
I have the same problem with you.
I am very happy that you have found a solution and have shared it with us.
I am totally new to discourse and ngnix, I do not know how to fix it even with your answer.
So could you please explain it more in detail?
Thanks very much.