Full site CDN acceleration for Discourse

I have noticed there is no “cloudfront.template.yml” in discourse_docker/templates/. So I am wondering:
Can CloudFront work using the same techniques ?

Also, can we use http2 ? Is the long polling stuff still needed when using http2 ?

If you’re using a the supported Docker-based install, HTTP2 should be working automatically! :sunny:

Long polling is still needed for notifications to appear live.

3 Likes

I think if you have cloudfront setup, it’s only delivering specific objects (images), rather than the site/application in it’s entirety with js and so on.

So the only thing you need is to have the correct cloudfront url for those images.

Some additional notes for anyone who decides to use HTTPS together with Full site CDN acceleration

  1. Discourse internally uses the value of SiteSetting.force_https to decide if your access-control-allow-origin: is the HTTP or HTTPS version of your site. If while polling you see an error in the browser console along the lines of preflight request doesn't pass access control check: The 'Access-Control-Allow-Origin' header has a value http doesn't match https, double check your force_https setting. Also note the protocol in your DISCOURSE_CORS_ORIGIN in your container definition (http|https) will be overridden by force_https.
    Don’t forget to add DISCORSE_ENABLE_CORS: true in your container definition.

  2. If you were planning to only do HTTPS from your end users to your CDN, and then HTTP from your CDN to your actual Discourse web_only containers, lots of custom configuration will be required.

  3. If your CDN is serving your site on HTTPS, then whatever Long Polling URL you setup must also be on HTTPS, so even if the CDN is handling your HTTPS, you must still setup HTTPS on your Discourse servers. If you run into an error about Same-origin policy, double check that you’re not trying to connect to HTTP instead of HTTPS

    • If you use letsencrypt to generate your certificates, note that fullchain.pem => /shared/ssl/ssl.crt (ssl_certificate)
    • privkey.pem => /shared/ssl/ssl.key (ssl_certificate_key)
  4. You might use the following templates in your container definition:

  - "templates/web.template.yml"
  - "templates/web.ssl.template.yml"
  - "templates/fastly.template.yml"
  • Towards the end of the hook:ssl inside templates/web.ssl.template.yml you’ll see this block being added to your /etc/nginx/conf.d/discourse.conf.
if ($http_host != $$ENV_DISCOURSE_HOSTNAME) {
    rewrite (.*) https://$$ENV_DISCOURSE_HOSTNAME$1 permanent;
}
  • You’ll need to need to comment these lines out, otherwise you’re long polling attempts always serve up 301 redirects back to your origin, instead of respecting whatever you set in SiteSetting.long_polling_base_url
  1. The easiest way I’ve found to do this is to copy templates/web.ssl.template.yml to local.web.ssl.template.yml and just remove those extra lines, and update your container reference to use your local template. If you go that route, you should periodically diff your local version with the origional version, because there are some security improvements that are regularly incorporated into this template.

Some of the error messages you’ll run into until things are configured correctly.

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://polling.example.com/message-bus/37c91c51e6cd4b0c95288b8fc29a0480/poll. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).

Reason: CORS header ‘Access-Control-Allow-Origin’ missing

Response to preflight request doesn’t pass access control check: No ‘Access-Control-Allow-Origin’ header is present on the requested resource

Failed to load https://polling.example.com/message-bus/8caefcec2cf94de3ae684c4b953a1084/poll: Response for preflight is invalid (redirect)

4 Likes

You can do this with a - replace in your app.yml similar to how it is described on Setting up Let’s Encrypt with Multiple Domains.

  after_ssl:
    - replace:
        filename: "/etc/nginx/conf.d/discourse.conf"
        from: /return 301 https.+/
        to: |
          return 301 https://$host$request_uri;

    - replace:
        filename: "/etc/nginx/conf.d/discourse.conf"
        from: /gzip on;[^\}]+\}/m
        to: |
          gzip on;
          add_header Strict-Transport-Security 'max-age=31536000'; # remember the certificate for a year and automatically connect to HTTPS for this domain
1 Like

@brahn, I was looking at using a pups replace line, but I couldn’t figure out how to do a multi-line match in pups…

Note that templates/web.ssl.template.yml is inside of the port 443 block, not the port 80 block.

Is there a way to use the pups, replace command to match the entire mult-line string?

if ($http_host != www.example.com) {
   rewrite (.*) https://www.example.com$1 permanent;
}

The most direct way I thought of inside the container definition is an exec line running perl, awk or sed to do the multiline replace… but then you’ve got shell escaping along with your target language to disentangle before it will work…

The first replace takes care of the redirect from http to https for multisite. Perhaps that one is not relevant for you.

The second replace is multi-line. It replaces everything from line 33 to line 39 that was added by the web.ssl template.

It just removes that whole rewrite block. I could not figure out what purpose it serves and it breaks mutlisite so…

You could do this in your app.yml:

after_ssl:
    - replace:
        filename: "/etc/nginx/conf.d/discourse.conf"
        from: /gzip on;[^\}]+\}/m
        to: |
          gzip on;
          add_header Strict-Transport-Security 'max-age=31536000'; # remember the certificate for a year and automatically connect to HTTPS for this domain
          if ($http_host != www.example.com) {
            rewrite (.*) https://www.example.com$1 permanent;
          }
1 Like

@sam

I’m curious to know one thing:
Eg. My discourse is hosted on forum.example.com
Can I set long polling base to poll.example.org which points to the same server IP?
Will it have any impact considering CSP?

2 Likes

Hi there. Why you guys recommends keep away from Cloudflare when we are in Discourse?

I’m trying to install it in my new VPS (Debian in Hetzner) and I thought that could be useful to keep Cloudflare on in my little server.

Thanks for your time.

CloudFlare isn’t a conventional CDN, it’s a network proxy. Some of their performance features alter the code between client and server.

Leaving those features on can break Discourse in new and interesting ways. If you turn them off, you’re just adding extra network hops between the Discourse app in your browser, and the server. More hops = a less responsive interface.

6 Likes

Well, I’m in Hetzner VPS and I read that could be good to use Cloudflare to keep my server safe (in eventual attacks). CDN could be OK too, because I’m in another country (America, not Germany).

What do you think about it?

I’m not going to comment on whether you should be worried about attacks. You need to make that assessment yourself, but don’t fall prey to FUD.

If you leave their performance features enabled we cannot support you here. As mentioned above they interfere with the javascript in ways that do no good.

You may be able to make the basic asset caching work with all of the other performance features turned off.

Even then, if Cloudflare is active during installation and setup, certificate enrolment will fail. Let’s Encrypt isn’t supported behind a Cloudflare proxy for initial enrolment.

3 Likes

Thanks for your answer Stephen. I’m facing issues trying to install Discourse and I thought that could be related to Cloudflare.

So, I can’t use it neither to manage DNS? How can I protect my server and keep Discourse without Cloudflare?

Press the orange cloud next to your hostname in the Cloudfare control panel so the cloud turns grey.
Then install Discourse. If you want to protect your server, press the grey cloud so it turns orange, but make sure to disable all performance features first.

6 Likes

I have a question concerning CDN technology.

Am I right that cdn can only cache the files in my site and can’t cache hotlinked files from another site?

I store catalogue files (mps and pictures) on another server and hotlink them on my Discourse site. Am I right that such files won’t be cached by cdn? Is there a way to cache the hotlinked files too?

Unless you tell discourse not to, it will pull those remote images to its own image store. The below assumes you turned that off.

You should put that site behind CDN and then share the CDN url to discourse.

If you put a CDN in front of your image site but share the origin url to discourse, You can use a built in feature to replace the origin url with the CDN url .

2 Likes

Am I right, that if I activate the CDN feature in Discourse and switch off saving the origin to Discourse, the cdn service will cache these external files and Discourse will replace all the links to files hotlinked from another site?

I doubt that can caches external files hotlinked from other sites. Can anybody approve it?

FYI - I’ve posted a new thread on how to setup full site CDN acceleration (and SSL termination) using AWS Cloudfront here (link below). There be dragons here - so tread lightly.

1 Like

Is that still required when the error Reason: CORS header ‘Access-Control-Allow-Origin’ missing happens?

I can’t find anything that uses that variable in the Github:


Edit: the settings comment still say it’s needed, and it does work after that. So I’ll just accept :slight_smile: