تسريع شبكة توصيل المحتوى (CDN) للموقع بالكامل لـ Discourse

Discourse · 24 أكتوبر 2014، 6:17ص

Fastly وCloudFlare وعدد قليل من شبكات توصيل المحتوى (CDNs) الأخرى تقدم وضعًا تقوم فيه بتسريع المحتوى الديناميكي.

باختصار، توجه عنوان IP الخاص بالنطاق الخاص بك إلى شبكة توصيل المحتوى وستقرر الشبكة بذكاء كيفية التعامل مع الطلب.

يمكن تقديم المحتوى الثابت بسهولة من ذاكرة التخزين المؤقت
يمكن توجيه المحتوى الديناميكي إلى الموقع.

يوفر هذا بعض المزايا على مجرد شحن الأصول الثابتة التي تمت تغطيتها في دليل شبكة توصيل المحتوى.

يمكنك اختيار “الحماية shielding” التي تحمي موقعك من زيادات حركة المرور.
يمكن تسريع المحتوى الديناميكي باستخدام تقنيات مثل ضغط دلتا. (ملاحظة: بشكل عام، تتناسب حمولتنا مع 1 RTT لذا فإن هذا له تأثير أقل)
يمكن إجراء مفاوضات SSL عند الحافة مما يقلل من الرحلات ذهابًا وإيابًا المكلفة للمفاوضات.

إذا قمت بتمكين تسريع الموقع بالكامل باستخدام شبكة توصيل محتوى، فمن الضروري اتباع 3 قواعد

يجب تقديم “ناقل الرسائل” يجب أن يتم تقديمه من المصدر.
تحتاج إلى إعداد الثقة في X-Forwarded-For. بالنسبة لـ Cloudflare، أضف cloudflare.template.yml إلى ملف app.yml الخاص بك.
كن حذرًا للغاية بشأن التقنيات التي تطبق التحسين على الموقع، فالأشياء مثل Rocket Loader يمكن أن توقف عمل Discourse. تم تحسين Discourse بالفعل بشكل كبير، وهذا ليس ضروريًا.

لتقديم طلبات “الاستقصاء الطويل long polling” من نطاق مختلف، قم بتعيين إعداد الموقع المخفي long_polling_base_url إلى خادم المصدر. يتم تكوين هذا على أفضل وجه عن طريق إضافة متغير البيئة DISCOURSE_LONG_POLLING_BASE_URL في ملف app.yml الخاص بك، أو عبر وحدة تحكم Rails.

على سبيل المثال، إذا كان موقعك على “http://forum.example.com”، فيجب عليك إعداد http://forum-direct.example.com/ للربط بإعداد الموقع. إذا لم تفعل ذلك، فسيتعطل موقعك.

إذا كنت تضع Discourse خلف Varnish، فمن المحتمل أن ترغب في اتباع نفس الحيلة هنا وتجاوز Varnish لطلبات ناقل الرسائل.

ملاحظات فنية مملة:

إن تحقيق ناقل رسائل يعمل على نطاق مختلف تمامًا أمر صعب للغاية. ناقل الرسائل لدينا على دراية بالمستخدم الذي يقوم بالاستقصاء، وقد لا يحتوي النطاق الآخر على ملف تعريف ارتباط مُعد، لذا دون مساس هناك مشكلتان. أولاً، لا يمكنك حتى إجراء طلبات ajax قياسية عبر النطاقات دون رقصة CORS هائلة.

ثانيًا، احتجنا إلى آلية لإبلاغ النطاق الآخر بهوية المستخدم حتى نتمكن من الاستقصاء للحصول على المعلومات الصحيحة.

عندما يتم تغيير عنوان URL الأساسي للاستقصاء الطويل، تقوم Discourse بشحن علامة وصف (meta tag) إضافية تشارك رمز مصادقة “عبر النطاقات”. يتم تمرير هذا الرمز باستخدام رأس مخصص مرة أخرى إلى ناقل الرسائل. تنتهي صلاحية الرمز المميز بعد 7 أيام أو بمجرد تسجيل خروج المستخدم.

يمكنك رؤية معظم التنفيذ هنا: FEATURE: allow long polling to go to a different url · discourse/discourse@aa9b3bb · GitHub

fantasticfears · 30 أكتوبر 2014، 1:24ص

I don’t know what it means… fits in 1 RTT?

sam · 30 أكتوبر 2014، 1:50ص

1 Round trip, read up abount TCP congestion control, initial windows and so on.

http://samsaffron.com/archive/2012/03/01/why-upgrading-your-linux-kernel-will-make-your-customers-much-happier

renoirb · 2 مايو 2015، 2:29ص

UPDATE 2015-04-02: I just made the full example more complete.

I am confused with this definition.

But I think this should clarify it. Please, don’t hesitate to tell me if i’m doing something wrong.

The confusing part is that if “http://some-origin.com/”. If you are behind Fastly, you have to use a CNAME entry and then you have to have a sub domain name and not the top level.

Background: In DNS, a top level domain name (i.e. “some-origin.com”) can only have A records. Since Fastly requires we use a CNAME entry, we have no choice but to use a sub domain name.

Let’s say that we will then use “http://discourse.some-origin.com/” to serve our Discourse forum so we can use Fastly.

Now there’s this thing called “long polling” which is basically an OPTION HTTP request with a long time before returning anything. If we use the Fastly or Varnish address, as Discourse would by default, Varnish will time out and “long polling” won’t work.

More background: Varnish has this option to bypass in known contexts through vcl_pipe which is roughly a raw TCP socket. But Fastly doesn’t offer it because of the size of their setup.

Proposed setup

Let’s enable long polling and expose our site under Fastly. We’ll need two names, one pointing to Fastly’s and the other to the IP addresses we give within the service dashboard.

discourse.some-origin.com that’s our desired Discourse site domain name
discoursepolling.some-origin.com (pick any name) that we’ll configure in Discourse to access directly to our public facing frontend web server

In my case, I generally have many web apps running that are only accessible from my internal network. I refer to them as “upstream”; the same term NGINX uses in their config. Since this number of web apps you would host on a site can fluctuate, you might still want the number public IP address to remain stable. That’s why I setup a NGINX server in front that proxies to internal web app server. I refer to them as “frontends”.

Let’s say you have two public facing frontends running NGINX.

Those would be the ones you setup in Fastly like this.

Here we see two Backends in Fastly pannel at Configure -> Hosts.

Notice that in this example i’m using 443 port because my backends are configured to communicate between Fastly and my frontends through TLS. But you don’t need to.

Quoting again @sam;

[quote=“sam, post:1, topic:21467”]
To server “long polling” requests from a different domain, set the Site Setting long polling base url to the origin server.[/quote]

Really means here is that we would have to put one of those IP addresses in Discourse settings.

What I’d recommend is to create a list of A entries for all your frontends.

In the end we need three things:

What’s the public name that Fastly will serve
Which IPs are the frontends
Which hostname we want to use for long polling and we’ll add it to our VirtualHost

The zone file would look like this;

# The public facing URL
discourse.some-origin.com.  IN CNAME global.prod.fastly.net.

# The list of IP addresses you’d give to Fastly as origins/backends
frontends.some-origin.com.  IN A 8.8.8.113
frontends.some-origin.com.  IN A 8.8.8.115

# The long polling URL entry
discoursepolling.some-origin.com.  IN CNAME frontends.some-origin.com.

That way you can setup the “long polling base url” correctly without setting a single point of failure.

Then, we can go in Discourse admin zone and adjust the “long polling base url” to our other domain name.

# /etc/nginx/sites-enabled/10-discourse

# Let’s redirect to SSL, in case somebody tries to access the direct IP with
# host header.
server {
    listen      80;
    server_name discoursepolling.some-origin.com discourse.some-origin.com;
    include     common_params;
    return      301 https://$server_name$request_uri;
}

server {
    listen      443 ssl;
    server_name discoursepolling.some-origin.com discourse.some-origin.com;
    # Rest of NGINX server block
    # Also, I would make a condition if we are in discoursepolling but not
    # under using anything specific to polling.
    # #TODO; find paths specific to polling
}

To see if it works; look at your web browser developer tool “Network inspector” for /poll calls on discoursepolling.some-origin.com, and see if you have 200 OK status code.

brahn · 23 يناير 2017، 4:39ص

To clarify something here, in a multisite configuration, all sites should use the same long polling url? It looks to me like the this line is making that a requirement:

github.com/discourse/discourse

config/initializers/004-message_bus.rb

5dbd6a304


      
          group_ids = if is_admin
            # special rule, admin is allowed access to all groups
            Group.pluck(:id)
          elsif user
            user.groups.pluck('groups.id')
          end
          
          hash = {
            extra_headers:
              {
                "Access-Control-Allow-Origin" => Discourse.base_url_no_prefix,
                "Access-Control-Allow-Methods" => "GET, POST",
                "Access-Control-Allow-Headers" => "X-SILENCE-LOGGER, X-Shared-Session-Key, Dont-Chunk"
              },
            user_id: user_id,
            group_ids: group_ids,
            is_admin: is_admin,
            site_id: RailsMultisite::ConnectionManagement.current_db
          
          }
          env["__mb"] = hash

thanks!

Edit: No wait, that doesn’t work.

base site: example.com
long polling url: origin.example.com

multisite 1: mysite.com

If mysite uses origin.example.com as the long polling address I get:

XMLHttpRequest cannot load https://origin.example.com/message-bus/634dd18187094c6c950c0bf14f74c239/poll. Response to preflight request doesn't pass access control check: The 'Access-Control-Allow-Origin' header has a value 'https://example.com' that is not equal to the supplied origin. Origin 'https://mysite.com' is therefore not allowed access.

If mysite uses it’s own long polling origin as the domain I get this:

XMLHttpRequest cannot load https://origin.mysite.com/message-bus/b35c9c8e958f44f78d0d4773dc6d75f3/poll. Response to preflight request doesn't pass access control check: The 'Access-Control-Allow-Origin' header has a value 'https://example.com' that is not equal to the supplied origin. Origin 'https://mysite.com' is therefore not allowed access.

Is this because of "Access-Control-Allow-Origin" => Discourse.base_url_no_prefix ?

aurelien · 13 سبتمبر 2017، 12:09م

I have noticed there is no “cloudfront.template.yml” in discourse_docker/templates/. So I am wondering:
Can CloudFront work using the same techniques ?

aurelien · 15 سبتمبر 2017، 9:51ص

Also, can we use http2 ? Is the long polling stuff still needed when using http2 ?

fefrei · 15 سبتمبر 2017، 12:37م

If you’re using a the supported Docker-based install, HTTP2 should be working automatically!

Long polling is still needed for notifications to appear live.

SouperC · 15 سبتمبر 2017، 8:04م

I think if you have cloudfront setup, it’s only delivering specific objects (images), rather than the site/application in it’s entirety with js and so on.

So the only thing you need is to have the correct cloudfront url for those images.

ryanerwin · 20 مايو 2018، 5:13م

Some additional notes for anyone who decides to use HTTPS together with Full site CDN acceleration

Discourse internally uses the value of SiteSetting.force_https to decide if your access-control-allow-origin: is the HTTP or HTTPS version of your site. If while polling you see an error in the browser console along the lines of preflight request doesn't pass access control check: The 'Access-Control-Allow-Origin' header has a value http doesn't match https, double check your force_https setting. Also note the protocol in your DISCOURSE_CORS_ORIGIN in your container definition (http|https) will be overridden by force_https.
Don’t forget to add DISCORSE_ENABLE_CORS: true in your container definition.
If you were planning to only do HTTPS from your end users to your CDN, and then HTTP from your CDN to your actual Discourse web_only containers, lots of custom configuration will be required.
If your CDN is serving your site on HTTPS, then whatever Long Polling URL you setup must also be on HTTPS, so even if the CDN is handling your HTTPS, you must still setup HTTPS on your Discourse servers. If you run into an error about Same-origin policy, double check that you’re not trying to connect to HTTP instead of HTTPS
- If you use letsencrypt to generate your certificates, note that fullchain.pem => /shared/ssl/ssl.crt (ssl_certificate)
- privkey.pem => /shared/ssl/ssl.key (ssl_certificate_key)
You might use the following templates in your container definition:

  - "templates/web.template.yml"
  - "templates/web.ssl.template.yml"
  - "templates/fastly.template.yml"

Towards the end of the hook:ssl inside templates/web.ssl.template.yml you’ll see this block being added to your /etc/nginx/conf.d/discourse.conf.

if ($http_host != $$ENV_DISCOURSE_HOSTNAME) {
    rewrite (.*) https://$$ENV_DISCOURSE_HOSTNAME$1 permanent;
}

You’ll need to need to comment these lines out, otherwise you’re long polling attempts always serve up 301 redirects back to your origin, instead of respecting whatever you set in SiteSetting.long_polling_base_url

The easiest way I’ve found to do this is to copy templates/web.ssl.template.yml to local.web.ssl.template.yml and just remove those extra lines, and update your container reference to use your local template. If you go that route, you should periodically diff your local version with the origional version, because there are some security improvements that are regularly incorporated into this template.

Some of the error messages you’ll run into until things are configured correctly.

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://polling.example.com/message-bus/37c91c51e6cd4b0c95288b8fc29a0480/poll. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).

Reason: CORS header ‘Access-Control-Allow-Origin’ missing

Response to preflight request doesn’t pass access control check: No ‘Access-Control-Allow-Origin’ header is present on the requested resource

Failed to load https://polling.example.com/message-bus/8caefcec2cf94de3ae684c4b953a1084/poll: Response for preflight is invalid (redirect)

brahn · 21 مايو 2018، 2:46ص

You can do this with a - replace in your app.yml similar to how it is described on Setting up Let’s Encrypt with Multiple Domains.

  after_ssl:
    - replace:
        filename: "/etc/nginx/conf.d/discourse.conf"
        from: /return 301 https.+/
        to: |
          return 301 https://$host$request_uri;

    - replace:
        filename: "/etc/nginx/conf.d/discourse.conf"
        from: /gzip on;[^\}]+\}/m
        to: |
          gzip on;
          add_header Strict-Transport-Security 'max-age=31536000'; # remember the certificate for a year and automatically connect to HTTPS for this domain

ryanerwin · 21 مايو 2018، 3:22ص

@brahn, I was looking at using a pups replace line, but I couldn’t figure out how to do a multi-line match in pups…

Note that templates/web.ssl.template.yml is inside of the port 443 block, not the port 80 block.

Is there a way to use the pups, replace command to match the entire mult-line string?

if ($http_host != www.example.com) {
   rewrite (.*) https://www.example.com$1 permanent;
}

The most direct way I thought of inside the container definition is an exec line running perl, awk or sed to do the multiline replace… but then you’ve got shell escaping along with your target language to disentangle before it will work…

brahn · 21 مايو 2018، 3:52ص

The first replace takes care of the redirect from http to https for multisite. Perhaps that one is not relevant for you.

The second replace is multi-line. It replaces everything from line 33 to line 39 that was added by the web.ssl template.

It just removes that whole rewrite block. I could not figure out what purpose it serves and it breaks mutlisite so…

You could do this in your app.yml:

after_ssl:
    - replace:
        filename: "/etc/nginx/conf.d/discourse.conf"
        from: /gzip on;[^\}]+\}/m
        to: |
          gzip on;
          add_header Strict-Transport-Security 'max-age=31536000'; # remember the certificate for a year and automatically connect to HTTPS for this domain
          if ($http_host != www.example.com) {
            rewrite (.*) https://www.example.com$1 permanent;
          }

itsbhanusharma · 23 يناير 2019، 7:44ص

@sam

I’m curious to know one thing:
Eg. My discourse is hosted on forum.example.com
Can I set long polling base to poll.example.org which points to the same server IP?
Will it have any impact considering CSP?

satonotdead · 7 أبريل 2020، 6:50ص

Hi there. Why you guys recommends keep away from Cloudflare when we are in Discourse?

I’m trying to install it in my new VPS (Debian in Hetzner) and I thought that could be useful to keep Cloudflare on in my little server.

Thanks for your time.

Stephen · 7 أبريل 2020، 6:55ص

لا تُعد CloudFlare شبكة توصيل محتوى (CDN) تقليدية، بل هي وكيل شبكة. فبعض ميزات الأداء الخاصة بها تُعدّل الكود بين العميل والخادم.

ترك هذه الميزات مفعّلة قد يتسبب في تعطل Discourse بطرق جديدة ومثيرة للاهتمام. أما إذا قمت بإيقافها، فإنك تضيف فقط قفزات شبكية إضافية بين تطبيق Discourse في متصفحك والخادم. وكلما زادت القفزات، أصبح الواجهة أقل استجابة.

satonotdead · 7 أبريل 2020، 7:02ص

Well, I’m in Hetzner VPS and I read that could be good to use Cloudflare to keep my server safe (in eventual attacks). CDN could be OK too, because I’m in another country (America, not Germany).

What do you think about it?

Stephen · 7 أبريل 2020، 7:18ص

I’m not going to comment on whether you should be worried about attacks. You need to make that assessment yourself, but don’t fall prey to FUD.

If you leave their performance features enabled we cannot support you here. As mentioned above they interfere with the javascript in ways that do no good.

You may be able to make the basic asset caching work with all of the other performance features turned off.

Even then, if Cloudflare is active during installation and setup, certificate enrolment will fail. Let’s Encrypt isn’t supported behind a Cloudflare proxy for initial enrolment.

satonotdead · 7 أبريل 2020، 7:33ص

Thanks for your answer Stephen. I’m facing issues trying to install Discourse and I thought that could be related to Cloudflare.

So, I can’t use it neither to manage DNS? How can I protect my server and keep Discourse without Cloudflare?

RGJ · 7 أبريل 2020، 7:57ص

Press the orange cloud next to your hostname in the Cloudfare control panel so the cloud turns grey.
Then install Discourse. If you want to protect your server, press the grey cloud so it turns orange, but make sure to disable all performance features first.

الموضوع		الردود	مرات العرض
Enable a CDN for your Discourse Self-Hosting cdn , configuring , how-to	72	294023	4 أكتوبر 2025
MessageBus short polling is not working Bug	12	2810	29 سبتمبر 2017
Discourse & Cloudflare Self-hosting	49	5034	26 يناير 2024
My discourse speed is very slow Self-hosting	24	4826	4 مارس 2021
Discourse CDNs are blocked by privacy badger Site feedback	47	12849	15 أغسطس 2017

تسريع شبكة توصيل المحتوى (CDN) للموقع بالكامل لـ Discourse

ملاحظات فنية مملة:

Proposed setup

الموضوعات ذات الصلة