Does anyone have experience with hosing Discourse behind a Varnish instance? I use it for load balancing.
Which cookies do I need to whitelist (ie. enable to affect the page) when using Varnish?
I have it working very well - as far as I can see Discourse behaves well towards Varnish and you can aggressively cache everything under the /assets directory.
backend discourse {
.host = "x.x.x.x";
.port = "81";
}
sub vcl_recv {
if (req.http.host == "forum.myhost.org.uk") {
set req.backend = discourse;
}
if (req.http.host == "forum.myhost.org.uk") {
if (req.url ~ "^/assets") {
return(lookup);
} else {
return(pipe);
}
}
Obviously there is more in my varnish config, but that is the basic / cache specific bit. I proxy through Apache, so in the virtual host have something like this:
<LocationMatch "^/assets/.*$">
Header set Cache-Control "public, max-age = 3600"
</LocationMatch>
In my setup, the max-age is used to tell varnish to cache, though different cache headings can be used.
There might be a few other things that can be safely cached (comments anyone?) though the above seems to work great.
It takes load off Apache, on which I run mod_security so the stack is both fast and pretty secure 
)
Enjoy…
3 Likes
I would strongly recommend going with a origin pull CDN for assets as opposed to caching locally, CDNs are so cheap anyway.
4 Likes
That makes a lot of sense, thanks 
Reviving this briefly. I’ve already got Varnish running on my web server, so including Discourse’s static assets in it has basically zero additional cost to me.
On the other hand, I’m running Discourse behind Nginx with Passenger; Nginx is quite good at serving static assets—in fact, under low-to-moderate load, it’s almost always faster than Varnish. Varnish wins as load begins to ramp, though.
Right now (since my Discourse forum is LAN-only and really just for me to play with) I’ve got a statement in vcl_recv to just pass all traffic with my Discourse URL in it, but will it be problematic to enable it? @richp10, have you noticed any cookies placed that you need to strip?
I’ve gone ahead and turned on Varnish caching for my assets directory, using a slightly modified version of @richp10’s config:
# Cache only the static assets in Discourse's "assets" dir and pass everything else
if (req.http.host ~"discourse.bigdinosaur.org") {
if (!(req.url ~ "^/assets/")) {
return (pass);
}
}
This is a hell of a lot more convenient for me than dealing with a CDN, since I’m already using Varnish on the web server. Plus, there’s quite a bit of stuff in there; keeping the static assets in Varnish will ensure that they’re accessed from RAM instead of having to push IO down onto the disk.
I’m not anticipating a massive performance increase or anything, but it seems like just a smart thing to do.
Thanks! I needed this.
Maybe you can do a little “how to” tutorial for this plz?
Thanks!
I’ve changed my discourse varnish config since 2013. Looks like this now:
sub vcl_recv {
...
# Cache only the static assets and pass everything else
if (req.http.host ~"discourse.bigdinosaur.org") {
if (req.url ~ "^/uploads/" ) {
return (lookup);
}
elseif (req.url ~ "^/assets/") {
return (lookup);
}
elseif (req.url ~ "^/user_avatar/") {
return (lookup);
}
else {
return (pass);
}
}
...
}
sub vcl_pass {
set bereq.http.connection = "close";
# Fix broken behavior showing tons of requests from 127.0.0.1 with Discourse
if (req.http.X-Forwarded-For) {
set bereq.http.X-Forwarded-For = req.http.X-Forwarded-For;
} else {
set bereq.http.X-Forwarded-For = regsub(client.ip, ":.*", "");
}
}
Note also this is for Varnish 3. I’ve been putting off the migration to Varnish 4 because the VCL syntax changes and I haven’t put in the time to sit down and figure out how to translate my config.
2 Likes
Upgraded Varnish to V4, bringing with it some subtle but important VCL syntax changes. My Discourse block now looks like this, but it’s working only inconsistently:
# Cache only the static assets in Discourse's' "assets" dir and pass everything else
if (req.http.host ~"discourse.bigdinosaur.org") {
if (req.url ~ "^/uploads/" ) {
return (hash);
}
elseif (req.url ~ "^/assets/") {
return (hash);
}
elseif (req.url ~ "^/user_avatar/") {
return (hash);
}
else {
return (pipe);
}
}
(lol hash pipe)
I’m seeing weirdly inconsistent cache hits and I have absolutely no clue why. Some objects that match the regexes above (like user avatars) are being cached, and some others (like…other user avatars) aren’t.
Haven’t really had the time yet to sit down and try to figure out what might be wrong or misconfigured, since switching over to varnish v4 meant putting out a bunch of other fires, but I’ll poke at it tomorrow, maybe.
6 Likes
Quick update on this a year later: this is the Varnish 4.1/5 Discourse config I’ve been using that appears to work without any issues:
if (req.http.host ~"discourse.bigdinosaur.org") {
if (!(req.url ~ "(^/uploads/|^/assets/|^/user_avatar/)" )) {
return (pass);
}
}
Could probably also tweak it a bit, since it’s ignoring and not caching some emoji (specifically emojii with query strings on the end of their URLs) and a couple of other things (scripts, css). But it works!
edit - this is in sub vcl_recv, obviously.
5 Likes
People here seem to be using Varnish + an extra nginx
The nginx bit seems unecessary, plus Discourse already uses nginx in it’s container
Is there a way for Varnish to go directly to the container and not use an extra nginx or apache
ok I figured it out, dumb question
and it improves loading times dramatically to remove other proxies
Also I used @Lee_Ars 's caching config and it produces odd behaviour like certain avis just not displaying so everything is return (pass); for now. Have you not gotten any odd behaviour with your caching @Lee_Ars ?
@Lee_Ars ant new updates in Varnish 6.x? Thanks.
1 Like