Varnish Configuration¶
Below you will find detailed Varnish configuration recommendations for the features provided by this library. The configuration is provided for Varnish 3 and 4.
Basic Varnish Configuration¶
To invalidate cached objects in Varnish, begin by adding an ACL to your Varnish configuration. This ACL determines which IPs are allowed to issue invalidation requests. Let’s call the ACL invalidators. The ACL below will be used throughout the Varnish examples on this page.
# /etc/varnish/your_varnish.vcl
acl invalidators {
"localhost";
# Add any other IP addresses that your application runs on and that you
# want to allow invalidation requests from. For instance:
# "192.168.1.0"/24;
}
Important
Make sure that all web servers running your application that may trigger invalidation are whitelisted here. Otherwise, lost cache invalidation requests will lead to lots of confusion.
Purge¶
To configure Varnish for handling PURGE requests:
Purge removes a specific URL (including query strings) in all its variants (as specified by the Vary
header).
- Varnish 4
1 2 3 4 5 6 7 8
sub vcl_recv { if (req.method == "PURGE") { if (!client.ip ~ invalidators) { return (synth(405, "Not allowed")); } return (purge); } }
- Varnish 3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
sub vcl_recv { if (req.request == "PURGE") { if (!client.ip ~ invalidators) { error 405 "Not allowed"; } return (lookup); } } sub vcl_hit { if (req.request == "PURGE") { purge; error 204 "Purged"; } } # The purge in vcl_miss is necessary to purge all variants in the cases where # you hit an object, but miss a particular variant. sub vcl_miss { if (req.request == "PURGE") { purge; error 204 "Purged (Not in cache)"; } }
Refresh¶
If you want to invalidate cached objects by forcing a refresh add the following to your Varnish configuration:
Refresh invalidates a specific URL including the query string, but not its variants.
1 2 3 4 5 | sub vcl_recv {
if (req.http.Cache-Control ~ "no-cache" && client.ip ~ invalidators) {
set req.hash_always_miss = true;
}
}
|
Ban¶
To configure Varnish for handling BAN requests:
- Varnish 4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
sub vcl_recv { if (req.method == "BAN") { if (!client.ip ~ invalidators) { return (synth(405, "Not allowed")); } ban("obj.http.X-Host ~ " + req.http.X-Host + " && obj.http.X-Url ~ " + req.http.X-Url + " && obj.http.content-type ~ " + req.http.X-Content-Type ); return (synth(200, "Banned")); } } sub vcl_backend_response { # Set ban-lurker friendly custom headers set beresp.http.X-Url = bereq.url; set beresp.http.X-Host = bereq.http.host; } sub vcl_deliver { # Keep ban-lurker headers only if debugging is enabled if (!resp.http.X-Cache-Debug) { # Remove ban-lurker friendly custom headers when delivering to client unset resp.http.X-Url; unset resp.http.X-Host; unset resp.http.X-Cache-Tags; } }
- Varnish 3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
sub vcl_recv { if (req.request == "BAN") { if (!client.ip ~ invalidators) { error 405 "Not allowed."; } ban("obj.http.X-Host ~ " + req.http.X-Host + " && obj.http.X-Url ~ " + req.http.X-Url + " && obj.http.content-type ~ " + req.http.X-Content-Type ); error 200 "Banned"; } } sub vcl_fetch { # Set ban-lurker friendly custom headers set beresp.http.X-Url = req.url; set beresp.http.X-Host = req.http.host; } sub vcl_deliver { # Keep ban-lurker headers only if debugging is enabled if (!resp.http.X-Cache-Debug) { # Remove ban-lurker friendly custom headers when delivering to client unset resp.http.X-Url; unset resp.http.X-Host; unset resp.http.X-Cache-Tags; } }
Varnish contains a ban lurker that crawls the content to eventually throw out banned data even when it’s not requested by any client.
Tagging¶
Add the following to your Varnish configuration to enable cache tagging.
Note
The custom X-Cache-Tags
header should match the tagging header
configured in the cache invalidator.
- Varnish 4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41
sub vcl_recv { if (req.method == "BAN") { if (!client.ip ~ invalidators) { return (synth(405, "Not allowed")); } if (req.http.X-Cache-Tags) { ban("obj.http.X-Host ~ " + req.http.X-Host + " && obj.http.X-Url ~ " + req.http.X-Url + " && obj.http.content-type ~ " + req.http.X-Content-Type + " && obj.http.X-Cache-Tags ~ " + req.http.X-Cache-Tags ); } else { ban("obj.http.X-Host ~ " + req.http.X-Host + " && obj.http.X-Url ~ " + req.http.X-Url + " && obj.http.content-type ~ " + req.http.X-Content-Type ); } return (synth(200, "Banned")); } } sub vcl_backend_response { # Set ban-lurker friendly custom headers set beresp.http.X-Url = bereq.url; set beresp.http.X-Host = bereq.http.host; } sub vcl_deliver { # Keep ban-lurker headers only if debugging is enabled if (!resp.http.X-Cache-Debug) { # Remove ban-lurker friendly custom headers when delivering to client unset resp.http.X-Url; unset resp.http.X-Host; unset resp.http.X-Cache-Tags; } }
- Varnish 3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41
sub vcl_recv { if (req.request == "BAN") { if (!client.ip ~ invalidators) { error 405 "Not allowed."; } if (req.http.X-Cache-Tags) { ban("obj.http.X-Host ~ " + req.http.X-Host + " && obj.http.X-Url ~ " + req.http.X-Url + " && obj.http.content-type ~ " + req.http.X-Content-Type + " && obj.http.X-Cache-Tags ~ " + req.http.X-Cache-Tags ); } else { ban("obj.http.X-Host ~ " + req.http.X-Host + " && obj.http.X-Url ~ " + req.http.X-Url + " && obj.http.content-type ~ " + req.http.X-Content-Type ); } error 200 "Banned"; } } sub vcl_fetch { # Set ban-lurker friendly custom headers set beresp.http.X-Url = req.url; set beresp.http.X-Host = req.http.host; } sub vcl_deliver { # Keep ban-lurker headers only if debugging is enabled if (!resp.http.X-Cache-Debug) { # Remove ban-lurker friendly custom headers when delivering to client unset resp.http.X-Url; unset resp.http.X-Host; unset resp.http.X-Cache-Tags; } }
User Context¶
To support user context hashing you need to add some logic
to the recv
and the deliver
methods:
- Varnish 4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86
sub vcl_recv { # Prevent tampering attacks on the hash mechanism if (req.restarts == 0 && (req.http.accept ~ "application/vnd.fos.user-context-hash" || req.http.X-User-Context-Hash ) ) { return (synth(400)); } # Lookup the context hash if there are credentials on the request # Only do this for cacheable requests. Returning a hash lookup discards the request body. # https://www.varnish-cache.org/trac/ticket/652 if (req.restarts == 0 && (req.http.cookie || req.http.authorization) && (req.method == "GET" || req.method == "HEAD") ) { # Backup accept header, if set if (req.http.accept) { set req.http.X-Fos-Original-Accept = req.http.accept; } set req.http.accept = "application/vnd.fos.user-context-hash"; # Backup original URL set req.http.X-Fos-Original-Url = req.url; set req.url = "/_fos_user_context_hash"; # Force the lookup, the backend must tell not to cache or vary on all # headers that are used to build the hash. return (hash); } # Rebuild the original request which now has the hash. if (req.restarts > 0 && req.http.accept == "application/vnd.fos.user-context-hash" ) { set req.url = req.http.X-Fos-Original-Url; unset req.http.X-Fos-Original-Url; if (req.http.X-Fos-Original-Accept) { set req.http.accept = req.http.X-Fos-Original-Accept; unset req.http.X-Fos-Original-Accept; } else { # If accept header was not set in original request, remove the header here. unset req.http.accept; } # Force the lookup, the backend must tell not to cache or vary on the # user hash to properly separate cached data. return (hash); } } sub vcl_backend_response { if (bereq.http.accept ~ "application/vnd.fos.user-context-hash" && beresp.status >= 500 ) { return (abandon); } } sub vcl_deliver { # On receiving the hash response, copy the hash header to the original # request and restart. if (req.restarts == 0 && resp.http.content-type ~ "application/vnd.fos.user-context-hash" ) { set req.http.X-User-Context-Hash = resp.http.X-User-Context-Hash; return (restart); } # If we get here, this is a real response that gets sent to the client. # Remove the vary on context user hash, this is nothing public. Keep all # other vary headers. set resp.http.Vary = regsub(resp.http.Vary, "(?i),? *X-User-Context-Hash *", ""); set resp.http.Vary = regsub(resp.http.Vary, "^, *", ""); if (resp.http.Vary == "") { unset resp.http.Vary; } # Sanity check to prevent ever exposing the hash to a client. unset resp.http.X-User-Context-Hash; }
- Varnish 3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88
sub vcl_recv { # Prevent tampering attacks on the hash mechanism if (req.restarts == 0 && (req.http.accept ~ "application/vnd.fos.user-context-hash" || req.http.X-User-Context-Hash ) ) { error 400; } # Lookup the context hash if there are credentials on the request # Only do this for cacheable requests. Returning a hash lookup discards the request body. # https://www.varnish-cache.org/trac/ticket/652 if (req.restarts == 0 && (req.http.cookie || req.http.authorization) && (req.request == "GET" || req.request == "HEAD") ) { # Backup accept header, if set if (req.http.accept) { set req.http.X-Fos-Original-Accept = req.http.accept; } set req.http.accept = "application/vnd.fos.user-context-hash"; # Backup original URL set req.http.X-Fos-Original-Url = req.url; set req.url = "/_fos_user_context_hash"; # Force the lookup, the backend must tell not to cache or vary on all # headers that are used to build the hash. return (lookup); } # Rebuild the original request which now has the hash. if (req.restarts > 0 && req.http.accept == "application/vnd.fos.user-context-hash" ) { set req.url = req.http.X-Fos-Original-Url; unset req.http.X-Fos-Original-Url; if (req.http.X-Fos-Original-Accept) { set req.http.accept = req.http.X-Fos-Original-Accept; unset req.http.X-Fos-Original-Accept; } else { # If accept header was not set in original request, remove the header here. unset req.http.accept; } # Force the lookup, the backend must tell not to cache or vary on the # user hash to properly separate cached data. return (lookup); } } sub vcl_fetch { if (req.restarts == 0 && req.http.accept ~ "application/vnd.fos.user-context-hash" && beresp.status >= 500 ) { error 503 "Hash error"; } } sub vcl_deliver { # On receiving the hash response, copy the hash header to the original # request and restart. if (req.restarts == 0 && resp.http.content-type ~ "application/vnd.fos.user-context-hash" && resp.status == 200 ) { set req.http.X-User-Context-Hash = resp.http.X-User-Context-Hash; return (restart); } # If we get here, this is a real response that gets sent to the client. # Remove the vary on context user hash, this is nothing public. Keep all # other vary headers. set resp.http.Vary = regsub(resp.http.Vary, "(?i),? *X-User-Context-Hash *", ""); set resp.http.Vary = regsub(resp.http.Vary, "^, *", ""); if (resp.http.Vary == "") { remove resp.http.Vary; } # Sanity check to prevent ever exposing the hash to a client. remove resp.http.X-User-Context-Hash; }
Your backend application should respond to the application/vnd.fos.user-context-hash
request with a proper user hash.
Note
We do not use X-Original-Url
here, as the header will be sent to the
backend and some applications look at this header, which would lead to
problems. For example, the Microsoft IIS rewriting module uses this header
and Symfony has to look into that header to support IIS.
Note
If you want the context hash to be cached, you need to always set the
req.url
to the same URL, or Varnish will cache every hash lookup
separately.
However, if you have a paywall scenario, you need to leave the original URL unchanged.
Cleaning the Cookie Header¶
In the examples above, an unaltered Cookie header is passed to the backend to use for determining the user context hash. However, cookies as they are sent by a browser are unreliable. For instance, when using Google Analytics, cookie values are different for each request. Because of this, the hash request would not be cached, but multiple hashes would be generated for one and the same user.
To make the hash request cacheable, you must extract a stable user session id. You can do this as explained in the Varnish documentation:
1 2 3 4 5 6 7 8 9 10 11 | sub vcl_recv {
# ...
set req.http.cookie = ";" + req.http.cookie;
set req.http.cookie = regsuball(req.http.cookie, "; +", ";");
set req.http.cookie = regsuball(req.http.cookie, ";(PHPSESSID)=", "; \1=");
set req.http.cookie = regsuball(req.http.cookie, ";[^ ][^;]*", "");
set req.http.cookie = regsuball(req.http.cookie, "^[; ]+|[; ]+$", "");
# ...
}
|
Note
If your application’s user authentication is based on a cookie other than
PHPSESSID, change PHPSESSID
to your cookie name.
Debugging¶
Configure your Varnish to set a custom header (X-Cache) that shows whether a cache hit or miss occurred. This header will only be set if your application sends an X-Cache-Debug header:
- Varnish 4
1 2 3 4 5 6 7 8 9 10 11 12 13
sub vcl_deliver { # Add extra headers if debugging is enabled # In Varnish 4 the obj.hits counter behaviour has changed, so we use a # different method: if X-Varnish contains only 1 id, we have a miss, if it # contains more (and therefore a space), we have a hit. if (resp.http.X-Cache-Debug) { if (resp.http.X-Varnish ~ " ") { set resp.http.X-Cache = "HIT"; } else { set resp.http.X-Cache = "MISS"; } } }
- Varnish 3
1 2 3 4 5 6 7 8 9 10
sub vcl_deliver { # Add extra headers if debugging is enabled if (resp.http.X-Cache-Debug) { if (obj.hits > 0) { set resp.http.X-Cache = "HIT"; } else { set resp.http.X-Cache = "MISS"; } } }