Varnish Cache Invalidation with Super Fast Purger

A couple of years ago we introduced the Super Fast Purger (SFP) as an essential feature shipped with the Varnish Administration Console (VAC). You can execute your purge directly against Varnish or against the VAC.

We all know that we cache really fast so we also need something to invalidate the content really fast. When your web server updates content, Varnish needs a notification in order to expunge the outdated content. Our high performance cache invalidation is an HTTP client, custom-built  for Varnish Cache, designed to do exactly this and can handle tens of thousands of invalidation events across multiple data centers.

Set up

No additional set up is required after installation of VAC; by default the listening port for Super Fast Purger is 8088, which can be configured. The configurable items that are specific to SPF are specified in /opt/vac/etc/default file. To purge using SFP, you must first have a VCL that handles it. The simplest purge example in Varnish4 looks like this:

Upon receiving an HTTP request, the SFP will look at its "X-Cache-Group" header, and duplicate the request to all the members of that group. The responses' headers are aggregated into one message and then sent back to the client.

Curl can issue a simple PURGE request to the SFP:

$ curl  -X PURGE http://vac-server:8088/itemToPurge -H "Host: localhost" -H "X-Cache-Group: Production" -H "X-HMAC: key"

The purge from the VAC will act upon all caches in the “Production” group. It will distribute the purge requests to group(s) of caches and can be used for multiple servers. Super Fast Purger specific headers are required to specify authentication and the targeted cache group, among them X-Cache-Group header is mandatory to specify the name of the group of Varnish cache(s) you wish to purge. SFP’s first use is purging content from caches but it has other uses, like warming the cache of a whole cluster.

Should anybody purge?

Security and flexibility were our concern when building SFP, and it doesn't change how you do the purge against Varnish Cache. You do not want all your users to be able to purge, so you can protect this operation using ACL, i.e., IP ranges:

Alternatively, you can use the digest VMOD to build check an HMAC sent in the request. For example:

Here, we hash a private key ("super_secret"), the host header and the url of the request. If the header "X-HMAC" doesn't match, the request is invalid (most probably, the issuer didn't know the private key).

The following headers will be present when the Super Fast Purger responds to the request.

X-Caches is the SFP specific header that contains the names of the caches within the group. cli_start refers to the time the connection was established and a thread has accepted the work. req_start refers to the last request's start time since cli_start and req_end refers to the last request's end time since cli_start. X-ReqNum shows the number of purges processed on this connection.

The Super Fast Purger is a feature that is exclusive to Varnish Plus. If you would like to evaluate SFP you can contact our sales department (sales (@) varnish-software.com) and they will set you up with a trial account.

Image (c) 2007 Will Luo used under Creative Commons license.