Two-Minute Tech Tuesdays

In this week's 2-Minute Tech Tuesday, we'll be discussing banning, which is a mechanism in Varnish to remove outdated content from the cache so that new content can be presented immediately, rather than waiting for the cache to expire.

Banning allows Varnish to remove multiple objects from the cache at once through ban expressions. Here's an example of such a ban expression:

obj.status > 200

As you can see, there's a field to match, there's an operator, and there's a value. So in this case, we're removing all the objects from the cache that have a status code greater than 200.

Here's another example:

req.http.host == example.com && req.url ~ ^/admin

We have a double equal sign operator to do an exact match, and we use a logical AND operator to chain multiple conditions to one another.

Bans can be invoked through Varnish ADM by running the ban subcommands. The ban expression that we want to match against the object is added as an extra argument.

varnishadm ban "req.url ~ ^admin && req.http.host == example.com"

We can also use HTTP to get the job done, and for that, we need VCL.

vcl 4.1;

sub vcl_recv {
    if (req.mthod == "PURGE" && req.http.x-invalidate-pattern) {
        ban("obj.http.x-url ~ " + req.http.x-invalidate-pattern
            + " && obj.http.x-host == " + req.http.host);
        return (synth(200,"Ban added"));
    }
}

sub vcl_backend_response {
    set beresp.http.x-url = bereq.url;
    set beresp.http.x-host = bereq.http.host;
}

sub vcl_deliver {
    unset resp.http.x-url;
    unset resp.http.x-host;
}

This VCL code looks surprisingly similar to the one we saw in last week's video. We're actually reusing the purge request method to trigger the invocation. However, we're adding an extra request header called x-invalidate-pattern. The value of the x-invalidate-pattern request header contains the URL pattern that we want to match against the objects and remove from the cache.

We use this value, as well as the host header, to compose a full ban expression and to invoke through VCL's ban function. In the end, we return synthetic HTTP confirming the ban invocation. The rest of the code is just there to optimize the processing of the bans.

In a production environment, an extra layer of security needs to be added to avoid unauthorized bans. We do this by defining an access control list (ACL) that contains IP addresses, hostnames, and subnets where the client can connect from. If the client IP address doesn't match the ACL, a 405 Method Not Allowed error is returned.

Here's an example of an HTTP request that is used to trigger a ban:

PURGE / HTTP/1.1
X-Invalidate-Pattern: ^/admin
Host: example.com

It's important to use the purge request method. It is also essential that you specify a URL pattern through the x-invalidate-pattern request header. And of course, the right host header needs to be specified as well.

This is the potential HTTP response you may get from Varnish:

HTTP/1.1 200 Ban added
Date: Tue, 20 Oct 2020 13:30:12 GMT
Server: Varnish
X-Varnish: 32770
Content-Type: text/html; charset=utf-8
Retry-After: 5
Content-Length: 249
Accept-Ranges: bytes
Connection: keep-alive

<!DOCTYPE html>
<html>
  <head>
   <title>200 Ban added</title>
  </head>
  <body>
   <h1>Error 200 Ban added</h1>
   <p>Ban added</p>
   <h3>Guru Meditation:</h3>
   <p>XID: 32770</p>
   <hr>
   <p>Varnish cache server</p>
  </body>
</html>

In terms of implementation, we can translate the HTTP request into the following curl command, or use whatever HTTP client or dedicated Varnish plugin that your framework offers.

curl -XPURGE -H"X-Invalidate-Pattern: ^/admin" http://example.com/

If you looked at last week's episode about purging, you can now make the comparison between purging and banning and see what works best for you in terms of cache invalidation.

Two-Minute Tech Tuesdays - Banning

Cache Invalidation in Varnish

Caching “Uncacheable” Artifactory Resources to Boost Cache Efficiency