Uncovering 3 "hidden" VMODs

Varnish Cache and Varnish Plus can both be extended by Varnish Modules, or VMODs.

The aim of a VMOD is to extend and expand the range of functionality that Varnish already provides. Varnish Software engineers, of course, create VMODs, but with the flexibility of the VCL configuration language, many people have created their own VMODs, which gets to the core of why Varnish is seen as a kind of “Swiss army knife”. It can do a lot of different things, and is not at all limited by the constraints of what we at Varnish Software have created. Nevertheless, we engineers do write and maintain a fair number of VMODs ourselves, some of which can be considered as core Varnish features.

They are imported and called from your VCL to offer a larger set of capabilities to your setup.

What’s new?

Varnish Plus 4.1.7 has just been released (as of the 28th of June) and among the shiny, new features that are a part of the release, we have also released new VMODs. In this blog post we are going to talk about three VMODs that might as well be hidden because they are not very well-known. It’s time to bring them out of hiding.

VMOD_SESSION
This VMOD lets you manipulate session local variables. For now only the timeout_idle can be tweaked. The timeout_idle is a timeout for client connections. A connection is considered idle until we have received the full request headers.

import session;

sub vcl_recv {
    session.set_timeout_idle(30s);
}

By default the timeout_idle is a global parameter that you can set either through varnishadm or, for a permanent solution, you can set it among the daemon parameters varnishd uses on startup.

This VMOD gives you the freedom to decide what will be the value of the timeout_idle based on the incoming request Varnish is seeing, so this timeout value is no longer set as “global” but is instead now “per session”.

Once you have installed the lastest Varnish Plus, you can read the full documentation typing from command line:  `man vmod_session`

VMOD_BODYACCESS
This module provides a set of functions that will help you to manipulate the request body. This VMOD makes possible to buffer, have regular expression matching on request body and open up for caching POST requests.

When a POST request is received, we might want to make sure it is not a malicious request before sending it to the backend. Using the functions provided in this VMOD, we can run security checks; for instance, we can combine regular expression matching with the request body length and make sure it's not a suspicious request.

sub vcl_recv {
    # need to buffer the whole request body
    std.cache_req_body(1KB);
    set req.http.x-len = bodyaccess.len_req_body();
    set req.http.x-re = bodyaccess.rematch_req_body("Regex");
}

sub vcl_hash {
    bodyaccess.hash_req_body();
    return (lookup);
}

sub vcl_backend_fetch {
    set bereq.method = bereq.http.x-method;
}

sub vcl_deliver {
    set resp.http.x-len = req.http.x-len;
    set resp.http.x-re = req.http.x-re;
}

The VMOD is part of the varnish-modules collection; type `man vmod_bodyaccess` for complete documentation and examples.

VMOD_REWRITE

If you stare for a second at your VCL you might notice some of the lines of the entire file are dedicated to operations, such as URLs or headers rewrite/manipulation. In most cases you don’t want to rewrite a single URL, but you’ll want to manipulate a whole range of headers/URLs based on specific rules that will lead up to a long list of if-else clauses, which are not easy to maintain or to debug.

The aim of this VMOD is to simplify this whole process, reducing the number of VCL lines you’ll have to write to specify any rewrite rule.

Prior to this VMOD, the VCL would have looked like:

sub vcl_recv {
    if (req.url ~ "pattern1") {
        set req.url = regsub(req.url, "regex1", "subtitute1");
    else if (req.url ~ "pattern2") {
        set req.url = regsub(req.url, "regex2", "subtitute2");
    }
}

While if you use the VMOD everything boils down to just a few VCL lines:

import rewrite; 

sub vcl_init {
    new rs = rewrite.ruleset("/path/to/file.rules");
}

sub vcl_recv {
    set req.url = rs.replace(req.url);
}

The file file.rules contains a set of rules to be applied based on the incoming URL.

You can take a look at the complete documentation for this VMOD.

Plenty of VMODs seem to lurk in the shadows but do exist and serve specific and useful purposes; we will try to shine a light on them more in the future.