Caching post requests with varnish

I’ve written a Varnish module (VMOD) called libvmod-bodyaccess that lets you get access to the request body. The VMOD has a set of functions that expand the already existing function for buffering the request body. Retrieving the request body allows us to manipulate it and here lies the value. Altering the request body allows you to do all sort of neat stuff.

Usually the request body isn't buffered and because of this it can only be consumed once. This VMOD makes it possible to buffer, have regular expression matching on request body and to open up for caching POST/PUT requests.

The VMOD is available for Varnish 4 releases and includes different functions:

buffer_req_body(BYTES size): buffers the request body if it is smaller than size. This function is a “better” (bug-free**) copy of std.CacheReqBody(), so please use this one instead of the one provided by the libvmod-std. Please note that retrieving req.body makes it possible to retry pass operations(POST, PUT).

len_req_body(): returns the request body length.

rematch_req_body(STRING re): regular expression match on request body.

hash_req_body(): adds body bytes to the input hash key.

The VMOD with examples and documentation can be found here: https://github.com/aondio/libvmod-bodyaccess

What's the idea behind this VMOD?

Starting from Varnish 4 we have the ability to buffer the request body data before passing it to the backend.

When a POST or PUT request is received, we might want to do some validation on the request body before sending to the backend. These checks could for instance be security checks. Combining regular expression matches on request body and its length, we are able to identify possible malicious POST/PUT requests and discard them. Once we are sure we have a trusted request body, we might want to add available request body bytes to the lookup hash key, giving the option to cache some POST requests.

Tips, tricks and things you should know about the VMOD

Varnish converts all request methods to GET in the backend request. Now with this new VMOD a POST/PUT request can be handled in a better way, and we can “force” the use of those methods on the backend side. This can be done by writing something like this to your vcl:

pcre sometimes gives us a hard time; so far, in my testing I haven't hit any problems, but if problems arise, the first thing you might want to tune is pcre parameters, such as “pcre_match_limit” and “pcre_match_limit_recursion”.

Buffering the request body before doing any kind of operation on it is mandatory because a non-buffered req.body can be consumed only once.

Why is this VMOD good for you?

Adding a few lines to your VCL configuration you will be able to validate the incoming request body making it available for various manipulations and for caching PUT/POST requests.Here is a complete VCL example: