Partitioning your Varnish Cache

If you have multiple virtual hosts being handled by the same server you sometimes want some sort of QoS to apply to your caching. You might want to reserve a certain amount of memory for each virtual host. Let me show you how it can be done.

Varnish has pretty few built-in features compared to some of the other reverse proxies out there. Other proxies have explicit support for things, such as denying hotlinking or redirection. Varnish doesn't really need to add such features because it is trivial to add support for this yourself using VCL. This keeps the complexity of the Varnish core reasonably at bay without limiting what the users can do with Varnish.

You can have multiple storage backends in Varnish. Internally these are called stevedores, indicating a worker that loads or unloads something, usually a boat. You can have multiple stevedores active at the same time. In fact, Varnish usually has at least two of these, the user defined one and the transient stevedore, used for short-lived objects or used for emergencies. To set up Varnish to start with multiple stevedores, edit the command line. You want something like this:

Here we have three user-defined stevedores, default, foo and bar. In vcl_backend_response (4.0) or vcl_fetch (3.0) you can assign a response to a certain stevedore. As with anything in Varnish you can create arbitrary rules that govern what response is stored where. You could assign space in your cache for different user agents, IP ranges or, more likely, virtual hosts. Here, I'll use the three different stevedores based on the Host: header coming from the client.

It is worth pointing out that we must use the bereq object in vcl_backend_response as the req object is not available in vcl_backend_response, as you now can get into vcl_backend_response without a req object, due to asynchrounous backend requests. As you can see I've added a debug header here, to aid in testing.

Testing

As you can see the x-storage response header is set. If I look at varnishstat I'll see the storage backend has been kicked into action. 

You'll see both the number of allocator requests and the number of bytes allocated has increased. Success!

Image is (c) 2010 Dalal Al Mudhaf and used under a CC license.