Sometimes, have you ever had an itch to scratch, so you start a tiny prototype, and it grows a bit too big, so you need to make it a real project, and release it, and then you just can’t stop, because there’s always that one feature that makes sense to add?
Welp, varnish-go has been just that, and it’s turned into a project I’m quite happy with, so in this post, I’ll explain why I needed to build it, and then I’ll do my best to explain why you might need it.
varnishtest is one of the unsung heroes of the Varnish ecosystem. It’s the test framework that powers 99% of the CI pipeline, allowing us to create controlled clients, servers and Varnish instances that can run in parallel and ephemerally to test everything from our VCL and vmods, but also HTTP protocol issues, counter or logs discrepancies, etc. Virtually any code related to Varnish will have some Varnish Test Case (.vtc) files somewhere in its tree, with the Varnish tree alone having more than 1100 of them, and that number keeps growing.
There’s a full step-by-step tutorial on the syntax on the Dev Portal, but to give you a taste, here is what a VTC can look like:
Just feed the file to varnishtest and enjoy the success of your test passing:
It’s an amazing tool that allows us to verify that the Varnish internals are correct and also allows you to write tests against your VCL with fake, or real backends, while writing a minimal amount of code.
You can also verify counters, logs, do clustering, run shell scripts, inject headers or status in the requests and responses. Honestly, that tool is a great Swiss Army knife of a test framework for our Swiss Army knife of an HTTP server.
VTCs are written using a Domain Specific Language, which makes sense as it’s pretty...well, specific to our use case. But it creates three issues.
First, the language barrier is real: new users need to learn a new language to write tests. AI agents are surprisingly competent at it, but the learning corpus is still orders of magnitude smaller than a “general” language.
Second and third stem from the same origin: it’s tailored for Varnish core testing, and therefore there’s a misalignment with what a user wanting to their VCL logic, or even performance of their setup:
varnishtest is able to do a lot of “weird” things at the protocol level to stress Varnish out: inject HTTP/2 frames, corrupt chunks, etc. It’s great that it exists, but most VCL writers will never worry about this since the protocol is an abstraction.
The other side of that coin is that some things are impossible to do in VTC. For example, servers can’t receive a request, sign it and send the signature back. You’ll need your own server that needs to be awkwardly plugged into the test.
All this isn’t ideal for users who want to just test their VCL and/or have more control over their clients/servers in the tests.
varnishtest-goFrom there, the question becomes: what if we could write tests in a well-known language, with HTTP and testing batteries included, while retaining the power and composability of varnishtest?
That’s how varnishtest-go came to be, about a year ago. As mentioned in the introduction, it was just a toy, built to help a couple of customers automate their tests, and while it’s quite rough and minimal, it ticks the first few boxes:
It’s pure go
You create an net/http/httptest server, controlling exactly what a response should be
Then you create a Varnish object, feed it the URL of your server, and start Varnish
Ask the Varnish server its URL and create an net/http.Client from it
Then run all the tests you want by firing requests and checking the responses using the regular go primitives
We get the full power of go at our fingertips! We can easily use test backends, real ones, run tests in parallel off the same Varnish server, while being able to leverage the whole go ecosystem. It’s quite brilliant.
Have I mentioned that it’s quite rough and minimal, though?
varnishtest-go varnish-goThis is where the “just one more feature” slippery slope reveals itself: varnishtest-go already opened a varnishadm connection to pilot the Varnish instance, but then it just YOLO’d its way through, allowing the API to just send and receive text messages through it. And the feature ideas started rolling in:
How about offering a properly typed API for varnishadm, with idiomatic error handling, and parsing of the responses?
But also, we might want to look at the Varnish counters to verify that the internals are working correctly? For this we need a VSC API.
And what about logs? It would be cool to read the VSL so tests can inspect transactions to debug a case more easily, right?
Syslogs are also important, we should capture and expose those.
Varnish Cache’s and Varnish Enterprise’s APIs are very similar, so we might as well support both, it’s just a couple of special cases…
Now that I think about it, what if…
At this point, we already had the ability to:
Start a Varnish instance with arbitrary parameters, and monitor it
Communicate with the admin socket to read and write parameters, reload VCL, etc.
Read logs
Read metrics
It was time to realize that it wasn’t about replacing varnishtest anymore and that instead it was more sensible to just create varnish-go, a generic Varnish SDK and let the varnishtest bits use that SDK.
First, you can do some fun stuff, check out the example repository for some tools that can visualize your logs, metrics and VCLs in ways that are possible, but quite possibly annoying and error-prone in C.
Second, because it gives users a lot more power to manipulate Varnish. The Varnish ecosystem has always been about providing generic tools to the users, rather than forcing them down a specific path. varnish-go follows the same philosophy, exposing the CLI features through a language that is infinitely more powerful than shell scripting. It doesn’t (and can’t) dictate what you will do with it, but it gives you the tools to do it.
As a teaser, route-builder is based on varnish-go and is pretty much a straight up implementation of this old blog post of mine. It will be used very soon in a project that I’ll be excited to reveal soon™.
And lastly, to go back to LLMs, having a go API opens up the space considerably, because go is a very well spread language and it’s easy for an AI agent to produce decent code in it (warning: make sure to drink vibe-code responsibly). I’m a firm believer that you should understand what you produce, but sometimes, the code quality doesn’t matter much. Here’s a handful of throwaway tools that proved very useful and were one-shot by an AI agent:
Filter faulty backend requests for chunk encoding error, and immediately reproduce the request in go to test them to know if the error was transient or a one-off
Complex filtering of requests where a header and its hash (found in a second header) didn’t match
Rate limiting logging based and variable querying based on Varnish counters
There’s little appeal to making any of these an actual project, because they are just so easy to build on the spot, for exactly the case you need. In the first example, the VSL query was actually hard-coded, instead of going the usual route of CLI arguments when a tool needs to be reusable.
varnish-go is already useful, but the road ahead is wide and goes far. I’ve recently pulled in the VCL parser module from my colleague Per Buer and we have a couple of fun ideas for what to do with it.
If you have ideas of your own, need help or just want to hang out with other Varnish users and builders, join us on this discord and say hi!