Analyzing varnishlog output with Ease: Introducing varnishlog-parser

Varnishlog is a powerful tool that allows you to perform an in-depth analysis of the requests that flow through the subroutines of Varnish, from the initial client request to the backend fetch and the final response. Its output consists of a plain text representation organized by VSL tags (Varnish Shared Memory tags), each with details of the current event.

While varnishlog includes filtering flags and works great with Unix utilities like grep, sometimes you need more than raw text. Sometimes, you want to see the bigger picture: to visualize how requests and transactions interconnect without squinting at endless lines of log data. Also, it's good to leave your terminal sometimes.

Enter varnishlog-parser

varnishlog-parser is both a library and a web interface designed to parse and analyze varnishlog output with ease. Instead of scrutinizing raw logs in the terminal, this tool lets you copy and paste the output into a browser, where you can view transactions in a more organized, visual manner. You can choose to combine all transactions into a single view or split them into multiple browser tabs.

When the logs are parsed, VSL transactions are grouped into "transaction groups" based on parent-child relationships. From there, detailed information about each transaction becomes accessible through a series of tabs:

Overview

This tab presents a sequence diagram segregated by transaction groups. At a glance, you can see key details like response codes, whether it was a cache hit or not, or which backend fulfilled the fetch in case of a miss.

Headers

Working in an environment with a complex Varnish setup often leads to troubleshooting rogue headers. I work in a company that offers a CDN service to its customers, based (of course) on Varnish. Sometimes a request proxied through the CDN fails but works fine when routed directly to the origin, and the manipulation of HTTP headers has been the culprit in many cases.

The Headers tab is your ally in these scenarios. It highlights the state of each header in every transaction, whether it's the original client header or one modified by your VCL code. This makes debugging and tracking header behavior a breeze.

VCL Log Tree

This is one of my favorite features! The VCL Log Tree visually represents varnishlog entries. When encountering a Link VSL tag, it directly renders the child transaction that is triggered, so it’s shown as part of a hierarchy, with the parent transaction resuming after the child completes. This structure reflects the flow of events precisely.

To make things more human-readable, certain tags like timestamps and accounting details are displayed in a friendlier format.

Timestamps

If performance and latency are on your radar, this tab will quickly become your go-to. It enables you to generate charts plotting Timestamp VSL tags, filtering for specific events. You can even add custom std.timestamp("label") entries directly in your VCL to track the timing of specific VCL operations.

Request Builder

Have you ever had the joy of manually extracting the host, URL, and headers from a varnishlog output to construct a curl command? It’s not exactly fun.

The Request Builder tab does the heavy lifting for you. Just select the transaction you're interested in, adjust settings like whether to use the original or final URL and headers, choose the backend to target, and voilà, your curl command is ready to go!

Future Plans

There’s always room for improvement. One idea I’d love to explore is a feature to display a "diff" between transactions, but I’m still mulling over how best to implement it.

If you have suggestions or ideas for new features, feel free to open an issue or submit a pull request on the GitHub repository. I’m all ears! 😊

About the Author

Manuel is a DevOps engineer based in Spain and has been working with Varnish Cache (and Varnish Enterprise) for more than four years. In that time, he has analyzed thousands of varnishlog logs from the customers of the company’s CDN (Transparent Edge). He loves automating tasks and making his job and the job of his colleagues easier, which is why he developed this tool.