Real-time statistics API with VAC

Varnish Administration Console (VAC) 3.0.0 release is just around the corner! To get you excited about our new release planned for next week, here’s is a little sneak peak at the Real-time Statistics API and UI.

In 3.0.0, VAC is broken up into two major components. Its Management UI, and a RESTful API, with the UI utilising the API. There are a diverse number of features supported by the API, for example, change management, statistics and etc. This blog post aims to showcase the Real-time Statistics API.

Varnishstat, that ships with Varnish, is an extremely useful open source tool for gauging the health of a Varnish instance. It is a command-line tool and displays current and lifetime values for all the internal counters in Varnish.

The power of Varnishstat is greatly extended with VAC via its in-memory round-robin database (RRD) exposed as a queriable JSON API. We call it the Real-time Statistics API.

Easily configurable real-time stats

With the help of varnish-agent, a snapshot of Varnishstat is pushed to VAC and aggregated in memory. The end result is a time series for each key counter for each Varnish instance. Therefore with the upcoming release of VAC, you are no longer are you restricted to statistics for a single point in time, or a single aggregated average for the entire lifetime. Out of the box, these time series are real-time and up to two hours worth of queriable data. Naturally, the counters of interest, and the sampling resolution and window are completely configurable within VAC.

VAC users will ascertain the health of their Varnish installation in real-time through the Realtime Statistics API. This is further enhanced with the management user interface (UI) in VAC. As an example, the bad things graph is a real-time graph that utilises the API and renders the rate of change based on counters that are indicative of an undesirable behaviour.

The Real-time Statistic API supports queries based on time range in seconds past epoch, and number of samples based on current time. The API also supports the usual consolidation function, average, min, max, last, first, and total. VAC is a natural datasource for providing statistics to any existing monitoring tool.

Note that seconds past epoch is easily ascertained with the following command.

	$> date +%s

Querying a single counter for a group of Varnish

The cache_hit counter is used for the following example.

Assuming 1 second resolution per sample, the following query illustrates how to obtain a time series based on Epoch time range of 1383549551 to 1383549553 using average consolidation function.

	$> curl -u user:passwd 
		http://vac/api/v1/group/group_id/stats/cache_hit/1383549551/1383549553/1/average

The same query can be performed with simply specifying samples and a resolution. Assuming the current time is 1383549551, the query indicates 1 second resolution per sample, for 3 samples using average consolidation function.

	$> curl -u user:passwd http://vac/api/v1/group/group_id/stats/cache_hit/1/3/average

An example of output data would be

	{
		"nodes": [ {
	        "id": "52769fe7b2da640380243e4c",
	        "name": "varnish1",
	        "stats": {
	            "counter": "cache_hit",
	            "values": [
	                {
	                    "time": 1383549551,
	                    "value": 1449
	                },
	                {
	                    "time": 1383549552,
	                    "value": 1363
	                },
	                {
	                    "time": 1383549553,
	                    "value": 1416
	                }
	            ]
	        }
	    } 
	] }

The aforementioned example illustrates how the cache_hit counter can be queried via Epoch and samples and resolution. The cache_hit counter is indicative of how many cache hits has occurred per second.

Configuring key counters in VAC

Key counters and its resolution are synonymous to data sources and archives with rrdtools, the industry standard tool for high performance data logging and graphing of time series data. These data sources and its archives can be configured in an XML file. Note that VAC does not utilise rrdtool for storage but its own in-memory RDD component to mitigate high levels of disk IO and thus providing the high performance needed for real-time data query and storage.