Return to Blog
March 25, 2014
4 min read time

VCL change management and contiguous integration with your Varnish using VAC API and Git

Yves Hwang

 

Managing multiple Varnish instances and their respective VCL is made significantly easier with Varnish Administration Console (VAC) through its API. This blog post aims to illustrate an example of VCL change management and continuous integration with multiple Varnish instances using VAC API and a little magic from Git.

Use cases (benefits)

  • Auto deploy VCL changes to a number of Varnish servers
  • Version track VCL changes
  • Rollback failed VCL changes
  • Maintain and ensure the correct VCL is used even if Varnish child process restarts

Precursor

VAC manages multiple Varnish servers, particularly medium to large setups. Therefore the notion of a group of Varnish servers is first class citizens in VAC. Varnish servers belonging to the same group are assumed to have identical VCL and parameter changesets. This is known as Martin’s Law.

VCL management in VAC is inspired by the branch/commit model used in Git. In the context of VAC, a single group can be associated with a single active VCL branch. The branch allows for tracking VCL changes down to timestamp and the user who made the change. The head of the branch, which contains the latest VCL content, is deployed to the Varnish servers. Therefore rolling back VCL consists of pointing the head to another commit on the same branch, and deploying the content to the servers. This means that integrating VAC into a Git workflow is seamless.

VAC models aspects of Varnish in JSON and each JSON model has an associated ObjectId as an unique identifier. This is permeated throughout the API. (Refer to the API documentation for more information on the JSON schema and example models.)

Configuration

The following configuration item will reside a file called .vacrc  within the Git repo.
  • A group of Varnish servers, known as the “Production Group”, with the following ID 521c5a464e2ceea0eb18334c
  • An active VCL branch associated to the aforementioned group, with the following ID 52849728051e0430abe8de11
  • A simple repo containing a VCL file called main.vcl
  • VAC location and credential
## content of .vacrc ##
GROUP_ID 521c5a464e2ceea0eb18334c
VCL_ID 52849728051e0430abe8de11
USER_PASSWD vac:vac
VAC_LOCATION http://localhost:8080
FILE_NAME main.vcl

Git hooks

The update-hook can be used just prior to updating the ref. This will ensure that the new VCL pushed to the repo can be verified and deployed to the Varnish servers via VAC.

Workflow

  1. New VCL is pushed to the Git repo
  2. All VCL is verified and saved to the appropriate VCL branch in VAC
  3. New VCL is deployed to the appropriate group in VAC
  4. Any failures, rollback to previous VCL, and discard commit in Git

Putting it all together

By default, VAC will ensure that the managed Varnish servers will always load the intended VCL even if the child process restarts. This feature can be turned off in the VAC configuration file. 
#!/bin/bash

# git hook file
newrev="$3"
tree_sha1=$(git cat-file -p "$newrev" | awk '$1 == "tree" { print $2 }')
settings_sha1=$(git cat-file -p "$tree_sha1" | awk '$4 == ".vacrc" { print $3 }')
GROUP_ID="$(git cat-file -p "$settings_sha1" | awk '$1 == "GROUP_ID" { print $2}')"
VCL_ID="$(git cat-file -p "$settings_sha1" | awk '$1 == "VCL_ID" { print $2}')"
USER_PASSWD="$(git cat-file -p "$settings_sha1" | awk '$1 == "USER_PASSWD" { print $2}')"
VAC_LOCATION="$(git cat-file -p "$settings_sha1" | awk '$1 == "VAC_LOCATION" { print $2}')"
FILE_NAME="$(git cat-file -p "$settings_sha1" | awk '$1 == "FILE_NAME" { print $2}')"

PREVIOUS_HEAD=$(curl -u $USER_PASSWD $VAC_LOCATION/api/v1/vcl/$VCL_ID/head | python -mjson.tool  | grep \$oid | awk -F \" '{ print $4 }' )

echo VCL branch id is: $VCL_ID
echo Group id is:      $GROUP_ID
echo Previous head is: $PREVIOUS_HEAD

echo Save and verify VCL for compile or syntax error against the group
SAVE_STATUS=$(curl -u $USER_PASSWD -H "Content-Type: text/plain" -X POST —data-binary "@$FILE_NAME" $VAC_LOCATION/api/v1/vcl/$VCL_ID/push -s -w "%{http_code}" -o /dev/null )

# if the save status is 200, then the VCL is uploaded ok and saved on all varnish cache, ready for deployment
# 400 status code means there are syntax error (see the payload for more details)

rollback() {
    echo "Rolling back to last known state"
    ROLLBACK_STATUS=$(curl -u $USER_PASSWD -H "Content-Type: text/plain" -X POST $VAC_LOCATION/api/v1/vcl/$VCL_ID/push/$PREVIOUS_HEAD -s -w "%{http_code}" -o /dev/null )
    echo $ROLLBACK_STATUS 
    echo Rollback complete
    exit 1
}

if [ $SAVE_STATUS -eq 200 ]
    then 
        echo Verified. Now deploying VCL...
        DEPLOY_STATUS=$(curl -u $USER_PASSWD -X PUT $VAC_LOCATION/api/v1/group/$GROUP_ID/vcl/$VCL_ID/deploy -s -w "%{http_code}" -o /dev/null )
        if [ $DEPLOY_STATUS -eq 200 ]
            then
                echo Deployed.
                exit 0 
            else
                echo Deployment unsuccessful. Rolling back...
                rollback
        fi
    else
        echo Roll back to last known state
        rollback
fi

Summary

This blog post demonstrated the steps required for a git-update hook with VAC API. The example provided was designed to address VCL change management and continuous integration for anybody that needs to manage multiple Varnish instances. The VAC API is a flexible platform for managing groups of Varnish instances, and together with Git, this integrated solution delivers tracking VCL changes, auto deployment of VCL, rollback of VCL and maintaining that the VCL stays consistent in an event of Varnish restart.

Want to take a peek at our Varnish Administration Console? Sign up for the VAC demo.

Try the VAC demo now