Top 5 things to keep in mind when migrating from Varnish 3 to 4

Varnish Cache 3.0 (the open source project) was released back in 2011 while Varnish Cache 4.0 made its debut in 2014. Since then, we’ve seen Varnish Cache 4.1.x appear, and now, the most recent, Varnish 5.0, which was only recently released.

Varnish Cache 3.0 has reached end-of-life, and is no longer maintained. That means you should definitely migrate to a more stable, supported, better-performing and feature-complete Varnish Cache version.

Migrations are always time-consuming and potentially problematic, but hopefully this blog post will help you to have a smoother transition.

• Threading model: In Varnish 3.0 a single thread would serve each client, the thread would fetch the content from cache or from the backend.
  In Varnish 4.0 there are two threads: one for the client and one for the backend side.
  This change introduced a lot of improvements, among them the most important of which is the ability to refresh content in the background while serving stale content to the client.

• VCL
  VCL workflow: As a result of the new threading model, the VCL workflow has changed a bit.
  What is important for you as a user is that there are subroutines that are backend thread only:
  • vcl_backend_fetch
  • vcl_backend_response
  • vcl_backend_error
    
    
  VCL naming: Minor changes happened on this side, mostly to be more RFC compliant. More on this in “man vcl”.
  

• Grace behavior: Before Varnish 4.0 all backends would have to be unhealthy before Varnish could use a stale object to fulfill a client request. Starting from Varnish Cache 4.0 there is native support for the stale-while-revalidate behavior, which really means that Varnish, if no fresh content is present in cache, will serve stale content. If it is found, Varnish will deliver an “old” object to the client and trigger an asynchronous backend request to refresh the content.
  Here’s the VCL to use for any Varnish version after 4.0:
  
sub vcl_hit {
  if (obj.ttl >= 0s) {
    # fresh object present in cache
    # normal hit
    return (deliver);
  }

  # We have no fresh objects. Let’s look at the stale ones
  if (std.healthy(req.backend_hint)) {
    # Backend is healthy. Limit age to 10s
    # A background fetch is triggered to fetch a fresh object.
    if (obj.ttl + 10s > 0s) {
      set req.http.grace = "normal(limited)";
      return (deliver);
    } else {
      # No candidate for grace. Fetch a fresh object.
      return(fetch);
    }
  } else {
    # backend is sick - use full grace
    # a background fetch is triggered to try to fetch fresh content
    if (obj.ttl + obj.grace > 0s) {
      set req.http.grace = "full";
      return (deliver);
    } else {
      # no graced object.
      return (fetch);
    }
  }
}


• Directors: In Varnish Cache 3.0, a director was a logical group of backends whose scope was to let Varnish choose a backend server from among several if one was down.
  Starting from Varnish Cache 4.0 to make the backend selection logic easier to extend, the whole concept of directors has been moved to VMOD (Varnish modules). This really gives you the freedom to develop your own logic with regard to the way Varnish handles load balancing.
  If you don’t want to define your own director you can still use one of the default options the directors VMOD gives you: round robin, fallback, random and hashing.
  

  More on this at “man vmod_directors”.

• Purge: In Varnish Cache 3.0 you could use the keyword “purge” to invalidate and flush away the content in your cache. In Varnish Cache 4.0 the keyword “purge” has been retired and it’s now possible to do cache invalidations with purges via VCL:

sub vcl_recv {
  …
  if (req.method == “PURGE”) {
    return(purge);
  }
}

On top of these changes, more new features have been developed in the commercial version of Varnish, between Varnish Cache Plus 3.0 and the newer Varnish Cache Plus 4.x.

Just to name some of them:

• Massive Storage Engine (MSE)
• Persistent MSE
• TLS/SSL support for client and backend connections

There can be plenty of pitfalls in migrating, but hopefully this helps you as you make the important migration from Varnish 3 to Varnish 4 and take advantage of the improvements Varnish 4 offers. 

And, even when you're not migrating, there are some common Varnish mistakes you can avoid while configuring. Watch our webinar, 10 Varnish Cache mistakes and how to avoid them, to find out more.

Photo (c) 2015 Ben Blash used under Creative Commons license.