Porting your Fastly VCL to Varnish

Fastly VCL is a domain-specific programming language which has evolved from the Varnish proxy cache, which is part of Fastly's platform architecture. Varnish Enterprise VCL, instead, is specific to the Varnish Software solutions. There are some architectural similarities between Varnish Cache open source, Fastly and Varnish Enterprise, but as it is said, the devil’s in the details, hence this blog post.

This is a general guide to porting VCL (Varnish Configuration Language) from your Fastly CDN service to a Varnish Enterprise setup. In general, much of the VCL used on the Fastly platform can directly port over to Varnish but there are a few caveats and differences to consider.

As a general rule, the ‘whatever’ caching logic defined via Fastly, e.g. tenant-specific origin shield, user throttling and origin server selection, can be implemented with Varnish Enterprise too. While the logic remains the same, the syntax between Fastly and Varnish Enterprise is sometimes different. Below are some considerations:

URLs and query strings

The req.url variable contains the URL (path and query) being requested by the client, and is copied into bereq.url when making a request to a backend. This statement is equally true for both Fastly and Varnish Enterprise.

In Fastly, the path and query can be separately accessed using two other different variables , while with Varnish Enterprise you can manipulate and access the different variables of a URL using vmod_urlplus. With this Varnish module, not only can you access the path and the query of each URL, but you can also use it to define more complex logics, e.g. normalize different URLs which will lead to improving your cache hit ratio with better caching storage management.

Cookies

Cookie manipulation is possibly one of the most important, valuable and risky operations within a caching setup. With Fastly you can access a named cookie using subfield accessor syntax. Often this is usefully combined with a regular expression match to extract parts of a structured cookie value. 

For example you could use the following VCL to isolate the cookie, and then extract the various parts of it into distinct HTTP headers:

sub vcl_recv { ... }

Fastly VCL


if (req.http.cookie:auth ~ "^([0-9a-f]+).(\d+).([\w-]+)$") {
  set req.http.Auth-SessionID = re.group.1;
  set req.http.Auth-CreditCount = re.group.2;
  set req.http.Auth-DisplayName = re.group.3;
 }


With Varnish Enterprise such an operations is simplified via the use of vmod_cookieplus and the above VCL becomes similar to the following:
        vcl 4.0;
        import cookieplus;
        sub vcl_recv
        {
       
         // Keep these Cookies
         cookieplus.keep("JSESSIONID");
         cookieplus.keep_regex("^BNES_SSESS");
         cookieplus.keep_regex("^SSESS[a-zA-Z0-9]+$");
         cookieplus.keep_regex("^SESS[a-zA-Z0-9]+$");

         // Any Cookie that is not kept will be removed
         cookieplus.write();
         }

The same VMOD can be used for Set-Cookie manipulations.

For example, with Fastly you can set “Set-Cookie” header using the same lines of VCL

sub vcl_deliver { ... }

Fastly VCL

add resp.http.set-cookie = "auth=52b93cff.165826435.d783dad8-ebb9-4475-b6fb-68ce83f90f12; max-age=86400; path=/";

set resp.http.cache-control = "private, no-store";

If you replace the command “add” with a “set” suddenly the VCL becomes perfect for a Varnish Enterprise installation too.

sub vcl_deliver { ... }

Varnish VCL

set resp.http.set-cookie = "auth=52b93cff.165826435.d783dad8-ebb9-4475-b6fb-68ce83f90f12; max-age=86400; path=/";

set resp.http.cache-control = "private, no-store";

Not only that, but you may decide to combine vmod_cookieplus with vmod_digest, which can compute the digest/hash of a specific Set-Cookie header.

Controlling the cache

Both Fastly and Varnish Enterprise honor, by default, HTTP headers as sent by origin responses, such as Cache-Control, Last-Modified, and Expires. With Fastly, you can override this behavior using VCL in vcl_fetch, by setting the values of beresp.ttl, beresp.stale_while_revalidate, and beresp.stale_if_error.

sub vcl_fetch { ... }

Fastly VCL


set beresp.ttl = 30m;


With Varnish Enterprise you can also override HTTP headers. This operation, unlike Fastly, must be done in vcl_bakend_response which is the VCL subroutine triggered by an origin response, whereas in Fastly the same operation gets done when a new backend request gets issued by Fastly, independent of the response status or reason sent by the origin server.

With Varnish Enterprise the above VCL becomes:

sub vcl_backend_response {

set beresp.ttl = 30m;

}

Furthermore, with Varnish Enterprise, you can also modify freshness attributes of assets already stored in cache. This is achieved using vmod_stale, which is particularly handy if you need to implement stale-while-revalidate or stale-if-error logics.

These are just a few examples of how existing Fastly VCL can be quickly ported to Varnish Enterprise VCL. We have many customers take this approach to reduce their Fastly costs, add more custom logic at the Edge or add a cost-effective failover solution so they stay online when Fastly goes down.