The most convenient way to terminate SSL/TLS connections in Varnish is by using Hitch. To install Hitch, you can follow the steps in the Hitch repository. If you are a Varnish Plus customer, we recommend that you install it from the Varnish Plus repository as described in the Varnish Book or Varnish Software Customer Guide. Three main sources to answer most of Hitch and PROXY related questions are: Hitch documentation, hitch man page, and vcl man page.

Hitch implements the PROXY protocol, which provides a convenient way to safely transport connection information such as a client's address across multiple layers of NAT or TCP proxies. This protocol allows Hitch to send the client connection information in a HTTP header field to Varnish and the backend. The PROXY header describes which IP address and port was used to connect to the proxy (Hitch), and which IP address and port was connected to. We recommend that you read this blog post which explains in detail how to deal with the PROXY protocol.

Other useful links with documentation about Hitch are:

• Latest Varnish Plus documentation.
• Blog post on how to use Let's Encrypt with Hitch and Varnish in CentOS 7.
• Client TLS/SSL termination documentation.
• Documentation in open source code.

To improve your Varnish Configuration Language (VCL) code, we recommend that you read the blog posts about best practices in Varnish. If your Varnish configuration doesn't behave as expected, there are many steps you can take to verify or find the problem in your VCL code.

The Varnish Massive Storage Engine (MSE) is an enhanced storage method for Varnish Plus. MSE is designed to store and handle over 100TB of data with persistence, which makes it very useful for video streaming. If you want your storage to be persistent, MSE needs a bookkeeping file, which is created with the store file system.

In order to calculate the size of the bookkeeping file in MSE, please follow the size recommendations in the Varnish Plus documentation. We also advise you to take a look at the MSE-specific counters in man varnish-counters, so you can monitor the disk utilization. In addition, you might find the man page of mkfse.mse useful for getting the description of all MSE size-related parameters.

Varnish High Availability (VHA) has a content-replicator agent called vha-agent. vha-agent reads the varnishlog, and for each object insertion detected in server A, it sends an HTTP HEAD request to a Varnish server B. If server B does not have the object, it requests it from server A. As a result, the same object is cached in both servers with only one single backend fetch.

If the URL length of your web resources is very long, you might need to adjust the http_req_hdr_len parameter of varnishd, so that the URLs are not truncated. vsl_reclen is another parameter to consider with very long HTTP header fields. These and other parameters are available in the man pages of varnishd and vha-agent.

As an additional troubleshooting action, we advise you to ensure that you have the latest versions of VHA and Varnish Cache Plus. Instructions for installing and upgrading VHA are available in the VHA installation guide.

Finally, if the issue you are facing persists, please do not forget to run the varnishgather script, upload the output to filebin, and send the link of your upload when you contact Varnish support. All details about varnishgather and its usage are available at https://github.com/varnish/varnishgather.

Varnish Custom Statistics (VCS) is a data stream management system (DSMS) implementation for Varnish. VCS allows you to analyze the traffic from multiple Varnish servers in real-time to compute traffic statistics and detect critical conditions. This is possible by continuously extracting transactions with the vcs-key tags in your VSL. Thus, VCS does not slow down your Varnish servers.

VCS collects traffic information from one or more Varnish servers. A typical problem is due to wrong port configuration or firewall rules blocking traffic on the configured ports. You can check connectivity issues by running the vstatdprobe in the foreground with debugging info:

vstatdprobe -Fg -n <varnish instance name> -p <port vstatd server> <IP address vstatd server>

Details about all possible parameters of vstatdprobe are available in its man page.

Other typical configuration issue arise when VCS is configured to analyze data from many hours. If you are experiencing high memory usage, you can get an explanation by multiplying the value in bucket_len \(\cdot\) the number of vcs-key \(\cdot\) average size of vcs-key in bytes. A common value for bytes is around 12KB. If your calculation confirms the size of bucket_len is the problem, you should reduce it.

As an additional troubleshooting action, we advise you to double check that you have the latest version. For that, please see the VCS installation instructions.

We advise you to use the Varnish Software repository and verify whether you have the most recent Varnish version installed. Instructions on how to configure your repository are available in the Varnish Book or Varnish Software Customer Guide. Then, to verify you have the latest Varnish version in Ubuntu, you can use the commands below to update your Varnish installation.

apt-get update
apt-cache policy varnish-plus

You might also want to read the major changes between versions. To find the files describing changes, you can run commands like the following:

dpkg -L varnish-plus | grep changes

A Varnish module (VMOD) is a shared library that can be called from VCL code. VMODs are an excellent way to modularize (and hopefully share) your code.

At Varnish Software we are working to better document the available VMODs for Varnish Cache and Varnish Plus in Varnish Plus packages. If you encounter a problem with a VMOD packaged in Varnish Plus, you can get help from Varnish support for it. The first recommendation to solve problems with VMODs packaged in Varnish Plus is to make sure you have the most recent Varnish Plus and varnish-plus-vmods-extra packages installed. For installation instructions, please visit https://docs.varnish-software.com/varnish-cache-plus/installation/.

VMODs can be very different from one another. The main advice is to first look at the manual page of the VMOD you are working with. Manual pages have as prefix man vmod_<name>. If documentation is not enough to solve your issues, please run the varnishgather script, upload the output to filebin, and send the link of your upload to Varnish support.

In the Varnish Software lab, we are working on the vcl-migrator project. We will publish a blog post when this is ready. In the meantime, the varnish3to4 script is the main recommendation to syntactically migrate VCL code from Varnish version 3 to version 4.

varnish3to4 is a script that assists you migrating a VCL file from Varnish 3 to 4. The script aims to replace most of the syntactical changes in VCL code from Varnish 3 to Varnish 4, but it is not exhaustive. You can download the script from https://github.com/fgsch/varnish3to4. Usage and up-to-date details about the script is in the same repository.

If your VCL code is complex and the varnish3to4 script is not enough, you can get help from our experts by using the Varnish Professional services.

Health checks and Saint mode are mechanisms in Varnish to check the health of backends or specific resources, and mark them as healthy or sick. Health checks are done with probes. Failing probes may lead to sick backends, and when all backends from a director are sick you may encounter the no backend selected error. To identify when the probes are failing, we advise you to analyze the health probes with the following command:

varnishlog -g raw -q 'vxid == 0' -i Backend_health
varnishlog -d -q FetchError

The format of the result of a backend health probe is described in the manual page of vsl. Another useful command to debug health checks is varnishadm backend.list -p. The manual page of varnish-cli describes this and related commands.

If a backend is sick, serving stale objects is of great help. Stale objects are served using grace mode. An additional source to understand how stale objects are served is our blog on stale while revalidate.

If the commands in this post are not enough to diagnose the backend issues you are facing, please do not forget to add the result of the varnishgather script to Varnish support.

Varnish has different mechanisms to store the cache, namely malloc, file, and mse (Varnish Plus only). If you change the storage configuration, you have to restart Varnish for the changes to take effect. However, the persistence functionality of mse (Massive Storage Engine) allows you to implement an easy work around to effectively expand your storage on-the-fly.

The workaround consists of configuring the parameters of the Varnish daemon varnishd, by way of having two storage backends, the old and the new one. Then, you modify your VCL to instruct Varnish to use both storage backends based on any condition you wish. For example:

sub vcl_backend_response {
  if(condition) {
    set beresp.storage_hint = "MSE_old"; }
  else {
    set beresp.storage_hint = "MSE_new"; }
  }
}

Finally you will have to restart Varnish for changes to take effect. Remember that since you are using MSE with persistence enabled, the cache of the previous configuration is preserved. This workaround allows you to increase or decrease the storage backend without overloading your backend.