Using pipe in Varnish

Pipe was added to Varnish early on to have some way of dealing with unknown behaviour in HTTP. It is pretty simple stuff, really. It turns Varnish into a TCP proxy. Since Varnish uses one thread per connection the thread stops worrying about HTTP and just starts shuffling bytes back and forth. Early on in Varnish development this was very useful when we found some weird bug in Varnish where Varnish would crash whenever it saw something strange, like a content-length header that was out of bounds. In these situations we would just do return(pipe) and let the client and the server talk directly without having Varnish interfering. 

Having the client and server talk directly has a couple of disadvantages. They are:

Varnish has no idea of what is going on.

Since it is just passing bytes back and forth there is no logging of what is going on. This is bad for obvious reasons and a frequently asked question on the #varnish IRC channel. The initial request is logged, sort of. Since Varnish doesn't know when the reponse ends it cannot log the size of the response and such.

Varnish can’t intercept the discussion anymore

Again, since we’re passing bytes back and forth there are no VCL hooks to handle anything. So, if the client decides to ask the server for another request over the same connection Varnish won’t stop it. Since keep-alives is the norm these days care must be taken in order to stop the client and server from keeping the conversation going. 

The recommended way of doing this is to add some VCL code to vcl_pipe:

sub vcl_pipe {
  set bereq.http.connection = "close";
}

Using pipe without it is sort of dangerous as you might have requests being routed to the wrong backend. You choose your backend in vcl_recv and vcl_recv will not be run for any subsequent requests over the same connection. By adding "Connection: close" we ask the backend to close the connection when it has responded to the request.

There is obviously no caching

Since Varnish is shuffling bytes there cannot be any caching.

On the other hand there are a couple of advantages that you should consider

Stable memory footprint

Since pipe only allocates buffers the memory footprint will be minimal, no matter what sort of resources the client requests. Even if the client requests a 4GB ISO Varnish will hardly spend any memory. This is even more relevant if you have some sort of progressive download of a video stream that never ends.

Handling HTTP weirdness

When websockets were launched Varnish was able to support them from day one. We would just look for the relevant HTTP request headers and “upgrade” the connection. VCL code:

sub vcl_pipe {
    if (req.http.upgrade) {
        # copy the upgrade header from the client request to
        # the backend request
        set bereq.http.upgrade = req.http.upgrade;
    }
}
sub vcl_recv {
    if (req.http.Upgrade ~ "(?i)websocket") {
        return (pipe);
    }
}