VCL functions

  • Start off with a cheat-sheet for variables
  • Go through the remaining vcl functions: hash, pipe, miss, pass, hit, error and deliver.
  • Add some “features” with VCL.

The following chapter will cover the parts of VCL where you typically venture to customize the behavior of Varnish and not define caching policy.

These functions can be used to add custom headers, change the appearance of the Varnish error message, add HTTP redirect features in Varnish, purge content and define what parts of a cached object is unique.

After this chapter, you should know what all the VCL functions can be used for, and you’ll be ready to dive into more advanced features of Varnish and VCL.

Variable availability in VCL

Variable recv fetch pass miss hit error deliver pipe hash
req.* R/W R/W R/W R/W R/W R/W R/W R/W R/W
bereq.*   R/W R/W R/W       R/W  
obj.hits         R   R    
obj.ttl         R/W R/W      
obj.grace         R/W        
obj.*         R R/W      
beresp.*   R/W              
resp.*           R/W R/W    

The above is a map of the most important variables and where you can read (R) from them and write (W) to them.

Some variables are left out: client.* and server.* are by and large accessible everywhere, as is the now variable.

Remember that changes made to beresp are stored in obj afterwards. And the resp.* variables are copies of what’s about to be returned - possibly of obj. A change to beresp will, in other words, affect future obj.* and resp.* variables. Similar semantics apply to req.* and bereq.*. bereq.* is the “backend request” as created from the original request. It may differ slightly - Varnish can convert HEAD requests to GET for example.

Note

Many of these variables will be self-explaining during while you’re working through these exercises, but we’ll explain all of them towards the end of the chapter to make sure there’s no confusion.

VCL - vcl_hash

  • Defines what is unique about a request.
  • Executed directly after vcl_recv
sub vcl_hash {
    hash_data(req.url);
    if (req.http.host) {
        hash_data(req.http.host);
    } else {
        hash_data(server.ip);
    }
    return (hash);
}

vcl_hash defines the hash key to be used for a cached object. Or in other words: What separates one cached object from the next.

One usage of vcl_hash could be to add a user-name in the cache hash to cache user-specific data. However, be warned that caching user-data should only be done cautiously. A better alternative might be to cache differently based on whether a user is logged in or not, but not necessarily what user it is.

The default VCL for vcl_hash adds the hostname (or IP) and the URL to the cache hash.

Note

The handling of the Vary-header is separate from the cache hash.

VCL - vcl_hit

  • Right after an object has been found (hit) in the cache
  • You can change the TTL or issue purge;
  • Often used to throw out an old object
sub vcl_hit {
    return (deliver);
}

VCL - vcl_miss

  • Right after an object was looked up and not found in cache
  • Mostly used to issue purge;
  • Can also be used to modify backend request headers
sub vcl_miss {
    return (fetch);
}

The subroutines vcl_hit and vcl_miss are closely related. It’s rare that you can use them, and when you do, it’s typically related to internal Varnish tricks - not debug-feedback or backend modification.

One example is using purge; to invalidate an object (more on this later), another is to rewrite a backend request when you want the ESI fragments to get the unmodified data.

You can also modify the backend request headers in vcl_miss. This is very uncommon, as you’ve likely done this in vcl_recv already. However, if you do not wish to send an X-Varnish header to the backend server, you need to remove it in in vcl_miss and vcl_pass using unset bereq.http.x-varnish;.

VCl - vcl_pass

  • Run after a pass in vcl_recv OR after a lookup that returned a hitpass
  • Not run after vcl_fetch.
sub vcl_pass {
    return (pass);
}

The vcl_pass function belongs in the same group as vcl_hit and vcl_miss. It is run right after either a cache lookup or vcl_recv determined that this isn’t a cached item and it’s not going to be cached.

The usefulness of vcl_pass is limited, but it typically serves as an important catch-all for features you’ve implemented in vcl_hit and vcl_miss. The prime example is the PURGE method, where you want to avoid sending a PURGE request to a backend.

VCL - vcl_deliver

  • Common last exit point for all (except vcl_pipe) code paths
  • Often used to add and remove debug-headers
sub vcl_deliver {
    return (deliver);
}

While the vcl_deliver function is simple, it is also very useful for modifying the output of Varnish. If you need to remove a header, or add one that isn’t supposed to be stored in the cache, vcl_deliver is the place to do it.

The main building blocks of vcl_deliver are:

resp.http.*
Headers that will be sent to the client. They can be set and unset.
resp.status
The status code (200, 404, 503, etc).
resp.response
The response message (“OK”, “File not found”, “Service Unavailable”).
obj.hits
The number of hits a cached object has made. This can be evaluated and sued as a string to easily reveal if a request was a cache hit or miss.
req.restarts
The number of restarts issued in VCL - 0 if none were made.

VCL - vcl_error

  • Used to generate content from within Varnish, without talking to a web server
  • Error messages go here by default
  • Other use cases: Redirecting users (301/302 Redirects)
sub vcl_error {
    set obj.http.Content-Type = "text/html; charset=utf-8";
    set obj.http.Retry-After = "5";
    synthetic {"
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
  <head>
    <title>"} + obj.status + " " + obj.response + {"</title>
  </head>
  <body>
    <h1>Error "} + obj.status + " " + obj.response + {"</h1>
    <p>"} + obj.response + {"</p>
    <h3>Guru Meditation:</h3>
    <p>XID: "} + req.xid + {"</p>
    <hr>
    <p>Varnish cache server</p>
  </body>
</html>
"};
    return (deliver);
}

Note

Note how you can use {” and “} to make multi-line strings. This is not limited to synthetic, but can be used anywhere.

Example: Redirecting users with vcl_error

sub vcl_recv {
        if (req.http.host == "www.example.com") {
                set req.http.Location = "http://example.com" + req.url;
                error 750 "Permanently moved";
        }
}

sub vcl_error {
        if (obj.status == 750) {
                set obj.http.location = req.http.Location;
                set obj.status = 301;
                return (deliver);
        }
}

Redirecting with VCL is fairly easy - and fast. If you know a pattern, it’s even easier.

Basic redirects in HTTP work by having the server return either 301 “Permanently moved” or 302 “Found”, with a Location header telling the web browser where to look. The 301 can affect how browser prioritize history and how search engines treat the content. 302 are more temporary and will not affect search engines as greatly.

The above example illustrates how you can use Varnish to generate meta-content.

Exercise: Modify the error message and headers

  • Make the default error message more friendly.
  • Add a header saying either HIT or MISS
  • “Rename” the Age header to X-Age.

Solution: Modify the error message and headers

sub vcl_error {
        synthetic "<html><body><!-- Blank page must mean it's a browser issue! --></body></html>";
        set obj.status = 200;
        return (deliver);
}

sub vcl_deliver {
        set resp.http.X-Age = resp.http.Age;
        unset resp.http.Age;

        if (obj.hits > 0) {
                set resp.http.X-Cache = "HIT";
        } else {
                set resp.http.X-Cache = "MISS";
        }
}

The solution doesn’t use the “long-string” syntax that the default uses, but regular strings. This is all up to you.

Note

It’s safer to make sure a variable has a sensible value before using it to make a string. That’s why it’s better to always check that obj.hits > 0 (and not just != 0) before you try using it. While there are no known bugs with obj.hits at the moment, string-conversion has been an area where there have been some bugs in the past when the variable to be converted had an unexpected value (for example if you tried using obj.hits after a pass in vcl_recv).

This applies to all variables - and all languages for that matter