Cache invalidation

  • Explicit invalidation of cache
  • purge; removes all variants of an object from cache, freeing up memory
  • set req.hash_always_miss = true; can refresh content explicitly
  • ban(); can be used to invalidate objects based on regular expressions, but does not necessarily free up memory any time soon.
  • Which to use when?
  • What about this ban lurker?
  • obj.ttl = 0s; is obsolete.

Whenever you deal with a cache, you will eventually have to deal with the challenge of cache invalidation, or refreshing content. There are many motives behind such a task, and Varnish addresses the problem in several slightly different ways.

Some questions you need to ask whenever the topic of cache invalidation comes up are:

  • Am I invalidating one specific object, or many?
  • Do I need to free up memory, or just replace the content?
  • Does it take a long time to replace the content?
  • Is this a regular task, or a one-off task?

The rest of the chapter will hopefully give you the knowledge you need to know which solution to pick when.

Naming confusion

  • The course material uses Varnish 3-terminology if nothing else is stated.
  • Varnish 3 uses the term ban and banning for what was known as purge and purging in Varnish 2. They are the same.
  • Varnish 3 has a new function called purge; that did not exist in VCL in Varnish 2.
  • purge; is a much improved way of doing what was in Varnish 2 done using set obj.ttl = 0s;.
  • Sorry about the confusion!

With Varnish 3, an attempt was made to clean up some terminology. Unfortunately, this might have made things slightly worse, until people forget everything about Varnish 2.

The function called purge() in Varnish 2 is known as ban() in Varnish 3. This course material will use that terminology, but you are likely to run across material that refers to purge() where you should read it as ban().

On top of that, Varnish 3 introduced the purge; function that’s accessible in vcl_hit and vcl_miss. This replaces the usage of set obj.ttl = 0s;, which was common in Varnish 2, though the latter is still valid in Varnish 3.

All the terms will be discussed in more detail, of course. This is just a heads-up about possible naming confusion.

Removing a single object

  • If you know exactly what to remove, use purge;.
  • It must be used in both vcl_hit and vcl_miss
  • Frees up memory, removes all Vary:-variants of the object.
  • Leaves it to the next client to refresh the content
  • Often combined with return(restart);

The purge; keyword is the simplest manner of removing content from the cache explicitly.

A resource can exist in multiple Vary:-variants. For example you could have a desktop version, a tablet version and a smartphone version of your site and use Vary in combination with device detection to store different variants of the same resource.

If you update your content you can use purge; to evict all variants of that content from the cache. This is done in both vcl_hit and vcl_miss. This is typically done by letting your content management system send a special HTTP request to Varnish. Since the content management system doesn’t necessarily hit a variant of the object that is cached, you have to issue purge; in vcl_miss too. This ensures that all variants of that resource are evicted from cache.

The biggest down-side of using purge; is that you evict the content from cache before you know if Varnish can fetch a new copy from a web server. If the web server is down, Varnish has no old copy of the content.

Example: purge;

acl purgers {
        "127.0.0.1";
        "192.168.0.0"/24;
}

sub vcl_recv {
        if (req.request == "PURGE") {
                if (!client.ip ~ purgers) {
                        error 405 "Method not allowed";
                }
                return (lookup);
        }
}

sub vcl_hit {
        if (req.request == "PURGE") {
                purge;
                error 200 "Purged";
        }
}
sub vcl_miss {
        if (req.request == "PURGE") {
                purge;
                error 404 "Not in cache";
        }
}
sub vcl_pass {
        if (req.request == "PURGE") {
                error 502 "PURGE on a passed object";
        }
}

The PURGE example above is fairly complete and deals with a non-standard method. Using purge; will remove all Vary:-variants of the object, unlike the older method of using obj.ttl = 0s; which had to be issued for each variants of an object.

Note

ACLs have not been explained yet, but will be explained in detail in later chapters.

The lookup that always misses

  • req.hash_always_miss = true; in vcl_recv will cause Varnish to look the object up in cache, but ignore any copy it finds.
  • Useful way to do a controlled refresh of a specific object, for instance if you use a script to refresh some slowly generated content.
  • Depending on Varnish-version, it may leave extra copies in the cache
  • If the server is down, the old content is left untouched

Using return (pass); in vcl_recv, you will always ask a backend for content, but this will never put it into the cache. Using purge; will remove old content, but what if the web server is down?

Using req.has_always_miss = true; tells Varnish to look the content up but, as the name indicates, always miss. This means that Varnish will first hit vcl_miss then (presumably) fetch the content from the web server, run vcl_fetch and (again, presumably) cache the updated copy of the content. If the backend server is down or unresponsive, the current copy of the content is untouched and any client that does not use req.hash_always_miss=true; will keep getting the old content as long as this goes on.

The two use-cases for this is controlling who takes the penalty for waiting around for the updated content, and ensuring that content isn’t evicted until it’s safe.

Warning

Varnish up until 3.0.2 does not do anything to evict old content after you have used req.hash_always_miss to update it. This means that you will have multiple copies of the content in cache. The newest copy will always be used, but if you cache your content for a long period of time, the memory usage will gradually increase.

This is a known bug, and hopefully fixed by the time you read this warning.

Banning

  • Ban on anything
  • Does not free up memory
  • ban req.url ~ "/foo"
  • ban req.http.host ~ "example.com" && obj.http.content-type ~ "text"
  • ban.list
  • In VCL: ban("req.url ~ /foo");

Banning in the context of Varnish refers to adding a ban to the ban-list. It can be done both through the command line interface, and through VCL, and the syntax is almost the same.

A ban is one or more statements in VCL-like syntax that will be tested against objects in the cache when they are looked up in the cache hash. A ban statement might be “the url starts with /sport” or “the object has a Server-header matching lighttpd”.

Each object in the cache always points to an entry on the ban-list. This is the entry that they were last checked against. Whenever Varnish retrieves something from the cache, it checks if the objects pointer to the ban list is point to the top of the list. If it does not point to the top of the list, it will test the object against all new entries on the ban list and, if the object did not match any of them, update the pointer of the ban list.

There are pros and cons to this approach. The most obvious con is that no memory is freed: Objects are only tested once a client asks for them. A second con is that the ban list can get fairly large if there are objects in the cache that are rarely, if ever, accessed. To remedy this, Varnish tries to remove duplicate bans by marking them as “gone” (indicated by a G on the ban list). Gone bans are left on the list because an object is pointing to them, but are never again tested against, as there is a newer ban that superseeds it.

The biggest pro of the ban-list approach is that Varnish can add bans to the ban-list in constant time. Even if you have three million objects in your cache, adding a ban is instantaneous. The load is spread over time as the objects are requested, and they will never need to be tested if they expire first.

Tip

If the cache is completely empty, bans you add will not show up in the ban list. This can often happen when testing your VCL code during debugging.

VCL contexts when adding bans

  • The context is that of the client present when testing, not the client that initiated the request that resulted in the fetch from the backend.
  • In VCL, there is also the context of the client adding the item to the ban list. This is the context used when no quotation marks are present.

ban("req.url == " + req.http.x-url);

  • req.url from the future client that will trigger the test against the object is used.
  • req.http.x-url is the x-url header of the client that puts the ban on the ban list.

One of the typical examples of purging reads ban("req.url == " + req.url), which looks fairly strange. The important thing to remember is that in VCL, you are essentially just creating one big string.

Tip

To avoid confusion in VCL, keep as much as possible within quotation marks, then verify that it works the way you planned by reviewing the ban list through the cli, using ban.list.

Smart bans

  • When Varnish tests bans, any req.*-reference has to come from whatever client triggered the test.
  • A “ban lurker” thread runs in the background to test bans on less accessed objects
  • The ban lurker has no req.*-structure. It has no URL or Hostname.
  • Smart bans are bans that only references obj.*
  • Store the URL and Hostname on the object
  • set beresp.http.x-url = req.url;
  • set beresp.http.x-host = req.http.host;
  • ban obj.http.x-url ~ /something/.*

Varnish now has a ban lurker thread, which will test old objects against bans periodically, without a client. For it to work, your bans can not refer to anything starting with req, as the ban lurker doesn’t have any request data structure.

If you wish to ban on url, it can be a good idea to store the URL to the object, in vcl_fetch:

set beresp.http.x-url = req.url;

Then use that instead of req.url in your bans, in vcl_recv:

ban("obj.http.x-url == " +  req.url);

This will allow Varnish to test the bans against less frequently accessed objects, so they do not linger in your cache just because no client asks for them just to discover they have been banned.

ban() or purge;?

  • Banning is more flexible than purge;, but also slightly more complex
  • Banning can be done from CLI and VCL, while purge; is only possible in VCL.
  • Smart bans require that your VCL stores req.url (or any other fields you intend to ban on) ahead of time, even though banning on req.url directly will still work.
  • Banning is not designed to free up memory, but smart bans using the ban lurker will still do this.

There is rarely a need to pick either bans or purges in Varnish, as you can have both. Some guidelines for selection, though:

  • Any frequent automated or semi-automated cache invalidation will likely require VCL changes for the best effect, be it purge; or setting up smart bans.
  • If you are invalidating more than one item at a time, you will either need a whole list, or need to use bans.
  • If it takes a long time to pull content into Varnish, it’s often a good idea to use req.hash_always_miss to control which client ends up waiting for the new copy. E.g: a script you control.

Exercise: Write a VCL for bans and purges

Write a VCL implementing a PURGE and BAN request method, which issues purge; and ban(); respectively. The ban method should use the request headers req.http.X-Ban-url and req.http.X-Ban-host respectively. The VCL should use smart bans.

Do you get any artifacts from using smart bans, and can you avoid them?

To build further on this, you can also have a REFRESH method that fetches new content, using req.hash_always_miss.

To test this exercise you can use lwp-request. Example commands:

lwp-request -f -m PURGE http://localhost/testpage
lwp-request -f -m BAN -H 'X-Ban-Url: .*html$' -H 'X-Ban-Host: .*\.example\.com' http://localhost/
lwp-request -f -m REFRESH http://localhost/testpage

You may want to add -USsed to those commands to see the request and response headers.

Solution: Write a VCL for bans and purges

sub vcl_recv {
        if (req.request == "PURGE") {
                return (lookup);
        }
        if (req.request == "BAN") {
                ban("obj.http.x-url ~ " + req.http.x-ban-url +
                    " && obj.http.x-host ~ " + req.http.x-ban-host);
                error 200 "Banned";
        }
        if (req.request == "REFRESH") {
                set req.request = "GET";
                set req.hash_always_miss = true;
        }
}

sub vcl_hit {
        if (req.request == "PURGE") {
                purge;
                error 200 "Purged";
        }
}

sub vcl_miss {
        if (req.request == "PURGE") {
                purge;
                error 404 "Not in cache";
        }
}

sub vcl_fetch {
        set beresp.http.x-url = req.url;
        set beresp.http.x-host = req.http.host;
}

sub vcl_deliver {
        unset resp.http.x-url;
        unset resp.http.x-host;
}

Exercise : PURGE an article from the backend

  • Send a PURGE request to Varnish from your backend server after an article is published. The publication part will be simulated.
  • The result should be that the article must be purged in Varnish.

Now you know that purging can be as easy as sending a specific HTTP request In order to help you have access to the file article.php which fakes an article. It is recommended to create a new page called purgearticle.php.

Solution : PURGE an article from the backend

article.php

<?php
header( "Cache-Control: public, must-revalidate, max-age=3600, s-maxage=3600"  );

$date = new DateTime();
$now = $date->format( DateTime::RFC2822 );
?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
    <head></head>
    <body>
        <h1>This is an article, cached for 1 hour</h1>

        <h2>Now is <?php echo $now; ?></h2>
        <a href="<?=$_SERVER['PHP_SELF']?>">Refresh this page</a>
    </body>
</html>

purgearticle.php

<?php
header( 'Content-Type: text/plain' );
header( 'Cache-Control: max-age=0' );
$hostname = 'localhost';
$port     = 80;
$URL      = '/article.php';
$debug    = true;

print "Updating the article in the database ...\n";
purgeURL( $hostname, $port, $URL, $debug );

function purgeURL( $hostname, $port, $purgeURL, $debug )
{
    $finalURL = sprintf(
        "http://%s:%d%s", $hostname, $port, $purgeURL
    );

    print( "Purging ${finalURL}\n" );

    $curlOptionList = array(
        CURLOPT_RETURNTRANSFER    => true,
        CURLOPT_CUSTOMREQUEST     => 'PURGE',
        CURLOPT_HEADER            => true ,
        CURLOPT_NOBODY            => true,
        CURLOPT_URL               => $finalURL,
        CURLOPT_CONNECTTIMEOUT_MS => 2000
    );

    $fd = false;
    if( $debug == true ) {
        print "\n---- Curl debug -----\n";
        $fd = fopen("php://output", 'w+');
        $curlOptionList[CURLOPT_VERBOSE] = true;
        $curlOptionList[CURLOPT_STDERR]  = $fd;
    }

    $curlHandler = curl_init();
    curl_setopt_array( $curlHandler, $curlOptionList );
    curl_exec( $curlHandler );
    curl_close( $curlHandler );
    if( $fd !== false ) {
        fclose( $fd );
    }
}
?>

default.vcl

backend default { .host = "localhost"; .port = "80"; }
acl purgers { "127.0.0.1"; }
sub vcl_recv {
    if (req.request == "PURGE") {
        if (!client.ip ~ purgers) {
            error 405 "Not allowed.";
        }
        return (lookup);
    }
}
sub vcl_hit {
        if (req.request == "PURGE") {
                purge;
                error 200 "Purged.";
        }
}
sub vcl_miss {
        if (req.request == "PURGE") {
                purge;
                error 200 "Purged.";
        }
}