Varnish Custom Statistics: our little xmas make over

Varnish Custom Statistics: our little xmas make over

Varnish Custom Statistics (VCS) now ships with a front end, helping you debug your production backend servers out of the box. With the power of VCL and the wealth of operational data contained in varnishlog, we are extremely excited about the 1.1 launch and the new possibilities the VCS now has to offer.

As the year is drawing towards an end, Varnish Software is bursting with its usual Xmas spirit. Our office filled with gingerbread and Xmas decor whilst our little Varnish helpers busily plug away, working to make the internet a better place.

In the spirit of Xmas,  it is fitting to mention that we are releasing Varnish Custom Statistics (VCS) version 1.1 with a front end makeover (and a brand new name)!* Akin to Santa, we are delivering something awesome for you good boys and girls out there.

VCS now ships with a front end, helping you debug your production backend servers out of the box. With the power of VCL and the wealth of operational data contained in varnishlog, we are extremely excited about the 1.1 launch and the new possibilities the VCS now has to offer.

Some background information

Varnishlog is a shared memory round-robin log that stores a ton of information on all requests flowing in and out of Varnish Cache. VCS is an extremely flexible data gathering tool, built specifically for Varnish, that enables you to customize the statistics output within the constraints of VCL. This is achieved by identifying the precise points of interest in VCL with a simple standard log line, then these outputs are aggregated by VCS. In short, VCS can provide unique, and customisable, statistical views of all requests from VCL variables that you find applicable for your needs.

Some examples of how you can use Varnish Custom Statistics

Varnish Software AS hosts both varnish-software.com and varnish-cache.org. I want to know in run-time, how many requests are distributed between the two domains, how many 500 status codes are served to the clients, and the speed at which this content is served per domain basis. Last but not least, I want to know the most popular content on varnish-software.com and on varnish-cache.org respectively.

Varnishtop aggregates data from the shared memory log. However, it is unable to provide the aforementioned information as it aggregates data from Varnish as a whole, not on per domain basis. Varnishlog is a valid alternative to get the information we need. But to pull them out is quite a pain as one would have to run multiple varnishlog processes to separate data from different domains with loads of different regex for different use cases. Like Rudolf the red nosed reindeer, here comes VCS to the rescue.

Here’s a little sneak peak on the VCS top list:

VCStop 10 listing

Let’s say you want to log on per domain basis, we first customize the VCL by inserting the following simple line of VCL to the vcl_recv method. This will tell VCS to start aggregating based on unique HTTP host header and URL.

    std.log(  "vstatd-key:" + req.http.host + req.url);

That single line will log requests information, status codes, and time to first byte on miss or hit basis. This line also forms the key used in VCS. If you are wondering why time to first byte is important? It is how fast the client receives your content. It is ultimately the true indicator of how your customers perceive your website / application.  

The status codes are also aggregated on per key basis and thus making this an extremely handy tool to debug your production. Hark! The joys of seeing zeros on 5xx status codes.

No 500 errors! Hooray!

As mentioned earlier, you can do matching queries with regular expressions directly in VCS key box by supplying the /match/ keyword.

Regex with the match keyword showing 2xx response codes

Upcoming webinars on Varnish tools

Varnish Santa’s little workshop has indeed been busy. We are planning to do a webinar on the VCS and other Varnish tools in January. Watch this space for more info, get in touch with us if you want more information on VCS, and Varnish Software of course wishes you a merry xmas and happy holidays.

 

* NOTE: VCS was formerly known as vstatd

 

Add comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
Type the characters you see in this picture. (verify using audio)
Type the characters you see in the picture above; if you can't read them, submit the form and a new image will be generated. Not case sensitive.

Comments

Awesome article.