Doing the right thing: Magento + Varnish Cache

Doing the right thing: Magento + Varnish Cache

Here is a series of posts highlighting Varnish Cache and its usage in the real world, solving real problems.

HTTP is the universal transport protocol and the only universally deployed API accross devices, clients, servers running all kind of operating systems. Varnish Cache being the HTTP "Swiss Army Knife" or the HTTP "Husqvarna Extreme Logging Chainsaw" if you will, can potentially be used to solve lots of different problems. At Varnish Software I am in contact on a daily basis with people using Varnish to do things we had not even thought of after all these years of making and providing services for the software. With Varnish you can compress, cut&paste, upgrade, secure, add performance and speed up your website's content delivery, and a whole lot of other things that I will blog about the coming months.

Today i will have a take on Magento, a fast growing Open Source E-commerce platform which is known for being flexible, but not so much for being fast. Enter:

Since my last post on this subject, many things have happened in the Magento world, with the main highlight being that Magento Inc was bought by Ebay, which was not really surprising as they made an investment in the company already in early 2010. Still, this means that the Magento Platform has a rather interesting future as a crucial piece for eBay's success (as are PayPal and X.Commerce). So without any further ado:

A 250x Faster Magento is possible!

This is what is up in the interwebs regarding Varnish and Magento! And I see two main trends:

  • A lot of people doing great work with the Magento and Varnish Cache combo
  • Lots of PHP code and VCL sauce lying around in blogs, presentations and different websites

As I will focus on the people doing great work in this post, I want first to give a piece of advice to those relying on random code and configuration files:

It is better to stand on the shoulders of giants, right?

Your (or your customer's) e-commerce site is the last possible place on the web where you should be testing random VCL snippets and unknown PHP code on. It is easy to write a how-to but it is hard to maintain it, specially as software develops and things change. People create and maintain software for a reason, so when doing performance related changes to your website, think twice! Be careful not to burn your revenue stream.

If you still want to go this road I have two choices for you:

Re-invent the wheel: Do-It-Yourself®

If you have time and no deadlines: DIY Magento+Varnish is the solution for you. Learning how to use Magento, understand HTTP Headers and get a hold of reverse proxy caching and Varnish Cache specifically will take you some time. For you I have one piece of advice: “There are only two hard things in Computer Science: cache invalidation and naming things.” quote originally from Philip Lewis Karlton. Good luck!

Use the interwebs and blogs

As I stated above: Unmaintained and outdated information applied wrongly is self-harming. Once that is said, there are lots of nice written blogs on how to architect and deploy a website running Magento and accelerate it with Varnish Cache. As usual, google is your friend here. Good luck to you too!

Let's stand on each other's shoulders!

Some people take pride in their job and take it serious. People that do so often achieve what is described as success. Putting your eggs in the right basket is what you have to do. Doing the right thing means choosing the right plugin for you! So, instead of doing things all by yourself, try and use someone else's software and help them make it better if you want. To the table I bring to you (in chronological order of development):

  • Pagecache powered by Varnish - Phoenix Media
    • Description:
      This plug-in was made in the beggining of 2011. It is architecturally approved by Varnish Software as all Varnish related configurations and settings have been reviewed by the core team in close cooperation with Phoenix Media. The bundled VCL allows it to run in one or multi server environments with Magento and to purge Varnish’s cache either on demand or event based (e.g. when a product gets out of stock or a CMS page is changed) right from the Magento backend, which is pretty cool. Also, the PageCache module supports Magento design exceptions to cache different versions of the same shop page per device (think browser and iphone). Support for ESI coming in Q1 2012. Supports Varnish 2.1.x and 3.0.x
    • Pricing and Availability:
      Two editions: Free Community Edition & Enterprise edition for €1,000.- with additional advanced features and support. Works with Magento CE > 1.4 & PE, EE > 1.7
      Developed and supported by our Integration Partner and Magento Gold Partner Phoenix Media with a proven track record of making Magento extensions. Available on Magento Connect.
  • Magneto-Varnish - Madalin Oprea
    • Description:
      Started in May 2011. It features a basic administration interface to add your server IP but no caching granularity or any additional control. Supports Varnish 2.1.x and additional 3.0.x support was added recently.
    • Pricing and Availability:
      Free. Not sure about which Magento editions and versions it supports.
      Maintained by Madalin Oprea and lives on Github.
  • AOE StaticFabrizio Branca
    • Description:
      Started in May 2011. It has some nice features as it makes caching just work. It is more like like a proof-of-concept but handles also session-users through nifty tricks (javascript and such). For real sites it has some drawbacks as no management interface is provided and thus no caching granularity to speak of here either, but it is a good start.
    • Pricing and Availability:
      Free. Works under Magento CE 1.5 & 1.6 and Magento EE 1.10 & 1.11. Might work in newer versions as well.
      Maintained by Fabrizio Branca and lives on Github.
  • Cache Extension powered by VarnishcreativeStyle
    • Description:
      As far as I could see this module contains code from blog posts around the web. Although it aims high, it is not very sophisticated and lacks documentation. VCL editing in your browser without checking the configuration first could harm your installation badly. It has a forum of users where you can ask and probably get help.
    • Pricing and Availability:
      Free. Works on Magento Community edition only, not sure which versions it supports.
      Maintained by creativeStyle, company that as far as I could see has no record on any Magento extensions. Available on Magento Connect.

As you see there are many solutions you can use, have a look and decide for yourself. My pick:

The most feature rich solution is the Pagecache powered by Varnish, by Phoenix Media. Also I believe this to be the best case of professional Magento Varnish acceleration as it combines support while staying community friendly, very much as Magento Commerce itself. It comes in an open source version and a commercially supported version for Magento users wanting even more features and support for a critical piece of software. Also they have a good track record of making Magento plugins, so you can feel safe that they are not experimenting while aiding you to add better performance to your webshop.

Want to use Varnish beyond Magento?

Now that you have discover the power of Varnish Cache, why not deploy it on a cluster and serve all your websites, not only Magento, with it?

For that Varnish Software offers Enterprise Subscription services which include our professional support from core developers, Open Source assurance, IP infringement protection and specialized key tools, such as the Varnish Administration Console for management and real-time monitoring of your group(s) of Varnish Cache servers, created by the makers of the software. Read more on our subscription levels and price plan, and once you are ready email me or send us a request and I will gladly help you figuring out the best solution for you and your e-commerce site.

Add comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
Type the characters you see in this picture. (verify using audio)
Type the characters you see in the picture above; if you can't read them, submit the form and a new image will be generated. Not case sensitive.

Comments

I am new to magento and no experience with varnish.

 
Do i have to install Varnish on the server before using this extension. If yes, please give step-by-step instructions.  

Thanks for your attention.

We do acceleration consulting for Varnish and other acceleration software on the web. With that in mind:

One piece of advice to Anonymous below:

Get into Magento first, see whether you like it or not.

Varnish is a highly sophisticated (and incredibly versatile) piece of software, which, even using one of these plugins, you would probably have to tune yourself. That's right - editing config files, learning VCL (Varnish Configuration Language) and more. The setup will be different for each usage, thus no step-by-step instructions can be given.

Consider using external acceleration services (hosted acceleration), or maybe a CDN for your static files, first.

Varnish is definitely very sweet, but you have to be patient with it & to invest some time and love.

Max

Thanks for helping Magento run faster.

Hi,

I started yesterday with testing this setup. It works very fast, so thank you very mutch. Ik have only one problem (at te moment ;-)). When I log In to Magento and log out agen. The NO_CACHE cookie issn't removed. How can I fix this?

Best regards,

Otto Smittenaar

Hi there, 

I recently released yet another Magento+Varnish extension.

This one however, contrary to the extensions in your article, uses ESI Includes and allows the caching of pages containing Session Based data (like the cart). This means that even if your customer is logged in the pages he visits are still loaded from the cache.

You can find the extension here: https://github.com/huguesalary/Magento-Varnish

It still has some bugs and may not be fully suitable for production, but it works.

It is currently used on www.betabrand.com

-Hugues

Rubén Romero's picture

Thanks for the tip Hugues.

Haven't had a look at the extensions since this post. It looks promising although a bit complex for some, since it adds redis to the mix.

Is it very much tailored towards your site or could it be applied elsewhere as it is?

-Rubén.

I stumbled upon your billet, which is very good but should be updated.

You should add Nexcess Turpentine http://www.magentocommerce.com/magento-connect/turpentine-6332.html.
Based on Hugues Alary ESI code, "Full Page Caching, with hole-punching via Varnish ESI and/or AJAX, even for logged-in visitors"

Hi Ruben,

I just saw your message.

The extension is not specific to my website. I have got multiple reports from other developers who installed it successfully.

-Hugues.

i would like to mention another extension https://github.com/nexcess/magento-turpentine what seems to be most recent and actively developed

also any comments on the issues with ESI listed here?: http://www.fabrizio-branca.de/magento-varnish-ajax-vs-esi.html