Varnish Web Developer Wiki Highlights: Cache Invalidation in Varnish with Examples

 

At the risk of sounding repetitive, cache invalidation is routinely seen, thanks to a now-immortal statement from Phil Karlton, as one of computer science's two most difficult things. 

And it's true: due to the difficult nature of maintaining real-time, up-to-date cache coherency, cache invalidation is a challenge. It's one we've been working on for just about as long as Varnish has existed.

To delve further into how exactly the different components of cache invalidation work with Varnish, here's an excerpt from the Varnish Wiki on this very subject. 

As usual, I'd like to remind you that the Wiki isn't just ours - it's also yours. I encourage you not only to take a look at it but also to contribute to it based on your own Varnish experience.

Read More

6/22/17 12:44 PM
by Rubén Romero

Varnish Web Developer Wiki Highlights: Magento2 Varnished

A few years ago I wrote the "Doing the right thing: Magento + Varnish Cache" blog post which has been very popular and I have planned to write a new one for a while. I didn't. Instead we added a " Varnish with Magento 2" tutorials section in the Varnish Web Developer Wiki, which includes many resources such as this Step by Step Guide to make Magento .2 and Varnish work like a charm.

It's a wiki, meaning that we hope and expect that you will help us to expand it and keep it up to date. I'm publishing it here in the blog for reference. In this tutorial, we give the step-by-step process that will help you install and configure Varnish to take your Magento2 e-commerce site to the next level. So read on:

Magento is a popular and powerful e-commerce platform for marketing, catalog management and search engine optimization. Magento provides online merchants with a flexible shopping cart system and control over a user-friendly WUI with content and various functionality for online users.

Magento is popular because it's got all this good stuff going for it, but there are some important considerations when implementing a Magento-based e-commerce site. It needs a bit of help in the acceleration department. This is where Varnish comes in; Varnish is a great complement to Magento sites because it adds significant speed and the potential for a real reduction in maintenance costs. 

Magento2 is a PHP-based e-commerce platform. There are two editions:

  • Community edition (CE)
  • Enterprise edition (EE)

Whichever works best for you, you can get started with a performance increase for your website right away!

Let’s get started!

This article assumes that you have a running instance of Magento2 and that you have administrator rights for said instance, both at the OS and application level. We have tested this using Ubuntu LTS 16.04, Varnish Cache 4.1 and Magento2.

If you need guidance on the installation of Magento, please visit the Magento-Site

1. Installing Varnish

In case you also do not have Varnish, you will need to follow the instructional section on how to Install Varnish before we can continue.

2. How to place Magento2 behind Varnish

Here we discuss how to configure your Magento2 behind Varnish. The Magento2 admin states that the built-in application cache is not recommended for production use, but that does not mean that Varnish is already configured. Varnish needs to be installed and the configuration file suitably configured and deployed.

We assume that you have Magento2 installed and running on your backend servers and that your server is a Debian-based Linux server.

Configuring Magento

  1. Visit the Magento2 admin page and go to:
  • > Stores
  • > Advanced
  • > System
  • > Full Page Cache

Here switch caching application to Varnish

This is what the Magento2 mdmin page should look like:

Image courtesy: Magento Site

  1. Configuring for Varnish
  • Under the Additional section, find a button for exporting the ready-made configuration file for Varnish 3 or 4. We recommend you use Varnish 4, as earlier versions are no longer supported.
Click on::
-> Export VCL for Varnish 4 This is usually named varnish.vcl

Place the file in a Varnish folder for configuration (any place that is safe for you).

  • Next, to add Varnish repository, follow the instructions at Install Varnish
  1. Testing your setup

Now check if your services are up and running:

$ ps -e | grep 'apache2|varnish'

If you receive outputs like the ones below:

1143 ?        00:00:03 varnishd
1148 ?        00:00:17 varnishd
1366 ?        00:02:02 varnishlog
1591 ?        00:00:01 apache2
11743 ?       00:00:10 apache2
11744 ?       00:00:10 apache2

Congratulations! You have your services running on the backend.

Now we need to check the Magento2 frontend: As you may have already noticed above, there is a varnishlog process running as well.

3. Restarting services after making changes

sudo systemctl restart varnish.service

sudo systemctl restart apache2.service

4. Basic caching

Varnish does not cache cookies or their headers. But some cookies are marked as safe by the Magento site. Therefore it is recommended to remove or ignore these cookies so that Varnish can cache anything.

An example, such as the following, will help to unset/remove unwanted cookies.

Update your default.vcl with this code or a similar code of your choice.

Under the vcl_recv add the following code.

Warning: Make sure to add the code below the default code given for vcl_recv

sub vcl_recv {
  if (req.http.cookie) {
    set req.http.cookie = ";" + req.http.cookie;
    set req.http.cookie = regsuball(req.http.cookie, "; +", ";");
    set req.http.cookie = regsuball(req.http.cookie, ";(COOKIE1|COOKIE2|COOKIE3)=", "; \1=");
    set req.http.cookie = regsuball(req.http.cookie, ";[^ ][^;]*", "");
    set req.http.cookie = regsuball(req.http.cookie, "^[; ]+|[; ]+$", "");

    if (req.http.cookie == "") {
      remove req.http.cookie;
    }
  }

5. Excluding certain URLs

Not all URLs should be cached. Especially not in sites that deal with personal information such as credit cards or financial details.

if (req.url ~ "magento_admin|magento_login") {

return (pass);

}

6. Extended caching

There is a subroutine called vcl_fetch which is by default set to 120 seconds as can be seen. You can extend this caching value by:

sub vcl_fetch {
       set beresp.ttl = 5s;
}

This sets the TTL to 5 seconds, making Varnish pick up changes every 5 seconds. Add this subroutine right below the backend default.

However, there is a downside to short TTL values in that they increase the load not only on the backend servers but also on frontend servers. Of course this gives you a better control over your cache, but it also increases overheads, such as network traffic, slower response times, which diminish the whole purpose of Varnish.

Decreasing TTL values is not a good solution for high-traffic servers. Varnish has a better solution for that.

7. Specific TTL-based caching

Varnish creates a TTL value for every object in the cache. The most effective way of increasing a website’s hit ratio is to increase the time-to-live (TTL) of the objects.

8. Go further

If you are interested in Varnish, you can always give Varnish Plus a go with a free trial. You can capture real-time traffic statistics, create a paywall for premium content, simultaneously work on administration across all Varnish servers, record relationships between web pages for easy content maintenance, detect devices used for browsing, accelerate APIs and more.

Check out the links below to take your Varnish-Magento site to new performance heights.

9. For more guidance

You can always refer to the Configure and Use Varnish at the Magento site.

To see the guide on installing and configuring Magento with Varnish on web servers, please look at here.

If you are interested in trying out an installation, download Marko’s Vagrant Box marko_magento2github. His installation used nginx with Varnish and Magento. You can also read more about in Marko’s blog post about placing Magento2 behind Varnish marko_magento2post.

Read More

4/4/17 10:49 AM
by Rubén Romero

Varnish Web Developer Wiki Highlights: Drupal with Varnish step-by-step

It will probably not come as a big surprise to anyone that some of the world's most popular and powerful content management systems - as great as they are - can use a little bit of help when it comes to performance. It will also not be surprising to many that the flexibility of Varnish Cache makes it an ideal complement to these CMSes in boosting performance. Last week, we shared the tutorial on using Varnish with WordPress via the Varnish Developer Wiki... and this week, we bring you another tutorial on another major CMS: straight from the Varnish with Drupal section please read the step-by-step guide to make Drupal 8 site fly with Varnish.

It's a wiki, meaning that we hope and expect that you will help us to expand it and keep it up to date. I'm publishing it here in the blog for reference. In this tutorial, we give the step-by-step process that will help you install and configure Varnish to take your Drupal-based site to the next level. So let's get started.

Read More

2/17/17 11:00 AM
by Rubén Romero

Varnish wiki highlights: WordPress with Varnish step-by-step

Important Note

Based on the Step-by-step: Speed Up WordPress with Varnish Software article originally written by Federico G. Schwindt and published on Web Designer Magazine - http://www.webdesignermag.co.uk/ as well as on the Varnish Software blog.

It is now available in the Varnish Web Developer Wiki, specifically under the Varnish with WordPress Tutorials section. As with any other wiki docs you can help us to expand it and keep it up to date. I'm publishing it here in the blog for reference. Read on:

Read More

2/9/17 11:30 AM
by Rubén Romero

Varnish Software Blog

The Varnish blog is where our team writes about all things related to Varnish Cache and Varnish Software...or simply vents.

 

 
 

Posts by Topic

see all
 

SUBSCRIBE TO OUR BLOG