Varnish and Microservices

Microservices are great: they isolate concerns, provide modular and reusable components and forces us, developers, to think about proper APIs, leading to better overall designs.

However, the co-evolution of microservices and cloud computing has led at least some devops to forget the forest for the trees. Notably, the ease of deployment and scalability is seen as the end-all, be-all in terms of adaptability to traffic, when in practice it’s only one of the many compromises one can make when trying to scale.

In this blog, instead of just throwing money and hardware at scalability issues, we’ll look at caching, and more specifically when it’s a good idea to use it, what the benefits are and what you need to make it happen.

Reusability

Microservices are often cacheable. Yup, I said it, there’s no walking that back! A lot of people assume their API endpoints are uncacheable when they are in fact “only” dynamic. Let me present a couple of example:

• User bank statement? Cacheable, but you probably need to include the user’s id/cookie into the cache key
• An endpoint that returns the latest build of your app? Very cacheable! Just set a short enough TTL (Time To Live, the duration the object will spend in cache), or invalidate that endpoint’s cache when you build a new version.
• An API that gives you the current time, down to the millisecond, depending on the timezone? CACHEABLE! Just tell Varnish to cache the data until the next millisecond.

• Formulate what makes a request unique (host, path, maybe some cookies?)
• Decide how long that response is going to be valid, or trigger an invalidation when the data becomes stale

Then you have reusability, and therefore cacheability. And most microservices only do two things: they either fetch data (from a database, or from other services), or compute things (based on user input and/or fetched data).

And usually, if your input is the same, your output is the same, so it’s cacheable, at least for a while. All in all, microservices are just services, and we can cache them like any other services. Obviously, it’s easier with Varnish because of how granular and configurable we can be, but the base point still stands.

Caching and composability

Now, I will admit that caching microservices can seem daunting, because of the possible number of services, and of how they can interact with each other. Notably, this won’t work*:

But that’s because it’s flatly applying a “CDN” mindset, where caching is an afterthought and it lies outside of the platform. With that architecture, each response that gets to Varnish will probably be an unholy mix of cacheable and uncacheable data. That’s no good! We can fix it by making caching part of the platform, and caching at a more granular level:

This way, you can finely dictate what’s reusable and what isn’t, speeding fetches internally before they are merged into an uncacheable object. I know what it looks like: I’m adding more complexity, but we are actually dividing, and we’ll be ready to conquer very, very soon, thanks to the next section.

Developing with caching in mind

All blog posts can’t be all fun and games, in this section I need to be a bit preachy. But it’s for your own good, I promise. Here’s the big lesson: turn caching ON when developing!

Developers hate caching, mainly because the caricatural cycle goes like this:

1. Build an app
2. Test locally with a few manual users
3. Publish and notice performance issues
4. Scale up/out until you can serve users properly
5. Receive the infrastructure bill
6. Hastily deploy caching
7. Caching breaks the app because some rule was a bit too broad and caught something it shouldn’t have.

So yeah, of course, if you ignore everything before point 7, it’s ultimately the caching layer’s fault. But this can easily be avoided by making the cache part of the platform. Between docker compose (we have docker images!), kubernetes (we have helm charts!) and hyperscalers (we have AMIs!) it’s very easy to deploy your app AND a tiny varnish node in front of it.

Also, and this is important: you don’t need to write a ton of Varnish configuration every time you change the app! HTTP has provisions for cache-control headers, and by default, Varnish will trust what your app says to do, meaning you can pilot caching directly from your code. How comfortable is that?

Invalidation

Cache invalidation is often offered as a counter-argument to caching. The wording will vary, but the spirit goes something like this: if an object gets invalidated, it needs to be purged, as well as all the responses it was any part of, directly or indirectly.

To which I answer: yessir, absolutely, correctomundo. And it’s not a big task, BUT, one needs to have prepared a tiny bit. The very easy solution here is to ask services to keep track of the resources they use, and to pass that information downstream.

It’s been a long time since we looked at a diagram, let’s have one:

Very simply, every service “producing” content can attach a resource id to a specific header (if the information comes from a database, the object’s primary key is a perfect candidate), and services that mash objects together just merge the headers. This way, tags just “bubble up” towards the edge.

The result is that each response carries this header containing the “ingredients” it was created with. From there, Varnish can use the magic of tag-based cache invalidation: mark a specific id as obsolete and all and only the objects that carry it will be invalidated.

Start small, expand

Like testing, caching is something that is easier to tackle early on, and incrementally. Thankfully, the very nature of microservices make it easy to begin timidly with a couple of services before generalizing the approach.

It’s also pretty easy to know where to start, especially if you have tracing on your platform: look where you are spending the most time fetching or generating reusable content. The answer will vary from setup to setup, of course, but good observability will produce numbers you can rely on, trust them. Contact us to learn more.

* if you correctly identified a reference to KRAZAM’s excellent skit, please get yourself one (1) Internet point, you deserve it.