Paywall challenges

When implementing a Paywall in an organization there are a lots of changes needed. There are editorial changes, marketing aspects, tech support and much more. I can’t say I know much about the editorial changes a newspaper must go through when implementing a paid-content strategy, but we’ve been part of quite a few technical teams setting up the logic for paid-content and we’ve learned a few lessons I’d like to share.

Payment is easy

When planning paywalls quite a few newspaper fear this. None of the ones that we’ve talked to have had any problems implementing payments. It’s become a very well understood commodity and all the kinks have been worked out by the e-commerce market. Do not fear this and do not give away big chunks of your revenue to have someone handle it.

Access control is harder

Implementing access control is considered a much more difficult problem. There are three ways to solve it:

1. Clientside, using Javascript. Technically this is a bit awkward as you don’t really want to trust the client to hide the content if the Javascript decides the user doesn’t have access. However, since we’re not talking about building Fort Knox, it may suffice for many publications.
2. Using a CDN with ESI. We’ve seen paywalls using ESI on CDNs. Basically you set a cookie and then you use the ESI processing to either include the full version of the page or the preview depending on whether or not the user has access to the content. There are potential lock-in effects here and the value of pushing the main content through a CDN might not always be obvious. For most sites with text content the main content is only a very small fraction of the payload so your own infrastructure is probably more than capable of handling the load. Furthermore, many CDNs can introduce long latencies for cache invalidations which can be painful if you are dependant on serving up-to-date content.
3. Serverside, using either Varnish, implementing it in the application servers or in the caching layer. Implementing access control in the application layer makes sense if you have limited distribution. Implementing it in an edge cache such as Varnish is in my obviously bias opinion the perfect place to do it. You’ll maintain performance enough to handle almost any traffic and control enough to be able to update content in milliseconds.

Access rules need to be simple

We’ve implemented quite a few sets of access rules in various paywalls. Over time we’ve noticed that the complex ones tend to get simplified as organizations struggle to convey the rules to their readership. Having three different kind of metered quotas along with time variations isn’t easy to communicate. Implementing it is SMOP - Simple Matter Of Programming.
Generally there are two subtopics to this. Authentication and authorization. Authentication is pretty well understood these days and there is lots of software that can help you. Authorization is where you should spend the time.

Devops makes sense

If you decide to put the paywall inside Varnish or you decide on putting any slightly complex Varnish configuration you’ve added logic in the edge caching layer. The changes in the edge layer have to match up with frontend and backend code. So, when you deploy you want the same people and the same process to be able to deploy Varnish configuration and backend code. Please, let the developers own the VCL configuration.

Tests should include Varnish
Since your application now spans Varnish your have to make sure to test it. This will increase the complexity of your test suite. However, there isn’t really that much magic. Any HTTP test tool should do. A tool you might want to consider is varnishtest, the special tool built for testing Varnish and VCL. Our own paywall has tests built with varnishtest.

Please don't hesitate to get in touch if you have some paywall challenges of your own. We may be able to help you out or give some good advice.

This picture is (c) Lee Roylland