Build your own CDN in 5 steps

We, as users, are consuming (and producing) more content than ever, Cisco predicts that “global Internet traffic in 2021 will be equivalent to 127 times the volume of the entire global Internet in 2005. Globally, Internet traffic will reach 30 GB per capita by 2021, up from 10 GB per capita in 2016.”

This is why CDNs, commercial and private, are more important than ever. Devices constantly send and receive HTTP requests/responses and to sustain this immense exchange of bytes, a well-planned backbone is a must. Hence the accelerating growth of DIY/private and hybrid CDNs. 

Lately many companies have started to implement their own private or hybrid content delivery network. A private CDN, contrary to commercial CDNs, can give you full control over your content because you don’t need to share storage/machines/servers with any other website(s) using the same infrastructure. Also, you are completely free to plan and deploy your own CDN as it best fits your specific use case, flexibility that is very difficult if not impossible to achieve with any of the commercial CDN solutions out there, which use a one-fits-all approach.

And let’s not underestimate the cost savings factor: commercial CDNs usually sell per used bandwidth and number of requests handled by each PoP (point of presence). If the traffic your website sees is in the range of GB per second these traffic-related costs can be noticeably reduced with a private CDN.

Our DevOps-friendly DIY CDN solution can be set up in five steps. All you need to set-up a CDN:

• origin server and PoP (points of presence)
• ISP
• CDN and caching software

Following these steps you will find yourself with an origin server, a few PoPs strategically distributed based on your audience, to which you will be able to serve content following web performance best practices. You can also watch our on-demand webinar — How to Build Your Own CDN in Under 1 Hour — to see a demo of how it's done.

1. Cloud or on-premise?

The very first decision you will have to make is whether to deploy your content delivery network using cloud instances or physical machines.

You will have to ask yourself this question twice: both for the origin server(s) and then for your PoPs. It’s not a mutually exclusive decision; you can decide to implement a portion of your CDN using one (or more) cloud solution(s) while another portion lives on-premise.The following are a few points to consider when making this decision:

1. Availability: does the cloud provider I want to use have machines available in the region I want to serve content to? Does my company own data centers in the region to which we want to deliver content?
2. Performance: will I gain better performance using cloud instances or physical machines?
3. Costs: Given my web traffic load per second, will it be cheaper to pay for the infrastructure or to own it? Also, will it be cheaper to use IaaS or to hire a specific team to maintain such an architecture?
4. Scale: Which strategy will help my business scale: Cloud, on-prem or hybrid?
5. Security: What are the security pros and cons of the different options for my business?

Whether you choose the “on-prem” route is really as easy as installing packages on your servers, e.g. “apt-get install varnish-plus” or “yum install varnish-plus”. You can reach out to us for further info at sales@varnish-software.com 

In the cloud you can follow the instructions on the AWS marketplace or our extensive documentation on how to get started with Varnish and AWS.

2. ISP and routing

Find yourself the best ISP for your needs and also consider that you will need a solution that can route each request to the closest PoP.A few factors worth consideration:

1. Geographic presence: Find an ISP that coverages the region(s) you want to serve.
2. Connection speed: Ensure you have enough speed for both your average day and peak times as well.
3. Availability and reliability: As with all parts of your infrastructure an ISP needs to be always available as it will be the backbone of your business.
4. Cost efficiency: Map out service costs carefully, as costs from an ISP will vary widely depending upon a number of factors, and you will have to find the proper balance between services and costs.

The market offers many ISP and requests routing options, and what you choose to serve your specific needs will depend on the balance that makes the most sense for your setup and traffic expectations.

3. Origin server and PoPs

Let’s start with the juicy part here!
You can think of a CDN as a huge cache with distributed nodes that help to deliver content faster to end users. To do so and be efficient you need to decide, based on factors you consider to be of strategic importance for your business, in which geographical location you want your origin server(s) to be located. The same reasoning applies to each of your PoPs.

The origin server decision is straightforward, as you will want your web server as close as possible to have full control of your content. But when it comes to deciding where to locate the PoPs and how many you need, there are a few factors you should consider:

1. Where is your most important audience? Do you expect business to grow in certain regions? If so, which regions? Also, it’s important to analyze where and how they are concentrated to deliver the best possible experience.
2. Latency between your origin server and your audience: this will help you to decide where to locate each PoP.
3. Traffic patterns. Analyzing them allows you to provide an excellent user experience and optimize your infrastructure

4. Install Varnish DIY CDN software

Varnish is extremely flexible due to its unique VCL configuration language. VCL lets you set caching rules and make decisions at the edge.

Varnish DIY CDN comes with of complete set of content delivery features:

1. Load balancing
2. Prefetch option to keep your cache warm
3. Persistent storage engine
4. High availability
5. Security, in both a secure connection and a fully encrypted cache

Varnish DIY CDN will have to be installed in each of your PoPs.
This is what we will achieve at the end of this quick setup:

As you can see we will install Varnish DIY CDN in:

• Origin shield: this helps protect your origin server(s) and offload some of the requests. In this example the origin server will be based in London.
• PoP A: is one of the two PoPs we’ll develop to set up a very basic CDN. For the purpose of this example we’ll place it in Frankfurt, Germany, within the EU.
• PoP B: Our other PoP will serve content to the North American audience. It will be New York based.

Each PoP and the origin shield will have a symmetrical and coupled configuration to ensure each PoP is always available even if one of the two Varnish nodes is under maintenance or is not available.

Each Varnish Cache Plus tuple is synchronised using VHA, which in addition to replicating content among different caching nodes, also reduces the stress on your backends, relieving them of too much strain consuming less bandwidth and requiring fewer roundtrips.

5. Configure Varnish

To have a basic and working configuration you need to:

1. Let Varnish know which backends it can talk to: Open “/etc/varnish/default.vcl” and specify your backed IP address and port

2. Configure which port on which Varnish listens to incoming request:

Open “etc/default/varnish.param” and set “-a”: 80.

Restart Varnish, and you are all set. You have your very own CDN up and running. Once you have understood what type of infrastructure best meets your needs and how your audience is distributed, following these steps to build your own private CDN using Varnish should only take a few hours.