As consumers, we expect to access any content we want on any device with within-millisecond-availability, playing on-demand. We don’t usually think about what is required technically to make this work. One of the biggest components in this equation is storage and the process of accessing selected content from storage to serve it. With massive libraries of both in-demand and long-tail content available, the challenges of high-performance, fast, available and consistent delivery are growing.
Storage is one of those things most of us probably don’t think much about when it comes to provisioning enough for our data. With innovations in cloud storage and the universality of streaming services, most of us - from the average consumer to the small business - don’t have massive storage demands. But what happens when those needs grow? When the business in question is growing, and along with the company, the storage and retrieval needs grow, too? We as consumers rely on on-demand streaming services, for example, but where do we think that storage capacity lives? And how do we get instant access to massive libraries of video or music content? The press-a-button-and-it’s-here near-magic we now expect is nearly miraculous when you consider all the elements that must go right to deliver this content fast and consistently on-demand.
Most companies and their content libraries, however small the data set, can benefit from a streamlined, more efficient, smarter storage setup. But where it really makes a difference is in designing storage for the high-performance needs of video distribution, CDNs, and large-cache use cases. That is, the companies consumers turn to for on-the-go music of their choosing and the content that fuels the cultural phenomenon we know as “Netflix and chill”, whether or not it’s even Netflix we’re using.
When Waoo, a content and fiber broadband provider in Denmark, needed to scale for their massive video content delivery needs and to create their own CDNs that could serve traffic both inside and outside their own network, they looked at Varnish Plus to help deliver fast and reliably.
Why did we create the Massive Storage Engine (MSE), how it works and who should be using it?