Optimizing cloud use for the new e-commerce reality

As the Covid-19 crisis started ramping up, and lockdowns became inevitable, e-commerce sites were among the first to take a hit. This was especially true for e-commerce retailers providing essential services, such as online grocery shopping and delivery. Consumers rushed to these sites to start stockpiling, and even as lockdowns have eased, shopping habits have changed permanently. Some examples: 

• More than a third of UK consumers are reported to be doing online grocery shopping, and will continue to do so even when Covid-related restrictions end.
• Most Singaporeans are opting for online shopping to purchase daily goods, and online sales are set to reach S$9.5 billion (USD 6.9 billion) in 2020. Similar trends were echoed throughout Asian markets.
• The second quarter of 2020 saw e-commerce grow a whopping 45% in the United States. About 80% of Americans have shopped online for groceries during the outbreak, up 39% from pre-pandemic figures.

Supporting IT infrastructure endured a trial-by-fire in this unusual period of change. E-commerce, as we’ve long predicted, was already becoming an everyday thing. Covid accelerated this process, forced companies to build in new functionality to serve consistently higher levels of traffic as well as manage spikes, and has tested the limits of cloud capacity and elasticity. 

Coming out successfully on the other side of the initial wave of the crisis, having recovered stability and site reliability, e-commerce platforms have shown that it is possible to prepare their infrastructure to manage extraordinary and unpredictable traffic and mitigate the problems this traffic introduces while continuing to serve consumers. The cloud isn’t infallible but when managed and optimized properly, it plays a key role in everyday performance and in high-traffic situations.

Setting the scene: E-commerce boom, cloud bust

Imagine, if you will, the scene: from early on in the pandemic (February 2020), online grocery shopping began to go bananas: people were already stocking up and pulling back from the physical shopping experience. By mid-March, when lockdowns started going into effect, e-commerce sites saw immediate spikes. For example, one Varnish customer reported that a lockdown announcement from their country’s head of state happened one evening. Traffic plummeted during the announcement, and 20 minutes later, traffic on the website suddenly boomed to at least 4x its usual volume. The site completely crashed soon thereafter, and competitors -- and indeed most e-commerce retailers -- experienced the same. 

The week that followed left online retailers scrambling to fix infrastructural problems, remove any non-essential functionality that dragged performance down, increase cache time-to-live parameters and introduce new traffic management tools, such as queuing systems to help flatten out traffic spikes. Eventually this combination of tricks got booming traffic under control.

But one thing most retailers didn’t count on was a cloud computing shortage. Ever since cloud technology became the go-to for flexible, elastic compute capacity, no one really challenged the idea of the cloud as a limited resource. In mid-March, for a brief period, the major cloud providers simply didn’t have any extra capacity available as every kind of online business attempted to scale out and up simultaneously. While this was a very temporary situation, it refocused the need for optimization of cloud use.

Evaluating cloud prospects 

During the pandemic, the e-commerce sector has seen an acceleration of growth that under normal circumstances would have taken years to develop -- all while other sectors of the economy have struggled. Big box stores have prospered during lockdown periods, making their brick-and-mortar stores into local delivery or pick-up hubs to facilitate the next-day delivery/pick-up offers consumers demand. While for big retailers, local infrastructure has been a boon, helping manage the tricky supply-chain questions raised by Covid, the technical constraints supporting this boom have posed challenges. 

In the highest-intensity moments for IT teams in large e-commerce companies, the frenzy to spin up cloud instances (and not being able to) required, for example, shutting down non-production environments internally to free up CPUs to run extra servers for production load. Once queue systems were introduced to smooth traffic out and clear bottlenecks, and emergency measures, such as those listed above, were deployed, such sites were protected from massive surges (which are the killer). 

Once these fixes were in place, and crunch time was over, e-commerce companies were able to take stock and see that infrastructure improvements and removing bottlenecks had long-term positive performance improvements, in some cases, cutting down on page rendering time and being able to handle greater capacity with some basic streamlining of resources. And this, to make a long story short, is where the evaluation of the cloud comes into play.

It’s a cloud world, and we’re just living in it

For the most part, cloud resources have been reliable and available in all but the most extreme circumstances, making them ideal for flexibly spinning up more instances during predictable spikes like Black Friday. This will -- and should -- continue to be the case because there’s no better way to scale flexibly as demand requires.

But for many retailers, these extremes invited the opportunity, when things calmed down, to look at how optimizing cloud use could better balance actual use and predicted use and the tendency to overprovision cloud services and drive up costs. If anything the Covid crisis has moved companies into a new, more sophisticated relationship with the cloud: where once flipping a switch and turning on more capacity was easy and almost “out of sight, out of mind”, reframing the cloud as a potentially limited resource has highlighted the need to: 

• predict and provision for what is actually needed
• consider the reasons for a multi-cloud solution
• understand and manage cloud costs, particularly in a crisis or high-intensity situation

As a number of e-commerce categories, such as groceries and pharmacy, have been classified during the pandemic as “critical infrastructure” by national governments, the pressure to perform became greater than ever. For the most part, e-commerce platforms responded to issues and maximized resources to stay available and serve consumers. 

Whether preparing for disaster or getting ready for the predictable holiday rush, it’s a cloud world, in which cloud resources offer ways to get cache, infrastructure and software-as-a-service applications easily. 

Adopting Varnish Enterprise in the cloud (available on AWS, GCP, Azure, and Oracle) lets you can reap the benefits of offloading content delivery, reducing backend server load by up to 99%, making costs more predictable and content delivery more efficient while setting your e-commerce platform up for greater resilience.