Industry standard servers have played a big role in reducing the cost of networking across the enterprise. But there is a fair amount of nuance that needs to be appreciated to understand how to achieve that goal. One of the best examples is the way Amazon Web Services offloads network services from industry standard servers.

AWS has the largest amount of x86 server infrastructure on the planet. But even with all that infrastructure, AWS spent several million dollars developing its own network infrastructure to offload networking functions from those servers. At the recent AWS re:invent 2016 conference, James Hamilton, vice president and distinguished engineer for AWS, described how AWS is employing custom 25G routers and 10G network interface controller (NIC) cards based on commodity processors to scale networking services in the cloud.

A key element of strategy, Hamilton said, was the decision to offload as much of the processing from the server to the NICs by employing a separate network operating system that make more server capacity available for applications, while also reducing the amount of energy consumed in the data center.

“If you can offload the hardware you can run roughly a tenth the latency, roughly a tenth the power and roughly tenth the cost,” Hamilton said.

In addition, Hamilton noted the data center is more secure because if a hypervisor is compromised on the network it remains secure because it’s running its own separate operating system. In the future, Hamilton said that AWS will build its own Annapura ASIC to further enhance networking performance based on silicon technology it originally acquired in 2015.

Naturally, every IT organization that builds out its own cloud platform is going to encounter the same problem. They just won’t have millions of dollars to throw at research and development to solve that engineering problem.

That’s where new computing architectures in the data center will play a critical role. Vendors such as Netronome and others are pioneering the development of commercial products that enable IT organizations to offload network processing from server CPUs in much the same way AWS does but at a fraction of the cost. The primary difference is that Netronome keeps costs under control by enabling the same performance and efficiencies on COTS servers for all IT organizations.  In addition, Netronome’s XVIO network interface is less disruptive to existing applications than the SR-IOV interface that AWS uses to support cloud-native applications.

It should be apparent to every IT professional that the relationship between servers and networks is fundamentally changing. There will continue to be infrastructure dedicated to both functions. But the processing of repetitive network traffic will occur on a NIC directly plugged into a server in a way that dramatically reduces the latency that gets created when network traffic is processed on a router or switch.

The challenge facing IT organizations today is to figure out how best to take advantage of these new capabilities to achieve similar levels of networking performance and scale as a public cloud service provider without needing a multi-million IT budget to accomplish it.