Battle of the Multi-Clouds: SD-WANs, VPNs, and More

Remember the Cisco Intercloud Fabric? It was part of Cisco’s failed Intercloud strategy, which launched in 2014 and was shut down in 2016 with end-of-sales in early 2017. The Intercloud Fabric was supposed to be a magic substrate that would connect private clouds to public clouds as well as connect across clouds. Cisco ran into market and execution problems on its ambitious vision, and Intercloud joined the ranks of past Cisco initiatives with limited market traction like Application-Oriented Networking (AON).

Fast-forward to 2019, and here we are again talking about multi-cloud connectivity and how enterprises need a strategy to manage network connections across multiple clouds, both private and public. However, there’s not a clear answer to the problem, and AvidThink expects things will get more complicated before we see some resolution.

There are quite a few categories of cloud connectivity. I’m not going into the variations in this post, but they range from private express lanes, like AWS Direct Connect or Azure ExpressRoute, to remote-site connectivity to public clouds, to cross-cloud networking for container workloads. The ones I will cover for now, because they overlap with the SD-WAN market, are those with connectivity into virtual private clouds (VPC) from remote sites and cross-cloud VPC connectivity.

So why is cross-cloud or multi-cloud important? For one, enterprises now are a lot more savvy about the use of public and private clouds. CIOs who used to think multiple availability zones (AZ) were sufficiently robust are now mandating that critical applications must be able to port easily between different clouds, and in some cases, run simultaneously on different clouds. Furthermore, development teams are now building some cross-cloud applications that take advantage of best-of-breed native services in different clouds.

If your VPCs are within one provider, such as AWS, there are multiple options. You can do peering between VPCs with little effort. Although this is not without its complications. For example, on AWS peering VPCs are not transitive. Meaning that if VPC A is peered with VPC B and VPC A is also peered with VPC C, that doesn’t imply VPC B and VPC C can communicate. Things get more challenging when you have to connect different VPCs in different locations and even more complicated when you need to bring in private clouds or remote offices.

Basic Connectivity With Routers and VPNs (Plus Firewalls)

There are simple ways to provide external connectivity into a VPC. All the major clouds have a software gateway you can attach into a VPC with an external IP address. These gateways usually can perform network address translation (NAT), so you can hide your private address space in the VPC from the outside world. You can even selectively pick servers inside your VPC subnet to expose to the outside world with a corresponding external IP. This provides connectivity from external sites to services within a VPC.

For VPC-to-VPC connectivity, using routers with NAT is possible, but most often, enterprises will turn to their trusty virtual private network (VPN) services to provide more flexibility and control across server-to-server connections from within different VPCs. Good ol’ VPN and IPsec can connect across sites within the same clouds and across clouds. In these situations, most enterprises turn to third-party firewall/VPN gateways or open source VPN/firewall/routing software to tie these clouds together. Any of your favorite commercial software-based firewalls that load into Amazon Web Services (AWS), Microsoft Azure, or Google Cloud Platform (GCP) will do the trick. Alternatively, you can always use an open source Linux or *BSD version. The advantage of picking a firewall/VPN solution is the added security rules that come with it.

Wait, SD-WANs Want a Bite of the Opportunity, Too

As it turns out, software firewalls with VPNs aren’t the only option to connect across VPCs. SD-WANs are also muscling in. Fresh from their wins connecting remote sites to HQ and private and public clouds, SD-WANs are now pushing into the use case of VPC-to-VPC connectivity.

SD-WAN vendors claim that their approach is potentially better for the enterprise because they provide a consistent network policy across the entire enterprise, from the edge to corporate HQ to all clouds — private and public. Their central cloud control ensures a single pane of glass that can universally manage and monitor an entire enterprise network. Not to mention compression and WAN optimization as well as QoS capabilities, which can potentially improve application performance across these VPC-to-VPC or remote-site-to-VPC connections.

And, of course, these SD-WAN solutions already provide encryption, so they can replace the firewall/VPNs. Indeed, many SD-WANs have sophisticated user, device, and role-based policies that dictate what type of application traffic is allowed in each site.

So here’s another instance, back to an article I wrote about firewalls and SD-WAN, where next-generation firewalls (NGFW) and SD-WAN are overlapping and competing. And the list of firewalls that now claim SD-WAN features continues to grow. Again, is SD-WAN part of the feature-set for a NGFW or is the NGFW simply a feature of an SD-WAN?

Transit VPCs as Evidence of the Emerging Battlefield

As evidence of the evolution of edge and cloud connectivity from routers to firewalls and now to SD-WANs, let’s take a quick look at transit VPCs. I’m going to use the term transit VPCs in a more generic context, as opposed to the AWS Transit VPC solution. The concept of a transit VPC is equally applicable to Azure and GCP. Essentially, in a deployment, a transit VPC uses a dedicated VPC as the “transit VPC” which contains either a software router, like the Cisco CSR1000v, or a firewall/VPN from your favorite vendors — think Juniper, Cisco, or Palo Alto Networks — or now, an SD-WAN solution, such as Riverbed, SilverPeak, or VMware-Velocloud NSX SD-WAN. This VPC becomes the hub in a hub-and-spoke architecture with other spoke VPCs connecting through it as it controls access to and from both internal VPCs and external VPCs or sites in other clouds.

Instead of a full mesh, with the attendant complications of a N-squared set of connections to set up and govern, it reduces to simpler N connections. The downside is that all traffic has to flow through the hub, which could be a single point of failure that needs to be made redundant and that requires scaling capabilities to match performance requirements as needed. We’ll leave the detailed discussion of transit VPCs for another day, but I wanted to share a slide from Amazon’s re:Invent 2018 that shows the varieties of options for VPC connectivity:

Notice the routers, firewall/VPNs, and SD-WAN options? Plus Aviatrix, which doesn’t quite fall into any of the categories, but defines a new wave of cloud network orchestrators that facilitate VPC-to-VPC communication and cloud networking in general. This slide is a simple depiction of the battlefield that will define cloud-to-cloud networking in the next few years.

Hold Your Horses, What About Container Networking?

But wait, we’re not quite done. We’ve only discussed VPC-to-VPC and remote-site-to-VPC connectivity. We haven’t even delved into container pod-to-pod, or VM-to-VM connectivity. There’s a complete set of considerations on that front and another set of players. Aside from the commercial options, there’s open source options like Calico, Canal, Cilium, Flannel, Kube-Router, Romana, and WeaveNet, though some of them aren’t as well maintained as they used to be. We’ll revisit all of these another day.

So just like regular networking, cloud networking has layer upon layer of complexity. Is it any wonder that application developers and DevOps yearn for a world of complete Level 3 public addressability and reachability and where service meshes gate and control connectivity, policy, security, and load distribution? I would too, if I were a DevOps engineer.