There’s been a lot of focus in the industry around overlay networking as a vehicle for delivering networking-as-a-service or networking-on-demand in order to interconnect communities of interest quickly without having to manually configure all the network elements across the network. An overlay network approach works by creating L2 tunnels over an existing IP network, thus enabling a pair of end systems (real or virtualized) to peer with each other as if on the same L2 subnetwork. Overlay networking is a powerful tool that can help enable scalable, rapid connectivity and elastic bandwidth between virtualized compute and storage resources programmatically driven by applications and business logic -- not manually configured by operations staff. The momentum behind protocols such as VXLAN, GRE, and NVGRE to support the increasingly agile networking solution within and between data centers, including multi-tenancy support through virtualized address spaces, is good evidence of the traction the overlay paradigm is gaining in the industry.

However, despite the potential usefulness that overlay networking can provide, there are inherent drawbacks. Tunnels, by virtue of how they operate, have neither visibility nor control of the network elements of the underlying physical network, sometimes referred to as the underlay. As more and more tunnels are overlaid on a common shared network, one can imagine network congestion and contention for bandwidth resources becoming an issue, particularly in the wide area network (WAN) and in high peak-to-valley traffic scenarios where over-provisioning is undesirable or not economically viable. Without proper control mechanisms such as traffic engineering in the WAN, large-scale overlay networking can create difficulties for service providers in maintaining service level agreements (SLAs) or for cloud network operators/providers in ensuring appropriate levels of application performance. The resulting traffic congestion can lead to packet loss and protocol time-outs that cannot be remedied by overlay tunneling protocols, thus adversely affecting application-level traffic flows. So while overlay networking can provide significant benefits within the data center, it is critical for service providers to be capable of managing the underlay outside of the data center to mitigate such situations.

In the WAN, where the optical transport network plays a key role, SDN is now beginning to emerge as a new technology tool that offers the promise of managing the underlay. With emerging dynamic optical transport solutions that can support a mix of integrated packet, OTN, and optical switching functions, SDN is gaining interest as a way for operators to dynamically program transport bandwidth and orchestrate resources across both the IP and optical transport layers. This technology can not only accelerate and automate the network configuration that best meets the needs of the overlaid traffic, but it can also optimize network resources for performance and utilization, as well as ensure QoS and committed SLAs.

WAN networks are typically constructed with two layers: an IP/MPLS layer and an optical transport layer. With the broad adoption of integrated switching technologies into optical transport platforms, including ROADM, OTN, and/or packet, the optical transport layer has evolved into a much more agile networking layer. This type of solution presents an opportunity for network operators to leverage the optical layer directly for certain traffic types (e.g., elephant flows), particularly when the cost per bit to transit a transport system is much less than the cost per bit for transiting a router.

In order to access and leverage these network resources in an optimal fashion, one cannot rely solely on the overlay model, where all traffic management decisions are relegated to the IP/MPLS router layer. Without a view into the underlying optical transport and transmission layer and some multi-layer intelligence, the IP/MPLS layer cannot make optimal decisions that incorporate the capabilities and characteristics of transport layers, as it only has a view of the IP topology. While this approach was common practice in the days of static point-to-point wavelengths, carriers today are seeking to leverage the agility of today’s transport networks to help improve the overall efficiency of the combined IP and optical layers.

In an underlay model, the topology of the transport layer, along with its optical and digital switching capabilities is exposed to the SDN control layer, ideally in a vendor-agnostic fashion. This facilitates the exposure of the underlay network resources and, when combined with the view of the router layer, creates a globalized view of the multi-layer core. It is only when the SDN control layer gains visibility to both of these core networking layers (and how they are interconnected) that cross-layer coordination and optimization of network resources can then be achieved, and tradeoffs between the router and the transport layer can be best realized.

One example illustrating the value of managing the underlay is core network re-grooming and re-optimization. Network operators often design and engineer a “flow” through the core network, based on its current state, but as the network state evolves, what was once an optimal path may no longer be the best. Non-uniform traffic conditions and patterns at the network level along with changes in topology at different networking layers can impact the degree of optimality for traffic flows, as well as the quality of service experienced by the flows. Periodic re-optimization of the network to gain more efficiency and utilization out of the existing network’s resources is a function often desired by operators, but not easily attainable. Some operators have developed homegrown tools, but proactive analysis of the network and identification of opportunities to optimize a flow’s path through the multi-layer network are not common practices.

SDN’s principle of logically centralized multi-layer knowledge, fortunately, presents an opportunity to concurrently examine resources and traffic flows at both the router and optical transport layers, along with topological connectivity between these layers, and to implement “globally” optimized paths that meet the network operator’s objectives through multi-layer provisioning and orchestration. By coordinating resources concurrently at each layer, a new level of traffic engineering can be attained that transports bits through the most appropriate layer of the network. Large steady flows with a need for lower latency or better performance, for example, can be offloaded from intermediate routers and instead be tunneled beneath the router layer and transported via the packet/OTN/optical transport layer, saving relatively expensive transit router ports and fabric resources.

While there is great value in overlay networking in certain use cases, it’s not a technique that applies to all scenarios, particularly when it comes to multi-ayer SDN for carrier networks, where reductions in cost/bit for services along with acceleration of the introduction of new services both continue to be critical issues. If we’re to see true optimization in the core, then coordination and optimization of resources within, as well as between, the IP and converged optical transport layers will be necessary.  In order to achieve this objective, one can’t hide everything under an overlay – one has to know what’s going on under the covers.