I’ve spent quite a few years in my career in the networking test and measurement (T&M) business and know that those in that business have a unique perspective on new networking technologies. T&M vendors see all the early challenges with the roll-out of new technologies and tend to have a good handle on which vendor’s products work as advertised and which do not, as well as when products are ready for prime-time. With that in mind, I sat down to have a conversation with IXIA’s Michael Haugh, Product Marketing Director, to get his take on the readiness of SDN. Mike has been involved in SDN since the early days and works closely with the ONF Testing & Interoperability working group and has been involved in numerous plug-fests.
Ixia has a range of products for SDN and OpenFlow testing and you’ve been involved with many ONF member companies in their testing of SDN over the last few years. Where would you say the SDN movement is today? Are SDN solutions production and carrier-grade ready? Or still in the early days?
Haugh: The SDN movement has progressed rapidly since the start with the creation of the ONF in the spring of 2011. Now, SDN can mean many different technical solutions; one that has become well adopted is OpenFlow. We introduced OpenFlow test capability in early 2012 and worked with about a dozen early adopters and that has grown to almost every network equipment manufacturer (NEM) we work with and most service providers. OpenFlow has progressed to version 1.3 being widely adopted, and it brings in many features which increase the applications beyond the data center where it was first used. Now, with features like MPLS, PBB, and IPv6, it is being evaluated for more places in the network like metro for Carrier Ethernet and mobile backhaul. In addition to many more OpenFlow-enabled switches coming to market, there are many more controllers. The majority of switch vendors are offering their own controllers and applications to have a complete SDN offering. This has increased the market for testing OpenFlow controllers, for which Ixia has switch emulation.
At this point, I would say SDN is becoming carrier-grade. Most of the early solutions we worked with were software-based implementations or hybrid implementations with limited feature set. We are beginning to see a new range of products targeted toward carrier networks which offer high-performance forwarding, high table capacity, redundancy/resiliency, and many other required features.
I think we are still in the early days for SDN. We have seen great progress on the southbound API side, like OpenFlow, but there is still a lot of work to do on the northbound API, controller-to-controller, and many other areas.
What are the most common tests that you are involved with today? Do they involve regression testing, feature and conformance testing, interoperability testing or performance testing?
Haugh: We are involved in a wide range of testing from feature and functional testing to scale and performance testing to specific use-case/application testing. Products vary from pure software-based switches, like Open vSwitch (OVS), to purpose-built devices that leverage programmable network processors.
Most OpenFlow switches we test first start with feature development and require our IxANVL suite for conformance since they want to conform to the ONF specification. There has been decent demand for this based on the v1.0 specification, and now with the v1.3 certification program coming there are a lot of vendors preparing for the certification with IxANVL.
Once they get past feature testing and conformance, they move on to performance and specific application testing. Since the OpenFlow spec now covers a wide variety of features (v1.3 can perform match on up to 40 fields of the packet header) there are a lot of combinations, especially when leveraging multi-table/pipeline designs.
What’s the most challenging element in testing SDN? What’s different from traditional networking testing? Specifically, what have you seen in terms of OpenFlow testing?
Haugh: The most challenging aspect of SDN testing, in terms of OpenFlow, is that so many aspects of the specification are optional. This creates a lot of interoperability issues and requires a fairly detailed feature comparison to determine if a controller (and application) is going to work with a switch. A simple example is that using the flow_mod to add/remove a VLAN tag is optional, so if the application requires it and the switch does not support it — the two won’t work together. That is a simple example, and the feature_request/reply has flags to indicate what actions are supported by the switch. It gets more complicated when the switch cannot do a certain combination of actions. Right now, the OpenFlow protocol exchange cannot communicate specific capabilities or limitations.
In terms of testing we have done, we have seen everything from features not being supported (even mandatory features) to features being implemented in software only, which affects the forwarding performance. In the early days it is very common to find bugs, implementation issues or performance penalties very early in the testing.
Do you generally test software-only solutions or hardware-based solutions?
Haugh: We test both, although the traditional Ixia user is more focused on hardware-based solutions. Quite a few years back now we virtualized our Ixia port, which we have productized as IxVM. This virtual Ixia port can run on a VM of a virtualized server. This has been very useful in testing virtual environments as well as testing vSwitch performance.
What about network virtualization solutions based on overlays? Has that been easy to test?
Haugh: Software-based SDN solutions have become quite popular in the data center — like the VMware (former Nicira) solution, which can provide overlay (tunnels) to create virtual networks over legacy physical networks. Our virtual test ports can be used to verify the network service and forwarding performance.
Where it gets interesting is that there are many flavors of overlays, each using a different tunneling technology. Ixia can test over any technology, and for some, like VxLAN, we also support full emulation.
In some of the new capabilities of OpenFlow (later releases like 1.3+), how do you go about testing multi-table pipelines?
Haugh: Our test solution, IxNetwork, provides full v1.3 support. This allows us to test all the match/action combination for each table and test various multi-table pipelines. The main issue is that there is no protocol in place which lets you pre-determine and agree on the pipeline. There is working going on within the ONF Forwarding Abstractions Work Group (FAWG) which is defining TTPs for this purpose. Once that is in place, it will be much easier. What we find is that the software- and network processor-based solutions are very flexible. Other solutions, like those based on Broadcom chipsets, are very rigid and only support specific pipelines. Check out their OF-DPA document, which describes this. In that case, the application would need to be written for that specific pipeline.
Longer-term, as northbound applications mature, how do you envision testing those? Do you see standardization happening?
Haugh: The northbound API is an area of debate. Some want to see it standardized, some do not. The ONF has a group working on the NBI and will provide some recommendations and guidelines. It seems most are standardizing on RESTful based APIs and are publishing them, so applications can be developed. For Ixia, it provides opportunity to help test and verify the NBI.
How will Ixia approach these types of testing? Strictly API-based, or use-case and end-user metrics-based? Or a blend?
Haugh: Ixia will need to provide a very flexible solution so that many different APIs can be tested. Since the community is standardizing on RESTful APIs, this helps to limit the scope of the support required for testing. Ixia will be able to surround the DUT, in this case the controller, and test it on the NBI and SBI.
Have you had demand for inter-controller testing? What about hybrid deployments (SDN + traditional)?
Haugh: Now with OpenFlow v1.3 defining multiple controllers (as master/slave or equal roles) this requires controller-to-controller communication, referred to as an east-west API. The issue here is that it is not standardized and is proprietary for each controller vendor. Some vendors are looking to leverage BGP, which is very interesting to us. As long as this gets standardized, Ixia will support testing it.
In the bigger picture, with SDN + traditional, you will have more requirements to test automation and orchestration. In most cases that is done through an OpenStack plug-in. For Ixia, we work with several orchestration partners, one of which is QualiSystems, who offers TestShell to automate and orchestrate in the lab, and they have OpenStack integration.
How do you envision testing evolving over the next two to three years as SDN matures?
Haugh: SDN is moving fast, so the more features and capability, the more testing that is needed, which is good for Ixia. I see more of the testing being focused on the controller and ensuring the application is going to work when written for the NBI. We are beginning to see some of this demand and recently worked with NEC to test their controller to ensure it scaled to 200 devices as the data sheet stated. There are also more devices coming to market which are built for carrier networks, so the scale and performance will raise the bar from where it is today.