Network functions virtualization (NFV) is picking up pace and communications service providers (CSP) are going into production with real use-cases. As mentioned by Prodip Sen, the first chair of the ETSI ISG, most CSPs haven’t gone far in their journey. A major reason is the support ecosystem that needs to be around for the CSPs to rely on. One of the most critical elements of the ecosystem is virtual network functions (VNF) validation. VNF validation services based on well-developed, functional, and deployment test plans make sure that a VNF being tested:

• Can live within a telco cloud.
• Can perform functionally the way its supposed to.
• Can co-exist with other network elements.

VNF validation services can later encompass working with the VNF vendors to make sure a particular VNF can function properly within the exact client environment. Most of the time it also involves VNF manager integration services as a part of a larger engagement with the VNF vendors.

The need for VNF validation is two-fold:

• First, vendors are launching new virtual appliances.
• Second, because OpenStack/software-defined networking (SDN) and VNF are upgraded to a new version every few months, a Continuous Integration/Continuous-Development (CI/CD) approach is required.

The challenge is that every company out there is claiming to be the VNF validation and certification authority. It is very hard for service providers to assess who they should work with for this critical task. The following are the most critical elements service providers should look for when selecting a vendor:

• Involvement of the validation company in open source projects is very critical. There is a lot of effort being done in this space within OpenStack and OPNFV projects. Companies need to be evaluated on their upstream contribution to these projects. For example, a very relevant group working under the OPNFV platform is Dovetail. Another project that has been proposed is Yardstick that is working on verifying the infrastructure compliance when running VNF applications. Other related efforts are groups working around platform testing and benchmarking like Tempest and Rally. The idea is to see if the vendor is involved in open source efforts.

• How big is the lab? VNF validation is done at many levels. One of the critical aspects to validate are scalability capabilities of the VNF being tested. Many companies have a very limited setup, and they claim to do all sorts of VNF validation. VNF validation needs to mimic CSPs environment as closely as possible. For that reason, a substantial investment from the vendor needs to be done to set up a lab that can properly assess a VNF on test cases that involve shifting workloads across racks and even wide area network (WAN) links. A serious lab setup is also needed to properly test for scalability of a complex VNF. With multiple OpenStack distribution and SDN combinations, it is critical to understand how many different environments (Mirantis, Canonical, ODL, OC) are available in the lab to test a VNF in parallel.

• Are there independent service integrators for VNF validation? With so many different solutions out in the market, SP's need their use-cases to be mimicked, which normally involves multiple vendor software components. Getting this critical part of the entire life-cycle done by a specific software vendor kills the purpose. Most interesting are the big service integrators that have good working relationships with all the big vendors and can mimic the exact environment of the customer, regardless of the solution that is being used.

The following skill sets need to be evaluated for the VNF validation company.

• Familiarity with 3GPP and Internet Engineering Task Force (IETF) standards to come up with the right test plans.
• Expertise around different open source/proprietary solutions.
• In-house software development teams for northbound/southbound integration.
• Strong system administration skills around OpenStack and SDN solutions.