Thanks to everyone who joined us on October 24 for the Plexxi DemoFriday on Big Data fabrics. For those of you who missed it, Ed Henry of Plexxi product management and Nils Swart of Plexxi business development demonstrated how to set up a Big Data fabric integrated with Cloudera, which produces enterprise data hubs built on Apache Hadoop.
After the demo, Henry and Swart took questions from DemoFriday participants. Read the Q&A below, watch the full presentation, or check out the teaser video and other resources below.
At what scale (number of racks) does Plexxi become a preferred network solution?
Plexxi: Let’s forget about the network specifics for a moment, though we will come back to them. The one thing that I wanted to highlight with my demonstration of our product was how easy it is to take the Plexxi solution and tie it into a workflow that you have present, or could build, within your organization. Looking at the simple operational aspects of deploying and administrating a CDH cluster, you can find the typical challenges that most shops will face in terms of scaling and maintaining that infrastructure component of the cluster. Submission of a change request to the network team, that request sitting in the queue within that team for as long as is necessary until it is finally completed can take upwards of a few weeks. Ultimately halting the ability for the cluster to continue providing valuable data oriented feedback to the business.
What I wanted to highlight was the fact that request wouldn’t even necessarily have to be submitted to the network team. The Cloudera integration made use of the Plexxi DSE and discovered the new nodes that were added to the cluster and added them to the respective Affinity Groups within Plexxi Control, allowing that node to adopt the same policies defined, with respect to the network, that all other nodes of that cluster have already inherited and had been approved previously by the network team of the organization.
It’s a change in thought process for most organizations to think about integrations like this with respect to infrastructure. As IT administrators, we’re used to the idea of bolting two pieces of software together to create a workflow, but we’re not used to being able to logically couple pieces of infrastructure together and creating robust workflows with the glue code that we’re writing. What I showed in the demo is just the beginning. You, as the administrator, could take that simple integration and build on it, creating your own workflows. We at Plexxi, are making a truly consumable network platform.
Now, stepping back from the workflow/software specific integration portion, we can talk speeds and feeds with respect to the cluster and how Plexxi fits into the picture.
The inherent architectural difference between the Plexxi product and the other networking vendor’s solutions within the industry position us to be entirely more flexible than most competitor architectures. Our multipathing algorithms that we use to compute all possible paths between two nodes within a Plexxi network, and then the ability to define unequal cost multipathing policy through our fabric, allows us to utilize the bandwidth available between two given switches more efficiently and effectively. We are also able to adapt to usage between those two nodes to better fit the given conditions of the network, and help bolster capacity between two switches on the fly.
These two particular traits of a Plexxi network that I’ve discussed are all available whether you have a four switch deployment, or a 40 switch deployment. Performance gains can be seen from the operational overhead of running an IT infrastructure along with application performance. I’ll talk about applying a Plexxi network to the popular Lambda Architecture in an upcoming post as well, where we can talk about job serialization with respect to cluster resources and how we can integrate the network for resource management with something like YARN much like we can on the cluster level that I showed during the demo.
So Affinity groups — are they applied on a per-policy match basis, or is it more per VM? Per virtualized network element?
Plexxi: Affinity Groups, as I’d conveyed in the demo, are an abstraction of relationships that exists between entities that are connected to the Plexxi network. In order to identify which nodes should be part of an affinity group, we need to take a look at how a Plexxi network can collect this unique identifier for each node with respect to the network. Since we are an Ethernet network, we can identify a node by the MAC address and physical interface that that MAC address is learned on. We can also identify which Affinity Group that node should be a part of by what VLAN that node is a member of. That way we can apply that policy to any node connected within a specific VLAN.
Thinking about this, we can start to build some interesting workflows around using some of the configuration management frameworks that exist within the IT community today. Tools such as Chef, Puppet, Ansible, Salt, etc. all have a “Facts” store that houses information that helps to identify nodes by specific types of attributes. That said, depending on the types of policies you have defined within your configuration management frameworks, you can tie a specific node back to that policy according to its MAC or VLAN membership.
If I were to apply this logic to something like Cloudera Manager’s approach to node management, I could look at it from the aspect of querying what hosts have specific host templates applied and then tie a specific policy to the set of nodes that are identified as being members of a specific host template. Once that host template is deployed to the correct nodes within the cluster, the proper policies can be submitted to Plexxi control at the same time. Once the policy is applied, you now have the network portion of deployment tied into your entire operational workflows.
In all, the Plexxi platform offers much in the way of ease of use, flexibility and policy application models that don’t exist in other Ethernet networking products that exist today.