The biggest driver of data center infrastructure development these days is Big Data. Whether it is advanced cloud architectures within the data center or on third-party infrastructure, or scale-out data warehousing or compute capabilities, the specter of massive volumes of data just around the corner is always present.
But as I’ve mentioned before, the key challenge to Big Data is not just to find someplace to store it all, but to make it available to diverse and disparate sets of resources quickly and at relatively low cost. In this way, network infrastructure has just as big a role to play as compute and storage, and will in fact pose an even greater challenge, considering the way in which networking impacts virtually every aspect of the data stack.
Tech consultant Yves de Montcheuil, for one, recognizes that improving and accelerating the business process is one of the key functions of Big Data, and that this job becomes more difficult as the number and variety of data creation points increases. This means the enterprise needs to become more adept at accessing data, manipulating diverse volumes, developing and applying new applications and hopefully cobbling this broad new collection of systems and services into a real-time knowledge-enhancing machine. In this way, Big Data is not about building a new kind of data ecosystem. It’s about dramatically upping the game of the existing one.
Networking’s role in all of this is clear when it comes to the broad strokes, but still fuzzy in the details. Advanced fabric technologies will undoubtedly provide a more dynamic interchange between disparate data points and data sets, but it will take some doing to get there, and there is still no consensus as to the best way to go about it.
SDN developer Plexxi, for example, sees tighter integration between its network hardware and the analytics platforms that will run Big Data processes as a key solution. The company recently gained certification for its Plexxi Switch under Cloudera’s Enterprise 5 Hadoop platform, which enables the creation of a single-tier, scale-out network architecture for large workloads. The Plexxi switch provides a high-density Ethernet environment featuring optical switching and highly dynamic configuration capabilities that provide both the high bandwidth and system adaptability that Big Data environments require.
Application delivery is also poised to see a dramatic rise in importance as Big Data hits the enterprise. To that end, Cisco recently inked a deal with A10 Networks to integrate the Thunder application delivery controller (ADC) into the Cisco Application Centric Infrastructure (ACI) fabric. This provides a new level of automation for the deployment of L4-L7 application and security services by targeting key functions like dynamic scaling and policy-based management and governance. The partners are already looking to expand their relationship by incorporating advanced capabilities like service chaining, Web Application Firewall (WAF) development and global server load balancing.
And Big Switch Networks is looking to integrate its Big Cloud Fabric into leading data center platforms like VMware and Dell in a bid to leverage its technology as a bare metal solution for hyperscale environments. By forging a tighter bond with vSphere, for example, Big Switch can integrate its fabric controller into vCenter, where it can exert greater control of virtual machines across disparate network topologies, including OpenStack environments. With Dell, the company gains a foothold on Open Networking switches like the S6000-N and tighter coordination with Dell’s other Big Data partners, like Cumulus Networks and Midokura.
If there is any commonality among all these approaches, it is that the network will not be a passive mover of data in scale-out, Big Data environments. Rather, there will be a great deal of interaction between the physical switch, the virtual layer and higher order data and application loads to ensure that the proper resources are being utilized to produce optimal outcomes.
There is still quite a bit of work to do to perfect these solutions, but at least the right people are having the right conversations to bring it all closer to reality.
Photo courtesy of Shutterstock.
