With the recent release of the Xeon 5500 (previously code-named Nehalem) and an accompanying 10Gbps network controller, Intel not only gave the folks in the server room something to talk about, it provided improvements to Ethernet throughput that put the chipmaker on the leading edge of network evolution.
According to Steve Schultz, Intel, a pioneer in promoting Ethernet back in the 1970s, has long had a vision of Ethernet as the “common fabric” in the data center, “working with various partners over multiple years to introduce standards in the IEEE to help upgrade Ethernet to a lossless fabric.” The company furthered that vision when it released its first 10GbE network controller – the 82598 – in 2003. It was the first product in the industry to have a feature variously called data center bridging (DCB) or data center Ethernet (DCE), which lets network administrators set traffic classes on the 10GbE link, and to prioritize those classes.
“Now we’re introducing the next generation of that product, which has further enhancements, including iSCSI acceleration and Fiber Channel over Ethernet [FCoE] offloads,” Schultz told Enterprise Networking Planet. In other words, with the 82599 controller, the Ethernet link is now better equipped to transport not only voice/media and data traffic, but storage traffic as well.
The new capabilities reflect some new techniques for moving data around within and among the platform components – specifically in a virtualized environment.
“On the virtualization side we have a number of enhancements to address some of the overhead associated in a virtualized environment,” Schultz said, “where you’re not only virtualizing the servers you’re virtualizing that first layer of an Ethernet switch that’s running on the host processor.”
In emulation, the normal mode of virtualization, according to Ahluwalia, “You have multiple virtual machines running on the host processor. And then there’s the virtual switch [vSwitch] – a small VMM [virtual machine monitor] layer that’s provided by VMware or Xen or Microsoft – and the vSwitch essentially talks to the networking port.”
“We found that there’s a huge I/O penalty associated with that software overhead,” he continued. To combat that penalty, Intel introduced a technology called virtual machine device queues (VMDqs). into the previous generation network controller, the 82598. “That technology really accelerates this vSwitch functionality; and it actually moves a whole bunch of distribution and data routing requirements and features into the networking silicon,” Sunil Ahluwalia, 10 Gigabit Ethernet product line manager, Intel LAN Access Division, said.
But the 82599 takes what Ahluwalia called the “next step in terms of the evolution of performance enhancement,” a brand new technique or alternative to emulation mode, direct assignment, wherein a virtual machine has a one-to-one association with a virtual port on the NIC. “We’ve introduced a technology called VM direct connect (VMDc), based on industry standard SR-IOV, that helps achieve near-native performance in a virtualized environment.”
The proof of the pudding is performance testing in which the new processor/controller duo achieved nearly 20Gbps throughput in a VMware EXS environment – nearly twice the throughput of the older platform.
Intel efforts at boosting FCoE performance in the new platform center around features and capabilities designed to accelerate the data path, according to Ahluwalia – “things like FCoE-capable direct data placement (DDP) where you can directly write into application memory.” Other new performance-enhancing features include CRC offloads and “Large sequence offloads,” he said.
Intel is also spearheading efforts to get native FCoE ‘initiators’ (instruction sets) built into the various operating systems. “Our approach is to have the networking stack in the right place in the operating system. That is something we are driving in Linux and Windows and [VMware] ESX distributions,” Ahluwalia said, noting that this has already happened in the Linux community. “That software is available in the latest Linux kernel releases.”
As to the already-established Ethernet storage technology, iSCSI, for which native initiators are already embedded in those operating systems, Intel is taking steps to accelerate iSCSI traffic with the new platform. For example, “We’ve released a new CRC instruction set in order to be able to do iSCSI CRC in less than one instruction.” Together with other newly introduced traffic accelerating techniques, such as receive side coalescing, (also called large receive offload) – in which incoming data is grouped and sent up to the processor in larger chunks – the new platform yields vastly improved iSCSI performance, topping out at some 500,000 I/O operations per second, nearly double that of its predecessor.
Finally, Intel has integrated a number of what previously were separate components into the silicon for the 82599 controller, including a clock and PHYs for several different interface types. According to Schultz, these integrations bring down both the cost of the controller and its size, making it appropriate for use on NICS, mezzanine cards, and LAN on motherboard designs.
Can you say 10GbE blade servers?