When Troutman Sanders centralized its IT operations and revamped its WAN, it realized it also needed to upgrade it monitoring tools.
“We didn’t have good visibility into our WAN,” says William H. Horn, an enterprise systems engineer at Troutman Sanders LLP, an international law firm headquartered in Atlanta. “We couldn’t look across a circuit and see what was going at a given point in time.”
Serving Fortune 500 firms, Troutman Sanders employs more than 700 attorneys on three continents, with close to 2,000 support staff, including a team of twelve in the network and systems engineering department. Its systems run in an 80 percent virtualized Microsoft environment. Each office has two Cisco routers linking to fully redundant WANs.
As part of its strategy to centralize its IT operations, Troutman Sanders recently retired its backup WAN, downgraded its old primary point-to-point WAN to the backup slot and installed a new primary virtual private LAN service (VPLS) WAN. Riverbed WAN acceleration appliances were used to shape the traffic and reduce latency.
“There were some things that people had done in the past in configuring the network that we weren’t comfortable with, so we figured we would get a fresh start with a new WAN,” says Horn. “Once we implemented the new network, we felt we had a good grasp on QoS and the traffic shaping policies, but we still could not see what was traveling on the network.”
The firm was previously using Compuware’s Vantage agentless software for application level analysis, but this didn’t show WAN traffic. It augmented this with Multi Router Traffic Grapher (MRTG) to display network traffic levels and provide congestion alerts. However, MRTG doesn’t tell what users or applications make up that traffic.
“You can set up the network to do exactly what you want it to do,” says Horn, “but if you are utilizing circuits at rates above 50 percent you really have to know exactly where your traffic is and what that traffic is in order to know what you need to classify and what you need to prioritize with QoS.”
Using IP Accounting on a router, he said, can deliver some of the information needed. But again, it was not a complete solution.
“IP Accounting doesn’t provide real visibility, nor can you see historical information,” he says. “All the accounting data is building over time until you clear it, so you may be looking at something that is aged and the conversation is already over.”
While none of these approaches provided the level of visibility required at Troutman Sanders, it looked like NetFlow monitoring would do the trick. NetFlow provides information on who, what and where is communicating through the network device. It consists of two different elements – a flow generator and a flow collector. A flow consists of a stream of packets being transmitted from a source to a destination.
Traditional NetFlow uses seven key fields (source IP address, destination IP address, source port, destination port, Layer 3 protocol, Type of Service byte, and input interface) to identify a packet and determine if it belongs to a particular flow. A later version, called Flexible NetFlow, lets administrators determine their own fields. Once the conversation has ended or is summarized it is sent to the collector. A single NetFlow packet contains conversation details on anywhere from 24-30 conversations. Generally the load on the NetFlow capable hardware is negligible as is the volume of traffic it causes across the network. Administrators can use the real time or historical data to identify such items as who are the top talkers on a link, who they are communicating with, what protocol/application they are using and how long have they been doing it.
When time to look for a NetFlow collector, Troutman opted to go with Scrutinizer from Plixer International – in this case, the paid edition that includes historical data and support. He figures it was cheaper to pay for a robust, supported product than to spend staff time working with the free version.
“Just being able to send the configuration data to the routers using SNMP tools from the Scrutinizer server saves us a ton of work,” says Horn. “No one wants to spend a week to build an open source server to do the same thing.”
The software was loaded on its own virtual server with two interfaces – one to collect the NetFlow export data from the routers and the other for the RDP (Remote Desktop Protocol) and web interfaces used by the network management staff.
“We discovered early on that it doesn’t work well if you are sending all the NetFlow exports into the same interface that you are using to browse the box or RDP into the box,” says Horn. “There is a traffic flow issue and the NIC receiving the traffic will slow down.”
But by directing the traffic to separate ports, the routers can send their data without problem. He foresees no issues even if the company runs NetFlow on 100 switches and routers.
“The NetFlow exports don’t create enough traffic to be a significant volume, so we will always just use the one server,” says Horn. “It is useful to be able, by going to a web page, to look at the primary circuit, see what is on it at the moment, or go back and look at the history.”