Enterprise Networking Planet content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Over the years, the network threat landscape has evolved from hacking for fun and fame to commercialized and targeted persistent attacks. Along this rocky road, Network Intrusion Detection and Prevention Systems (NIDS / NIPS) became an enterprise best practice–now widely considered essential at the network edge as well as at key junctions leading to high-value, mission-critical assets (e.g., data centers). A NIDS focuses on spotting attacks by analyzing observed packets with various methods, while a NIPS attempts to automatically block detected attacks based on policy.
According to Gartner, the worldwide market for stand-alone IPS products reached $939 million last year, up 5 percent from 2009. Growth rate was down from 2008, reflecting both a weak economy and displacement by network-generation firewalls with embedded IPS capabilities.
In this buyer’s guide, we examine the capabilities and features offered by some of today’s best-selling NIPS appliances. Although the specific business needs of each enterprise network may differ, we look at key questions that every enterprise should consider when shopping for solutions in this enterprise network security device category.
Fitting NIPS into your network
A NIDS is composed of passive devices–sensors–which are typically dropped from network taps or connected to switch mirror ports. But, given that NIPS must not only detect attacks, but actively stop them, NIPS is an active in-line technology, often situated in between “the unknown untrusted outside” and “the valuable trusted inside.”
However, this placement does not mean NIPS is always used to block suspicious traffic. In fact, Gartner estimates that 25 percent of enterprise NIPS are initially deployed in an “IDS-only mode,” with blocking features disabled. Administrators start by watching what the NIPS actually detects to build confidence and hone policies, without risking business disruption. Advanced detection and blocking features tend to be enabled over time, depending on each organization’s tolerance for risk (false negatives) versus downtime (false positives).
Furthermore, a stand-alone NIPS appliance is not the only available form factor. Many SMBs and branch offices prefer to deploy perimeter firewalls with embedded IPS features (such as Unified Threat Management appliances). Some enterprises would rather bolt an IPS blade into a data center switch (or slip an IPS card into a router) instead of dropping in an IPS appliance. It is common for network equipment vendors to offer form factors that fit into several of these deployment models, all powered by the same IPS engine. Choosing a network architecture to best fit a given business scenario is therefore an essential part of NIPS selection.
Finally, when choosing any in-line security technology, it is important to consider what happens when systems reach capacity or fail. Consider how a NIPS can be scaled (e.g., by adding appliances/cards, managed by the same console). Examine load balancing and high availability features and ask whether the NIPS “fails open” (i.e., permits all traffic without blocking to avoid business disruption).
Inspection depth versus speed
NIPS performance and therefore design is influenced by the need to inspect traffic at near-wire-speed versus the need to reassemble messages and drill deeply to match patterns, observe behaviors, and enforce policies. If you plan to deploy NIPS at a large network’s perimeter or in front of a server pool with high transaction volume, be sure to compare throughput requirements to product benchmarks – measured with IPS enabled. Policy number and complexity can have a significant impact; the stand-alone NIPS market has been moving to purpose-built hardware to meet ever-higher expectations.
However, deployment location can also impact required depth. Network edge and branch office NIPS are often deployed as relatively coarse measures, ensuring that common easily-recognized threats cannot get very far. Narrowly-focused data center NIPS are more likely to drill deeper – for example, searching for database threats, web apps threats, and behavioral anomalies. Thus, business goals and likely impact on policies should be considered when establishing both performance and functional requirements. Don’t be fooled by a speedy product that cannot detect threats of importance to you.
In fact, Gartner reports that 65 percent of new NIPS deployments stick to basic vendor-recommended configurations; just 10 percent of enterprises end up aggressively using advanced features like custom signatures and network behavior analysis. This suggests that many organizations could be getting more value out of their NIPS investments, given a better understanding of capabilities, easier ways to put them to work, and higher confidence that utilizing them would not impede legitimate traffic.
Detection and response methods
NIPS products have evolved over time and will continue to evolve, working to better detect and prevent new threats. Today, it is not unusual for a NIPS product to combine several threat detection methods to create a layered defense and offset weaknesses associated with any single method.
The foundation used by most NIPS is signature-based detection: looking for pre-defined patterns associated with reported vulnerabilities and exploits. Like anti-virus signatures, NIPS signatures should be updated often, driven by new threat intelligence and signatures supplied by your NIPS vendor. In addition, many NIPS can be extended with custom signatures that you may develop to reflect threats unique to your business –for example, detecting exploits against proprietary applications.
It is also common for NIPS to provide protocol anomaly and rate-based detection: watching for spikes in traffic load, out-of-sequence packets, or nonsensical headers that are often in Denial of Service (DoS) attacks. Here, look for tuning parameters that can avoid false positives should you experience a legitimate but sudden increase in traffic.
Together, these methods can detect many threats – but perhaps not brand new zero-day threats that have never been seen before. To this end, a NIPS may offer network behavior analysis: using an established baseline of normal traffic to flag suspicious traffic that could represent intruder activity or trojan back-channels. Behavior analysis can be powerful, but it can also be more difficult to tune and requires a solid baseline. Here, look for automated base-lining, self-tuning aids, manual tuning options, and how easily exceptions can be made to work around false positives without disabling the NIPS.
Finally, consider how a NIPS responds when threats are detected. Policies generally control whether a NIPS just generates alerts or launches automated response (e.g., TCP reset, IP address quarantine, ARP redirection). In addition, look at how much useful information the NIPS provides, whether related alerts are correlated to each other and to users, how well severities and thresholds enable focus on top-priority incidents, and whether forensic details are captured to enable investigation long after the incident. A good NIPS should strike a balance between too much information and not enough, using automation, post-processing, and GUI features to promote operational efficiency.
Evaluation criteria
Start your own product search by getting a handle on where your NIPS will be deployed, the assets it will be expected to protect, the primary risks it must be able to address, and the speed at which it must do so. Given this as a foundation, it’s time to start looking at individual NIPS products and their capabilities.
One useful framework for NIPS comparison was developed by NSS Labs, a research firm that conducts annual independent tests on various security products, including NIPS vulnerability, exploit and evasion tests. During its recently-completed 4Q10 NIPS Group Test, NSS Labs compared tested products based on the following criteria:
- Security effectiveness: As noted above, many NIPS products are simply deployed with vendor-recommended default settings that provide a basic level of intrusion detection. Consider the level of security offered by those settings, as well as the security that can be achieved by tuning those settings. Consider not just vulnerability/exploit coverage, but also successful attack results (e.g., arbitrary code execution, buffer overflow, code injection, cross-site scripting, directory traversal, privilege escalation).
- Effectiveness by attack vector: In the early days, NIPS focused primarily on incoming traffic, sent by external attackers. However, the threat landscape has shifted towards compromised internal systems, requiring analysis of outgoing traffic as well. Consider how well any NIPS handles threats in both directions.
- Effectiveness by disclosure date: Every NIPS must be continuously updated to defend against new threats, including rapid response to newly-emerged zero day attacks. But what about sustained defense against old threats? In fact, many of the biggest security incidents begin with an exploit against a relatively old security vulnerability that victims had not yet patched. Consider whether default and tuned NIPS policies cover both old and new threats.
- Resistance to evasion: Given commercialization of the threat landscape and targeted attacks, hackers have incentive to go “low and slow” trying to not just avoid raising human suspicion but also evading automated threat detection. Ask how a NIPS deals with well-known evasion techniques, such as IP fragmentation, TCP segmentation, RPC fragmentation, URL obfuscation, and FTP evasion, as well as more advanced techniques like PDF or JavaScript evasion.
- Impact of evasion: How a NIPS handles basic evasion techniques like IP fragmentation or TCP segmentation heavily impacts its effectiveness at handling most threats, since it will not be able to look for higher-layer exploits. Consider whether basic evasions would neutralize a NIPS deployed with default settings.
- Performance: As noted above, NIPS performance (e.g., throughput, connection rate, transaction delay) involves trade-off between signature number/complexity and throughput. NSS Labs benchmarks both maximum performance and real-world traffic, measured with default and tuned NIPS policies. Your own traffic content/load and NIPS settings may differ, but such benchmarks can help buyers “size” NIPS products and models.
- Total cost of ownership: Consider factors like purchase price, maintenance fees, installation effort, policy tuning, and routine maintenance when determining TCO. Don’t underestimate the cost of maintenance (e.g., testing and applying routine signature updates) or the cost of policy tuning (e.g., creating policies for new vulnerabilities or exploits, eliminating false positives, addressing new business needs).
To learn more about the above criteria, consult the NSS Labs NIPS Test Methodology [PDF] or read about a summary of its NIPS Group Test Results.
Example products
According to Gartner, 2010’s top-selling stand-alone IPS vendors today include Cisco Systems, HP Tipping Point, IBM ISS, Juniper Networks, McAfee, Radware, Top Layer, and Sourcefire. To illustrate available NIPS products, EnterpriseNetworkingPlanet will profile a few of these product lines over the next few weeks, including Sourcefire Snort and 3D Sensors, HP S Intrusion Prevention Systems, and Cisco IPS 4200 Series Sensors.
