Using ESP to Prevent Replay Attacks

The tighter your network’s security is, the more difficult it is for a hacker to break in. However, hackers tend to be clever and have lots of methods of getting into a network.

Prior to Windows 2000, hackers could use a method called a replay attack to break into even some of the most secure networks. Replay attacks are seldom used because of their complexity–often, less-complicated methods work just as well. The problem is that before Windows 2000, there were lots of ways to protect against the less sophisticated attacks, but few (if any) ways to protect against replay attacks.

Replay Attacks

In a replay attack, a hacker uses a protocol analyzer to monitor and copy packets as they flow across the network. Once the hacker has captured the necessary packets, he can filter them and extract the packets that contain things like digital signatures and various authentication codes. After these packets have been extracted, they can be put back on the network (or replayed), thus giving the hacker access to the desired access.

Replay attacks have existed for a long time. Years ago, replay attacks were simply aimed at stealing passwords. However, given the encryption strength of passwords these days, it’s often easier to steal digital signatures and keys.

Repelling Attacks with IPSec

Windows 2000 provides a way to protect against a replay attack: the IPSec subcomponent called Encapsulating Security Payload (ESP). The IPSec protocol is a security-enabled protocol that’s designed to run on IP networks. IPSec runs at the network level and is responsible for establishing secure communications between PCs. The actual method of providing these secure communications depends on the individual network. However, the method often involves a key exchange. ESP is the portion of IPSec that encrypts the data contained within the packet. This encryption is controlled by an ESP subcomponent called the Security Parameters Index (SPI).

In addition to the encryption, ESP can protect against replay attacks by using a mathematically generated sequence number. When a packet is sent to a recipient, the recipient extracts the sequence number and records the sequence number in a table. Now, suppose a hacker captured and replayed a packet. The recipient would extract the sequence number and compare it against the table that it has been recording. But the packet’s sequence number will already exist in the table, so the packet is assumed to be fraudulent and is therefore discarded. //

Brien M. Posey is an MCSE who works as a freelance writer. His past experience includes working as the director of information systems for a national chain of health care facilities and as a network engineer for the Department of Defense. Because of the extremely high volume of e-mail that Brien receives, it’s impossible for him to respond to every message, although he does read them all.