Understanding the IPSec protocol

By Brien M. Posey | Aug 24, 2000 | Print this Page
http://www.enterprisenetworkingplanet.com/netsp/article.php/623881/Understanding-the-IPSec-protocol.htm

I recently had a conversation with a friend in which he was telling me about a particular database that his company uses. He said that the information contained within the database demands the utmost confidentiality. However, the database had been around for many years and was written in a way that offers very little security beyond a basic password. Anyone who knew what to look for could easily see confidential information flowing freely across the network.

Later, what my friend had told me really bothered me. I started looking for a way to secure legacy applications, and I decided that perhaps the best way to secure such an application is with the IPSec protocol.

What is IPSec?

"Another cool security feature is the anti-replay code, which prevents the packets from being resent by a hacker. "

As the name implies, IPSec is an extension of the basic IP protocol. The main difference is that it's geared toward encrypting packets. Because the encryption occurs at the transport layer of the OSI model, the application doesn't have to support encryption for the encryption process to work. Therefore, all network traffic generated by applications can be encrypted regardless of the application.

IPSec works by setting up a negotiation between two computers. During this negotiation, the machines must agree on which keys and policies to use. Once the negotiation is complete, the computers use the Internet Security Association Key Management Protocol (ISAKMP) to exchange security keys. Once the computers are communicating under the protection of this protocol, they begin negotiating a second time. This time, they determine which encryption algorithms to use. Two different algorithms are generated, one for inbound communication and one for outbound communication. These algorithms are exchanged in the form of SA (Security Association) messages which also contain references to the IPSec protocols that will be used.

To make IPSec communications even more secure, IPSec supports some other security features. For example, IPSec performs an integrity check to make sure that the data being sent hasn't been altered. IPSec also supports digital signatures and the ability to rekey a message midstream without exposing the key. Another cool security feature is the anti-replay code, which prevents the packets from being resent by a hacker. Such a technique is often used as a way of stealing the encryption key and gaining access to an unauthorized system.

The IPSec policy

When you boot an IPSec-enabled computer, it retrieves an IPSec policy from either the Registry or the Active Directory. The location depends on several factors such as the computer's role and the connection to the network. The policy governs the basic rules regarding the way that IPSec will behave. For example, it establishes such guidelines as which IP traffic should be filtered and the types of authentication and key exchange that should be used. IPSec policies tend to get a little complicated; I'll write a more detailed article on IPSec policies in the future. //

Brien M. Posey is an MCSE who works as a freelance writer. His past experience includes working as the Director of Information Systems for a national chain of health care facilities and as a network engineer for the Department of Defense. Because of the extremely high volume of e-mail that Brien receives, it's impossible for him to respond to every message, although he does read them all.