Implement IPSec on Windows Server 2003
With IPSec, you can encrypt network traffic and protect it from malicious users. With the first of our two part series, we tell you how to implement it on Windows Server 2003.
In this, the latest installment in our series of Windows Server 2003 administration tutorials, we'll take a look at the process of implementing IPSec on a Windows Server 2003 system. We'll also look at some of the procedures you need to be aware of when performing this implementation on a live network. When we're all done, we'll look back at the process and weigh up the pros and cons, seeking to answer one simple question: Is it worth considering an IPSec implementation for your network?
Before wading into an explanation of how to implement IPSec, we should first take a moment to introduce you to, or refresh your knowledge of, this free and very effective method of securing network transmissions.
On a theoretical level, IPSec is a framework designed to provide security for IP based network traffic. On a practical level, it is a network layer protocol that encrypts data so that it cannot be 'sniffed' from the network and then subsequently read or altered. IPSec achieves this functionality through two protocols called IPSec Authentication Header (AH) and IPSec Encapsulating Security Payload (ESP).
IPSec AH does not actually encrypt data, but it does provide authentication and guaranteed data integrity. In other words, with IPSec AH, someone can read the data in a transmission, but they cannot alter it. Nor can someone fake the source of the data. IPSec ESP, in contrast, focuses on securing the data in the transmission, though it does also provide some authentication, and a measure of data integrity checking.
The good news is that in a Windows Server 2003 IPSec implementation, as with most others, you do not need to choose between IPSec AH and IPSec ESP. You can use both protocols to get the full benefit of IPSec – authentication, guaranteed integrity of data, and encrypted data transfer.
IPSec and Windows Server 2003
Now that we have recapped what IPSec is and does, we can get on with looking at how to implement it.
IPSec functionality is provided on a Windows Server 2003 system through the IPSec Services service. So, the first step in configuring IPSec is to make sure that this is running on your server by looking in the Services MMC. On a domain controller, the Services MMC can be accessed through the Administrative Tools menu. The IPSec service is configured to start automatically by default, so unless it has been stopped or disabled, your check should be nothing more than cursory.
(Click for a larger image)
IPSec policies, and their components, are configured through the IP Security Policy Management MMC snap-in. There is no shortcut to this MMC on the Administrative Tools menu, so you'll need to open a blank MMC and then add the snap-in to it. Once you have done this, you'll be able to start working in the IP Security Management MMC snap-in as shown in Figure 1.
Structure of Policies
The properties of an existing rule can be viewed by double-clicking the rule from within the IPSec Security Policies snap-in. The properties page for one of the default policies, which are discussed in a moment, is shown in Figure 2.
(Click for a larger image)
As you can see from Figure 2, there are three rules in this policy. The first defines that security should be requested for all IP traffic, and that the security should use Kerberos (define) for authentication and encryption. The second rule defines that all ICMP traffic (such as that associated with ping and tracert) (define) is permitted with no request for security. The third rule (<default>) defines what happens to traffic that does not conform to either of the first two rules.