Implement IPSec on Win2k3: Clients and Servers

Welcome back to our look at the process of implementing IPSec on Windows Server 2003. In Part One we looked at what IPSec can offer, and we started the process of implementation by examining IPSec policies. We also looked at the IP Security Policy Management MMC snap-in. In this installment, we’ll continue our look at the implementation process, and configure IPSec secured communications between a Windows XP Professional system and a Windows Server 2003 system.

Assigning Policies
As we discussed in part one, IPSec implementations on Windows Server 2003 are governed by policies. Once defined, these policies are assigned to the system and subsequently dictate behavior in respect to IPSec communications. As we also mentioned in the first part, Windows Server 2003 comes with three default IPSec policies. We’ll be using two of these policies – Client (Respond Only) and Server (Request Security) – for the purposes of this demonstration.

IPSec makes sense if you either feel that the data on your network is at risk, or if you value the security of your data enough to spend a small amount of time configuring your systems … You have nothing to lose by giving it a try in a test environment.

The Client (Request Security) is considered the most liberal of the three default policies. A server assigned this policy will provide an IPSec-secured connection for clients that request one, but otherwise will allow connections to be completed as normal. In this article, we’ll assign the Client (Request Security) policy to our Windows Server 2003 system. Then, when our client system asks for an IPSec connection, the server can provide it.

Before moving onto the configuration of IPSec, we should talk briefly about authentication systems. By default, the IPSec policies provided with Windows Server 2003 are configured to use Kerberos (define) authentication. Kerberos is the default authentication protocol associated with Windows Server and Active Directory, so it’s an obvious choice as an authentication mechanism for IPSec on Windows platforms. However, IPSec on Windows Server 2003 also supports digital certificates for authentication, as well as preshared keys.

A requirement of using Kerberos is that all systems involved in the IPSec process (client and server) are members of a domain in Active Directory. Therefore, if you are configuring IPSec for non-Windows systems, you’ll need to use one of the other authentication methods.

Before discussing policy assignment, we should issue a disclaimer. You should never run test configurations or try out new features like IPSec on a production server. The steps that follow are only intended for use on a test server. Although we are not going to suggest anything that might render a live server unavailable, not performing the steps as described could do exactly that. So, in the best interests of everyone, please keep your experiments to a test server.

To assign an IPSec policy, simply select the policy in the right-hand pane of the IP Security Policy Management MMC Snap-in, then right-click and choose Activate. A small green ‘+’ sign appears on the folder icon of the policy to indicate that it is assigned. You can see an example of an assigned policy in Figure 1.

Figure 1.
(Click for a larger image)

It should be noted that only one policy can be assigned at a time. You can’t combine policies. If an existing policy does not meet your IPSec needs, you’ll need to create a new one that does.

Because we are using the local security policy on the server, once you have assigned the chosen policy, there is no need to manually refresh the policy. However, you should keep in mind that if you were deploying IPSec through Group Policy, you would need to trigger a manual refresh of the policy or wait for the scheduled update cycle before your new settings would take effect.

That’s it! Your server is now ready to respond to requests for incoming IPSec connections. All that is needed is a client system using IPSec to make use of the new functionality.

Client Side Configuration
The process of configuring IPSec on Windows XP Professional is basically the same as on Windows Server 2003. For the purposes of this discussion, we will enable IPSec functionality manually, but if you intend to deploy IPSec to more than just a few computers, you should consider using Group Policy to make the necessary client side changes.

On a Windows XP Professional system, the IPSec Policies can be configured most easily by accessing the Local Security Policy MMC from Control Panel, Administrative Tools. As you can see from Figure 2, the IP Security Policies snap-in is added to this tool by default. As an alternative to navigating through the Control Panel, you can click Start, Run and type Secpol.msc in the Open field.

Figure 2.
(Click for a larger image)

Once in the IPSec Security Policies snap-in, you can either use an existing policy, or create a new one. Windows XP Professional comes with the same three default IPSec policies as Windows Server 2003.

For the purposes of this demonstration, we’ll enable the Server (Request Security) policy on the workstation. This will mean that requests to non-IPSec enabled hosts, like a Web server on the Internet for example, will be permitted without IPSec. Connections to hosts that do support IPSec will use encryption. This is a fairly common configuration for systems on a wireless network where you want to secure communications to internally hosted servers, but are less concerned about securing things like Web browsing traffic. Of course if you are accessing the Internet through a proxy server that is providing IPSec encryption, browsing traffic will also become encrypted, but that’s another discussion.

Once you have assigned the policy, you can establish a connection to the server. The IPSec encryption will occur in invisibly in the background, though you may notice a very slight pause when you first connect to a server. This is because the IPSec communication session needs to be established each time a connection to a new host is made. Other than that, the IPSec encryption establishment and maintenance should be completely transparent. In fact, there is no way to tell that IPSec is being used without actually going and looking for it.

Monitoring and Verifying IPSec Traffic

Figure 3.
(Click for a larger image)

Once you have set up your trial IPSec configuration, you’ll want to be sure that communications between your test workstation and server are indeed encrypted. There are a number of ways of doing this, but perhaps the easiest is to use the IP Security Monitor MMC snap-in. Once you have added the snap-in to an MMC on your server, navigate to the Statistics folder for your system, following the path shown in Figure 3

The statistics provided include the amount of data that has been sent and received in encrypted form, and the number of current security associations. This number represents the IPSec connections that are currently established between this server and other systems. To see details of these connections, click the Security Associations folder as shown in Figure 4.

As you can see, in this test scenario, there is a single IPSec session established between the server ( and the Windows XP Professional client (

Figure 4.
(Click for a larger image)

There are numerous other ways of ensuring that IPSec is being used, such as packet sniffing with a network monitoring application, but if all you are looking to establish is whether IPSec is being used or not, then these are probably the easiest methods available.

So, Is it Worth it?

At the beginning of Part One, we said that we would answer the question of whether implementing IPSec on your network was worth it. Hopefully we have demonstrated that the implementation process is very straightforward, and its operation completely transparent. In fact, it’s hard to find fault with IPSec.

However, one consideration is that IPSec adds an extra layer of complexity to network troubleshooting. Every time you experience a connectivity issue, you have to consider whether the problem is with the underlying network structure, or with IPSec. It may be that IPSec is not the cause of the problem, but it’s one more thing to consider, when you probably have enough to think about already. Additionally, on larger networks or those with already high network traffic levels, you should consider whether the additional (though minimal) network traffic associated with the setup and maintenance of IPSec connections would be a problem. Chances are it wont be, but it should be considered.

Ultimately, IPSec makes sense if you either feel that the data on your network is at risk, or if you value the security of your data enough to spend a small amount of time configuring your systems. Given that Microsoft provides all the software and tools need to configure and monitor IPSec, you have nothing to lose by giving it a try in a test environment.

Latest Articles

Follow Us On Social Media

Explore More