Windows Server 2003 Security the Hard Way

By Marcin Policht | Feb 13, 2004 | Print this Page
http://www.enterprisenetworkingplanet.com/netsecur/article.php/3313141/Windows-Server-2003-Security-the-Hard-Way.htm

When it comes to Windows Server 2003 security, most of the attention has been paid to software improvements built into the platform. However, there is also a hardware perspective to the security story that network managers need to know about — hardware-based cryptography.

While cryptography can be, and frequently is, implemented exclusively in software, such an approach is often inefficient, especially when dealing with larger amounts of data, asymmetric algorithms, and long encryption keys (i.e., those more than 1024 bits), even on 64-bit servers. In addition, in sensitive environments, the possibility of compromising software-based keys is an issue, since they remain in the server's memory even after they are no longer used (of course, such an exploit would require the attacker have the capability to analyze memory contents).

An increasing need for secure communication, especially in the e-commerce area, raised interest in alternative methods of implementing encryption, which would eliminate the performance and vulnerability issues described above. The new solution comes in the form of hardware security modules (HSMs). In addition to serving as a storage for private keys, HSMs provide a number of standard cryptography-related features, such as secure authentication and communication sessions via Secure Sockets Layer (SSL), by offloading the setup of computational-intensive SSL connections from Web servers to tamper-resistant hardware modules, and protecting access to Web services and custom applications.

Two vendors that currently offer such products for Windows Server 2003 are nCipher and Rainbow-Chrysalis.

Rainbow-Chrysalis, headquartered in Ottawa, Canada, makes the Luna CA3 HSM. Luna CA3 integrates with Windows 2000 and Windows 2003 via the custom Cryptographic Service Provider. It supports every commonly used cryptographic algorithm and key length, and is based on WHQL-compliant hardware. The hardware consists of several components: a token, a token reader, a PED authentication keypad (PIN Entry Device), a set of color-coded PED keys, and a PCI token reader card. The combination of an authentication keypad and keys allows for three-factor authentication (independent of the server to which the device is attached). The first factor involves using a PED key, the second a personal PIN assigned to each administrator, and the third (and optional) invokes the key-splitting feature. Note, however, that three-factor encryption requires the simultaneous presence of several administrators to perform a cryptography-related task.

The advantage of a separate keypad is that keystrokes cannot be captured by the operating system of the server to which the device is attached. The token reader contains two slots to allow secure copying of keys stored on one token to another. To get Luna CA3 operational, install Luna Cryptographic Services on the Windows server. The software setup must be followed: Install the PCI card with appropriate drivers and attach the reader with the PED authentication keypad. Next, initialize and activate the cryptographic token. Finally, configure the server to use Luna Cryptographic Services as the Cryptographic Service Provider.

A more detailed description of Luna CA3 module, its installation procedures, and its integration with Windows Public Key Infrastructure (PKI) can be found in a white paper published on Microsoft's Web site.

nShield HSM from nCipher offers similar capabilities. Based in Cambridge, England, the company has its U.S. headquarters in Woburn, Massachusetts. nShield provides the same degree of integration with Windows 2000 and Windows 2003 PKI as Rainbow-Chrysalis, including support for cryptographic algorithms and hardware compatibility. Its hardware is also similar, with the token reader card available in both PCI and SCSI factors. In addition, nShield has several additional features worth noting.

nShield employs nCipher proprietary Security World key management technology, providing such functionality as key storage, backup, and recovery. It also allows the implementation of security policies across HSM infrastructure and simplifies key transfer, which was one of the critical problems traditionally associated with hardware cryptography modules (and one of the factors hindering their broader acceptance on the market). nCipher's technology offers practically unlimited key storage because keys are stored in the form of encrypted files, external to the HSM (which also makes backup and restore procedures easier to implement).

Among the other features nShield supports are elaborate access control methods, which consist of granular Access Control Lists and management via split responsibility smart cards, where simultaneous permissions from several administrators are required for performing specific secured operations (basically the equivalent of Luna CA3's key-splitting feature described earlier), as well as key policies (linking key usage to factors such as time limits or the presence of token cards in the reader). Multiple modules installed on the same server can operate in a load-balanced or high availability configuration. Additional details about the nShield and its integration with Windows 2003 PKI, are available in a whitepaper published on nCipher's Web site.

Continued on Page 2: In Compliance

Continued from Page 1

Luna CA3's and nShield's high degree of security is evident in the products' compliance with with Federal Information Processing Standard 140-2 (FIPS 140-2), which is considered to be one of the most reliable industry recognized benchmarks defining security requirements for cryptographic modules. FIPS ratings are assigned after stringent testing in independent, accredited laboratories). FIPS 140-2 is the current standard (superceding FIPS 140-1) assigned through Cryptographic Module Validation Program and offered jointly by two government organizations: National Institute of Standards and Technology (NIST) and Communications Security Establishment (CSE).

FIPS is significant because its specifications define requirements sanctioned by federal agencies in both the United States and Canada, which practically guarantees a sufficient level of protection in any security-critical environment. FIPS 140-2 consists of four levels of assurance (with Level 4 the highest). Details and specifications of FIPS certification are posted on the NIST Web site. Note that all Cryptographic Service Providers included in Windows 2000 and Windows 2003 Server Certification Authorities are Level 1 FIPS compliant. When combined with products from Rainbow-Chrysalis or nCipher, their validation increases to Level 3.

In the past, HSM architectures have allocated one module for every system that requires encryption. This negatively affected capital cost as well as complicated administration and maintenance. Although this design is still viable for critical applications (where dedicated cryptography modules might be required), savings can be realized by employing a network-attached hardware security module (such as from nCipher's netHSM or Chrysalis' Luna SA), which can be shared across a number of systems. While the products are similar in many aspects, there are also some significant differences between them. In both cases, communication between shared HSMs and clients takes place over the network and is limited to systems registered manually with the device's management utilities. In addition, HSM device clients must have appropriate Cryptography Service Providers software installed on them.

Luna SA is an Ethernet-attached, 2U (19") rack-mountable appliance. It is capable of storing Windows 2000 and Windows 2003 Certificate Authority keys as well as SSL acceleration (for Web and ISA servers), handling up to 1200 SSL connections per server, and servicing up to 10 servers (with a combined capacity of up to 3600 transactions per second) using RSA 1024-bit keys.

netHSM is also an Ethernet-attached appliance. It boasts a slimmer (1U) rack-mountable form factor. The 1600 model offers up to 1600 transactions per second using RSA 1024-bit keys. netHSM, like nShield, implements Security World, which greatly simplifies the remote administration of multiple (both dedicated and shared) nCipher hardware security modules. This makes it easier to integrate netHSM into an environment containing other nCipher HSMs, as investments are protected and a unified management approach is accommodated. Security World also facilitates secure and unattended backups of all key materials. Another advantage of a netHSM-based infrastructure is its resiliency. By installing multiple network-shared devices, load balancing and failover capabilities can be implemented.

Although Security World is nCipher's proprietary technology, Luna SA offers several features described in the previous paragraph, albeit in a different manner. High availability (including failover and load balancing) is accomplished through implementation of its Ultimate Trust Security Platform (UTSP) solution. Luna SA devices are limited to 80 key objects, and Rainbow-Chrysalis offers External Identity Management, which grants stored keys additional layers of encryption protection in an external database (equivalent to capabilities of nCipher products). Luna SA can be managed remotely using Secure Command Line Interface over a network or via a local console port. However, without an additional backup key protection system, backups are performed manually, using tokens. In some cases, multiple tokens might be required to back up all key material on a single device.

As of press time, only netHSM had obtained FIPS 140 certification (i.e., Level 3); Chrisalis-ITS product's validation is pending. One possible security issue might result from the fact that although netHSM uses a hardened and cryptographically protected operating system with a strictly controlled user interface, without root-level access, Luna SA uses a standard Linux operating system. It is thus vulnerable to rogue software installations (via root account).

One final consideration is pricing. The advantages of netHSM are reflected in its base and connection license prices. However, when considering the actual operational cost of the Luna SA solution, the difference may turn out to be less significant. For more detailed information, refer to the appropriate sections of Rainbow-Chrysalis' and nCypher's Web sites.

Feature courtesy of ServerWatch.

» See All Articles by Columnist Marcin Policht