Whether you need low-, medium-, or high-security connections, IPSec policies do the job
Attacks that take place over the Internet receive a great deal of publicity, primarily because serious attacks often interrupt service to millions of users. Malicious users who prey on corporate data from inside a network receive much less fanfare but cause at least as much damage as Internet-based attacks. (For information about the frequency of internal network attacks, see the sidebar "Why You Need LAN-based Security," page 74.) Companies rarely publicize internal network abuse statistics; acknowledging such crimes is tantamount to declaring that corporate IT can't adequately secure sensitive information.
Windows 2000 provides two methods for securing access to sensitive data: tunneling protocols (which protect a TCP/IP packet by encapsulating it within another TCP/IP packet, much like placing a letter inside an envelope) and IP Security (IPSec). These methods can reduce and potentially eliminate many internal security violations, including data theft and unauthorized data modification. You can select PPTP or Layer 2 Tunneling Protocol (L2TP) as a tunneling protocol to secure client-to-client and client-to-server connections. The advantage of L2TP is that it interoperates with IPSec.
IPSec comprises an architecture and two protocols (Authentication Header—AH—and Encapsulating Security Payload—ESP) that you can employ to secure LAN and WAN connections in either a native Win2K or mixed-mode network. After you determine the requirements for a specific secure connection, you define an IPSec policy that implements your security requirements, then activate the policy on the computers at both ends of the connection. Win2K includes built-in policies to help you enforce low-, medium-, and high-security communications; these policies are sufficient for many common network security situations.
However, the built-in policies don't cover every common security scenario, so you also need to understand how to define and test a custom policy. See the sidebar "Creating and Testing a Custom Policy" for an example IPSec policy that mandates secure, encrypted communication between Win2K systems that aren't members of a Win2K domain. By the time you finish this article, you'll have the knowledge you need to implement secure IPSec connections for Win2K domain members, Win2K standalone servers, and Win2K servers in a Windows NT 4.0 domain.
How Does IPSec Work?
IPSec employs hash algorithms and public key cryptography to secure and protect network communication between two endpoints. In its most secure form, IPSec uses one key for signing and encrypting data and a second key to verify the message signature and decrypt the data. Together, signing and encryption can provide four types of packet protection: authentication, integrity, anti-replay, and—optionally—confidentiality.
First, IPSec authenticates and verifies every message's origin, which guarantees that the sending computer is a known entity. Second, IPSec guarantees data integrity, which means nobody can modify the data while it's in transit between the sending and receiving machines. Third, IPSec uniquely identifies each IP packet with a digital signature represented as a checksum, which prevents another user from reusing or replaying a message to establish a session (possibly with stolen credentials). Fourth, IPSec can optionally guarantee confidentiality by encrypting network packets, which ensures that only the person who has the key required to decrypt the message can read the message.
IPSec provides two protocols: AH and ESP. AH protocol authenticates the sender, guarantees data integrity, guarantees that no one can replay the packet, and signs each message with a digital checksum created from either a Secure Hash Algorithm (SHA) or an MD5 hash algorithm. (The hash algorithm generates the digital checksum that acts as a packet signature.) ESP protocol does everything that AH does and also encrypts data so that no one can view it during transit. ESP encrypts data with Data Encryption Standard (DES) or Triple DES (3DES) cryptography and optionally signs each message. You can combine these protocols to achieve the highest possible level of security for computer-to-computer authentication and data transfer.
When you deploy IPSec on your network, all computers and users must be known, verifiable entities. You can rest easy knowing that IPSec security features can eliminate much network abuse. Be aware, however, that when you configure a system to require ESP protocol and DES or 3DES encryption, the encryption and decryption processes introduce a 10 to 20 percent performance overhead on the secure endpoints. (For more information about how Win2K implements IPSec, see Tao Zhou, "IP Security in Windows 2000," http://www.win2000mag
.com/, InstantDoc ID 7976.)
What's an IPSec Security Association?
When you activate IPSec, the endpoints must establish a secure connection called a security association (SA) before they can securely exchange data. The SA's purpose is to verify each endpoint's identity—a process called mutual endpoint authentication—and to establish session keys. The endpoints employ Internet Security Association and Key Management Protocol (ISAKMP) to exchange the information they need to generate mutually agreed-upon keys. For security reasons, IPSec never sends the keys over the network; instead, IPSec sends the data that each endpoint needs to locally create the key. After the endpoints agree on encryption, hash algorithm, and authentication methods, they create one SA for inbound connections and one SA for outbound connections.
After the endpoints establish an SA, they can securely transfer data. An SA remains valid for 1 hour after the last packet transfer. If either endpoint tries to communicate with the other after the SA expires, the endpoints must create a new SA. You can override the expiration period and specify a different SA lifetime if the default doesn't suit your implementation. For example, on a very-high-security server, you might want SAs to expire only a few minutes after receipt of the last packet.
How Does IPSec Authenticate Endpoints?
To verify the identity of an endpoint, IPSec can use three different authentication methods: Kerberos, certificate, and preshared key. Kerberos is Win2K's default authentication method for computers and users. Kerberos authentication is the right choice if you're implementing IPSec in a native Win2K domain because you always have a Win2K domain controller to perform Kerberos authentication. In a native Win2K domain, you can safely activate Win2K's built-in IPSec policies because these policies require Kerberos authentication.
Several situations exist in which you can't implement IPSec with Kerberos authentication. If you install Win2K Advanced Server as a standalone server in an NT 4.0 domain, you have no Win2K domain controller and thus no Kerberos authentication service. The same is true if you're implementing IPSec to secure remote L2TP connections to a Win2K RRAS server (remote clients typically can't access a Win2K domain controller until they establish a valid SA). In both scenarios, you must configure IPSec policies to employ certificate or preshared key authentication.
If you opt for the certificate method, you need a Certificate Authority (CA) to assign valid certificates to computers and users. To successfully authenticate a certificate-based IPSec connection, each endpoint must have a computer certificate that verifies the endpoint's identity during the initial SA negotiation. Without a computer certificate, you'll never successfully establish an IPSec or L2TP/IPSec connection.
You can install Win2K AS's Microsoft Certification Authority, install a third-party CA, or use a public CA on the Internet. You can install Microsoft's CA in enterprise mode in a Win2K domain or in standalone mode on a Win2K server in an NT 4.0 domain. I recommend that you configure an installed CA to automatically issue computer and user certificates. The alternative requires that an administrator manually approve each request—
busywork we can all do without.
If you don't have a Win2K domain controller for Kerberos authentication and you don't have a CA, a preshared key is your only option for mutual endpoint authentication. Preshared key authentication requires both systems to pass the same text string during SA negotiation. Because the IPSec policy on each machine stores the string as clear text, this form of authentication is the least secure of the three. To protect the shared string from public eyes, you should restrict IPSec policy access to Administrators only on domain controllers and restrict access to System only on the computers where the assigned policy resides.
What's an IPSec Policy?
An IPSec policy defines the parameters for secure communication between the local system and other clients and servers on the network. Every policy must have at least one IPSec rule that specifies whether to use a tunnel, the type of endpoint authentication to negotiate, and—
optionally—the IP protocol to which the rule applies. The rule also specifies a filter, which provides TCP/IP source and destination addresses. (For information about TCP/IP filters, see Gary C. Kessler, Rik J. Dayvie, and Craig T. Martin, "The Power of Packet Filtering," January 2000.) Win2K includes a built-in rule, the <Default Response> rule, in the three native policies and in any custom policies that you define.
To view or edit a policy's rules, open Microsoft Management Console (MMC) and load the IP Security Policy Management snap-in for the local computer. Select IP Security Policies on Local Machine, as Figure 1, page 76, shows. The right pane lists the policies that reside on that machine. Client (Respond Only), Secure Server (Require Security), and Server (Request Security) are default policies that Win2K provides on every system. Each entry has a description and a Policy Assigned column that tells you whether the policy is active.
To display the IPSec configuration parameters for a policy—for example, the Client (Respond Only) policy—double-click the policy, then double-click the <Default Response> rule to display the Edit Rule Properties dialog box. Be forewarned: Unless you're an IPSec security guru, never modify the <Default Response> rule; every built-in policy uses this rule, so if you break it, none of the native policies will work.