Some research companies, notably IDC, predict that nearly 100 percent of Internet traffic will be encrypted by 2005. Although a portion of this traffic will consist of credit card transactions, pretty good privacy (PGP)–encrypted email, and encrypted file transfers, most will be protected because of the increasing use of VPNs—between company sites, business partners, or the office and employees' homes. Many companies that use Microsoft Internet Security and Acceleration (ISA) Server 2000 as a firewall and proxy server ask me whether they can also use the product to establish an Internet VPN. The product's robust and useful wizards can indeed help you quickly establish a client-to-gateway or gateway-to-gateway VPN. Still, setting up an ISA Server VPN involves many steps as well as Certificate Services, DHCP, DNS, and RRAS, so the process is more complex than just setting up a firewall.
VPN Tunneling
If you've never set up a VPN, the way in which they work can seem a bit awkward. First, you must use hardware or software to establish endpoints—one or more VPN clients and a VPN server. You typically can establish a VPN's physical and data-link layers (i.e., Layers 1 and 2 of the Open System Interconnection—OSI—model) over a dial-up line or a high-speed dedicated digital line. The endpoints don't need to use the same vendor's solution, but they must use the same tunneling protocol. Most VPN implementations use Layer Two Tunneling Protocol over IP Security (L2TP/IPSec), as the sidebar "ISA Server VPN Protocols," page 2, explains, and can partner with one another, but interoperability conflicts still abound, especially for ISA Server VPNs. (Most vendors' L2TP implementations vary at least slightly, like different dialects within a language.) The key to interoperability is that both endpoint solutions must support the same IPSec protocol and configuration options. (See the Web-exclusive sidebar "IPSec Protocols and Modes," http://
www.winnetmag.com/windowssecurity, InstantDoc ID 40596, for a discussion of IPSec protocols.)
You can configure a client-to-gateway VPN between a client and a server, or you can configure a gateway-to-gateway (aka site-to-site) VPN between two or more VPN network endpoints (servers or clients). ISA Server's two most common Internet VPN scenarios are between a Windows PC client and an ISA Server (client-to-gateway) and between two ISA Servers (gateway-to-gateway).
Clients at either end of a gateway-to-gateway VPN maintain their own identities and traffic domains, but remote clients on a client-to-gateway VPN become virtual hosts on the VPN server's network. The process of establishing a VPN connection often assigns these clients new IP addresses, subjecting the clients to the same traffic as physical hosts on the network. Therefore, remote clients often lose their local networking services and might not be able to connect to their local servers, printers, or Internet services. This type of configuration is known as a tunnel-mode VPN; remote clients in a tunnel-mode VPN have access only to network servers and printers and often access the Internet through the VPN. You can configure your DHCP server or RRAS to assign valid network IP addresses to remote clients. With the first method, RRAS gets a range of addresses from the DHCP to pass to the VPN clients; with the second method, RRAS assigns the addresses from its own DHCP server pool. If you need to pass advanced DHCP scope options to the remote clients, you must set up a DHCP relay agent on the RRAS computer. I suggest that you use a dedicated subnet for your VPN clients so that you can easily distinguish between VPN and LAN clients.
Alternatively, split-mode VPNs give remote clients simultaneous local and network access. Split-mode clients, however, can become unsecured gateways and can introduce rogue traffic into the VPN. Some VPN clients, including the Windows Network Connection VPN client, support both tunnel and split modes. The mode you choose depends on your objectives and your VPN solution.
The clients on either side of a VPN transmit unencrypted information; the VPN endpoints—typically routers, firewalls, or ISA Server systems, as Figure 1 shows—perform all encryption. The application in which the user created the data is usually unaware of the VPN. The VPN hardware or software decrypts the encapsulated data at the other end of the VPN, so if someone were to capture the encapsulated traffic, the most that person could read would be the IP header, and even that header usually isn't the original. IPSec protects TCP and UDP headers, source and destination addresses, and packet payload content.
To support L2TP/IPSec, you might need to open certain ports on any involved firewalls. You'll probably need to open UDP port 1701 for L2TP traffic, UDP port 500 for Internet Key Exchange (IKE) traffic, and UDP port 4500 for Network Address Translation Transversal (NAT-T) traffic if you use NAT-T. (See the Web-exclusive sidebar "NAT Transversal," http://www.winnetmag.com/windowssecurity, InstantDoc ID 40597, for more information about NAT-T.) Configuring IPSec connections on ISA Server creates the appropriate inbound and outbound packet filters for these ports, although you might need to open additional ports for additional client access (e.g., DHCP, DNS, NetBIOS). You'll probably also need to permit IP Type 50 Encapsulating Security Payload (ESP) packets and IP Type 51 Authentication Header (AH) packets. Also be prepared to use preinstalled machine certificates or to install certificates. (See the Web-exclusive sidebar "Certificate Authentication," http://www.winnetmag.com/windowssecurity, InstantDoc ID 40598, for information about certificates.) Now that you have a bit of background about how client-to-gateway and gateway-to-gateway ISA Server VPNs work, let's look at the steps involved in setting up these popular VPNs.