More and more organizations are instituting policies that require security and vulnerability assessments on servers and systems before they're placed on the network. Many of these policies demand that you use auditing tools to scan your servers for exploits, confirm patch levels, provide security lockdown tips, and so on. For Exchange Server 2003 and Exchange 2000 Server email servers, a vulnerability that you need to take just as seriously is the open relay.
What is an open relay? An SMTP feature permits one server to act as an intermediary and accept messages on behalf of the final destination. The server then retransmits (or relays) the messages to the destination. In Exchange, you can configure servers to permit specific machines and users to relay mail. However, when configuration problems arise and anyone can use the system to relay mail, the result is an open relay. An open relay is dangerous for at least two reasons: First, SMTP is inherently unsecure, which makes it easy to spoof or forge the sender in a message. Lately, with the proliferation of viruses such as Bagel, we've seen how forging the sender can cause confusion and increase workloads. An open relay compounds this problem because a forged message sent from an open relay adds a certain level of authenticity—the message appears to have originated from a server in the specified sender's domain.
Second, spammers often use open relays to propagate their payload. When a spammer uses an open relay, your server's resources are diverted from processing your organization's mail traffic. Not only are resources diverted and your mail delivery hampered but you also run the risk of losing business in the long term. Using spam-blocking software and blacklists such as MAPS (http://www.mail-abuse.com) can result in the rejection of all legitimate mail from your domain because of its reputation as an open relay. An open relay can inhibit your organization's ability to do business and jeopardize its reputation.
If open relays are such a threat, why would you want to let your server relay mail in the first place? Generally, you permit relaying when you have an application that needs to send SMTP mail but doesn't have the capability to determine how to get the message to the destination. You configure the application to give the message directly to your relay server, and the relay server uses SMTP routing to deliver the message to its recipients. Examples of these types of applications are POP and IMAP clients, or a Web server form that sends email confirmations when information is uploaded. In some situations, such as designing a Web server application, you might be able to instead engineer the application so that it uses Messaging API (MAPI) instead of SMTP, thereby foregoing the need for relaying. But trends have been leaning toward standard protocols (e.g., SMTP)—not away from them.
In the case of POP and IMAP, relaying is necessary because POP and IMAP aren't intended for sending mail but rather for retrieving mail from mailboxes. To send mail, the POP and IMAP protocols must be paired with SMTP and the client must be configured with the name of a server that permits relaying. Providing relay support for these applications doesn't mean that your server will necessarily become an open relay. When you need to use relaying, your goal must be to configure an Exchange system to allow relaying only from authorized senders. To determine who is authorized, you either need to identify a system by its IP address or authenticate a sender via logon credentials. In the sections that follow, I detail how to configure a server to act as a relay without putting the server at risk for exploitation.