Over the past several years, as email attacks against end users and their desktop systems have increased, direct attacks against email servers have decreased (although the decrease has been relative). However, servers are still vulnerable because attackers are still releasing exploits against Microsoft Exchange Server and even Sendmail. Let's look at a couple of common attacks and ways you can reduce or stop these attacks against your email servers.
Buffer-Overflow Exploits
A buffer overflow occurs when a software program, such as a mail server,
stores more data in a data buffer than was originally allowed for and no provision
exists for the unexpected input. Attackers can use this bug to make the mail
server execute other programs it was never intended to execute. If the mail
server runs at a privileged level, the entire system can be compromised. Even
if the mail server doesn't run in privileged mode, attackers can compromise
it and gain full access to its resources.
Although they can occur accidentally through programming errors, buffer overflows are a common security exploit against data integrity. In a buffer-overflow exploit scenario, the extra data can contain codes designed to trigger specific actions, such as sending new instructions to the attacked server that could damage user files, change data, or disclose confidential information.
In the past, attackers often used buffer-overflow exploits to enable the passing of worms between various servers on the Internet as well as to prove their prowess. More recently, however, buffer-overflow exploits have a more targeted purpose: They let attackers compromise a mail server so that they can then use the mail server to send spam.
This type of attack has two serious consequences. First, a compromised mail server means that attackers can read the email messages being sent to and from your company. The results can be devastating. Second, attackers can use the server resources of your company to send spam. This scenario can earn bad will for your company and violate your ISP contract, which often means termination of service.
It's important that you harden your mail servers (and any other public servers) against buffer-overflow exploits and other types of attacks. You can also take several other protective measures.
Server Hardening
The best way to reduce the chance of a mail server compromise is to harden
the mail server itself. In all situations, hardening is a worthwhile effort.
On hardened servers, especially Internetfacing servers, fewer services are available
for exploitation and those services are generally "compartmentalized." The following
measures are generally required for hardening:
- Physically securing the computer
- Updating OS and application software
- Enabling logging of administrative access and resource use
- Removing unnecessary applications, services, and tools
- Enabling local firewall services
- Restricting the use of privileged accounts
By hardening servers, you can dramatically reduce their vulnerability. Unfortunately,
hardening mail servers often isn't enough. A better solution is to both harden
the server and provide additional filtering for email traffic before it actually
hits the server. You can filter email traffic early by using network appliances,
managed services, and software integrated into an existing mail system (e.g.,
Microsoft Exchange). Keep in mind that you want to layer your defenses—for
example, by hardening internal mail servers while at the same time deploying
vendor-hardened network appliances to protect the perimeter.
Network Appliances
Mail-filtering network appliances are deployed in
front of internal mail servers. These appliances usually provide two types of
firewalls: a packet-filtering firewall and an application-level firewall. As
a packet-filtering firewall, a network appliance allows only valid TCP/IP traffic
to ports that mail services use (e.g., SMTP, often POP3 and IMAP). As an application-firewall,
the appliance ensures that the sending server properly uses SMTP and follows
relevant IEEE Requests for Comments (RFCs) and common practices (e.g., having
reverse DNS set up).
Network appliances tend not to be susceptible to attack for several reasons. First, most appliances run on heavily customized OSs. These OSs have been stripped of most extra services that would let attackers gain a foothold on the system (or the OS has been designed from scratch specifically for the appliance). Second, engineers typically follow best practices when hardening the appliance. Finally, an appliance permits only a limited set of traffic (i.e., traffic related to mail transport) to and from the mail server and even that traffic is carefully scrutinized.
Figure 1 shows a network appliance located
in front of an internal mail server. This placement lets the appliance protect
internal servers and lets you offload processing from the internal mail servers
to the appliance.