Exchange Ideas, December 2006

What approach should I take in developing an email-security strategy?
Because of the danger of viruses, Trojan horses, and spyware and because email is now the main attack vector, most organizations rely on multiple layers of defense. Those layers can include a packet-filtering firewall, an email firewall, and a demilitarized zone (DMZ) mail server.

The first layer of defense—and the layer that best protects the underlying network and provides a crucial level of protection for network-oriented applications—is the packet-filtering firewall. A packet-filtering firewall understands networks at the TCP/IP layer, including such matters as TCP, UDP, and ports. This type of firewall is configured to let only certain types of incoming packets through to specifically allowed ports on the internal hosts that the firewall protects. For example, a firewall might allow incoming packets on TCP port 25 on the DMZ mail server and TCP port 80 or TCP port 443 on the DMZ Web mail server.

The second layer of defense is an email firewall, one example of an application-level firewall. This type of firewall works at a higher level in the protocol stack. It not only understands SMTP but can scan the content of mail envelopes and mail content to detect spam, phishing attacks, and viruses. The email firewall is usually hardened against SMTP-based attacks (e.g., buffer-overflow attacks), so the DMZ mail server is less susceptible to such attacks. An email firewall protects email systems (i.e., computer systems that provide mail service) as well as providing a layer of protection for internal users from dangerous email messages in their mailboxes.

Note that email firewalls must provide comprehensive antivirus capabilities to properly defend against both known and unknown viruses. Much antivirus software has been reactive. However, because of how quickly viruses now spread and because many viruses are polymorphic, a reactive approach is no longer enough. Antivirus software must also provide predictive scanning, meaning that it should be able to perform heuristic scanning to detect key characteristics that identify a virus rather than needing to know an exact signature. Reactive scanning still has a place in virus defense, but real-time defense against zero-day threats requires predictive scanning from antivirus software.

The third layer of defense is a well-configured DMZ mail server. This server accepts only mail destined for the domains that it owns—that is, for internal users. This approach prevents spammers from using this mail server for relaying spam. The DMZ mail server is also hardened so that attacks that jump to it from the email firewall (e.g., invalid input that the email firewall accepts and passes on to the DMZ mail server) don't compromise it.

Finally, additional layers of defense can be beneficial, such as an intrusion detection system and a separate DMZ Web mail server. (Because Web mail servers usually run complex Web applications, they often provide an avenue for an attack that can compromise internal systems.)
Dustin Puryear