I've correctly configured a Microsoft SMTP server for IIS 5.0, but the mail stays in the Queue folder and isn't delivered. The error message an internal DNS error caused a failure to find the remote server is appearing in Event Viewer. All the NTFS permissions are correct, and I've configured the SMTP service to use the correct NIC. When I use Telnet to access port 25 on the SMTP server, the server responds, so I know the SMTP service is working. I'm not using a smart host, and I can ping the DNS server as well as use the Microsoft Windows 2000 Resource Kit's nslookup.exe tool to resolve the domain names for the mail. Why isn't the server delivering mail?
You seem to have covered almost all the bases. However, one other gotcha can cause this problem, and you're a good candidate for it because the server seems to be working. The DNS protocol specification, which Internet Engineering Task Force (IETF) Request for Comments (RFC) 1035 spells out, indicates that you must use port 53 on the DNS server. For you to successfully resolve the domain name in the email address, proxy servers and firewalls must be able to pass DNS queries over port 53. DNS resolution is working on your system or you wouldn't be able to ping the domains to which you're sending mail. However, the largest percentage of DNS queries uses UDP on port 53. Although TCP is more reliable than UDP, UDP is a connectionless protocol and thus more efficient. (For information about connectionless protocols, see the sidebar "Connectionless Protocols.") Thus, because traffic to and from a DNS server is brief and the packets are small, UDP is the preferred means of communicating with the server. Preferences aside, however, RFC 1035 states that a DNS server should respond to both TCP and UDP DNS queries.
As a safety measure, security administrators will often block TCP DNS traffic across a firewall, proxy server, or at the DNS server itself. This block is where the problem lies because, by default, the Microsoft SMTP service uses TCP for DNS traffic.
You can edit a metabase setting to switch the default behavior of the SMTP server to use UDP, but I've been unable to verify the exact setting. I do know that Microsoft doesn't support the configuration. You should be able to configure your firewalls and proxy servers to allow TCP traffic on port 53.
Another (and perhaps more likely) reason that you can't send mail with the SMTP server is that most large companies restrict outbound SMTP at their firewalls. Many companies control outbound SMTP by specifying which servers can forward outbound mail. So, you need to configure IIS's SMTP properties to relay the mail to another SMTP server that has rights to send email messages to the Internet. To do so, open the SMTP server's Properties dialog box, click the Delivery tab, then click Advanced. Enter the IP address of the corporate mail server in the Smart host field.