Have You Checked Your SMTP Mailbox Temporary Tables Lately?

We have an Exchange Server 2003 environment with a front-end and back-end infrastructure. There are dedicated bridgehead servers for incoming and outgoing email traffic. Symantec Mail Security 4.6 for Microsoft Exchange is installed on all our Exchange servers. Recently, I was troubleshooting another problem and discovered that one bridgehead server was holding numerous emails in the SMTP mailbox temporary tables. When the bridgehead SMTP service was re-started, users notified our Support Center that they were receiving old email messages. So, I initiated a call with Microsoft Product Support Services (PSS). When I was trouble-shooting this problem with PSS, I discovered that whenever event ID 348 occurred (which has the error message A message could not be virus scanned – this operation will be retried later), email messages became stuck inside the SMTP mailbox temporary tables. Although this event came from Exchange, I later learned that when Mail Security can’t scan email messages, it logs those events and the messages are stored in the SMTP mailbox temporary tables. If you’re unfamiliar with the SMTP mailbox and other special Exchange Server mailboxes, Evan Dodds has done an exceptional job explaining them in the three-part “Exchange Special Mailboxes” blog at http://blogs.technet.com/evand/archive/2004/12.aspx.

When email messages are stuck in SMTP mailbox temporary tables, you can’t use Outlook or other email clients to view them. You have to use a tool called Microsoft Foundation Classes MAPI (MFCMAPI). The Microsoft article “MFCMAPI demonstrates MAPI client code” (http://support.microsoft.com/?kb=291794) contains information about this tool as well as a link to download it.

MFCMAPI doesn’t have good documentation. So, if you want to look at what’s inside your SMTP mailbox temporary tables, follow these steps:

1. Download MFCMAPI and copy it to your Exchange server. Launch the tool and let it create a MAPI profile for you. Don’t try to create this profile manually.
2. Select Session and choose Logon and Display Store Tables.
3. Select MBD and choose Get Mailbox Table.
4. Enter the name of your Exchange server and click OK. If you’re checking a bridgehead server, the next window will show up quickly because there are only three mailboxes. If you’re checking a mailbox server, it might take a long time to open because MFCMAPI will display all the mailboxes.
5. Double-click the SMTP (<Server Name> (GUID)) mailbox, where Server Name is the SMTP server’s name.
6. Expand the Root Container. Under the Temp Tables folder, you’ll see the temporary tables, the number of which depends on how many SMTP virtual servers you have on your SMTP server. They’ll be marked TempTable#1, TempTable#2, and so on. Check each table for email messages. You might see messages in the temporary tables for a short period of time, but you shouldn’t see any messages sticking around in those tables. If you do, you might want to call PSS and discuss the problem. Note that you should be careful using MFCMAPI because any items you delete, whether intentionally or accidentally, will be permanently deleted.

I worked with PSS and Symantec representatives to make sure future email messages didn’t get stuck in the temporary tables. Following the instructions in the Symantec article “Error: ‘A message could not be virus scanned - this operation will be retried later.’ Event ID 348” (document ID 2004102615323554), I ended up making a couple of changes in the registry. Basically, I increased the timeout period and the number of input threads available to the Mail Security process. After the registry changes, I re-created the Exchange database on our bridgehead server. Because this database didn’t have any user mailboxes, I didn’t have any problems. (Had the problematic Exchange database been on the mailbox server, I would have given a lot more thought before deciding to re-create the database.)

Nowadays, whenever I restart the SMTP service, I always check the event log for event ID 348 and the SMTP mailbox temporary tables for stuck email messages. I haven’t seen either, so it appears the problem has been fixed.
—Brian Ko