Get back in production quickly and recover mail later
In the Windows 2000 Magazine article "Exchange 2000 Storage Exposed, Part 2" (August 2000), Jerry Cochran discussed the Exchange 2000 Server backup and restore process. Exchange 2000 improves on the Exchange Server 5.5 process because it lets you recover a single database while other databases in the storage group (SG) continue to service users. In a traditional recovery process, any users on a mailbox store are left without email while you recover the databases.
However, in some companies, you can't afford for your email system to be unavailable to any users while you're performing a restore. Let's look at a method for bringing an Exchange Server Information Store (IS) back into production quickly to let users back into the mail system immediately, effectively reducing downtime to zero. This method lets you focus on recovering existing mail as a background recovery process.
Reducing Downtime
You can reduce downtime in two ways: eliminate the cause of service interruptions or reduce the duration of an interruption that has already occurred. Let's assume that you've done everything you can to eliminate service interruptions by following the guidelines on topics such as hardware fault tolerance, clustering, and best practices for mission-critical applications. Nevertheless, you have a service interruption—one of your Exchange servers crashes and refuses to mount one of the stores in an SG. In Exchange Server 5.5, the IS service won't even start, but in Exchange 2000, the IS service starts even if one of the stores in an SG won't mount. You determine from the event log errors and by running Eseutil that the reason the store won't mount is that it's corrupted, and you must restore it from the last backup.
At this point, you typically would prepare for a full store recovery by tracking down the last full backup tape and additional backup tapes (unless you have the luxury of disk-based backup, which is becoming a popular option for those who can afford the extra hard disks). Regardless of your backup medium, the users whose mailboxes are on the affected store will have to wait while you recover the database files (the property database—.edb—files and the streaming database—.stm—files). I've found that outages last at least several hours, no matter how small the company size, or even all night, depending on the quality of the company's operational procedures and the experience of the people performing the restores.
However, you have another option: Restore service immediately and recover old mail and other items later. The advantage of this method is that users have immediate access to their mailboxes so that they can begin receiving and sending mail. What they won't have access to immediately is their old mail, calendar, tasks, and so forth. After you perform a background recovery, these items will show up in about the same length of time that users would ordinarily be waiting just for access to their Inbox.
This technique won't appeal to everyone, but the advantage of giving end users instant access to their mailbox might be worth the wait for the rest of the mail to reappear. In a POP3 email environment, the recovery might be totally invisible. Instead of receiving a Server not found error message while you're recovering the IS, the POP3 email client just won't receive any new mail until the phased recovery is complete. So, let's see how to perform this feat.
Prepare Your Environment
Well in advance of the disaster and recovery, you need to set up your recovery environment. At a minimum, you need two computers: an Exchange 2000 computer in the same domain name but in a second Active Directory (AD) forest, unrelated to your production AD, and a workstation—the "recovery console"—to perform the recovery operations. The workstation must be a separate computer for two reasons. First, you must change its domain membership during the recovery process. Second, you can install and test the recovery utilities on the separate machine before you begin. In some cases, the utilities don't run on the recovery Exchange server after you install all the applications and service packs.
For the recovery, you can use one server only if the server has sufficient processor power, memory, and storage capacity to run Exchange 2000 and act as the Win2K domain controller (DC). The Exchange server must have adequate disk space to write out the IS from backup and extract the mailboxes to Personal Folders (.pst files). Therefore, you must allow more than twice the size of the original database files: one for the databases, and another for the .pst files, which also break the single-instance ratio.
On the recovery servers, install Win2K and the same service pack as the production mailbox server has. You'll need a Win2K DC to hold AD and the user accounts, which the Mailbox Reconnect tool will extract. To promote the server to a DC in the new AD forest, you must install the prerequisite DNS. I prefer to configure DNS to be a secondary zone copy of the production primary DNS zone, but you can always make manual DNS entries, if needed.
The next step in setting up your environment is to install on the recovery workstation the necessary Win2K components (i.e., Network News Transfer Protocol—NNTP—and SMTP) for installing Exchange 2000. Install Exchange with the same organization name as the production system has. Apply the same service packs and hotfixes as the production Exchange system has. Make sure that the Exchange server has the same organization name and administrative group name as the production system, or the recovery won't work. The Exchange server name, however, doesn't have to match the mailbox server, so you can easily do this in advance. If your backup software uses an Exchange agent, add the service account to the local administrators group on the recovery mailbox server. This account typically is in the production domain, so you must type in the domain name; you can't resolve the name with the recovery domain.
On the recovery console/workstation, copy the Exmerge utility from the Exchange 2000 Server CD-ROM's support\utils\i386\exmerge directory. Exmerge requires that exchmem.dll be in the same directory or system path. Copy the Mailbox Reconnect tool (mbconn.exe) from the support\utils\i386 directory. I explain these utilities in more detail later.
Finally, make sure that you have a distribution list (DL) that contains all users on a particular IS. You'll see why this is necessary in a moment. Although you can create and manage the DL in several ways, the easiest way is to organize your users by organizational unit (OU) because you can right-click the OU and choose Add members to a group.