Unfortunately, many of the performance sizing tools for Exchange available from Microsoft and others don’t take into account the peaks and valleys that a typical storage solution encounters during regular usage. For instance, snapshots and backups can consume a significant amount of storage and many of the benchmarks available do not take these extra data protection activities into account.
In poor storage implementations, Exchange responsiveness can slow and even stop during the backup window. Therefore, it’s safest to be conservative in the number of IOPS per mailbox and not stray below 1.0.
Keep Exchange Virtual Machines Clean
In addition to the memory hit, the only other potential gotcha of virtualization involves the tendency by IT organizations to want to load additional virtual machines (VMs) on the same physical servers as the Exchange server. To many folks, it seems natural to combine collaborative applications such as Microsoft Office SharePoint Server, Office Communications Server, and Exchange 2007 on the same virtual server infrastructure and physical servers.
It’s difficult, however, to accurately size physical resource requirements for these applications because they each have dramatically different performance requirements. Since modeling the performance of different application characteristics is complex, time consuming, and error-prone, it’s best to keep Exchange virtual servers deployed on a physical host dedicated to Exchange.
Disaster Recovery and Business Continuity
Finally, many organizations are considering virtualization for use in email business continuity and disaster recovery scenarios. The ease of copying VMs to a remote site enables a relatively rapid “full fidelity” recovery of Exchange and makes not only the OSs, but the AD infrastructure and the storage servers available on a moment’s notice. Although replication technologies such as VMware’s VMotion aren’t recommended by Microsoft for use with Exchange 2007 as Microsoft hasn’t thoroughly tested them, these technologies should someday make site-to-site failover a simple reality.
In the meantime, customers are looking to storage-based replication solutions to mirror virtual servers and storage to secondary sites. Even when automated replication technologies for Exchange become available, in many cases a manual process will still be required to change critical DNS (e.g., MX) records and AD infrastructure to support the secondary site recovery in the event of a failure.
Lessons from Real-World Scenarios
Migrating from Exchange 2003 to Exchange 2007 carries with it a significant increase in datacenter power utilization. For example, two representative customers whom I’ve recently worked with on virtualization projects calculated that the additional servers required to support Exchange 2007 would drive power utilization in their datacenter from the pre-Exchange 2007 levels of 80 percent to 90 to 95 percent capacity. Because of the number of servers required and power limitations, it was impossible to add the servers necessary to migrate to Exchange 2007, while maintaining the current infrastructure.
By deploying a virtual infrastructure and leveraging their existing SANs, these companies were able to not only deploy Exchange 2007 but also reduce their power consumption through server consolidation. In addition, they now have a virtual server platform that can support their rapid growth projections over the next couple of years.
Both companies virtualized their Exchange 2007 system by using two quad-core processor servers, each with a minimum of 8GB of RAM, and redundant local hard drives running VMware ESX 3.5 as host machines. They mounted two Windows Server 2003 R2 64-bit OSs as guest machines from their SAN via iSCSI connections.
These two guest servers run as a mailbox server and a combined client access/hub transport server respectively. The physical servers use storage on their SAN to hold mailbox and public folder content, and also act as a quorum disk for the highly available mailbox server role.
Using the above configuration, each customer supports up to 500 mailboxes. Each has been running its Exchange infrastructure in production for over nine months.
What You Can Expect
Scalability claims vary around the number of mailboxes supported for Exchange 2007 running in a virtualized environment. In reality, Exchange 2007 exhibits the same storage resource constraints as prior releases of Exchange: Bottlenecks are typically due to I/O constraints on the storage solution.
In a typical 1,000-mailbox deployment running at a moderate email user configuration, with 10 percent or 100 mailboxes being accessed via higher impact BlackBerry devices, and supported with appropriate storage, the following profile is consistent for an Exchange 2007 mailbox server: • CPU utilization is low • Memory utilization is low to moderate • Storage utilization continues to be the key bottleneck, but is still moderate • Network traffic typically remains constant
Although this scenario isn’t indicative of every environment (since email loads and usage patterns vary widely), what is noteworthy is the low load level on the physical server and the available unused capacity. This is also the case with the client access and hub transport servers. Just as with the mailbox server, CPU utilization and memory utilization are typically very low, while storage I/O and network utilization remain low to moderate.
As for storage I/O and network utilization, which is moderate to high at peak usage in this configuration, they can both be addressed by adding additional storage drives to a SAN or managing the network more efficiently if these issues create a scalability problem.
As with any IT infrastructure, virtual servers must be managed. Loading too many VMs on a physical server will cause truly bizarre events and activities (such as the bouncing of client connections to Exchange forcing re-logins every minute or scheduled backups maxing out I/O, making the server inaccessible to new clients) to occur in network, storage, memory, and other physical resources when these are throttled arbitrarily by a virtual platform.
Appropriate sizing is a key requirement when deploying Exchange 2007 on a virtual platform, as are management tools. Lately I’ve been seeing Hyper-V achieve a slight edge over VMware on performance with Exchange, but customers should still choose their virtualization platform strategically based on their entire corporate virtualization roadmap.
One area that distinguishes the two platforms is management—VMware’s product is still much richer in capabilities in this area. To manage virtual and physical servers in the Hyper-V world, you will require either a separate Windows Server 2008 or Microsoft Systems Center Virtual Machine Manager.
Virtualizing Exchange Can Make a Difference
With the above caveats, virtualization is a safe and effective alternative platform for the successful operation of Exchange 2007. Quite a few companies are running full production deployments of Exchange 2007 in a virtualized environment. Although running on a virtual platform might not be suitable for all environments (such as smaller non-redundant Exchange architectures), it’s ideally suited and increasingly recommended for deploying Exchange 2007 in a highly available configuration.