Microsoft Hyper-V was introduced in Windows Server 2008. This enterprise-ready virtualization solution provides true hypervisor-based virtualization that enables virtual machine (VM) performance that’s equivalent to running on native hardware. Hyper-V uses failover clustering as the mechanism to create highly available Hyper-V environments. This feature enables the ability to move a VM from one node to another with minimal downtime through quick migration, which essentially saves the VM memory and state to disk, pauses the VM, dismounts the LUN on the current VM owner node, mounts the LUN on the target VM owner node, reads the memory and state information back into the VM on the target, and then starts the VM. This process is fairly fast but typically makes the VM unavailable for about 30 seconds (or longer, depending on configuration), which causes users to be disconnected.
One of biggest shortcomings of Server 2008’s Hyper-V is its inability to move a VM between nodes in a failover cluster without any downtime. For Hyper-V to realistically compete against other virtualization solutions, it needed an overhaul in Windows Server 2008 R2 for its VM migration technology. This revamp required two major changes. First, it was necessary that a VM’s memory and state could be copied between nodes while the VM was still running. This change would avoid the long downtime associated with saving memory to disk and then reading from disk on the target machine. Hyper-V Server 2008 R2 introduced Live Migration to address this issue. Second, Hyper-V 2008 R2 removed the LUN dismount and mount operation, which was necessary to make the configuration and Virtual Hard Disk (VHD) available to the source and target nodes simultaneously. This article focuses on the second Hyper-V change in Server 2008 R2.
The Shared Nothing Problem
NTFS is a very monogamous file system. An NTFS volume can be accessed and used by only one OS instance at a time; it wasn’t designed as a cluster file system. However, NTFS is also a very powerful file system: highly secure, industry tested, and with a huge support ecosystem that includes backup, defragmentation, and maintenance tools, as well as many services that rely on features of the file system.
A cluster consists of multiple nodes, each running an instance of Windows Server. But when NTFS volumes are used in a cluster, how is the integrity of NTFS maintained, and how are multiple concurrent mounts of an NTFS volume prevented? In a failover cluster, shared disks (i.e., disks that all the nodes in a cluster have a path to—which means they’re typically LUNs on a SAN) are resources of the cluster. Like other resources in a failover cluster, these resources have only one owner at any one time. Therefore, multiple mounts of a LUN are blocked because only one node can own the resource. The resource owner mounts the LUN and can access the NTFS volumes stored in the LUN.
Consider Hyper-V using failover clusters—particularly VMs. This lack of sharing introduces several design considerations. Hyper-V uses disks to store not only the VHD for the VM but also for the various configuration and state files. If an NTFS volume can’t be accessed by more than one node at a time, any VMs that share a LUN for storage of the VHD and configuration must run on the same node. Moving one VM on its own that shares a LUN for storage with another VM isn’t possible; all the VMs sharing the LUN must move as a collective unit. This restriction means that in Server 2008, each VM must have its own LUN so that each VM can be moved independently of other VMs around the cluster, by dismounting the LUN and mounting on the new node. A limit of one VM per LUN means administrators must deal with a lot of LUNs, as well as a lot of potential wasted space, because each LUN is provisioned with a certain amount of space and room to grow.
In addition to extra management of multiple LUNs and the potential for a lot of wasted space, another problem with Hyper-V in Server 2008 is the act of dismounting the LUN from the current owning node and mounting on the target node when a VM needs to be moved. This dismount/mount operation takes a lot of time. The goal in Server 2008 R2 was to achieve a zero downtime migration solution for VMs, with no visible end-user effects. The Hyper-V team implemented Live Migration to transfer memory and state information without stopping the VM. However, dismounting and mounting the LUN that contains the VHD requires a pause of the VHD, which results in downtime. To prevent the need to dismount and mount during moves and to avoid having one LUN per VM, as well as to save space, prevent administrative headaches, and—ultimately—save money, it’s necessary to have a LUN that’s accessible by all nodes in the cluster at once.