Closing In on Clusters

Let's say you decide to build your intranet using Microsoft SQL Server on one Intel-based server and Internet Information Server (IIS) on another. Suppose the IIS server dies and, within a few seconds--no user down time--all the IIS users and processing automatically switch to another server. What if Windows NT Server had that failover capability tightly integrated, and this capability required no proprietary hardware? Interested?

Or, suppose you convince your CEO that NT really can scale. You put a 100GB SQL Server database on your four-way Pentium Pro system to serve 5000 users, and it runs out of gas. Where do you go from there? Do you need to look at an eight-way parallel system, or can you just add another four-way Pentium Pro server and have it work on the same database?

Imagine you can keep your system running while you're applying updates to the operating system or key application software. For example, you can wait until an off-peak time and move an application easily from one server to another and then apply a new version or service pack to the offline server and run some tests. When you're comfortable with the change, you can move the application back, testing as you go. If a problem occurs, you can easily move the application over again. All this while, users are up and running the very application you're updating. Is this an administrator's dream?

Okay, now that you've seen a few scenarios, let me formally define clusters, the technology that allows such solutions. A cluster is a group of whole, standard computers working together as a unified computing resource that can create the illusion of being one machine--a single-system image. The term whole computer, which is synonymous with node, means a system that can run on its own, apart from the cluster.

Clusters have been addressing the problems of availability, performance, and scaleability for years. Until now, however, cluster vendors have focused on serving high-end customers, ignoring high-volume server markets. Gregory Pfister, a senior technical staff member at IBM's server group in Austin, Texas, wrote In Search of Clusters: The Coming Battle in Lowly Parallel Computing. In this book, Pfister identifies three keys to making clusters a high-volume solution: the speed of microprocessors, the availability of standard high-speed communications, and the existence of standard tools for distributed computing.

Pfister believes all three requirements, especially the speed of microprocessors, are now met. The time is ripe to move clusters into the high-volume market, like the one NT serves. The current crop of NT microprocessors--Intel, Alpha, MIPS, and PowerPC--are as fast as the CPUs in the largest computers. One example is Intel's new standard, high-volume (SHV) system, a four-way Pentium Pro-based motherboard that Intel developed to take advantage of NT's symmetrical multiprocessing (SMP) capabilities.

Intel is planning to create complete SHV systems for some OEMs, who will change only the machine's faceplate. With Intel in the system business, you can expect these high-end machines to roll out from more than 20 vendors within a few months, creating a ready-made SHV market for clusters.

Pfister predicts that vendors will greatly profit from developing a high-volume cluster market if they can take advantage of the timing and solve a few fundamental problems, such as creating a single-system image, forming a standard so that software vendors aren't locked into one hardware vendor's solutions, and pricing cluster software licenses so that the cost doesn't exceed that of large parallel systems. Solving these problems could drive a tenfold increase in the quantity of cluster solutions shipped each year.

To appreciate the significance and implications of clusters in the NT world, you can look at the work Microsoft and its partners are doing on NT clusters. You need to know about Microsoft's emerging Wolfpack cluster standard, get a perspective on what various cluster solutions can do, and understand what various vendors are presenting to the NT market. For some background on the book that predicted the direction of clusters, see my review of In Search of Clusters and my interview with Pfister.

What Is Wolfpack?
Several leading NT Server systems vendors, including Compaq, Digital Equipment, HP, NCR, and Tandem, have been independently working on clustering solutions for a few years. These vendors agreed to pool their expertise with Microsoft in an initiative to produce a cross-vendor standard for NT Server clusters. This group wanted to give NT Server customers the greater choice and flexibility they wanted. So in October 1995, Microsoft announced its intent to develop strategic partnerships to fashion a new clustering standard with the code name Wolfpack.

This name and many of its technology goals derive from Pfister's book. In Chapter 4, Pfister describes a cluster as a "pack of dogs." While searching for a code name for the API, Microsoft came across this book and decided to describe clusters with the name Wolfpack, which sounds a lot cooler than Dogpack.

Wolfpack is an alias for clusters, and the six core vendors in Microsoft's clustering project consider themselves members of the Wolfpack. These members are Digital, Compaq, Tandem, Intel, HP, and NCR. Each partner contributes key components of its existing technology. Other vendors, including Amdahl, IBM, Octopus, Vinca, Marathon, Stratus, and Cheyenne, have agreed to support the Wolfpack API. These vendors are part of Microsoft's Open Process, which includes about 60 vendors and customers who are part of design previews during various stages of Wolfpack development.

Wolfpack describes a set of cluster-aware APIs, NT cluster support, and a clustering solution (which means a vendor can claim to be Wolfpack compliant while competing with the Wolfpack solution on a different level--so if a vendor claims to support Wolfpack, you need to ask how). Here's a detailed explanation of each Wolfpack component.

Wolfpack: The API
You can make applications cluster aware by calling the Wolfpack API. The services the API accesses can speed recovery; let you take additional actions, such as proactively notifying users on failover; let you restart and reacquire nonstandard resources; and let you monitor and detect more subtle application faults than a simple crash or lock-up. Potentially, Wolfpack API services will let applications achieve higher scaleability and do dynamic load-balancing on a cluster. Microsoft has not yet announced details of how the BackOffice applications will exploit the Wolfpack API services to become cluster aware.

Wolfpack: The Cluster Support
Cluster support will make all NT Server applications Wolfpack compliant in the sense that they will run exactly the same on a server that has Wolfpack as on a non-Wolfpack server, and Wolfpack will be able to do basic failover recovery of any NT Server application, without any need for you to modify the application. Wolfpack handles failover of an unmodified application by executing it through a provided wrapper dynamic link library (DLL) that notifies the cluster manager of the application's existence and creates a basic heartbeat (a check-in on the other cluster machine and its answer, at regular intervals) so Wolfpack can tell whether the application goes down or locks up. The pricing and packaging of Wolfpack is not set, but I can imagine Microsoft adding cluster support to NT in the same way that NT includes SMP support today.

Wolfpack: The Solution
Microsoft will deliver Wolfpack, the solution, in two phases. Phase 1 is two-node availability and scaling clusters (a new version of SQL Server will let you work on the same database from two servers at once). Phase 2 will allow more than two nodes in a cluster.

Reread the first paragraph in this article. That scenario describes a June 1996 demonstration of a Wolfpack availability cluster solution at PC Expo in New York City. This two-node failover capability is the basis for Phase 1 of Wolfpack (early 1997 is the estimate for delivery). The price for Wolfpack's Phase 1 release is not set, but one rumor is that NT Server will include Wolfpack at no additional cost. As I write this article, Compaq, Digital, HP, NCR, Amdahl, Stratus, and Tandem have all announced plans to OEM the Wolfpack-based cluster solution.

The next step in Phase 1 (set for the second quarter of 1997) will be an open certification program with the goal of expanding the market for two-node cluster solutions and giving NT Server customers a greater selection to choose from. Microsoft is also committed to making Wolfpack available on Intel, Alpha, PowerPC, and MIPS chips.