Load Balancing Your Disks
Throwing more hardware at a bottleneck in the disk subsystem isn't always effective. By load balancing your existing disk subsystem, you can capitalize on your investment and better understand when and where to add disk hardware, if needed. The key to load balancing your NT Server disk subsystem is to identify which disk device is the bottleneck, understand the characteristics of the application using the disk device, and determine which files on the disk device the application is using. Performance Monitor is an excellent tool to help you identify which disk device is the bottleneck. (For information about using Performance Monitor to improve disk I/O, see "The Beginner's Guide to Optimizing Windows NT Server, Part 2," August 1997.) Unfortunately, Performance Monitor alone isn't enough. To understand how and where to distribute your disk workload, you must know which applications (processes) access which files on the disk device.
After you identify which disk device is the bottleneck, you can use two techniques to help you isolate the files on the disk device your applications use. The first technique involves using NT's internal auditing feature (available under Administrative Tools/UserManager/Policies/Audit) to alert you when applications access specific files. This technique is helpful, but the auditing feature generates and sends output to the Event Monitor and can be challenging to decrypt. Auditing also adds a lot of overhead to both the CPU and the disk subsystem selected for auditing.
The second technique, which I find easier to implement, is using Systems Internals' freeware tool, Filemon.exe, which is available at http://www.sysinternals.com. After you download this utility, you just unzip the file and run the utility during high usage periods to get a feel for the specific files your applications are accessing on the disk subsystem, as Screen 1 shows.
Filemon gives you the data you need to load balance your disk subsystem. As Screen 1 shows, you can determine which process is accessing which part of the disk subsystem. By using Filemon with Performance Monitor, you can decide which disk devices to move files from (disks with a high % Disk Time--a combination of read time and write time), which files to move, and where to move them (disk devices whose % Disk Time is low).
Not only does Filemon tell you which applications access which files on which disk devices, but it can also show you whether the disk requests are a read or write activity. As a result, you have more granular data to use when deciding which RAID level to use or when adding standalone disks.
Be careful when you move files around to different volumes under NT. Make sure that the shares and permissions are properly set after you move the files. If the files are associated with a specific application, you might need to move the files using the application or update the Registry. You can determine how you move the files in the disk subsystem according to the applications running in your environment. Regardless of your environment, you will want to have a complete system backup available to restore from if necessary. I also suggest that you make sure no active users are on your system when you make changes to your disk subsystem; otherwise, frustrated end users might complain. Always remember to test your application before and after you move any files to ensure you don't accidentally break anything.
Tuning the Allocation Unit Size
NTFS uses clusters as the fundamental unit of disk allocation. A cluster consists of a fixed number of disk sectors. When you use the Format command or NT's Disk Administrator, clusters are known as the allocation units. In NTFS, the default allocation unit size depends on the volume size. Using the Format command from the command line to format your NTFS volume, you can specify a variety of allocation unit sizes for a specific NT disk volume. For example, to format a volume using an 8KB allocation unit size, go to the command prompt and type
Format k: /f:NTFS /A:8192
For information about the Format command, go to the command prompt and type
Format /? | more
The default allocation unit size that NT provides is a great place to start if you're unaware of the disk workload characteristics. Before you set up a RAID array or new standalone disks, you need to determine the size of the average disk transfer on your disk subsystem and set the allocation unit size to match it as closely as possible. By matching the allocation unit size with the amount of data that you typically transfer to and from the disk, you'll incur lower disk subsystem overhead and gain better overall performance. To determine the size of your average disk transfer, use Performance Monitor to review two counters (Avg. Disk Bytes/Read and Avg. Disk Bytes/Write) under the LogicalDisk object. The Avg. Disk Bytes/Read counter measures the average number of bytes transferred from the disk during read operations, and the Avg. Disk Bytes/Write counter measures the average number of bytes transferred to the disk during write operations.
After you measure the number of bytes written to and read from your disk subsystem, you can set the allocation unit size so that you can achieve maximum performance. Of course, if you want to change the allocation unit size for a file system on the disk device, you have to back up the data from the disk device, use the new allocation unit size to format the disk device, and restore your data. Just as you need to use multiple RAID levels or standalone disks to accommodate different disk workloads, you will want to use this same approach when formatting multiple RAID arrays and standalone disks. Customize each disk device with the allocation unit size appropriate for the projected workload characteristics. If you can't determine the disk workload characteristics, use a smaller allocation unit size on devices that you expect to be random or read-intensive and use a larger allocation unit size for disk devices that will experience sequential and write-intensive workloads.
Your Results May Vary
Just how much your disk subsystem performance will improve using the tuning techniques I've described in this article depends on your environment. To test the differences in matching the allocation unit size, I ran a stress test. My test system consisted of an NT Server 4.0 file server configured with a 100Base-T network card, a 200MHz Pentium Pro processor, 64MB of RAM, a SCSI-based hardware RAID array controller with 16MB of onboard cache, a hard disk for NT, a hard disk for the paging file, and a three-disk RAID 5 array. I tested this system by using the Dynameasure file-server stress-testing tool from Bluecurve (http://www.bluecurve.com) operating from four 400MHz Pentium II processor clients with 128MB of RAM and 100Base-TX networking cards. While running the baseline test, I used Performance Monitor to obtain the disk characteristics of the RAID 5 array. I used this information to reformat the RAID 5 array from the default allocation unit size of 4KB to an allocation unit size of 64KB according to my Performance Monitor results. When I retested the system with the same workload, the results were dramatic, as Figure 1 shows. In this chart, higher values are better. As you can see, once the load increased on the file server, the performance of the tuned RAID array improved by more than 5Mbps in some instances.
Know Your Environment
Although this article presents general performance recommendations, you need to recognize that you'll achieve the best RAID or standalone disk performance improvements when you understand what is occurring on your particular NT system. Then you can make informed decisions when you customize your NT system to match your specific needs.