OPEN/stor 2.0

Hierarchical Storage Management (HSM) has been around for almost 20 years in the mainframe environment. The basic premise of HSM centers on managing aging data. Now this storage strategy is available for Windows NT. One of the products leading the way in HSM for NT is Eastman Software's OPEN/stor. I evaluated version 2.0--version 2.05 is now available.

Understanding HSM
As you add files to a computer hard disk, you typically access the information less and less, and the files spend more time sitting idle on your hard disk. So why not move these files to an Iomega Jaz drive or similar device and retrieve them as you need them? The answer is simple. Once you move these files, you need a way to find them when you need them.

By placing a file marker when HSM moves the files offline, HSM solves this problem of finding archived files. If you want to locate a particular file, the HSM database finds the file and passes it to you. To the user, the file transfer and retrieval process is invisible. The goal of HSM is to maintain optimal use of storage space.

HSM is an application for controlling and managing data storage on various storage devices. As you see in Figure 1, page 92, these storage devices are shown in a hierarchical fashion, with the most expensive, fastest devices at the top and the cheapest, slowest devices at the bottom. This hierarchy consists of different layers with local fast hard disks at the top, intermediate hard disks as the first migration level, and optional magneto-optic (MO) and tape jukeboxes at the lower levels.

Once you establish some migration rules, HSM can migrate seldom-accessed files from the local hard disk to a migration volume on the HSM server. HSM leaves a pointer on the local hard disk to the original file, and NT Explorer or File Manager displays the filename. If you double-click the filename, you can retrieve the file almost as fast as you can retrieve it from a local hard disk (unless HSM is storing the file on an MO or tape device).

HSM is not a backup application. In fact, Eastman Software's OPEN/stor, the original PC HSM product, specifically states that it is not a backup system. OPEN/stor simply moves files from one device to another in a hierarchy of storage devices. The overall goal is to maintain defined levels of files on NT servers and NT workstations.

OPEN/stor is not the only HSM product on the market, but because of the complexity of terminology and setup, it's the only product I'm going to discuss in this article. Microsoft plans to include elements of OPEN/stor in NT 5.0, so OPEN/stor is a preeminent product at the moment. Windows NT Magazine will look at other HSM products in future issues.

Basic Terminology and Rules
Before you can install and use HSM, you need a thorough understanding of storage concepts and the rules of migration. You also need to pre-plan your environment and setup because once you install HSM, changing components and devices can be painstaking.

Active vs. Inactive Data. Hard disks have a mix of active and inactive data. Eastman Software estimates that you can consider 20 percent of the files on a hard disk active, while 80 percent are inactive. Of course, not everyone agrees on the terms active and inactive because not everyone can agree on how long a file is not used before it is considered inactive.

With OPEN/stor, the terms active and inactive are operationally defined in an administrator context. For example, one administrator may deem a file inactive if no one has accessed the file in 60 days, but another administrator may set 30 days as the criterion. With OPEN/stor, you have to define the rules for what's active and what's inactive (setting these rules is an OPEN/stor configuration option).

Movement of Inactive Data. After you define what files are active and inactive, you have to decide when to move the inactive files. Typically, you determine this setting according to the amount of free space on a managed disk (i.e., a volume). OPEN/stor defaults to using very high percentages (a 97 percent critical level, a 95 percent acceptable level, and a 93 percent optimal level) when calculating the amount of free disk space to maintain.

This issue is not trivial because files occupying more than 60 percent of a hard disk can cause serious fragmentation and dramatically slow down the hard disk I/O. Replacing large files with small placeholders helps maintain hard disk volume size.

Movement of Files. Eastman Software refers to moving files from the primary storage to secondary storage as migration. When the movement occurs, OPEN/stor leaves a placeholder in the original file's place. These placeholders maintain file location information and serve as a link to the secondary storage of the migrated file. If you scan your hard disk with NT Explorer or an application such as Word, NT sees the placeholder as the original file. OPEN/stor 2.0 creates a 420byte placeholder file. In future builds, the placeholder file size will match the original file size so you can see the original file size.

You can use the following Registry setting to control this option (I like to see the 420byte file size because I then know the file is a migrated file):

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\ AvFilter\Parameters\ ChangeFileSize

The value is a REG_DWORD. Setting the value to 0 will result in the 420byte placeholder file size. If you set this value to 1, you'll see the true file size and not the placeholder file size.