In “Troubleshooting Active Directory Replication,” Sean Deuby presented several strategies for solving Active Directory (AD) replication problems. To troubleshoot AD replication at a deeper level, it helps to have an in-depth understanding of how replication works when changes occur in the directory. AD was one of the first LDAP directories to introduce multi-master replication, whereby changes can originate in any instance of the directory (i.e., on any domain controller—DC). Previously, such as with Windows NT 4.0, changes could originate on only one DC, the Primary Domain Controller (PDC). Multi-master replication has numerous inherent benefits, but it presents a complex problem: how to coalesce and replicate changes to the directory. Throughout this article, I’ll refer back to the simple three-DC domain that Figure 1 shows.
Figure 1: Sample AD replication topology
Keeping Track of Replication
AD uses several counters and tables to ensure that every DC has the most current information for each attribute and object and to prevent any endless replication loops. AD uses naming contexts (NCs), also known as directory partitions, to segment replication. Every forest has a minimum of three NCs: the domain NC, the configuration NC, and the schema NC. AD also supports special NCs, often known as application partitions or non-domain naming contexts (NDNCs). DNS uses NDNCs (e.g., DomainDnsZones, ForestDnsZones). Each NC or NDNC replicates independently of one another.
Every DC maintains a special counter known as an update sequence number (USN) counter. The USN is a 64-bit number that you can think of like a clock. The USN counter is never decremented, and a USN can never be reused. Each DC maintains a separate USN counter that starts during the Dcpromo process and is incremented over the lifetime of a DC. It’s improbable that any two DCs in a forest will ever have the exact same USN at the same time. The USN counter is incremented each time a transaction occurs on a DC. Transactions are typically create, update, or delete operations against an object. An update transaction might include updates to a single attribute, or it might include updates to many attributes. In the event that a transaction fails and is rolled back, the USN assigned to the transaction isn’t reused. When an object is modified (or created), the usnChanged attribute of that object is stamped with the USN of the transaction that caused the change. You can therefore keep track of changes to AD by asking a DC for all the objects for which the usnChanged attribute is greater than the highest USN the last time you checked.
Table 1 illustrates a simple example of the changes to the USNs of two DCs over time. Consider a scenario in which you create five new groups on DC-A. This action will increment DC-A’s USN counter by five. DC-A then replicates these groups to DC-B, whose USN counter is incremented by five. (Note that the initial USN values in Table 1 were chosen for illustration purposes.) Subsequently, you edit the name of one of the groups on DC-B. DC-B’s USN counter is incremented by one. When the change in name replicates to DC-A, DC-A’s USN counter is incremented by one.
Table 1: USN Changes Over Time
From the perspective of DC-A, when the five groups are created, this is considered to be an originating write. From the perspective of DC-B, this is a replicated write. Conversely, when a group’s name is updated on DC-B, DC-B considers this action an originated write and DC-A considers it a replicated write.
The AD replication process identifies all the DCs participating in the replication process using two globally unique identifiers (GUIDs). The first GUID, the Directory Service Agent (DSA) GUID, is established during Dcpromo and doesn’t change for the lifetime of the DC. The DSA GUID is stored in the objectGuid attribute of the NTDS Settings object under the DC as shown in Active Directory Sites and Services. The second GUID, the Invocation ID, is the identifier for the DC during the replication process; it might change during the DC’s lifetime.
The Invocation ID is stored in the invocationId attribute of the NTDS Settings object. Any time a restore is performed on a DC by a supported restoration process, such as Windows Server Backup or NTBACKUP, that DC’s Invocation ID is reset. By resetting the Invocation ID, AD is able to ensure that the DC receives a copy of any changes that occurred on that DC between when the backup was taken and when the restore was performed. Because the Invocation ID is the unique identifier for the DC during the replication process, the reset of the Invocation ID effectively ensures that the DC enters the replication process as a new DC and there are no assumptions about data that might already exist on that DC. Improper restoration of a virtualized DC, such as restoring or reverting back to a saved snapshot, won’t reset the DC’s invocation ID. This leads to a situation known as USN rollback, which can cause severe replication problems.