Managing Database Availability Groups

Applies to: Exchange Server 2010

A database availability group (DAG) is a set of up to 16 Microsoft Exchange Server 2010 Mailbox servers that provides automatic database-level recovery from a database, server, or network failure. DAGs use continuous replication and a subset of Windows failover clustering technologies to provide continuous mailbox availability. Mailbox servers in a DAG monitor each other for failures. When a Mailbox server is added to a DAG, it works with the other servers in the DAG to provide automatic, database-level recovery from database failures.

When you create a DAG, it is initially empty, and a directory object is created in Active Directory that represents the DAG. The directory object is used to store relevant information about the DAG, such as server membership information. When you add the first server to a DAG, a failover cluster is automatically created for the DAG. In addition, the infrastructure that monitors the servers for network or server failures is initiated. The failover cluster heartbeat mechanism and cluster database are then used to track and manage information about the DAG that can change quickly, such as database mount status, replication status, and last mounted location.

Contents

Creating Database Availability Groups

Database Availability Group Membership

Configuring Database Availability Group Properties

Database Availability Group Networks

Shutting Down Database Availability Group Members

Installing Update Rollups to Database Availability Group Members

Creating Database Availability Groups

A DAG can be created using the New Database Availability Group wizard in the Exchange Management Console (EMC), or by running the New-DatabaseAvailabilityGroup cmdlet in the Exchange Management Shell. When creating a DAG, you provide a name for the DAG, and optional witness server and witness directory settings. In addition, one or more IP addresses are assigned to the DAG, either by using static IP addresses or by allowing the DAG to be automatically assigned the necessary IP addresses using Dynamic Host Configuration Protocol (DHCP). You can manually assign IP addresses to the DAG by using the DatabaseAvailablityGroupIpAddresses parameter. If you omit this parameter, the DAG will attempt to obtain an IP address by using a DHCP server on your network.

For detailed steps to create a DAG, see Create a Database Availability Group.

When you create a DAG, an empty object representing the DAG with the name you specified and an object class of msExchMDBAvailabilityGroup is created in Active Directory.

DAGs use a subset of Windows failover clustering technologies, such as the cluster heartbeat, cluster networks, and cluster database (for storing data that changes or can change quickly, such as database state changes from active to passive or the reverse, or from mounted to dismounted or the reverse). Because DAGs rely on Windows failover clustering, they can only be created on Exchange 2010 Mailbox servers running the Windows Server 2008 Enterprise operating system or Windows Server 2008 R2 Enterprise operating system.

Database Availability Group Witness Server and Witness Directory

When creating a DAG, you need to specify a name for the DAG no longer than 15 characters that's unique within the Active Directory forest. In addition, each DAG is configured with a witness server and witness directory. The witness server and its directory are used only for quorum purposes where there's an even number of members in the DAG. You don't need to create the witness directory in advance. Exchange will automatically create and secure the directory for you on the witness server. The directory shouldn't be used for any purpose other than for the DAG witness server.

The requirements for the witness server are as follows:

• The witness server can't be a member of the DAG.
• The witness server must be in the same Active Directory forest as the DAG.
• The witness server must be running Windows Server 2008 R2, Windows Server 2008, Windows Server 2003 R2, or Windows Server 2003.
• A single server can serve as a witness for multiple DAGs; however, each DAG requires its own witness directory.

We recommend that you use an Exchange 2010 Hub Transport server in the Active Directory site containing the DAG. This allows the witness server and directory to remain under the control of an Exchange administrator. Regardless of what server is used as the witness server, if the Windows Firewall is enabled on the intended witness server, you must enable the Windows Firewall exception for File and Printer Sharing.

Neither the witness server nor the witness directory needs to be fault tolerant or use any form of redundancy or high availability. There's no need to use a clustered file server for the witness server, or employ any other form of resiliency for the witness server. There are several reasons for this. With larger DAGs (for example, six members or more) several failures are required before there's a need for the witness server. Because a six-member DAG can tolerate as many as two voter failures without losing quorum, it would take as many as three voters failing before the witness server would be needed to maintain a quorum. Also, if there is a failure that affects your current witness server (for example, you lose the witness server because of a hardware failure), you can use the Set-DatabaseAvailabilityGroup cmdlet to configure a new witness server and witness directory (provided you have a quorum).

As a best practice, in an environment where a DAG is extended across multiple datacenters (and Active Directory sites) and configured for site resilience, we recommend that you use a witness server in your primary datacenter (the datacenter containing the majority of your user population). If each datacenter has a similar number of users, the datacenter you choose to host the witness server will be considered to be the primary datacenter from the solution's perspective. If the witness server is in the datacenter with the majority of the client population, the majority of clients retain access after a failure.

If the datacenter is remote to large user populations, this may affect your decision. You would then need to determine if there's a requirement for the primary datacenter to remain healthy and active if there is a loss of wide are network (WAN) connectivity to the other two datacenters. In that event, the witness server should also be in the primary datacenter.

Although it's supported to use a witness server in a third datacenter, we don't recommend this scenario. From an Exchange perspective, this configuration doesn't provide you with greater availability. It's important that you examine the critical path factors if you use a witness server in a third datacenter. For example, if the WAN connection between the primary datacenter and the second and third datacenter fails, the solution in the primary datacenter will become unavailable.

Specifying a Witness Server and Witness Directory During DAG Creation

When creating a DAG, you must provide a name for the DAG. You can optionally also specify a witness server and directory. If you specify a witness server, we recommend that you use a Hub Transport server, because this allows an Exchange administrator to be aware of the availability of the witness server.

When creating a DAG, the following combinations of options and behaviors are available:

• You can specify only a name for the DAG, and leave the Witness server and Witness directory fields blank. In this scenario, the wizard will search for a Hub Transport server that doesn't have the Mailbox server role installed, and it will automatically create the default directory (%SystemDrive%:\DAGFileShareWitnesses\<DAGFQDN>) and default share (<DAGFQDN>) on that server and use that Hub Transport server as the witness server. For example, consider a witness server named EXMBX3 on which the operating system has been installed onto the C drive. A DAG named DAG1 in a domain named contoso.com would use a default witness directory of C:\DAGFileShareWitnesses\DAG1.contoso.com, which would be shared as \\EXMBX3\DAG1.contoso.com.
• You can specify a name for the DAG, the witness server that you want to use, and the directory you want created and shared on the witness server.
• You can specify a name for the DAG and the witness server that you want to use, and leave the Witness directory field blank. In this scenario, the wizard will create the default directory on the specified witness server.
• You can specify a name for the DAG, leave the Witness server field blank, and specify the directory you want created and shared on the witness server. In this scenario, the wizard will search for a Hub Transport server that doesn't have the Mailbox server role installed, and it will automatically create the specified DAG on that server, share the directory, and use that Hub Transport server as the witness server.

When a DAG is formed, it will initially use the Node Majority quorum model. When the second Mailbox server is added to the DAG, the quorum is automatically changed to a Node and File Share Majority quorum model. When this change occurs, the DAG begins using the witness server for maintaining a quorum. If the witness directory doesn't exist, Exchange automatically creates it, shares it, and provisions the share with full control permissions for the cluster network object (CNO) computer account for the DAG.

If the Windows firewall is enabled on the witness server before the DAG is created, it may block the creation of the DAG. Exchange uses Windows Management Instrumentation (WMI) to create the directory and file share on the witness server. If the Windows firewall is enabled on the witness server and there are no firewall exceptions configured for WMI, then the New-DatabaseAvailabilityGroup cmdlet will fail with an error. If you specify a witness server, but not a witness directory, you'll get the following error:

If you specify a witness server and witness directory, you'll get the following warning:

If the Windows firewall is enabled on the witness server after the DAG is created but before servers are added, it may block the addition or removal of DAG members. If the Windows firewall is enabled on the witness server and there are no firewall exceptions configured for WMI, then the Add-DatabaseAvailabilityGroupServer cmdlet will display the following warning:

To resolve the preceding error and warnings, do one of the following:

• Pre-create the witness directory and share on the witness server
• Enable the WMI exception in Windows firewall
• Disable the Windows firewall

Return to top

Database Availability Group Membership

After a DAG has been created, you can add servers to or remove servers from the DAG by using the Manage Database Availability Group wizard in the EMC, or by using the Add-DatabaseAvailbilityGroupServer or the Remove-DatabaseAvailbilityGroupServer cmdlets in the Shell. For detailed steps to manage DAG membership, see Manage Database Availability Group Membership.

If the Mailbox server being added to a DAG doesn't have the failover clustering component installed, the method used to add the server (for example, the Add-DatabaseAvailabilityGroupServer cmdlet or the Manage Database Availability Group wizard) will install the failover clustering feature.

When the first Mailbox server is added to a DAG, the following occurs:

• The Windows failover clustering component is installed, if it isn't already installed.
• A failover cluster is created using the name of the DAG. This failover cluster is used exclusively by the DAG, and the cluster must be dedicated to the DAG. Use of the cluster for any other purpose is not supported.
• A cluster network object (CNO) is created in the default computers container.
• The name and IP address of the DAG is registered as a Host (A) record in DNS.
• The server is added to the DAG object in Active Directory.
• The cluster database is updated with information on the databases mounted on the added server.

In a large or multi-site environment, especially those in which the DAG is extended to multiple Active Directory sites, you must wait for Active Directory replication of the DAG object containing the first DAG member to complete. If this Active Directory object isn't replicated throughout your environment, adding the second server may cause a new cluster (and new CNO) to be created for the DAG. This is because the DAG object will appear empty from the perspective of the second member being added, thereby causing the Add-DatabaseAvailabilityGroupServer cmdlet to create a new cluster and CNO for the DAG, even though these objects already exist. To verify that the DAG object containing the first DAG server has been replicated, use the Get-DatabaseAvailabilityGroup cmdlet on the second server being added to verify that the first server you added is listed as a member of the DAG.

When the second and subsequent servers are added to the DAG, the following occurs:

• The server is joined to the Windows failover cluster for the DAG.
• The quorum model is automatically adjusted:
  • A Node Majority quorum model is used for DAGs with an odd number of members.
  • A Node and File Share Majority quorum model is used for DAGs with an even number of members.
• The witness directory and share are automatically created by Exchange when needed.
• The server is added to the DAG object in Active Directory.
• The cluster database is updated with information about mounted databases.

Pre-Staging the Cluster Network Object for a Database Availability Group

The CNO is a computer account that's created in Active Directory and associated with the cluster's Name resource. The cluster's Name resource is tied to the CNO, which is a Kerberos-enabled object that acts as the cluster's identity and provides the cluster's security context. In Exchange 2007, this Kerberos-enabled machine account was created in the domain using the security context of the user performing the tasks. This required the user’s account to have permissions to create and enable machine accounts in the domain or that the computer account was properly pre-staged and provisioned.

As stated above, the formation of the DAG's underlying cluster and the CNO for that cluster is performed when the first member is added to the DAG. When the first server is added to the DAG, Powershell contacts the Microsoft Exchange Replication service on the Mailbox server being added. The Microsoft Exchange Replication service installs the Failover Clustering feature (if it's not already installed) and begins the cluster creation process. The Microsoft Exchange Replication service runs under the LOCAL SYSTEM security context, and it's under this context in which cluster creation is performed.

In environments where computer account creation is restricted, or where computer accounts are created in a container other than the default computers container, you can pre-stage and provision the CNO. You create and disable a computer account for the CNO and then either:

• Assign full control of the computer account to the computer account of the first Mailbox you are adding to the DAG.
• Assign full control of the computer account to the Exchange Trusted Subsystem universal security group.

Assigning full control of the computer account to the computer account of the first Mailbox you are adding to the DAG ensures that the LOCAL SYSTEM security context will be able to manage the pre-staged computer account. Assigning full control of the computer account to the Exchange Trusted Subsystem universal security group can be used instead because the Exchange Trusted Subsystem group contains the machine accounts of all Exchange servers in the domain.

For detailed steps about how to pre-stage and provision the CNO for a DAG, see Pre-stage the Cluster Network Object for a Database Availability Group.

Removing Servers from a Database Availability Group

Mailbox servers can be removed from a DAG by using the Manage Database Availability Group wizard in the EMC, or the Remove-DatabaseAvailbilityGroupServer cmdlet in the Shell. Before a Mailbox server can be removed from a DAG, all replicated mailbox databases must first be removed from the server. If you attempt to remove a Mailbox server with replicated mailbox databases from a DAG, the task will fail.

There are certain scenarios in which you must remove a Mailbox server from a DAG before performing certain operations. These scenarios include:

• Performing a server recovery operation   If a Mailbox server that's a member of a database availability group is lost, or otherwise fails and is unrecoverable and needs replacement, you can perform a server recovery operation using the Setup /m:RecoverServer switch. However, before you can perform the recovery operation, you must first remove the server from the DAG by using the Remove-DatabaseAvailabilityGroupServer cmdlet with the ConfigurationOnly parameter.
• Removing the database availability    There may be situations in which you need to remove a DAG (for example, when disabling third-party replication mode). If you need to remove a DAG, you must first remove all servers from the DAG. If you attempt to remove a DAG that contains one or more members, the task will fail.

Return to top

Configuring Database Availability Group Properties

After servers have been added to the DAG, you can use the EMC or the Shell to configure the properties of a DAG, including the witness server and directory used by the DAG, and the IP addresses assigned to the DAG.

Configurable properties include:

• Witness server   The name of the server that you want to host the file share for the file share witness. We recommend that you specify a Hub Transport server outside the DAG as the witness server. This enables the system to automatically configure, secure, and use the share, as needed.
• Witness directory   The name of a directory that will be used to store file share witness data. This directory will automatically be created by the system on the specified witness server.
• Database availability group IP addresses   One or more IP addresses assigned to the DAG. These addresses can be configured using manually assigned static IP addresses, or they can be automatically assigned to the DAG by using a DHCP server in your organization.

The Shell enables you to configure DAG properties that aren't available in the EMC, such as DAG IP addresses, network encryption and compression settings, network discovery, the TCP port used for replication, and alternate witness server and witness directory settings, and to enable datacenter activation coordination mode.

For detailed steps to configure database availability group properties, see Configure Database Availability Group Properties.

Database Availability Group Network Encryption

DAGs support the use of encryption by leveraging the encryption capabilities of the Windows Server operating system. DAGs use Kerberos authentication between Exchange servers. Microsoft Kerberos security support provider (SSP) EncryptMessage/DecryptMessage APIs handle encryption of DAG network traffic. Microsoft Kerberos SSP supports multiple encryption algorithms. (For the complete list, see reference section 3.1.5.2, "Encryption Types" of Kerberos Protocol Extensions) The Kerberos authentication handshake select5s the strongest encryption protocol supported in the list: typically Advanced Encryption Standard (AES) 256-bit, potentially with a SHA Hash-based Message Authentication Code (HMAC) to maintain integrity of the data. For details, see HMAC.

Network encryption is a property of the DAG, and not a DAG network. You can configure DAG network encryption by using the Set-DatabaseAvailabilityGroup cmdlet in the Shell. The possible encryption settings for DAG network communications are shown in the following table.

DAG network communication encryption settings

Database Availability Group Network Compression

DAGs also support built-in compression. When compression is enabled, DAG network communication uses XPRESS, which is the Microsoft implementation of the LZ77 algorithm. For details, see LZ77 algorithm and reference, section 3.1.7.2, "Compression Algorithm" of Wire Format Protocol Specification. This is the same type of compression used in many Microsoft protocols, in particular, MAPI RPC compression between Microsoft Outlook and Exchange.

As with network encryption, network compression is also a property of the DAG, and not a DAG network. You configure DAG network compression by using the Set-DatabaseAvailabilityGroup cmdlet in the Shell. The possible compression settings for DAG network communications are shown in the following table.

DAG network communication compression settings

Return to top

Database Availability Group Networks

Although a single network adapter and path is supported, we recommend that each DAG have a minimum of two DAG networks. A DAG network is a collection of one or more subnets used for either replication traffic or MAPI traffic. In a two-network configuration, one network is typically dedicated for replication traffic, and the other network is used primarily for MAPI traffic. You can create additional networks in a DAG, and configure them as replication networks for redundancy.

You can use the New Database Availability Group Network wizard in the EMC or the New-DatabaseAvailabilityGroupNetwork cmdlet in the Shell to create a DAG network. For detailed steps to create a DAG network, see Create a Database Availability Group Network.

You can use the DAG network's Properties dialog box in the EMC or the Set-DatabaseAvailabilityGroupNetwork cmdlet in the Shell to configure DAG network properties. For detailed steps to configure DAG network properties, see Configure Database Availability Group Network Properties. Each DAG network has required and optional parameters to configure:

• Network name   A unique name for the DAG network of up to 128 characters.
• Network description   An optional description for the DAG network of up to 256 characters.
• Network subnets   One or more subnets entered using a format of IPAddress/Bitmask (for example, 192.168.1.0/24 for IPv4 subnets; 2001:DB8:0:C000::/64 for IPv6 subnets).
• Enable replication   In the EMC, select the check box to dedicate the DAG network to replication traffic, and block MAPI traffic. Clear the check box to prevent replication from using the DAG network, and to enable MAPI traffic. In the Shell, use the ReplicationEnabled parameter in the Set-DatabaseAvailabilityGroupNetwork cmdlet to enable and disable replication.

DAG Networks and iSCSI Networks

By default, DAGs will perform discovery of all networks detected and configured for use by its underlying cluster. This includes any Internet SCSI (iSCSI) networks that are in use as a result of using iSCSI storage for one or more DAG members. As a best practice, iSCSI storage should use dedicated networks and network interface cards. These networks shouldn't be managed by the DAG or its cluster, or used as DAG networks (MAPI or replication). Instead, they should be manually disabled from use by the DAG, so they can be dedicated to iSCSI storage traffic. There are two steps to perform to disable iSCSI networks from being detected and used as DAG networks:

1. Configure the DAG to ignore any currently detected iSCSI networks by using the Set-DatabaseAvailabilityGroupNetwork cmdlet, as shown in this example.

   Set-DatabaseAvailabilityGroupNetwork -Identity DAG2\DAGNetwork02 -ReplicationEnabled:$false -IgnoreNetwork:$true
   

2. Disable the network for use by the cluster by running the following command from an elevated command prompt or PowerShell window. (Run Cluster network to display the names of the cluster networks.)

   Cluster network ClusterNetworkName /prop Role=0
   

After disabling all iSCSI networks for use by the DAG and its cluster, you can optionally force network discovery by running the Set-DatabaseAvailabilityGroup cmdlet with the DiscoverNetworks parameter.

Return to top

Shutting Down Database Availability Group Members

The Exchange 2010 high availability solution is integrated with the Windows shutdown process. If an administrator or application initiates a shutdown of a Windows server in a DAG that has a mounted database that's replicated to one or more DAG members, the system will try to activate another copy of the mounted database prior to allowing the shutdown process to complete.

However, this new behavior doesn't guarantee that all of the databases on the server being shut down will experience a loss-less activation. As a result, it's a best practice to perform a server switchover prior to shutting down a server that's a member of a DAG. For detailed steps to perform a server switchover, see Perform a Server Switchover.

Return to top

Installing Update Rollups on Database Availability Group Members

Installing Microsoft Exchange Server 2010 update rollups on a server that is a member of a database availability group (DAG) is a relatively straightforward process. When you install an update rollup on a server that is a member of a DAG, several services will be stopped during the installation, including all Exchange services, and the Windows Cluster service. The general process for applying update rollups to a DAG member is as follows:

• Perform a server switchover so that all databases on the server are passive copies.
• Suspend activation for the databases on the server being updated.
• Install the update rollup.
• Resume activation for the databases on the updated server.
• Perform database switchovers as needed.

You can download the latest update rollup for Exchange 2010 from the Microsoft Download Center. For detailed steps to install an update rollup on a DAG member, see Installing Update Rollups on Database Availability Group Members.

Return to top