Compartir a través de

Exchange: Lingering/Orphaned In-Place Holds on mailboxes blocking cleanup/moves.

  • Artículo

This blog post covers two different scenarios when an In-Place Hold is removed off an eDiscovery Mailbox Search, but that corresponding entry is not cleaned up properly on the mailboxes formerly targeted by that Mailbox Search. When this happens, you can’t remove the Mailbox Search and you also can’t migrate mailboxes back to older versions of Exchange, and this blog post covers how these IDs can become “orphaned” as well as how to remediate the situation. Additionally, a link to a script is provided at the bottom to help with the remediation.

DISCLAIMER: You should not be manually manipulating the In-Place Holds entries on mailboxes unless you are absolutely sure you know what you are doing and that you are indeed cleaning up only orphaned entries. Doing otherwise could result in legal and/or political issues within your organization.

Background:

eDiscovery Mailbox Searches created with an In-Place Hold on them, receive what is called “InPlaceHoldIdentity” GUID associated with that Mailbox Search. That GUID is then stamped in the InPlaceHolds (in AD the attribute is called msExchUserHoldPolicies) of each mailbox listed/targeted in that Mailbox Search. This is so the mailboxes are linked to the Mailbox Search and vice versa. This link helps ensure a Search with an active Hold can’t be accidentally deleted and the associated data currently being held is unexpectedly lost.

Normally when the Hold is removed from an eDiscovery search, within an hour the corresponding InPlaceHolds ID is removed off the associated mailbox(es), but there are two scenarios (covered below) in which this doesn’t work as expected. When this happens, the Search can’t be removed (even though the Hold has been turned off) and you receive this error (either through the GUI or PowerShell):

 Can’t remove mailbox search ‘DANTEST3” because there is still at lease one mailbox on In-Place Hold.

NOTE: You should always check the Mailbox Search, either by name or In-Place Hold ID, to see if the Hold has truly been turned off before using the cleanup method described in this post. If it hasn't you need to stop and turn it off if you truly want to decommission the Mailbox Search.

Here is an example of a Mailbox Search with the Hold enabled and the associated Hold Identity:

Also user mailboxes with orphaned InPlaceHolds entries will receive the following error when they are attempted to be migrated to older versionblos of Exchange (such as Exchange 2010):

 The mailbox ‘Sheehan, Daniel’ is place on In-Place Hold and can’t be moved to versions of Exchange earlier than Exchange 2013.

Orphaned In-Place Holds in a Hybrid Environment

In a Hybrid environment the InPlaceHolds entries on a mailbox are replicated up to EXO through Azure AD Connect. Mailbox Searches that have a Hold placed on them are also replicated to EXO (but that’s not the source of the issue) to maintain parity between both systems. As a mailbox is migrated to EXO, the values in the InPlaceHolds are kept as is (again because the Mailbox Searches with a hold on them are persisted into EXO).

Even after taking the Hold off a Mailbox Search, you continue to receive the above error over and over again no matter how many times you change the hold status of the Mailbox Search. Here are examples of a Mailbox Search where the Hold has been removed:

As mentioned previously, the InPlaceHolds entries (both on local and remote mailboxes) in a Hybrid configuration are kept in sync for parity, so if one side is missing the InPlaceHolds entries on a mailbox, then Azure AD Connect will repopulate them from the other side. Normally this is be a good thing except there is a potential timing issue with Azure AD connect as follows:

  1. The Hold is removed from the Mailbox Search via On-Premises.
  2. The removal is replicated up to EXO through Azure AD Connect, and the EXO mailbox has the GUIID removed.
  3. Unexpectedly, Azure AD Connect then repopulates the InPlaceHolds information back On-Premises (presumably because it was set on the EXO mailbox before it was blanked).
  4. At the next replication interval Azure AD Connect will notice a GUID in the InPlaceHolds attribute of the On-Premises mailbox and replicate it back up to EXO.

In a sense, this Azure AD Connect timing issue causes a loop where the GUIDs can’t be removed in a single pass from the mailboxes that were once in Hold in a Mailbox Search.

To permanent resolve this situation in a Hybrid environment, use the following steps:

  1. Verify the Mailbox Search is no longer configured to have any Hold enabled.
  2. Remove the Mailbox Search InPlaceHoldIdentity GUID from the affected mailboxes InPlaceHolds attribute in the On-Premises side.
  3. Wait for Azure AD Connect replication to sync the removal up to EXO where the GUID is removed from the EXO mailbox, but is repopulated back on to the On-Premises mailbox by Azure AD connect.
  4. Immediately after that synchronization has completed, remove the GUID a second time from the affected On-Premises mailboxes InPlaceHolds attribute.

Once the GUID is blanked on both sides for each affected mailbox(es), it stays blanked and you can safely remove the Mailbox Search that previously had a hold place on it. More on how to do this programmatically later.

Orphaned In-Place Holds in a non-Hybrid Environment

I have also seen the issue with orphaned InPlaceHolds entries on mailboxes in a non-Hybrid environment, where the Hold was removed some time ago from the Mailbox Search, and that scenario is usually attributed to various scenarios:

  1. AD user (with mailboxes) object restores (where the Hold ID was populated from before it was removed).
  2. Object moves between domains where the Hold ID removal was just lost in transition.
  3. Even the odd issue where the removal of the Hold from the Mailbox Search never manifested itself as a removal on the targeted mailboxes. I.E. You turned off the Hold on the Mailbox Search, but it never got cleaned off/removed from the user mailbox(es).

In this scenario the In-Place Hold ID just needs to be removed from the affected mailboxes once as there is nothing like Azure AD Connect that will repopulate it.

Cleaning Up Orphaned In-Place Holds

The assumption at this point is the Hold has already been removed some time ago from the Mailbox Search and the Policy IDs just weren’t removed from the mailbox(es), or the Mailbox Search itself was removed some time ago but the Policy ID was orphaned on the mailbox(es). If there is any situation where the Mailbox Search is still needed with a Hold, DO NOT USE THIS PROCESS TO CLEAN UP THE POLCIY ID FROM MAILBOXES OR YOU COULD CAUSE MAJOR eDISCOVERY ISSUES.

While cleaning up the GUIDs from the mailbox InPlaceHolds attribute can be done on a case by case basis via ADSIEdit, or the Attributed Editor in Active Directory Users and Computers (ADUC), this really doesn’t scale to a large number of mailboxes especially if you have to do it twice and the mailboxes span multiple domains. Example of manipulating the attributes in the ADUC Attribute Editor:

To assist with this cleanup, I wrote a script to completely automate this process, including a –WhatIf switch that reports only (doesn’t actually make changes) switch, and it can output to a CSV file what was done. All you have to do is run the script (twice for the Hybrid scenario) with the GUID you want to remove across all mailboxes (again both local and remote). Example:

To use the script, you will need to obtain the Mailbox Search In Place Hold ID you want to clean up. You can pull that from the InPlaceHolds attribute of the affected mailbox(es) through the Exchange Management Shell as follows:

Once you have the ID, you should ALWAYS use the Exchange Management Shell and the following command to verify the Hold status of a matching Mailbox Search is disabled. If it's not, stop and follow the normal procedure of disabling it and allowing Exchange time to try and remove the ID off the affected mailboxes:

 
Get-MailboxSearch -InPlaceHoldIdentity <InPlaceHoldID> | FL Name,InPlaceHoldEnabled

Example of a Mailbox Search where the Hold has NOT been disabled.:

NOTE: The script will perform this same check just to be sure, but it’s a good idea to always manually check the existence of Mailbox Searches and their Hold status manually. If the script finds there is a hold still enabled on the Mailbox Search, it will automatically stop:

Closing Thoughts

I can’t stress this enough – you should not be manually manipulating the InPlaceHolds entries on mailboxes unless you are absolutely sure you know what you are doing and that you are indeed cleaning up only orphaned entries. Again, I tried to put safety guards into the script to help prevent you from shooting yourself in the foot, both legally and politically, but nothing is fool proof.

Otherwise hopefully now you know why In-Place Holds can get orphaned on mailboxes, especially in a Hybrid scenario, and have a script to help you clean them up as needed

Please feel free to leave me comments here if you wish, I promise I will try to respond to each in kind.

Thanks!
Dan Sheehan
Senior Premier Field Engineer