MSDTC Fails to Mutually Authenticate

Article
10/19/2016

Applies To: Windows 10, Windows 7, Windows 8, Windows 8.1, Windows Server 2008, Windows Server 2008 R2, Windows Server 2012, Windows Server 2012 R2, Windows Server Technical Preview, Windows Vista

Note

This topic is adapted from Microsoft Knowledge Base Article 827805 (MSDTC Fails to Mutually Authenticate When Computers Do Not Run in the Same Domain). This topic supersedes the Knowledge Base Article, as it contains information that is more recent and accurate.

Warning

This topic contains information about modifying the registry. Before you modify the registry, make sure to back it up and make sure that you understand how to restore the registry if a problem occurs. For information about how to back up, restore, and edit the registry, click the following article number to view the article in the Microsoft Knowledge Base:

256986 Description of the Microsoft Windows Registry

When two computers that are running in workgroups, or in domains that do not trust each other, use Microsoft Distributed Transaction Coordinator (MSDTC) for communication and for distributed transactions, the mutual authentication may fail.

Symptoms

When you run a distributed transaction against a linked server in Microsoft SQL Server 2000 on a computer that is running Microsoft Windows Server 2003 Enterprise Edition, you may receive the following error message:

Server: Msg 7391, Level 16, State 1, Line 2   
The operation could not be performed because the OLE DB provider 'SQLOLEDB' was unable to begin a distributed transaction. OLE/DB provider returned message: New transaction cannot enlist in the specified transaction coordinator.

Cause

The MSDTC proxy may not correctly authenticate MSDTC when the communicating computers are in workgroups, or in different domains that do not trust each other.

Resolution

Note

If you use Registry Editor incorrectly, you may cause serious problems that may require you to reinstall your operating system. Microsoft cannot guarantee that you can solve problems that result from using Registry Editor incorrectly. Use Registry Editor at your own risk.

For information about how to edit the registry, view the "Change Keys and Values" Help topic in Registry Editor (Regedit.exe) or the "Add and Delete Information in the Registry" and "Edit Registry Data" Help topics in Regedt32.exe. Microsoft recommends that you back up the registry before you edit it.

To turn off RPC security, follow these steps:

Start Registry Editor (Regedt32.exe).
Locate the following key in the registry: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC
On the Edit menu, click Add Value, and then add the following registry value:

Value Name Data Type Value

TurnOffRpcSecurity REG_DWORD 1
Quit Registry Editor.

More Information

Steps to Reproduce the Behavior

Make sure that both computers are running Windows Server 2003.
Install SQL Server 2000 with Service Pack 3 (SP3) on both the computers.
Make sure that MSDTC is started on both the computers.
On the first computer, start the SQL Query Analyzer (Isqlw.exe) utility, and then connect to the local SQL Server.
Add the second computer as the linked server. To do so, run the following Transact-SQL statement in SQL Query Analyzer:
```
EXEC sp_addlinkedserver 'remote_server', N'SQL SERVER'  
GO  
```
Note

Replace remote_server with the name of the second computer.

Run the following Transact-SQL statement in SQL Query Analyzer:

SET xact_abort ON  
GO  
USE pubs  
GO  
BEGIN DISTRIBUTED TRANSACTION  
SELECT * FROM remote_server.pubs.dbo.authors  
COMMIT TRAN  
GO

You may receive the error message that is mentioned in the "Symptoms" section of this topic.

For additional information, view the following Microsoft Knowledge Base Article:

329332 You receive Error 7391 when you run a distributed transaction against a linked server

Share via