Poison Message Handling in MSMQ 3.0
This sample demonstrates how to perform poison message handling in a service. This sample is based on the Transacted MSMQ Binding sample. This sample uses the netMsmqBinding. The service is a self-hosted console application that enables you to observe the service receiving queued messages.
In queued communication, the client communicates to the service using a queue. More precisely, the client sends messages to a queue. The service receives messages from the queue. The service and client therefore, do not have to be running at the same time to communicate using a queue.
A poison message is a message that is repeatedly read from a queue when the service reading the message cannot process the message and therefore terminates the transaction under which the message is read. In such cases, the message is retried again. This can theoretically go on forever if there is a problem with the message. Note that this can only occur when you use transactions to read from the queue and invoke the service operation.
Based on the version of MSMQ, the NetMsmqBinding supports limited detection to full detection of poison messages. After the message has been detected as poisoned, then it can be handled in several ways.
This sample illustrates the limited poison facilities provided on Windows Server 2003 and Windows XP platforms and the full poison facilities provided on Windows Vista. In both samples, the objective is to move the poison message out of the queue to another queue which then can be serviced by a poison message service.
MSMQ v3.0 (Windows 2003 and Windows XP) Poison Handling Sample
There is no poison sub-queue support in MSMQ v3.0. As such, in this sample, we create a queue to hold poison messages. This is the preferred method of dealing with poison messages in MSMQ v3.0 to prevent data loss.
The poison message detection in Windows Server 2003 and Windows XP is restricted to configuring a single property ReceiveRetryCount. The ReceiveRetryCount is the number of times the message can be read repeatedly from the queue. Windows Communication Foundation (WCF) maintains this count in memory. Once the count is reached (that is, the message is poisoned) we then transition to how to handle the poison message. The enumeration ReceiveErrorHandling property on the binding provides possible values it can take. The enumeration values are:
Fault (default): To fault the listener and also the service host.
Drop: To drop the message.
Move: To move the message to the poison message sub-queue. This value is available only on Windows Vista.
Reject: To reject the message by sending the message back to the sender's dead-letter queue. This value is available only on Windows Vista.
Of these, only Fault and Drop are valid values, when using MSMQ v3.0. The sample illustrates how to handle poison messages using MSMQ v3.0. This sample can also be run in MSMQ v4.0 and works exactly as in MSMQ v3.0.
The service contract is IOrderProcessor, which defines a one-way service that is suitable for use with queues.
[ServiceContract(Namespace="http://Microsoft.ServiceModel.Samples")]
public interface IOrderProcessor
{
[OperationContract(IsOneWay = true)]
void SubmitPurchaseOrder(PurchaseOrder po);
}
The service operation displays a message stating it is processing the order. To demonstrate the poison message functionality, the SubmitPurchaseOrder service operation throws an exception to rollback the transaction on some random invocation of the service. This causes the message to be put back in the queue. Eventually the message is marked as poison. The configuration is set to fault on poison messages. Once the message is poisoned, the queued channel throws a MsmqPoisonMessageException that contains the MessageLookupId of the poisoned message and faults the ServiceHost. An error handler is attached to monitor for this exception and take appropriate action. To add the error handler, we create a PoisonErrorBehaviorAttribute and annotate the OrderProcessorService with this behavior.
// Service class that implements the service contract.
// Added code to write output to the console window.
[PoisonErrorBehavior]
public class OrderProcessorService : IOrderProcessor
{
static Random r = new Random(137);
public static string QueueName;
public static string PoisonQueueName;
[OperationBehavior(TransactionScopeRequired = true, TransactionAutoComplete = true)]
public void SubmitPurchaseOrder(PurchaseOrder po)
{
int randomNumber = r.Next(10);
if (randomNumber % 2 == 0)
{
Orders.Add(po);
Console.WriteLine("Processing {0} ", po);
}
else
{
Console.WriteLine("Aborting transaction, cannot process purchase order: " + po.PONumber);
Console.WriteLine();
throw new Exception("Cannot process purchase order: " + po.PONumber);
}
}
public static void StartThreadProc(object stateInfo)
{
StartService();
}
public static void StartService()
{
// Get MSMQ queue name from app settings in configuration
QueueName = ConfigurationManager.AppSettings["queueName"];
// Get MSMQ queue name for the final poison queue
PoisonQueueName = ConfigurationManager.AppSettings["poisonQueueName"];
// Create the transacted MSMQ queue if necessary.
if (!System.Messaging.MessageQueue.Exists(QueueName))
System.Messaging.MessageQueue.Create(QueueName, true);
// Create the transacted poison message MSMQ queue if necessary.
if (!System.Messaging.MessageQueue.Exists(PoisonQueueName))
System.Messaging.MessageQueue.Create(PoisonQueueName, true);
// Get the base address that is used to listen for WS-MetaDataExchange requests
// This is useful to generate a proxy for the client
string baseAddress = ConfigurationManager.AppSettings["baseAddress"];
// Create a ServiceHost for the OrderProcessorService type.
ServiceHost serviceHost = new ServiceHost(typeof(OrderProcessorService), new Uri(baseAddress));
// Hook on to the service host faulted events
serviceHost.Faulted += new EventHandler(OnServiceFaulted);
// Open the ServiceHostBase to create listeners and start listening for messages.
serviceHost.Open();
}
public static void OnServiceFaulted(object sender, EventArgs e)
{
Console.WriteLine("Service Faulted");
}
// Host the service within this EXE console application.
public static void Main()
{
OrderProcessorService.StartService();
// The service can now be accessed.
Console.WriteLine("The service is ready.");
Console.WriteLine("Press <ENTER> to stop the service");
Console.ReadLine();
Console.WriteLine("Exiting service");
}
}
The PoisonErrorBehaviorAttribute installs the PoisonErrorHandler. The PoisonErrorHandler is an implementation of the IErrorHandler. The IErrorHandler is called whenever an exception is thrown in the system. In our implementation, we look for MsmqPoisonMessageException that indicates that a poison message was found. We retrieve the MSMQ message look up id from the exception and use the System.Messaging API to receive the poison message from the queue and move it to a specific poison message queue for later processing. Because the message may still be locked under the ServiceModel transaction, we wait to see if the read was unsuccessful and try again. Once the message has been moved to a different queue, the rest of the messages in the queue can be consumed. a new ServiceHost is created and opened for processing. The following code example demonstrates this behavior:
public class PoisonErrorHandler : IErrorHandler
{
static WaitCallback orderProcessingCallback = new WaitCallback(OrderProcessorService.StartThreadProc);
public void ProvideFault(Exception error, MessageVersion version, ref Message fault)
{
// no-op -we are not interested in this.
}
// Handle poison message exception by moving the offending message out of the way for regular processing to go on.
public bool HandleError(Exception error)
{
MsmqPoisonMessageException poisonException = error as MsmqPoisonMessageException;
if (null != poisonException)
{
long lookupId = poisonException.MessageLookupId;
Console.WriteLine(" Poisoned message -message look up id = {0}", lookupId);
// Get MSMQ queue name from app settings in configuration.
System.Messaging.MessageQueue orderQueue = new System.Messaging.MessageQueue(OrderProcessorService.QueueName);
System.Messaging.MessageQueue poisonMessageQueue = new System.Messaging.MessageQueue(OrderProcessorService.PoisonQueueName);
System.Messaging.Message message = null;
// Use a new transaction scope to remove the message from the main application queue and add it to the poison queue.
// The poison message service processes messages from the poison queue.
using(TransactionScope txScope= new TransactionScope(TransactionScopeOption.RequiresNew))
{
int retryCount = 0;
while(retryCount < 3)
{
retryCount++;
try
{
// Look up the poison message using the look up id.
message = orderQueue.ReceiveByLookupId(lookupId);
if(message != null)
{
// Send the message to the poison message queue.
poisonMessageQueue.Send(message, System.Messaging.MessageQueueTransactionType.Automatic);
// complete transaction scope
txScope.Complete();
Console.WriteLine("Moved poisoned message with look up id: {0} to poison queue: {1} ", lookupId, OrderProcessorService.PoisonQueueName);
break;
}
}
catch(InvalidOperationException )
{
//Code for the case when the message may still not be available in the queue because of a race in transaction or
//another node in the farm may actually have taken the message.
if (retryCount < 3)
{
Console.WriteLine("Trying to move poison message but message is not available, sleeping for 10 seconds before retrying");
Thread.Sleep(TimeSpan.FromSeconds(10));
}
else
{
Console.WriteLine("Giving up on trying to move the message");
}
}
}
}
// Restart the service host.
Console.WriteLine();
Console.WriteLine("Restarting the service to process rest of the messages in the queue");
Console.WriteLine("Press <ENTER> to stop the service");
ThreadPool.QueueUserWorkItem(orderProcessingCallback);
return true;
}
return false;
}
}
The service configuration includes the following poison message properties: receiveRetryCount, and receiveErrorHandling as shown in the following configuration file. The properties maxRetryCycles and retryCycleDelay are ignored when running on Windows Server 2003 and Windows XP.
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<appSettings>
<!-- Use appSetting to configure the MSMQ queue name. -->
<add key="queueName" value=".\private$\ServiceModelSamplesPoison" />
<add key="poisonQueueName" value=".\private$\OrderProcessorPoisonQueue" />
<add key="baseAddress" value="https://localhost:8000/orderProcessor/poisonSample"/>
</appSettings>
<system.serviceModel>
<services>
<service
name="Microsoft.ServiceModel.Samples.OrderProcessorService">
<!-- Define NetMsmqEndpoint -->
<endpoint address="net.msmq://localhost/private/ServiceModelSamplesPoison"
binding="netMsmqBinding"
bindingConfiguration="PoisonBinding"
contract="Microsoft.ServiceModel.Samples.IOrderProcessor" />
</service>
</services>
<bindings>
<netMsmqBinding>
<binding name="PoisonBinding"
receiveRetryCount="0"
maxRetryCycles="1"
retryCycleDelay="00:00:05"
receiveErrorHandling="Fault">
</binding>
</netMsmqBinding>
</bindings>
</system.serviceModel>
</configuration>
Processing messages from the poison message queue
The poison message service reads messages from the final poison message queue and processes them.
Messages in the poison message queue are messages that are addressed to the service that is processing the message, which could be different from the poison message service endpoint. Therefore, when the poison message service reads messages from the queue, the WCF channel layer finds the mismatch in endpoints and does not dispatch the message. In this case, the message is addressed to the order processing service but is being received by the poison message service. To continue to receive the message even if the message is addressed to a different endpoint, we must add a ServiceBehavior to filter addresses where the match criterion is to match any service endpoint the message is addressed to. This is required to successfully process messages that you read from the poison message queue.
The poison message service implementation itself is very similar to the service implementation. It implements the contract and processes the orders. The code example is as follows.
// Service class that implements the service contract.
// Added code to write output to the console window.
[ServiceBehavior(AddressFilterMode=AddressFilterMode.Any)]
public class OrderProcessorService : IOrderProcessor
{
[OperationBehavior(TransactionScopeRequired = true, TransactionAutoComplete = true)]
public void SubmitPurchaseOrder(PurchaseOrder po)
{
Orders.Add(po);
Console.WriteLine("Processing {0} ", po);
}
public static void OnServiceFaulted(object sender, EventArgs e)
{
Console.WriteLine("Service Faulted...exiting app");
Environment.Exit(1);
}
// Host the service within this EXE console application.
public static void Main()
{
// Create a ServiceHost for the OrderProcessorService type.
ServiceHost serviceHost = new ServiceHost(typeof(OrderProcessorService));
// Hook on to the service host faulted events.
serviceHost.Faulted += new EventHandler(OnServiceFaulted);
serviceHost.Open();
// The service can now be accessed.
Console.WriteLine("The poison message service is ready.");
Console.WriteLine("Press <ENTER> to terminate service.");
Console.WriteLine();
Console.ReadLine();
// Close the ServiceHostBase to shutdown the service.
if(serviceHost.State != CommunicationState.Faulted)
{
serviceHost.Close();
}
}
Unlike the order processing service that reads messages from the order queue, the poison message service reads messages from the poison sub-queue. The poison queue is a sub-queue of the main queue, is named "poison" and is automatically generated by MSMQ. To access it, provide the main queue name followed by a ";" and the sub-queue name, in this case -"poison", as shown in the following sample configuration.
.gif) Note: |
|---|
In the sample for MSMQ v3.0, the poison queue name is not a sub-queue, rather the queue that we moved the message to. |
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<system.serviceModel>
<services>
<service name="Microsoft.ServiceModel.Samples.OrderProcessorService">
<!-- Define NetMsmqEndpoint -->
<endpoint address="net.msmq://localhost/private/ServiceModelSamplesPoison;poison"
binding="netMsmqBinding"
contract="Microsoft.ServiceModel.Samples.IOrderProcessor" >
</endpoint>
</service>
</services>
</system.serviceModel>
</configuration>
When you run the sample, the client, service, and poison message service activities are displayed in console windows. You can see the service receive messages from the client. Press ENTER in each console window to shut down the services.
The service starts running, processing orders and at random starts to terminate processing. If the message indicates it has processed the order, you can run the client again to send another message until you see the service has actually terminated a message. Based on the configured poison settings, the message is tried once for processing before moving it to the final poison queue.
The service is ready.
Press <ENTER> to terminate service.
Processing Purchase Order: 0f063b71-93e0-42a1-aa3b-bca6c7a89546
Customer: somecustomer.com
OrderDetails
Order LineItem: 54 of Blue Widget @unit price: $29.99
Order LineItem: 890 of Red Widget @unit price: $45.89
Total cost of this order: $42461.56
Order status: Pending
Processing Purchase Order: 5ef9a4fa-5a30-4175-b455-2fb1396095fa
Customer: somecustomer.com
OrderDetails
Order LineItem: 54 of Blue Widget @unit price: $29.99
Order LineItem: 890 of Red Widget @unit price: $45.89
Total cost of this order: $42461.56
Order status: Pending
Aborting transaction, cannot process purchase order: 23e0b991-fbf9-4438-a0e2-20adf93a4f89
Start up the poison message service to read the poisoned message from the poison queue. In this example, the poison message service reads the message and processes it. You could see that the purchase order that was terminated and poisoned is read by the poison message service.
The service is ready.
Press <ENTER> to terminate service.
Processing Purchase Order: 23e0b991-fbf9-4438-a0e2-20adf93a4f89
Customer: somecustomer.com
OrderDetails
Order LineItem: 54 of Blue Widget @unit price: $29.99
Order LineItem: 890 of Red Widget @unit price: $45.89
Total cost of this order: $42461.56
Order status: Pending
To set up, build, and run the sample
Ensure that you have performed the One-Time Setup Procedure for the Windows Communication Foundation Samples.
To build the C# or Visual Basic .NET edition of the solution, follow the instructions in Building the Windows Communication Foundation Samples.
To run the sample in a single- or cross-machine configuration, change the queue names to reflect the actual hostname instead of localhost and follow the instructions in Running the Windows Communication Foundation Samples.
By default with the netMsmqBinding binding transport, security is enabled. Two properties, MsmqAuthenticationMode and MsmqProtectionLevel, together determine the type of transport security. By default, the authentication mode is set to Windows and the protection level is set to Sign. For MSMQ to provide the authentication and signing feature, it must be part of a domain. If you run this sample on a computer that is not part of a domain, you receive the following error: "User's internal message queuing certificate does not exist".
To run the sample on a computer joined to a workgroup
If your computer is not part of a domain, turn off transport security by setting the authentication mode and protection level to
Noneas shown in the following sample configuration:<bindings> <netMsmqBinding> <binding name="TransactedBinding"> <security mode="None"/> </binding> </netMsmqBinding> </bindings>Ensure the endpoint is associated with the binding by setting the endpoint's bindingConfiguration attribute.
Ensure that you change the configuration on the PoisonMessageServer, server and the client before you run the sample.
Note:Setting security mode to None is equivalent to setting MsmqAuthenticationMode, MsmqProtectionLevel, and Message security to None.
.gif)
Send comments about this topic to Microsoft.
© Microsoft Corporation. All rights reserved.