Understanding Transport Queues

Applies to: Exchange Server 2010

This topic provides an overview of queues in Microsoft Exchange Server 2010 and the queue management tasks that administrators can perform.

Looking for management tasks related to managing transport servers? See Managing Transport Servers.

Contents

Overview

Queue Database Files

Queue Management

Message Retry, Resubmit, and Expiration Intervals

Overview

A queue is a temporary holding location for messages that are waiting to enter the next stage of processing. Each queue represents a logical set of messages that a transport server processes in a specific order.

The Exchange Management Shell and Queue Viewer support two types of interaction with queues. You can use these interfaces to view the status and contents of queues and detailed message properties. You can also use these interfaces to perform actions that modify queues or the messages in the queues.

Exchange 2010 uses an Extensible Storage Engine (ESE) database for queue storage. Formerly known as JET, ESE is a method that defines a low-level API to the underlying database structures in Exchange.

Messages that come from and go to the Internet are queued at the computers that have the Edge Transport server role installed. Messages in transit within the Exchange 2010 organization are queued at the computers that have the Hub Transport server role installed.

Types of Queues

The routing of a message determines the type of queue in which a message is stored. The following types of queues are used in Exchange 2010:

• Submission queue   A persistent queue that's used by the categorizer to gather all messages that have to be resolved, routed, and processed by transport agents. The categorizer is a component of Exchange transport that processes all inbound messages and determines what to do with the messages based on information about the intended recipients. In Exchange 2010, the Edge Transport server uses the categorizer to route the message to the appropriate destination. The Hub Transport server uses the categorizer to expand distribution lists and to identify alternative recipients and forwarding addresses. After the categorizer retrieves full information about recipients, it uses that information to apply policies, route the message, and perform content conversion.
  All messages that are received by a transport server enter processing in the Submission queue. Messages are submitted through a Receive connector, the Pickup directory, or the store driver. The categorizer retrieves messages from this queue and, among other things, determines the location of the recipient and the route to that location. After categorization, the message is moved to a delivery queue or to the unreachable queue. Each Exchange 2010 transport server has only one Submission queue. Messages that are in the Submission queue can't be in other queues at the same time.
• Mailbox delivery queue   The mailbox delivery queues hold messages that are being delivered to a Mailbox server by using encrypted Exchange RPC. Mailbox delivery queues exist on Hub Transport servers only. Mailbox delivery queues hold messages that are being delivered to mailbox recipients whose mailbox data is stored on a Mailbox server that's located in the same site as the Hub Transport server. More than one mailbox delivery queue can exist on a Hub Transport server. The next hop for a mailbox delivery queue is the distinguished name of the mailbox store.
• Remote delivery queue   Remote delivery queues hold messages that are being delivered to a remote server by using SMTP. Remote delivery queues can exist on both Hub Transport servers and Edge Transport servers, and more than one remote delivery queue can exist on each server. Each remote delivery ** queue contains messages that are being routed to recipients that have the same delivery destination. On an Edge Transport server, these destinations are external SMTP domains or SMTP connectors. On a Hub Transport server, these destinations are outside the Active Directory site in which the Hub Transport server is located. Remote delivery queues are dynamically created when they're required and are automatically deleted from the server when they no longer hold messages and the configurable expiration time has passed. By default, a remote delivery queue is deleted three minutes after the last message has left the queue. The next hop for a remote delivery queue is an SMTP domain name, a smart host name or IP address, or an Active Directory site name.
• Poison message queue   The poison message queue is a special queue that's used to isolate messages that are detected to be potentially harmful to the Exchange 2010 system after a server failure. Messages that contain errors that are potentially fatal to the Exchange system are delivered to the poison message queue. This queue is typically empty, and if no poison messages exist, the queue doesn't appear in the queue viewing interfaces. The poison message queue is always in a ready state. By default, all messages in this queue are suspended. The messages can be deleted if they're considered to be harmful to the system. If the event that caused the message to enter the poison message queue is determined to be unrelated to the message, delivery of the message can be resumed. When delivery is resumed, the message enters the Submission queue.
• Unreachable queue   Each transport server can have only one Unreachable queue. The Unreachable queue contains messages that can't be routed to their destinations. Typically, an unreachable destination is caused by configuration changes that have modified the routing path for delivery. Regardless of destination, all messages that have unreachable recipients reside in this queue.

The following table lists the queues that exist on a Hub Transport server or Edge Transport server and their characteristics.

Queues that exist on a Hub Transport server or Edge Transport server

When a message is received by transport, a transport mail item is created and saved to the database. A unique identifier is assigned to the transport mail item when it enters the database. If a message, or transport mail item, is being routed to more than one recipient, the item can have more than one destination. Each destination represents a separate routing solution for the transport mail item, and each routing solution causes a routed mail item to be created.

The routed mail item is a reference to the transport mail item and is the unit of operation for queuing actions. If a transport mail item has more than one routing solution, more than one routed mail item references the same transport mail item. A message that's being sent to recipients in two different domains appears as two distinct messages in the delivery queues, even if only one transport mail item is in the database.

About the Poison Message Queue and the Unreachable Queue

The categorizer sends messages to the Unreachable queue when there's no known route to their destinations. Typically, an unreachable destination is caused by a configuration error that affects the delivery path. For example, the messages will be sent to the Unreachable queue if the following conditions are true:

• There are messages in the remote delivery queue named Contoso.com.
• You delete the Send connector that's used to reach the Contoso.com domain.

By default, the messages in the Unreachable queue have the status of Ready. Messages in the Unreachable queue are never automatically resubmitted. Messages remain in the Unreachable queue until they're manually resubmitted by an administrator, removed by an administrator, or the value specified in the MessageExpirationTimeOut parameter passes.

The poison message queue contains messages that are determined to be potentially harmful to the Exchange 2010 server after a server failure. The messages may be genuinely harmful in their content and format. Alternatively, they may be the results of a poorly written agent that has caused the Exchange server to fail when it processed the supposedly bad messages. All messages in the poison message queue are in a permanently suspended state. The poison message queue can't be resubmitted with the Retry-Queue cmdlet with the Resubmit parameter. To resubmit the messages in the poison message queue, you can use Queue Viewer or the Resume-Message cmdlet to resume the messages. The messages in the poison message queue are never automatically resumed or expired. Messages remain in the poison message queue until they're manually resumed or removed by an administrator.

Return to top

Queue Database Files

All the different queues are stored in a single ESE database. By default, this queue database is located at C:\Program Files\Microsoft\Exchange Server\TransportRoles\data\Queue.

Like any ESE database, the queue database uses log files to accept, track, and maintain data. To enhance performance, all message transactions are written first to log files and memory, and then to the database file. The checkpoint file tracks the transaction log entries that have been committed to the database. During an ordinary shutdown of the Microsoft Exchange Transport service, uncommitted database changes that are found in the transaction logs are always committed to the database.

Circular logging is used for the queue database. This means that the history of committed transactions that are found in the transaction logs isn't maintained. Any transaction logs that are older than the current checkpoint are immediately and automatically deleted. Therefore, the transaction logs can't be replayed for queue database recovery from backup.

The following table lists the files that constitute the queue database.

Files that constitute the queue database

Options for Configuring the Queue Database

You can't use the Exchange Management Console (EMC) or the Shell to configure the queue database. You configure the queue database by modifying the EdgeTransport.exe.config file. The EdgeTransport.exe.config file is an XML application configuration file that's associated with the EdgeTransport.exe file.

For more information about the EdgeTransport.exe.config file, see Understanding the EdgeTransport.exe.Config File.

The <appSettings> section of the EdgeTransport.exe.config file is where you can add new configuration options or modify existing configuration options. Many configuration options that are completely unrelated to the queue database are also available. However they're outside the scope of this topic and won't be discussed.

The configuration options for the queue database that are available in the EdgeTransport.exe.config file are described in the following table.

Message queue database configuration options that are available in the EdgeTransport.exe.config file

Return to top

Queue Management

When you experience a mail flow problem or an influx of spam, you can perform operations that modify the status of queues and messages that are located in queues. You can perform an action on a single object, or you can perform a bulk action on more than one selected object. Use the Queue Viewer graphical user interface and commands in the Shell in Exchange 2010 to retrieve information about messages and delivery queues. After you retrieve this information, you can select the queues and messages that you want to manage.

You use Queue Viewer or commands in the Shell to create filter criteria to identify the queues and messages that you want to manage. The filter criteria are based on the following attributes:

• Queue state
• Queue properties
• Message state
• Message properties

For more information about how to filter queues, see Filter Queues. For more information about how to filter messages, see Filter Messages in Queues.

Queue Management Tasks

You use Queue Viewer or commands in the Shell to view information about queues and messages. You can also use these tools to perform the following actions:

• Suspend queue   This action temporarily prevents delivery of messages that are currently in the queue. The queue continues to accept new messages, but no messages leave the queue. For more information, see Suspend Queues.
• Resume queue   This action reverses the effect of the suspend queue action and enables delivery of queued messages to resume. For more information, see Resume Queues.
• Retry queue   When a connection to the next hop for a queue fails, a retry timer is set. The retry timer schedules subsequent connection tries. The retry queue action overrides the next scheduled connection attempt and tries to connect to the next hop immediately. If no connection is made, the next retry time is reset. For more information, see Retry Queues.
  You can also use the Retry-Queue cmdlet together with the Resubmit parameter to cause the messages in the queue to be resubmitted to the Submission queue and to go back through the categorization process. You can manually resubmit messages that have the following status:
  • Mailbox delivery queues or remote delivery queues that have the status of Retry. The messages in the queues must not be in the Suspended state.
  • Messages in the Unreachable queue that aren't in the Suspended state.
  • Messages in the poison message queue.
    For more information, see Resubmit Messages in Queues.
• Suspend message   This action temporarily prevents delivery of a message. You can use the suspend message action to prevent delivery of a message to all the recipients in a specific queue or to all recipients in all queues. For more information, see Suspend Messages.
• Resume message   This action reverses the effect of the suspend message action and enables delivery of queued messages to resume. You can use the resume message action to resume delivery of a message to all the recipients in a specific queue or to all recipients in all queues. You can also use this action to resubmit messages in the poison message queue. For more information, see Resume Messages.
• Remove message   This action permanently prevents delivery of a message. You can use the remove message action to prevent delivery of a message to any recipients in a specified queue or to all recipients in all queues. You can also configure the remove message action to send a non-delivery report (NDR) to the sender when the message is removed. For more information, see Remove Messages from Queues.
• Export message   This action copies a message to the file path that you specify. The messages aren't deleted from the queue, but a copy of the message is saved to a file location. This enables administrators or officials in an organization to later examine the messages. Before you export a message, you must suspend the message in the queue so that typical delivery doesn't continue during the export process. The export format is compatible with e-mail applications such as Microsoft Office Outlook. Save the message in .eml format to make sure that the operating system associates the file with an e-mail application. For more information, see Export Messages from Queues.

Queue Filtering Scenarios

Filtering generates different views of the queues. You use the queue properties as filter options. By specifying filter criteria, you can quickly locate queues and take action on them. The following scenarios are examples of how you can use queue filtering to manage message flow:

• You receive a message from the Microsoft System Center Operations Manager that indicates that a queue length has exceeded the established threshold. You want to investigate whether a server-wide mail flow problem exists.
  You can create a filter to view all the queues that have a message count that exceeds what you consider typical. If a mail flow problem is indicated, you can select all the queues in the filter results and suspend the queues while you continue to investigate.
• You suspend several queues to investigate the cause of mail flow problems. You determine that the problem was caused by an incorrect connector configuration and is now fixed.
  You can create a filter to view all the queues that have a status of Suspended, and then select all the queues in the filter results and resume the queues.

Queue Properties to Use When Filtering Queues

You can use the queue properties to create a filter and locate queues that meet specified criteria. The following table lists the queue properties by which you can filter and the valid values for those properties.

Queue properties

Operators to Use When Filtering Queues

When you create a queue filter, you must include an operator for the property value to match. The following table shows the comparison operators that you can use in a filter expression and how each operator functions.

Filter expression operators

You can specify multiple expressions in your queue filter by using the -and operator in the Shell or by adding multiple expressions in Queue Viewer. Queues must meet all criteria to be included in the result set. For example, the results of the following command will display a list of queues that have a destination to any SMTP domain name that ends in Contoso.com and that currently contain more than 500 messages.

Get-Queue -Filter {Identity -like "*Contoso.com*" -and MessageCount -gt 500}

Message Filtering Scenarios

Filtering generates different views of the messages in queues. By specifying filter criteria, you can quickly locate messages and take action on them. When an e-mail message is sent to multiple recipients, the message may be located in multiple queues. When you filter by message properties, you can locate messages across all queues. The following scenarios are examples of how you might use message filtering to manage mail flow:

• The Submission queue on the computer that has the Edge Transport server role installed has a high volume of messages that are queued for delivery. Many of the messages have the same subject. Therefore, you suspect that spam is being sent to your organization. You can create a filter to view all the messages that meet the subject criteria. If you determine that the messages are spam, you can select them all and delete them from the delivery queue without sending an NDR.
• A user reports that mail flow is slow. You examine the queues and see that many messages that have random subjects appear to be coming from a single domain. You can create a filter to view all the queued messages from that domain. If you determine that the messages are spam, you can select them all and delete them from the queues without sending an NDR.

Message Properties to Use When Filtering Messages

You can use message properties to create a filter and locate messages that meet specified criteria. The following table lists the message properties by which you can filter and the values that are associated with those properties.

Message properties

Operators to Use When Filtering Messages

When you create a message filter, you must include an operator for the property value to match. The following table shows the comparison operators that you can use in a filter expression and how each operator functions.

Filter expression operators

You can specify a filter that evaluates multiple expressions by using the -and comparison operator in the Shell or by adding multiple expressions in Queue Viewer. To be included in the result set, messages must meet all conditions of the filter. For example, the results of the following command will display a list of messages that are sent from any e-mail address that has a domain name that ends in Contoso.com and that have an SCL that's greater than 5.

Get-Message -Filter {FromAddress -like "*Contoso.com*" -and SCL -gt 5}

Return to top

Message Retry, Resubmit, and Expiration Intervals

Messages that can't be successfully delivered are subject to various retry, resubmit, and expiration deadlines based on the message's source and destination. Retry is a renewed connection attempt with the destination domain, smart host, or Mailbox server. Resubmit is the act of sending messages back to the Submission queue for the categorizer to reprocess. The message is said to time-out or expire after all delivery efforts have failed over a specified period of time. After a message expires, the sender is notified of the delivery failure. Then the message is deleted from the queue.

In all three cases of retry, resubmit, or expire, you can manually intervene before the automatic actions are performed on the messages.

Configuration Options for Message Retry

When a transport server can't connect to the next hop, the queue is put in a status of Retry. Connection attempts continue until the queue expires or a connection is made.

Configuration Options for Automatic Message Retry

The configuration options that are available for message retry intervals are described in the following table.

Configuration options that are available for message retry intervals

The <appSettings> section of the EdgeTransport.exe.config file is where you can add new configuration options or modify existing configuration options. There are many configuration options available that are completely unrelated to the message retry, resubmit, and expiration intervals. Any configuration options that don't involve these intervals are outside the scope of this topic.

For more information about the EdgeTransport.exe.config file, see Understanding the EdgeTransport.exe.Config File.

For more information, see Configure Message Retry, Resubmit, and Expiration Intervals.

Configuration Options for Manual Message Retry

When a mailbox delivery queue or a remote delivery queue is in the status of Retry, you can manually force an immediate connection attempt by using Queue Viewer in the EMC or the Retry-Queue cmdlet in the Shell. The manual retry attempt overrides the next scheduled retry time. If the connection isn't successful, the retry interval timer is reset. The delivery queue must be in a status of Retry for this action to have any effect.

For more information, see Retry Queues.

Configuration Options for Delay DSN Messages

After each message delivery failure, the Edge Transport server or the Hub Transport server generates a delay delivery status notification (DSN) message and queues it for delivery to the sender of the undeliverable message. This delay DSN message is sent only after a specified delay notification time-out interval, and only if the failed message wasn't successfully delivered during that time. By default, the delay notification time-out interval is 4 hours. This delay prevents the sending of unnecessary delay DSN messages that may be caused by temporary message transmission failures. The sending of delay DSN notification messages can be selectively enabled or disabled for messages that originate inside or outside the Exchange organization.

The configuration options that are available for delay DSN notification messages are described in the following table.

Configuration options that are available for delay DSN notification messages

For more information, see Configure Message Retry, Resubmit, and Expiration Intervals.

Configuration Options for Message Resubmission

Message resubmission sends undelivered messages back to the Submission queue to be reprocessed by the categorizer.

Automatic Message Resubmission

Undelivered messages are automatically resubmitted if the delivery queue is in the status of Retry and has been unable to successfully deliver any messages for a specified period of time. That period of time is controlled by the MaxIdTimeBeforeResubmit parameter in the EdgeTransport.exe.config application configuration file. By default, the value of the MaxIdTimeBeforeResubmit parameter is 12 hours. Only messages in mailbox delivery queues or remote delivery queues are candidates for automatic resubmission.

For more information, see Configure Message Retry, Resubmit, and Expiration Intervals.

Manual Message Resubmission

You can manually resubmit messages that have the following status on a Hub Transport server or an Edge Transport server:

• Mailbox delivery queues or remote delivery queues that have the status of Retry. The messages in the queues must not be in the Suspended state.
• Messages that are in the Unreachable queue and aren't in the Suspended state.
• Messages that are in the poison message queue.

For more information about the poison message queue and the Unreachable queue, see "About the Poison Message Queue and the Unreachable Queue" earlier in this topic.

If you want to manually resubmit messages that are located in the mailbox delivery queues, the remote delivery queues, or the Unreachable queue without waiting for the time that's specified by the MaxIdleTimeBeforeResubmit parameter to pass, you must use the Retry-Queue cmdlet with the Resubmit parameter. To manually resubmit messages that are located in the poison message queue, you can use Queue Viewer or the Resume-Message cmdlet to resume the message.

For more information, see the following topics:

• Resubmit Messages in Queues
• Resume Messages

Another way that you can manually resubmit messages is to suspend the messages, export the messages to text files that have the .eml file name extension, and then copy the .eml files to the Replay directory on any Hub Transport server or Edge Transport server. This resubmission method works for messages that are located in the mailbox delivery queues, remote delivery queues, or the Unreachable queue. Messages that are located in the poison message queue are already in the Suspended state. Messages that are located in the Submission queue can't be suspended or exported.

For more information, see Export Messages from Queues and Resubmit Messages in Queues.

Configuration Options for Message Expiration

The message expiration time-out interval specifies the maximum length of time that an Edge Transport server or a Hub Transport server tries to deliver a failed message. If the message can't be successfully delivered before the expiration time-out interval has passed, an NDR that contains the original message or the message headers is delivered to the sender.

Automatic Message Expiration

The message expiration time-out interval is controlled by the MessageExpirationTimeOut parameter in the Set-TransportServer cmdlet or in the transport server properties in the EMC. By default, the value of the MessageExpirationTimeOut parameter is 2 days.

For more information, see the following topics:

• Configure Message Retry, Resubmit, and Expiration Intervals
• Set-TransportServer

Manual Message Expiration

Although you can't manually force messages to expire, you can manually remove messages from any queue, except the Submission queue, with or without an NDR.

For more information, see Remove Messages from Queues.

Return to top