Share via

Selection of Inbound STARTTLS Certificates

  • Article

Applies to: Exchange Server 2010

The selection of an inbound STARTTLS certificate occurs in the following scenarios:

  • SMTP hosts request Transport Layer Security (TLS) with Edge Transport servers. The host that requests TLS with the Edge Transport server may be any other SMTP host. This also describes the Domain Security scenario. For more information about Domain Security, see Understanding Domain Security.
  • SMTP clients, such as Microsoft Outlook Express, request TLS with Hub Transport servers.
  • Internet-facing Hub Transport servers request TLS with an Edge Transport server.

When an SMTP session is established, the receiving server initiates a certificate selection process to determine which certificate to use in the TLS negotiation. The sending server also performs a certificate selection process. For more information about that process, see Selection of Outbound Anonymous TLS Certificates.

This topic describes the certificate selection process for inbound STARTTLS. All the steps described in this topic are performed on the receiving server. The following figure shows the steps of this process.

Selection of an inbound STARTTLS certificate
Selection of an inbound STARTTLS certificate

  1. When the SMTP session is established, Microsoft Exchange calls a process to load the certificates.
  2. In the load certificate function, the Receive connector to which the session is connected is checked to see whether the AuthMechanism property is set to a value of TLS. You can set the AuthMechanism property on the Receive connector by using the Set-ReceiveConnector cmdlet. You can also set the AuthMechanism property to TLS by selecting Transport Security Layer (TLS) on the Authentication tab of a specific Receive connector.
    If TLS isn't enabled as an authentication mechanism, the server doesn't advertise X-STARTTLS as an option to the sending server and no certificate is loaded. If TLS is enabled as an authentication mechanism, the certificate selection process continues to the next step.
  3. The certificate selection process retrieves the fully qualified domain name (FQDN) value from the Receive connector configuration. If the FQDN value on the Receive connector is null, the server's physical FQDN is retrieved.
  4. The certificate selection process searches the local computer certificate store for certificates that match the FQDN. If a certificate isn't found, the server doesn't advertise X-STARTTLS, no certificate is loaded, and Event ID 12014 is logged in the Application log.
  5. The certificate selection process searches for all certificates in the certificate store that have a matching FQDN. From this list, the certificate selection process identifies a list of eligible certificates. Eligible certificates must meet the following criteria:
    • The certificate is an X.509 version 3 or a later version certificate.
    • The certificate has an associated private key.
    • The Subject or Subject Alternate Name fields contain the FQDN that was retrieved in step 3.
    • The certificate is enabled for SSL/TLS use. Specifically, the SMTP service has been enabled for this certificate by using the Enable-ExchangeCertificate cmdlet.
  6. If no eligible certificates are found after these checks, the server doesn't advertise X-STARTTLS, no certificate is loaded, and Event ID 12014 is logged in the Application log.
  7. From the eligible certificates, the best certificate is selected based on the following sequence:
    1. Sort eligible certificates by most recent Valid from date. Valid from is a Version 1 field on the certificate.
    2. The first valid public key infrastructure (PKI) certificate that's found in this list is used.
    3. If no valid PKI certificates are found, the first self-signed certificate is used.
  8. The certificate is checked to see whether it has expired. The Valid to field in the certificate properties is compared to the current date and time. If the certificate hasn't expired, STARTTLS is advertised. If the certificate has expired, Event ID 12016 is logged in the Application log, but STARTTLS is still advertised.