Using PKI on the Edge Transport Server for Domain Security

Applies to: Exchange Server 2010

Domain Security relies on mutual Transport Layer Security (TLS) for authentication. Successful mutual TLS authentication relies on a trusted, validated X.509 certificate chain for the TLS certificates that are used for Domain Security.

Therefore, before you can successfully deploy Domain Security, you must configure your Edge Transport server and your X.509 public key infrastructure (PKI) to accommodate certificate trusting and certificate validation.

Configuring Root Certification Authorities

To validate a given X.509 certificate, you must trust the root certification authority (CA) that issued the certificate. A root CA is the most trusted CA, which is at the top of a CA hierarchy. The root CA has a self-signed certificate. When you run an application that relies on certificate authentication, each certificate must have a certificate chain that ends in a certificate in the trusted root container of the local computer. The trusted root container contains certificates from root certification authorities.

To successfully send domain-secured e-mail, you must be able to validate the receiving server's X.509 certificate. Similarly, when someone sends domain-secured e-mail to your organization, the sending server must be able to validate your certificate.

There are two types of trusted root CAs that you can use to implement Domain Security: Built-in third-party root CAs and private root CAs.

Third-Party Root Certification Authorities

Microsoft Windows includes a set of built-in third-party root CAs. If you trust the certificates issued by these third-party root CAs, this means you can verify certificates issued by these CAs. If your organization and your partner organizations are using the default Windows installation and trust the built-in third-party root CAs, trust is automatic. In this scenario, additional trust configuration is not required.

Private Trusted Root Certification Authorities

A private trusted root CA is a root CA that has been deployed by a private or internal PKI. For example, when your organization or the organization that you exchange domain-secured e-mail with has deployed an internal PKI with its own root certificate, you must make additional trust configurations.

When private root CAs are used, you must update the Windows trusted root certificate store on the Edge Transport server for Domain Security to function correctly.

You can configure trust in two ways: direct root trust and cross-certification. You must understand that whenever the transport service picks a certificate, it validates the certificate before it uses it. Therefore, if you're using a private root CA to issue your certificates, you must include the private root CA in the trusted root certificate store on each Edge Transport server that sends or receives domain-secured e-mail.

Direct Root Trust

If you want to trust a certificate that has been issued by a private root CA, you can manually add that root certificate to the trusted root certificate store on the Edge Transport server computer. For more information about how to manually add certificates to the local certificate store, see the Help file for the Certificate Manager snap-in in Microsoft Management Console (MMC).

Cross-Certification

Cross-certification occurs when one CA signs a certificate that is generated by a different CA. Cross-certification is used to build trust from one PKI with another PKI. In the context of Domain Security, if you have your own PKI, instead of using direct manual trust for a root authority of a partner with an internal PKI, you might create a cross-certificate for the partner CA under your root authority. In this case, trust is established because the cross-certificate ultimately chains back to your trusted root.

You must understand that if you have an internal PKI and are using cross-certification, you must manually update the root certificate store on each Edge Transport server that receives domain-secured e-mail so that each Edge Transport server can validate certificates when they receive e-mail from partners that are trusted through cross-certificates.

For more information about how to manually add certificates to the local certificate store, see the Help file for the Certificate Manager snap-in in MMC.

Configuring Access to the Certificate Revocation List

Whenever the Transport service retrieves a certificate, it validates the certificate chain and validates the certificate. Validation of the certificate is a process in which many attributes of the certificate are confirmed. Most of these attributes can be confirmed on the local computer by the application that requests the certificate. For example, the intended use of the certificate, the expiration dates on the certificate, and similar attributes are verifiable outside the context of a PKI. However, verification that the certificate has not been revoked must be validated with the CA that issued the certificate. Therefore, most CAs make a certificate revocation list (CRL) available to the public to validate the revocation status.

To successfully use Domain Security, CRLs for CAs that you use or are used by your partners must be available to the Edge Transport servers that send and receive domain-secured e-mail. If the revocation check fails, the receiving Exchange server issues a temporary protocol rejection of the message. A transient revocation failure can occur. For example, the Web server that is used to publish the CRL can fail. Or, general network connectivity issues between the Edge Transport server and the CRL distribution point could fail the revocation check. Therefore, transient revocation failures only cause temporary mail delivery delays because the sending server will retry later. However, CRL validation is required for successful domain-secured e-mail transmission.

You must enable the following scenarios:

• Your Edge Transport servers must be able to access CRLs for external CAs   Each partner with which you exchange domain-secured e-mail must have publicly available CRLs that your organization's Edge Transport server can contact. In some cases, CRLs are only available with Lightweight Directory Access Protocol (LDAP). In most cases, with public CAs, CRLs are published via HTTP. Make sure that the appropriate outbound ports and proxies are configured to let the Edge Transport server contact the CRL. You can determine which protocol a given CRL distribution point accepts by opening a certificate in MMC and viewing the CRL Distribution Points field.
• You must make the CRL for the CA that issues your certificates publicly available   You must understand that even when an Edge Transport server retrieves a certificate from your own organization, it validates the certificate chain to validate the certificate. Therefore, the CRL for your CA must be available to your own Edge Transport servers. In addition, all partners that you exchange domain-secured e-mail with must be able to access the CRL for the CA that issues your certificates.

Configuring Proxy Settings for WinHTTP

Exchange 2010 transport servers rely on the underlying Microsoft Windows HTTP Services (WinHTTP) to manage all HTTP and HTTPS traffic. Both Hub Transport servers and Edge Transport servers may use HTTP to access updates for Microsoft Exchange 2010 Standard Anti-spam Filter Updates and for CRL validation.

For more information, see Configure Proxy Settings for WinHTTP.

Testing the PKI and Proxy Configuration

To verify your PKI and proxy configuration for a specific Edge Transport server, use Certutil.exe to verify the certificate chain for your Edge Transport server certificate. Certutil.exe is a command-line tool that is installed as part of Certificate Services in Windows Server 2008 operating systems. For more information, see Test PKI and Proxy Configuration.