Share via

Configure authentication and encryption on the server

  • Article

Applies To: Windows Server 2003, Windows Server 2003 R2, Windows Server 2003 with SP1, Windows Server 2003 with SP2

To configure authentication and encryption on the server

  • Configure the encryption level by using Group Policies

  • Using Terminal Services Configuration

Configure the encryption level by using Group Policies

  1. Open Group Policy.

  2. In Computer Configuration, Administrative Templates, Windows Components, Terminal Services, Encryption and Security, double-click the Set client connection encryption level setting, and then click Enabled.

  3. To set the encryption level, do one of the following:

    • To set the encryption level to Client Compatible, High Level, or Low Level, in the Encryption Level list, click the level that you want, and then click OK. For information about these encryption levels, see Note, at this end of this topic.

    • To enable FIPS compliant encryption, click OK to close the Set client connection encryption level Properties dialog box, and then navigate to Computer Configuration, Windows Settings, Security Settings, Local Policies, Security Options. Double-click System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing, and then click Enabled.

  4. If you set the encryption level to High Level or if you enabled FIPS compliant encryption and you want to use Transport Level Security (TLS) 1.0 to authenticate the server, you must enable TLS by using the Terminal Services Configuration tool and meet additional configuration requirements.

    You cannot use Group Policy to enable TLS authentication. For more information, see Using Terminal Services Configuration, later in this topic.

Important

  • Any encryption level settings that you configure in Group Policy override the configuration that you set by using the Terminal Services Configuration tool. Also, if you enable the System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing Group Policy setting, this setting overrides the Set client connection encryption level Group Policy setting.

  • In order for clients to be able to connect to a terminal server that uses FIPS compliant encryption, you must upgrade these clients to use the RDP 5.2 (Windows Server 2003) client. You can install this client from Windows Server 2003 terminal servers. For more information, see Remote Desktop Connection for Windows Server 2003 [5.2.3790] (https://go.microsoft.com/fwlink?/LinkID=41068).

  • Use this procedure to configure the local Group Policy object. To change a policy for a domain or an organizational unit, you must log on to the primary domain controller as an administrator. Then, you must open Group Policy by using the Active Directory Users and Computers snap-in.

  • You should thoroughly test any changes you make to Group Policy settings before applying them to users or computers. For more information on testing policy settings, see Resultant Set of Policy.

  • To perform this procedure, you must be a member of the Administrators group on the local computer, or you must have been delegated the appropriate authority. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure. As a security best practice, consider using Run as to perform this procedure. For more information, see Default local groups, Default groups, and Using Run as.

Using Terminal Services Configuration

  1. Open Terminal Services Configuration.

  2. In the console tree, click Connections.

  3. In the details pane, right-click the connection you want to modify, and then click Properties.

  4. On the General tab, in Security layer, select a security method. The security method that you select determines whether the terminal server is authenticated to the client, and the level of encryption that you can use. You can select from these security methods.

    • The Negotiate method uses TLS 1.0 to authenticate the server, if TLS is supported. If TLS is not supported, the server is not authenticated.

    • The RDP Security Layer method uses native Remote Desktop Protocol encryption to secure communications between the client and server. If you select this setting, the server is not authenticated.

    • The SSL method requires the use of TLS 1.0 to authenticate the server. If TLS is not supported, the connection fails. This method is only available if you select a valid certificate, as described in Step 6.

    If you select Negotiate or SSL, for TLS to function correctly, you must also set the encryption level to High, or you must enable FIPS compliant encryption by using Group Policy or Terminal Server Configuration. Additional server and client configuration requirements must also be met. For more information about requirements and tasks for configuring Terminal Server to support TLS authentication, see Configuring authentication and encryption.

  5. In Encryption level, click the level that you want. You can select Low, Client Compatible, High, or FIPS Compliant. For more information about these levels, see Notes, at the end of this topic.

  6. To use TLS 1.0 to authenticate the server, in Certificate, click Browse, click Select Certificate, and then click the certificate that you want to use. The certificate must be an X.509 certificate with a corresponding private key. For instructions on how to verify whether the certificate has a corresponding private key, see Notes.

  7. To specify that clients log on to the terminal server by typing their credentials in the default Windows logon dialog box, select the Use standard Windows logon interface check box.

Note

  • To perform this procedure, you must be a member of the Administrators group on the local computer, or you must have been delegated the appropriate authority. If the computer is joined to a domain, members of the Domain Admins group might be able to perform this procedure. As a security best practice, consider using Run as to perform this procedure.

  • To open Terminal Services Configuration, click Start, click Control Panel, double-click Administrative Tools, and then double-click Terminal Services Configuration.

  • Any encryption level settings that you configure in Group Policy override the configuration that you set by using the Terminal Services Configuration tool. Also, if you enable the System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing Group Policy setting, this setting overrides the Set client connection encryption level Group Policy setting.

  • When you change the encryption level, the new encryption level takes effect the next time a user logs on. If you require multiple levels of encryption on one server, install multiple network adapters and configure each adapter separately.

  • To verify that certificate has a corresponding private key, in Terminal Services Configuration, right-click the connection for which you want to view the certificate, click the General tab, click Edit, click the certificate that you want to view, and then click View Certificate. At the bottom of the General tab, the statement, "You have a private key that corresponds to this certificate" should appear. You can also view this information by using the Certificates snap-in.

  • The FIPS compliant setting (the System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing setting in Group Policy or the FIPS Compliant setting in Terminal Server Configuration) encrypts and decrypts data sent from the client to the server and from the server to the client, with the Federal Information Processing Standard (FIPS) 140-1 encryption algorithms, using Microsoft cryptographic modules. For more information, see FIPS 140 Evaluation (https://go.microsoft.com/fwlink/?LinkID=34627).

  • The High setting encrypts data sent from the client to the server and from the server to the client by using strong 128-bit encryption.

  • The Client Compatible setting encrypts data sent between the client and the server at the maximum key strength supported by the client.

  • The Low setting encrypts data sent from the client to the server using 56-bit encryption.

Information about functional differences

  • Your server might function differently based on the version and edition of the operating system that is installed, your account permissions, and your menu settings. For more information, see Viewing Help on the Web.

See Also

Concepts

Working with MMC console files
Configuring authentication and encryption
Request a computer certificate for server authentication
Request a certification authority certificate for the client
Configure authentication on the client by using Remote Desktop Connection
Configuring Terminal Services with Group Policy