Authentication Delay for sites Published through ISA server 2006 using Forms Based Authentication

Introduction

Consider the following scenario: users logging to the websites published through ISA server 2006 using FBA (Forms Based Authentication) with LDAPS as authentication method were take long time to logon. Once they were logged in, the performance was normal. The delay was around 15 to 20 seconds that clearly happened during the initial logon process, after typing the credentials on FBA.

Data Collection

In order to find out why the delay is happening we need to collect data while doing a repro of the issue as follows:

• Test client machine: logon to the website where we get delay in the logon process.
• ISA server: Use ISA Data packager in repro mode with web proxy and web publishing template to collect data, when user is trying to logon to the website.

Data Analysis

When reviewing the netmon captures from the internal NIC of ISA server we found that when ISA Server was trying to communicate with the domain controller there was a delay of 7 seconds that happened during the during SSL handshake as shown below:

The SSL handshake is expected in this case since ISA Server needs to authenticate the user using LDAPS, therefore the first step is to establish the SSL handshake, during this process the domain controller would present its certificate (server authentication certificate) to ISA server for authentication, once this authentication process completes, SSL handshake completes and SSL connection starts (reference : http://technet.microsoft.com/en-us/library/cc514301.aspx and http://support.microsoft.com/kb/257591 ). As you can see in the above capture, there is a delay in the SSL handshake process.

Troubleshooting and Resolution

There are many components in this process that could be causing such delay, best thing to do is to narrow it down which component is causing that. Here it is the checklist that was used in this scenario:

• Make sure that the server authentication certificate on the domain controller does not have “Client Authentication” attribute enabled as per article http://technet.microsoft.com/en-us/library/cc514301.aspx.
• Make sure that the CRL check is also not an issue as per http://blogs.technet.com/b/pki/archive/2006/11/30/basic-crl-checking-with-certutil.aspx and http://blogs.technet.com/b/yuridiogenes/archive/2009/03/28/unable-to-logon-using-forms-base-authentication-through-isa-server-2006.aspx.
• For this particular scenario the resolution was: initially it was detected that there was multiple certificates on the Domain Controller and one of the certificates had an unknown Enhanced Key usage type. In collaboration with Directory Services Team, we deleted the wrong certificates(took backup of these certificates along with the private keys) and only kept the correct server authentication certificate. The hotfix http://support.microsoft.com/kb/932834 was also applied, this hotfix makes sure that correct certificate is used on the domain controller and rebooted the domain controller. After that correct certificate was picked and we were able to logon to the websites quickly without delays seen earlier.

As you can see, in this particular scenario ISA Server 2006 was only a victim of an issue on the Domain Controller.

Author
Suraj Singh
Support Engineer
Microsoft CSS Forefront Security Edge Team

Technical Reviewer
Yuri Diogenes
Sr Support Escalation Engineer
Microsoft CSS Forefront Security Edge Team

Comments

• Anonymous
  November 11, 2010
  The comment has been removed
• Anonymous
  November 11, 2010
  Hi Suraj , As per my blog post mentioned above, I'm not sure what sets the permissions on these machine key files. I have a few other questions on the post too -  It would be great if you could shed some light on them.  I always planed to do some more tests and see under what conditions the machine keys had the wrong permissions set. Kind Regards, Rhys