Load Testing Web Services with Unit Tests

This post is going to focus on load testing web services.  The web services will be called from unit tests which will then be added to a load test.  The example will also show how to use timers in a unit test which will then show up in the load test monitor. 

I will use 2 Web Services for this example.  The default HelloWorld service and another service called HelloUniverse.  A slight delay has been added to each web service call to make it easier to see the timing differences in the load test. Here is the code for my sample web services.  You can add this to a Web Service web site project to run through the rest of this example.

using System;

using System.Web;

using System.Web.Services;

using System.Web.Services.Protocols;

[WebService(Namespace = "https://tempuri.org/")]

[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]

public class HelloService : System.Web.Services.WebService

{

    public HelloService () {

       

    }

    [WebMethod]

    public string HelloWorld() {

        //Sleep for half a second

        System.Threading.Thread.Sleep(500);

        return "Hello World";

    }

    [WebMethod]

    public string HelloUniverse()

    {

        //sleep for one second

        System.Threading.Thread.Sleep(1000);

        return "Hello Universe";

    }

   

}

 

Now we will create a unit test from the web service.  Follow this link for more in depth information on creating unit tests: Unit Test Walkthrough

1. Create a test project

2. Right Click on the test project and select Add Web Reference…

3. Enter the URL to your web service in the Add Web Reference dialog.  In my example, I entered the URL for the HelloService Service and I gave the reference a name of Hello.

4. Click Add Reference to add the reference.

5. The next step is to create the unit test.  Right Click on the project and select Add New Item…

6. Select Unit Test in the Add New Item dialog.

7. Now it is time to create the unit test which will call the web service.  My unit test looks like the following:

 

using System;

using System.Text;

using System.Collections.Generic;

using Microsoft.VisualStudio.TestTools.UnitTesting;

//add using reference to the webservice

using TestProject1.Hello;

namespace TestProject1

{

    /// <summary>

  /// Unit tests for HelloService

    /// </summary>

    [TestClass]

    public class UnitTest1

    {

        public UnitTest1()

        {

           

        }

        [TestMethod]

        public void SayHello()

        {

            //Create the HelloService Object

            HelloService hello = new HelloService();

            //call hello world service

            string response = hello.HelloWorld();

            //verify it was successful

            Assert.Equals(response, "Hello World");

     //call hello universe service

            response = hello.HelloUniverse();

            //verify it was successful

            Assert.Equals(response, "Hello Universe");

        }

    }

}

 

8. The next step is to verify the unit test works as expected.  You can do this by running the unit test from the Test View window.  You can open the test view window from the Test Menu.  Test -> Other Windows -> Test View

If the test is working as expected, we can create the load test.  Follow this link for more information on Load Tests:  Load Tests

1. Right click on the test project, select Add and then select Load Test… in the fly out menu.

2. The previous step will launch the Load Test Wizard.  Let’s walk through the wizard:

a. Click Next on the first screen.

b. The second screen defines the load test scenario.  Enter a name for the scenario.  Think Time profile does not affect unit tests.  If you want to add a delay between iterations of a unit test, you can Change the Think Time between test iterations. 

c. The next pane is for selecting the load profile.  There are 2 profiles available to you: Constant and Step Load.  Select the profile and settings for your test.

d. The next pane is for Adding tests to the load test.  Click the Add button and add your unit test to the load test.

e. The next pane is for adding counter sets to the load test.  Please follow the above link for more information on counter sets.  I would suggest mapping the IIS and ASP.Net counter sets to your web server.  This will instruct the load test to collect the counters defined in these sets on the web server machine.  If you do map counter sets to the web server, then the user that runs the load test needs to have permission on the web server for collecting performance counters.

f. The next pane is for configuring some other load test settings such as the duration of the load test.  Set the duration to 5 minutes so we can smoke test the Load Test.  Click Finish when you have completed these settings.

 

After clicking finish, you will be brought into the Load Test Editor.  At this point, you can click the play button on the load test editor tool bar to run the load test.  Clicking play will launch the Load Test Monitor.  The default graph will have 2 lines: User Load and Tests/Sec and it will look like the following:

 

 

You may also want to add the specific counters for this test.  These are found in the counter tree under: Scenario1 -> SayHello ->. 

 

 

You should also look at the counters which you are collecting on the web servers and other machines under test to see they are handling the user load.  If any threshold violations or errors have occurred, a link will appear on the status bar.  This is a good spot to start looking for problems.

 

Next, You should look through the available tables.  The test and transaction tables will provide useful information on how long it is taking to execute the web service calls.  The unit test which I have defined above does not have any transactions defined, so the transaction table will be blank.  Here is the Test Table:

 

Transactions are useful if you want to collect timing information on a subset of an entire test.  For example, the test defined above makes 2 web service calls.  The test table and average test time counter give me information on how long it takes to make both calls, but what if I want information for each call?  One way to do this is by defining transactions, which are also called timers in unit tests.  Here is the same unit test, but with 3 timers defined.  One for both calls, one for the HelloWorld call and one for the HelloUniverse call.

using System;

using System.Text;

using System.Collections.Generic;

using Microsoft.VisualStudio.TestTools.UnitTesting;

//add using reference to the webservice

using TestProject1.Hello;

namespace TestProject1

{

    /// <summary>

    /// Unit tests for HelloService

    /// </summary>

    [TestClass]

    public class UnitTest1

    {

        TestContext testContextInstance1;

        public UnitTest1()

        {

        

        }

        //used to call the base methods of TestContext

        public TestContext TestContext

        {

            get { return testContextInstance1; }

            set { testContextInstance1 = value; }

        }

        [TestMethod]

        public void SayHello()

        {

            //Create the HelloService Object

            HelloService hello = new HelloService();

            //Start HelloBoth and HelloWorld timers

            TestContext.BeginTimer("HelloBoth");

            TestContext.BeginTimer("HelloWorld");

            //call hello world service

            string response = hello.HelloWorld();

            //end HelloWorld Timer

            TestContext.EndTimer("HelloWorld");

            //verify it was successful

    Assert.Equals(response, "Hello World");

            //Start HelloUniverse timer

            TestContext.BeginTimer("HelloUniverse");

            //call hello universe service

            response = hello.HelloUniverse();

            //end HelloUniverse Timer

            TestContext.EndTimer("HelloUniverse");

            //end HelloBoth Timer

            TestContext.EndTimer("HelloBoth");

            //verify it was successful

            Assert.Equals(response, "Hello Universe");

        }

    }

}

Save and compile the unit tests then run the load test again. This time when you go to the transactions table, you will see the 3 transactions. This shows you that the unit test spends more time in the HelloUniverse transaction than the HelloWorld transaction.

 

 

 

Another option that you have with timers is you can graph them. Go back to the graph view. The timer/transaction counters are stored under the test they are defined in. For my example it is: Scenario1->SayHello ->Transactions->HelloBoth->Avg.Transaction Time.

 

 

 

You also have the option to run multiple unit tests inside of a load test. This allows you to test multiple web service or other api calls from one test and to see how your servers react to having each of the services called together. When adding mutliple tests, you can set a percentage on each test which indicates how often the test will be called. For example, now suppose my unit test class looks like the following. Notice that 2 unit tests are defined:

 

using System;

using System.Text;

using System.Collections.Generic;

using Microsoft.VisualStudio.TestTools.UnitTesting;

//add using reference to the webservice

using TestProject1.Hello;

namespace TestProject1

{

    /// <summary>

    /// Unit tests for HelloService

    /// </summary>

    [TestClass]

  public class UnitTest2

    {

        TestContext testContextInstance1;

        public UnitTest2()

        {

        }

        //used to call the base methods of TestContext

        public TestContext TestContext

        {

            get { return testContextInstance1; }

            set { testContextInstance1 = value; }

        }

        [TestMethod]

        public void CallHelloWorld()

        {

            //Create the HelloService Object

            HelloService hello = new HelloService();

            //Start HelloWorld timers

            TestContext.BeginTimer("HelloWorld");

            //call HelloWorld service

            string response = hello.HelloWorld();

            //end HelloWorld Timer

            TestContext.EndTimer("HelloWorld");

            //verify it was successful

            Assert.Equals(response, "Hello World");

           

        }

        [TestMethod]

        public void CallHelloUniverse()

        {

            //Create the HelloService Object

     HelloService hello = new HelloService();

            //Start HelloUniverse timer

            TestContext.BeginTimer("HelloUniverse");

            //call HelloUniverse service

            string response = hello.HelloUniverse();

            //end HelloUniverse Timer

            TestContext.EndTimer("HelloUniverse");

            //verify it was successful

            Assert.Equals(response, "Hello Universe");

        }

    }

}

I can now add both the CallHelloUniverse test and the CallHelloWorld test to a load test and both will be executed during the load test execution. If I know that about 75% of the users hit the CallHelloWorld method and 25% hit the CallHelloUniverse service, I can set these as the percentages in the load test and this is about how often each test will be called. If you perform these steps and run the load test, you can easily see the percentage differences by looking at the test table in the Load Test Monitor.

 

 

I hope this gives you an idea on how you can use unit tests to call web services and hit these services under load.   Please let me know if you have more questions on how this process works.

 

Here are the same code examples in VB:

Web Service:

Imports System.Web

Imports System.Web.Services

Imports System.Web.Services.Protocols

<WebService(Namespace:="https://tempuri.org/")> _

<WebServiceBinding(ConformsTo:=WsiProfiles.BasicProfile1_1)> _

<Global.Microsoft.VisualBasic.CompilerServices.DesignerGenerated()> _

Public Class HelloService

    Inherits System.Web.Services.WebService

    <WebMethod()> _

    Public Function HelloWorld() As String

        System.Threading.Thread.Sleep(500)

        Return "Hello World"

    End Function

    <WebMethod()> _

    Public Function HelloUniverse() As String

        System.Threading.Thread.Sleep(1000)

        Return "Hello Universe"

    End Function

End Class

Unit Test:

Imports System

Imports System.Text

Imports System.Collections.Generic

Imports Microsoft.VisualStudio.TestTools.UnitTesting

Imports TestProject1.Hello

<TestClass()> Public Class UnitTest1

    <TestMethod()> Public Sub SayHello()

        'Create the HelloService Object

        Dim hello As HelloService = New HelloService()

        'call hello world service

        Dim response As String = hello.HelloWorld()

        'verify it was successful

        Assert.Equals(response, "Hello World")

        'call hello universe service

        response = hello.HelloUniverse()

        'verify it was successful

        Assert.Equals(response, "Hello Universe")

    End Sub

End Class

Unit Test With Transcations:

Imports System

Imports System.Text

Imports System.Collections.Generic

Imports Microsoft.VisualStudio.TestTools.UnitTesting

Imports TestProject1.Hello

<TestClass()> Public Class UnitTest1

    Dim testContextInstance1 As TestContext

    'used to call the base methods of TestContext

  Public Property TestContext() As TestContext

        Get

            Return testContextInstance1

        End Get

        Set(ByVal Value As TestContext)

            testContextInstance1 = value

        End Set

    End Property

    <TestMethod()> Public Sub SayHello()

        'Create the HelloService Object

        Dim hello As HelloService = New HelloService()

        'Start HelloBoth and HelloWorld timers

        TestContext.BeginTimer("HelloBoth")

        TestContext.BeginTimer("HelloWorld")

       'call hello world service

        Dim response As String = hello.HelloWorld()

        'end HelloWorld Timer

        TestContext.EndTimer("HelloWorld")

        'verify it was successful

        Assert.Equals(response, "Hello World")

        'Start HelloUniverse timer

        TestContext.BeginTimer("HelloUniverse")

        'call hello universe service

        response = hello.HelloUniverse()

        'end HelloUniverse Timer

        TestContext.EndTimer("HelloUniverse")

        'end HelloBoth Timer

     TestContext.EndTimer("HelloBoth")

        'verify it was successful

        Assert.Equals(response, "Hello Universe")

    End Sub

End Class

Unit Test class with 2 unit tests:

Imports System

Imports System.Text

Imports System.Collections.Generic

Imports Microsoft.VisualStudio.TestTools.UnitTesting

Imports TestProject1.Hello

<TestClass()> Public Class UnitTest2

    Dim testContextInstance1 As TestContext

    'used to call the base methods of TestContext

    Public Property TestContext() As TestContext

        Get

            Return testContextInstance1

        End Get

        Set(ByVal Value As TestContext)

            testContextInstance1 = value

        End Set

    End Property

    <TestMethod()> Public Sub CallHelloWorld()

        'Create the HelloService Object

        Dim hello As HelloService = New HelloService()

        'Start HelloWorld timers

        TestContext.BeginTimer("HelloWorld")

        'call HelloWorld service

        Dim response As String = hello.HelloWorld()

     'end HelloWorld Timer

        TestContext.EndTimer("HelloWorld")

        'verify it was successful

        Assert.Equals(response, "Hello World")

    End Sub

    <TestMethod()> Public Sub CallHelloUniverse()

        'Create the HelloService Object

  Dim hello As HelloService = New HelloService()

        'Start HelloUniverse timer

        TestContext.BeginTimer("HelloUniverse")

        'call HelloUniverse service

        Dim response As String = hello.HelloUniverse()

        'end HelloUniverse Timer

        TestContext.EndTimer("HelloUniverse")

        'verify it was successful

        Assert.Equals(response, "Hello Universe")

    End Sub

End Class

Comments

• Anonymous
  December 15, 2006
  Here is an article which describes how to load test web services with unit tests: http://blogs.msdn.com/slumley/pages/load-testing-web-services-with-unit-tests.asp

• Anonymous
  June 21, 2007
  Hi , Its a very useful .  I have one issue in Avg.Response time and Avg. Transaction Time both the values for my loadtest  has no values  i mean '-' and '0'. Even i have defined transaction in my unit test as below,                TestContext.BeginTimer("t1");                List<PersonLocationInformation> actual = target.TestSummary(100);                TestContext.EndTimer("t1"); Please help me to fix this issue Thanks in advance

• Anonymous
  June 27, 2007
  Is there at least 1 iteration of the unit test completing.  Transaction counters are not reported until at least one iteration completes

• Anonymous
  July 15, 2007
  Hi Sean, I am new to this concept. I have to perform load testing of a web service and for this your article is quite informative, an excellent one. However, I need some guidence as well. I have to know the time in which my web service does its processing and sends back the xml response.I need to do this load testing for a constant load of 1000 users and want to know this performance unit(time it takes to process a request and send back the response) under such load.Can you please guide me regarding it? Moreover, could you please clarify if a load of 1000 user means that the load test would continously send 1000 request to the web service after each time interval as stated in sample rate or it means that it just send a request to 1000 users and then samples the response and gives the data accordingly. It would be of real help if you could guide me regarding these issue. Thanks in anticipation

• Anonymous
  August 15, 2007
  Rmehera, You could get the time that it takes for the web service calls by using the timer objects described in this article.  Just surround your call with begin and end timer calls. 1000 users means that we will have 1000 virtual users executing your unit test or web test.  So if the only thing in your unit test is a call to web service, then it basically means that there would be continuous requests from 1000 users.

• Anonymous
  February 10, 2010
  Hi, The above examples are not really testing the Web Services but instead the class libraries implementing these. The above tests are not calling the web services via a proxy. Consequently, any proxy instanciation and network overhead for calling the web services won't be outlined in the load tests.

• Anonymous
  June 15, 2010
  wow leveling guide Pretty good post. I just stumbled upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I'll be subscribing to your feed and I hope you post again soon.

• Anonymous
  November 25, 2010
  I rellay like this port.Thanks for writing such a nice post.

• Anonymous
  December 07, 2014
  Will it still work if I call a web service (eg. Amazon web service) and I have no control over their Web/SQL servers. I want to load test my unit test that call their service? and How do I set the performance counter? I want to know how long does my unit test take? and how long does the api call take? How can I load test them?