Share via


Create a Simple Data Collector to Integrate Data into Load Test Analyzer (Tables View)

With a custom data collector, you can collect various diagnostic performance information from your Web applications. In my previous post, I have highlighted the key points of integrating custom collector data into Load Test Analyzer. The purpose of this post is to go through the procedure by using a simple collector.

One of the questions some forum users asked is ‘Did Load Test insert item records into my table?’ To find out this, I add a load test containing a Web test that purchases orders from a online store (Web application) to my test project. I create a SimpleRowCountDataCollector to collect the counts of the Order table before and after the Load test run. Then I enable the data collector in a .testsettings file, run the Web test in a Load test, and check the row counts after the run completes.

Since this sample data collector can be downloaded from https://teamtestplugins.codeplex.com/, I will only outline the steps in this post. If you would like more detailed instructions, read this page.

Step 1 – Create a Data Collector

1. Create a new class library

2. Add references to the following dlls

Microsoft.VisualStudio.QualityTools.ExecutionCommon

Microsoft.VisualStudio.QualityTools.Common.dll

3. Add RowCountDataCollector.cs

Specify the URI and a friendly name for the collector

     [DataCollectorTypeUri("datacollector://MyCompany/SimpleRowCountDataCollector/1.0")]
    [DataCollectorFriendlyName("RowCountDataCollector", false)]
    [DataCollectorConfigurationEditor("configurationeditor://Microsoft/GenericEditor/1.0")]
    public class SimpleRowCountDataCollector : DataCollector

Below are the brief descriptions of the major functions.

3.1 Initialize() function

This function does three thing

- Init data members

- Retrieve the collector configuration from the test settings file

- Register the TestCaseStart and TestCaseEnd events

        // Required method called by the testing framework
        public override void Initialize(
            XmlElement configurationElement, 
            DataCollectionEvents events, 
            DataCollectionSink sink, 
            DataCollectionLogger logger, 
            DataCollectionEnvironmentContext environmentContext)
        {
            // 1. Init members 
            dataEvents = events; // The test events
            dataLogger = logger; // The error and warning log
            dataSink = sink;     // Saves collected data
            
            // Configuration from the test settings
            configurationSettings = configurationElement;

            // 2. Read collector configurations from the test settings
            CollectorConfigurationManager configManager = new CollectorConfigurationManager(configurationSettings);
            server = configManager["server"];
            if (string.IsNullOrEmpty(server))
            {
                server = Environment.MachineName;
            }

            database = configManager["database"];
            table = configManager["table"]; 

            // 3. Register TestCaseStart and TestCaseEnd events for the data collector
            dataEvents.TestCaseStart +=
                new EventHandler<TestCaseStartEventArgs>(OnTestCaseStart);

            dataEvents.TestCaseEnd +=
                new EventHandler<TestCaseEndEventArgs>(OnTestCaseEnd);
        }

3.2 OnTestCaseStart Event Handler

 This function get the order count from the Order table right before the load test is started. As I mentioned in the previous post, you can log warning or error messages in the code. The message will show up in the Load Test Errors Table.
         public void OnTestCaseStart(object sender, TestCaseEventArgs e)
        {
            // dataLogger.LogWarning(e.Context, "OnTestCaseStart. Time:" + DateTime.Now.ToString());
            rowCountBeforeTestRun = SQLHelper.GetRowCount(server, database, table);
        }

3.3 OnTestCaseEnd Event Handler

 This function get the order count from the Order table after the load test is completed, write the tabular data in XML format, and send the file back to client (VS).
        // Write the counts to a table in XML format
        public void OnTestCaseEnd(object sender, TestCaseEndEventArgs e)
        {
            // Get the row count after run ends
            rowCountAfterTestRun = SQLHelper.GetRowCount(server, database, table);

            // create the table to be integrated into load test database
            string filePath = Path.Combine(Path.GetTempPath(), "rowCounts.xml");
            RowCountXMLWriter.CreateDataTable();
            RowCountXMLWriter.AddDataRow("BeforeTestStart", rowCountBeforeTestRun);
            RowCountXMLWriter.AddDataRow("AfterTestEnd", rowCountAfterTestRun);
            RowCountXMLWriter.WriteRowCountsToXmlFile(filePath);

            // Send the file back
            dataSink.SendFileAsync(e.Context, filePath, false);
        }
 One important thing to keep in mind is that only files sent back in OnTestCaseStart and OnTestCaseEnd events can be added to Load Test Tables view in post run.

4. Add RowCountXMLWriter.cs file

This class creates a DataTable, add rows to the table, and write out the data in XML format

         static DataTable m_dataTable;

        public static void CreateDataTable()
        {
            m_dataTable = new DataTable();

            m_dataTable.Locale = CultureInfo.InvariantCulture;
            m_dataTable.Columns.Add(new DataColumn("Description", typeof(string)));
            m_dataTable.Columns.Add(new DataColumn("RowCount", typeof(Int32)));
        }

        public static void AddDataRow(string description, int rowCount)
        {
            DataRow row = m_dataTable.NewRow();
            row["Description"] = description;
            row["RowCount"] = rowCount;
            m_dataTable.Rows.Add(row);
        }

        public static void WriteRowCountsToXmlFile(string xmlFilePath)
        {
            DataSet dataSet = new DataSet();
            dataSet.Locale = CultureInfo.InvariantCulture;
            dataSet.Tables.Add(m_dataTable);

            using (System.IO.StreamWriter xmlStreamWriter = new System.IO.StreamWriter(xmlFilePath))
            {
                dataSet.WriteXml(xmlStreamWriter, XmlWriteMode.WriteSchema);
                xmlStreamWriter.Close();
            }

            dataSet.Dispose();
        }

5. Add ConfigurationManager.cs

This class reads the data collector settings into a dictionary. Please note, it is written for the format described in Step 6, section <DefaultConfiguration>.

6. Add an App.config file

Note this data collector is configured to use a Generic XML editor. If you want to create your own UI editor, read this MSDN page.

[DataCollectorConfigurationEditor("configurationeditor://Microsoft/GenericEditor/1.0")]
public class SimpleRowCountDataCollector : DataCollector

[App.config]

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSection>
    <section
      name="DataCollectorConfiguration"
      type="Microsoft.VisualStudio.QualityTools.Execution.DataCollectorConfigurationSection,
        Microsoft.visualStudio.QualityTools.ExecutionCommon,
        Version=4.0.0.0, Culture=neutral,
        PublicKeyToken=b03f5f7f11d50a3a" />
  </configSection>
  <DataCollectorConfiguration xmlns="https://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <DataCollector typeUri="datacollector://MyCompany/SimpleRowCountDataCollector/1.0">
    <DefaultConfiguration>
<Setting name="server" value="YourWebServer" />
<Setting name="database" value="Store" />
<Setting name="table" value="Order" />
  </DefaultConfiguration>
  </DataCollector>
  </DataCollectorConfiguration>
</configuration>

This data collector config file allows you to specify three settings, Web server, database, and table name.

Step 2 – Install the Data Collector

You need to drop a copy of the data collector on every machine you want to collect data.

Setup Requirement: You must install Agent on these machines.

Files to drop: RowCountDataCollector.dll, RowCountDataCollector.dll.config

Drop location: %Program Files%\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies\DataCollectors

An example of the setup

Machine1: VS2010 Ultimate, Controller, Agent

Machine2: Web Server, Agent, RowCountDataCollector.

For you to try out the sample collector, you can drop it on your local machine, configure it to point to a table in your local load test database (step 3), and then run the test(step 4).

Step 3 – Configure the Data Collector

Open the Testsetting file used by your load test run. Click on Data and Diagnostics. Enable the RowCountDataCollector.

image

Click on Configure, it will launch the default editor for you to modify the settings.

 image

Step 4 – Run your load test and view the data

Run your load test view the before/after order counts in post run table.

 image

Comments

  • Anonymous
    April 20, 2010
    Hi, It would be great if all VS test web and load could also be cross-posted in the Visual Studio Team Test blog. There are too many blogs to monitor. You should view all the test tools as one product (we use manual, coded UI, web/load testing portion from the tool, for those of us using VS Ultimate and doing multiple types of testing in our overall test strategy) Bertrand

  • Anonymous
    October 26, 2010
    Hi YuTong, First of all, thank you for this article. It's excellent. I have a question. Is it possible to use your custom data collector inside MTM (Microsoft Test Manager)? I need to add the new custom log file automatically in "Attachments" section right after the test was done along with other logs (Event Log and so on). Thank you in advance. Regards, Goran

  • Anonymous
    October 26, 2010
    Hi YuTong, First of all, thank you for this article. It's excellent. I have a question. Is it possible to use your custom data collector inside MTM (Microsoft Test Manager)? I need to add the new custom log file automatically in "Attachments" section right after the test was done along with other logs (Event Log and so on). Thank you in advance. Regards, Goran

  • Anonymous
    October 26, 2010
    The comment has been removed

  • Anonymous
    November 11, 2015
    Hi Get I get example for section 5. Add ConfigurationManager.cs? Thanks