Share via

Ordered Tests in TFS Build

  • Article

In an earlier article I discussed how to use and Ordered Test to control the execution order of Coded UI Tests (the same can be applied to other test types as well).  I received a few follow-up questions about how to do this in TFS Build so tests run in a particular order as part of a build.

Here’s one way that’s remarkably easy.

In my example, I have a project called JustTesting, which contains just a test project with 3 unit tests (which will always pass, BTW).

image

I put those tests into an ordered test:

image

In Solution Items, I open up my JustTesting.vsmdi file, create a new test list (called Ordered Tests), and add my ordered test to it.

image

Once that’s done, I check everything into TFS (my Team Project’s name is “Sample CMMI”.

Next, I set up a build definition (in Team Explorer, right-click Builds, and select “New Build Definition”).  Set whatever options you want (name, trigger, workspace, build defaults) but stop at “Process”.

In the section named “2. Basic”, you’ll see that by default the Automated Tests field is set to (something like): “Run tests in assemblies matching **\*test*.dll using settings from $/Sample CMMI/JustTesting/Local.testsettings”. 

image

Click on the ellipsis on the right of that to open the Automated Tests dialog:

image

Remove the entry you see (or leave it if you wish to include that test definition), and then click “Add”.

In the Add/Edit Test dialog, select the optoin for “Test metadata file (.vsmdi)”.  Use the browse button to find and select your desired .vsmdi file.  In my example, JustTesting.vsmdi.

Uncheck “Run all tests in this VSMDI file”, then check the box next to your test list containing the ordered test.  In my example, the test list is called “Ordered Tests”.  Your dialog should look something like this:

image

Click OK and you’re Automated Tests dialog should look like:

image

Click OK again, then save your build definition.

Queue a new build using this definition.  Once complete, look at the build report to see your test results.

image

image

It’s a few steps, but nothing ridiculous.  And I didn’t have to hack any XML files or do any custom coding.

Hope this helps!

Comments

  • Anonymous
    September 27, 2010
    The comment has been removed

  • Anonymous
    March 08, 2011
    I don't see the Process Option you mentioned. Can you please let me know, where I can find that option. (I can see General, Trigger, Workspace, Build Defaults, Project File and Retention Policy Options)

  • Anonymous
    August 22, 2012
    The comment has been removed

  • Anonymous
    August 23, 2012
    The results of an ordered test are not viewed (nor is an ordered test run) via MTM.  MTM is designed for more manual testing and test case management.  Ordered tests are executed either in Visual Studio or via mstest.