Quality dashboard (Agile and CMMI)

TFS 2017 | TFS 2015 | TFS 2013

You can use the Quality dashboard to obtain an overview of progress occurring in the test, development, and build areas as they relate to the quality of the software under development. The team can use the Quality dashboard to learn and make decisions that support team goals around product quality.

By using this dashboard, you can review test progress, build states, progress in resolving and closing bugs, rate of bug reactivations, the percentage of code that has been tested, and the trends in code changes. Each of these metrics is plotted for the most recent four weeks.

You can use this dashboard to answer the following questions:

- Is the test effort on track?
- Is the team testing the appropriate functionality?
- Are the team's bug fixes of high quality?
- Are tests stale?
- Does the team have sufficient tests?
- Are any bottlenecks occurring?

Requirements

Same requirements defined in Project portal dashboards.

Data displayed in the dashboard

Team members can use the Quality dashboard to determine the overall quality of the product that they are developing. In an ideal case, test pass rates, bugs, and code churn all show the same picture, but often they do not. When you find a discrepancy, you must examine more closely the appropriate build and data series. The Quality dashboard combines the test results, code coverage from testing, code churn, and bugs, to help you understand many perspectives at the same time.

To learn about the Web Parts that are displayed on the Quality dashboard, refer to the illustration and the table that follow.

To learn more about how to interpret, update, or customize the charts that appear in the Quality dashboard, see the topics in the following table.

Web Part	Data displayed	Related topic
	Stacked area graph of the test results of all Test cases grouped by their most recent recorded outcome - Never Run, Blocked, Failed, or Passed - during the most recent four weeks.	Test Plan Progress
	Stacked columns that show how many builds Failed or Succeeded during the most recent four weeks.	Build Status
	A stacked area graph of the cumulative count of all Bugs, grouped by state, during the most recent four weeks.	Bug Progress
	A stacked area graph of how many bugs the team has reactivated from the resolved or closed state during the most recent four weeks.	Bug Reactivations
	Line chart that shows the percentage of code tested by build verification tests (BVT) and other tests during the most recent four weeks.	Code Coverage
	Stacked area graph that shows how many lines of code the team added, removed, and changed in the check-ins before the build during the most recent four weeks.	Code Churn
	List of upcoming events. This list is derived from a SharePoint Web Part.	Not applicable
	Count of active, resolved, and closed work items. You can open the list of work items by choosing each number. This list is derived from a Team Web Access Web Part.	Not applicable
	List of recent builds and their status. You can view more details by choosing a specific build. This list is derived from a Team Web Access Web Part. Legend: : Build not started : Build in progress : Build succeeded : Build failed : Build stopped : Build partially Succeeded	Run, monitor, and manage
	List of the most recent check-ins. You can view more details by choosing a specific check-in. This list is derived from a Team Web Access Web Part.	Manage pending changes

Required activities for monitoring quality

For the Quality Dashboard to be useful and accurate, the team must perform the activities that this section describes.

Required activities for tracking test plan progress

For the Test Plan Progress report to be useful and accurate, the team must perform the following activities:

• Define test cases and user stories, and create Tested By links between test cases and user stories.

• Define test plans, and assign test cases to test plans.

• For manual tests, mark the results of each validation step in the test case as passed or failed.

  Important

  Testers must mark a test step with a status if it is a validation test step. The overall result for a test case reflects the status of all the test steps that the tester marked. Therefore, the test case will have a status of failed if the tester marked any test step as failed or did not mark it.

  For automated tests, each test case is automatically marked as passed or failed.

• (Optional) To support filtering, assign Iteration and Area paths to each test case.

  Note

  For information about how to define area and iteration paths, see Define area paths or Define iteration paths .

Required activities for tracking bug progress and bug reactivations

For the Bug Progress and Bug Reactivations reports to be useful and accurate, the team must perform the following activities:

• Define Bugs.

• Update the State of each Bug as the team fixes, verifies, closes, or reactivates it.

• (Optional) Specify the Iteration and Area paths of each Bug if you want to filter by those fields.

Required activities for tracking build status, code coverage, and code churn

For the Build Status, Code Coverage, and Code Churn reports to be useful and accurate, team members must perform the following activities:

• Configure a build system. To use Team Foundation Build, you must set up a build system.

  For more information, see Build and Release agents.

• Create build pipelines. You can create several build pipelines and then run each of them to produce code for a different platform. Also, you can run each build for a different configuration.

  For more information, see Get started with CI/CD.

• Define tests to run automatically as part of the build. As part of the build pipeline, you can define tests to run as part of the build or to fail if the tests fail.

  For more information, see Set up continuous testing for your builds.

• Configure tests to gather code coverage data. For code coverage data to appear in the report, team members must instrument tests to gather that data.

  For more information, see Run tests in your build process.

• Run builds regularly. You can run builds at regular intervals or after every check-in. You can create regular builds when you use the schedule trigger.

  For more information, see Build triggers.

  Note

  Although a team member can manually rate a build by using Build Explorer, this rating is not reflected in the Build Quality Indicators report. The build rating appears in the Build Summary report. For more information, see Rate the quality of a completed build and Build Summary.

Troubleshoot quality issues

The following table describes specific quality issues that the Quality dashboard can help you monitor and identify actions the team can take.

Related notes

Project portal dashboards