Preventing check-ins to TFS that contain code analysis warnings

Recently we have received several questions regarding Visual Studio Code Analysis integration with Team Foundation Server’s check-in policy and build server, so I thought it would be helpful to clarify the behavior and expose some relatively hidden functionality.

Source Control Check-in Policy

The first point of confusion that often arises is about the source control check-in policy.  A common question is “I have a check-in policy, so why does it let me check-in code that contains code analysis warnings?”, “I want a code analysis check-in policy, but I want to hold different projects (e.g. test and product code) to different standards, can I configure different check-in policies for different projects?”, or “How can I treat warnings as errors so I break the build on code analysis warnings?”

The important thing to note about the code analysis check-in policy is the rule set selected for the check-in policy sets the baseline for the minimum set of rules that must be run on the project before the code can be checked in.  All this does is enforce that the rules were run on the project before a check-in, it does not in fact enforce that the code checked-in be code analysis clean for the rule set that is specified in the check-in policy, nor does it actually require that exact rule set be run, as a superset of those rules can be run as long as each rule in the check-in policy rule set was run, the check-in will not be rejected.

The way to prevent checking in code with code analysis warnings is to modify the action for the rules from the default of “Warning” to “Error” for the rule set on the TFS server, setting the build definition trigger to be a “Gated Check-in”, and enabling Code Analysis as part of the build.  NOTE: taking this approach will also result in your builds breaking on code analysis warnings since you are treating them as errors.

Rule Sets

The first thing to do is pick the rule set(s) you want run for each project.  You can either use the built in rule sets, or create your own custom.   You will need to create at least one custom rule set if you want to use the same set of rules, but block check-ins for some projects, and with the same set of rules not block check-ins for other projects.  The easiest way to create a custom rule set is to start from an existing rule set:

• Open any project’s properties page
• Select the Code Analysis tab
• Choose the rule set you would like to make a copy of from the rule set drop down
• Click “Open”. 
  
• You will now be presented with the rule set editor, from the rule set editor change the action from “Warning” to “Error”
  
• When you save the modified rule set you will be presented with the “Save As…” dialogue since the built in rule sets are by default read only

You will now need to either place your new custom rule set on a share and set the project to use that rule set by selecting “<Browse…>” from the rule set dropdown, or you can drop the new custom rule set into the rule set directory (“%VS_INSTALL_DIRECTORY%\Team Tools\Static Analysis Tools\Rule Sets”) on each developer’s machine and the build server machine, and it will automatically appear as an option in the rule set dropdown.

Configuring the Build Definition

Next we need to configure the build definition:

1. Set the “Trigger” to “Gated Check-in”
   
2. Under “Process”, expand “Basic” and set “Perform Code Analysis” to “Always”
    

That’s it, now check-ins with code analysis warnings will be blocked for projects using the new custom rule set(s).

Comments

• Anonymous
  April 10, 2011
  I usually just toss my ruleset file into TFS; usually putting it into Solution Items so when a dev does a get latest on the solution they get any updates to the ruleset.

• Anonymous
  May 17, 2011
  Also check out Martin's Custom Check-in Policy Pack, which is part of the Team Foundation Server Power Tools.  It includes a code analysis policy that does not involve the build server.