Test Methods are neither Methods nor Tests
Put on your life vest & tankini. We’re heading off into the deep end again.
You’re doing TDD. You are happy with the results, enjoying the thrill of going fast.
As with most C# projects done test-first, you have some code that looks like this:
class C
{
public int F() { return 7; }
}
[TestFixture]
public class CTests
{
[Test]
public void Test1()
{
C c = new C();
Assert.AreEqual(c.F(), 7);
}
}
Find the unit test in this picture. If you answered “Test1()” you probably feel pretty smart.
But you’re wrong! Test1() is a method that contains a test. The actual test is this stuff:
C c = new C();
Assert.AreEqual(c.F(), 7);
The rest is just a convenient, language-acceptable package for the test. TestFixtures aren’t classes, they’re just a convenient way to package up a set of related tests.
Still don’t believe me? Try this one: Who calls Test1()? You’re probably thinking the answer is “NUnit”, but you’re hesitant to say that because I was so mean to you last time. Good.
The answer is: “no one”. Really. No one calls Test1(), it just runs.
Seem absurd? Try this instead: Who calls Main()? You’re thinking “The system”, but will answer “No one” because you think that’s what I want to hear.
Yep, no one calls Main(), it just runs. When you step in to your program in the debugger, you expect to see Main() at the bottom of the callstack; nothing else. Your program gets launched by the debugger, and Main() just runs.
It’s magic.
To repeat: Test methods are just convenient containers for tests. No one calls them, they just magically run.
If you don’t believe me, read again from the start. If you’re tired of re-reading, then pretend you believe me so you can continue.
So, what does it mean to mark a test method (or test fixture class) as ‘public’? It’s pointless. I don’t want anyone to ever instantiate my test fixtures or call my test methods.
Similarly, it’s pointless that my test methods have a parameter list of “()”. () means the parameter list is empty, but test methods have no parameter list. It’s the difference between “null” and “no value”.
And, finally, test methods have to be marked with the return type ‘void’, which is again pointless.
The only part that has value is the name. I definitely want to give my tests names.
It’s interesting to me that NUnit forces your test methods & test fixtures to be marked ‘public’. Do the Simplest Thing that Could Possibly Work says that the checks for ‘public’ should simply be removed. Further, the check that the return type must be ‘void’ should be removed. If there was a way to remove the check for the parameter list, I’d do that to. (NUnit would actually provide additional value by forcing test methods to be ‘private’, so no one accidentally calls one.)
Comments
- Anonymous
July 01, 2004
Doesn't "Main"/entry point method have to be public as well? - Anonymous
July 01, 2004
I'm not very familiar with the internals of NUnit, but I would have expected that NUnit would reflect on an assembly, find the classes/methods that are marked, and invoke them? If that is not the case, can you give some more insight on what kind of "magic" is going on? - Anonymous
July 01, 2004
Originally, NUnit 1.0 was based on JUnit. Java as you know doesn't have Attributes (just yet), so the idea there was that your test fixture would impement an interface ITestFixture and each public method that housed a test (or tests) followed a naming convention starting with "Test".
Thus JUnit (or NUnit 1.0) framework exe would load the package/assembly and reflect on all types that implement ITestFixture and call the public methods that started with Test.
With NUnit 2.0, since test methods are marked with a [Test] attribute, it shouldn't be necessary to have them public, since reflection could call private methods. However, NUnit 2.0 is backwards compatible in that you could implement ITestFixture (not sure if that's the exact name) and NUnit will run every public method in that class starting with "Test". - Anonymous
July 01, 2004
Interesting. I think I agree with you that tests are not methods. They do, however, call methods, right? How should NUnit distinguish methods that contain tests and those that are used by tests? By using something other than the public/private distinction? For example, that all methods used by tests should be outside the TestFixture? - Anonymous
July 01, 2004
If I follow you correctly, tests are orthogonal to code. Is test really an aspect of code?
Finally, do you think tests deserve to be a language feature?
public void MyMethod(Argument arg)
{
//do something
}
test MyMethodTest(MyMethod myMethod){
// test MyMethod
} - Anonymous
July 01, 2004
Chicken before the egg.
They had to start somewhere. What else could they do? They were not creating a language, but a tool to make the language more productive. Now if you want to extend C# to accomplish your goals, you are raising the bar to what only a select few are able to do.
But I love your reasoning, don't get me wrong there. The only thing I can say is setup/initialization may become a little more challenging under the proposed scheme.
Where do you put your tests that deal with validating coordination of parts? Do you artificially create a class in order to do this? - Anonymous
July 01, 2004
Julien: Main() does not have to be public. Neither does the class it lives in. In VS2002 / 2003, the wizard would create it as public, but it was unnecessary, and in my opinion wrong.
NUnit does reflect to find the tests methods to run; that's the magic I'm talking about. But I don't like to think "my test is called by reflection", just "my test runs". - Anonymous
July 01, 2004
nsp75: Yep, you got the history right. So, now that we have reflection & attributes, we can remove the 'public' restriction, right? - Anonymous
July 01, 2004
Dimitri: My tests are written in C# code, but I haven't decided if that's important or not.
I'm not proposing a new language feature; that would be alltogether too concrete for this post. We're talking about thinking, not about doing!
I find the 'public', 'void', and '()' parts annoying because they seem unnecessary. I only know how to get rid of the 'public' without changing the language, so that's as far as I'm going for now.
You get courage points for proposing something as radical as building TDD features into a language.
At a higher level, I do like to think of testability as a feature of my classes. "My object can create itself, describe itself, tell you when its attributes change, and verify that it is functioning correctly." For that reason, I think shipping unit tests is a good idea, but few agree with me. - Anonymous
July 01, 2004
David: Yes, tests typically call methods. NUnit already uses attributes to tag tests, whihc is fine. accessibility is not relevant. - Anonymous
July 02, 2004
Absolutely! There's no need for them to be public any longer. - Anonymous
July 05, 2004
Could the reasoning behind shipping unit tests be error reporting? Today, if an error occurs in the application, diagnostics pick up and offer to send information to the vendor. Would it be feasible to run unit tests at that time as well? - Anonymous
July 06, 2004
Dimitri: perhaps. In my experience, the tests that result from doing TDD are independent of most external dependencies. So, if they pass on the developer machine, they're pretty much guaranteed to pass on any other machine.
I have heard the NUnit maintainers say that they've had success with this approach, though. When they get support requests, the first thing they ask is for the user to run the unit tests.
I suppose that correct installation of NUnit, configuration to find a matching CLR, and CLR installation, all count as "external dependencies" that you would catch in your unit tests. - Anonymous
May 31, 2009
PingBack from http://outdoorceilingfansite.info/story.php?id=2421