Do Tests Get Too Much Respect?

respect joanneI’ve recently read Phil Haack’s post about “structuring unit tests”. And then Ayende’s response. I’ve also went over the comments on both posts, and clearly, most were in favor of having some kind of method to structure or organize test code.

Who’s right?

I used to work with someone who was very VERY organized. His Outlook had dozens of folders and sub-folders. He know exactly where to file each email, and consequently, where to find each email. If I were looking for something and couldn’t find it,  I’d turn to him, and he would find it rather quickly because everything was filed according to a “system”.

Mind you, mostly I find email. I keep all my past emails in one folder called “backlog” and use the search box.

Who’s system is more organized? My friend’s, obviously.

Which one is more effective? Now that’s a different story. The search box finds things much quicker. There’s also a penalty in the filing system. For each email, my friend needs to think where it belongs. What if it deserves to be in two places? The filing itself is costing him time.

See the resemblance?

Reading all the comments that favor the structuring method, or any method, and stating its not much work and effective, made me think: Have these people lost their mind? Have they lost sight of what’s really important?

Tests have a primary goal: to tell us we’ve broken something. Once I write a test, I hope and pray I’ll never ever see it again. Because if I do, something’s broken and I’ll need to fix it. Otherwise, this test vanishes between the other hundreds or thousands of tests. Which is fine by me. If the test fails, my test runner will get me there in two clicks.

It seems that along the way, we’ve placed too much respect upon our tests. No wonder, since we’ve crowned TDD as a “design method”. Obviously tests are not mere code, they are a design tool. and therefore should be treated even better than production code.

What’s important is working production code.

Tests are the most effective tools to get to working code. But let’s put things in perspective: they are just that – tools.

And the upcoming Typemock Isolator V7  is all about a new perspective.

Stay tuned.

  • Eric K.

    I think tests can have a few other uses, too, including:

    – A reference of the verified functionality of the system (documentation?)

    – A reference for new developers, or even developers who need to revisit code they haven’t seen in a while

    In both of those cases, a well thought-out test structure will be a great benefit over a big grab-bag-o-tests.

    As for “Once I write a test, I hope and pray I’ll never ever see it again”, I don’t work that way… I look at the tests frequently.

    • Eric,

      – I don’t like the documentation metaphor so much. That’s because I’ll call it documentation if I can read it, and the tests are short enough so I can understand for the exact scenario, with the exact scenario and result how it’s used. Most of the time, neither test names or bodies are not that descriptive.

      – See former point.

      If tests are not readable, you won’t care about their organization. If they are, their organization will at best point to the stream of thoughts the writer was experiencing at the time.

      And, I’ll bite: Why do you look at tests frequently?

      • Hi Gil. Thanks for the response. 

        I’ll concede that badly named, badly thought out tests don’t make for good documentation. That’s self evident. I’d add that they don’t make for good tests, either. But, of course, that’s another discussion, and that’s not what I was talking about…

        Say you have tests you can read, with useful, descriptive names, tests short enough that you can understand the exact scenario and the result … I would submit that that makes for very good documentation. (Functional documentation, that is. There are plenty of reasons for other types of documentation, too.) Any other kind of test may not be worth the time it takes to write it. My comment assumed such a decent set of tests.

        *edit* – I seem to have two Disqus accounts… I am the same Eric K. that you were replying to. 🙂

        Back to the point of your post, though… Assuming that the tests are worth looking at  (see above), organizing them in a decent structure will always be better than a giant directory-of-tests for precisely the reasons I gave previously. There is nothing better at describing what the code will do under a certain set of scenarios than tests that exercise those scenarios.

        (If, however, the tests are as incomprehensible as you are suggesting, then sure, I wouldn’t want to look at them again either.)

        I look at the tests frequently for exactly that reason. I need to make some changes; What scenarios have we already considered for this piece of code? What boundaries have been established and proven? What special scenarios have already had tests written around them? A quick glance at a well organized set of tests can give me that immediate insight in ways that nothing else can. This is especially important if I have never seen that code before, or if it’s simply been a while.

        In short, I can’t imagine *not* wanting to look at the tests.

  • I like the comparison to email filing, and I think your point is well made along those lines. But, I do think there’s another situation to consider:

    When I write tests, I tend to forget about them too and only revisit them when they break. However, if I’m maintaining someone else’s code, his tests give me a window into the intended usage scenarios of his class. So, any expressiveness of those tests (descriptive names, hierarchical organization, comments or notes about temporal dependencies, etc) save me time. I view such organization less as a favor to myself and a help for my own process and more as a favor for prospective maintenance programmers (which may include me if I have to revisit something many moons later and need to clear out the cobwebs a bit).

    • Erik,

      It may be that for you, the organization helps. For others – not so much. I can tell you that I’m more likely to look at test names, rather than which tests are their neighbors. 

      I really like the “help others” MO and the best way to do that is get someone to review your tests. This someone may comment on organization, or about something else. The sad truth is, there are always more important review comments than organizational ones.

      In any event – do what works for you, but keep in mind the cost that it carries.