Unit Testing is a Poor Example to Demonstrate a Complaint About Methodologies

code, tdd 0 comments suggest edit

Although I agree in spirit with most of Joel’s discussion of methodologies and rock star programmers, I’m in a bit of disagreement over the quote from Tamir he posts.

For instance, in software development, we like to have people unit-test their code. However, a good, experienced developer is about 100 times less likely to write bugs that will be uncovered during unit tests than a beginner. It is therefore practically useless for the former to write these…

I disagree with this, but in part because I think this is based on a flawed assumption over the purpose of unit testing. This point assumes that the only objective for unit tests is to uncover bugs. In reality, unit tests serve a much larger purpose.

1. Unit Tests promote better interfaces.\ Certainly, if you are truly a rock-star developer, writing a class that is extremely usable might come intuitively to you. But I think even rock stars can benefit from writing client code that uses a class the developer is building. This process helps to make sure that the interfaces to the class are thought out, and probably doesn’t take much more time than thinking through the class design before coding.

2. Unit Tests are a great form of documentation.\ When learning a new API, often the first (or second) thing I want to see is sample code that uses the API. Well written unit tests are a great source of documentation for how a particular class is meant to be used.

3. Unit Tests are great as regression tests\ So you’re a rock star who doesn’t write buggy code. Are you sure the person who is going to maintain your code is also a rock star? What about the person who wrote the class your code is dependent on? As Code Complete states, 80% of a project’s timeline is spent after the code is deployed in the maintenance phase. At some point, someone will come in and modify the code and perhaps change it in a subtle way that doesn’t appear to be a bug, but violates an assumption made by the initial programmer. There are a thousand ways in which a developer can write perfect code, but have it break either now or later. Well written unit tests can provide a high degree of confidence that bugs that are introduced later are discovered quickly. It’s no panacea, but it’s sure as heck a lot better than having none.

So while I agree that blindly following methodologies is a hindrance to truly talented developers, I do believe that there are some practices that are worthwhile across the board. In a team environment, communication is of the utmost importance. Most developers don’t work in a vacuum and a writing unit tests is one of those practices that really help communication within a project and beyond. It helps communications with current team members as well as future team members to come.

So yes, being a monkey in a methodology is bad, but I think there are better illustrations of this than Unit Testing.

UPDATE: I’m not the only one who believes this. Seems Roy has a similar opinion to mine. As does Jason Kemp in my comments.

Technorati Tags: TDD

Found a typo or error? Suggest an edit! If accepted, your contribution is listed automatically here.

Comments

avatar

7 responses

  1. Avatar for Jason Kemp
    Jason Kemp December 6th, 2004

    Amen, brother.

    I agreed with most of Joel's post up until that point myself. Joel even alluded to Eric Sink's article describing the difference between programmers and developers. He should have stopped there. The developers that Eric describes more closely coincide with the "rosh gadol" type that Joel wants on his team, IMO. The rosh gadol type sees what needs to be done, and does it, while still fitting in with the team.

  2. Avatar for Wayne Allen
    Wayne Allen December 22nd, 2004

    Unit tests are micro design + communication. See my post from March 2004

    http://weblogs.asp.net/wallen/archive/2004/03/22/94112.aspx

  3. Avatar for Tamir Nitzan
    Tamir Nitzan January 11th, 2005

    Little did I know an innocent letter to a co-employee about Joel Spolsky's post would stir up this much response....



    One cannot examine means without examining ends.



    In my book, the goal of commercial software developers is to produce the best software by the most efficient means possible.



    Inherently, that's a maximization problem with multiple parameters. "Best" means multiple things (fastest, most user friendly, least bugs, quickest out the door, etc.). So does "efficient". That means every development organization (and often, every project) needs to find its own balance point.



    For some, this balance includes unit tests. For some it includes other forms of automated tests (such as functional, scenario-based testing). For some it includes an SRS, and for some it doesn't.



    No such balance is "bad". The only "bad" thing is thinking that you found "THE winning formula" which can be applied in every situation, to any developer, and yield the best results.



    For the record, I agree 100% that there are situations that call for unit tests.



    However, consider a highly skilled developer that costs $100 / hour. Let's say it'll take 10,000 hours to code a project, which makes it a $1 million effort. Let's assume the cost of bugs given the skillset working on this is going to be $250k. Further assume that writing unit tests for this application will add 25% to the time spent coding it, and reduce the cost of bugs overall 10%. That means this developer will spend $250k writing unit tests, which will save $25k on bug cost. Not a very cost efficient model. Instead, say the same developer spends $100k instrumenting this system for scenario-based end to end tests. These have higher coverage than unit tests, and also check for algorithmic correctness.. so they might save $150k in cost of bugs. For this specific model, unit tests would be a waste of time, whereas the other (functional) tests are not.

    On the other hand, let's consider a NASA developer writing code for the next Jupiter rover vehicle. He might spend $1 million writing code, but any bug in this might cost $1 billion. Under the circumstances, he would be ill advised not to use unit tests even if they increase bug saftey just by 1%.



    Lastly to address the points you bring up:

    1. "Certainly, if you are truly a rock-star developer, writing a class that is extremely usable might come intuitively to you. But I think even rock stars can benefit from writing client code that uses a class the developer is building. This process helps to make sure that the interfaces to the class are thought out, and probably doesn't take much more time than thinking through the class design before coding."



    True. But unit tests are not the only way to accomplish that. In a good size system, there will be other components inside the system that are "clients" for the component you're currently writing. Using those to test can save you time.



    2. "When learning a new API, often the first (or second) thing I want to see is sample code that uses the API. Well written unit tests are a great source of documentation for how a particular class is meant to be used."



    It may well be more cost effective for you to spend more time studying the system than it takes someone else to write it. In addition, a well written system needs little (if any) internal documentation, and that would mostly be around the pure algorithmic sections (which unit tests don't usually address very well). Either way, forcing the entire world to write tests because that's your preferred method of study makes about as much sense as tossing lectures out the window because you prefer books. Personally, I like object model diagrams (I don't really, but this is an example).. will you generate one for each class for me?



    3. "So you're a rock star who doesn't write buggy code. Are you sure the person who is going to maintain your code is also a rock star? What about the person who wrote the class your code is dependent on? As Code Complete states, 80% of a project's timeline is spent after the code is deployed in the maintenance phase. At some point, someone will come in and modify the code and perhaps change it in a subtle way that doesn't appear to be a bug, but violates an assumption made by the initial programmer. There are a thousand ways in which a developer can write perfect code, but have it break either now or later. Well written unit tests will help make sure that bugs that are introduced later are discovered early. It's no panacea, but it's sure as heck a lot better than having none."



    If the developer following me doesn't know how to properly test and validate his stuff, then he should write unit tests for the existing code before modifying it :)

    Seriously though, unit tests are just one way of checking your system, and they are often limited by the boundaries of what they intend to test (sort of by definition).



    My goal is to get the best software out the door in the most efficient way possible. I find that flexibility and common sense are the most important tools that let me do that. Sometimes I find Unit Tests helpful (to be honest, not very often though), and sometimes I find them a waste of time. I don't see myself as an "expert developer"; but I value my time (and the company's money) and want to spend it where it gets the best bang.



    The whole premise of Spolsky's original posting was that following something blindly (he picked MSF, but of course the same is true for XP, Scrum, RUP, whatever) is inherently flawed. I find it rather amusing that a posting like that would start a religious war about why Unit Testing is superior to any other. Do whatever you must to produce the best software efficiently. And if you know how to do that well, UT or not, I will look forward to being on a team with you one day.

  4. Avatar for 尖锐湿疣
    尖锐湿疣 April 7th, 2005

    Good!

  5. Avatar for Bruno
    Bruno September 18th, 2006
  6. Avatar for you've been HAACKED
    you've been HAACKED November 13th, 2007

    Writing Testable Code Is About Managing Complexity

  7. Avatar for Community Blogs
    Community Blogs November 15th, 2007

    Frans Bouma wrote an interesting response to my last post, Writing Testable Code Is About Managing Complexity