Bug Driven Development

code, tdd 0 comments suggest edit

It is a sad fact of life that, in this day and age, arguments are not won with sound logic and reasoning. Instead, applying the principle of framing an argument is much more effective at swaying public opinion.

1364145387_b8cf994488 So the next time you try to make headway introducing Test Driven Development (or even simply introducing writing automated unit tests at all) into an organization and are rebuffed with…

Don’t bring your fancy schmancy flavor of the week agile manifesto infested “methodology” here kiddo. I’ve been writing software my way for a loooong time…

You can reply with…

I’m sorry, but I’m not a fan of Bug Driven Development. I think Test Driven Development is not without its challenges, but it’s a better alternative. Either you’re with us, or against us. Are you a bug lover? Bug Driven Development gives comfort to the bugs.

UPDATE: this is an example of my dry humor. I don’t believe that “Framing” is a good way to win an argument and I would never actually say or recommend saying anything similar to to this. It’s meant as a bit of a joke, but with a point.

A team that is not focused on automated testing of some sort throughout the lifecycle of the project is effectively embracing Bug Driven Development. Bugs are going to drive the development cycle at the end of the project.

Don’t believe me though, look at the research done by others. In Facts and Fallacies of Software Engineering, Robert Glass points out…

Fact 31. Error removal is the most time-consuming phase of the life cycle.

In Rapid Development, Steve McConnell relates…

Shortcutting 1 day of QA activity early in the project is likely to cost you from 3 to 10 days of activity downstream.

In other words, if you don’t control the bugs, the bugs control your schedule.

Found a typo or error? Suggest an edit! If accepted, your contribution is listed automatically here.



36 responses

  1. Avatar for Steven Harman
    Steven Harman September 23rd, 2007

    But the BDD acronym is already taken - Behavior Driven Development. And to be honest I'm still 100% clear on exactly what BDD is as it seems to be a summation of TDD, Stories, DDD, etc...

  2. Avatar for DotNetKicks.com
    DotNetKicks.com September 23rd, 2007

    You've been kicked (a good thing) - Trackback from DotNetKicks.com

  3. Avatar for Haacked
    Haacked September 23rd, 2007

    @Steve I don't care that BDD is taken. My BDD is better. ;)
    Actually, I think mine should be acronymed like this...
    BUg Driven Development or BUDD for short.
    As in, you must be smoking bud to think BUDD is a good way to develop software.

  4. Avatar for she
    she September 23rd, 2007

    The thing is, "bug driven development" is the most simple and straightforward one. Its a lot associated with experience too, the more experienced one gets, the less bugs he does (but it also depends on the language, and whether it supports certain coding schemes or not)
    So i can understand everyone who is sceptical, no matter how many OTHERS are persuaded that their model is better. :)
    I think however there should be some kind of contest. A fair contest, where we would measure different techniques with a somewhat similar "reallife usage case" (and it should be a quite complex problem too and the programmers involved should have some years experience too)

  5. Avatar for Josh Stodola
    Josh Stodola September 23rd, 2007

    Great post, Phil! A definite reminder.

    As in, you must be smoking bud to think BUDD is a good way to develop software.


  6. Avatar for Haacked
    Haacked September 23rd, 2007

    @she Well the competition would have to last at least two years. Maintenance typically consumes 40 to 80 percent of software costs.
    Assuming a software system lifecycle of two years here's a reasonable competition that would mimic real life.
    1. Two teams are divided into two groups each. Group 1 has 3 months to build an app. Group 2 doesn't get to watch.
    2. Teams get requirements from a person acting as a "client".
    3. Every week, the client throws in a requirements change.
    4. At the end of the project, Group 1 leaves and Group 2 comes in without any prior knowledge.
    5. Now for the next year and nine months, group 2 must maintain the app.
    6. Every week, the client will report changes to the app that the groups must make.
    7. All the while, the groups must fix any bugs reported by the QA team.
    I think this would be a relatively fair competition to determine the efficacy of TDD vs BuDD.

  7. Avatar for Howard
    Howard September 23rd, 2007

    The kind of "framing the argument" that you have in mind is what people sometimes do when they lack real arguments for their point. For example, "Death Tax" "Right to Life," and “Woman’s Right to Choose.”
    Test driven development is great in some contexts. For example, it's perfect for the ongoing development of a database engine that exposes a well-defined API. At the other extreme is what I have been involved with recently, namely incremental development, by a small team, of a database-driven workflow automation application -- the only place TDD has paid off for this has been in developing various helper functions. Writing a complete suite of autotmated tests would probably double our effort.
    Even if TDD eliminates 75% of the bugs – and how likely is that – the situation is qualitatively the same. You still need to expose the product to testers and field users – “Bug-Driven Development.”

  8. Avatar for Howard
    Howard September 23rd, 2007

    The “fair competition” should include the extra time that Group 2 took in the construction phase. The fact that TDD hasn’t been widely adopted proves to me that TDD adds substantially to the time to construct software.
    Also we should include the time for Group 2 to get up to speed on the tests and maintain them. This would include the work to update the tests that are continually getting broken because of the weekly client change requests. If you are smart enough to design your tests so they are well factored and tend not to get broken, then you are smart enough to design you application itself so that its parts are equally well factored. Then most bugs will appear in new functions that are being beta tested, and not as regression errors.

  9. Avatar for Ricky Clarkson
    Ricky Clarkson September 24th, 2007

    At best, TDD is Bug Driven Development, where the bug doesn't live for very long.
    I can equally say to you that I don't believe in your Test Driven Development, and that I now mandate Proof Driven Development. Here is a pile of maths books.

  10. Avatar for Haacked
    Haacked September 24th, 2007

    @Howard, well making sure the code is well factored also takes time. And no matter what, I guarantee you that when you need to make a small change to a codebase, it's a *lot* faster to be able to make that change when you have a decent suite of automated regression tests. Even in "well factored" code.
    How many times have you approached a codebase that was so well factored that you could make a change and have a high degree of confidence you didn't affect other code bases. Unit tests are no panacea, but they do provide confidence.

  11. Avatar for The Other Steve
    The Other Steve September 24th, 2007
    I’m sorry, but I’m not a fan of Bug Driven Development. I think Test Driven Development is not without its challenges, but it’s a better alternative. Either you’re with us, or against us. Are you a bug lover? Bug Driven Development gives comfort to the bugs.

    Not a great way to win over friends.
    Most people I know are not opposed to writing tests. What is lacking, is a good explanation as to how. Especially since most development done today is on an existing code base. I think evangelizing TDD could better be served by helping to show how easy it might be in the long run and helping people get past that initial learning curve.
    The app I've been working on the past year at work is a gigantic collosal mess which was developed using the Big Ball of Mud Methodology. There's no way to unit test this mass... I have a hard enough time just stepping through the code. Which leads me to another observation. It appears that TDD only works well with certain application architectures. I've recently been working with the MVP pattern you talked about earlier in a side project at home. Once I got that working, things started going together much easier. It was hard to get the first feature working, but as I added new functionality it was a breeze to put in place. But that's a way of development that in my experience, most people don't know or do.
    Again, people aren't opposed. It's just hard to see how you get from big ball of mud to something good.
    And make it idiot proof enough that even the idiot on your team can understand. (Remember that 80% of developers in the real world are nowhere near as bright as the other 20%)

  12. Avatar for Haacked
    Haacked September 24th, 2007
    The fact that TDD hasn’t been widely adopted proves to me that TDD adds substantially to the time to construct software.

    How is that a proof? Depends on what you mean by widely adopted. Is it over 50%? Perhaps not. Is it a significant number? Definitely. Not too hard to find a dev who gets TDD.
    And I did point out in my post, one would gain plenty of benefits by just starting to introduce automated unit tests and not completely adhering to the TDD dogma.
    However, having said that, adoption rate is not a proof that TDD adds a significant time or not. There are plenty of experienced TDD devs out there who can develop using TDD not much slower than an equivalent experienced developer can without TDD.
    I remember working on one particular VB app written to parse Fortran that was incredibly challenging to change without breaking. I had to write tests because every change would introduce some unforeseen bug that I would then spend hours tracking down and then fixing, not to mention the hours in which I didn't even know a bug had been introduced by the fix for another bug. Test coverage would've kept the overall time down.
    Both teams get 3 months, but both teams have to live with the bugs they introduced in that time.

  13. Avatar for engtech
    engtech September 24th, 2007

    This is our workflow in a nutshell.

  14. Avatar for Haacked
    Haacked September 24th, 2007
    Not a great way to win over friends.

    @TheOtherSteve - Well I was joking. I would never actually say that.

  15. Avatar for Haacked
    Haacked September 24th, 2007
    What is lacking, is a good explanation as to how. Especially since most development done today is on an existing code base. I think evangelizing TDD could better be served by helping to show how easy it might be in the long run and helping people get past that initial learning curve.

    I've heard good things about the book, Working With Legacy Code. I haven't read it myself yet, but I've personally had a lot of success adding unit tests after the fact to code and applying structured refactorings.
    Case in point, when I started with Subtext (taking the existing .TEXT code) there were 0 unit tests. Now there are 785 and 36% code coverage (and counting!).
    Now granted, .TEXT was no ball of mud, but testing after the fact can be done.
    I've worked with balls of mud in the past and the approach I took was iterative. Anytime I had to fix a bug, I would try to write a test that fails because of the bug, then fix the bug and make sure that the test passes because of the fix.
    Many times, I could find small chunks of code I could factor out into its own method. Over time, a ball of mud can get refactored into something nicer and more testable. But it takes discipline and patience.

  16. Avatar for Haacked
    Haacked September 24th, 2007
    Even if TDD eliminates 75% of the bugs – and how likely is that – the situation is qualitatively the same. You still need to expose the product to testers and field users – “Bug-Driven Development.”

    I'm definitely not arguing against that point. As I've said before.

    Automated unit tests are NOT a replacement for system testing, beta testing, integration testing, nor any other kind of testing. Unit tests are only one small part of the testing equation, but they are an important part, just as all the other types of testing are important.

    The point is not that you're going to be able to eradicate all bugs. Rather, it's using all tools at your disposal (in their proper contexts) to gain the upper hand against having bugs run your schedule.
    Sure, TDD may not work so well in all contexts, such as developing a workflow site or a sharepoint site perhaps. But in the contexts where it does work well, it works well.
    For web apps, I personally recommend a combination of unit testing and using something like WATIN for web testing tricky parts of the web UI. I think it's worth the test maintenance cost to be able to run a QA cycle in half the time or less because much of it is automated.

  17. Avatar for Mr. Teflon
    Mr. Teflon September 24th, 2007

    I don't need no stinkin' TDD, I line my software up with Teflon, bugs just bounce off! :-)
    I think that the answer is always it depends. It depends on the team that you have, quality of the developers.
    For example, I have been working on a very complex library with over 200,000 lines of code (written from scratch) for last 5 years with changes and additions made almost every single day. I do not have many tests for that library, yet it is of very high quality with over 10,000 deployments.
    There are bugs of course but there just aren't that many given the size of the project. Looking at my accurate release notes over 5 years, there were more features added than bugs found.
    So what is the secret here? First, I am one and the only one working on that code. Second, I do not under any conditions commit the code that I have not executed, lot of times line by line in debugger, to make sure it works. I can't stress how important this last step actually is.
    It works great for me.
    However, once you get into the teams, not everyone can be this disciplined. Actually most people aren't. I've seen people check in branches of code that I know they did not execute. I have seen people check in the code that even breaks the builds... That is where TDD and continuous builds help.
    Actually thinking about it, if you have more than 2 people working on the code base you should be well served with having at least some tests as part of your code base.

  18. Avatar for Christopher Steen
    Christopher Steen September 24th, 2007

    Link Listing - September 24, 2007

  19. Avatar for Chris Moorhouse
    Chris Moorhouse September 24th, 2007
    Even if TDD eliminates 75% of the bugs – and how likely is that – the situation is qualitatively the same. You still need to expose the product to testers and field users – “Bug-Driven Development.”

    So, when the user types in X, Y information should be displayed. Oh, heck, did I just write a test? Geez, I thought we weren't going to use that useless Test Driven stuff on this project.

  20. Avatar for craig cameron
    craig cameron September 24th, 2007

    In relation to mr teflon, working as a single developer is much much easier. I once worked on a project with a few junior developers and the bugs spiraled out of control. I then kicked them off the project and we went to only one developer and the bugs started to disappear. While this is not always an option if you can create functional silos in the code and then assign responsibility to individuals you will tend to be better off.

  21. Avatar for Frans Bouma
    Frans Bouma September 24th, 2007

    I'm all for weeding out bugs early on as much as possible, and unit tests can help. However, the focus should be on weeding out bugs, not on writing tests. The thing is: writing tests is OK, but will never be enough to squash most bugs, simply because some software will simply require an unlimited amount of tests to be sure all bugs are gone.
    What helps more is proof that the software written is correct. This is tedious work as well, but can also be done earlier on. So added to the tests which can show that the software changes and new feature(s) are indeed working in the context of the tests (because that's what you're testing), you have to have proof that the code is correct. Proving that code is correct isn't that hard, it just takes time. But lacking proof and thinking the tests are the proof is a mistake: the tests only proof what's tested in the tests. If you think the tests combined define 100% of the cases the software will be used in, you're mistaken as well. A simple multi-db oriented o/r mapper API alone has an unlimited amount of corner cases where it can go wrong, so you need proof that a routine is correct.

  22. Avatar for Andy Stopford
    Andy Stopford September 24th, 2007

    Frans, some good points and what you want to avoid is writing masses and masses of tests (which would be avoided by most manageemnt) or writing too little tests. Two things help here, frameworks that reduce this and do the work for you (MbUnits row and pairwise or XUnits theory attributes for example) coupled with a frameworks test input factories and random drivers.
    Prehhaps this is where TDD had a big distiniction over just writing tests. TDD uses tests to map to the user stories so your tests will model the apps functionality so you are pinpointing areas that are suppose to work the way you require.
    Tests coupled with requirements are the best deal and with tests you can adjust your tests to meet the need functionality and refactor with no fear of introducing regression bugs. Refactoring with no tests is like swimming with sharks with your leg hanging off.

  23. Avatar for Derik Whittaker
    Derik Whittaker September 24th, 2007

    Today I came across a nice post on 'Bug Driven Development' . This is something that most developers

  24. Avatar for Stephan
    Stephan September 24th, 2007

    How about DeDD - for Defect Driven Development?

  25. Avatar for Kevin Dan
    Kevin Dan September 25th, 2007

    Bug Driven Development == Code and Fix, an approach should be replaced by Test Driven Development, that is, Test always comes first.
    Legacy code == code without testing counterpart

  26. Avatar for Karthik Hariharan
    Karthik Hariharan September 26th, 2007

    Couldn't agree more. But there's another equally frustrating response you get when trying to implement TDD.
    "Yeah thats a great idea, we really should start doing that..."
    3 months later...nothing has changed.

  27. Avatar for Michael White
    Michael White September 27th, 2007

    Granted, this definitely does NOT solve the problems involved in working with legacy code - but building at solid framework WITH tests, and basing your application code on that framework will work wonders. By tests I do not simply mean test methods that are intended to catch bugs but rather test in a QA fashion. Build mini-apps that take care of feature A, B, C etc. The features these mini-apps are tests for are the features that are contained directly in the framework base code. Anything outside of that is project based testing. Frameworks take various time scales to develop and test thoroughly depending on size - but once you have your framework running smoothly, you will find your bug count drop considerably, and if you have a framework of really useful code then application development time will drop considerably as well. This reduces overall time tremendously in most cases. I have done a few projects now on a framework I designed and it has cut development by 2/3 and testing finds and extremely low number of bugs that are usually very easy to find and fix.
    Final point: There is no single "best" way to do anything. Often it is project based and you just have to hand pick the methodology and systems you (and your team) work best with.

  28. Avatar for Alex
    Alex September 27th, 2007

    With all of this talk of eliminating bugs, in my opinion, the best way to do this is to pair program and have mandatory code reviews. This way, younger developers are continually in check and they learn coding styles from more seasoned devs. Code reviews should be mandatory regardless. If you have trouble getting the company to buy off with TDD, then good luck with XP. I cannot stand when businesses try and cut corners with software.

  29. Avatar for Tod McKenna
    Tod McKenna September 29th, 2007

    Code reviews are indeed a good practice. There are those who don't see the worth (bad experience, perhaps?), but I've seen firsthand how code reviews + test driven development + instrumentation can make a gigantic impact on the quality of the released software.
    Of course, if you're a lone star, or code in your underwear, then maybe finding a good code reviewer isn't an option!
    Anyway, thanks Mr. Haack for yet another great read...

  30. Avatar for arakiel
    arakiel October 2nd, 2007

    Well lookie here, it's the latest version of
    "X is better then Y because I said so"
    And once again, nothing to back up WHY X is so much better then Y, we just have to take the dogma preachers word for it.
    If your not going to bother explaining why your dogma is better then Y then your no better then those who dismiss it.

  31. Avatar for Brian
    Brian October 4th, 2007

    In the restaurant industry there is a saying among waiters. "Run your table or they will run you."
    I'm not surprised that our industry cannot use logic. We write it all the time, but cant use it. Maybe thats why our projects fail a lot.

  32. Avatar for Greg D
    Greg D October 21st, 2007

    The dysfunctional flavor of bug-driven development that I find most maddening is the attitude that a devteam won't fix known bugs until a [em]customer[/em] finds them first. So yeah, I know of at least a dozen different crash errors in the rotting codebase that I just inherited, but I'm not allowed to go in and fix 'em because none of our customers have reported them yet.
    Of course, I suppose I shouldn't be too surprised when we consider killing and restarting the software to be a "quick and easy" workaround for most of the bugs in this thing.
    Is anybody out there hiring software devs who care about the quality of their software?

  33. Avatar for Eric
    Eric October 25th, 2007

    I might have to add this to my list of Little Known Development Drivers: http://coderoshi.com/2007/0...

  34. Avatar for Rajesh Kumar N.Ch
    Rajesh Kumar N.Ch August 16th, 2011

    To start follow TDD to get into production implement BDD.
    To implement BDD follow TDD.

  35. Avatar for haacked
    haacked March 18th, 2012
  36. Avatar for Calmarius
    Calmarius August 3rd, 2012

    I usually write tests after I modify my code, because I don't know what tests I need.