Bug Driven Development

Sep 24, 2007 code tdd methodologies suggest edit

It is a sad fact of life that, in this day and age, arguments are not won with sound logic and reasoning. Instead, applying the principle of framing an argument is much more effective at swaying public opinion.

1364145387_b8cf994488 So the next time you try to make headway introducing Test Driven Development (or even simply introducing writing automated unit tests at all) into an organization and are rebuffed with…

Don’t bring your fancy schmancy flavor of the week agile manifesto infested “methodology” here kiddo. I’ve been writing software my way for a loooong time…

You can reply with…

I’m sorry, but I’m not a fan of Bug Driven Development. I think Test Driven Development is not without its challenges, but it’s a better alternative. Either you’re with us, or against us. Are you a bug lover? Bug Driven Development gives comfort to the bugs.

UPDATE: this is an example of my dry humor. I don’t believe that “Framing” is a good way to win an argument and I would never actually say or recommend saying anything similar to to this. It’s meant as a bit of a joke, but with a point.

A team that is not focused on automated testing of some sort throughout the lifecycle of the project is effectively embracing Bug Driven Development. Bugs are going to drive the development cycle at the end of the project.

Don’t believe me though, look at the research done by others. In Facts and Fallacies of Software Engineering, Robert Glass points out…

Fact 31. Error removal is the most time-consuming phase of the life cycle.

In Rapid Development, Steve McConnell relates…

Shortcutting 1 day of QA activity early in the project is likely to cost you from 3 to 10 days of activity downstream.

In other words, if you don’t control the bugs, the bugs control your schedule.

Found a typo or mistake in the post? suggest edit

Comments

36 responses

Steven Harman • September 23rd, 2007
But the BDD acronym is already taken - Behavior Driven Development. And to be honest I'm still 100% clear on exactly what BDD is as it seems to be a summation of TDD, Stories, DDD, etc...
DotNetKicks.com • September 23rd, 2007
You've been kicked (a good thing) - Trackback from DotNetKicks.com
Haacked • September 23rd, 2007
@Steve I don't care that BDD is taken. My BDD is better. ;)
Actually, I think mine should be acronymed like this...
BUg Driven Development or BUDD for short.
As in, you must be smoking bud to think BUDD is a good way to develop software.
she • September 23rd, 2007
The thing is, "bug driven development" is the most simple and straightforward one. Its a lot associated with experience too, the more experienced one gets, the less bugs he does (but it also depends on the language, and whether it supports certain coding schemes or not)
So i can understand everyone who is sceptical, no matter how many OTHERS are persuaded that their model is better. :)
I think however there should be some kind of contest. A fair contest, where we would measure different techniques with a somewhat similar "reallife usage case" (and it should be a quite complex problem too and the programmers involved should have some years experience too)
Josh Stodola • September 23rd, 2007
Great post, Phil! A definite reminder.

As in, you must be smoking bud to think BUDD is a good way to develop software.

LMAO!
Haacked • September 23rd, 2007
@she Well the competition would have to last at least two years. Maintenance typically consumes 40 to 80 percent of software costs.
Assuming a software system lifecycle of two years here's a reasonable competition that would mimic real life.
1. Two teams are divided into two groups each. Group 1 has 3 months to build an app. Group 2 doesn't get to watch.
2. Teams get requirements from a person acting as a "client".
3. Every week, the client throws in a requirements change.
4. At the end of the project, Group 1 leaves and Group 2 comes in without any prior knowledge.
5. Now for the next year and nine months, group 2 must maintain the app.
6. Every week, the client will report changes to the app that the groups must make.
7. All the while, the groups must fix any bugs reported by the QA team.
I think this would be a relatively fair competition to determine the efficacy of TDD vs BuDD.
Howard • September 23rd, 2007
The kind of "framing the argument" that you have in mind is what people sometimes do when they lack real arguments for their point. For example, "Death Tax" "Right to Life," and “Woman’s Right to Choose.”
Test driven development is great in some contexts. For example, it's perfect for the ongoing development of a database engine that exposes a well-defined API. At the other extreme is what I have been involved with recently, namely incremental development, by a small team, of a database-driven workflow automation application -- the only place TDD has paid off for this has been in developing various helper functions. Writing a complete suite of autotmated tests would probably double our effort.
Even if TDD eliminates 75% of the bugs – and how likely is that – the situation is qualitatively the same. You still need to expose the product to testers and field users – “Bug-Driven Development.”
Howard • September 23rd, 2007
The “fair competition” should include the extra time that Group 2 took in the construction phase. The fact that TDD hasn’t been widely adopted proves to me that TDD adds substantially to the time to construct software.
Also we should include the time for Group 2 to get up to speed on the tests and maintain them. This would include the work to update the tests that are continually getting broken because of the weekly client change requests. If you are smart enough to design your tests so they are well factored and tend not to get broken, then you are smart enough to design you application itself so that its parts are equally well factored. Then most bugs will appear in new functions that are being beta tested, and not as regression errors.
Ricky Clarkson • September 24th, 2007
At best, TDD is Bug Driven Development, where the bug doesn't live for very long.
I can equally say to you that I don't believe in your Test Driven Development, and that I now mandate Proof Driven Development. Here is a pile of maths books.
Haacked • September 24th, 2007
@Howard, well making sure the code is well factored also takes time. And no matter what, I guarantee you that when you need to make a small change to a codebase, it's a *lot* faster to be able to make that change when you have a decent suite of automated regression tests. Even in "well factored" code.
How many times have you approached a codebase that was so well factored that you could make a change and have a high degree of confidence you didn't affect other code bases. Unit tests are no panacea, but they do provide confidence.
The Other Steve • September 24th, 2007
I’m sorry, but I’m not a fan of Bug Driven Development. I think Test Driven Development is not without its challenges, but it’s a better alternative. Either you’re with us, or against us. Are you a bug lover? Bug Driven Development gives comfort to the bugs.

Not a great way to win over friends.
Most people I know are not opposed to writing tests. What is lacking, is a good explanation as to how. Especially since most development done today is on an existing code base. I think evangelizing TDD could better be served by helping to show how easy it might be in the long run and helping people get past that initial learning curve.
The app I've been working on the past year at work is a gigantic collosal mess which was developed using the Big Ball of Mud Methodology. There's no way to unit test this mass... I have a hard enough time just stepping through the code. Which leads me to another observation. It appears that TDD only works well with certain application architectures. I've recently been working with the MVP pattern you talked about earlier in a side project at home. Once I got that working, things started going together much easier. It was hard to get the first feature working, but as I added new functionality it was a breeze to put in place. But that's a way of development that in my experience, most people don't know or do.
Again, people aren't opposed. It's just hard to see how you get from big ball of mud to something good.
And make it idiot proof enough that even the idiot on your team can understand. (Remember that 80% of developers in the real world are nowhere near as bright as the other 20%)
Haacked • September 24th, 2007
The fact that TDD hasn’t been widely adopted proves to me that TDD adds substantially to the time to construct software.

How is that a proof? Depends on what you mean by widely adopted. Is it over 50%? Perhaps not. Is it a significant number? Definitely. Not too hard to find a dev who gets TDD.
And I did point out in my post, one would gain plenty of benefits by just starting to introduce automated unit tests and not completely adhering to the TDD dogma.
However, having said that, adoption rate is not a proof that TDD adds a significant time or not. There are plenty of experienced TDD devs out there who can develop using TDD not much slower than an equivalent experienced developer can without TDD.
I remember working on one particular VB app written to parse Fortran that was incredibly challenging to change without breaking. I had to write tests because every change would introduce some unforeseen bug that I would then spend hours tracking down and then fixing, not to mention the hours in which I didn't even know a bug had been introduced by the fix for another bug. Test coverage would've kept the overall time down.
Both teams get 3 months, but both teams have to live with the bugs they introduced in that time.
engtech • September 24th, 2007
This is our workflow in a nutshell.
ugh
Haacked • September 24th, 2007
Not a great way to win over friends.

@TheOtherSteve - Well I was joking. I would never actually say that.
Haacked • September 24th, 2007
What is lacking, is a good explanation as to how. Especially since most development done today is on an existing code base. I think evangelizing TDD could better be served by helping to show how easy it might be in the long run and helping people get past that initial learning curve.

I've heard good things about the book, Working With Legacy Code. I haven't read it myself yet, but I've personally had a lot of success adding unit tests after the fact to code and applying structured refactorings.
Case in point, when I started with Subtext (taking the existing .TEXT code) there were 0 unit tests. Now there are 785 and 36% code coverage (and counting!).
Now granted, .TEXT was no ball of mud, but testing after the fact can be done.
I've worked with balls of mud in the past and the approach I took was iterative. Anytime I had to fix a bug, I would try to write a test that fails because of the bug, then fix the bug and make sure that the test passes because of the fix.
Many times, I could find small chunks of code I could factor out into its own method. Over time, a ball of mud can get refactored into something nicer and more testable. But it takes discipline and patience.
Haacked • September 24th, 2007
Even if TDD eliminates 75% of the bugs – and how likely is that – the situation is qualitatively the same. You still need to expose the product to testers and field users – “Bug-Driven Development.”

I'm definitely not arguing against that point. As I've said before.

Automated unit tests are NOT a replacement for system testing, beta testing, integration testing, nor any other kind of testing. Unit tests are only one small part of the testing equation, but they are an important part, just as all the other types of testing are important.

The point is not that you're going to be able to eradicate all bugs. Rather, it's using all tools at your disposal (in their proper contexts) to gain the upper hand against having bugs run your schedule.
Sure, TDD may not work so well in all contexts, such as developing a workflow site or a sharepoint site perhaps. But in the contexts where it does work well, it works well.
For web apps, I personally recommend a combination of unit testing and using something like WATIN for web testing tricky parts of the web UI. I think it's worth the test maintenance cost to be able to run a QA cycle in half the time or less because much of it is automated.
Mr. Teflon • September 24th, 2007
I don't need no stinkin' TDD, I line my software up with Teflon, bugs just bounce off! :-)
I think that the answer is always it depends. It depends on the team that you have, quality of the developers.
For example, I have been working on a very complex library with over 200,000 lines of code (written from scratch) for last 5 years with changes and additions made almost every single day. I do not have many tests for that library, yet it is of very high quality with over 10,000 deployments.
There are bugs of course but there just aren't that many given the size of the project. Looking at my accurate release notes over 5 years, there were more features added than bugs found.
So what is the secret here? First, I am one and the only one working on that code. Second, I do not under any conditions commit the code that I have not executed, lot of times line by line in debugger, to make sure it works. I can't stress how important this last step actually is.
It works great for me.
However, once you get into the teams, not everyone can be this disciplined. Actually most people aren't. I've seen people check in branches of code that I know they did not execute. I have seen people check in the code that even breaks the builds... That is where TDD and continuous builds help.
Actually thinking about it, if you have more than 2 people working on the code base you should be well served with having at least some tests as part of your code base.
Christopher Steen • September 24th, 2007
Link Listing - September 24, 2007
Chris Moorhouse • September 24th, 2007
Even if TDD eliminates 75% of the bugs – and how likely is that – the situation is qualitatively the same. You still need to expose the product to testers and field users – “Bug-Driven Development.”

So, when the user types in X, Y information should be displayed. Oh, heck, did I just write a test? Geez, I thought we weren't going to use that useless Test Driven stuff on this project.
craig cameron • September 24th, 2007
In relation to mr teflon, working as a single developer is much much easier. I once worked on a project with a few junior developers and the bugs spiraled out of control. I then kicked them off the project and we went to only one developer and the bugs started to disappear. While this is not always an option if you can create functional silos in the code and then assign responsibility to individuals you will tend to be better off.
Frans Bouma • September 24th, 2007
I'm all for weeding out bugs early on as much as possible, and unit tests can help. However, the focus should be on weeding out bugs, not on writing tests. The thing is: writing tests is OK, but will never be enough to squash most bugs, simply because some software will simply require an unlimited amount of tests to be sure all bugs are gone.
What helps more is proof that the software written is correct. This is tedious work as well, but can also be done earlier on. So added to the tests which can show that the software changes and new feature(s) are indeed working in the context of the tests (because that's what you're testing), you have to have proof that the code is correct. Proving that code is correct isn't that hard, it just takes time. But lacking proof and thinking the tests are the proof is a mistake: the tests only proof what's tested in the tests. If you think the tests combined define 100% of the cases the software will be used in, you're mistaken as well. A simple multi-db oriented o/r mapper API alone has an unlimited amount of corner cases where it can go wrong, so you need proof that a routine is correct.
Andy Stopford • September 24th, 2007
Frans, some good points and what you want to avoid is writing masses and masses of tests (which would be avoided by most manageemnt) or writing too little tests. Two things help here, frameworks that reduce this and do the work for you (MbUnits row and pairwise or XUnits theory attributes for example) coupled with a frameworks test input factories and random drivers.
Prehhaps this is where TDD had a big distiniction over just writing tests. TDD uses tests to map to the user stories so your tests will model the apps functionality so you are pinpointing areas that are suppose to work the way you require.
Tests coupled with requirements are the best deal and with tests you can adjust your tests to meet the need functionality and refactor with no fear of introducing regression bugs. Refactoring with no tests is like swimming with sharks with your leg hanging off.
Derik Whittaker • September 24th, 2007
Today I came across a nice post on &#39;Bug Driven Development&#39; . This is something that most developers
Stephan • September 24th, 2007
How about DeDD - for Defect Driven Development?
Kevin Dan • September 25th, 2007
Bug Driven Development == Code and Fix, an approach should be replaced by Test Driven Development, that is, Test always comes first.
Legacy code == code without testing counterpart
Karthik Hariharan • September 26th, 2007
Couldn't agree more. But there's another equally frustrating response you get when trying to implement TDD.
"Yeah thats a great idea, we really should start doing that..."
3 months later...nothing has changed.
Michael White • September 27th, 2007
Granted, this definitely does NOT solve the problems involved in working with legacy code - but building at solid framework WITH tests, and basing your application code on that framework will work wonders. By tests I do not simply mean test methods that are intended to catch bugs but rather test in a QA fashion. Build mini-apps that take care of feature A, B, C etc. The features these mini-apps are tests for are the features that are contained directly in the framework base code. Anything outside of that is project based testing. Frameworks take various time scales to develop and test thoroughly depending on size - but once you have your framework running smoothly, you will find your bug count drop considerably, and if you have a framework of really useful code then application development time will drop considerably as well. This reduces overall time tremendously in most cases. I have done a few projects now on a framework I designed and it has cut development by 2/3 and testing finds and extremely low number of bugs that are usually very easy to find and fix.
Final point: There is no single "best" way to do anything. Often it is project based and you just have to hand pick the methodology and systems you (and your team) work best with.
Alex • September 27th, 2007
With all of this talk of eliminating bugs, in my opinion, the best way to do this is to pair program and have mandatory code reviews. This way, younger developers are continually in check and they learn coding styles from more seasoned devs. Code reviews should be mandatory regardless. If you have trouble getting the company to buy off with TDD, then good luck with XP. I cannot stand when businesses try and cut corners with software.
Tod McKenna • September 29th, 2007
Code reviews are indeed a good practice. There are those who don't see the worth (bad experience, perhaps?), but I've seen firsthand how code reviews + test driven development + instrumentation can make a gigantic impact on the quality of the released software.
Of course, if you're a lone star, or code in your underwear, then maybe finding a good code reviewer isn't an option!
Anyway, thanks Mr. Haack for yet another great read...
arakiel • October 2nd, 2007
Well lookie here, it's the latest version of
"X is better then Y because I said so"
And once again, nothing to back up WHY X is so much better then Y, we just have to take the dogma preachers word for it.
If your not going to bother explaining why your dogma is better then Y then your no better then those who dismiss it.
Brian • October 4th, 2007
In the restaurant industry there is a saying among waiters. "Run your table or they will run you."
I'm not surprised that our industry cannot use logic. We write it all the time, but cant use it. Maybe thats why our projects fail a lot.
Greg D • October 21st, 2007
The dysfunctional flavor of bug-driven development that I find most maddening is the attitude that a devteam won't fix known bugs until a [em]customer[/em] finds them first. So yeah, I know of at least a dozen different crash errors in the rotting codebase that I just inherited, but I'm not allowed to go in and fix 'em because none of our customers have reported them yet.
Of course, I suppose I shouldn't be too surprised when we consider killing and restarting the software to be a "quick and easy" workaround for most of the bugs in this thing.
Is anybody out there hiring software devs who care about the quality of their software?
Eric • October 25th, 2007
I might have to add this to my list of Little Known Development Drivers: http://coderoshi.com/2007/0...
Rajesh Kumar N.Ch • August 16th, 2011
To start follow TDD to get into production implement BDD.
To implement BDD follow TDD.
haacked • March 18th, 2012
@arakiel Research supports the effectiveness of TDD
Calmarius • August 3rd, 2012
I usually write tests after I modify my code, because I don't know what tests I need.