What Integrated Circuits Say About Testing Your Code

A while back I talked about how testable code helps manage complexity. In that post, I mentioned one common rebuttal to certain design decisions made in code in order to make it more testable.

Why would I want to do XYZ just do improve testability?

integrated-circuit Recently, I heard one variation of this comment in the comments to my post on unit test boundaries. Several people suggested that it’s fine to have unit tests access the database, after all, the code relies on data from the database, it should be tested.

Implicit in this statement is the question, “Why would I want to abstract away the data access just to improve testability?

Keep in mind, I never said you shouldn’t test your code’s interaction with the database. You absolutely should. I merely categorized that sort of test as a different sort of test - an integration test. You might still use your favorite unit testing framework to automate such a test, but I suggest trying to keep it in a separate test suite.

The authors of The Pragmatic Programmer: From Journeyman to Master have a great answer to this question with their comparison to Integrated Circuit's, which have features designed specifically to enable testability.

The “Design For Test” Wikipedia entry refers to name as encompassing a range of design techniques for adding features to microelectronic hardware in order to make it testable. Examples of these techniques show up as early as the 1940s/50s. So designing for testability is not some whiz-bangy latest methodology flavor of the day the crazy kids are doing.

One key benefit to these techniques is that components can be tested in relative isolation. You don’t have to place them into a product in order to test them, though at the same time, they can be tested while within the product.

So in answer to the original question, I’d ask, “Why wouldn’t we design for testability?”

I think this analogy illustrates one reason why I don’t want my unit tests talking to the database (apart from wanting the tests to run fast). Ideally, someone else down the road, new to the project, should be able to get the latest code from source control and run the unit tests immediately without having to go through the pains of setting up an environment with the correct database.

Another benefit of abstracting away the database so that your code is testable and doesn’t cross boundaries is that your code is then not so dependent on a particular database. I used to argue that there’s no need to insulate your code from the particular database that you are using. I’ve never been on a project where the customer suddenly switches from SQL Server to Oracle. That sort of drastic change very rarely happens.

But it turns out that I have been on projects where we switched from SQL Server 6.5 to 7 (and from 7 to 2000 and so on). Upgrades can be nearly as drastic as choosing a different database vendor. Having your code isolated from your choice of database provides some nice peace of mind here.

What others have said

Requesting Gravatar... Mike Swaim Aug 04, 2008 3:36 AM
# re: What Integrated Circuits Say About Testing Your Code
Thus pointing out that unit tests aren't a silver bullet. You also need integration tests. (I had code break when we went from SQL Server 2000 to 2005. Fortunately, I had test code that caught it.)
Requesting Gravatar... Kevin Dente Aug 04, 2008 3:48 AM
# re: What Integrated Circuits Say About Testing Your Code
While it may be uncommon for a given system to switch database platforms, it is quite common for ISVs to support multiple database platforms. The less coupled the app logic to the database, the easier that is.
Requesting Gravatar... Meisinger Aug 04, 2008 8:55 AM
# re: What Integrated Circuits Say About Testing Your Code
I think this should also expand to testing File I/O

I have recently been caught by this... it was my fault but I was caught by it none the less :)
Requesting Gravatar... haacked Aug 04, 2008 1:00 PM
# re: What Integrated Circuits Say About Testing Your Code
@Mike Swaim - Definitely true. I rarely see anyone touting unit tests as a Silver Bullet. Merely a good thing to do.
Requesting Gravatar... Robz Aug 04, 2008 1:34 PM
# re: What Integrated Circuits Say About Testing Your Code
Great post!
Requesting Gravatar... Ngu Soon Hui Aug 05, 2008 10:34 AM
# re: What Integrated Circuits Say About Testing Your Code
Speaking about why should one make the code testable, it reminds me of this post:

itscommonsensestupid.blogspot.com/.../...ream.html
Requesting Gravatar... Seth Petry-Johnson Aug 08, 2008 3:05 AM
# re: What Integrated Circuits Say About Testing Your Code
Ideally, someone else down the road, new to the project, should be able to get the latest code from source control and run the unit tests immediately without having to go through the pains of setting up an environment with the correct database.


IMHO, you shouldn't write integration tests unless you also automate the DB creation or initialization. It's no good if your tests rely on data that is maintained somewhere OUTSIDE of the test project: this creates friction when running the tests, which usually means they don't get run. And if they don't get run the test DB starts to lag behind production, and eventually the tests get junked. Keeping a test DB in sync with production is kind of painful, but I think it's a necessary evil.

Requesting Gravatar... David Nelson Aug 11, 2008 1:23 AM
# re: What Integrated Circuits Say About Testing Your Code
"Why wouldn’t we design for testability?"

Because it can sometimes (not always, but sometimes) lead to a compromised design. For example (and I really don't want to start this debate here, I am just using it as an example), the Framework Design Guidelines recommend using abstract classes instead of interfaces, for a number of very good reasons. The TDD crowd has long argued against that guideline, primarily because it makes mocking more difficult. That is, they would rather give up the advantages of using abstract classes instead of interfaces for no other reason than testability. I don't think that's a good way to design software; testability is important, but so is usability, maintainability, and extensibility. If you can include testability in your design while still adhering to other good design principles, that's great. But if I have to make a choice, testability is not at the top of my priority list.
Requesting Gravatar... Matt Holmes Aug 13, 2008 7:07 AM
# re: What Integrated Circuits Say About Testing Your Code
I agree with David Nelson. I design my software for stability, usability, maintainability and extensibility first. I worry about things like testability later. I guess my biggest problem with a lot of these new movements that come around, is that they sound great on paper, and work in an academic setting, then you put them in the hands of the grunts in the trenches like me, and they fall apart. Those of us who develop with deadlines and dollars on the line, aren't likely to re-envision the way we design software so that a few unit tests can be easily created. I would much rather mold my unit tests around my well designed software, than mold my software around a bunch of unit tests. My customers don't see unit tests, they see software. Other developers that get hired on don't care how my unit tests are structured, they care how the meat of my code is structured and designed. If I were to go to my boss and say "Well boss, our code design probably isn't optimal, but darn tootin' if it doesn't test well", I'd probably get fired.

I also think that using integrated circuits as an analogy is sort of silly. Integrated circuits would be nigh impossible to test without facilities, they are integrated circuits. I can test my software without ever putting a single piece of test rigging inside of it. That's not to say I don't value unit tests, but I think look at the microprocessor designers and going "well, they design hardware XYZ way, we should design software XYZ way" is not the most productive leap to make.
Requesting Gravatar... haacked Aug 13, 2008 8:16 AM
# re: What Integrated Circuits Say About Testing Your Code
I don't think the micro-processor analogy is silly. Your customers don't see microprocessors, they see cell-phones, set-top boxes. These things happen to be made up of components such as microprocessors which need to be tested in isolation.

I was a consultant for many years for paying customers, probably like yours, with tight deadlines and dollars on the line. And I wrote unit tests, not as an academic exercise, but because it helped me meet the deadlines with quality code.

I think the debate between "practical" vs "ivory tower" is a false one. The reason that this sort of TDD style development doesn't seem practical in a deadline driven development project is because it isn't, if you're completely new to it. But if you can practice it on the side somehow, and get very good at it, you realize it is a very practical technique.

I'm not saying it's required to deliver value to your customers. Many consultants I know do so. I'm suggesting it's a useful tool that is very practical for delivering value to your customers. And it is similar to a tradition of delivering quality well established with microprocessor engineering.
Requesting Gravatar... Rob Conery Aug 13, 2008 8:44 AM
# re: What Integrated Circuits Say About Testing Your Code
>>>I design my software for stability, usability, maintainability and extensibility first. <<<

I think those are great goals. How do you back that up? Can you quantify this? Or is it just an assertion you're making here?

>>>I guess my biggest problem with a lot of these new movements that come around, is that they sound great on paper, and work in an academic setting, then you put them in the hands of the grunts in the trenches like me, and they fall apart. <<<

Why do they fall apart?

>>>I would much rather mold my unit tests around my well designed software, than mold my software around a bunch of unit tests.<<<

Wouldn't we all :). You must be very good at what you do :).

>>>Other developers that get hired on don't care how my unit tests are structured, they care how the meat of my code is structured and designed.<<<

A bunch of well-written tests would show them all they needed to know....

>>>If I were to go to my boss and say "Well boss, our code design probably isn't optimal, but darn tootin' if it doesn't test well", I'd probably get fired.<<<

This is a silly statement (unless you really use "darn tootin" regularaly... which you might... just sayin...). Your code is your code, always. Your tests are your PROOF that you met the requirements as handed to you. So, to rephrase your statement "Well boss here's the requirements, here's the test coverage and results - can I have a raise?".

Here's the deail - I was, like you are now, a nay-sayer. I didn't see the benefit of writing code for code's sake, and thought that this was yet another way to charge the client for money.

And then I decided to hang up the presumption and try it (Green Lights and Ham should be the name of a TDD book). And wouldn't ya know... it SAVED me time after time.

I'm not going to try and convince anyone - but ask Phil, I used to give him a hard time constantly. And now I eat those words whenever I see him.

Just try it is all I'm suggesting.
Requesting Gravatar... Kevin Aug 13, 2008 9:54 AM
# re: What Integrated Circuits Say About Testing Your Code
>>then you put them in the hands of the grunts in the trenches like me
Those techniques came from people who were "grunts in the trenches" and are used by many people who are "grunts in the trenches" with much praise. You should rephrase your statement to be something like "then you put them in the hands of someone who doesn't want to learn and already has a predisposition against it and who thinks pretty damned highly of their abilities as is and they fall apart mainly because of the lack of effort, understanding and perhaps intelligence of the person behind the keyboard"
Requesting Gravatar... jsmith Jan 17, 2009 1:05 PM
# re: What Integrated Circuits Say About Testing Your Code
Besides, If you don't TDD how can you "drive out the design" of your code. After all, TDD is really about design. Right? Right.

It's getting pretty deep when MS employees are beating us over the head with this.
Requesting Gravatar... Gary Woodfine Nov 06, 2011 10:08 PM
# re: What Integrated Circuits Say About Testing Your Code
Great post. One more thing to raise is that the database should have it's own set of unit tests. That test that it is preforming correctly.

100% on the integration tests, these are different type of tests and are not part of your development unit tests.

What do you have to say?

(will show your gravatar)
Please add 3 and 1 and type the answer here: