What Integrated Circuits Say About Testing Your Code

tdd, code 0 comments suggest edit

A while back I talked about how testable code helps manage complexity. In that post, I mentioned one common rebuttal to certain design decisions made in code in order to make it more testable.

Why would I want to do XYZ just do improve testability?

integrated-circuit Recently, I heard one variation of this comment in the comments to my post on unit test boundaries. Several people suggested that it’s fine to have unit tests access the database, after all, the code relies on data from the database, it should be tested.

Implicit in this statement is the question, “Why would I want to abstract away the data access just to improve testability?

Keep in mind, I never said you shouldn’t test your code’s interaction with the database. You absolutely should. I merely categorized that sort of test as a different sort of test - an integration test. You might still use your favorite unit testing framework to automate such a test, but I suggest trying to keep it in a separate test suite.

The authors of The Pragmatic Programmer: From Journeyman to Master have a great answer to this question with their comparison to Integrated Circuit’s, which have features designed specifically to enable testability.

The “Design For Test” Wikipedia entry refers to name as encompassing a range of design techniques for adding features to microelectronic hardware in order to make it testable. Examples of these techniques show up as early as the 1940s/50s. So designing for testability is not some whiz-bangy latest methodology flavor of the day the crazy kids are doing.

One key benefit to these techniques is that components can be tested in relative isolation. You don’t have to place them into a product in order to test them, though at the same time, they can be tested while within the product.

So in answer to the original question, I’d ask, “Why wouldn’t we design for testability?”

I think this analogy illustrates one reason why I don’t want my unit tests talking to the database (apart from wanting the tests to run fast). Ideally, someone else down the road, new to the project, should be able to get the latest code from source control and run the unit tests immediately without having to go through the pains of setting up an environment with the correct database.

Another benefit of abstracting away the database so that your code is testable and doesn’t cross boundaries is that your code is then not so dependent on a particular database. I used to argue that there’s no need to insulate your code from the particular database that you are using. I’ve never been on a project where the customer suddenly switches from SQL Server to Oracle. That sort of drastic change very rarely happens.

But it turns out that I have been on projects where we switched from SQL Server 6.5 to 7 (and from 7 to 2000 and so on). Upgrades can be nearly as drastic as choosing a different database vendor. Having your code isolated from your choice of database provides some nice peace of mind here.

Tags: IC , Integrated Circuit , TDD , Unit Testing

Found a typo or error? Suggest an edit! If accepted, your contribution is listed automatically here.

Comments

avatar

14 responses

  1. Avatar for Mike Swaim
    Mike Swaim August 4th, 2008

    Thus pointing out that unit tests aren't a silver bullet. You also need integration tests. (I had code break when we went from SQL Server 2000 to 2005. Fortunately, I had test code that caught it.)

  2. Avatar for Kevin Dente
    Kevin Dente August 4th, 2008

    While it may be uncommon for a given system to switch database platforms, it is quite common for ISVs to support multiple database platforms. The less coupled the app logic to the database, the easier that is.

  3. Avatar for Meisinger
    Meisinger August 4th, 2008

    I think this should also expand to testing File I/O
    I have recently been caught by this... it was my fault but I was caught by it none the less :)

  4. Avatar for haacked
    haacked August 4th, 2008

    @Mike Swaim - Definitely true. I rarely see anyone touting unit tests as a Silver Bullet. Merely a good thing to do.

  5. Avatar for Robz
    Robz August 4th, 2008

    Great post!

  6. Avatar for nsoonhui
    nsoonhui August 5th, 2008

    Speaking about why should one make the code testable, it reminds me of this post:
    itscommonsensestupid.blogspot.com/.../...ream.html

  7. Avatar for Seth Petry-Johnson
    Seth Petry-Johnson August 8th, 2008
    Ideally, someone else down the road, new to the project, should be able to get the latest code from source control and run the unit tests immediately without having to go through the pains of setting up an environment with the correct database.


    IMHO, you shouldn't write integration tests unless you also automate the DB creation or initialization. It's no good if your tests rely on data that is maintained somewhere OUTSIDE of the test project: this creates friction when running the tests, which usually means they don't get run. And if they don't get run the test DB starts to lag behind production, and eventually the tests get junked. Keeping a test DB in sync with production is kind of painful, but I think it's a necessary evil.

  8. Avatar for David Nelson
    David Nelson August 10th, 2008

    "Why wouldn’t we design for testability?"
    Because it can sometimes (not always, but sometimes) lead to a compromised design. For example (and I really don't want to start this debate here, I am just using it as an example), the Framework Design Guidelines recommend using abstract classes instead of interfaces, for a number of very good reasons. The TDD crowd has long argued against that guideline, primarily because it makes mocking more difficult. That is, they would rather give up the advantages of using abstract classes instead of interfaces for no other reason than testability. I don't think that's a good way to design software; testability is important, but so is usability, maintainability, and extensibility. If you can include testability in your design while still adhering to other good design principles, that's great. But if I have to make a choice, testability is not at the top of my priority list.

  9. Avatar for Matt Holmes
    Matt Holmes August 13th, 2008

    I agree with David Nelson. I design my software for stability, usability, maintainability and extensibility first. I worry about things like testability later. I guess my biggest problem with a lot of these new movements that come around, is that they sound great on paper, and work in an academic setting, then you put them in the hands of the grunts in the trenches like me, and they fall apart. Those of us who develop with deadlines and dollars on the line, aren't likely to re-envision the way we design software so that a few unit tests can be easily created. I would much rather mold my unit tests around my well designed software, than mold my software around a bunch of unit tests. My customers don't see unit tests, they see software. Other developers that get hired on don't care how my unit tests are structured, they care how the meat of my code is structured and designed. If I were to go to my boss and say "Well boss, our code design probably isn't optimal, but darn tootin' if it doesn't test well", I'd probably get fired.
    I also think that using integrated circuits as an analogy is sort of silly. Integrated circuits would be nigh impossible to test without facilities, they are integrated circuits. I can test my software without ever putting a single piece of test rigging inside of it. That's not to say I don't value unit tests, but I think look at the microprocessor designers and going "well, they design hardware XYZ way, we should design software XYZ way" is not the most productive leap to make.

  10. Avatar for haacked
    haacked August 13th, 2008

    I don't think the micro-processor analogy is silly. Your customers don't see microprocessors, they see cell-phones, set-top boxes. These things happen to be made up of components such as microprocessors which need to be tested in isolation.
    I was a consultant for many years for paying customers, probably like yours, with tight deadlines and dollars on the line. And I wrote unit tests, not as an academic exercise, but because it helped me meet the deadlines with quality code.
    I think the debate between "practical" vs "ivory tower" is a false one. The reason that this sort of TDD style development doesn't seem practical in a deadline driven development project is because it isn't, if you're completely new to it. But if you can practice it on the side somehow, and get very good at it, you realize it is a very practical technique.
    I'm not saying it's required to deliver value to your customers. Many consultants I know do so. I'm suggesting it's a useful tool that is very practical for delivering value to your customers. And it is similar to a tradition of delivering quality well established with microprocessor engineering.

  11. Avatar for Rob Conery
    Rob Conery August 13th, 2008

    >>>I design my software for stability, usability, maintainability and extensibility first. <<<
    I think those are great goals. How do you back that up? Can you quantify this? Or is it just an assertion you're making here?
    >>>I guess my biggest problem with a lot of these new movements that come around, is that they sound great on paper, and work in an academic setting, then you put them in the hands of the grunts in the trenches like me, and they fall apart. <<<
    Why do they fall apart?
    >>>I would much rather mold my unit tests around my well designed software, than mold my software around a bunch of unit tests.<<<
    Wouldn't we all :). You must be very good at what you do :).
    >>>Other developers that get hired on don't care how my unit tests are structured, they care how the meat of my code is structured and designed.<<<
    A bunch of well-written tests would show them all they needed to know....
    >>>If I were to go to my boss and say "Well boss, our code design probably isn't optimal, but darn tootin' if it doesn't test well", I'd probably get fired.<<<
    This is a silly statement (unless you really use "darn tootin" regularaly... which you might... just sayin...). Your code is your code, always. Your tests are your PROOF that you met the requirements as handed to you. So, to rephrase your statement "Well boss here's the requirements, here's the test coverage and results - can I have a raise?".
    Here's the deail - I was, like you are now, a nay-sayer. I didn't see the benefit of writing code for code's sake, and thought that this was yet another way to charge the client for money.
    And then I decided to hang up the presumption and try it (Green Lights and Ham should be the name of a TDD book). And wouldn't ya know... it SAVED me time after time.
    I'm not going to try and convince anyone - but ask Phil, I used to give him a hard time constantly. And now I eat those words whenever I see him.
    Just try it is all I'm suggesting.

  12. Avatar for Kevin
    Kevin August 13th, 2008

    >>then you put them in the hands of the grunts in the trenches like me
    Those techniques came from people who were "grunts in the trenches" and are used by many people who are "grunts in the trenches" with much praise. You should rephrase your statement to be something like "then you put them in the hands of someone who doesn't want to learn and already has a predisposition against it and who thinks pretty damned highly of their abilities as is and they fall apart mainly because of the lack of effort, understanding and perhaps intelligence of the person behind the keyboard"

  13. Avatar for jsmith
    jsmith January 17th, 2009

    Besides, If you don't TDD how can you "drive out the design" of your code. After all, TDD is really about design. Right? Right.
    It's getting pretty deep when MS employees are beating us over the head with this.

  14. Avatar for Gary Woodfine
    Gary Woodfine November 6th, 2011

    Great post. One more thing to raise is that the database should have it's own set of unit tests. That test that it is preforming correctly.
    100% on the integration tests, these are different type of tests and are not part of your development unit tests.