Better Testers

code 0 comments suggest edit

In a recent post, Test Better, I suggested that developers can and ought do a better job of testing their own code. If you haven’t read it, I recommend you read that post first. I’m totally not biased in saying this at all. GO DO IT ALREADY!

There was some interesting pushback in the comments. Some took it to mean that we should get rid of all the testers. Whoa whoa whoa there! Slow down folks.

I can see how some might come to that conclusion. I did mention that my colleague Drew wants to destroy the role of QA. But it’s not because we want to just watch it burn.

Rather, we’re interested in something better rising from the ashes. It’s not that there’s no need for testers in a software shop. It’s that what we need is a better idea of what a tester is.

Testers Are Not Second Class Citizens

Perhaps you’ve had different experiences than me with testers. Good for you, here’s a Twinkie. For the vast majority of you, you can probably relate to the following.

At almost every position I’ve been at, developers treated testers like second class citizens in the pecking order of employees. Testers were just a notch above unskilled labor.

Not every tester mind you. There were always standouts. But I can’t tell you how many times developers would joke that testers are just wannabe developers who didn’t make the cut. The general attitude was that you could replace these folks with Amazon’s Mechanical Turk and not know the difference.

mechanical turkMechanical Turk from Wikimedia Commons. Public Domain image.

And in some cases, it was true. At Microsoft, for example, it’s easier to be hired as a tester than as a developer. And once your foot is in the door, you can eventually make that transition to developer if you’re half way decent.

This makes it very challenging to hire and retain good testers.

Elevate the Profession

But it shouldn’t be this way. The problem is we need to elevate the profession of tester. Drew and I talk about these things from time to time and he told me to think of testers as folks who provide a service to developers. Developers should test their own code, but testers can provide guidance on how to better test the code, suggest usage scenarios, set up test labs, etc.

Then it hit me. We already do have testers that are well respected by developers and may serve as a model for what we mean by the concept of better testers.

Security Testers

By most accounts, good security testers are well respected and not seen as folks who are developer wannabes. If not respected, they are feared for what they might do to your system should you piss them off. One security expert I know mentioned developers never click on links he sends without setting up a Virtual Machine to try it in first. Either way, it works.

Perhaps by looking at some of the qualities of security testers and how they integrate into the typical development flow, we can tease out some ideas on what better testers look like.

Like regular testers, many security testers test code that’s ready to ship. Many sites hire white hat penetration testers to attempt to locate and exploit vulnerabilities in a site that’s already been deployed. These folks are experts who keep up to date on the latest in security testing. They are not folks you can just replace with a Mechanical Turk.

Of course, smart developers don’t wait till code is deployed to get a security expert involved. That’s way too late. Security testers can help in the early stages of planning. Provide guidance on what patterns to avoid, what to look out for, some good practices to follow. During the coding stages they can provide code reviews with an eye towards security or simply answer questions you may have about tricky situations.

Testing as a Service

There are other testers that also follow a similar model. If you need to target 12 languages, you’ll definitely want to work with a localization/internationalization tester. If you value usability you may want to work with a usability expert. The list goes on.

It’s just not possible for a developer to be an expert in all these possible areas. I’d expect developers to have a basic understanding of these areas. Perhaps be quite knowledgeable in each, but never as knowledgeable as someone who is focused on these areas all the time.

The common theme among these testers is that they are providing a service to developers. They are sought out for their expertise.

General feature and quality testers should be no different. Good testers spend all their time learning and thinking about better and more efficient ways to test products. They are advocates for the end users and just as concerned about shipping software as developers. They are not gate keepers. They are enablers. They enable developers to ship better code.

This idea of testers as a service is not mine. It’s something Drew told me (he seriously needs to start his blog up again) that struck me.

By necessity, these would be folks who are great developers who have chosen to focus their efforts on the art and science of testing, just as another developer might choose to focus their efforts on the art and science of native clients, or reactive programming.

I love working with someone who knows way more about testing software and building in quality from the start than I do.

This is one of the motivations for me to test my own code better. If I’m going to leverage the skills of a great tester, it’s a matter of pride not to embarrass myself with stupid bugs I should have caught in my own testing. I want to impress these folks with crazy hard bugs I have no idea how to test.

Ok, maybe that last bit didn’t come out the way I intended. The point is when you work with experts, you don’t want them spending all their time with softballs. You want their help with the meaty stuff.

Found a typo or error? Suggest an edit! If accepted, your contribution is listed automatically here.



8 responses

  1. Avatar for Pranav Ainavolu
    Pranav Ainavolu April 24th, 2013

    So true. Interesting writeup

  2. Avatar for JimW
    JimW April 24th, 2013

    Some companies (I can think of a telecom or two, but won't embarrass them) consider testing to be an entry level position for developers! So, they start out as 2nd class citizens in the culture.
    I have great respect for testers, and recognize they have a different mind-set from developers. In my view they like to "break things". To me this is the mark of a good tester. They can teach developer how to be more effective testers of their own code, and should be encouraged to. They bring a lot to the table.

  3. Avatar for Kirk
    Kirk April 24th, 2013

    I guess I have been lucky. In both companies that I have worked for with testers, neither considered that an entry to programming. Both looked for some experience in the industry served. My only issue is that I wished we had more developer support for the tester. I believe it would be good to develop a test to replicate every bug found. Then that be tested without a human giving them more time to find new bugs.

  4. Avatar for ChadF
    ChadF April 24th, 2013

    Years ago I worked on a project with a "tester" that wasn't actually part of the development team.. If I recall correctly, he was a customer advocate/liaison (it was an internal project for another group in the organization), but he made a good tester since he could break it quickly and expose any front-end bugs. Since he wasn't limited by the mindset of the development team's expectations of "how it is _suppose_ to be used", he would usually try things unexpected and trigger outcomes that weren't handled right in the code. Effectively it was like getting User Acceptance Testing through out the development cycle (not just the end).

  5. Avatar for Tom Ritter
    Tom Ritter April 24th, 2013

    Very insightful, and I can't say I fully made the connection myself... even considering my background. I started as a Web Developer (ASP.Net WebForms) in a Financial Services company. We thankfully weren't one of the Daily WTF companies - we cared about the quality of our code, and our alumni went on to Etsy among other similarly advanced shops. We had QA in India, and they were treated as second class citizens. It also kind of bugged me - I felt we should be testing more ourselves, writing more unit tests, building frameworks. When I did some of that myself (as did some other devs on other teams) and worked more with QA to make their lives easier - my area's quality went up.

    I left that company to go do security testing, and 2 1/2 years later, I'm at one of the more specialized and high end consultancies.

    I think I still have a few more years in the field until I figure out how the 'wins' I made as a developer working with QA (unit tests) can be applied generally to security testing and development. Every app I test is so varied maybe they can't. (As an example, imagine a framework that logs in as normal user A and tries hitting every single URL route in your app, including ones they shouldn't have access to. How to detect that a load failed or succeeded is different in every single app; and how to determine if a user should or shouldn't be able to see the page is also different.) Perhaps there is no general way, which is why security scanning services (Veracode, Nessus, WebInspect, etc) are never as thorough as a person.

    Etsy (and specifically Zane Lackey) I think has made a lot of those wins by taking a security tester (Zane, who used to work at my company) and putting him in a role where he can dictate development practices (query standards, logging standards, etc) such that you *can* detect dangerous things (or just 'weird' which usually means dangerous but you don't know how to exploit it). His presentation on security at Etsy is well worth a read.

    And I think the model of getting a good tester (security or otherwise) the pull to dictate development practices is good. (I'm not sayin make them a boss, I'm sayin give them strong input.) As a final example, going back to my first job - we would often have bad data. Testers spent a lot of time writing up bugs about bad data. We decided to cut out the middle man and architect a way to catch that long before they had to test for it. Had the testers had the pull to do that, we would have been developing better, sooner.

  6. Avatar for Block Scientific
    Block Scientific May 3rd, 2013

    Informative article, its really helpful.

  7. Avatar for Jessica Dodson
    Jessica Dodson August 21st, 2013

    The thing to remember about testers is that they aren't out to embarrass the developer and point fingers; their job is to make sure the final product is great! No one likes being told their baby is ugly but you need that completely unbiased, 3rd party eye to point out the problems in your own work.

  8. Avatar for kunal
    kunal February 20th, 2014

    I worked at MS and with due respect it is not easy to get in a SDET compared to SDE. It's a myth. So many good devs are turned down if they are not creative enough to test the software and find bugs.