Thursday, May 6, 2021

Making Sure It Works

The Association for Software Testing is crowd-sourcing a book, Navigating the World as a Context-Driven Tester, which aims to provide responses to common questions and statements about testing from a context-driven perspective.

It's being edited by Lee Hawkins who is posing questions on TwitterLinkedInSlack, and the AST mailing list and then collating the replies, focusing on practice over theory.

I've decided to contribute by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be.

Perhaps you'd like to join me?


"We test to make sure it works"

I know what you mean but, as we both know, there are few certainties in life. I'd probably be more nuanced and perhaps try to capture the intent with something like this: we test it to help increase the chances that people using it for the tasks we expected get value from it. 

If I'm testing the product or feature, I might explore who is likely to be using it, the things they might use it for, and what they would hope to get from it. I'll also be interested in why we built this particular solution, what else we tried, and why the others were rejected.

I'll be expecting to constrain the budget for my testing to a level that is proportionate to the business value we expect to generate.

This naturally means that I won't be able to test everything that I can possibly think of so I will prioritise my testing based on perceived risks to that business value. This is part art and part science so don't be surprised if the risk calculations change as I explore.

One last thing, as well as seeing whether the product works for you, I'll be interested in whether the product does things that we don't intend it to, or that users expressly do not want, and what the effects of those things might be.


  1. "We test to make sure it works"

    Yes, that's what you do ten minutes before your boss demos the product to People Who Matter. But building an entire test strategy on that premise is never going to result in a robust product that can go out into the world and take on all situations. Testing to "make sure it works" gives you a product where the Happy Path is sound, but doing anything that at all deviates from that path is fraught with possible disaster.