The Association for Software Testing is crowd-sourcing a book, Navigating the World as a Context-Driven Tester, which aims to provide responses to common questions and statements about testing from a context-driven perspective.
It's being edited by Lee Hawkins who is posing questions on Twitter, LinkedIn, Slack, and the AST mailing list and then collating the replies, focusing on practice over theory.
I've decided to contribute by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be.Perhaps you'd like to join me?
--00--
"Why didn’t you find those issues before we shipped?"
Assuming that's an unloaded request for information, I'd say that your answer will lie somewhere in here:
- we didn't look, or
- we didn't look in the right places, or
- we looked in the right places but didn't provoke the issue, or
- we looked in the right places and provoked the issue, but didn't realise
- we looked in the right places, and provoked the issue, and realised, but didn't understand the impact
If you're also interested in the reasons for those things, here's some possibilities:
- our ideas of what to review didn't cover this
- our risk assessment prioritised other areas
- our budget didn't permit us to test all of the places we thought of
- our ability to control the product behaviour is limited
- our visibility of the product behaviour is limited
- our access to environments in which to test is limited
And, again, if you were to ask me to explain that list, I'd suggest this kind of thing:
- we didn't understand the domain implications
- we didn't understand the code to a relevant depth or breadth
- we didn't understand the requirements, implicit or explicit
- we didn't spend enough time with enough perspectives to think of this possibility
- we didn't place a high value on checking our work relative to building it
- we didn't place a high value on checking our work relative to shipping it
From previous experience, I know you've heard of the Five Whys, and if I sense that you're going to keep on in this direction I might stop and ask you a question to try to understand your motives:
Who do you think the "we" I'm referring to is?
I'm fearing that, because you're asking me, you will say "the tester."
If you do, I'll know that it wasn't an unloaded enquiry after all and we can begin to have a conversation about collective responsibility, from management down, for what we deliver.
If you don't then we're already a couple of steps along the road and we can have an honest and open conversation about causes and potential mitigations and their costs.
Image: https://flic.kr/p/LM1NpN
Comments
Post a Comment