Thursday, May 6, 2021

Just Automate the Testing


The Association for Software Testing is crowd-sourcing a book, Navigating the World as a Context-Driven Tester, which aims to provide responses to common questions and statements about testing from a context-driven perspective.

It's being edited by Lee Hawkins who is posing questions on TwitterLinkedInSlack, and the AST mailing list and then collating the replies, focusing on practice over theory.

I've decided to contribute by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be.

Perhaps you'd like to join me?


"Let's Just Automate the Testing"

Yes, let's talk about how automation can help us test! I'd usually start by trying to understand what problem you want to solve but as we don't have that let's start with some ideas about what you might want to automate.

At one end of a spectrum, perhaps there is some repetitive confirmatory work that humans currently do that is time-consuming and boring and is amenable to being implemented in code? That seems plausible as a target. We'd probably want to consider effort and value factors such as the costs associated with developing and maintaining the automation, the risks it would mitigate, and the potential multipliers that it could bring, such as documenting behaviour, and being able run in different environments.

At the other, perhaps there is a way of automating the thinking work of a skilled human who looks carefully at a product, the context it was created for, and its development process and considers the risks associated with building that thing that way for that purpose? That seems less plausible doesn't it? But that's important information if you want to build the right kind of product at the right kind of price and the right kind of time.

Automation can seem attractive: save time, save costs, and bake quality in! Even better, an ever-increasing number of test cases with a high pass percentage scratches that management-level spreadsheet itch in ways that testing, whatever it is, just can't.

Does that sound jaundiced? Don't get me wrong, automation is a powerful tool and I use it a lot. I just don't think automation replaces testing. For me, testing is knowledge work. While there's often a great deal of scope for automating a path after it's been trodden, it takes intent and agency to find the paths worth following and the things to look for while walking along them.

No comments:

Post a Comment