Thursday, April 15, 2021

Testing By Exploring

This week I listened to a Ministry of Testing podcast on exploratory testing.  After the intros, as the panelists  attempted in turn to give their definitions of exploratory testing, I realised that I didn't have one for myself.

It's not that I don't know or haven't seen definitions. Here's one I've heard a few times, from back in 2008, in A Tutorial in Exploratory Testing by Cem Kaner: 

Exploratory software testing is a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.
It's not that I think The Exploratory Elders like Kaner got it right back in the day and their words should be carved on monumental stones and polished on Sundays, or that I'm not interested any more. I was reading Maaret Pyhäjärvi's post, Exploratory Testing the Verb and the Noun, just this morning.

It's not that I think definitions are just playing semantics either. Spending the time to understand what I mean when I say something can be enlightening, as well as respectful to the people I choose to say it to. Further, I use definitions as hands-on, practical tools. This is one of my favourites, due to Jerry Weinberg:

A problem is a difference between things as desired and things as perceived.
Exploration is intimately wrapped up in the way I want to test and it was intriguing to find that I hadn't given some explicit thought to the term exploratory testing before. After a little reflection, I think the answer is that I made my definition of testing accommodate it.
Testing is the pursuit of relevant incongruity.
I'll use whatever approach I think is most suitable given the context to implement my testing. Predominantly that is exploration and I don't think  exploration in software is materially different to exploration in general. Here's a couple of snapshots of the term from the Oxford English Dictionary:

The action of examining thoroughly; investigation, scrutiny, study; an instance of this.
The action of travelling to or around an uncharted or unknown area for the purposes of discovery and gathering information; (later also) the action or activity of going to or around an unfamiliar place in order to learn about it. Also as a count noun: an instance of this; an expedition for the purpose of discovery.
When I think of explorers my mind's eye conjures visions of Victorian men with large mutton chops hacking their way through the jungle, planting flags on icy  mountain summits, and sailing merrily into the areas of the map labelled Here Be Monsters.

And that's how many people seem to see exploratory testing too. Perhaps not the facial hair, but certainly the low-tech blood-sweat-and-tears manual labour.

Unsurprisingly, that's not my perspective. What I see are people striving to achieve a goal with intent and agency, while fighting against a range of constraints and in the face of considerable uncertainties.

They use tactics such as surveying, scouting, and time-boxing. They will take measurements, observations, and samples. They will use tools in the manner intended and creatively in other ways when the need arises. 

They will exploit their surroundings by moving left, right, forward, and back, but also up trees or onto high ground to get a different view or down into the ground to understand the terrain. They will build on existing knowledge, frame experiments to create new knowledge, and collect material as they go for later consumption and absorption into their growing body of knowledge.

They will frequently pause to take stock and decide what the next target is in the wider context of their overall mission. From the crow's nest they might see a landmass and decide to try to circumnavigate it, land on it, or ignore it, whichever will serve their mission of finding a lost island, or making contact with native tribes, or locating the Northwest Passage.

They will react to what they observe, and to what the environment throws at them. They will try to square their observations with data they have picked up from elsewhere, and they'll assiduously document as they go, for their own benefit, for their sponsors' interest, and for those who will follow on behind them.

I'll grant you that, yes, those whiskery chaps in unsuitable clothing seem to be doing things the hard way. But that's because they lived 150 years ago and the cutting edge of their times looks blunt today. You can bet your best stovepipe hat that if they'd had access to Land Rovers, satellites, or global internet connectivity they'd have used them in the cause of their mission.

Familiar? It feels so to me, and I am comfortable that I don't have a definition of exploratory testing. Unless you count this:

Exploratory testing is testing by exploring.

Image: Michael Palin (from Erebus, a book about Victorian explorers my dad lent me and I'm reading at the moment)


  1. How very strange. I've just picked that same book off the shelf as my next history read.