Tuesday, March 31, 2020

Meta is Better

Much of Definition of The Engineering Method by Billy Vaughn Koen chimes with what I've come to believe about testing. 

In part, I think, this is because thinkers who have influenced my thinking were themselves influenced by Koen's thoughts. In part, also, it's because some of my self-learned experience of testing is prefigured in the article. In part, finally, it's because I'm reading it through biased lenses, wanting to find positive analogy to something I care about. 

I recognise that this last one is dangerous. As Richard Feynman said in Surely You're Joking, Mr. Feynman!: "I could find a way of making up an analog with any subject ... I don’t consider such analogs meaningful.” 

This is a short series of posts which will take an aspect of Definition of The Engineering Method that I found interesting and explore why, taking care not to over-analogise.

It is Koen's contention that engineering is applied heuristics. He takes the term sota to be the set of heuristics known by an individual or a group at a specific time (see Sota so Good) and expects that an outcome will be motivated by heuristics from the sotas of the parties involved in the problem definition and solution.

Having determined that, he looks for a general rule for engineering (p. 41):
Since every specific implementation of the engineering method is completely defined by the heuristic it uses, [the quest for a rule to implement the engineering method] is reduced to finding a heuristic that will tell the individual engineer what to do and when to do it.
For those of you worrying quietly at the back, by this point Koen has acknowledged that heuristics are fallible rules of thumb. However, I'm worrying with you when wondering quite what it means for an implementation to be defined by a heuristic. My  current interpretation is something like this: "choosing and applying a relevant engineering heuristic is an instance of the engineering method (for Koen)".

I am categorically not worrying about Where he goes next, though. Just look at this (p. 42):
My Rule of Engineering is in every instance to choose the heuristic to use from what my personal sota takes to be the sota representing the best engineering practice at the time I am required to choose.

I covered "best practice" in Sota so Good so I'll ignore it here and move on to what I love about this rule:

  • it reminds the engineer to check their personal biases
  • it admits time (and so broader context) to the equation
  • it shows us that we can use heuristics to choose a suitable heuristic

Koen stretches the concept further by saying that the intersection of the sotas of all engineers across all times will contain only one heuristic (p. 42):
The Rule of Engineering is: Do what you think represents best practice at the time you must decide, and only this rule must be present.
Worriers, if you're concerned that this has a whiff of the No True Scotsman about it, then so am I: is the heuristic defined by being in the sota of all engineers, or is the set of engineers defined by their having this heuristic in their sota?

I'm prepared to put this quibble to one side, though. I find Koen's formulation beautiful. A heuristic which helps us decide which approach to take is a useful step back from any situation and implicitly admits context into its interpretation and application.

When defining testing for myself I strove to achieve that kind of generality, to pack layers of nuance into a simple principle, to provide tools for decision-making at the point of use. This is what I came up with:
Testing the is the pursuit of relevant incongruity.
I like it and find it valuable, but still have concerns that the terms I've used hinder easy interpretation by others. I have always admired how Jerry Weinberg managed turn that kind of trick so well. These three lines have been part of my daily life for years:

  • A problem is the difference between what is perceived and what is desired.
  • Quality is value to some person.
  • Things are the way they are because they got that way.

I think Koen's Rule of Engineering might be joining them.

No comments:

Post a Comment