Developer! Developer! Developer! Tester!

Last weekend, one of the testers from my team was speaking at Developer! Developer! Developer! East Anglia, a .NET community event. Naturally, because I'm a caring and supportive manager — and also as it was in Cambridge, and free — I went down to have a look. Despite not being a developer, it wasn't hard to find something of interest in all five sessions, although it's a shame the two talks on testing were scheduled against each other. Here's thumbnails from my notes.

Building a better Web API architecture using CQRS (Joseph Woodward): Command Query Responsibility Segregation is a design pattern that promotes the separation of responsibility for reading data (query) from writing data (command). A useful intro to some concepts and terminology for me, it went deeper into the code than I generally need, and in a language and libraries that I'm unfamiliar with. I found Martin Fowler's higher-level description from 2011 more consumable.

I do like listening to practitioners talk about topics they care about, though. Particularly enjoyable here was the shared relief in the room when it became apparent that many of the attendees, despite being advocates of CQRS, found that they all violate a core principle (commands don't return data) in favour of practicality from time to time (e.g. when creating users and returning a unique ID).

Client-side web performance for back-end developers (Bart Read): Chrome's developer tools got a massive big-upping, particularly the memory monitoring and task manager. Bart provided an interesting list of heuristics for improving user experience on the client side which included: load only enough to show the first thing that that the user needs to see, do the rest lazily; inline what you can for that first load; if you can fit it into a single packet even better because that reduces the cost of latency; It's the latency, stupid; load all adverts last, really last, totally the last thing that you do on a page, honestly never do anything with adverts until you've done every other thing you have to do.

Visual note-taking workshop (Ian Johnson): I've thought a lot about my own note-taking over the years and I know that it's heavy on text. I'm very comfortable with that, but I like drawing and I'm interested in trying sketchnoting to see whether going out of my comfort zone can give me another perspective or perhaps technique to roll into my usual approach.

This talk was a primer: some basic iconography, some suggestions for placement (corners for metadata such as dates, speaker name, conference); thoughts on prominence (bold, colours, boxes, underlines, ...); reminders that sketch notes are not about realism; exhortations to just go for it and not worry if it doesn't work out; and this rule of thumb for ordering noting activity: content then boxes then colours. (Related material from Ian is here.)

Testing Demystified (Karo Stoltzenburg): Karo's talk is the reason I was at the conference but she's written about it already in Three Ways to get Started with (Exploratory) Testing as a non-Tester so I won't say more. I will, however, mention that I took the opportunity to practice my new-found sketchnoting skills in her talk. As expected, I found it hard to resist writing a lot of text.

Monitoring-First Development (Benji Weber): Unruly are an XP shop applying XP development practices in a wider context. In the product they'll write a failing test then code to make it pass, and in production they'll write a failing monitor (such as checking for equivalence between two data items using a tool such as Nagios) and then implement whatever functionality provides the data for the monitor. A neat idea, and it works in their context. (Similar content here in an earlier lightning talk by Benji.)

I was really impressed with DDD: 300 attendees, friendly atmosphere, just enough organisation, free, and good spread of talks.
Image: DDD!

Forget It

Now where was I?

Oh yes, The Organized Mind by erm ... hold on a minute ... it's on the tip of my tongue ... err ... ummm ... there you go: The Organized Mind by Daniel Levitin, a self-subtitled guide to thinking straight in the age of information overload. And surely we all need a bit of that, eh?

One of the most productive ways to get your mind organised, according to Levitin, is to stop trying to organise your mind (p. 35):

The most fundamental principle of the organized mind, the one most critical to keeping us from forgetting or losing things, is to shift the burden of organizing from our brains to the external world ... This is not because of the limited capacity of our brains — rather it's because of the nature of memory storage and retrieval in our brains.

Essentially, memory is unreliable. There are numerous reasons for this, including: novelty being prioritised over familiarity, successful recall being reliant on having a suitable cue, and — somewhat scarily — that the act of remembering can itself cause memories to change.

To get around this, Levitin favours lodging information you'll need later somewhere outside of your head, in the place that you need it, in a form that'll help you to use it straight away. He likens this to affordances as described by Don Norman in his book The Design of Everyday Things which I blogged about in Can You Afford Me?  From there:

an affordance is the possibility of an action that something provides, and that is perceived by the user of that thing. An affordance of a chair is that you can stand on it. The chair affords (in some sense is for) supporting, and standing on it utilises that support.

Failing to externalise can lead to competition for mental resources when a new primary task comes along but in the background your mind is rehashing earlier ones (p. 68):

... those thoughts will churn around in your brain until you deal with them somehow. Writing them down gets them out of your head, clearing your brain of the the clutter that is interfering with being able to focus on what you want to focus on.

Perhaps you're worried that too much organisation inhibits creativity? Quite the opposite, Levitin claims (p. 87):

Finding things without rummaging saves mental energy for more important and creative tasks. 

Which brings me to testing and a conversation I was having with one my team a couple of weeks ago. In it, we agreed that experience has taught us to prefer earlier rather than later organisation of our test notes, the data we're collecting, and our ideas for other possible tests.

I've also written at some length about how I deposit thoughts in files, arranged in folders for particular purposes such as 1-1 meetings, testing experiments, or essay ideas where they may later be of value (e.g. Taking Note and A Field of My Stone).

Even posts here on Hiccupps serve a related purpose. I find that I don't easily remember stuff, but curating material that I find interesting and writing about it, and adding tags, and cross-referencing to other thoughts helps to me to retain and reinforce and later recall it.

Try to keep everything in my head? Forget it.
Image: Amazon

Bog Standard, A Parable

There was once a young man, a handsome, moral, and brave young man, dexterous beyond compare, sharp of eye, voracious in the consumption of information and with sagacity and recall to rival that of the wisest of elephants. Oh yes, and he was also a software tester.

This preternaturally blessed young man, over the course of time, had occasion to visit many conveniences, both public and private. As was his way, he declined to waste those periods of forced repose and so took the opportunity to exercise and practice the skills that served him so well elsewhere in his life while sequestered in those littlest of rooms.

To this end, over repeated visits to a particular water closet he began to observe that, while the cleanliness in general could not be faulted, there was one area which was reliably hygienic to a significantly lower standard than the rest. This region, populated by dust, tiny fragments of tissue, hair, and other detritus, he noted, was along the base of the wall directly in the eyeline of one seated on the porcelain throne.

"How could the cleaners of this thunderbox, clearly evidenced to be otherwise assiduous in their duties, have regularly missed this blatant, glaring filth", he wondered?

As he reflected on this question, he studied the topography of the chamber. From his vantage point, seated with his back to one of the short edges of the rectangular room, the door was to his left with handle closest to him and hinges on the far side. On opening, the door would swing inwards and away from him, to rest against the far wall, the one whose junction with the floor had so captured his attention. This wall was only slightly wider than the door itself and, from his perch, he guessed he would need merely a second arm's length to be able to touch it, making this a small latrine, but not unusually so.

He began to form a theory concerning a toilet provider employing a toilet attendant, tasked with making the toilet experience acceptable to the toilet users, but operating within constraints on time, materials, the physical layout and so on. Equipped with mop, toilet brush, vacuum cleaner, duster and detergent, on pushing open the door, the attendant would squeeze into the confined space with elbows around their ears to clean and tidy before backing awkwardly out again.

In spite of any good intentions for this cubicle in particular and high standard of cleanliness elsewhere in the facility, it was still clearly possible for the attendant to miss something significant and negative and obvious to users within moments of their first experience of the service. In this scenario, the door would never — could never — have closed, and the attendant never have had either cause nor indeed opportunity to adopt the position, pose and perspective of the crapper's patrons.

"This is an exciting thought!" he thought excitedly, but sadly it was lost for ever when a new project with terrifyingly high priority came crashing onto his to do list: someone had used the last piece of toilet paper ...
Image: https://flic.kr/p/8mEGWT

Quite the Reverse

Cohen and Medley, in Stop Working & Start Thinking, say:

Simple tests are not experiments ... A chef will bake a cake at different temperatures and find the one that gives the best results ... [but] we should only include [this test] in classical "science" if the "normal" situation is included as a control ... Every careful observation of a puzzling or new phenomenon should be matched to similar observations of well-understood or classical material.

They go on to introduce some useful terminology: variables are the things that you will aim to alter in the experiment; all other factors that could vary, but which you will aim not to vary, are parameters.

And they then describe three types of experiment concerned with investigating the possibility of a causal relationship between a variable, A, and an outcome, X.

Deficit: run one experiment with A and one without A. Monitor the presence of X in both cases. If X is seen with A but not without A then perhaps we have some evidence that lack of A inhibits X.

Result reversal: run a single experiment during which A is first removed and then reintroduced, holding all parameters constant, If X disappears when A is removed and reappears when A is brought back, then there is stronger evidence of a causal relationship between A and X.

Demi-reversal: when it's not possible to control the parameters in the experimental context  — for example in real world ecological investigations — then introducing A into an environment which is already understood and looking for changes in X can be indicative of a causal relationship too.

Result reversal is convincing, they say, because "there are so many more ways to lose function than there are to regain it. You can only fix your car by repairing the fault, but your car can fail for a billion and one reasons."

Once you have a hypothesis, its value will be dependent upon its power of prediction of the outcomes in new experiments. When you can't control the parameters enough to run new experiments, you may be able to use retrodiction, where historical data is analysed to see whether the hypothesis can be seen to have held in relevant situations in the past.

It's sometimes dense and dry, and it gives deeper-than-is-helpful-to-a-non-scientist detail about some of the experiments described. Despite this, I really liked Stop Working & Start Thinking as a basic guide to experimental practice, and a reminder to step back from the coal face, written by seasoned practitioners. I pulled out a couple of other aspects that I found particularly enjoyable in A Different Class and Ignorance, Recognised.
Image: https://flic.kr/p/5bLjxD

A Different Class

In Stop Working & Start Thinking (which I also mentioned the other day)  Jack Cohen and Graham Medley want scientists to consider what science is and how they do it, as well as just getting on with it. To help explain this, they partition scientific answer-seeking like so:

• observation
• measurement
• investigation
• experiment

And that's interesting in and of itself.  But the authors have been round the block and so recognise that this categorisation is not absolute, and that sometimes it might not be clear where a particular activity sits, and that some activities probably sit in multiple categories at different times and even at the same time.

In science — and thinking, so this applies to you too, testers — generalisations are useful because they help us to frame hypotheses at relevant granularities. We’re all made up of atoms but a description of social deprivation in inner cities at an atomic level would unhelpfully obscure, for example, that higher-level concepts such as social class can be useful factors in such an analysis. Further, this can be true despite social class itself being a fluid notion.

But generalisations are problematic for scientists — and thinkers, which includes us testers. It’s easy to be lulled into a false sense of security when, for example, all known observations show one general thing (swans are white) but then a new observation doesn’t fit (it looks just like a swan, but it’s black). There’s a human tendency to want to avoid complicating a simple model, and so reject the data (that’s not a swan!) which doesn't improve the theory.

Scientists, Cohen and Medley say — and I'd add testers also — should retain a critical distance when classifying: what does it mean to be a black swan? Would, say, a white stripe under one wing affect this? Could there be shades of black? Could there be greys? What explanatory power does a given classification permit? What does it prevent?

Further, they add, involving tools in categorisation, or more generally in any measurement, requires another consideration:

... when measurement is automated, the final figure, the one you get from the machine, incorporates the prejudice of the designer, not yours!

Class.
Image: https://flic.kr/p/pEuUf8

Edit: I wrote another post from this book too: Quite the Reverse.  

See You Triangulater

Perhaps it's true that there's nothing new under the sun, but that doesn't mean that what's already known is necessarily uninteresting. Here's a quick example: I was recently reflecting on how talking to multiple people about their perspectives, finding data from several independent sources, or asking the same question in different ways felt analogous to a technique from surveying, triangulation.

Triangulation is an ancient but still widespread method of mapping a landscape in which a network of points are plotted in relationship to one another, with each point always connected to two others, making triangles. Building one triangle against the next, and the next, and the next allows the whole space under consideration to be covered.

I'm nowhere near the first here, though, as a quick search established:

In the social sciences, triangle is often used to indicate that two (or more) methods are used in a study in order to check the results of one and the same subject ... By combining multiple observers, theories, methods, and empirical materials, researchers can hope to overcome the weakness or intrinsic biases and the problems that come from single method, single-observer and single-theory studies.

So what's my point? Testing frequently involves collations, connections, and comparisons. Triangulation is an interesting model of those activities to consider, for me, right now, even if there's likely no solar system in which it's a novel one.
Image: https://flic.kr/p/5p4mBt

On Mapping Non-testable Papers

The Test team book club at Linguamatics read On Testing Non-testable Programs by Elaine Wyuker this week. As usual, the discussion was interesting and, as usual, the reading material was only the starting point and, as usual, we found ourselves exploring our own context and sharing our own experiences and ideas.

I find this kind of interaction invigorating and energising. It remains fascinating to me that we each bring common and unique perspectives to these meetings and I thrive on hearing others on my team talk about how they see the topic, I covet the time I spend thinking about how I do, and then I enjoy immensely contrasting the two.

I had wondered, while reading the paper, whether I could extract some kind of ontology of oracles from it. Informally, it seemed that Weyuker had structured her analysis in this way: programs are testable or not; these are characteristics of non-testable programs; non-testable programs are of three types; these approaches to oracles can be used with such and such a type, ...

So, to explore that notion, I went back to the paper and gave myself an hour to re-read it, watch a short video by Cem Kaner which references it, and make a mind map. (I'm also interested in experimenting with getting value from mind maps.)

Unsurprisingly, you might say, when studying the paper more closely, with a particular purpose in mind, I found that my informal analysis was a simplistic analysis. In order to map the details of the structure I was interested in, I had to think harder than when I was merely absorbing the higher-level points.

To give one example, Weyuker breaks non-testable programs into three types early in the paper:

It is interesting to attempt to identify classes of programs which are non-testable. These include: (1) programs which were written to determine the answer. If the correct answer were known, there would have been no need to write the program; (2) programs which produce so much output that it is impractical to verify all of it; (3) programs for which the tester has a misconception.

Then, later, in a section about approaches for dealing with two of those types, she says

For those programs deemed non-testable due to a lack of knowledge of the correct answer ... In the case of programs which produce excessive amount of output ... One other class of non-testable programs deserves mention. These are programs for which not only an oracle is lacking, but it is not even possible to determine the plausibility of the output.

Is this third class the same as the original third class? Or different? (And, as a writer, what can I learn from that?)

Here's the map I produced in my timebox:

Image: https://flic.kr/p/rHrgSt