Hiccupps

As a tester, comparison, and confidence in your ability to compare, are key parts of your toolkit. So, like me, you might find this video humbling. It talks about metamers (where different things look the same) and anti-metamers (the same things look different) and shows how easy it is to mislead our visual systems even while explaining how the images were created and why the optical illusion works. Video: https://www.youtube.com/watch?v=tQ9oUfyEc1k

Managing a complex set of variables, of variables that interact, of interactions that are interdependent, of interdependencies that are opaque, of opacities that are ... well, you get the idea. That can be hard. And that's just the job some days. Investigating an apparent performance issue recently, I had variables including platform, product version, build type (debug vs release), compiler options, hardware, machine configuration, data sources and more. I was working with both our Dev and Ops teams to determine which of these seemed most likely in what combination to be able to explain the observations I'd made. Up to my neck in a potential combinatorial explosion, it occurred to me that in order to proceed I was adopting an approach similar to the ideas behind chart parsing in linguistics. Essentially: keep track of all findings to date, but don't necessarily commit to them (yet) maintain multiple potentially contradictory analyses (hopefully efficiently) pack ...

The Cambridge Tester Meetup  last night was a discussion on testing standards. Not on the specific question of ISO 29119  (for which see Huib Schoots' excellent resource ) but more generally on the possibility of there being a standard at all. It was structured along the lines of Lean Coffee with thoughts and questions being thrown down on post-its and then grouped together for brief discussion. I've recorded the content of the stickies here with just a little post-hoc editing to remove some duplication or occasionally disambiguate. The titles were just handles to attach stickies to once we had a few in an area and I haven't tried to rationalise them or rearrange the content. Enhance/Inhibit Testing Testing is a creative process so can't be standardised. Testing doesn't fit into a standard format, so how can there be a standard for it? (Do we mean "good testing" whatever that is?) New tools, technology might not fit into a standard. Standardis...

You can find Weinberg's famous Rule of Three in a variety of formulations. Here's a couple that showed up when I went looking just now ( 1 , 2 ): If you can't think of at least three things that might be wrong with your understanding of the problem, you don't understand the problem.  If I can’t think of at least three different interpretations of what I received, I haven’t thought enough about what it might mean.  At work I challenge  myself to come up with ideas in threes and, to keep that facility oiled , when I'm not testing I challenge myself to turn the crap joke I've just thought of into a triad.  By providing constraints on the problem I find the intellectual joking challenge usefully similar to the intellectual testing challenge. Here's an example from last week where, after I happened onto the first one, I constrained the structure of the others to be the same and restricted the subject matter to animals: If I had three pet lions I wo...

Most working days I go for a walk round the block after I've eaten my dinner. As I leave our office I've got two primary choices: lift or stairs. I say primary because there's obviously many things I could do at that point (although I have promised my wife that naked star jumps will not feature in my daily exercise regimen ... again). In any case I go for the stairs. At the bottom of the stairs I have two more choices: left or right. Each takes me to a different exit from the building but both open onto the circuit that I stroll round, and if I leave by one of them I will naturally arrive back at the other so there's (again, to the level of granularity that I care about) no significant difference between them. I can't go straight on at the bottom of the stairs because the lift is there and a u-turn sends me back into work so every day I am forced to make the choice - left or right.  And every day until recently I've been making that choice without any con...

Early yesterday morning I was setting up a bunch of scripts to run a bunch of requests against a bunch of servers in a bunch of permutations over the course of a few days. Bitter experience has taught me that this kind of thing rarely works right the first time so if I can, I find a way to start with a small version of the setup.  Here, that meant choosing a sensible subset of the requests, letting things run for a few minutes, keeping an eye on progress (files created in the right place, servers generating the kind of messages, execution times reasonable, CPU load and number of processes in the right ballpark and so on) and then inspecting logs at the end. As I worked the bugs out of the configuration I had a little time to kill on each iteration. But what to do? I'd stumbled across a couple of haiku I'd enjoyed just the day before and I'd been pondering something  Chris George  had said about factory testers during  Cambridge Lean Coffee  last w...

So the other day I bleeted  about how I like to write to help me collect my thoughts  and how that feels like a dialogue through the page. Somewhat ironically, you might think, I hadn't intended that action to be more than jotting down the realisation I'd just had.  But, of course, as soon as it was out there I began to challenge it, and by proxy myself. Here's a sample: "When I need to think through an issue, I write."  Really? Always? Does getting the ideas down free mental resource for inspection of the ideas?  Does making it concrete mean that it's easier to spot inconsistency? I know people who are adept at maintaining multiple views of a thing. When a different angle of attack is used a different kind of defence is made. The defences are not compatible, but because they are never seen together, this can be overlooked. Why didn't I talk about pictures? I draw a lot too. I recalled that James Lyndsay mentioned the other day that he makes ...

A trend is better than a snapshot, right? That's Joel Montvelisky, introducing the State of Testing Survey 2015 . I'm certainly in  favour of data  and I'd agree that a trend can be better than a snapshot. But if you want to know the state of some system right now for the investigation you're performing right now and you've no reason to think that right now is related to back then, then perhaps right now you'll take the snapshot, right? Openness and openness to challenge was one of the things I liked most about the previous, inaugural, survey. In the discussion between Jerry Weinberg and Fiona Charles about the results ( transcript here ) Weinberg's opening remarks include: We need to be careful on how we interpret this data [...] One way to look at the survey is that it’s giving information about what information we should be getting.  I'm looking forward to seeing what was learned. Image:  https://flic.kr/p/oq5E3x

When I need to think through an issue, I write. And when I do that I feel I'm having a dialogue with myself. I write. I challenge. I rewrite. I re-challenge. Within or across drafts. Dynamically or with reflection. At length or fleetingly. As a means to an end, or as an end in itself. It both clarifies and exposes the need for clarification. For me. When I asked on Twitter  I got a couple of useful references to similar things: The Feynman Technique Expressive Writing I'd be very interested in any others. Edit: I followed up on this post  later. Image:  https://flic.kr/p/5UWSs9

Linguamatics hosted James Lyndsay at the Cambridge Tester Meetup  last night. His workshop started with some improv games based on the work of Keith Johnstone  which, by exposing our awkwardness, showed us that we were conditioned to behave in certain ways, that we have patterns of operation. As testers we want to be free to think, investigate, explore, perform. A second round of exercises had us giving and receiving imaginary presents to illustrate the notions of offers and blocking . Here, one person creates a context for the other (the offer) which can be accepted or rejected, but both parties must be aware of the ways in which they might constrain that context (the block). For example, I might mime the shape of something to pass to my partner and then, as their hands reach for it, change the shape. This constitutes a block - I am not collaborating as fully as I might. Blocks come in many varieties; the receiver may block by not accepting the gift or refut...