Hiccupps

  Testing is inherently about looking because, put simply:  If you don't look, you're not likely to find. The interesting challenge is to look in the right way at the right place at the right time. This is what motivates any kind of intentional testing: how can we put ourselves in a strong position to look for, see, and — crucially —  recognise  the things that matter to the people that matter, when they matter?   --00-- Tools can help with this mission: using tools we can look more deeply, more broadly, for more complex patterns, for harder-to-spot traces, faster, more often, more efficiently, and so on.  There's a trade-off, naturally. As the tool takes us further from the material being worked on we must either trust it more or check its results more thoroughly ... to the extent that we care about the results. At a crude level, think of it as a spectrum. At one end we might have a knife. It's a tool I can use ...

  Over the years I've seen, heard, and been in many conversations about the tools testers must learn .  So many specific tools mentioned in that time, but I prefer an answer like this: Learn the tools that are important for your context. Perhaps that sounds like a cop-out, because it means testers will have to think about what they're doing, why they're doing it, the results they want to get from it, ways that they might get those results, the relative trade-offs of alternative approaches, how deep to go, the amount of time it's worth investing, when to invest it, and so on. Tough. There are easier alternatives. For example, just learn whatever tools someone on LinkedIn said that someone else on Linked said that someone else on LinkedIn said that someone else on LinkedIn said that ... But that's a merry-go-round anyone would soon want to get off.  If you were asking the question and urged me to be specific I'd point you to  My Favourite Tool .  If you read it an...

I'm continuing to be inspired by Patrick Prill's series of posts on the way AI is impacting personal, societal, and economic systems and today that leads me to share another short analogy. See also Hedging Your Bets  and A SpotifAI Model? Into the Groove by Jonathan Scott recounts the history of recorded music. You probably already have some background awareness of a timeline something like this: wax cylinders were the early commercial format, overtaken later by 78 rpm shellac discs which in turn preceded vinyl 33s and 45s, before CDs and now streaming. What you may not be aware of is the explosion of formats, players, converters, novelty features, quality innovations, pricing strategies, lawsuits, and mergers that lead ultimately to standardisation and consolidation.  Formats: the medium is one obvious difference between formats but there are others including the needle, the type of amplification, the speed of rotation, whether the needle moves left/right or up/down in the ...

You've almost certainly heard of the Prisoner's Dilemma : two members of a crimimal gang are captured and interrogated separately. The police tell them both they are looking at a year in prison but offer a deal if they squeal: less time for incriminating their colleague. There are three scenarios: No-one talks: the default one year each One talks: the talker goes free , the other gets three years Both talk: both get two years  The dilemma is whether to remain shtum and trust the other will do the same  or  to spill the beans with the risk of an extended stay in prison but the potential reward of walking away scot free.  Given that they are sure the other would be prepared to rat them out, the rational individual's strategy, according to game theory, is to talk even though there is a better group outcome in not talking. --00--   The Prisoner's Dilemma is merely a well-known example but there are many other games with different setups and constraints. I came acros...

  Hands up if you think the world needs another format for retrospectives.  Yeah, right.  Yet here I am. Last October it was clear that we were becoming jaded. After a phase of novelty themes (at the festival, NSYNC, 8-bit video games, ...) and a long stretch of Stop Start Continue , engagement in our retros was sliding. So when it was my turn to facilitate I wanted to find a different format and I had some specific requirements in mind: a simple and intuitive structure, invitations to express feelings, and opportunities to be publicly grateful.  I didn't find an existing template that fit the bill so I made one up, just four columns in a basic table with these headers: Kudos to ... 'cos of ... I like it when ... I feel meh about ... I don't like it when ...   I expected it to be a one-off palate cleanser but it's proven popular and we've been using it for six months straight.  There's no originality in those categories, by the way: I see a strong fami...

My friend Patrick Prill continues to dissect the growth, use, and risks of AI in our industry at the rate of an article or two a day, each one teasing out a particular thread from the very tangled issue ball. ( Here's yesterday's.) I recommend reading them for a thoughtful and detailed analysis and reflection on today's software and social ecosystems and the ways that AI is reshaping them.  They certainly make me think and, as with last week's Hedging Your Bets on AI , I find it helpful to capture those thoughts as analogies, so here's another. A very long time ago I wrote a music fanzine: A5, photocopied, 50p or a quid, sold by a scruffy long-hair in a German army jacket out of a battered record bag in the back rooms of pubs between bands. I remember well when domestic studio and CD-R technology became affordable because the volume of submissions for review increased dramatically ... and the overall quality likewise decreased. The technolog...

My friend and colleague Patrick Prill has been writing articles about AI in our industry at a prodigious rate over the last couple of weeks. (Here's  the second of yesterday's pieces.) I recommend reading them for a thoughtful and detailed analysis and reflection on today's software and social ecosystems and the ways that AI is reshaping them. I can't compete with the breadth, depth, or frequency of his output so, although this post is on the same topic, I offer only a short analogy.  Step back in time 60, 70 or 80 years and wander through England's pleasant pastures. You will see lanes and fields bordered with hedgerows maintained by skilled workers, living and breathing boundaries, functional for keeping stock in and intruders out but also a component of the local ecological web. Hand-laid hedges are habitats. Birds nest in them, rodents and small mammals use them like a road network to travel safely around the landscape, larger animals use them as windbreaks or f...

In 2024 I wrote a post, An AI Red Light , about a creative project I did with my daughter using both AI and "traditional" tooling. This was my conclusion: I think we were pragmatic in our choice of tools. Where our vision required control (video) we used a tool that gave us that control. Where we were happy to cede some control in exchange for output at a quality we could not produce ourselves (vocals) we used a different tool.  I didn't set out to do some kind of AI experiment. Instead I set out to have some fun with my daughter and found tools that enabled us to produce something, some art, that we're proud of and I'm happy is our work. For sure, we were sometimes more creators and sometimes more curators, but at no point were we ever mere spectators.  Two years down the road I have access to, and use, LLMs in my work. With them, I have been able to make things that I would not have done without them such as the user interface on my model-based dialog walker .  ...

There was a time when testing was all about the mnemonics . Well, we had no AI back then so thinking about our human craft and how to share what we had learned using our human intelligence with other engaged humans for later recall in their human heads seemed important. But this post isn't about dumping on AI. It isn't about mnemonics either even if WTTWTH does look like something that'd fit well into that ancient world.  No, this post is just a snappier version of what I said to my team this week when I was walking through some testing I'd done the day before. It was concerned with a change to a particular turn in the dialog system we're building where multiple variables are in play, some populated by an external call to an LLM service.  I wanted to make the point that the bulk of the testing work was done in the research I did and the spreadsheet I made, not in the interaction with our product. That spreadsheet was the result of me looking in our service's cod...