Skip to main content

Posts

Better Isolate Than Never

How do you test a product with distributed logic and non-deterministic components? Well, it depends on the product and when the question is asked. This post is one answer, for now. A service I work on exposes an API for internal and external clients and calls multiple other services. It orchestrates a reasonably complex journey for a user, with their inputs and the responses of the services determining each step.  Our customers can configure the service to provide a variant of the standard journey for their users. This gives them choices on the available steps, the order of the steps, and, importantly, some of the logic for taking particular steps. This configuration lives in a different repository to the service, not owned by us, and it can be changed and deployed independently from the service. As you can imagine, we test our service in multiple ways at different granularities using the usual tooling, including unit tests fo...
Read more
Recent posts

Just Looking

In Looking Good, Testers!  I made the case that testing is about looking and that delegating the looking to someone or something  else involves a risk calculation: do you both understand the task the same way, is the third party likely to carry out the task in the way you requested, how easily can you check any results they give back, could they complement you in some way, how will you keep your knowledge and skills up to date if you don't do this task any more, how much does this task matter, how much do you care about any of that, ...? I didn't say anything there about selecting what to look at or for  ... and I'm not going to talk about it here either because it's essentially the whole of testing. I will note one thing, though: to find things other people don't find,  look where other people don't look .  What I do want to cover here is looking at things that, arguably, you don't need to.  My team was recently asked to add a new layer to a service we ow...
Read more

Looking Good, Testers!

  Testing is inherently about looking because, put simply:  If you don't look, you're not likely to find. The interesting challenge is to look in the right way at the right place at the right time. This is what motivates any kind of intentional testing: how can we put ourselves in a strong position to look for, see, and — crucially —  recognise  the things that matter to the people that matter, when they matter?   --00-- Tools can help with this mission: using tools we can look more deeply, more broadly, for more complex patterns, for harder-to-spot traces, faster, more often, more efficiently, and so on.  There's a trade-off, naturally. As the tool takes us further from the material being worked on we must either trust it more or check its results more thoroughly ... to the extent that we care about the results. At a crude level, think of it as a spectrum. At one end we might have a knife. It's a tool I can use ...
Read more

What Tools to Learn for Testing?

  Over the years I've seen, heard, and been in many conversations about the tools testers must learn .  So many specific tools mentioned in that time, but I prefer an answer like this: Learn the tools that are important for your context. Perhaps that sounds like a cop-out, because it means testers will have to think about what they're doing, why they're doing it, the results they want to get from it, ways that they might get those results, the relative trade-offs of alternative approaches, how deep to go, the amount of time it's worth investing, when to invest it, and so on. Tough. There are easier alternatives. For example, just learn whatever tools someone on LinkedIn said that someone else on Linked said that someone else on LinkedIn said that someone else on LinkedIn said that ... But that's a merry-go-round anyone would soon want to get off.  If you were asking the question and urged me to be specific I'd point you to  My Favourite Tool .  If you read it an...
Read more

The Needle and the Damage Done

I'm continuing to be inspired by Patrick Prill's series of posts on the way AI is impacting personal, societal, and economic systems and today that leads me to share another short analogy. See also Hedging Your Bets  and A SpotifAI Model? Into the Groove by Jonathan Scott recounts the history of recorded music. You probably already have some background awareness of a timeline something like this: wax cylinders were the early commercial format, overtaken later by 78 rpm shellac discs which in turn preceded vinyl 33s and 45s, before CDs and now streaming. What you may not be aware of is the explosion of formats, players, converters, novelty features, quality innovations, pricing strategies, lawsuits, and mergers that lead ultimately to standardisation and consolidation.  Formats: the medium is one obvious difference between formats but there are others including the needle, the type of amplification, the speed of rotation, whether the needle moves left/right or up/down in the ...
Read more

Isn't It Obvious?

You've almost certainly heard of the Prisoner's Dilemma : two members of a crimimal gang are captured and interrogated separately. The police tell them both they are looking at a year in prison but offer a deal if they squeal: less time for incriminating their colleague. There are three scenarios: No-one talks: the default one year each One talks: the talker goes free , the other gets three years Both talk: both get two years  The dilemma is whether to remain shtum and trust the other will do the same  or  to spill the beans with the risk of an extended stay in prison but the potential reward of walking away scot free.  Given that they are sure the other would be prepared to rat them out, the rational individual's strategy, according to game theory, is to talk even though there is a better group outcome in not talking. --00--   The Prisoner's Dilemma is merely a well-known example but there are many other games with different setups and constraints. I came acros...
Read more

It's Called Gratitude

  Hands up if you think the world needs another format for retrospectives.  Yeah, right.  Yet here I am. Last October it was clear that we were becoming jaded. After a phase of novelty themes (at the festival, NSYNC, 8-bit video games, ...) and a long stretch of Stop Start Continue , engagement in our retros was sliding. So when it was my turn to facilitate I wanted to find a different format and I had some specific requirements in mind: a simple and intuitive structure, invitations to express feelings, and opportunities to be publicly grateful.  I didn't find an existing template that fit the bill so I made one up, just four columns in a basic table with these headers: Kudos to ... 'cos of ... I like it when ... I feel meh about ... I don't like it when ...   I expected it to be a one-off palate cleanser but it's proven popular and we've been using it for six months straight.  There's no originality in those categories, by the way: I see a strong fami...
Read more

A SpotifAI Model?

My friend Patrick Prill continues to dissect the growth, use, and risks of AI in our industry at the rate of an article or two a day, each one teasing out a particular thread from the very tangled issue ball. ( Here's yesterday's.) I recommend reading them for a thoughtful and detailed analysis and reflection on today's software and social ecosystems and the ways that AI is reshaping them.  They certainly make me think and, as with last week's Hedging Your Bets on AI , I find it helpful to capture those thoughts as analogies, so here's another. A very long time ago I wrote a music fanzine: A5, photocopied, 50p or a quid, sold by a scruffy long-hair in a German army jacket out of a battered record bag in the back rooms of pubs between bands. I remember well when domestic studio and CD-R technology became affordable because the volume of submissions for review increased dramatically ... and the overall quality likewise decreased. The technolog...
Read more

Hedging Your Bets on AI

My friend and colleague Patrick Prill has been writing articles about AI in our industry at a prodigious rate over the last couple of weeks. (Here's  the second of yesterday's pieces.) I recommend reading them for a thoughtful and detailed analysis and reflection on today's software and social ecosystems and the ways that AI is reshaping them. I can't compete with the breadth, depth, or frequency of his output so, although this post is on the same topic, I offer only a short analogy.  Step back in time 60, 70 or 80 years and wander through England's pleasant pastures. You will see lanes and fields bordered with hedgerows maintained by skilled workers, living and breathing boundaries, functional for keeping stock in and intruders out but also a component of the local ecological web. Hand-laid hedges are habitats. Birds nest in them, rodents and small mammals use them like a road network to travel safely around the landscape, larger animals use them as windbreaks or f...
Read more

Create, Curate, or Spectate?

In 2024 I wrote a post, An AI Red Light , about a creative project I did with my daughter using both AI and "traditional" tooling. This was my conclusion: I think we were pragmatic in our choice of tools. Where our vision required control (video) we used a tool that gave us that control. Where we were happy to cede some control in exchange for output at a quality we could not produce ourselves (vocals) we used a different tool.  I didn't set out to do some kind of AI experiment. Instead I set out to have some fun with my daughter and found tools that enabled us to produce something, some art, that we're proud of and I'm happy is our work. For sure, we were sometimes more creators and sometimes more curators, but at no point were we ever mere spectators.  Two years down the road I have access to, and use, LLMs in my work. With them, I have been able to make things that I would not have done without them such as the user interface on my model-based dialog walker .  ...
Read more