Hiccupps

  Testing is inherently about looking because, put simply:  If you don't look, you're not likely to find. The interesting challenge is to look in the right way at the right place at the right time. This is what motivates any kind of intentional testing: how can we put ourselves in a strong position to look for, see, and — crucially —  recognise  the things that matter to the people that matter, when they matter?   --00-- Tools can help with this mission: using tools we can look more deeply, more broadly, for more complex patterns, for harder-to-spot traces, faster, more often, more efficiently, and so on.  There's a trade-off, naturally. As the tool takes us further from the material being worked on we must either trust it more or check its results more thoroughly ... to the extent that we care about the results. At a crude level, think of it as a spectrum. At one end we might have a knife. It's a tool I can use ...

  Over the years I've seen, heard, and been in many conversations about the tools testers must learn .  So many specific tools mentioned in that time, but I prefer an answer like this: Learn the tools that are important for your context. Perhaps that sounds like a cop-out, because it means testers will have to think about what they're doing, why they're doing it, the results they want to get from it, ways that they might get those results, the relative trade-offs of alternative approaches, how deep to go, the amount of time it's worth investing, when to invest it, and so on. Tough. There are easier alternatives. For example, just learn whatever tools someone on LinkedIn said that someone else on Linked said that someone else on LinkedIn said that someone else on LinkedIn said that ... But that's a merry-go-round anyone would soon want to get off.  If you were asking the question and urged me to be specific I'd point you to  My Favourite Tool .  If you read it an...

  My friend Mirek wrote an interesting post recently: Where Rust fits for me . In it, he made a hierarchy of the programming languages he reaches for on a regular basis and why he picks a particular one for any specific task. I'd summarise it crudely like this: Shell: only for very basic setup. Anything with non-trivial logic belongs somewhere else. Python: his go-to but with caveats about the expected lifetime of the code: if it needs to persist without regular attention then Python won't do. Rust: Whatever doesn't fit in the other two categories.  I enjoy seeing people be thoughtful about their tools and reflect on the way that they work. Understanding what you do and why is a helpful first step on improving what you do and why, should you want to. And I always want to. So I thought I'd attempt the same kind of analysis and started listing languages that I use regularly: Groovy : I use Groovy exclusively in Jenkins pipelines. It's useful to have a little familiar...

  A few months ago I stopped having therapy because I felt I had stabilised myself enough to navigate life without it. For the time being, anyway.  I'm sure the counselling helped me but I couldn't tell you how and I've chosen not to look deeply into it. For someone who is usually pretty analytical this is perhaps an interesting decision but I knew that I didn't want to be second-guessing my counsellor, Sue, or mentally cross-referencing stuff that I'd researched while we were talking. And talk was what we mostly did, with Sue suggesting hardly any specific tools for me to try. One that she did recommend was finding a happy place to visualise, somewhere that I could be out of the moment for a moment to calm disruptive thoughts. (Something like this .) Surprisingly, I found that I couldn't conjure anywhere up inside my head. That's when I realised that I've always had difficulty seeing with my mind's eye but never called it out. If I try to imagine ev...

For over a year the roadworks near our house have been a riot of signage, inspiring me and my daughter to make silly songs using their words for lyrics as we walked to school.  Then she got the idea that we should make a "proper" song. So I downloaded n-Track and we hacked together a techno instrumental over a few evenings. Unfortunately, recording decent quality vocals at home without much equipment is non-trvial and then real life intervened anyway so the project stopped. A few months later I came aross Suno , an AI song generator, and had a lightbulb moment. Suno exposes a prompt for musical style and a prompt for lyrics so we had it make a song, When Red Light Shows , based on the signs. I think it was my daughter that suggested we should make a video too, so we did, with a non-AI tool, CapCut . The lyrics came from signs, the music came from AI, and we made the images yet I feel that we were creative across all of those areas. Reflecting on our experience, ...

This week I was working with a developer to make a change to a legacy codebase in an area neither of us are very familiar with. The need is easy to describe in general and some occurrences of the behaviour we want to alter are common, straightforward to identify in use, and clear in the code. Unfortunately, the logic in the application is complex, the data used is domain-specialised and the behaviour we are interested in can occur in extremely specific combinations that are hard for a layperson to predict. I had no confidence that cases we knew were all of the cases. My colleague did a round of work and asked me to take a look. I exercised the application in a few ways while inspecting its logs in real time so that I could see the effect of the changes (or not) immediately. This gave up some rarer examples which had not been covered in the code. I added a couple of tests to characterise them and he identified another code change. Next, rather than continue exercising the product lookin...

A couple of weeks ago I was wondering what testing patina might look like .  What do I mean by patina in this context? I think I'm looking for artefacts and side-effects of work, visible on tools and places of work, that demonstrate something about the length of time, depth, and breadth of work, and ways of working.  I'm seeking things that other practitioners could recognise and appreciate as evidence of that work. But, and this is important, the patina is not the work itself. So here's the list of things I've come up with so far: Patina might be visible in an IDE I've been using for a long time through a litter of plug-ins, some for defunct tooling, or obsolete languages, with multiple plug-ins for the same file format, and so on. Patina might be visible at work from my Confluence home page where I collect links to the internal talks and demos I've done. (Top-right in the image at the top, and deliberately obfuscated I'm afraid.) Patina might be visible in...

The product I'm working on at the moment is a Ada, a symptom checking app . The basic idea is that users enter a few details about themselves and their current symptoms and are then guided through a series of questions which leads to them being given a list of probability-ranked conditions they might have based on their answers. As you'd expect in this field, with this kind of data, a lot of care and concern is taken to understand how the app performs and what caused any changes in that performance across releases. There are many layers of testing. In one of those layers we have medical test cases. (And these are literally cases in the medical sense.) Each one represents data about an individual who might present to a doctor with a particular set of symptoms, a given medical history, possible comorbidities and so on. Each also comes with a set of acceptable condition suggestions and other expectations about how the software should behave when asking this kind of user questions...

This week I was testing a new piece of functionality that will sanitise query parameters on an API endpoint. The developer had implemented a first cut and I was looking at the code and the unit tests for a few minutes before we got together for a walkthrough. After exercising the new function through the unit tests, trying other values, and checking that the tests could fail, I wanted to send some requests from an external client. I already had the service set up in IntelliJ and running under the debugger, so ... Sending requests from Postman and inspecting sanitised values in a breakpoint was OK, but there was too much  jumping back and forth between applications, clicking on fields, scrolling and so on. Each iteration was many seconds long. Luckily, we'd already  decided to log sanitised strings for monitoring and later review,  so I could disable the breakpoints and just look at the service console. That was better but still too heavy fo...

And this is how it goes. One thing to another ... At the weekend I  was listening to Gene Kim's Idealcast interviews with Dr. Nicole Forsgren and Jez Humble . Jez Humble was talking about the importance of rapid feedback and referenced a  Brett Victor  talk that had stuck with him over the years: ... he built this little JavaScript game and he was changing parameters and the game was changing in real time. And his whole comment is like, if you can change something and see the change as you are changing it, it's incredibly powerful.   And so I looked it up in the show notes and watched it. Wow ... Inventing in Principle shows examples of experimental tooling for creative activities, particularly those that include a temporal dimension. The essential idea is to reduce the friction of having to maintain a model outside of the tool. In the image at the top, the left side is a traditional IDE and the right side is a dynamic visualisation of the function being develo...

Last week I did a presentation to a software testing course at EC Utbildning in Sweden titled Exploring with Automation where I demoed ways in which I use software tools to help me to test. Following up later, one of the students asked whether I had a favourite tool. A favourite tool? Wow, so simple but sooo deep!  Asking for a favourite tool could make a great interview question, to understand the breadth and depth of a candidate's knowledge about tools, how they think about an apparently basic request with deep complexity beneath (favourite for what task, on what basis, in what contexts, over what timescale?  what is a tool anyway?) and how they formulate a response to take all of that into account. I could truthfully but unhelpfully answer this question with a curt Yes or No. Or I could try and give something more nuanced. I went for the latter. At an extremely meta level I would echo Jerry Weinberg in Perfect Software : The number o...

  I am a believer in open-notebook testing . I make my work visible to anyone who wants to look at it, while it is in progress. Why? Well, I dislike information silos, publishing keeps me thoughtful about my work and my standards high, and sometimes someone will spot something I've missed or mistaken. But I also want my testing to be friction-free. In this context that has two aspects: I need to be able to (a) record and (b) share what I'm doing with as little impact on my work as I can manage. I've written before about the way I take notes in a text editor using a simple markup language. In my previous job I ran a little script on my testing notes and pasted the output straight into the Mediawiki instance we used. Unfortunately, in my new job we use Confluence. Also unfortunately, I found that support for even its own markup language was unreliable and so I had to find a new route. What I've iterated my way to over the last four months is, agai...

When I changed jobs recently, joining a team responsible for a middleware server, I expected that much of my work would be against its API. This has proved correct, although I've been able to test at all layers in our stack, from the external front-end applications down to the very back-end tooling used by our domain experts to generate data for our application, and that data itself. Before I joined, I knew that the server was written in Kotlin and that the team use IntelliJ for development. As I've only scripted in a text editor in recent years I invested some time in getting some beginner-level skills in the new language and IDE with the help of YouTube videos and exercism.io .  I won't tell you that I didn't curse while doing it. Because I did, a lot. Why? For example, because of the new key bindings for the text editor, a new and complex environment wrapped around that editor, tons of helpful functionality with a high discovery cost and a steep learning curve, a ne...

Last week I started a new job. The team I've joined owns a back-end service and, along with all the usual onboarding process, inevitable IT hassles, and necessary context-gathering, one of my goals for my first week was to get a local instance of it running and explore the API. Which I did. Getting the service running was mostly about ensuring the right tools and dependencies were available on my machine. Fortunately the team has wiki checklists for that stuff, and my colleagues were extremely helpful when something was missing, out of date, or needed an extra configuration tweak. Starting to explore the service was boosted by having ReDoc for the endpoints and a Postman collection of example requests against them. I was able to send requests, inspect responses, compare both to the doc, and then make adjustments to see what effects they had. If that's testing of any kind, it's probably what I call pathetic testing : There's this mental ima...

This week I wanted to monitor several pieces of software that talk to one another via HTTP and HTTPS. All are running on the same machine, three are Linux services, and one is a standalone script. I was interested in being able to see all of the communications between them in one place, in time order. I know a couple of ways of capturing this kind of data: proxying and network sniffing .  My default approach would be to have the applications configured to proxy via Fiddler running on my laptop inside the work network. Easy, right? Err, no, because I had forgotten that the machine in question is on a network that isn't considered secure and firewalls prevent that connection. In my proof of concept experiment, the standalone script just hung failing to find the proxy I had specified. Interesting behaviour, and I later reported it, but not what I needed right then. Next! As all of the software is on the same machine, capturing network traffic int...