Hiccupps

This week I was working with a developer to make a change to a legacy codebase in an area neither of us are very familiar with. The need is easy to describe in general and some occurrences of the behaviour we want to alter are common, straightforward to identify in use, and clear in the code. Unfortunately, the logic in the application is complex, the data used is domain-specialised and the behaviour we are interested in can occur in extremely specific combinations that are hard for a layperson to predict. I had no confidence that cases we knew were all of the cases. My colleague did a round of work and asked me to take a look. I exercised the application in a few ways while inspecting its logs in real time so that I could see the effect of the changes (or not) immediately. This gave up some rarer examples which had not been covered in the code. I added a couple of tests to characterise them and he identified another code change. Next, rather than continue exercising the product lookin...

  The Association for Software Testing is crowd-sourcing a book,  Navigating the World as a Context-Driven Tester , which aims to provide  responses to common questions and statements about testing from a  context-driven perspective . It's being edited by  Lee Hawkins  who is  posing questions on  Twitter ,   LinkedIn , Mastodon , Slack , and the AST  mailing list  and then collating the replies, focusing on practice over theory. I've decided to  contribute  by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be. Perhaps you'd like to join me?   --00-- "When the build is green, the product is of sufficient quality to release" An interesting take, and one I wouldn't agree with in gener...

  I was listening to the Radiolab podcast Stochasticity yesterday as I walked to the shop. The presenters were talking to two women, both named Laura Buxton, who became friends years ago because one of them released a balloon with their name and address attached from their back garden and the other found it 150 miles away in their back garden.  After Laura Two got in touch with Laura One they discovered other incredible similarities: they were both tall for their age, had brown hair and blue eyes, both had a labrador, black, a rabbit, grey, and a guinea pig with orange markings. They ended up in the local newspaper, and on national and even international television talking about how glad they were that fate had brought them together. If that sounds astonishing, how about this? As I strolled along the river and the podcast turned to the question of the probability of Laura-like events, David Spiegelhalter , a renowned statistician, jogged past me. What are the chances? The po...

A few weeks ago, in A Good Tester is All Over the Place , Joep Schuurkes described a model of testing work based on three axes: do testing yourself or support testing by others be embedded in a team or be part of a separate team do your job or improve the system It resonated with me and the other testers I shared it with at work, and it resurfaced in my mind while I was reflecting on some of the tasks I've picked up recently and what they have involved, at least in the way I've chosen to address them. Here's three examples: Documentation Generation We have an internal tool that generates documentation in Confluence by extracting and combining images and text from a handful of sources. Although useful, it ran very slowly or not at all so one of the developers performed major surgery on it. Up to that point, I had never taken much interest in the tool and I could have safely ignored this piece of work too because it would have been tested by ...

Dear Bug, After all we've been through I didn't expect to see you again the other day. Perhaps you thought I'd forgotten about us? Well, no, I remember you very well although without fondness.  Our relationship was intense but short-lived. It started in the dev environment when I glimpsed you out of the corner of my eye and knew immediately that I wanted to hold you close. Love at first sight? I wouldn't say that, but I recollect you teasingly jinking this way and that as I followed you through the architecture of our service, laughing and crying in turn. Eventually I caught up and found that I had been pursuing twins, two effects from the same underlying cause. We went on double dates (triple dates?) with a developer friend of mine until it became apparent that she understood you way better than I did. And that was the beginning of the end for us. As soon as she had finished with the bug she was seeing, you left me to be with her. That's OK, even before you and she...

The Association for Software Testing is crowd-sourcing a book,  Navigating the World as a Context-Driven Tester , which aims to provide  responses to common questions and statements about testing from a  context-driven perspective . It's being edited by  Lee Hawkins  who is  posing questions on  Twitter ,   LinkedIn , Mastodon , Slack , and the AST  mailing list  and then collating the replies, focusing on practice over theory. I've decided to  contribute  by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be. Perhaps you'd like to join me?   --00-- "Are observability and monitoring part of testing?" You'd like a simple answer first? OK, here you go: yes, they can be, but simply having, say, instrumente...

I'm reading Rick Rubin's The Creative Act: A Way of Being . It's spiritual without being religious, simultaneously vague and specific, and unerringly positive about the power and ubiquity of creativity.  We artists — and we are all artists he says — can boost our creativity by being open and welcoming to knowledge and experiences and layering them with past knowledge and experiences to create new knowledge and experiences.  If that sounds a little New Age to you, well it does to me too, yet also fits with how I think about how I work. This is in part due to that vagueness, in part due to the human tendency to pattern-match, and in part because it's true. I'm only about a quarter of the way through the book but already I am making connections to things that I think and that I have thought in the past. For example, in some ways it resembles essay-format Oblique Strategy cards and I wrote about the potential value of them to testers 12 years ago. This week I found the...

On Friday, in the weekly ensemble I have with some of the medical practitioners at work, I suggested we take a look at the first challenge from The Testing Map .  We all spend a lot of time testing although the others focus more on the domain side than the pure software side. That focus was reflected in the kinds of checks they tended towards: input strings that have obvious human semantics but which, for a seasoned software tester, would probably fall into a single equivalence class. While exploring some those strings we stumbled into security testing with the input "I don't know" because, due to the apostrophe, the testing challenge credited us with an attempted SQL injection. From there we were able to talk about script and HTML injection, and that slid into opening the developer tools in our browser and poking around the source, network traffic, cookies, and so on. The consensus when debriefing was that coverage can be improved when looki...

"How did you work that out so quickly?" my friend asked earlier this week when I sent him a handful of additional ways to view the problem we'd been pairing on.   "Stack Overflow and lots of small, fast, cheap experiments," I replied. And that's often true. It might not be Stack Overflow, but it's usually breaking a problem down, iterating on coherent chunks of it, composing the pieces into larger chunks, and building back to the original problem. The approach is reliable for me, but asking if it will be quick is like asking how long the proverbial string is . Sometimes it takes longer and I need to iterate on the approach itself, finding other "seams" along which to break the problem apart into units that can be addressed independently.  Sometimes I just give up and wait for the next opportunity to learn . Whether this is the right thing depends on all sorts of factors including whether I need an answer, whether I need the answer,...

  I was taking a look at some documentation last week. It listed a bunch of quality initiatives and said what each of them ensured. Ensured.   I had to stop and take a deep breath.  Several deep breaths. And a brisk walk around the block. I am triggered by wording like this. When someone says they've done some kind of testing, all that it ensures, assuming we can believe them , is that they've done something that they call that kind of testing. Literally, that's it. Setting some thresholds for a static analysis tool doesn't ensure quality. It ensures that the tool, when it is run , will look for certain kinds of patterns, in the code that it sees , and alert when a certain level of positive matches are found. The rules it uses may or may not conform to our idea of what quality means and, even if they do, there are likely bugs in either our thinking or the tool. Configuring a traditional pull request workflow doesn't ensure quality. It ensures that there are blocker...