Hiccupps

The Association for Software Testing is crowd-sourcing a book,  Navigating the World as a Context-Driven Tester , which aims to provide  responses to common questions and statements about testing from a  context-driven perspective . It's being edited by  Lee Hawkins  who is  posing questions on  Twitter ,   LinkedIn , Mastodon , Slack , and the AST  mailing list  and then collating the replies, focusing on practice over theory. I've decided to  contribute  by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be. Perhaps you'd like to join me?   --00-- "Are observability and monitoring part of testing?" You'd like a simple answer first? OK, here you go: yes, they can be, but simply having, say, instrumented code, dashboards, alerting, o

I'm reading Rick Rubin's The Creative Act: A Way of Being . It's spiritual without being religious, simultaneously vague and specific, and unerring positive about the power and ubiquity of creativity.  We artists — and we are all artists he says — can boost our creativity by being open and welcoming to knowledge and experiences and layering them with past knowledge and experiences to create new knowledge and experiences.  If that sounds a little New Age to you, well it does to me too, yet also fits with how I think about how I work. This is in part due to that vagueness, in part due to the human tendency to pattern-match, and in part because it's true. I'm only about a quarter of the way through the book but already I am making connections to things that I think and that I have thought in the past. For example, in some ways it resembles essay-format Oblique Strategy cards and I wrote about the potential value of them to testers 12 years ago. This week I found the f

On Friday, in the weekly ensemble I have with some of the medical practitioners at work, I suggested we take a look at the first challenge from The Testing Map .  We all spend a lot of time testing although the others focus more on the domain side than the pure software side. That focus was reflected in the kinds of checks they tended towards: input strings that have obvious human semantics but which, for a seasoned software tester, would probably fall into a single equivalence class. While exploring some those strings we stumbled into security testing with the input "I don't know" because, due to the apostrophe, the testing challenge credited us with an attempted SQL injection. From there we were able to talk about script and HTML injection, and that slid into opening the developer tools in our browser and poking around the source, network traffic, cookies, and so on. The consensus when debriefing was that coverage can be improved when looki

"How did you work that out so quickly?" my friend asked earlier this week when I sent him a handful of additional ways to view the problem we'd been pairing on.   "Stack Overflow and lots of small, fast, cheap experiments," I replied. And that's often true. It might not be Stack Overflow, but it's usually breaking a problem down, iterating on coherent chunks of it, composing the pieces into larger chunks, and building back to the original problem. The approach is reliable for me, but asking if it will be quick is like asking how long the proverbial string is . Sometimes it takes longer and I need to iterate on the approach itself, finding other "seams" along which to break the problem apart into units that can be addressed independently.  Sometimes I just give up and wait for the next opportunity to learn . Whether this is the right thing depends on all sorts of factors including whether I need an answer, whether I need the answer,

  I was taking a look at some documentation last week. It listed a bunch of quality initiatives and said what each of them ensured. Ensured.   I had to stop and take a deep breath.  Several deep breaths. And a brisk walk around the block. I am triggered by wording like this. When someone says they've done some kind of testing, all that it ensures, assuming we can believe them , is that they've done something that they call that kind of testing. Literally, that's it. Setting some thresholds for a static analysis tool doesn't ensure quality. It ensures that the tool, when it is run , will look for certain kinds of patterns, in the code that it sees , and alert when a certain level of positive matches are found. The rules it uses may or may not conform to our idea of what quality means and, even if they do, there are likely bugs in either our thinking or the tool. Configuring a traditional pull request workflow doesn't ensure quality. It ensures that there are blocker

  I often review and collaborate on unit tests at work. One of the patterns I see a lot is this: there are a handful of tests, each about a page long the tests share a lot of functionality, copy-pasted the test data is a complex object, created inside the test the test data varies little from test to test. In Kotlin-ish pseudocode, each unit test might look something like this: @Test fun `test input against response for endpoint` () { setupMocks() setupTestContext() ... val input = Object(a, OtherObject(b, c), AnotherObject(d)) ... val response = someHttpCall(endPoint, method, headers, createBodyFromInput(input) ) ... val expected = Object(w, OtherObject(x, y), AnotherObject (z)) val output = Object(process(response.getField()), otherProcess(response.getOtherField()), response.getLastField()) assertEquals(expected, output) } ... While these tests are generally functional, and I rarely have reason to doubt that they

  When I opened Blogger the other morning I saw that the Hiccupps all-time page view counter had ticked up to exactly 500000. My initial feeling was pleasure. Ooh! A milestone! Half a million views! Get me ! But that was fleeting.  As with many metrics, today's value is almost the least engaging aspect. What's more interesting is what the number represents, what it is desired that it should represent, and how closely those two concepts align on axes that matter. Also interesting: who wanted this metric (and there may be multiple interested parties) and whether they still want it, what assumptions are being made about it, what values (or milestones) matter, what it is being plugged into, and what decisions are made using it. That's not all, of course, because one sample is just one sample. More interesting is where it sits in context to others, if and how it is trending, and whether it correlates with relevant related metrics over the same period. I could go on. I don't

When my dad retired around 20 years ago, I got him a second hand PC and have been serving as first, second, and third-line tech support for him and occasionally his mates ever since.  He's an inveterate story teller and, for a laugh, one time I told him he should write all of his tales down and give them numbers. Then, instead of telling them (again) he could just say the number and we'd all laugh, saving everybody time. Well, he called my bluff so I'm now mid-way through editing a 300-page Word document he sent me and if I ask him about a detail of one of the stories, he recounts the entire thing. Clearly my idea has backfired spectacularly. Even funnier, this opus only describes his early life in Smethwick and the various jobs he had between leaving school with no qualifications and retirement. Our family stories are in another epic file I haven't even seen yet! The joke is well and truly on me. Still, the pages are full of gold and h ere I'm sharing an escapade

After four years, three of them as Vice President, I'm standing down from the board of the Association for Software Testing . Let me say up front that I am an unapologetic romantic about my craft. (And, yeah , I called it a craft. Sue me.) I believe in what AST stands for, its mission , and in context-driven testing , so it's been an absolute privilege to be involved in running the organisation. It's also been fun, and full of difficult situations and choices, and hard work on top of family life and a day job. There also was the small matter of the global Covid pandemic to deal with. The immediate impact was on CAST, our annual conference , and in some ways the beating heart of the AST. We had to variously cancel, reschedule, and move CAST online and we are still experiencing the after-effects as we organise the 2023 in-person event . So why am I leaving? Well, first, I'm not leaving the organisation, only the board. I am a life member and