Hiccupps

I joined a new project this week. Me and a couple of engineers from my team are being asked to build a service to provide access to data sources, both in-house and external. As the lack of this data is being felt some distance from our team's main product, we haven't been involved in conversations up to this point. A kick-off meeting was the first opportunity for us to meet the various stakeholders and hear them say in their own words what they're looking for. This is usually an interesting phase in a project because there's lots of scope for ambiguity, conflicting vision, missing requirements and so on. But how to navigate it? No two projects are the same so I don't have a standard approach. The things I did around the meeting are reasonably typical and include: I looked for project documentation on Confluence, read it, and tagged the pages. I looked for other Confluence pages on the same topics, and tagged them too. I looked in Jira for recent tickets on the same ...

In recent times I've spoken and written about how much fun and how productive it's been to build random walkers to help me to test services I've been working on: Walking the Talk webinar , A Model Student , and Navigate, Survey, and Explore . The walkers are clients which use dice rolls to make decisions as they navigate paths through a service, asserting things about the state as they go. Traditional unit tests tend to be extremely specific. They imagine the system in a particular state with a hard-coded input and an expected output.  In my walkers, the assertions are relatively generic, for example: a service response payload conforms to a schema the value of a field representing progress will not decrease across a series of interactions some values in service responses must be in a particular relationship to others, or to the input. I doubt this is a technically-correct use of the term but I've been thinking of th...

I've created a state model-based testing tool for the service I work on using Altwalker . I call the tool a walker because it walks the model, interacting with the service as it traverses edges and making assertions about the state in each node. The service itself is stateless so I've actually been building a model of the kinds of interactions that it allows. Initially I modelled the journeys our clients make and this was helpful to get a better understanding of our client needs and gripes and also some of the nuances of our API that I hadn't been aware of before. However, while building and exploring the model I've discovered some other permitted, functional, consistent, but unintended ways to interact with the service.  I decided to keep them in the model because they exist and can be exercised and if their behaviour changes it might tell us we've done something we weren't expecting. But I've been adding configuration options to allow them to be turned o...

The Association for Software Testing is crowd-sourcing a book,  Navigating the World as a Context-Driven Tester , which aims to provide  responses to common questions and statements about testing from a  context-driven perspective . It's being edited by  Lee Hawkins  who is  posing questions on  Twitter ,   LinkedIn ,   Slack , and the AST  mailing list  and then collating the replies, focusing on practice over theory. I've decided to  contribute  by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be. Perhaps you'd like to join me?   --00-- "What is the best testing metric?" Now there's a question that has occupied many minds for many long hours. Don't worry, though, because this will...

Last week I attended a meetup on API testing with Mark Winteringham . In it, he talked through some HTTP and REST basics, introduced us to Postman by making requests against his Restful Booker bed and breakfast application, and encouraged us to enter the Test.bash() 2022 API Challenge which was about to close. The challenge is to make a 20-minute video for use at the Ministry of Testing's October Test.bash() showing the use of automation to check that it's possible to create a room using the Restful Booker API. I talk and write about exploring with automation a lot (next time is 14th October 2022, for an Association for Software Testing webinar ) and I thought it might be interesting to show that I am not a great developer and spend plenty of time Googling, copy-pasting, and introducing and removing typos. So I did and my video is now available in the Ministry of Testing Dojo . The script I hacked during the video is up in GitHub . My ...

I am totally loving Oddly Influenced, Brian Marick's new podcast. The latest episode covers ways in which schools of thought and practice can inhibit the cross-fertilisation of ideas.  It includes a case study in experimental physics from Peter Galison's book, Image and Logic , where two different approaches to the same particle analysis problem seem to run on separate, parallel tracks: In the 'head to world' tradition, you use your head to carefully construct situations that allow the world to express its subtle truths ... In the 'world to head' tradition, you make yourself ever more sensitive to the world’s self-expressed truths ... The first of these wants to theorise and then craft an experiment using statistics while the latter wants to gather data and try to understand it visually. Marick is pessimistic about the scope for crossover in this kind of situation: How do you bridge traditions that differ on aesthetics, on different standards of what counts as ...

I attended another Lean Coffee online with the Association for Software Testing this morning. Here's a few aggregated notes from the conversation. Teaching Developers About Testing There's a lot of debate in the community about this, with two extreme perspectives. Anybody can test and we don't need specialised testers. Nobody except specialists can test. There is huge value in teaching developers about testing. Anyone who is involved in building software should be involved in testing software. We've been experimenting with ensemble testing. We started with regular session every two weeks, bringing a task for testing from a customer perspective. The testers helped prepare for the session (areas to cover, environments, data, etc). The session included members of the quality group and the teams. It evolved into various parts of the organisation using this approach for exploratory testing. Is "teaching developers about testing" a good framing for the concern here...

I've been working on my talk for the Testing, Diversity, AI conference run by the Software Testing interest group of the British Computer Society.  In it, I'm thinking about the tooling I built to help me explore a chatbot API. It exploits random choice to walk through the extremely large space of possible chats in a medical symptom checking application. As I reflected on the combination of tools and testing I found it convenient to label three activities that involve both. Navigat e Navigation is about finding a path to an endpoint. While navigating I am very interested to notice assumptions I'm making, workarounds that are required, and any questions that come to mind, but my main focus is on reaching the goal. In the first instance, on this project, I needed a basic framework that would enable my code to start a chat, walk through all of the interactions with the service, and stop. As I was writing code to create my initial ...

  The Great Post Office Scandal by Nick Wallis is a depressing, dispiriting, and disheartening read. For anyone that cares about fairness and ethics in the relationship that business and technology has with individuals and wider society, at least. As a software tester working in the healthcare sector who has signed up to the ACM code of ethics through my membership of the Association for Software Testing I put myself firmly in that camp. Wallis does extraordinarily well to weave a compelling and readable narrative out of a years-long story with a large and constantly-changing cast and depth across subjects ranging from the intensely personal to extremely technical, and through procedure, jurisprudence, politics, and corporate governance. I won't try to summarise that story here (although Wikipedia takes a couple of stabs at it ) but I'll pull out a handful of threads that I think testers might be interested in: The unbelievable naivety which lead to Horizon (the system at th...

  In episode three of Oddly Influenced, jUnit and What Makes a Successful Tool , Brian Marick is speculating about the relative lack of adoption of test-driven development compared to the tooling that supported it.  After putting forward a particular theory he wonders whether it's "true" and says: That ... gives me an opportunity to state a theme that’s been in my work for decades. I read books ... about how people do their work. I’m not so concerned if the theories are true as if they are suggestive – that is, do they give me ideas about how software people should do our work. [The] next task is trying out whether those ideas have good results, in our work. Because everyone’s theory about people is somewhere between fully wrong and fully right, and is always incomplete. I like this perspective a lot. It puts me in mind of Paul Feyerabend's Against Method, a book I failed to finish because it was so dense and widely-read, bu...