Hiccupps

The Association for Software Testing is crowd-sourcing a book,  Navigating the World as a Context-Driven Tester , which aims to provide  responses to common questions and statements about testing from a  context-driven perspective . It's being edited by  Lee Hawkins  who is  posing questions on  Twitter ,   LinkedIn , Mastodon , Slack , and the AST  mailing list  and then collating the replies, focusing on practice over theory. I've decided to  contribute  by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be. Perhaps you'd like to join me?   --00-- "What's the best format for a test plan?" I'll side-step the conversation about what a test plan is and just say that the format you should use is one that works for you, your coll

I have been pairing with a new developer colleague recently. In our last session he asked me "is this normal testing?" saying that he'd never seen anything like it anywhere else that he'd worked. We finished the task we were on and then chatted about his question for a few minutes. This is a short summary of what I said. I would describe myself as context-driven . I don't take the same approach to testing every time, except in a meta way. I try to understand the important questions, who they are important to, and what the constraints on the work are. With that knowledge I look for productive, pragmatic, ways to explore whatever we're looking at to uncover valuable information or find a way to move on. I write test notes as I work in a format that I have found to be useful to me, colleagues, and stakeholders. For me, the notes should clearly state the mission and give a tl;dr summary of the findings and I like them to be public while I'm working not just w

I'm definitely no expert in Large Language Models but expertise is not required, given all the publicity, to know that variation in the responses from the technology is expected.  To illustrate that, I asked ChatGPT  "In one sentence, why do LLM responses vary on the same prompt?" and got three different, although clearly related and overlapping, answers: LLM responses vary on the same prompt due to the model's stochastic nature, diverse training data, and sensitivity to slight changes in input phrasing.  LLM responses vary on the same prompt due to the model's probabilistic nature, employing diverse sampling techniques and leveraging its extensive training data to generate a range of contextually plausible and grammatically coherent outputs. LLM responses vary on the same prompt due to the model's inherent stochasticity and diverse training data, leading to dif