Hiccupps

  The Association for Software Testing is crowd-sourcing a book,  Navigating the World as a Context-Driven Tester , which aims to provide  responses to common questions and statements about testing from a  context-driven perspective . It's being edited by  Lee Hawkins  who is  posing questions on  Twitter ,   LinkedIn , Mastodon , Slack , and the AST  mailing list  and then collating the replies, focusing on practice over theory. I've decided to  contribute  by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be. Perhaps you'd like to join me?   --00-- "When the build is green, the product is of sufficient quality to release" An interesting take, and one I wouldn't agree with in gener...

  I was taking a look at some documentation last week. It listed a bunch of quality initiatives and said what each of them ensured. Ensured.   I had to stop and take a deep breath.  Several deep breaths. And a brisk walk around the block. I am triggered by wording like this. When someone says they've done some kind of testing, all that it ensures, assuming we can believe them , is that they've done something that they call that kind of testing. Literally, that's it. Setting some thresholds for a static analysis tool doesn't ensure quality. It ensures that the tool, when it is run , will look for certain kinds of patterns, in the code that it sees , and alert when a certain level of positive matches are found. The rules it uses may or may not conform to our idea of what quality means and, even if they do, there are likely bugs in either our thinking or the tool. Configuring a traditional pull request workflow doesn't ensure quality. It ensures that there are blocker...

I'm lucky that my current role at Ada Health gives me, and the rest of the staff, a fortnightly community day for sharing and learning. I've done my, erm, share of sharing, but today I took advantage of the learning on offer to attend a workshop on our approach to making medical terminology accessible to non-experts, a presentation on how we manage our medical knowledgebase, another on the single sign-on infrastructure we're using in our customer integrations, and a riskstorming workshop using TestSphere to assess an air fryer. So that would have been a great day by itself, but I, erm, capped it off by attending Capgemini's TestJam event, to see the keynotes by Janet Gregory and Lisi Hocke . Janet talked about holistic testing, or the kinds of critical review, discovery, and mitigation activities that can take place at any point in the software development (and deployment, and release) life cycle. The foundation for all of this is good communication and relationship...

Last night I attended What Does the 'Coach' in 'Quality Coach' Mean? , a talk by Vernon Richards for the Ministry of Testing meetup in Brighton. Vern described a common transition from tester through quality engineer and into quality coach on three dimensions: process, product, and team. In that view, a tester tends work in one team on one product and have influence on team-level process; a quality engineer is still team-based but takes a broader view on process; and a quality coach extends the process oversight further still and works across teams and products. This would typically mean that the problems in front of a quality coach are less about the technology and more about the people. How should a coach deal with that? By providing perspective to those fighting the fires, always taking a position of "unconditional positive regard" which is to say that the coachee has the right information and abilities to solve their current problem. Coach and tester skill...

  I'm in the middle of BBST Bug Advocacy at the Association for  Software Testing right now.  As you might imagine, on a course with that name there's been plenty of talk about what we mean by terms like bug, value, and quality. One of the great things about the four-week course is the mix of book work and application, so we students are repeatedly challenged with situations in which the learning can be practically applied. I have a lot of time for both Seth Godin and Shane Parrish so I'd have been listening carefully to Seth's appearance on the Knowledge Project podcast anyway but, given the context I'm in, the passage I've transcribed below stood out. It's about how the concept of quality is concretised as conformance to spec, and how that in turn directly drives physical actions. It starts at around 1:04:45: There's lots to be said about the spec. First lets talk about Edwards Deming and what spec and quality mean. Quality is not luxury, quality ...

Last night I attended Influencing Quality & tackling the problems that come with it... , a panel webinar hosted by Burns Sheehan and chaired by Nicola Martin . The panellists were: Marc Abraham , Head of Product Engagement at Asos Antonello Caboni , Engineering Coach at Treatwell Marie Drake , Principal Test Automation Engineer at NewsUK Pedro Vilela , Engineering Manager at Curve As you might guess from that line-up the material covered was broader than simply testing, taking in subjects such as quality, value, hiring, and the delivery process across the engineering organisation. The signal to noise ratio was high, so my sketchnotes capture only a few of the points made on each topic. The idea that quality is a whole team responsibility came up several times. Quality is a notoriously slippery and subjective concept, so in the Q&A I asked whether any of the speakers had ever tried to create a Definition of Quality for their team, something like a traditional Definition of Done...

Last night I attended a Heart of England Scrum User Group meetup where Mike Harris was asking So where did all this agile stuff come from? Luckily he was answering also: W. Edwards Deming . Mike's presentation was a high-level overview of the history of Lean and Agile in which he traced back to foundational work done by Deming and then to Deming's influence Walter Shewhart who integrated scientific methodology and statistics into industrial quality practices. I have read a little Deming but I'm motivated to look more deeply after this. In particular, Mike drew attention to the fact that Plan, Do, Study, Act and Plan, Do, Check, Act were, for Deming, very much not the same thing. I had never realised this.  The paper Circling Back: Clearing up myths about the Deming cycle and seeing how it keeps evolving by Ronald D. Moen and Clifford L. Norman talks about the strength of Deming's feeling about it, quoting him: The...

Should the public care about software testing? That's the question that the Association for Software Testing and the BCS Software Testing Specialist Group asked at our first joint peer conference on 22nd November 2020. The remit was: It’s our contention that most people don’t think very much about software development and software testing, despite software being deeply embedded in almost all aspects of our lives. When there’s a publicised issue such as a national bank failing to process customer orders for several days, a government IT project overrunning for years and then being canned, or a self-driving car causing a fatal accident, then society might take notice for a while. However, even when that happens it’s rare for the complexities and risks associated with the creation, integration, and maintenance of software systems to be front and centre in the discussion, and testing almost never makes it ...

    Jerry Weinberg discusses quality in Quality Software Management: Systems Thinking with an anecdote about a book his niece wrote. In the story, her book is released with significant sections missing due to bugs in the word processor she was using.  Weinberg happened to be working for the company that produced it and asked the project manager what was going on. The manager said that they knew about the issues but were unlikely to fix them any time soon: ... out of more than a hundred thousand customers we probably didn't have ten [who might have seen these problems] ... Eventually we'll probably fix them, but for now, chances are we would introduce a worse bug - one that would affect hundreds or thousands of customers. I believe we did the right thing. This situation motivates Weinberg's idea that quality is relative, encapsulated in the famous definition linking quality to value:     Quality is value to some person Hundreds of thousands of customers get...

The other day, the Ministry of Testing tweeted this : Great question from Cassandra:  "Could you share any tips on how to let go of that idea of personal perfection, when part of our job as testers is to aim for perfection?".  Is this something you have advice on? I've certainly had this conversation with testers in the past but I've had it with teams from other disciplines that I've managed too. This was the answer I gave to the tweet: Reframe 'success' from being the pursuit of perfection to something like getting a good solution at the right time for a reasonable cost. 'Good', 'right', and 'reasonable' are context-dependent and stakeholders should be able to guide the team on what they mean and when. The more general version is that I've found that people with skills in a particular area can tend to feel compromised if they haven't utilised their skills t...

We don't have any business analysts at our place but I'm interested in the role and the potential insights a BA can bring to software development. Historically, I've wanted my testing to be concerned with more than whether the desired scope was implemented: I want it to wonder whether that scope is targeting the right problem and, if it is, whether some other scopes could solve it too, and what the relative merits of each are. I think this kind of work overlaps with the things a BA might do, but perhaps with sharper tools, and I perceive that there's crossover in the skill set too. To give my thoughts some more grounding I looked for an intro book and chose Business Analysis Agility by James Robertson and Suzanne Robertson. Why that one? Because it talks end-to-end about software development in contemporary environments, because it's got worked examples, and because I liked what Johanna Rothman's five-star review on Amazon said about it: This book bu...

Anne-Marie Charrett delivered a beta version of her Testbash Manchester keynote at the Cambridge Tester meetup this week . Her core message was that quality and testing are not the same thing: there are non-testing aspects of software development that contribute to product quality there are non-product aspects of quality which should be considered in software development. A theme of the talk was that customer benefit could be threatened by the second of these, by factors such as code hygiene, speed of delivery, and time to recover after a failure in production. Testers, and others in software development, were urged to reframe their view of quality to encompass these kinds of activities. A Venn diagram represented it like this: Interesting, but it didn't quite hang together for me. I slept on it. In the morning, I found myself thinking that what Anne-Marie was trying to visualise really had two notions of quality, and they were not the same. Perhaps she could mov...

I tell you what: of all the things I might've expected to see on the first slide at Quality Jam London 2017 , my own professional-work-photo-grinning, shiny-pated, blue-tinted face peering back down at me from behind a massive Thank You! wasn't it. Expectations are grist to the working tester's mill, yet also often the bane of their lives. Tony Bruce , in  Manual Testing is Dead. Long Live Manual Testing , called for testers to set the expectations of the people that they interact with. The term "manual testing" undersells what testing is, or can be, with its connotations of manual labour, unthinking monotony, apparent separation from (woo! sexy!) automation and the historical association with scripted test cases. For Bruce, testing is "the pursuit of information" but he doesn't necessarily rush into meetings spouting from that kind of lexicon (although he's singing my kind of song right there). Instead he promotes the use of PAC (purpos...

Hands up if you're suspicious of vendor-run conferences. Yeah, me too. But I'm pleased to report that QASymphony didn't ram qTest down our throats at  Quality Jam London 2017 . Better still, they had a programme that included some speakers who had (a) nothing to do with the tools, and (b) interesting stuff to say.  At 100 people, and with a single track, it was a good size and shape for my taste, and having the talks in the (quite cosy) repurposed subterranean beer tank of an old Whitbread brewery added to the atmosphere and charm. I took the opportunity to practice sketchnoting again in all the talks. Mixed results, for me, but here's the ones I'm prepared to share. See my notes on the conference too.

Jerry Weinberg's definition of quality is well known. It is generally applied to encapsulate a relationship between a person and a product, at a particular time, and goes like this: Quality is value to some person It is intended to be a practical tool, and I think Weinberg would agree with something like this as a gloss for it: theoretical assessments of quality — perceived quality — are less important than those which are motivated by action. For example, a property is worth what someone actually pays for it. Without the action, it's just philosophy. What someone is willing to pay, or sacrifice, determines the quality (to them) at that moment. I've thought about this definition a lot over the years. In particular I've found myself speculating about the granularity of the definition. Back in 2012 I was wondering whether it was interesting to consider quality in terms of the aggregation of a set of qualities ;  more recently  I was thinking about the way that p...

Good Products Bad Products by James L. Adams seeks, according to its cover, to describe "essential elements to achieving superior quality." Sounds good! As I said in my first (and failed) attempt to blog about this book, I'm interested in quality. But in the introduction (p. 2) Adams is cautious about what he means by it: Quality is a slippery, complex, and sometimes abstract concept ... Philosophers have spent a great deal of time dealing with the concept of quality. This is not a book on semantics or philosophy, so for our purposes we will simply assume that quality means "good." But, of course, that leaves us with "good for whom?" "good for what?" "good when?" "good where?" and if you really like to pick nits, "what do you mean by good?" I won't go there, either. My bias is towards being interested in the semantics and so I'd have liked not to have seen a concept apparently so fundamental to t...

A couple of years ago, in It's a Mandate , I described mandated science : science carried out with the express purpose of informing public policy. There can be tension in such science because it is being directed to find results which can speak to specific policy questions while still being expected to conform to the norms of scientific investigation. Further, such work is often misinterpreted, or its results reframed, to serve the needs of the policy maker. Last night I was watching a lecture by Harry Collins  in which he talks about the relationship between science and democracy and policy. The slide below shows how the values of science and democracy overlap (evidence-based, preference for reproducible results, clarity and others) but how science's results are wrapped by democracy's interests and perspectives and politics to create policies. I spent some time thinking about how these views can serve as analogies for testing as a service to stakeholders. But Co...