Hiccupps

Peers Exchanging Ideas : The AST Peer Conference Guide The Association for Software Testing is an organisation dedicated to advancing the craft of software testing and one of the ways we do that is by promoting and sponsoring peer conferences. When we say peer conference we don’t have a particular format in mind. Adrian Segar describes a peer conference as a conference that is small, attendee-driven, inclusive, structured, safe, supportive, interactive, community-building, and provides opportunities for personal and group reflection and action. We like that kind of framing as it provides room for many local variants, although we’d add that disseminating the results of the conference could be an important outcome too. If we had to boil down our own perspective, it would be something like this: peers exchanging ideas. To help the conferences that we sponsor, we decided to write a checklist based on the experiences of the AST Board members and other organisers that we'...

Behind the hyperbole of repeatedly saying that we don't know what we're doing in my talk at CEWT #7 a couple of weeks ago, my core message was that, for me: In order to be a great tester you have to embrace that not knowing. You have to be able to work within uncertainty, without being confident that anything will stand still, taking into account that your lack of knowledge of something might be the key issue. The project without uncertainty never existed and will never exist. You won't be sure you ran the experiment you thought you did, or generated the data you were hoping to, or analysed the results without imposing your own biases. Stakeholders often won't know what they really want, developers often won't be sure they understood what the stakeholders said they wanted, you often won't be sure that the developer implemented the thing they intended to. The company might appear confident that the bets it is laying this quarter will pay off, b...

The four presentations at CEWT #7 were on the topic of Dirty Testing Secrets. Here's my brief summary. According to Karo Stoltzenburg , we testers have a bad case of hubris about the uniqueness and value of our work. Not to put too fine a point on it, half of what we do is pointless and in any case could be done by someone else. Testers, she says, pride themselves on questioning, communicating and facilitating communication, and finding the important bugs but really they should find a bit of time to take a long, hard look at themselves. Questions? She's heard better from developers, subject matter experts, product owners. Testers have no monopoly on critical thinking and people in other roles have information and experience to bring to the table that testers often won't. Communication? Sure, it's common for testers to bring people together but we're also often then an extra node in the information flow network, a contributor to the cacophony of Chinese whisp...

After the presentations and discussions at  CEWT #7  we split into groups for a Lean Coffee session to pick up threads from the day. Here's the topics and aggregated comments from the group that I was in. I would love to see Devs discuss their purpose like this/Why does testing give itself such a hard time about value? Thinking about this stuff allows people to focus their time in the areas they want to work in. It helps managers know what motivates their staff. The purpose of a developer is to solve problems for people. Testing is never voted highly enough to get discussed at developer Lean Coffees I've attended. Testing is evolving because the context in which we work is evolving, so we need to reflect on our role constantly. Self-reflection is a strong testing skill ... we expect thoughtful analysis of other things by testers, so why not also of themselves? This group (at CEWT) is self-selecting and not representative of testers in general. Do we really feel...

The topic at CEWT #7  last weekend was Dirty Testing Secrets. I decided to present something reasonably provocative as a conversation starter. I think it worked. The essay below is a pretty version of the notes I prepared in advance. --00-- Quality Assurance. QA. It's getting less common, but it's still not unusual for people in software to talk about getting something into QA or to asking us to QA their stuff. I've worked hard over the years at our place to spread the word that I don't think of my team in that way. I do an induction for all new employees and explain how testing is a creative and intellectual activity, not a checkbox ticking drudge. Sadly, I still encounter career testers who think that their role is to confirm that requirements are met and no more. But my sense is that that's an open secret rather than a dirty one. This isn't a dirty secret either, although it might be a surprise to some: That's not to say that we ar...

At CEWT #7 today I surprised the participants by asking them for some definitions as they arrived. I used the definitions in my talk, which I've blogged about , but for now here's a simple list. Add your own definitions in the comments if you like. Quality How well someone perceives something works and meets a set of requirements. Value to someone who matters at some time. Quality is value to some person that matters. You'll know it when you see it - when something is good, shiny, runs smoothly. External quality is a positive characteristic of software encompassing robustness, correctness, and lack of bugs. An outcome that satisfies all stakeholders + customers. The extent to which any given attribute of a "thing" meets its intended or desired purpose. A positive attribute of something (as in "a quality of ..") Quality is subjective and hence hard to define as what's quality for some is not for others. Initially I was going to say wh...

The six presentations at CEWT #6   took different angles on the topic of good testing or good testers. Here's my brief summary of each of them: Whatever good testing is, Karo Stoltzenburg said, it's not likely to be improved by putting a box around its practitioners. In fact, pretty much any box you care to construct will fit some other profession just about as well at it fits testing. Testers, like people in general, are individuals and it's in the combination of diverse individuals that we spread our risk of missing something and so increase our chances of doing the good testing we all want. What makes a senior tester? That's the question Neil Younger has been asking himself, and his colleagues at DisplayLink. He shared some of his preliminary thoughts with us. Like Karo he wants to avoid boxes, or at least to reduce their rigidity, but against that there's a desire from some for objective criteria, some kind of definition of done that moves them from one...

At CEWT #6 we used Lean Coffee as a way to reflect on some of the threads we'd discussed during the day . Here's a few brief, aggregated comments and questions that came up. Different Perspectives Claire and Helen's talk was about how the testers and test manager on a project had a very different view on the quality of the testing. The developer perspective is also interesting, and often different. Whose needs are being met by the testing? Which lens are we looking through: experience, knowledge, context, ..? Good testing is inherently perspective-based. The relative rule . What about outside of software, e.g. in laboratory science? Stop Working and Start Thinking . What makes good tests? Consistent, deterministic, a specific outcome. Really? What about if the software is non-deterministic? Isn't testing about information? Is gathering data (e.g. performance data) testing? There needs to be a pass/fail. Really ? Is the tester the best per...

At CEWT #6  we discussed what good testing and good testers are. We didn't set ourselves the mission of coming up with some kind of definition, we didn't explicitly take that mission on during the course of the day, and to my mind we didn't converge on anything particularly concrete that could form the basis of one either. Reviewing my notes, I thought it might be interesting to just list some of the factors that were thrown into the conversation during the day. Here they are: Good relative to what? Good relative to who? Good relative to when? Good for what? Good for who? Good for when? Goodness can be quantified. The existence of bugs found by non-testers is a way to judge testers or testing. Goodness cannot be quantified. The existence of bugs found by non-testers is not a way to judge testers or testing. Goodness can't be separated from context. Goodness can't be separated from perspective. The value and quality of testing is subjective. Good t...

At CEWT #6 we were asking what constitutes good testing. Here's prettied-up versions of the notes I used for my talk. One line of argument that I've heard in this context and others is that even though it's hard to say what good testing is, we  know it when we see it . But there's an obvious potential chicken and egg problem with that. I'm going to simplify things by assuming that the chicken came first. In my scenario, we'll assert that we  can know what good testing would look like for a project which is very similar to projects we've done before, where the results of testing were praised by stakeholders. The problem I'll address is that we don't have anyone who can do that project, so we have to hire. The question is: What can we do as recruiters, during recruitment, to understand the potential for a candidate to deliver that good testing? I've been recruiting technical staff for around 15 years, predominantly testers in the last t...

When Sneha Bhat asked if I'd present with her at CEWT #5 the talk we produced was Theoreticus Prime vs Praktikertron . In this essay we've tidied up the notes we wrote in preparation and included a few of the sketches we made when we were developing our model. The title comes from the Transformers we gave the participants at CEWT to explore in an attempt to illustrate different kinds of theory being discovered and shared. CEWT #5 asked this question: theory over practice or practice over theory? It's an age-old conundrum, represented in popular culture by memes like these that you would have seen as you avoid both theory and practice by grazing on social media when you should be working: In theory, there is no difference between theory and practice. But, in practice, there is. ( Wiki ) Theory is when you know everything but nothing works. Practice is when everything works but no one knows why. In our lab, theory and practice are combined: nothing works and no...

Games that can say something about testing, then. Christmas brought a new one to our house, for my kids, from my Aunty Wendy. It's called Hedbanz and it's a game you probably already know as 20 Questions or Who Am I ? Essentially, each player takes a card and, without looking at its face, places it on a band around their head so that the other players can see it. The player then has a limited time to ask the others yes/no questions about their card, in an attempt to work out what it is. A typical game might go like this: Player: Am I alive? Players: no. Player: Am I manufactured? Players: no. Player: Do you find me in the home? Players: yes. And so on. A skill in the game is asking questions that together partition the space of all things in different ways to narrow down to a specific item. I like playing the game with my kids because it allows me to see their logic at work, and allows me to show them different strategies for getting to an answer. For our an...

At CEWT #5 we used Lean Coffee as a way to reflect on some of the threads we'd discussed during the day . Here's a few brief, aggregated comments and questions that came up. Can we identify testing short cuts? In particular, can we find short cuts without adverse side-effects? Short cuts sometimes have assumptions built into them (e.g. that the side gate you're going to use to get into work at the weekend is open then.) Some of the things you used to hold as axiomatic are no longer relevant so you can short cut your old thinking. Can the shortness be in depth rather than length? ... and you can gain breadth first, as a kind of short cut in testing. You can plan training to isolate particular skills and short cut potential confusion. You can tell someone they'll waste their time trying something, based on your experience. But you might deny them some learning. And you deny them the opportunity to learn to recognise a waste of time. All your learning can...

CEWT is the Cambridge Exploratory Workshop on Testing, and for its fifth meeting , hosted at Linguamatics , a bunch of local testers gathered to consider a question: Theory Over Practice or Practice Over Theory?  A couple of months before CEWT #5 it was posed . For a day during CEWT #5 it was explored . Immediately after CEWT #5 it was .... still mostly undecided. But along the way we did at least have a unicycle, some Transformers, an Alien Dance Party, a selection of retro Nokia phones, and Batman. Oh, and Lean Coffee, and a retrospective, and six talks. First up in the talks Karo Stoltzenburg tested the question. Taking Weinberg and Gause's classic Are Your Lights On? as her guide, she wondered whether there was a problem here and, if so, whose problem it was, how it got to be a problem, and whether or not it was worth solving. Noting the potential for ambiguity in key terms in the question and the Call For Participation , she carefully teased out possible interp...

CEWT #4  was about test management and test managers. One of the things that became apparent during the day was how much of a moveable feast the role associated with this title is. And that reflects my own experience. A few months ago, when discussing courses for the line managers in the Test team, a trainer outlined what his course would cover and asked whether I'd got any heuristics for management. I gave him these, none of which were included in his synopsis: Clear and present . (Say what you think and why, and what you are committed to; encourage and answer any question; be approachable, available and responsive, or say when you can be) It’s all about MOI . (Motivation: explain why we are doing what we’re doing; Organisation: set things up to facilitate work, opportunities; Innovation: be ready with ideas when they’re needed) Congruency in all decisions . (Consider the other person, the context, yourself) In advance of CEWT, one of my team asked me what I felt...