Context-Driven, or What?

I was interviewed on Testers' Island Discs a few months ago. During the conversation, Mark Winteringham asked me about the Association for Software Testing and I said this:

The AST is a professional body for software testers. It's a non-profit — we're all volunteers — and its mission is to advance understanding of the science and practice of software testing according to context-driven principles ... which sounds like a real mouthful.

The way I like to gloss it, and the way it speaks to me, is that we value both expertise and experience when it comes to testing and generally speaking we favour context-driven testing too.

My sense is that, these days, most people would, even if they don't really know the terms, probably be doing some kind of context-driven testing. The proportion of people doing old-school stuff, test cases and so on, is shrinking as far as I can see.

I love talking about AST (why not come and join us?) and I very am happy with the way I interpret our mission as being about valuing expertise and experience ... but the way I referred to context-driven testing has been nagging away at me.

Why? Because I talked about "some kind of context-driven testing ..." as if there are flavours, and contrasted it with up-front test cases as if that's all it is.

What we say in the moment often isn't the optimal way we could express ourselves and I'm not vain enough to think that anyone listening was hanging on my every syllable in any case. However, I am interested in reflecting on what I said to better internalise what I think and make it more natural for me to say it concisely and accurately next time.

So, let's start at context-driven-testing.com. It describes the set of principles that underpin the Association for Software Testing's mission:

• The value of any practice depends on its context.
• There are good practices in context, but there are no best practices.
• People, working together, are the most important part of any project’s context.
• Projects unfold over time in ways that are often not predictable.
• The product is a solution. If the problem isn’t solved, the product doesn’t work.
• Good software testing is a challenging intellectual process.
• Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products.

These are opinionated principles, for sure, but they are principles that I can sign up to. 

The rest of context-driven-testing.com attempts to carve out a space in which context-driven testing is distinguished from testing where context is somehow considered, testing where context is not considered, agile testing, and even exploratory testing.

I am not under any illusions that being driven by context is any guarantee of good testing, or of consistent testing. There just isn't "the context" but instead "the contexts." Every person in a situation has their own take on what they see and hear at any given time, some parts shared with others and some not. The method of observing, the observations, and the tester's biases of interpretation will all contribute to determining any resulting actions.

On top of that, even for a given person with their own context, it's not possible to consider the whole context the whole time. Where are the edges of a context? How to know that the relevant context has been included and irrelevant context excluded? How to track when the context changes in a significant way?

Testers who are taking a context-driven approach will recognise these problems and try to balance them by, for instance, repeated observation, applying heuristics, finding similar examples from their experience, calling on their expertise, looking for collaboration, making regular attempts to synchronise with the people who matter on the project, and aligning work with current goals and constraints.

It's not a thought that I've had before, but I have the feeling that I have wrappd up context-driven approaches with being congruent. In congruent communication I want to make sure that I give fair consideration to myself, the other parties, and the wider situation. That's how I approach testing too. That's how I try to approach pretty much everything that matters to me.

I have also just noticed that I didn't use the term context-driven in my recent How To Test Anything talk even though I would readily describe the values it put forward and the approaches it described as context-driven.

What's that about, then? Good question. My immediate reaction is that I'm wary of labels and the preconceptions that go with them. There are so many X-driven-Y approaches these days, for example. That's a shame because context-driven, for me, provides scaffolding to stand on, not a stick to be beaten with.

My second reaction is that I'm unwittingly precious about the term and what it represents and that I don't want to dignify testing that doesn't deserve to be called context-driven by suggesting that it is. 

Back to Testers' Island Discs, then. When I said "most people will be doing some kind of context-driven testing ..." I think what I really wanted to say was something like "most people will be setting up their testing to take context into account rather than, for example, simply starting with a big set of test cases that they've derived from the requirements."

Which is all well and good yada yada blah blah blah semantics, you might say. Fair enough, but understanding what I think helps me to guide how I act and I'm very interested in acting in ways that I am comfortable with. This kind of writing helps me to get there.
Image: https://flic.kr/p/tkdaSh

Leaving Linguamatics

The big news in my world at the moment is that I'm leaving Linguamatics. It's a massive decision for me because I'm one of the founders and I've ploughed untold time and effort, blood, sweat, and (yes) tears into it over the years. 

I realise that I'm very fortunate to even be able to contemplate leaving the company given the global situation. However, despite all that's going on and the uncertain future, now still feels like a good time to part ways, for me and for the company. 

I'm immensely proud of what we've achieved at Linguamatics and what the company is set up to achieve next. My Test team have been challenging and inspiring in equal measure (if you don't like everything you say being questioned, don't  build a team of excellent testers) and I am confident they can continue to do good work without me. I also manage a small team of technical authors and I've been delighted to see what people who agree what a good job looks like and care to do it efficiently can achieve.

I was going to write a blog post about this, but I realised that I'd said it all in my email to the company already, so below there's a lightly-edited version of that.

--00--

Hi everyone,

I wanted to let you all know that I'll be leaving Linguamatics at the end of the year.  As you can imagine, that'll be a moment of very mixed emotions for me.

Linguamatics was started around 20 years ago after me and the other founders were made redundant from our previous employer. I think our first meeting as a company was around a dinner table and we sat back-to-back in a tiny office in St John's Innovation Centre for years as we created I2E from the ground up, learning how to build software, how to be a business, and how to work together.

Along the way were moments of great tension (like the time I accidentally unplugged the indexing servers) and great innovation (like the time I took the case off the web server and gaffa-taped a desk fan to it for cooling) and great satisfaction (like getting our first patent for the technology we invented).

We've taken on many wonderful people over the years - and that includes you all. I'm thrilled to have been involved in a significant amount of that recruitment and proud when I see how colleagues that I helped to hire have contributed and grown and become key to the company's success. With your help we've turned one product into a suite of products which are dedicated to the same vision we had originally: using natural language processing to extract information from unstructured text powerfully, efficiently, and with the ability to tune precision and recall to suit any need.

Where we've ended up is amazing. We have made Linguamatics into an incredible international business with a top-notch reputation doing work that helps make the world a better place. That's not to be sniffed at. When the opportunity to become part of IQVIA appeared there was a tough choice to make but I believed then, and continue to believe, that the two companies are a strong strategic fit and complement each other well.

On a personal level I've learned a tremendous amount from working at Linguamatics. I've done pretty much everything in the company at some point, including accidentally becoming the health and safety officer because I happened to pick up the phone when the local Council called asking for one. I'm always delighted when a colleague does something that I didn't know how to do, or shows me where my knowledge is lacking, because that's an opportunity to improve my skills and add to my toolbox. There have been lots of those occasions in the last two decades :)

Which brings me to today. It's been a long journey for me and, frankly, I'm now very tired. I want to take a break and then I want to look for a new challenge with new opportunities to learn. I'm extremely conscious that my entire working career has been at Linguamatics and I wonder how much of what I know is specific to here, and what could apply in other contexts, and what I could find out about how to test from other people in those other contexts. There's also a bunch of courses I have wanted to do for years but not found the time for, and I want to put some energy into my role as Vice President of the Association for Software Testing.

As I said, it's a time of mixed emotions, but after Linguamatics was basically my life for so long I think I can now move on without the company and I'm confident that the company can move on just fine without me too.

Thank you very much for being great colleagues and making Linguamatics a great place to work.

James

Image:https://flic.kr/p/57p1gv

Like This

Last week I started to see tweets with the simple text "Like this tweet and I’ll write something about you." The replies, all by the author of the tweet, were fascinating: numbers apparently representing a person followed by praise, reminiscence, and gratitude. Elisabeth Hendrickson's thread is a particularly uplifting example. 

I was intrigued, so I liked a couple of them. Shortly afterwards I got a direct message which explained that the tweet was part of a project for suicide prevention month with the aim of celebrating people while they are alive. If I wanted to join in I should provide a number to represent myself so that the author could write something about me, and I was invited to do the same for others.

I'm naturally cautious of these chain letter-like activities and I'd heard of Suicide Prevention Day and Week, but not month. However, I found that September does appear to be promoted as suicide prevention awareness month by the National Alliance on Mental Illness. Despite my concerns, the idea of helping to populate Twitter with positive messages appealed, so I joined in and ended up writing around 10 responses for people who liked my tweet.

I hope that someone who needs it right now found something valuable by seeing what me or others wrote. If not that, then I hope that some people who happened across these threads simply felt a rise in their mood and passed that on to others they interacted with.

Unexpectedly, I found the process of capturing only positive thoughts about the people who liked my tweet edifying. It was challenging at first because, perhaps by nature or perhaps by the nature of my job, I'm typically looking for a balanced perspective, seeing multiple sides, and avoiding blanket statements and absolutes. But it was tremendously pleasurable and enlightening to deliberately take the time to shed that skin and craft a couple of sentences that reflect the appreciation I have for someone I know or have found for someone I didn't know before. I like this. 
Image: https://flic.kr/p/xpmvFW

Disclaimer: I am no expert on suicide or its prevention. These links may be useful for those who want more:

• Help for suicidal thoughts (NHS, UK)
• About self-harm and suicide (on Twitter)
  
• World suicide helplines (Wikipedia)
  

Most Brutally Honest

 Maaike Brinkhof tweeted this last week:  

Give me your most brutally honest definition of software testing.

 As it happens, I have a definition of software testing that works for me, and it is (honestly) my honest answer:

   Testing is the pursuit of relevant incongruity.

But Maaike asked for the most brutally honest definition. How might that change what I'd say?

Well, what is brutal honesty? As I interpret the term, it describes a statement of perceived truth given with no regards for the feelings of anyone who might hear it.

That gives me a route in: can I find someone with a perspective that might offend us testing snowflakes, someone who is relevant to us, and someone whose view I can characterise with extreme candour?

I thought about this for about, um, one second and chose THE BUSINESS.

A perspective that might offend? Check. In the main, businesses run for the benefit of the business and, particularly as they get larger, can lose touch with their staff as individuals, become insensitive to employees' feelings about their work and why they do it, and blind to the craft that any given person learned and takes a pride in applying on a daily basis.

Relevant to us? Check. One of the key yardsticks for the work we choose to schedule should be the extent to which it would contribute to business value.

A perspective amenable to candour? Check. To the business, software testing is one of multiple potential risk mitigation activities, to be traded against one another and against the current business need in the current market against the current competition. 

From this point of view, software testing is nothing special, and can be simply described as, and my answer to Maaike's question:

Expendable insurance.

That is, a premium paid by the business which it is thought may not be necessary but which will pay out (by reducing losses) when the product does not meet business needs. It's expendable because it is viewed as a resource which can be cancelled, or the supplier changed, or the terms of engagement altered as the business requires.
 
Reflecting on this, I don't find myself offended. Yes, I find myself looking somewhat cynical and feeling perhaps sad about the way capitalism operates. But, realistically, this view of testing is not so different from my own in some respects.

I regard the testing my team provides as an optional service to the projects we work on. It's up to the project stakeholders (with my help, for sure) to decide what of their budget they want to invest in testing as opposed to peer review, or beta releases to the customers, or bringing in extra domain expertise, or other risk mitigation strategies. A key part of our development process is agreeing with the major stakeholders how they would like to spread the risk mitigation software testing can bring across the projects we're working on.

Naturally, my approach doesn't leave me with the bitter taste of cynicism or sadness in my mouth. But why? 

First, I have found a way to frame my work so that I don't feel frustrated by business considerations; doing a good job means exercising my skills in an effective, appropriate way given the project constraints.

Second, and perhaps more important, I typically operate at a granularity where I can still care about the people and the individual outcomes involved and I find that's important to me. 

Third, and related, I try to lead with empathy and this, I think, is what turns brutal honesty into radical candour and means that I can take the business perspective not as an insult but as a challenge to explain what testing could bring, to who, when, and when testing might not be appropriate.

Maaike shared her own definition later on:

Slapping people in the face with a dose of reality. Ad Infinitum

I love that it's also both candid and brutal.
Image: https://flic.kr/p/ed2sWW

Testers' Island Links

 

I was invited onto the Testers' Island Discs podcast recently. Testers' Island Discs is

... a podcast focused around software testing and storytelling hosted by Mark Winteringham.

The concept is straightforward: each episode, Mark interviews a different member of the global software testing community to talk about their career, interspersed with clips and discussions of the music that they love. Every guest gets to select five songs and one book to take with them to the island.

I've written about the music I chose in Island Records. This post is for links to things that came up in the conversation.

• Puns
• Your Testing is a Joke from EuroSTAR 2015
• The Rule of Three and Me
• CEWT
• How it started (Blog)
• Topics covered (List of CEWTs)
• CEWT 7 report
• CEWT 6 report
• CEWT 5 report
• Association for Software Testing
• Join the AST!
• Peers Exchanging Ideas ‘how to’ guide for peer conferences
• Context-Driven Testing 
• How to Test Anything
• My slides and video
• OnlineTestConf videos
• Overcoming nerves as a speaker
• Tech Voices 
• Toastmasters
• My blog about my experience
• A book
• Ron Jeffries - Extreme Programming Adventures in C# (My review)
• My book for MoT: When Support Calls
• Ron Jeffries Asteroids blog post series 
• Other notes
• Gather collaboration tool
• Trackers (software sequencers)
• Joint post with Lena Wiberg
• Galumphing with James Bach (not spelunking)
• K-cards

Image: Discogs 

Island Records

I was invited onto the Testers' Island Discs podcast recently. Testers' Island Discs is

... a podcast focused around software testing and storytelling hosted by Mark Winteringham.

The concept is straightforward: each episode, Mark interviews a different member of the global software testing community to talk about their career, interspersed with clips and discussions of the music that they love. Every guest gets to select five songs and one book to take with them to the island.

Music was my life for a long, long time. I was in bands, I did my own solo music, I wrote a fanzine and occasionally for "proper" publications too, I attended gigs several nights a week, and I DJed on student and community radio stations. I built up a big record collection stacked around the house and in the loft but when my daughters got old enough to need their own bedrooms I had to sell it to make space.

Although I never counted, I estimated that there were around 12-15000 albums, singles, flexidiscs, and CDs driven away in the bloke from Cambridge market's long wheelbase Ford Transit that sad, sad day.

Getting down to five tunes was tough. I originally thought I might choose from records whose sleeves I've used at the top of posts here on Hiccupps but, in the end, I settled for a handful that gave and continue to give me an emotional reaction. I hope you enjoy them.

Run DMC, Sucker M.C.'s (Profile) from Run DMC LP

Run DMC were the first band I got deeply into. It's a long time ago now, 1986 when their Raising Hell album came out, when I heard them on, I think, Janice Long's evening show on Radio 1. Hard, basic beats, chunky scratching, and rock guitar topped off with tag-team rapping.

I went up to Birmingham the next weekend and bought it then went back in later weeks to get the first album, Run DMC, and the second, King of Rock. All three are classics. Playing anything from these takes me back to that time when music had just got electrifying for me.

Cable, Oubliette (Infectious) from Down-lift the Up-trodden 10"

By the mid 1990s I'd become a bit bored by indie guitar bands but Cable (and also Compulsion with tracks like Why Do We Care?) reignited the excitement for me when they pinned me to the back wall of a tiny venue in Reading. I hoovered up their back catalogue on Krunch! and everything they put out afterwards.

I interviewed them at Nottingham Rock City and heard som outrageous stories about being kicked out of Oasis's studio at gunpoint! An oubliette is a kind of dungeon that you can only access through a trapdoor in its ceiling, by the way.

Jerry Reed, Thank You Girl (RCA) from When You're Hot You're Hot LP

For my sins, I have a soft spot for that extremely 1970s film, Smokey and the Bandit.  Burt Reynolds stars as The Bandit while his truck-driving sidekick, The Snowman, is played by the musician Jerry Reed. Reed also sings on the theme song for the film, East Bound and Down. It's great song and I used to sing it to my daughters to help get them off to sleep when they were babies.

I bought the soundtrack album on vinyl but when it arrived the sleeve was in awful condition. The place I ordered it from offered to let me choose something else as compensation and I picked a Jerry Reed album, When You're Hot You're Hot. Thank You Girl spoke to me immediately: "I wanna thank you girl/For loving me/I wanna thank you girl/For all the happiness you brought to me".

If it sounds cheesy to say it out loud, in the song it's simply magical. When me and my wife got married we chose this, Slade's Coz I Luv You, and I Walk The Line by Johnny Cash to play while we signed the register.

Pram, Cumulus (Too Pure) from Iron Lung EP 12"

The last term of my Master's degree was for project work, to write a dissertation. Naturally I spent a large part of it on a future-retro, techno-fear, short black and white film recorded on a silent Super-8 camera with a couple of my mates from the course. The film was called Wireless and it's about how phono terrorists are trying to break the state's stranglehold on the population which they control with mind-numbing music broadcast everywhere all of the time. We got all this old computer equipment out of skips at the Engineering department and borrowed a hall to set them up in from the college I was living in. Editing it was amazing, all these little strips of film hanging up on washing lines around my flat for weeks until we got it spliced together and transferred to video.

That was a major milestone, but we still needed to put a soundtrack on it and because I was a DJ and had a little DJ mixer and VCR that you could use to dub sound onto video tape it fell to me to do it, but the equipment was so primitive I mixed it live with three or four tape decks and a turntable. The sounds effects were all on audio tape, wound back to just the right place that I could put them into a tape deck and play them without needing to cue them up. A large part  of the soundtrack was an amazingly atmospheric track, Cumulus, by Pram which I had on a red vinyl 12" single called Iron Lung.

Years later, for reasons I don't recall, me and my daughters used to play a weird version of follow-the-leader to the same track, leaping around the living room and trying to pull the strangest poses we could. So when I hear this track I have two wonderful memories.

The Guy Who Invented Fire, Glow (Intellectos) from I Didn't Get Where I Am Today EP

No YouTube for this one, but I've made an MP3 available.

I used to make music in bands, in an Anglo-French duo, and alone as The Guy Who Invented Fire. I used an Amiga computer and tracker freeware based on Ultimate Soundtracker which was a four-channel software sequencer that played 8-bit samples.

This track is one of mine, probably the one I'm most proud of, in part because it was the point at which I realised I didn't need to be able to play anything more than once to be able to use it. I could simply record myself tinkering and then curate the best bits from it. I didn't consider it this way at the time, but it's really a step back from work to meta-work.

It was released on an EP called I Didn't Get Where I Am Today by a U.S. label, Intellectos. The title is from The Fall and Rise of Reginald Perrin, fact fans.

I've published a list of links to stuff I talked about on the podcast at Testers' Island Links.

Images: Pinterest/ebid, Discogs

What Price Quality?

 

 

Jerry Weinberg discusses quality in Quality Software Management: Systems Thinking with an anecdote about a book his niece wrote. In the story, her book is released with significant sections missing due to bugs in the word processor she was using. 

Weinberg happened to be working for the company that produced it and asked the project manager what was going on. The manager said that they knew about the issues but were unlikely to fix them any time soon:

... out of more than a hundred thousand customers we probably didn't have ten [who might have seen these problems] ... Eventually we'll probably fix them, but for now, chances are we would introduce a worse bug - one that would affect hundreds or thousands of customers. I believe we did the right thing.

This situation motivates Weinberg's idea that quality is relative, encapsulated in the famous definition linking quality to value:

    Quality is value to some person

Hundreds of thousands of customers get value from this particular product and many will consider it high quality. His neice ... well, not so much of either.

In The Shape of Actions Harry Collins and Martin Kusch link the value of some technology to the extent to which we are prepared to accommodate its behaviour. 

They call this Repair, Attribution and all That, and illustrate it by describing a calculator whch returns 6.9999996 for the calculation (7/11) x 11. The human operator understands that this really "means" 7 and will correct for it wherever the data is actually used.  

With those things in mind, it was fun to read a recent article in The Verge, Scientists rename human genes to stop Microsoft Excel from misreading them as dates, where a long-standing issue with the use of Excel in genetics had been worked around by changing genetics:

... over the past year or so, some 27 human genes have been renamed, all because Microsoft Excel kept misreading their symbols as dates ... when a user inputs a gene’s alphanumeric symbol into a spreadsheet, like MARCH1 — short for "Membrane Associated Ring-CH-Type Finger 1" — Excel converts that into a date: 1-Mar.

This is extremely frustrating, even dangerous, corrupting data that scientists have to sort through by hand to restore. It’s also surprisingly widespread and affects even peer-reviewed scientific work. One study from 2016 examined genetic data shared alongside 3,597 published papers and found that roughly one-fifth had been affected by Excel errors.

[...]

Microsoft did not respond to a request for comment, but [coordinator of HUGO Gene Nomenclature Committee, Elspeth Bruford]’s theory is that it’s simply not worth the trouble to change. "This is quite a limited use case of the Excel software," she says. "There is very little incentive for Microsoft to make a significant change to features that are used extremely widely by the rest of the massive community of Excel users."

Fun, yes, but it also makes me wonder: what price quality? who pays? how?
Image: https://flic.kr/p/26zeSd7