Who Cares?

Should the public care about software testing? That's the question that the Association for Software Testing and the BCS Software Testing Specialist Group asked at our first joint peer conference on 22nd November 2020. The remit was:

It’s our contention that most people don’t think very much about software development and software testing, despite software being deeply embedded in almost all aspects of our lives.

When there’s a publicised issue such as a national bank failing to process customer orders for several days, a government IT project overrunning for years and then being canned, or a self-driving car causing a fatal accident, then society might take notice for a while.

However, even when that happens it’s rare for the complexities and risks associated with the creation, integration, and maintenance of software systems to be front and centre in the discussion, and testing almost never makes it to the agenda at all.

In this workshop we aim to explore why that is, and what testers, testing organisations, software development companies, and governments could do to persuade the public that software testing is worth understanding and caring about.

As with so many events in these troubled times, we shifted an in-person experience online and were concerned that the buzz that comes from being in the same room as other people who are both knowledgeable and care deeply about the topic would be lost. 

I'll blog about the way we set it up later, but I'm pleased to report that the buzz was still there. I'll also blog about the breadth and depth of the conversations we had around the presentations but here I'm just going to summarise what the speakers said.

--00--

We kicked things off with Fiona Charles using Qantas flight QF72 as an example of how well-intentioned automation, complexity, and fail-safe mechanisms can result in at best unwanted behaviour and, at worst, disastrous outcomes. In that incident, faulty sensors caused on-board systems to misinterpret the plane's attitude as dangerous and take unnecessary action to correct it. Flight envelope constraints prevented the pilot from overriding the manoeuvre, leaving him a helpless spectator as passengers were tossed violently around the cabin.

Fiona posed the question "what level of control should we keep for ourselves?" and suggested that we have to build products that keep people at the centre of operations, facilitating and enhancing skilled human decision-making and enabling the human operator to work in whatever way is best for them, not fight against a system built for its own convenience.

Many modern systems (particularly those labelled AI) rely heavily on data, but data is not knowledge, and it comes with biases. As an industry and a scociety we are increasingly recognising this, but we tend not to recognise that the way the data is collected is biased, the architecture of the systems is biased, the models underlying the design are biased, the decision processes are biased, and, in fact, anything humans create is likely biased in some way.  

To help ourselves to see that bias and counteract it, Fiona said, we need to take seriously such questions as what could possibly go wrong, what possible outcomes there could be, and how does our product solve well-known concerns such as security or accessibility? Oh yes, and we should look to tune our bullshit detectors.

Harmful outcomes also concern Amit Wertheimer. In his presentation he took the position that software testing as an activity in its own right is just one way to help to create working products that do no harm. He described a separate team of testers distinct from developers as a crutch, providing a way for the builders of a product to disengage from the need for checking their work.

For Amit, in general, developers should be reviewing what they create and should be ready to "feel the pain" of it being found wanting by its users. That's not to say that the critical thinking skills that we testers like to claim have no value, but rather that other people, directly involved in development, can provide them too. He makes an exception for highly-specialised fields where domain knowledge may be instrumental in understanding the desired and observed behaviour of a product.

The public should not care about these details, though, he says. The public should care whether their product works and does no damage, not whether testers were involved in its production, or how.

Huib Schoots offered this perspective too, and also the opposite: the public should not care about software testing because that's the responsibility of the producers, yet the public should care about software testing because they need to be able to trust products, particularly in safety-critical situations.

He asked us to consider what happens when there's some kind of significant software failure. People tweet about it for a while, it blows up in the news perhaps, and then we all forget about it. Where is the forum in which the public can ask questions and request improvements? How can regulation and the law evolve with rapidly-changing technology?

He called for efforts to make an environment in which the public and software producers can get together to discuss how people's needs and concerns can be met, but then also warned about the risks of these being overrun by fringe views and conspiracy theories and noise.

Eric Proegler had no qualms about coming down on one side of the question: the public must care about software testing because of the increasing dependency of our lives and lifestyles on software.

Testers also have something to think about, he says. With current technology, it's easier than ever to create an "AI solution" and push it out to innumerable devices at a terrifying pace with huge numbers of poorly-understood and dynamic dependencies.

Software testing is under siege, in Eric's view.  One way forward is to find ways to incentivise better testing. Weak testing is motivated by short-termism, by getting to market, and by meeting quarterly targets. The business need trumps the societal need most times and Eric challenged us to wonder how we can change that.

The BCS are trying to change that, Adam Leon Smith claims, by engaging with government, the media, the public, and industry to highlight failures in IT and understand how they can be reduced both in frequency and impact. "We shouldn't hinder innovation" shouldn't be a popular opinion, in his opinion.

Regulation and certification is one avenue that can be explored. While this might seem like an anathema to some, Adam noted that software testing in some parts of some industries is already regulated, and cited the UK gaming machine testing strategy regulation as an example.

Acknowledging that testing is context-dependent, Adam speculated that there is scope for wider regulation of this kind. He would begin with high-risk scenarios where context could be restricted sufficiently that a non-generic standard for testing could plausibly be created. The high-level results of this kind of work could be communicated to consumers using a simple labelling scheme, similar to that used on food packaging to summarise the "healthiness" of the contents.

Take a step back, Janet Gregory urged us. We need to think about risks while producing our products and then find a way to communicate the risks to our users. A labelling scheme of the kind Adam suggested could certainly help, but Janet would like to see something like industry-wide checklists for things to consider on purchase, similar to the paperwork packaged with medicines that list potential side-effects.

She also asked for much tighter controls on the way in which products are advertised and cited self-driving cars as an example. For those kinds of products, adverts might claim "this model is an industry standard for safety" but consumers need to be aware that this doesn't mean getting in, pressing the take-me-to-mother's-house button and then going to sleep.

It's not enough to simply make this information available, of course. It needs to be made available in format that consumers can and, crucially, want to engage with. Techniques such as visualisations and analogy could be employed here. Bodies like AST and BCS can definitely be part of a dialogue around that.

Lalit Bhamare rounded things off for us with a call to action for consumers themselves. He talked about clean software — parallel to clean air — and said that unless consumers demand it, producers are unlikely to sacrifice convenience and profit to provide it.

What is clean software? Features that Lalit mentioned included reliability, quality, customer service, and a strong consideration of a product's impact on the world and not just the bottom line. Companies today largely take the public's acceptance of low-quality software for granted.

Testing is a crucial part of Lalit's idea. The public should care about software testing because testing helps them to have confidence that the software is clean, and if there's a public conversation about testing then manufacturers will be forced to pay attention to it.

The material created at the conference is jointly owned by the participants: Lalitkumar Bhamare, Fiona Charles, Janet Gregory, Paul Holland, Nicola Martin, Eric Proegler, Huib Schoots, Adam Leon Smith, James Thomas, and Amit Wertheimer.
Image: Rare Records