Skip to main content

Fail Here or Fail There

The First Law of Systems-Survival, according to John Gall, is this:

A SYSTEM THAT IGNORES FEEDBACK HAS ALREADY BEGUN THE PROCESS OF TERMINAL INSTABILITY
Laws are all-caps in Systemantics. Not just laws, but also theorems, axioms, and corollaries. There are many of them so here's another (location 2393-2394):
JUST CALLING IT “FEEDBACK” DOESN’T MEAN THAT IT HAS ACTUALLY FED BACK

There was a point when I realised, as the capitalised aphorisms rolled by, that I was sinking into the warm and sweetly-scented comforting foamy bathwater of confirmatory bias. Seen, seen, seen! Tick, tick, tick!

I took the opportunity to let myself know that I'd been caught in the act, and that I needed to get out of the tub and start to challenge the content. 

Intervening at that moment was congruent: I was in a context where I would accept it and prepared to change because of it. Of course, I enjoyed the deep irony of nodding along with Gall when he talked about that too (2456-2458):

Feedback is likely to cause trouble if it is either too slow or too prompt. It must be adjusted to the response rhythms of the system as well as to the tempo of the actual events — a double restriction.
So what might I challenge?

  • The lack of data to back up claims.
  • The overwhelming landslide of those upper-case one-liners.
  • The desert-dry commentary that weaves dangerously along the line between sniper sharp-shooting and sniping foot-shooting.

Looking around, I see that other readers have made similar observations. Cristiano Rastelli's review notes that everyone else he has spoken to about the book thought it was bullshit and simply stopped reading.

But I liked it despite its flaws. As a collection of practical, cynical, and even pathological heuristics it's a useful reminder of the power systems to do their own thing, to be realistic about the extent to which we can influence them, and to note again that all complex systems run broken:

IF IT DOESN’T FAIL HERE, IT WILL FAIL THERE (1562-1563)

I've pulled out a few quotes on topics that speak to my experience, including:

  • iterate systems into existence
  • seek the minimum necessary process
  • restrict only what must be restricted
  • align with natural tendencies if you can
  • make small changes where possible
  • pause to observe the effects of changes, emergent and intended, local and remote
  • remain humble about what you understand and the extent to which you understand it

--00-- 

Systemantics ... is almost a form of Guerilla Theater. It is the collection of pragmatic insights snatched from painful contact with the burning issues and ongoing problems of the day. (463-464)

NEW SYSTEMS MEAN NEW PROBLEMS (504-505)

COMPLEX SYSTEMS EXHIBIT UNEXPECTED BEHAVIOR (631-632)

Most people would like to think of themselves as anticipating all contingencies. (652-653)

A LARGE SYSTEM, PRODUCED BY EXPANDING THE DIMENSIONS OF A SMALLER SYSTEM, DOES NOT BEHAVE LIKE THE SMALLER SYSTEM (686-688)

SYSTEMS TEND TO OPPOSE THEIR OWN PROPER FUNCTIONS (703-704)

THE GHOST OF THE OLD SYSTEM CONTINUES TO HAUNT THE NEW (847-848)

In general, the larger and more complex the System, the less the resemblance between a particular function and the name it bears. (886-887)

THE SYSTEM ITSELF DOES NOT DO WHAT IT SAYS IT IS DOING (902-903)

A SYSTEM IS NO BETTER THAN ITS SENSORY ORGANS (1011-1012)

THE BIGGER THE SYSTEM, THE NARROWER AND MORE SPECIALIZED THE INTERFACE WITH INDIVIDUALS (1033-1035)

THE END RESULT OF EXTREME COMPETITION IS BIZARRENESS (1199-1200)

BIG SYSTEMS EITHER WORK ON THEIR OWN OR THEY DON’T. IF THEY DON’T, YOU CAN’T MAKE THEM (1257-1258)

Even today, the futility of Pushing On The System is widely unappreciated. (1273-1274)

THE MODE OF FAILURE OF A COMPLEX SYSTEM CANNOT ORDINARILY BE DETERMINED FROM ITS STRUCTURE (1472-1473)

The problem of evaluating “success” or “failure” as applied to large Systems is compounded by the difficulty of finding proper criteria for such evaluation. (1366-1367)

The idea that Bugs will disappear as components become increasingly reliable is, of course, merely wishful thinking. (1557-1557)

ONE DOES NOT KNOW ALL THE EXPECTED EFFECTS OF KNOWN BUGS (1587-1589)

NEW STRUCTURE IMPLIES NEW FUNCTIONS (1645-1646)

The designers had built a machine with that capability, but they knew not what they had wrought. . . until Experience demonstrated it to them. (1642-1644)

AS SYSTEMS GROW IN SIZE AND COMPLEXITY, THEY TEND TO LOSE BASIC FUNCTIONS (1663-1664)

THE MEANING OF A COMMUNICATION IS THE BEHAVIOR THAT RESULTS (1879-1880)

We must not assume that a message sent will automatically go to a central Thinking Brain, there to be intelligently processed, routed to the appropriate sub-centers, and responded to. (1992-1994)

The student proficient in the Creative Tack asks such questions as: What can I do right now and succeed at it? For which problem do my current resources promise an elegant solution? (2170-2172)

DO IT WITHOUT A NEW SYSTEM IF YOU CAN (2186-2186)

AVOID UNNECESSARY SYSTEMS (SYSTEMS SHOULD NOT BE MULTIPLIED UNNECESSARILY) (2187-2189)

LOOSE SYSTEMS LAST LONGER AND FUNCTION BETTER (2264-2265)

A System represents someone’s solution to a Problem. The System itself does not solve Problems. Yet, whenever a particular problem is puzzling enough to be considered a Capital-P Problem, people rush in to design Systems which, they hope, will solve that Problem. (2521-2525)

How many features of the present System, and at what level, are to be corrected at once? If more than three, the plan is grandiose and will fail. (2584-2586)

IF IT’S WORTH DOING AT ALL, IT’S WORTH DOING POORLY (2603-2604)

IN ORDER TO BE EFFECTIVE, AN INTERVENTION MUST INTRODUCE A CHANGE AT THE CORRECT LOGICAL LEVEL (2689-2690)

It seems clear enough that changing actors does not improve the dialogue of a play, nor can it influence the outcome. Punishing the actors is equally ineffective. Control of such matters lies at the level of the script, not at the level of the actors. In general, and as a minimal requirement: (2686-2689)

Exploratory behavior constitutes a series of probes, each of which elicits a piece of behavior from the System. The accumulation of those pieces of behavior allows the rat (or person) eventually to obtain a perspective as to the range of behaviors the System is capable of exhibiting in response to typical probing behaviors. (2714-2716)

ALWAYS ACT SO AS TO INCREASE YOUR OPTIONS (2724-2724)

THE SYSTEM IS ALTERED BY THE PROBE USED TO TEST IT [...] THE PROBE IS ALTERED ALSO (2774-2775)

All reference locations are from the Kindle edition.
Image: Wikipedia

Comments

Popular posts from this blog

Can Code, Can't Code, Is Useful

The Association for Software Testing is crowd-sourcing a book,  Navigating the World as a Context-Driven Tester , which aims to provide  responses to common questions and statements about testing from a  context-driven perspective . It's being edited by  Lee Hawkins  who is  posing questions on  Twitter ,   LinkedIn , Mastodon , Slack , and the AST  mailing list  and then collating the replies, focusing on practice over theory. I've decided to  contribute  by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be. Perhaps you'd like to join me?   --00-- "If testers can’t code, they’re of no use to us" My first reaction is to wonder what you expect from your testers. I am immediately interested in your working context and the way

Meet Me Halfway?

  The Association for Software Testing is crowd-sourcing a book,  Navigating the World as a Context-Driven Tester , which aims to provide  responses to common questions and statements about testing from a  context-driven perspective . It's being edited by  Lee Hawkins  who is  posing questions on  Twitter ,   LinkedIn , Mastodon , Slack , and the AST  mailing list  and then collating the replies, focusing on practice over theory. I've decided to  contribute  by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be. Perhaps you'd like to join me?   --00-- "Stop answering my questions with questions." Sure, I can do that. In return, please stop asking me questions so open to interpretation that any answer would be almost meaningless and certa

Testing (AI) is Testing

Last November I gave a talk, Random Exploration of a Chatbot API , at the BCS Testing, Diversity, AI Conference .  It was a nice surprise afterwards to be offered a book from their catalogue and I chose Artificial Intelligence and Software Testing by Rex Black, James Davenport, Joanna Olszewska, Jeremias Rößler, Adam Leon Smith, and Jonathon Wright.  This week, on a couple of train journeys around East Anglia, I read it and made sketchnotes. As someone not deeply into this field, but who has been experimenting with AI as a testing tool at work, I found the landscape view provided by the book interesting, particularly the lists: of challenges in testing AI, of approaches to testing AI, and of quality aspects to consider when evaluating AI.  Despite the hype around the area right now there's much that any competent tester will be familiar with, and skills that translate directly. Where there's likely to be novelty is in the technology, and the technical domain, and the effect of

Testers are Gate-Crashers

  The Association for Software Testing is crowd-sourcing a book,  Navigating the World as a Context-Driven Tester , which aims to provide  responses to common questions and statements about testing from a  context-driven perspective . It's being edited by  Lee Hawkins  who is  posing questions on  Twitter ,   LinkedIn , Mastodon , Slack , and the AST  mailing list  and then collating the replies, focusing on practice over theory. I've decided to  contribute  by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be. Perhaps you'd like to join me?   --00-- "Testers are the gatekeepers of quality" Instinctively I don't like the sound of that, but I wonder what you mean by it. Perhaps one or more of these? Testers set the quality sta

Postman Curlections

My team has been building a new service over the last few months. Until recently all the data it needs has been ingested at startup and our focus has been on the logic that processes the data, architecture, and infrastructure. This week we introduced a couple of new endpoints that enable the creation (through an HTTP POST) and update (PUT) of the fundamental data type (we call it a definition ) that the service operates on. I picked up the task of smoke testing the first implementations. I started out by asking the system under test to show me what it can do by using Postman to submit requests and inspecting the results. It was the kinds of things you'd imagine, including: submit some definitions (of various structure, size, intent, name, identifiers, etc) resubmit the same definitions (identical, sharing keys, with variations, etc) retrieve the submitted definitions (using whatever endpoints exist to show some view of them) compare definitions I submitted fro

Build Quality

  The Association for Software Testing is crowd-sourcing a book,  Navigating the World as a Context-Driven Tester , which aims to provide  responses to common questions and statements about testing from a  context-driven perspective . It's being edited by  Lee Hawkins  who is  posing questions on  Twitter ,   LinkedIn , Mastodon , Slack , and the AST  mailing list  and then collating the replies, focusing on practice over theory. I've decided to  contribute  by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be. Perhaps you'd like to join me?   --00-- "When the build is green, the product is of sufficient quality to release" An interesting take, and one I wouldn't agree with in general. That surprises you? Well, ho

Make, Fix, and Test

A few weeks ago, in A Good Tester is All Over the Place , Joep Schuurkes described a model of testing work based on three axes: do testing yourself or support testing by others be embedded in a team or be part of a separate team do your job or improve the system It resonated with me and the other testers I shared it with at work, and it resurfaced in my mind while I was reflecting on some of the tasks I've picked up recently and what they have involved, at least in the way I've chosen to address them. Here's three examples: Documentation Generation We have an internal tool that generates documentation in Confluence by extracting and combining images and text from a handful of sources. Although useful, it ran very slowly or not at all so one of the developers performed major surgery on it. Up to that point, I had never taken much interest in the tool and I could have safely ignored this piece of work too because it would have been tested by

Am I Wrong?

I happened across Exploratory Testing: Why Is It Not Ideal for Agile Projects? by Vitaly Prus this week and I was triggered. But why? I took a few minutes to think that through. Partly, I guess, I feel directly challenged. I work on an agile project (by the definition in the article) and I would say that I use exclusively exploratory testing. Naturally, I like to think I'm doing a good job. Am I wrong? After calming down, and re-reading the article a couple of times, I don't think so. 😸 From the start, even the title makes me tense. The ideal solution is a perfect solution, the best solution. My context-driven instincts are reluctant to accept the premise, and I wonder what the author thinks is an ideal solution for an agile project, or any project. I notice also that I slid so easily from "an approach is not ideal" into "I am not doing a good job" and, in retrospect, that makes me smile. It doesn't do any harm to be reminded that your cognitive bias

Test Now

The Association for Software Testing is crowd-sourcing a book,  Navigating the World as a Context-Driven Tester , which aims to provide  responses to common questions and statements about testing from a  context-driven perspective . It's being edited by  Lee Hawkins  who is  posing questions on  Twitter ,   LinkedIn , Mastodon , Slack , and the AST  mailing list  and then collating the replies, focusing on practice over theory. I've decided to  contribute  by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be. Perhaps you'd like to join me?   --00-- "When is the best time to test?" Twenty posts in , I hope you're not expecting an answer without nuance? You are? Well, I'll do my best. For me, the best time to test is when there

Vanilla Flavour Testing

I have been pairing with a new developer colleague recently. In our last session he asked me "is this normal testing?" saying that he'd never seen anything like it anywhere else that he'd worked. We finished the task we were on and then chatted about his question for a few minutes. This is a short summary of what I said. I would describe myself as context-driven . I don't take the same approach to testing every time, except in a meta way. I try to understand the important questions, who they are important to, and what the constraints on the work are. With that knowledge I look for productive, pragmatic, ways to explore whatever we're looking at to uncover valuable information or find a way to move on. I write test notes as I work in a format that I have found to be useful to me, colleagues, and stakeholders. For me, the notes should clearly state the mission and give a tl;dr summary of the findings and I like them to be public while I'm working not just w