Skip to main content

Farewell AST

After four years, three of them as Vice President, I'm standing down from the board of the Association for Software Testing.

Let me say up front that I am an unapologetic romantic about my craft. (And, yeah, I called it a craft. Sue me.) I believe in what AST stands for, its mission, and in context-driven testing, so it's been an absolute privilege to be involved in running the organisation. It's also been fun, and full of difficult situations and choices, and hard work on top of family life and a day job.

There also was the small matter of the global Covid pandemic to deal with. The immediate impact was on CAST, our annual conference, and in some ways the beating heart of the AST. We had to variously cancel, reschedule, and move CAST online and we are still experiencing the after-effects as we organise the 2023 in-person event.

So why am I leaving? Well, first, I'm not leaving the organisation, only the board. I am a life member and I hope to remain an active one. But, second, I have decided not to stand for re-election because I'm tired and my family situation has become more complicated recently. I can't commit the kind of time and energy that I would like to, to do the kind of job I want to do.

And what kind of job would that be? I reflected on the question while making my decision and also a couple of years ago when I wondering whether to stand for a second term.

When I joined the board I saw that we needed to strengthen our operations in some key areas to remain functional: membership management, internal operations, and value proposition. These are fundamental to any member organisation, particularly one staffed by volunteers. 

Given that, I have focussed on foundational changes; things that I hope will help AST to operate with reduced friction, and to demonstrate who we are, what we do, and why, to existing and potential members. 

I've listed some of that work at the end here and in retrospect it looks like a lot. Frankly, it is a lot and I'm very proud of my contributions. But it's also not just my work. I definitely brought energy and the skills to analyse, advocate for, plan, co-ordinate, and implement some projects, but my colleagues on the board and volunteer members also have those kinds of skills and provided all kinds of input, feedback, assistance, and support along the way too.

Not only that, but the AST board is run on a consensus basis. We talk about what we're trying to do and why, and what our options are, and come to a group decision. My sense is that this gives the group greater cohesion and helps us all to feel involved despite the physical distances between us. 

There are also significant areas I didn't touch directly, such as CAST and BBST training, that others were taking care of in parallel. And I don't want to give the impression that AST is a basket case. It's not, but it does have a legacy orgbase with all of the things-are-the-way-they-are-because-they-got-that-way quirks that you'd expect.

I'm not so vain as to think that the work I did is done done. I reckon AST has a more stable, workable, underpinning than before I joined, but I'm looking forward to seeing how the testers who take the organisation forward can reinforce, replace, extend, and build on top of it.

I wish good luck to all the continuing board members, and to the candidates in the current elections. I hope the next board has the same level of enjoyment and satisfaction that I've had doing this job.

Oh yeah, there's one other thing I got from AST that I'm exceptionally grateful for: being introduced to tater tots at a Tex-Mex joint in Atlanta during CAST. Man, I adore those crunchy potato love bombs and, after I've had a break, if anyone is interested in forming the Association for Scarfing Tots give me a shout.

--00--

This is a list of stuff I've done at AST that I made while I was thinking about this post.

Membership Management

It may seem obvious to say that members are the lifeblood of a membership organisation. It bears repeating, though, and it motivated a lot of my work. When I joined the board we didn't have much visibility of our who our members were and who they had been. I built and incrementally improved semi-automated regular reports that have helped us to understand that better.

This lead to the revision of our membership offering, from a one-price-fits-all approach to pay-what-you-can-afford with multiple price points for the same member benefits. Significant effort on this project was trying to first model, and then later monitor, the kinds of benefits we hoped for, such as increased membership from countries with lower average incomes or testers with less-senior role titles.

It's hard enough proposing potential approaches, trying to understand how they might work relative to one another and the existing system, and negotiating agreement from the board. Add on top of that the problems of migrating to the chosen system and keeping all relevant parties informed along the way and you'll see that this was a big undertaking.

To make things worse, the software we were using for membership management had been ancient for a long time and wasn't serving us well, but the mammoth task of researching alternatives, choosing one, and then migrating to it had been postponed repeatedly. I took it on, and now we have a functional and more modern system for managing membership, subscriptions, and events.

Organisation

Being nearly 20 years old, AST has had time to accumulate a lot of cruft anywhere that we store stuff. We've all seen this kind of thing, probably on a wiki or file share at work or searching for docs for some tool online. It's kind of bearable until it reaches a critical mass where it's hard to find what you need, where multiple approaches to the same task have been used in different places, and where it's clear that whoever was here before you didn't try to clean up. I cleaned up our Google drive.

Our web site had similar problems and I worked on that too. I used what analytics we had to try to optimise pages that were being hit most frequently and some basic SEO to try to promote the pages we'd prefer to be hit more often. 

I ran a volunteer project to remove the worst of the mess as well. We deleted over half of the pages, rationalised the site structure, standardised the layout of related pages, and made some templates for pages that we create on a regular basis. We discovered a bunch of unexpected problems along the way, such as broken spam filtering plug-ins on our contact forms, which we fixed as well.

As a remote-first organisation, and one in which the personnel changes regularly, and one in which some tasks only happen annually, it can be hard to find opportunities to share knowledge about how we do things. This is one of the causes of the cruft problem. 

To help overcome that, I wrote a lot of runbooks. I was very mindful that maintaining the runbooks shouldn't become a heavy maintenance burden in itself, so I tried hard to keep them to necessary and sufficient information, presented clearly. I also organised the runbooks for discoverability, using Google Drive symlinks to place them in a central location and in relevant folders for their task.

I've demonstrated and encouraged more use of the data we have for decision-making. Over the last two or three years we have developed a very rich financial model of our CAST conferences which is helping us to make more informed choices when it comes to costs and ticket pricing. Building this model and tuning it for sustainability is part of our one and three year goals, a simple planning and prioritisation framework we introduced at my suggestion. 

One of the perennial problems of volunteer organisations is the discrepancy between the number of good ideas and the time, energy, and resources to implement them. Having a process that explicitly nominates staggered goals, and reviewing our activities against them regularly, has helped to keep our focus on what we consider important.

Value Proposition

To help AST explain its value to potential members I researched the kinds of things that other, similar, membership organisations claim. This lead to a wide-ranging analysis of what we felt AST could claim and to what extent, what we felt that we couldn't claim and didn't want to, and what we couldn't currently claim but would like to. We were able to plan initiatives, such as initially writing a clearer set of membership benefits, based on the outcome of that work.

A pipeline of incoming members are one concern, but we also need to find ways to show existing members that we are here and doing worthwhile things. I think AST suffers from not making the most of the good work that it does. For example, running an interesting webinar but advertising it minimally or late, not telling members that it happened and what the key take-aways were, and not making it available on our YouTube channel despite having the recording. 

I tried to help change that culture by writing material for our monthly newsletter and prompting for it to be used, showing how Hootsuite could be invoked relatively easily to improve publicity reach, fixing permissions issues with our YouTube channel, and documenting how to add videos with minimal effort.

We also don't share or get the most from what we've done in the past. I implemented automation with Zapier to fetch an item from our archives and post it on social media every day. I also kicked off a volunteer project, which is still running, to create a GitHub repo and retrospectively collect presentations from CAST conferences and AST webinars in it.

I've attempted to set up some ways in which long-term value can be created too. I wrote an e-book, Peers Exchanging Ideas, about running peer conferences, publised on AST's GitHub repo. We used it for, and updated it after, a joint peer conference with BCS SIG which also produced a paper, Should the Public Care About Software Testing.

I came up with the idea for a crowd-sourced book, Navigating the World as a Context-Driven Tester, and collaborated with a long-term AST member, Lee Hawkins, to work out the details about how we'd like it to run. Lee's done a tremendous job over the last two years, and we've had contributions from over 60 testers from within and outside AST.

I set up and ran Steel Yourselves, an innovative webinar format in which testers challenge themselves to argue for a case they disagree with. I experimented with publicity materials and promotion too, introducing Canva for flyers and some simple configuration for emails to attendees before and after.
Image: https://flic.kr/p/kyG3nr

Popular posts from this blog

Meet Me Halfway?

  The Association for Software Testing is crowd-sourcing a book,  Navigating the World as a Context-Driven Tester , which aims to provide  responses to common questions and statements about testing from a  context-driven perspective . It's being edited by  Lee Hawkins  who is  posing questions on  Twitter ,   LinkedIn , Mastodon , Slack , and the AST  mailing list  and then collating the replies, focusing on practice over theory. I've decided to  contribute  by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be. Perhaps you'd like to join me?   --00-- "Stop answering my questions with questions." Sure, I can do that. In return, please stop asking me questions so open to interpretation that any answ...

How do I Test AI?

  Recently a few people have asked me how I test AI. I'm happy to share my experiences, but I frame the question more broadly, perhaps something like this: what kinds of things do I consider when testing systems with artificial intelligence components .  I freestyled liberally the first time I answered but when the question came up again I thought I'd write a few bullets to help me remember key things. This post is the latest iteration of that list. Caveats: I'm not an expert; what you see below is a reminder of things to pick up on during conversations so it's quite minimal; it's also messy; it's absolutely not a guide or a set of best practices; each point should be applied in context; the categories are very rough; it's certainly not complete.  Also note that I work with teams who really know what they're doing on the domain, tech, and medical safety fronts and some of the things listed here are things they'd typically do some or all of. Testing ...

The Best Programmer Dan Knows

  I was pairing with my friend Vernon at work last week, on a tool I've been developing. He was smiling broadly as I talked him through what I'd done because we've been here before. The tool facilitates a task that's time-consuming, inefficient, error-prone, tiresome, and important to get right. Vern knows that those kinds of factors trigger me to change or build something, and that's why he was struggling not to laugh out loud. He held himself together and asked a bunch of sensible questions about the need, the desired outcome, and the approach I'd taken. Then he mentioned a talk by Daniel Terhorst-North, called The Best Programmer I Know, and said that much of it paralleled what he sees me doing. It was my turn to laugh then, because I am not a good programmer, and I thought he knew that already. What I do accept, though, is that I am focussed on the value that programs can give, and getting some of that value as early as possible. He sent me a link to the ta...

Notes on Testing Notes

Ben Dowen pinged me and others on Twitter last week , asking for "a nice concise resource to link to for a blog post - about taking good Testing notes." I didn't have one so I thought I'd write a few words on how I'm doing it at the moment for my work at Ada Health, alongside Ben. You may have read previously that I use a script to upload Markdown-based text files to Confluence . Here's the template that I start from: # Date + Title # Mission # Summary WIP! # Notes Then I fill out what I plan to do. The Mission can be as high or low level as I want it to be. Sometimes, if deeper context might be valuable I'll add a Background subsection to it. I don't fill in the Summary section until the end. It's a high-level overview of what I did, what I found, risks identified, value provided, and so on. Between the Mission and Summary I hope that a reader can see what I initially intended and what actually...

Reasonable Doubt

In Your job is to deliver code you have proven to work  Simon Willison writes: As software engineers we ... need to deliver code that works — and we need to include proof that it works as well.  He is coming at this from the perspective of LLM-assisted coding, but most of what he says applies in general. I think this is a reasonable consise summary of his requirements for developers: Manual happy paths: get the system into an initial state, exercise the code, check that it has the desired effect on the state. Manual edge cases: no advice given, just a note that skill here is a sign of a senior engineer.  Automated tests: should demonstrate the change like Manual happy paths  but also fail if the change is reverted.  He notes that, even though LLM tooling can write automated tests, it's humans who are accountable for the code and it's on us to "include evidence that it works as it should." Coincidentally, just the week before I read his post I told one of my...

On Herding Cats

Last night I was at the Cambridge Tester meetup for a workshop on leadership. It was a two-parter with Drew Pontikis facilitating conversation about workplace scenarios followed by an AMA with a group of experienced managers. I can't come to work this week, my cat died. Drew opened by asking us what our first thoughts would be as managers on seeing that sentence. Naturally, sadness and sympathy,  followed by a week ? for a cat ? and I only got a day for my gran! Then practicalities such as maybe there's company policy that covers that , and then the acknowledgement that it's contextual: perhaps this was a long-time emotional support animal . Having established that management decisions are a mixture of emotion, logic, and contingency Drew noted that most of us don't get training in management or leadership then split us into small groups and confronted us with three situations to talk through: Setting personal development goals for others. Dropping a clange...

Great Shot, Kid

This week I've been playing with altwalker , a model-based testing tool. To get the hang of it, I attempted to build a very simple model of a workflow that is supported by the service my team owns. Hacking away at the example code, and looking frequently at the docs, I was able to get up and running in a few hours, creating: a basic model: nodes for system states, edges for operations simple assertions: mainly consistency checks on the states client: HTTP client to implement the operations against the service's API I configured this so that altwalker will perform a random walk of the model, starting state data is randomised, and the client will choose randomly whenever offered an option. Why so much randomness? Because it means that, over successive runs, more of the infinite space of possible workflow executions will be covered. Once I had that basically working I wrote a shell script that would run this loop a number of times: call altwalker ...

Bottom-up or Top-down?

The theme at  LLEWT this year was Rules and constraints to ensure better quality.   My experience report concerned a team I'd been on for several years which developed (bottom-up) a set of working practices that we called team agreements.   The agreements survived "natural" variation such as people leaving and joining and even some structural reorganisation which preserved most of the team members but changed the team's responsibilities or merged in a few people from a disbanded team. The agreements did not, however, persist through a significant round of (top-down) redundancies where the team was merged with two others.  I'm interested in thinking about the ways in which constraints on how people work affect the work and whether there are patterns that could help us to apply the right kinds of constraints at times they are likely to be useful.  I'm going to use this post to dump my thoughts. My starting po...

LLEWT 2024

This weekend I was at LLEWT 2024, a peer conference on Anglesey , north Wales, discussing communication. Given the day jobs of the participants, it was no surprise that the experience reports and the conversations that followed them mostly focussed on software development contexts.  Notes from my presentation are in Express, Listen, and Field . I made sketchnotes (below) for each presentation and a mindmap (above) to try to summarise the whole. Without much reflection yet, I guess I would pull these common high-level threads from the day: There are multiple reasons that communication fails  ... like, duh! ... but having multiple strategies for framing a message can help ... and having multiple tactics for delivering a message can help too. Understanding what you want from an interaction is key ... so setting the context to make that more likely is wise ... which might mean meta-conversation, being transparent, or changing your approach...

Exploring It!

This week the test team at Linguamatics held our first internal conference. There was no topic, but three broad categories could be seen in the talks and workshops that were given: experience reports, tooling, and alternative perspectives on our work. (The latter included the life cycle of a bug, and psychology in testing.) My contribution was an experience report looking at how I explore both inside and outside of testing. I've tidied up some of my notes from the prep for it below. There are testing skills that I use elsewhere in my life. Or perhaps there are skills from my life that I bring to testing. Maybe I'm so far down life's road that it's hard to tell quite what started where? Maybe I'm naturally this way and becoming a tester with an interest in improvement amped things up? Maybe I've so tangled up my work, life, and hobby that deciding where one starts and another ends is problematic? The answers to those questions is, I think, almost certai...