Guilty Pleasure

In Episode 10 of The Guilty Tester podcast Dave Duke took questions from the Twitterverse. The first one was this:

Think of a piece of work you’ve done recently, what did you actually do to test it and looking back, what surprised you?

In his answer, Dave told a story about a recent series of bugs, his involvement in sharing knowledge about them, and how that prompted an unexpected fix. It's a nice story, but a different reading of the original question interests me too: how do we  test our own work? As he was asking for more questions, I suggested it to him, and he tweeted back:

That's an interesting question. Back at you though. Do you test your own work differently to how you test others? Do you practice what you preach? Is it realistic to expect people to test their own work the same as they test others? Is there an advantage though.

Juicy! I didn't want to spend loads of time trying to craft an answer into a sequence of 240-character soundbites but I liked the idea of thinking it through, so I gave myself an hour to write a response. This is it.

I like how you loaded your questions. Take "differently to" and "the same as." When I hear those terms, I want to ask you which dimensions you have in mind. On a very strict interpretation, an identity relationship, I'd say that testing my work and testing others' work is not the same. But then I'd also suggest that two testers given identical applications to test in identical environments would not test the same either. In fact, my instinct is that the same tester given the same application in the same environment on two different occasions would be unlikely to test the same.

But, without asking for your clarification, I can answer the question this way: I do test the same way, because I always test the same way. I spent time working out what testing is for me (the pursuit of relevant incongruity) and so, at a strategic level, if I'm testing I'm doing the same thing.  However I also never test the same way, because I strive to take the context into account when I'm testing. At a tactical level, the context for two pieces of work is rarely truly identical in all important dimensions, not least because I am in the context and I am not reliably identical over time.

Just semantics? Leave it aside then. What about my work and others' work? As a test manager, my work doesn't overlap much with the people whose work I test. Whose work do I test? I'd say it includes developers, business development staff, testers, technical authors, technical support staff, and others. What overlaps with what I do? Some examples would be when I code (tools, test automation), or test (features, ideas), or write (usually raw doc, process doc). These would be places where I might have an opportunity to test my work in the same way as I'd test others'.

What might stop me from doing that? Here's a few thoughts: I would expect that I already understand the motivation for the work, and for the approach I took with it, so I might not spend time exploring it. Likewise, I might not look again at the problem I'm trying to solve with this work and wonder whether other approaches could be better; in particular I won't know of approaches that the implementer didn't know about. I might not want to criticise the approach taken, because I know it'll mean more work for me. I will also likely be subject to the same biases I had when building the solution.

Given that, what kinds of approaches do I use to test my own work? I use writing a great deal to provide a way to externalise thoughts in a way that detaches them from me. I then find that I can challenge them in a way that's less easy when they reside in my head. It doesn't have to be prose (although I find that helps me to nail down what I think), a mind map or a table or a sketch can all help. I also seek review; I might ask for review of ideas and proposals before implementing something, or I might run some kind of retrospective afterwards, or both.

Over time I have found that working incrementally, and finding ways to cheaply check that I'm on track at each step, helps me to produce the kind of work I want to produce and interleaves the implementation and the testing of it. I like to monitor and review myself. I am a believer in open notebook testing, so I like to commit my code to a public repository, I post my test notes to our company wiki. I may not ask for review of these things all the time, but they are available for anyone who wants to look and I am open to suggestions on them at any time. I make myself available for questions, on any topic, and I try to treat all questions with respect and humility, in order that people are not discouraged from asking them.

Finally, do I practice what I preach? Unlike pretty much everywhere else in this answer, I can give you a resounding YES here! It's a point of honour for me not to be a hypocrite. I have high standards and a desire to improve tempered with a pragmatic edge and, particularly as a manager, I feel it's on me to measure up and demonstrate the kinds of behaviours I try to encourage in my team, and the value that it brings. I also don't want to lose the enjoyment, credibility, and satisfaction that I get from being a practitioner.

You might ask how I tested this essay. I gave myself 20 minutes to collect and categorise thoughts (a kind of  mini fieldstone approach) and then 30 minutes to write each of those categories into a paragraph. At each stage I was testing that the thoughts were coherent and that I believed them. For example, I asked myself how happy I was to commit to "I always test the same way" and looked for reasons why I might not want to. Here's one I considered: perhaps I'll change my definition of what testing is.

I then spent 10 minutes organising the paragraphs, again looking for coherence, repetition of words or concepts, flow, and, finally, mindful of my time box, I pasted it into Blogger from my text editor and formatted it. I've allowed myself a few extra minutes to find a picture and relevant links and do a final proof read. Apart from typos I won't change anything now.
Image: https://flic.kr/p/m2s7Q

Questions or Suggestions

Coaching, training, mentoring, teaching and all the rest then, eh? What are the differences? When do they matter? I'm interested in that stuff, because I find theory useful to inform my practice. But as a practitioner I want to prioritise congruence with the situation over being constrained by a label.

How can theory inform my practice? A book like The Coach's Casebook provides a set of scenarios, talks about possible coaching approaches to them, and reflects on a fictionalised coach's attempts to work through them. This provides me with knowledge of tools and areas of potential application. I can call on these when someone comes to me with a problem. (Which they do, often.)

What I generally won't be doing at that point is trying to open a meta-discussion in which we negotiate whether we should try coaching or mentoring right now. At best this can be clunky and a distraction for  the person I'm trying to help. At worst it blocks the conversation and maybe buries or worsens the problem.

I also generally won't be deciding solo and up front to act as a trainer, or a teacher because that's imposing a restriction that could hinder my ability to assist. In fact, it's my experience that techniques frequently associated with different roles can be valuable in the same conversation. That's why I favour congruence.

My ability to be congruent is enhanced if I can understand the needs of the person I'm talking to better and, while I won't negotiate the terms of a coaching agreement in the moment, I have found that asking "are you looking for questions or suggestions?" has proved to be a great shorthand: not clumsy enough to break the conversational flow, but distinctive enough to provide an opportunity to direct it.

Questions mean that I will attempt to help them to explore the problem and their views on it (and I'll often reference Weinberg's definition of a problem as part of that) while suggestions will see me proposing options: actions or perspectives, for example.

I used this approach in the mentoring experiment with Claire that we wrote up recently, and it proved  helpful. In that case, we had also discussed this particular point before we started working together:

Although we use the terms “mentor” and “mentee” we don’t have specific definitions in mind, and will accept what others might term coaching, teaching, mentoring etc as we feel is most suitable and acceptable at any given time.

In summary: for me, the hat being worn is usually unimportant compared to the strength and value of the relationship being built. Knowledge of tools is beneficial but understanding the need and reacting to it appropriately are crucial.
Image: https://flic.kr/p/XZy3r

Edit: In short Twitter conversations Gus Evangelisti and Adam Knight both made the point that there are times where it's important to be clear about the kind of relationship that's being established. I agree, and I really like Gus' blog post From Consulting to Coaching with its simple yet effective visualisations of different modes of engagement and how they might vary in applicablity.

Don't be a Prick

This is a joint post with Claire Banks about a mentoring experiment we conducted for six months last year. We describe how it started, how we organised it, how it went, why we stopped, and how we feel about it now.

Starting

Claire: I’ve been testing for more years than I care to remember. I am one of many that ‘fell into testing’ around the time of the hyped Y2K bug. I have held a variety of roles from being a test team member through to manager at numerous different companies and industries over the last 19 years.

Given this wealth of experience, why on earth would I want someone I can talk things through with and seek advice from? Surely I should know it all? Well, no. I don’t know it all and never will. Plus I have a confidence issue.

At CEWT #4 I presented on why I would never be a test manager again. I reflected back on my career and was struck by the realisation that I’d had:

• 16 years in testing
• 12 different companies
• 25 test managers
• 3 great test managers

WOW! Only three test managers that I had felt inspired by? F*ck!!

The very next day, and I really do mean the very next day, I had a 1:1 with my test manager, the greatest I have encountered to date. I was asked if I would consider being a manager, and given lots of reasons why the company felt I was the one they wanted.

I spent some time considering it, reminding them of the presentation I had just done and discussing my doubts. I shared my fear of failing, of being one of the bad managers that drives people to unhappiness and ultimately resignation.

I eventually agreed to give it a go. Then panicked. Then along came James riding in on a white horse to save me … OK so maybe not the white horse or the saving part.

I encountered James at the local testing meetups where we would chat about testing. I started to follow him on Twitter, read his blog posts, and attend his presentations. I felt inspired: he likes to challenge ideas and try out new approaches.

Some members of James’ team also attended the meetups and I would observe their interactions. This confirmed to me that I’d quite like James to be my boss! I got a genuine sense of respect, support and encouragement, basically great camaraderie that went both ways.

I figured that James must be a decent person. If he wasn’t going to be my boss then I decided he would be my mentor, in a stealthy way without declaring my intention. We met a couple of times over coffee and I picked his brains.

On the second occasion, over coffee and the world's rudest plant (see the picture up top), we ended up agreeing to trial a mentor-mentee experiment. I think we did really well to get to that point. I was making a conscious effort not to look at the plant, which of course meant I spent most of the time staring at it in absolute awe, imagination running wild, mostly distracted from the purpose of our meeting.

James: Claire was starting a new managerial role and looking for some advice. We already knew each other from local meetups and CEWT so Claire had some idea about where I was coming from. I can sum it up in a handful of principles:

• Be congruent: consider the other person, the context, and yourself
• Be MOI: provide motivation, organisation, innovation (and the occasional jiggle)
• Be clear and present: as open with information and as available as possible

Michael Lopp, who writes as Rands, is a technical manager that I have a great deal of time for. He sums his philosophy up even more tightly: don’t be a prick:

My definition of a great manager is someone that you can make a connection with no matter where you sit in the organization chart. What exactly I mean by connection varies wildly by who you are and want you want and, yes, that means great managers have to work terribly hard to see the subtle differences in each of the people [who] work for them.

I don’t want to be a prick, and it was clear that Claire didn’t either. I shared some experiences and perspectives and, as I’d been reading a lot around coaching, mentoring, and line management, a few references that had been useful to me.

Towards the end of the conversation we sort of slid into negotiating a mentoring arrangement. Claire says she had this as an aim all along, but I don’t recall that I took much persuading and I’m certain I didn’t have a white horse. I do recall the plant being a major distraction.

Understanding

During that first conversation we sketched out the kinds of terms under which we’d like to work together and later on created a Google doc to record our shared understanding. Here’s the main bits (all of the extracts from our agreement will be in blue):

We’re entering into a mentor-mentee (James-Claire) relationship on a trial basis for 6 months. At that point we’ll review whether we want to continue.

Although we use the terms “mentor” and “mentee” we don’t have specific definitions in mind, and will accept what others might term coaching, teaching, mentoring etc as we feel is most suitable and acceptable at any given time.

It’s important to emphasise that

• Claire remains totally in control of, and responsible for, her actions. She can choose to adopt, tweak, reject, or otherwise anything that James says.
• James will not judge Claire for what she does with anything he says.

If either of us decides it’s not working for us before the 6 month trial period ends, we can just stop it.

We commit to a 30 minute monthly meeting where Claire brings topics.

The content of all discussions are confidential except for explicitly agreed exceptions.
There is no need to provide an agenda beforehand.
If an agenda is provided beforehand there’s no requirement on James to have prepared.

We are open to receive feedback on any aspect of this process at any time.
We are open to give feedback on any aspect of this process at any time.
We accept that all feedback given will be given with good intent. We will strive to receive all feedback in that spirit.

James: I’ve mentioned already that openness is important to me and it was my proposal that we put down in writing what it was that we thought we were agreeing to. Reading it back now, it’s an interesting mix of what we’re committed to, for how long, how we could get out of that commitment easily, thoughts about observing and refining our interactions: elements of flexibility and safety for us both.

Claire: I didn’t really know what to expect from this. Looking back, the idea James had to capture a jointly-agreed framework was very useful. My thoughts and approaches can be rather unstructured at times, so setting mutually-agreed ground rules helped me focus my time in preparing for our sessions. I also liked that we could end it at any point with no hard feelings.

I didn’t go into this thinking it was all about me, but upfront I didn’t really appreciate how it could impact James nor the sense of responsibility he felt.

Motivation

The Google doc also recorded why we were doing it:

Claire: My spidey sense tells me that James is one of the good guys who has no ulterior motive for helping me out. I was trying to be stealthy in viewing James as a mentor during a previous chat, without actually telling him, so having James agree to do this is fabulous. I am going through some changes in work, therefore I would like to talk through any issues and receive guidance as applicable, via a non-work-related medium.

James: Because Claire asked. Because I like her and am keen to help her (and other people). Because I haven’t done this before and I enjoy trying new things. Because it doesn’t cost me much. Because I might learn something about myself, and how I work.

Claire: Stepping into the world of management yet again, I wanted to build a support network for the new challenges I would face and very much wanted James to be part of that. I had the support of my two prior managers but felt that as we were all at the same company, any advice would be skewed towards what is best for the company, not necessarily what is best for me, as a person. I felt James would be perfect in that sense as his only interest was me.

James: Pretty much all of my working experience has been at Linguamatics in a context that I helped to create. I’m not sure I’d ever knowingly met a software tester when I started the Test team there. Amongst the reasons that I appreciate the local and wider testing communities are that I can take ideas from them and that I can test my ideas and experiences against them.

Claire says my only interest was her and it’s true that helping her was a strong motivation for me. However, this experiment was also a chance to see how well approaches that I’d evolved at work could generalise to someone that I had less knowledge of, in a context I had only a restricted view of, and no way of influencing.

Success

Asking what success looks like could have generated simply a restatement of our motivations, but perhaps we got something extra by asking the extra question:

James: Success will be that Claire has got net value from it, and can tell me what that value is. I’d like to have learned something about myself, and find that we haven’t fallen out.

Claire: Opening myself up which I don’t find easy, to someone I don’t really know but instinctively trust. To better equip myself to deal with situations. At the end of this I would like to feel more confident about tackling issues instead of running away and also learn more about myself.

James: In hindsight I like that my first measure of success is success for Claire, and that the second is that I can understand what she thinks constitutes that success. The most important thing really is her, but I want to get something for me too, something that will help me to see how the actions I took had some effect, which might help me to choose those actions, or some others, in future.

Claire: As time went on I found it easier to open up to James. This is most likely due to feeling that I knew him better and growing in trust.

I have learnt the following about myself:

• Recognising a destructive behavioral trait. When I encounter something new I automatically put up barriers to failure. If I then think I can't do what is required I find myself lost in negativity. The effort to overcome this is hard, I don't tend to enjoy the experience and it probably takes longer to accomplish the task in hand.
• I am listening more, asking more questions and challenging statements in others. This has proved quite effective in uncovering underlying issues that aren't being said that can then be addressed.

An unexpected bonus is that I’ve had a positive effect on other people. I was hoping that I would learn new ways to respond to situations, but didn't consider how my behaviour towards others helped them feel positive, empowered even, when tackling their own problems. Being told by someone that I have really helped them gives me face ache from all the smiling I do.

Iteration

Along the way, we found things we thought it would be interesting to add to the doc. This is what we said we feared:

James: I have a few fears: I give some advice that’s taken but that turns out bad; I come across as patronising; I lose Claire’s friendship; I have nothing to give when Claire has a really pressing need.

Claire: I fear James thinking I’m stupid, immature and that I must be a rubbish tester based on the situations I will seek advice for. I fear opening up and being completely honest, but I have to do this to benefit. I may come across as whiny and needy. I may not have anything to discuss.

After two or three meetings, we added another monthly strand specifically on reflection, interleaved with the original strand, so that we were meeting every couple of weeks:

The reflection meetings have a specific requirement: Claire brings with her, written down (so that she is forced to think it out), thoughts on what we said two weeks before:

• the problem
• what changes you made (or deliberately didn't)
• what outcomes you observed
• how you feel about it.

Claire: Very early on, James let me know that it would be OK if I emailed him with any issues in between our scheduled meetups. I decided against doing this as I wanted to stick to our agreed schedule and not become a drain on his time.

I prefer to look at people when I’m talking to them. I felt that walking side by side didn’t allow me to do this as I was focusing on the pavement in front, checking for trip hazards and poop. I was on the verge of suggesting we stop walking and sit and chat over coffee before realising that being static and having more intense eye contact would probably make me feel uncomfortable. I gave James feedback on this about half way through but decided against suggesting we reconsider it, so we continued with the walk and talk approach.

James: One of the fears that I mentioned was coming across as patronising, but with hindsight I could have generalised that to any number of traits that I might, after Rands, summarise as being a prick: things like not listening, not adjusting to Claire’s context, not caring, monopolising, mansplaining, gaslighting, not adjusting to Claire’s mood or other signals.

I am sensitive, perhaps over-sensitive, to this kind of thing. Those who’ve worked for me may not see it this way, but I strive to be careful about when I give advice, particularly unbidden.

Stopping

We agreed at the start that we’d run the experiment for six months. In the end it was a little longer, but we also missed a couple of meetings due to holidays and work commitments so it evened out. Like everything else we decided, we recorded our agreement in the doc:

We discussed whether we wanted to carry on. Claire was concerned that she wouldn’t have enough to ask about in future so we’ve agreed to stop regular meetings and instead have on-request ad-hoc ones whenever Claire wants them.

James: As it happened, we didn’t have any more mentoring meetings. I had wondered a little selfishly whether I might get some kind of ego deflation and find stopping difficult. Perhaps I would worry that there was something I should have done differently, that would have made my advice strong enough that Claire wanted to carry on.

In the event, I found that I was able to let it go easily. Our initial agreement was clear that Claire was in control of what she did with the advice, and that must include not wanting it at all. I liked that we left the door open to continue, though.

Claire: I felt relief once we had concluded our last scheduled meetup. This wasn’t a reflection on my time with James and I left the door to continue ajar. I was running on empty. I had been quite unhappy for a while and hadn’t shared the depth of feeling with James. I was figuring out what changes I should make and looking towards a new path. Therefore, it seemed to me I had reached a natural end that fulfilled our agreement.

I was confident James would be there when I was ready to re-open the door. Which I have subsequently found out he is :-)

Retrospection: Claire

Overall, I found the sessions very helpful, especially as James had no personal or work gain that may have influenced his support. I came away each time feeling better about a situation, either in terms of having reading material recommended or having James question/challenge some aspect of my issue that made me think about it in a different way.

I had slight difficulties throughout with regard to balancing the amount of information I could give. I had to ensure I wasn’t compromising anything confidential in relation to the company or individuals that James may know from the local tester meetups.

Presenting my problems was difficult the first couple of times as I was convincing myself that James would think I was a testing fraud. I have enormous respect for him and wanted to portray myself in a certain light. I soon stopped worrying about this, but did remain conscious throughout the remainder of our time that I was being overly negative. But then I figured if I was so awful he’d stop meeting up with me :-)

After one session where I had talked about a particularly difficult task I felt I had performed poorly on, James emailed me to say well done and that I should feel good about what happened. That was unexpected and made me feel good about what had happened.

Adding in the reflections part was a good idea as it made me think about the choices I did or didn’t make and the reasons for those decisions. It's a pity this wasn’t included from the outset. What a marvellous thing hindsight is.

James never gave me confirmation in anyway that my fears had come true. However, I still inwardly chuckle at a comment he made towards the end of our time. I find it rather hilarious and it went something like this:

James: You should consider taking on three new mentors.
Claire: (Pondering over the significance of three) Why's that?
James: Because you need a lot of help!!!

Here’s my thoughts on attributes to look for when considering choosing a mentor, in no particular order:

• Honest - someone who will tell you if you're being a prick.
• Asker - someone curious to question and tease out what isn't being said.
• Relevant - someone experienced in the same area.
• Listening - an active listener who can summarise their understanding.
• Objective - has no real stake in anything other than you.
• Trust - someone you can feel safe, comfortable and vulnerable with.
• Steer - to be guided by, rather than dictated to.

Retrospection: James

Reading it back, I like our initial agreement very much. We didn’t change the basic terms of it at all, only adding things that we thought could be useful as we went. Time was a major constraint on us both, so we arranged for the collaboration to not take much time. It’s interesting that the shape of the agreement reflects that without ever making it explicit.

In fact, this was an extremely cheap experiment for me: we met for 30 minutes once or twice a month while walking around the Cambridge Science Park. I do that walk most days after eating my dinner anyway, so I didn’t have to find a new time slot. Beyond looking up references for stuff I had recommended, there was little or nothing to do between times.

This lack of effort shouldn’t be taken as lack of commitment. I want Claire to succeed and I was focussed on her for the entirety of the time we spent together. But from a practical side, we both had to fit this in around our working and private lives, and we showed ourselves that we can get benefit from a small investment like this.

I am an inveterate note-taker, but I deliberately didn’t take any notes about my meetings with Claire. Why? Several reasons, of which the first is time: I was trying to squeeze my commitment down to the least possible while still being valuable. The second is confidentiality: no record means less likelihood of an accidental leak.

The final reason is that it challenges me; notes are a safety net of sorts and without them I have to either find a way to remember or accept that I am likely to forget details, and that that might be exposed. In this relationship, the mostly episodic nature of the questions meant that keeping a running context wasn’t so important and so no notes (until now, trying to remember details about what we did) wasn’t an issue.

Claire liked the reflection meetings, and I did too. A couple of weeks proved to be a good length of time for keeping the advice in mind, deciding how to act, and seeing some results of that action. Even here I found that my lack of notes wasn’t too important as Claire generally needed to summarise the problem before explaining her approach to it.

My fears never really came to pass: I was worried that we might fall out and we didn’t. I also don't recall not having something to say when asked. I did frequently have concern that what I was saying wouldn’t be relevant to Claire: she has mentioned her tendency to negativity, and it can be hard to make suggestions to someone who, in the moment, is prone to reject. I found that the assertiveness training I’ve had was useful here.

Claire suggested early on that we might blog or talk about what we were doing and we sketched out some terms for it, as you’d expect. Writing is my favourite way to reflect, but reflecting in writing in tandem has been particularly interesting.

We’re working in a shared Google doc and are having side conversations in chat about the content as we go along. Claire has said things in here that I didn’t know until now, for example that she held back her depth of feeling about her position and that was part of finishing the experiment for her. I would definitely consider writing a joint document as a debriefing tool in future.

Conclusions

Claire: This has been a positive experience for me and I’m glad we did it. I actually can’t remember if I ever thanked James. I’m sure I must have, I am overly polite... but... THANK YOU!

I feel I have gained a friend. James has been a good mentor for my special brand of crazy. For all the reasons above and probably additional ones that I’ll think of as soon as this blog post is published.

A small, personal measure of success for me is that it’s been about one year since my transition to management and no one on my team has quit. Whilst I don’t conclude that alone should grant me access to the elusive Good Managers Club, I’d like to think I haven’t been awful and screwed up anyone.

I’m a big fan of lists, rules and order so having the agreement upfront meant I knew what the parameters were and could play safely around within these.

As this has been a positive experience for me I feel that I would like to give something back. I don’t have the confidence to offer my services as a mentor for anyone though, heck, I don’t even have the confidence to attend the Cambridge Software Testing Clinic as a mentor! I guess for now I’ll just offer my services as and when requested by my colleagues and see where that goes.

James: In mentoring Claire I was hoping to learn something about myself and in particular about whether my experience, thoughts, and advice could travel well outside of my own context.

I’m happy from Claire’s reactions that they can, although I’ve done enough of this kind of thing with enough people now to know that what works today might not work tomorrow and that what works for Bill might not work for Betty. If there are meta rules that I’m following they probably run something like this:

• Frame suggestions as experiments.
• Explain that you are experimenting.
• Explain why you are experimenting.

How might that look in real life? Maybe along these lines: I don’t have a good answer but, in a similar situation a few years ago, I tried solution X with someone else. It worked out well for them, and it seems possible it could fix the problem Y you’re describing. How would you feel about trying it for a couple of weeks and then reviewing?

I’m happy that the personal management principles that I outlined at the top were useful in this mentoring relationship. They are heavily influenced by Jerry Weinberg’s thoughts on management and have been instrumental in guiding how I behave at work.

I found that I was taking things from work to try with Claire, and vice versa. For example asking “would you like questions or suggestions?” as a kind of short-cut to coaching or mentoring approaches. It was great to be able to exercise this kind of stuff with someone who is not in a formal working relationship with me — particularly one in which there is a hierarchical element — and who has no reason not to tell me to shut up or get lost if what I’m saying makes no sense to them.

Claire has complimented me a few times in this post, and I’d like to say something back: it was extremely brave of her to ask for help, particularly knowing how exposed it would make her feel. I may have offered thoughts, but all of the actions that motivated the positive results she describes here were hers.

Claire and James: We were both clear that we didn't want to be pricks and, thankfully, the closest we came to a prick during the entire experiment was that plant on the table right at the beginning.

Team Values: Reflection

The testers at Linguamatics decided to explore the adoption of a set of team values and this short series of posts describes how we got to them through extended and open discussion.

If you find the posts read like one of those "what I did in the holidays" essays you used to be forced to write at school then I'll have achieved my aim. I don't have a recipe to be followed here, only the story of what we did, in the order we did it, with a little commentary and hindsight.

• Introduction
• Why?
• Teasing Them Out
• Living With Them
• Reflection

--00--

We took our own sweet time to arrive at a set of team values. That's okay with me. It felt like a good pace for us given all of the variables: the ongoing work, the people, the novelty, and the uncertainty of the outcome. We took small steps, with space for review, with clarity on the next stage, and with consensus.

We've now been living with our values for a few months and I thought it might be interesting to wonder whether the questions we asked ourselves back at the start were helpful, and whether we resolved them. They were:

• Why are we interested in doing this?
• What do we want to achieve with it?
• How will we go about achieving it?

For me, those questions framed the whole arc of our approach. The order feels logical for this kind of endeavour and provided us with a grounding that we could look back to if we needed it. That's not to say that the Why and What we created were complete, nor that we wouldn't change them. No-one with any experience and sense of reality about the world would expect perfection up front ... would they?

Interestingly, the Why and What were relatively easy compared to the How. We quickly established good arguments for a set of values, and also the kinds of actions that we felt would motivate those values. Working out how to achieve it took a lot more and was, it seems to me, separated into two distinct parts: what mechanisms will we use to get to a set of values? and what should we do with our set of values to get the benefits we want? We managed to evolve an approach for the first question, with a lot of conversation along the way. We're still in the middle of the second.

Did we achieve our aims for the values, then? I think we did, in the sense that the values we generated do feel like they cover the areas we wanted, namely to:

• Encourage
• Emphasise
• Empower
• Explain

We are encouraging approaches that focus on risk and value; we are empowering ourselves to find ways (and time) to improve, and to feel safe to speak and act; we are setting ourselves up to explain what we're doing and why; and we are emphasising that all of these things are, to us, productive and valuable ways to go about our business.

A important question: could we have achieved these aims in a different way? There is no doubt about it, we definitely could. This was one path through a massive space of possible approaches and we negotiated it reasonably cautiously. It felt crucial to me that we didn't lose anyone en route if these were to be shared values, but getting unanimity from a group of fifteen people is not always easy.

Related: could this approach have achieved different values? Again, there is no doubt about it. A different set of people, in a different mood, on a different day, with a different facilitator or facilitation could easily have taken another route. And I think that it's possible that we'll revisit and change the values ourselves.

As I've said, we didn't rush into these values. It was my intuition at the beginning that slow and steady, with plenty of scope and time for checking and chewing over, was the way to go. The elapsed period was around 8 or 9 months but at any given point we didn't require much time from each of us and, when we did commit time, we were committed to the task.

I don't recall ever stating it explicitly, but one of my high-level goals was to try to keep the mechanics of the process out of the process as much as possible. When we got together to discuss this stuff, I wanted to be talking about the values much more frequently than how we would arrive at them.  And how we arrived at them was incrementally.

I might say that we were feeling our way into a process. You might say that we were (and particularly I was) making it up with each step. It wouldn't take much persuasion for me to agree that was a fair description too. But it wasn't a dictatorship: we talked about the process over the content at times, for example to decide that attendance would be optional and to time-box meetings at 30 minutes.

In a story of values, it'd be good to wonder whether this effort was valuable. Instinctively, I do feel that it was but I don't have an objective measure to judge it by. Subjectively, I think that we understand each other and our attitude to our work better. I get the feeling that, despite our differences — and believe me, we are very different people — we share a lot of outlook, desire to help the company craft great products, and care to do a good job inside business constraints.

It occurred to me at several points along the way, and again while writing up, that there's a strong analogy to the software development business here. In this case, our team took the role of stakeholders, implementers, and end users. Needing to state what we wanted, having to arbitrate in conversations between stakeholders, agreeing to relax requirements in the name of producing a thing, actually producing a thing, showing a thing to stakeholders, having a thing criticised by users ... all of these are roles performed by people we have to deal with all the time. And it doesn't hurt us to walk in their shoes from time to time.

On a personal level, in writing this I wondered whether there were in fact two stories, or perhaps three, intertwined: the values we got, the way we got to those values, the way that I wanted to participate in getting us to the values.

I briefly wondered about trying to disentangle them, but found that it read better to me with my commentary combined into the conversation itself. If I had to pull a few guiding principles out, I'd say these were core to me:

• Not getting bogged down with process; as long as we were talking productively we continued.
• Accepting that we could leave some aspects ambiguous; until it's critical to resolve them.
• Giving all a chance to be heard; and feel they're being listened to.

I have been on this kind of journey before to some extent, as a solo traveller. I have worked through what my management principles are, and encouraged the line managers in the Test team to do the same. I've also organised a cross-company sharing of management principles where many of the line managers presented three things that they felt were fundamental to how they work.

Also, a year or two ago I found myself working through a definition of testing which involved many entertaining hours pondering semantics and my own perspectives on my work. The cut-and-thrust of that propose/challenge/respond dynamic, happening with two instances of me inside my own head, gives me enormous amounts of enjoyment. But it's not to everyone's taste and perhaps not so suited to a large group.

On the other hand, the benefits that I have accrued from having a definition was something that I thought and hoped could be more generally acceptable: as a reference point, an anchor, for decision making. I ask myself: should I be testing now? If the answer is yes, then I can see whether or not the activity qualifies as testing for me. If it doesn't then I have to consider my priorities. If it does, then I can wonder whether it's the right testing.

We can do the same thing as a team, using our values as a motivator, as an envelope, as a guide. But in all these cases, it must be heuristic — there may be circumstances in which we choose to do something else, but at least we'll expect to have taken a deliberate decision to do that, and understand why.

I have found that reference to our values in conversations about priorities or how to react to circumstances can be extremely helpful. They might not tell us precisely what to do, but do give us guidance in the direction we should be looking to take. As I've repeatedly said (and it bears repetition) the conversations around what we're about were immensely informative and pleasurable devices for building a shared understanding, team spirit, and empathy.

I'd like to thank: the Test team at Linguamatics for making and participating so deeply in such an interesting project, and crafting this set of tools for ourselves; Daniel Karlsson and Keith Klain who were kind enough to give me some of their time and background in what they were trying to achieve in their own similar projects.
Image: https://flic.kr/p/oGMUQ

Solid State

Respect can be earned in many ways, with reliability, consistency and longevity among them: say you'll do your thing, do your thing, improve your thing, repeat, repeat, repeat.

And so respect is due to the organisers of the State of Testing survey for their continuing efforts to record annual snapshots of our industry.

The 2019 survey is open now and results from the previous five iterations are still available.
Image: https://flic.kr/p/auDHZX

Team Values: Living With Them

The testers at Linguamatics decided to explore the adoption of a set of team values and this short series of posts describes how we got to them through extended and open discussion.

If you find the posts read like one of those "what I did in the holidays" essays you used to be forced to write at school then I'll have achieved my aim. I don't have a recipe to be followed here, only the story of what we did, in the order we did it, with a little commentary and hindsight.

• Introduction
• Why?
• Teasing Them Out
• Living With Them
• Reflection

--00--

Having got to a longlist of seven values that we were struggling to cut down further, we decided to live with them for a while and then see how we felt. I made small cards listing the values and short descriptions and distributed them to the team. I also made the values prominent on our team's pages in the company wiki and wrote about our experiment with them in our internal company newsletter.

We waited three months or so before getting back together for a kind of retrospective. Gratifyingly, we found that we were in agreement on general comments like:

• There are broadly positive feelings towards the values.
• They are hard to remember, perhaps because there's seven of them.
• They reflect what we do rather than challenge us to do more.
• They could be more goal-directed, or perhaps have a mission on top of them.

Some specific comments and questions reflected shared feelings too, including:

• Risk-driven should include some strong relationship with the mission.
• Should we say something about how we want to use the values, e.g. can we take them as an open invitation to give each other feedback?

Unfortunately, despite the consensus on how we felt, there was little convergence on what we should do. So we agreed to spend another month living with the existing list and then reconvene with at least one concrete suggestion per person for how we'd like to proceed.

The requirement to come with a suggestion was intended to seed discussion and help break the deadlock. One of the team later told me that it was a strong disincentive for them: because they had no suggestion they thought perhaps they shouldn't or couldn't attend. I can see that and, if I wanted to use this tactic again, I'd think very carefully about how I phrased it.

A month later and we were off to a running start with a bunch of proposals for our next move. Here's a sample:

• Reduce the list to risk-driven, continuous improvement, collaboration. These are the three that I find myself reaching for most often. 
• Introduce the idea of advocacy.
• Put transparency either in one of the categories mentioned, or as its own category.
• Add something concrete about testing, to make it clear that we're serious about being the testing experts in the company.

We kicked the suggestions around for a while, and came to some joint decisions:

• Accept the proposed three values.
• Rewrite the definitions to include advocacy, transparency, and testing expertise.
• Make the definitions shorter.
• Introduce some kind of mission statement to frame the values.
• Check in with our colleagues who hadn't attended this meeting.

One of the team offered to help with the editing, which I appreciated. We both found the task challenging but I think that two heads helped to reduce the personal bias I would certainly have introduced by doing it alone.

To avoid it turning into some kind of wordsmith marathon we defined a time-boxed method to follow:

• Each take 30 minutes to draft proposals.
• Get together for 30 minutes to try to merge them.
• Brief email redrafting,
• Review by one of our technical authors.
• Final redraft.

The 30 minute merging overran as we strove to find a way to bring together two different versions of each definition. In the end we found that (a) radical candour and (b) writing both definitions in parallel on a whiteboard in different colours and then annotating them by crossing out or borrowing or rewriting together worked for us.

We shared the final result with the team, no-one dissented, and this what we've been working with for the last few months:

Mission: We regard ourselves as testing specialists and aim to do relevant testing in any given context to support our colleagues and the business in providing value to our end users.

We will always do our best to be ...

• Risk-driven - by taking actions that uncover relevant information and advocating for outcomes that mitigate risk or add value.
• Continually improving - by increasing expertise in our craft through experimenting, learning and sharing.
• Collaborative - by working with our colleagues in a friendly, non-judgemental and open way.

In the next post I'll reflect a little.
Image: https://flic.kr/p/oGMUQ

Team Values: Teasing Them Out

The testers at Linguamatics decided to explore the adoption of a set of team values and this short series of posts describes how we got to them through extended and open discussion.

If you find the posts read like one of those "what I did in the holidays" essays you used to be forced to write at school then I'll have achieved my aim. I don't have a recipe to be followed here, only the story of what we did, in the order we did it, with a little commentary and hindsight.

• Introduction
• Why?
• Teasing Them Out
• Living With Them
• Reflection

--00--

So far we've thought of a load of reasons that a team might want values, and loosely grouped them. That's the Why. Next step: the What.

In a second meeting following the same lightly-facilitated format as the first, we asked ourselves what kinds of values might be good fits to each of our four motivational categories: encourage, emphasise, empower, and explain.

We were flexible about what constituted a value, accepting philosophies, actions, outcomes, or whatever. We aimed to get the ideas out without too much filtering and, again, we used a mind map to document our balanced, open discussion.

It was sometimes hard to fit suggestions into our categories, both because the categories overlapped and because none of the categories seemed appropriate. We didn't sweat it; so long as the suggestion was recorded it could go into any of the four, or a fifth, Other, that we added as we went. These are a few examples:

• Encourage: communication, integrity
• Emphasise: expertise, risk-driven
• Empower: continuous improvement, safety
• Explain: transparency, possible outcomes
• Other: openness, non-judgemental

We mostly didn't record detailed justifications for assigning a value to a category. At the time that felt right: did we really want to break the flow of the conversation or increase the paperwork burden? (My answer: no.)

A consequence of this is that I can no longer tell why, for instance, we didn't we make openness and transparency a single entry. Maybe there was a good reason, maybe it was oversight, or maybe I transcribed things incorrectly. Regardless, what we had was good enough to move forwards in the moment, and with momentum.

Following another period of reflection we gathered again to choose the subset of the values which would become our focus. We had generated 40 or 50 and I judged that a simple discussion would not easily get us to consensus. So I created a spreadsheet listing all of the values and we split into small groups to rank them, expecting that the top-ranked values would encode those that we felt most strongly about, for whatever reason.

Interestingly, and painfully for me, each of the groups attacked this task with a unique twist: some provided a strict ranking that represented their combined opinion, some scored individually which meant multiple values could have the same ranking, some simply selected the values they felt were important without any scoring at all, some chose a few values and some chose many, some merged values together and scored the combination, and the merged values were not necessarily the same across groups.

The effort in that session was deep and the conversations were sometimes quite intense. We'd hoped to combine the results on the day but, due to the complexities of the varied ranking schemes, we didn't even try. So, instead, we shared all of the results back to each other, with a little commentary from each group, and ended the meeting.

For me, ranking at this point was productive but in retrospect I think it could have been easier to have proposed or imposed a more specific ranking scheme. Of course, this might have necessitated a meta conversation around ranking, which itself could have been difficult to resolve.

Systemic discrepancies aside, there was real, countable, actionable data generated in those ranking conversations and, to do justice to it, I invested time in looking for ways to aggregate it fairly. In the end I presented several different normalisation strategies back to the team.

The simplest merely counted the number of times a value was ranked at all, while more complex attempts scaled everyone's rankings to have a total "score" of 1, with weighting for merged items. I also created a graphic which showed visually which group had ranked which values, and which values had been merged.

This visualisation was helpful because it clearly showed clusters in our categorisations and in our choice of values. The numerical analyses were interesting because they turned out to be quite consistent about the smaller set of values that scored well. They were:

• Risk-driven
• Collaboration
• Communication
• Experimentation
• Non-judgemental
• Continuous improvement
• Expertise

After agreeing that the analysis was reasonable we decided to take this set of seven values and begin to develop a non-shallow understanding of what we meant by them.

The same groups as before each came up with short definitions for a value or two and we debated them together. Lesson learned: nuance is everything.  My role here reverted back to light facilitation, recording outcomes, keeping time, and aiming to ensure that all voices had an opportunity to be heard.

It was delightful to witness the conversations we had across the weeks. Even if we had eventually failed to arrive at a set of values, the dialogue and the engagement with which we were having it (most of our team attended most of the meetings), felt like a valuable way to share perspectives and build empathy with one another.

Outside of the meeting cycle I composed revised definitions from my notes, each with associated heuristics for their use that had emerged from the debate. Here's an example:

Risk-driven: Use relevant data to try to identify the important risks, their likelihoods, and their impacts, and deal with them in a reasonable priority order.

• Collect data from multiple sources, including stakeholders, the context into which the product will be placed, changes to the software, ourselves, ...
• Consider risks to the company reputation, the bottom line, the product, the customer, the customer data, ...
• Strive not to do this work in isolation from others.
• Recognise that we won't always find all of the risks, and we won't always assess those we do identify correctly.
• Be alert to the possibility that our analysis can change based on new data.

In the next review we agreed that we'd captured reasonably well something that the group as a whole could agree with. Yay! We then tried and failed to reduce the set further. Boo! But pragmatism reigned and we decided that we'd live with the large set for a while before thinking again.

Stopping seemed like a good choice to me. Despite the enthusiasm and continued strong level of participation across the team, I was wary of us becoming fatigued with the effort. Now felt like a good time to put the values into use, gauge our comfort with them and then iterate on the basis of experience.

In the next post I'll talk about that experience and how it resulted in us cutting our list down to three core values.
Image: https://flic.kr/p/oGMUQ