Hiccupps

I think a lot about the user experience when I'm testing . Not only for conformance with the UX guide we refer to but also in less tangible, subjective, respects such as how the software will  feel  to different kinds of users. The UX guide can't help here. Although it has value and the potential to save time - for example, by reducing the number of discussions about capitalisation policy for dialog title bars  to merely single figures per release - at close to 900 pages it is not a shining example of usability itself and it doesn't try to answer the question  who's a user?  In particular, it has only a handful of mentions of new users outside of the section on first-timers which itself is largely about setting up the product rather than using it. Inexperienced users will at various points form a significant enough proportion of your userbase that they merit special attention and when that happens, you'll need to think about lack of experience with ...

If extracting feature motivations, requirements and priorities from stakeholders is an art, presenting the analysis requires artfulness. The MoSCoW method suggests using English words M ust, S hould, C ould and W on't as an alternative to purely numeric priorities to make it more transparent that not everything listed will be delivered. But reversing the MoSCoW rules can help to obtain  the implicit prioritisations. You'll frequently hear "we must have a solution to X" or "Y should be improved in this release" or "we'd like to get Z into the payload if we could ". Key verbs like this  can be mapped to prioritisations, e.g. P1 (must), P2 (should), P3 (could). We use these clues to bootstrap priority discussions and, perhaps surprisingly, I often apply it to my own bug reports to get an idea of my intuition on an issue. You ought to beware that there will not b e a 1:1 mapping between key word and priority, and negation does not help. Fo...

One of the reasons that software testing is challenging, both intellectually and practically, is that the information about the state of the system under test is partial. It's part of the testing role to formulate a model (or, more usually, a cloud of overlapping, incomplete and contradictory models) that represent our best view of the system at any given time and we've developed a collection of monochrome boxes  that reflect the idea that access to source code can help make sense of it. But even that doesn't equate to an understanding of the model that the software has when it operates. For example: The tester may not follow the source code (completely). External libraries may implement a substantial part of the functionality but appear minimally in the source. Interactions with other layers, such as the operating system for file operations, will form part of the model without being part of the codebase. If the source code is compiled, it may be optimised in ways th...

As the tickets pour in and your will to live seeps out you need something to cling on to, something to keep you afloat, a software development life raft, if you will (with patches, natch). Unfortunately, this is not it. Instead, this is a bunch of those drug, chug, smug and shrug report definitions you've seen all over the place that the Dev manager and me have come up with in triage meetings over the years. We tried using them as buoyancy aids but, frankly, it takes more than Archimedes to lift this mood. debug report : A note from the developer to his future self as insurance: "Well, I did submit a report about that issue back when I committed the code". Yes, instead of fixing it. doodlebug report : Everyone knows it's coming and hopes it doesn't land on them. Doug report : The premise is fair, all the points made are apparently plausible, each step in the logic seems reasonable and internally the report appears to be consistent but you know the c...

So long ago it feels like another geological age - the Pre-Childrian, anyone? - I used to be in bands and I even made a few records. I don't have any formal schooling in music, and I didn't make much effort to learn by myself. Playing guitar was a creative endeavour and I took the view that I could do it however I wanted and when it didn't sound too good I just turned up the distortion because that always sounded good. When I started using trackers  to compose on the computer I unlearned what little I'd found out about music and the skill required to make it. Now I could focus on the selection as much as production; which samples to employ, how to arrange them, ways to process them to generate something new. The machine gave me space to improvise (read: bash the keyboard randomly) and then pick the bits that sounded good afterwards without ever having to be able to play them again. I embraced this freedom and began to collaborate with other people  using the same...

The focus/defocus distinction is important in Rapid Software Testing . It encourages the tester to consciously employ both logical reasoning and lateral thinking and to switch between them particularly at those times when one or the other is not productive. But while reasoning can often be ground out, it's hard to make intuitive leaps just because you want to, to simply hop out of the rut when you find you're bogged down, to free your mind from its box, especially when you haven't realised it's in a box yet, let alone that it's shaped like a straightjacket. So it's gladdening to find that we'll soon be able to pop on an electro hat and all our problems will be solved , literally.  I foresee a future where the brainly-enhanced QA staff trample all over the latest build seconds after it's delivered, booting it back to the Dev team for reworking and gleefully following it with an eruption of bug reports like Mount Etna on Ash Wednesday, all the while...

The Dev manager scooted his chair over to my desk yesterday with that particular smile on his face that means he knows something I don't know. Dev: Got a question for you. [Grin] QA: One of those questions with implications? [Grimace] Dev: Hidden implications. [Toothpaste advert] QA: Lots of hidden implications?  [Frown] Dev: I'm sitting on the tip of an iceberg of implications. [ Cheshire cat ] This is not a novel phrase   for not a novel idea  but it reminds me to keep myself in check when I'm trying to think through whatever the issue turns out to be. Dev: We need to audit these components against specific criteria for a stakeholder. They're used everywhere in the product. [Richard Branson] Hmm. I can see what's above the water for free; I can get an idea of what's just below well enough; I could dive deep down  but I have to get my gear ready and the currents and the winds and new snowfall will have shifted things by the time I resurface....

Sometimes it's hard to resist the temptation to just go for it, to run at the fresh build and leave long trails of filthy QA bootprints all over its virgin white snow. But hold on, Captain Scott ! Sometimes you want to know it's not ice before you start stamping. Sometimes includes projects with significant risk outside of the software itself, like running over budget or headed for late delivery or even non-delivery. When that sometimes is now it's reasonable (although, as always, not always) to try to shorten each iteration somehow, to limit effort until the likelihood of it being well spent is higher. That kind of somehow should look to fail a build early and at low cost and, to do that, somebody needs to find cheap and discriminatory  tests. Of course, the somebody who'll have to do that is you and you want tests which, while they might not have the power to explain what the issue is, can say that there's an important issue someplace. It's more than a ...

You can waste a lot of time on formatting. In The Power of Fancy Plain Text I said that I prefer plain text when possible and for a while I've been using Ascii Flow , a web-based tool, for making quick and dirty flowcharts and simple block diagrams that I can store in text documents too. Here's a bit of a chart  I made recently to plot out some test scenarios for a timing-related issue I was investigating and wanted to share on our wiki. The blocks represent actions occurring over time and their relationship to a some significant times and events. t1 t2 + + +---|-----------------------------|------> time | | | +----+ +----+ | +----+ E1 | | A | | B | | | C | | +----+ +----+ | +----+ | | +-----------------------------------------+ E9 | V ...

When you're testing you're observing, mapping, inspecting. Sometimes it's non-specific - you're exploring, trying to understand the size, shape, performance envelope, functionality, scope, stability, usability or other characteristics of an application. Sometimes it's very specific - you are implementing an attack vector that you think you can exploit to expose a bug. Sometimes it's neither of those things or a combination of those things. Whatever it is, it always involves looking and when I'm looking at a system I try to apply what I think of as peripheral testing  by analogy to  peripheral vision ,  the ability to see objects and movement outside of the direct line of vision . I have my main focus on the thing I'm interested in, but I permit and encourage part of my focus to wander briefly, just a little, but often, while I'm there. Let's say we're working in a GUI. I'll take a few seconds to hover over all the components in a d...