Hiccupps

I sometimes consciously split the functionality I'm testing into two parts:   general : behaviour that is the same, or similar, regardless of where it appears, how it is invoked and so on; and  specific : which differs according to function, context, time, data types etc.  I'll tend to do this more on larger projects when  the areas are new to me, or to the product, or if they're complex, or I think the test framework will be complex, or the specific is heavily dependent for its delivery on the general, or perhaps when the specific details are certain to change but the general will be stable.   I'll be looking to implement automation that concentrates first on general functionality and self-consistency and that will serve as a backstop when I move on to the more specific material.  To speed things up, to get wider coverage easily, and to avoid dependencies, I'll try to avoid crafting new test data by looking for data already in the company that can...

Co-location is intrinsic to  some software development  and it can also have social benefits, build an esprit de corps and smooth out the kinds of communication issues that time zones and typing often cause.  But, for me, there's another softer reason why co-location is advantageous - I learn stuff in passing from the natural interactions I have in the course of a working day.  When I go to a colleague's desk to ask about some functionality, and they pull up source files for inspection, I'm looking at the text, but I'm also interested in the editor they're using, the powerful ways it lets them search/replace and the fact that it has a plug-in for fancy diffing that I wasn't aware of and that I can use myself next time. Sitting with a developer as they write code is a welcome insight into the mindset of someone who really knows the nuts, bolts, screws, rivets, nails and other fixings when my skills, relatively speaking, extend to being able to tell a  brad...

The concept of test doubles  is well-established in unit testing with mocking probably the most familiar. The idea is that you fake enough of an API to permit unit tests to run against it. You can control the way that the mock API responds, tailor your test coverage and avoid external executables, dependencies on other code and so on to run your tests. It can be useful to use similar concepts at the component level too. For example: for diagnosis and investigation of misbehaviour in complex systems you can replace components in a workflow. I like this especially when the problem behaviour is hard to reproduce naturally. Part of our core product is  a server that calls out to other software for some tasks and returns data back to a client. Replacing a server-side executable with something that generates a specific output (e.g. returning an error response to the server, or bad data to the client) or behaviour (e.g. taking so long to respond that the server is forced to ti...

I read my daughter a story last night, about an elephant that thinks she's a mouse. It seems reasonable, her big book of knowledge says mice can be grey with big ears and skinny tails. Note to new testers: specifications are seldom sufficient . Think broadly. She moves in with a mouse family but it doesn't go well. Luckily Granny Mouse has the wisdom of age and experience, works out what's gone wrong and takes Nelly to the zoo to be with her own kind. Unfortunately, one of the mice then reads Nelly' s book and gets the idea he's an elephant. Note to old-hand testers: spread your insight through the team. A few times recently I've found myself talking to less-experienced colleagues and realising that I've segued from the mouse to the elephant, assuming they were following. They weren't, and the discussion got confused. We started off on a small acute, practical problem of right now and I expanded out to the big, theoretical potential solution o...

If you aren't using automation to get you to a place where you can efficiently apply intelligence you're almost certainly wasting time and effort. Michael Bolton put it this way on Twitter last week :  I like ... using automation as a taxi to drop me off at the site of the real work. When we moved offices a couple of months ago Marketing updated the contact details in their collateral and then requested that we review the changed PDFs. Simple approaches to this task would include reading all of the documents from top to bottom, skimming them looking for addresses to check or using a PDF reader's search functionality in each document to look for likely mistakes.  All of these would have been slow, prone to error and without a good way to make efficiencies. What we actually did was automate, using command line tools to do the heavy lifting. First, we transformed the PDF files into a text format in which they could easily be searched as a set:  $ f...

So the Dev Manager  finally started his own blog . Obviously, I was more than happy to pour feedback on it but more than disappointed that there's no public bug tracker, so I had to content myself with exaggerated outrage and inflated prioritisations via email. Which just isn't the same. But, joking aside, we should offer up a toast to him for writing positively about Weinberg's Perfect Software in one of his first posts even if he misinterpreted my face when I passed him the book as hopeful. In fact it was merely reflecting my ennui, despair, frustration, coffee intake and thoughts of an impending holiday . Weinberg wants to know  about how his writing has affected people's lives and I'm anxiously awaiting developments here too. Image:  http://flic.kr/p/9V2bKU

Maybe I'm just an oversensitive, subtle and thoughtful kind of guy, or attuned to ambiguity, or unempathic, or unimaginative, or a git, or plain old-fashioned thick, but being precise and clear with language is a skill that I value highly and strive to achieve . With questions, I find that I'm unlikely to assume the context if it's not already present or given. So, for example, I probably would not have a simple answer to " should the product do X?" Are we discussing a version of the product in the field or the product in development? Is it my personal opinion you're after? My user-head opinion? If so what kind of user? Perhaps it's what the spec says or said once upon a time? Maybe X is a reasonable result under certain circumstances but not others? Are you implicitly telling me that the software does do X and you're not sure whether it's right? Or that you think it might be intended to do X, and it doesn't? Or asking whether we need X in...

As the wartime slogan had it, careless talk costs much time in reimplementation and testing and ultimately late delivery of software that contains less than you wanted at a quality level of just-about-bearable and which will be followed immediately by a patch release, no two, no three patch, no four patch releases. It's incumbent on everyone in a development project to share knowledge efficiently by getting key points across economically. If you're not comprehensive, coherent, correct and concise, you risk other people missing your point because they never heard about it, couldn't follow it, didn't believe in it or lost interest in it, and that leads inexorably to extra costs. In that spirit, the meat  in this post is that you should endeavour be as open as possible, full in description and slight in length. When these are in conflict, strive to remove the need for fullness. If you must be full, then structure your content accordingly. Software teams are often...

I think a lot about the user experience when I'm testing . Not only for conformance with the UX guide we refer to but also in less tangible, subjective, respects such as how the software will  feel  to different kinds of users. The UX guide can't help here. Although it has value and the potential to save time - for example, by reducing the number of discussions about capitalisation policy for dialog title bars  to merely single figures per release - at close to 900 pages it is not a shining example of usability itself and it doesn't try to answer the question  who's a user?  In particular, it has only a handful of mentions of new users outside of the section on first-timers which itself is largely about setting up the product rather than using it. Inexperienced users will at various points form a significant enough proportion of your userbase that they merit special attention and when that happens, you'll need to think about lack of experience with ...

If extracting feature motivations, requirements and priorities from stakeholders is an art, presenting the analysis requires artfulness. The MoSCoW method suggests using English words M ust, S hould, C ould and W on't as an alternative to purely numeric priorities to make it more transparent that not everything listed will be delivered. But reversing the MoSCoW rules can help to obtain  the implicit prioritisations. You'll frequently hear "we must have a solution to X" or "Y should be improved in this release" or "we'd like to get Z into the payload if we could ". Key verbs like this  can be mapped to prioritisations, e.g. P1 (must), P2 (should), P3 (could). We use these clues to bootstrap priority discussions and, perhaps surprisingly, I often apply it to my own bug reports to get an idea of my intuition on an issue. You ought to beware that there will not b e a 1:1 mapping between key word and priority, and negation does not help. Fo...