At last night's Cambridge Tester Meetup, Karo talked about heuristics (slides here). After a brief introduction to the topic, she walked us through a couple of testing mnemonics:
We then split into two groups for an exercise. While the other group applied FCC CUTS VIDS to testing Karo's kitchen - in fact, a schematic and floor plan of it - the group I was in took FEW HICCUPPS and a beer glass used at the 42nd Cambridge Beer Festival.
There's plenty of pictures of the glass at #cbf42 but to give a quick description: it's a pint glass with a loosely-themed Hitchiker's Guide to the Galaxy/Beer Festival mashup logo (because it's the 42nd festival, we assume) on one side and a Campaign for Real Ale (Camra) logo along with the festival name and dates on the other. It has calibrations for different amounts of beer, apparently in accordance with some kind of volume marking regulations which have a CE logo.
HICCUPPS is a set of consistency oracles and we agreed to use each of them as springboards for test ideas rather than receptacles of them, to avoid being constrained by whether an idea was "in" the category we happened to be discussing and risk losing it.
Here's a selection of the ideas we came up with. I haven't edited much, only to combine some overlapping items and lose some repetition and the notes I can't understand this morning. We didn't use the internet for the exercise, but I've looked up some references while writing this post and we could certainly use it for evidence and to inspire more questions.
Images: Twitter, Wikipedia, Amazon
We then split into two groups for an exercise. While the other group applied FCC CUTS VIDS to testing Karo's kitchen - in fact, a schematic and floor plan of it - the group I was in took FEW HICCUPPS and a beer glass used at the 42nd Cambridge Beer Festival.
There's plenty of pictures of the glass at #cbf42 but to give a quick description: it's a pint glass with a loosely-themed Hitchiker's Guide to the Galaxy/Beer Festival mashup logo (because it's the 42nd festival, we assume) on one side and a Campaign for Real Ale (Camra) logo along with the festival name and dates on the other. It has calibrations for different amounts of beer, apparently in accordance with some kind of volume marking regulations which have a CE logo.
HICCUPPS is a set of consistency oracles and we agreed to use each of them as springboards for test ideas rather than receptacles of them, to avoid being constrained by whether an idea was "in" the category we happened to be discussing and risk losing it.
Here's a selection of the ideas we came up with. I haven't edited much, only to combine some overlapping items and lose some repetition and the notes I can't understand this morning. We didn't use the internet for the exercise, but I've looked up some references while writing this post and we could certainly use it for evidence and to inspire more questions.
History
- is the glass supplied at the festival always a pint glass, this shape, this size, of this manufacturing quality?
- is the logo in the same position, in the same proportions, in the same style across festivals? (e.g. compare images in the festival's Flickr account)
- are the same volume measures always printed on it (pint, half-pint, third-of-a-pint)?
- does the glass always show the certification of volume using the CE volume mark?
- is there always a theme to the beer festival? Does it need to be reflected on the glass?
- is it important to the festival that there is continuity or consistency across festivals, glasses etc?
Image
- is the festival logo on the glass intended to look this amateurish? (First impression: it's like a student rag picture)
- There's plenty of space on the glass, why squash the logo up in the way that has been done? (The mice have detail that's hard to see)
- Would a simpler graphic design have been more striking?
- is it important that the measurements are accurate? (To what degree?)
- are the fonts chosen appropriate for the audience? (No comic sans!)
- is the use of colour appropriate? (The logo has to sit in front of many different colours of liquid)
- is there any relationship between Hitchikers Guide to the Galaxy and Camra? Are there any potential negative connotations that could be made?
- is the glass consistent with the festival beer listing booklet, posters, staff uniforms etc?
Comparable Products
- is there a standard shape, size, material etc for beer glasses at festivals? How about at Camra festivals? Cambridge festivals?
- what about non-UK drinkers, what would they expect from a glass? In Britain, we still use imperial measurements but Cambridge is multicultural
- pubs often don't have oversized beer glasses (as this one is, where the pint mark is below the top of the glass)
- how easy is it to clean vs similar products?
- what do similar kinds of events do about glasses? e.g. do wine festivals expect drinkers to use the same glass for red and white? Do they provide cleaning facilities for glasses? Is that part of this product?
- is the glass solid? Will it break easily if dropped? Is the flooring chosen to be gentle on dropped glasses?
- do Camra members have any expectations about the glass based on Camra conventions?
- we observed what we thought were injection moulding marks on our glass - would hand-made glass be expected by any attendees? (They are already connoisseurs to some extent by going to the festival.)
Claims
- are the volume markings correct?
- is the time and date information printed on the glass correct?
- is the vessel suitable for drinking beer from? Is it optimal? (What is the optimal glass for beer? cider? perry? soft drinks? Does it differ across beers?)
- is the glass dishwasher safe?
- would a glass from earlier beer festivals be honoured at this festival?
- what does other festival material say, show, suggest about this glass?
- does the glass alter the taste of its contents?
- does the logo imply some endorsement from Douglas Adams' estate? (Particularly since Adams was from Cambridge)
- is the glass built to last? (If so, last for what duration? The festival, life?)
- is this really the 42nd festival? (according to who?)
User Desires
- is it easy to drink from? to hold? to pass between people (e.g. friends for trying a taste, to the bar staff?)
- is it stable when put down?
- should it be more tactile, e.g. with 3D logo on it?
- can this design of glass be stacked? is it stable when stacked?
- is it easy to fill, can the measures be seen by the bar staff?
- is it easy to carry multiple glasses (e.g. three in a triangle)
- is it unique (e.g. for collectors)
- it is robust?
- does it have appropriate thermal properties (e.g. help to keep cold beer cold?)
- is it safe (e.g. will it break into sharp shards when dropped?)
- do customers desire gender-specific glasses? ("Do you want that in a lady's glass?" )
- do customers want a glass that signifies no alcohol is in the drink? How about other kinds of specialist desires e.g. markings for gluten-free or vegetarian beer (is there such a thing, we asked? Yes.)
- how are the glasses packaged for transport? Are they space-efficient?
- are the production costs reasonable? affordable?
Product
- what is the product here? (we have permitted ourselves to switch between the glass, use of the glass, the festival ...)
- is the thickness of the glass appropriate, comfortable to drink from?
- do all of the instances of this glass at the festival look the same? Should they? To what tolerance?
- should there be half-pint glasses too?
- is the glass consistent with other aspects of branding?
- what is the Camra logo about? It's looks like it has a lid. Is that intentional?
Purpose
- do I want to drink out of it?
- is it obvious that it's a receptable for liquids? For drinking from?
- is it suitable for display?
- does it look good in a collection of such glasses?
- can it be easily, safely, efficiently transported and stored?
- will the colours and other markings fade?
- what else could it be used for? (e.g. holding coins or pens, as a vase, watering flowers, magnifying glass ... but this is a different testing exercise)
- does it chip easily?
- could you hurt people with it? (deliberately or not?)
- is the glass inert?
- can you stick it in your pocket when you need your hands free?
- is it compatible with devices for holding glasses (e.g. deckchairs, belts)
Statutes
- what is the CE volume marking? Would we need to test it in some way (e.g. check that the manufacturer is licensed to use it?)
- are there hygiene standards for drinking vessels (e.g. certain grade, thickness, transparency of glass?)
- are there conventions, contractual agreements, regulations about using the name of Cambridge or Camra in association with events?
- does the festival have a license to sell beer?
- do the bar staff need licenses or training to serve beer?
- some brands of beer might require their product to be served in glasses branded for them?
- does the logo conform to copyright law (e.g. with Hitchikers Guide to the Galaxy images)
- does the glass fulfil the needs of the beer? (e.g. to have its head displayed, show bubbles, permit its colour to be appreciated, compared with others etc)
- are glasses required to be round? (if so, how round? Could it be square? elliptical?)
Familiarity
- what are common problems of any kind of branded product? branding wearing off, typos, correct copy etc
- glasses with handles are often a pain to fit into a cupboard
Explainability
- the logo might not be obvious to people not in the intersection of Hitchikers and Camra fans
- explainability is a kind of testability heuristic
- the precise location of the festival isn't given, only the town. Should it be precise?
World
- this is a pint glass to most Brits at least. To others it might just be a glass
- is it obvious a beer glass? Probably it is to those familiar with the conventions of such glasses
- does it obey the laws of physics?
- is it a practical object, or a collectors item?
- why does it have a wooden barrel in the logo when the beer at the festival no longer uses them?
- does the audience expect nostalgia?
- is it quintessentially English?
Images: Twitter, Wikipedia, Amazon
Comments
Thanks for the examples, these are not always immediately apparent.
I still struggle with the difference between World & Comparable products and I suspect that you do too, from what I read here.
I can't seem to find a clear difference between the two from your examples.
But maybe you can enlighten me?
At the meetup we talked a bit about this. I don't worry too much about where the idea belongs so long as I get the idea down. But I think you're right in general that there's the potential for overlap between the elements of FEW HICCUPPS.
Even Michael Bolton's explanation of World (at the link I gave in the post) refers to another of the heuristics directly: "Often this kind of inconsistency leads us to recognize that the product is inconsistent with its purpose ..."
When testing software we might naturally look for/find comparable products that are also software and, in this case, World might inspire us to look outside of software. Those two things are blurred when testing non-software as in the post.
Also, things or relationships in the world might not constitute comparable products. For example, I might make an analogy between some entity in the software product and an object in real life and find useful test ideas that way. Imagine associating Clippy the notorious assistant in MS Office 97 (some element of a product) and a knowledgeable friend that you can ask for help. What might Clippy do that a friend does not?
great examples how to use those mnemonics in real world.
I envy you, because you had a lot of fun on this meetup.
Here is my comment. I think that you used few hiccupps in wrong context.
This mnemonic helps you with oracles, which means when you are suspecting that something with a product is an issue, it helps you to confirm that it is an issue or not.
FCC CUTS VIDS is a tour mnemonic and helps you to create test ideas.
You should used it on beer glass example instead of FEW HICCUPPS.
Regards, Karlo.
I don't think I have quite as strict a view as you here.
We had no spec, requirements, stories, stakeholder to get information from and so the consistency heuristic seems a plausible one to try.
In this exercise, using HICCUPPS helped us to identify a lot of things that we could decide to take further (or not) based on whatever constraints might be in context if we were required to test the glass.
Some of the questions we had suggested that we might be able to use certain kinds of oracles (e.g. for historical reference we might perhaps use Flickr) to help us make a call.
In real life, if I was using something like HICCUPPS and found I wasn't getting the value I wanted from it I'd try something else.
Post a Comment