Skip to main content

Stretching a Pint

At last night's Cambridge Tester Meetup, Karo talked about heuristics (slides here). After a brief introduction to the topic, she walked us through a couple of testing mnemonics:


We then split into two groups for an exercise. While the other group applied FCC CUTS VIDS to testing Karo's kitchen - in fact, a schematic and floor plan of it  - the group I was in took FEW HICCUPPS and a beer glass used at the 42nd Cambridge Beer Festival.

There's plenty of pictures of the glass at #cbf42 but to give a quick description: it's a pint glass with a loosely-themed Hitchiker's Guide to the Galaxy/Beer Festival mashup logo (because it's the 42nd festival, we assume) on one side and a Campaign for Real Ale (Camra) logo along with the festival name and dates on the other. It has calibrations for different amounts of beer, apparently in accordance with some kind of volume marking regulations which have a CE logo.

HICCUPPS is a set of consistency oracles and we agreed to use each of them as springboards for test ideas rather than receptacles of them, to avoid being constrained by whether an idea was "in" the category we happened to be discussing and risk losing it.

Here's a selection of the ideas we came up with. I haven't edited much, only to combine some overlapping items and lose some repetition and the notes I can't understand this morning. We didn't use the internet for the exercise, but I've looked up some references while writing this post and we could certainly use it for evidence and to inspire more questions.

History


  • is the glass supplied at the festival always a pint glass, this shape, this size, of this manufacturing quality? 
  • is the logo in the same position, in the same proportions, in the same style across festivals? (e.g. compare images in the festival's Flickr account)
  • are the same volume measures always printed on it (pint, half-pint, third-of-a-pint)?
  • does the glass always show the certification of volume using the CE volume mark
  • is there always a theme to the beer festival? Does it need to be reflected on the glass?
  • is it important to the festival that there is continuity or consistency across festivals, glasses etc?

Image


  • is the festival logo on the glass intended to look this amateurish? (First impression: it's like a student rag picture) 
  • There's plenty of space on the glass, why squash the logo up in the way that has been done? (The mice have detail that's hard to see)
  • Would a simpler graphic design have been more striking? 
  • is it important that the measurements are accurate? (To what degree?)
  • are the fonts chosen appropriate for the audience? (No comic sans!)
  • is the use of colour appropriate? (The logo has to sit in front of many different colours of liquid)
  • is there any relationship between Hitchikers Guide to the Galaxy and Camra? Are there any potential negative connotations that could be made? 
  • is the glass consistent with the festival beer listing booklet, posters, staff uniforms etc?

Comparable Products


  • is there a standard shape, size, material etc for beer glasses at festivals? How about at Camra festivals? Cambridge festivals?
  • what about non-UK drinkers, what would they expect from a glass? In Britain, we still use imperial measurements but Cambridge is multicultural
  • pubs often don't have oversized beer glasses (as this one is, where the pint mark is below the top of the glass)
  • how easy is it to clean vs similar products?
  • what do similar kinds of events do about glasses? e.g. do wine festivals expect drinkers to use the same glass for red and white? Do they provide cleaning facilities for glasses? Is that part of this product?
  • is the glass solid? Will it break easily if dropped? Is the flooring chosen to be gentle on dropped glasses?
  • do Camra members have any expectations about the glass based on Camra conventions?
  • we observed what we thought were injection moulding marks on our glass - would hand-made glass be expected by any attendees? (They are already connoisseurs to some extent by going to the festival.)

Claims


  • are the volume markings correct?
  • is the time and date information printed on the glass correct?
  • is the vessel suitable for drinking beer from? Is it optimal? (What is the optimal glass for beer? cider? perry? soft drinks? Does it differ across beers?)
  • is the glass dishwasher safe?
  • would a glass from earlier beer festivals be honoured at this festival?
  • what does other festival material say, show, suggest about this glass?
  • does the glass alter the taste of its contents?
  • does the logo imply some endorsement from Douglas Adams' estate? (Particularly since Adams was from Cambridge)
  • is the glass built to last? (If so, last for what duration? The festival, life?)
  • is this really the 42nd festival? (according to who?)

User Desires


  • is it easy to drink from? to hold? to pass between people (e.g. friends for trying a taste, to the bar staff?)
  • is it stable when put down?
  • should it be more tactile, e.g. with 3D logo on it?
  • can this design of glass be stacked? is it stable when stacked?
  • is it easy to fill, can the measures be seen by the bar staff?
  • is it easy to carry multiple glasses (e.g. three in a triangle)
  • is it unique (e.g. for collectors)
  • it is robust?
  • does it have appropriate thermal properties (e.g. help to keep cold beer cold?)
  • is it safe (e.g. will it break into sharp shards when dropped?)
  • do customers desire gender-specific glasses? ("Do you want that in a lady's glass?" )
  • do customers want a glass that signifies no alcohol is in the drink? How about other kinds of specialist desires e.g. markings for gluten-free or vegetarian beer (is there such a thing, we asked? Yes.)
  • how are the glasses packaged for transport? Are they space-efficient?
  • are the production costs reasonable? affordable? 

Product


  • what is the product here? (we have permitted ourselves to switch between the glass, use of the glass, the festival ...)
  • is the thickness of the glass appropriate, comfortable to drink from?
  • do all of the instances of this glass at the festival look the same? Should they? To what tolerance?
  • should there be half-pint glasses too?
  • is the glass consistent with other aspects of branding?
  • what is the Camra logo about? It's looks like it has a lid. Is that intentional?

Purpose


  • do I want to drink out of it?
  • is it obvious that it's a receptable for liquids? For drinking from?
  • is it suitable for display?
  • does it look good in a collection of such glasses?
  • can it be easily, safely, efficiently transported and stored?
  • will the colours and other markings fade?
  • what else could it be used for? (e.g. holding coins or pens, as a vase, watering flowers, magnifying glass ... but this is a different testing exercise)
  • does it chip easily?
  • could you hurt people with it? (deliberately or not?)
  • is the glass inert?
  • can you stick it in your pocket when you need your hands free?
  • is it compatible with devices for holding glasses (e.g. deckchairs, belts)

Statutes


  • what is the CE volume marking? Would we need to test it in some way (e.g. check that the manufacturer is licensed to use it?)
  • are there hygiene standards for drinking vessels (e.g. certain grade, thickness, transparency of glass?)
  • are there conventions, contractual agreements, regulations about using the name of Cambridge or Camra in association with events?
  • does the festival have a license to sell beer?
  • do the bar staff need licenses or training to serve beer?
  • some brands of beer might require their product to be served in glasses branded for them?
  • does the logo conform to copyright law (e.g. with Hitchikers Guide to the Galaxy images)
  • does the glass fulfil the needs of the beer? (e.g. to have its head displayed, show bubbles, permit its colour to be appreciated, compared with others etc)
  • are glasses required to be round? (if so, how round? Could it be square? elliptical?)

Familiarity


  • what are common problems of any kind of branded product? branding wearing off, typos, correct copy etc
  • glasses with handles are often a pain to fit into a cupboard

Explainability


  • the logo might not be obvious to people not in the intersection of Hitchikers and Camra  fans
  • explainability is a kind of testability heuristic
  • the precise location of the festival isn't given, only the town. Should it be precise?

World


  • this is a pint glass to most Brits at least. To others it might just be a glass
  • is it obvious a beer glass? Probably it is to those familiar with the conventions of such glasses
  • does it obey the laws of physics?
  • is it a practical object, or a collectors item?
  • why does it have a wooden barrel in the logo when the beer at the festival no longer uses them?
  • does the audience expect nostalgia?
  • is it quintessentially English?

Images: Twitter, Wikipedia, Amazon 

Comments

  1. James,

    Thanks for the examples, these are not always immediately apparent.
    I still struggle with the difference between World & Comparable products and I suspect that you do too, from what I read here.
    I can't seem to find a clear difference between the two from your examples.

    But maybe you can enlighten me?

    ReplyDelete
    Replies
    1. Hi, thanks for taking the time to comment.

      At the meetup we talked a bit about this. I don't worry too much about where the idea belongs so long as I get the idea down. But I think you're right in general that there's the potential for overlap between the elements of FEW HICCUPPS.

      Even Michael Bolton's explanation of World (at the link I gave in the post) refers to another of the heuristics directly: "Often this kind of inconsistency leads us to recognize that the product is inconsistent with its purpose ..."

      When testing software we might naturally look for/find comparable products that are also software and, in this case, World might inspire us to look outside of software. Those two things are blurred when testing non-software as in the post.

      Also, things or relationships in the world might not constitute comparable products. For example, I might make an analogy between some entity in the software product and an object in real life and find useful test ideas that way. Imagine associating Clippy the notorious assistant in MS Office 97 (some element of a product) and a knowledgeable friend that you can ask for help. What might Clippy do that a friend does not?

      Delete
  2. Hi James,
    great examples how to use those mnemonics in real world.
    I envy you, because you had a lot of fun on this meetup.
    Here is my comment. I think that you used few hiccupps in wrong context.
    This mnemonic helps you with oracles, which means when you are suspecting that something with a product is an issue, it helps you to confirm that it is an issue or not.

    FCC CUTS VIDS is a tour mnemonic and helps you to create test ideas.
    You should used it on beer glass example instead of FEW HICCUPPS.

    Regards, Karlo.

    ReplyDelete
  3. Hi Karlo, thanks for commenting.

    I don't think I have quite as strict a view as you here.

    We had no spec, requirements, stories, stakeholder to get information from and so the consistency heuristic seems a plausible one to try.

    In this exercise, using HICCUPPS helped us to identify a lot of things that we could decide to take further (or not) based on whatever constraints might be in context if we were required to test the glass.

    Some of the questions we had suggested that we might be able to use certain kinds of oracles (e.g. for historical reference we might perhaps use Flickr) to help us make a call.

    In real life, if I was using something like HICCUPPS and found I wasn't getting the value I wanted from it I'd try something else.

    ReplyDelete

Post a Comment

Popular posts from this blog

Can Code, Can't Code, Is Useful

The Association for Software Testing is crowd-sourcing a book,  Navigating the World as a Context-Driven Tester , which aims to provide  responses to common questions and statements about testing from a  context-driven perspective . It's being edited by  Lee Hawkins  who is  posing questions on  Twitter ,   LinkedIn , Mastodon , Slack , and the AST  mailing list  and then collating the replies, focusing on practice over theory. I've decided to  contribute  by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be. Perhaps you'd like to join me?   --00-- "If testers can’t code, they’re of no use to us" My first reaction is to wonder what you expect from your testers. I am immediately interested in your working context and the way

Meet Me Halfway?

  The Association for Software Testing is crowd-sourcing a book,  Navigating the World as a Context-Driven Tester , which aims to provide  responses to common questions and statements about testing from a  context-driven perspective . It's being edited by  Lee Hawkins  who is  posing questions on  Twitter ,   LinkedIn , Mastodon , Slack , and the AST  mailing list  and then collating the replies, focusing on practice over theory. I've decided to  contribute  by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be. Perhaps you'd like to join me?   --00-- "Stop answering my questions with questions." Sure, I can do that. In return, please stop asking me questions so open to interpretation that any answer would be almost meaningless and certa

Not Strictly for the Birds

  One of my chores takes me outside early in the morning and, if I time it right, I get to hear a charming chorus of birdsong from the trees in the gardens down our road, a relaxing layered soundscape of tuneful calls, chatter, and chirrupping. Interestingly, although I can tell from the number and variety of trills that there must be a large number of birds around, they are tricky to spot. I have found that by staring loosely at something, such as the silhouette of a tree's crown against the slowly brightening sky, I see more birds out of the corner of my eye than if I scan to look for them. The reason seems to be that my peripheral vision picks up movement against the wider background that direct inspection can miss. An optometrist I am not, but I do find myself staring at data a great deal, seeking relationships, patterns, or gaps. I idly wondered whether, if I filled my visual field with data, I might be able to exploit my peripheral vision in that quest. I have a wide monito

Testing (AI) is Testing

Last November I gave a talk, Random Exploration of a Chatbot API , at the BCS Testing, Diversity, AI Conference .  It was a nice surprise afterwards to be offered a book from their catalogue and I chose Artificial Intelligence and Software Testing by Rex Black, James Davenport, Joanna Olszewska, Jeremias Rößler, Adam Leon Smith, and Jonathon Wright.  This week, on a couple of train journeys around East Anglia, I read it and made sketchnotes. As someone not deeply into this field, but who has been experimenting with AI as a testing tool at work, I found the landscape view provided by the book interesting, particularly the lists: of challenges in testing AI, of approaches to testing AI, and of quality aspects to consider when evaluating AI.  Despite the hype around the area right now there's much that any competent tester will be familiar with, and skills that translate directly. Where there's likely to be novelty is in the technology, and the technical domain, and the effect of

Postman Curlections

My team has been building a new service over the last few months. Until recently all the data it needs has been ingested at startup and our focus has been on the logic that processes the data, architecture, and infrastructure. This week we introduced a couple of new endpoints that enable the creation (through an HTTP POST) and update (PUT) of the fundamental data type (we call it a definition ) that the service operates on. I picked up the task of smoke testing the first implementations. I started out by asking the system under test to show me what it can do by using Postman to submit requests and inspecting the results. It was the kinds of things you'd imagine, including: submit some definitions (of various structure, size, intent, name, identifiers, etc) resubmit the same definitions (identical, sharing keys, with variations, etc) retrieve the submitted definitions (using whatever endpoints exist to show some view of them) compare definitions I submitted fro

Testers are Gate-Crashers

  The Association for Software Testing is crowd-sourcing a book,  Navigating the World as a Context-Driven Tester , which aims to provide  responses to common questions and statements about testing from a  context-driven perspective . It's being edited by  Lee Hawkins  who is  posing questions on  Twitter ,   LinkedIn , Mastodon , Slack , and the AST  mailing list  and then collating the replies, focusing on practice over theory. I've decided to  contribute  by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be. Perhaps you'd like to join me?   --00-- "Testers are the gatekeepers of quality" Instinctively I don't like the sound of that, but I wonder what you mean by it. Perhaps one or more of these? Testers set the quality sta

Vanilla Flavour Testing

I have been pairing with a new developer colleague recently. In our last session he asked me "is this normal testing?" saying that he'd never seen anything like it anywhere else that he'd worked. We finished the task we were on and then chatted about his question for a few minutes. This is a short summary of what I said. I would describe myself as context-driven . I don't take the same approach to testing every time, except in a meta way. I try to understand the important questions, who they are important to, and what the constraints on the work are. With that knowledge I look for productive, pragmatic, ways to explore whatever we're looking at to uncover valuable information or find a way to move on. I write test notes as I work in a format that I have found to be useful to me, colleagues, and stakeholders. For me, the notes should clearly state the mission and give a tl;dr summary of the findings and I like them to be public while I'm working not just w

Build Quality

  The Association for Software Testing is crowd-sourcing a book,  Navigating the World as a Context-Driven Tester , which aims to provide  responses to common questions and statements about testing from a  context-driven perspective . It's being edited by  Lee Hawkins  who is  posing questions on  Twitter ,   LinkedIn , Mastodon , Slack , and the AST  mailing list  and then collating the replies, focusing on practice over theory. I've decided to  contribute  by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be. Perhaps you'd like to join me?   --00-- "When the build is green, the product is of sufficient quality to release" An interesting take, and one I wouldn't agree with in general. That surprises you? Well, ho

Make, Fix, and Test

A few weeks ago, in A Good Tester is All Over the Place , Joep Schuurkes described a model of testing work based on three axes: do testing yourself or support testing by others be embedded in a team or be part of a separate team do your job or improve the system It resonated with me and the other testers I shared it with at work, and it resurfaced in my mind while I was reflecting on some of the tasks I've picked up recently and what they have involved, at least in the way I've chosen to address them. Here's three examples: Documentation Generation We have an internal tool that generates documentation in Confluence by extracting and combining images and text from a handful of sources. Although useful, it ran very slowly or not at all so one of the developers performed major surgery on it. Up to that point, I had never taken much interest in the tool and I could have safely ignored this piece of work too because it would have been tested by

The Best Laid Test Plans

The Association for Software Testing is crowd-sourcing a book,  Navigating the World as a Context-Driven Tester , which aims to provide  responses to common questions and statements about testing from a  context-driven perspective . It's being edited by  Lee Hawkins  who is  posing questions on  Twitter ,   LinkedIn , Mastodon , Slack , and the AST  mailing list  and then collating the replies, focusing on practice over theory. I've decided to  contribute  by answering briefly, and without a lot of editing or crafting, by imagining that I'm speaking to someone in software development who's acting in good faith, cares about their work and mine, but doesn't have much visibility of what testing can be. Perhaps you'd like to join me?   --00-- "What's the best format for a test plan?" I'll side-step the conversation about what a test plan is and just say that the format you should use is one that works for you, your coll