Rage Against the Machinery

 

I often review and collaborate on unit tests at work. One of the patterns I see a lot is this:

• there are a handful of tests, each about a page long
• the tests share a lot of functionality, copy-pasted
• the test data is a complex object, created inside the test
• the test data varies little from test to test.

In Kotlin-ish pseudocode, each unit test might look something like this:

@Test
fun `test input against response for endpoint`() {

	setupMocks()
	setupTestContext()
	...
	
	val input = Object(a, OtherObject(b, c), AnotherObject(d))
	...
	
	val response = someHttpCall(endPoint, 
		method,
		headers,
		createBodyFromInput(input)
		)
        ...

	val expected = Object(w, OtherObject(x, y), AnotherObject (z))		
	val output = Object(process(response.getField()), otherProcess(response.getOtherField()), response.getLastField())
	
	assertEquals(expected, output)
}

...

While these tests are generally functional, and I rarely have reason to doubt that they're useful, getting an idea of the intent versus the implementation of each test, and the coverage of the tests as a whole, is difficult or time-consuming, or both, and readers pay some kind of cost every time they come to the tests. 

The machinery is in the way of the work. Rage!

At its heart, the example above is checking, for some function, that an input (a, b, c, d) produces an output (w, x, y, z). So, what I try to do, if the investment seems worthwhile given other priorities, is refactor to separate the test machinery from the test data.

For our example, it might look something like this:

@ParamterisedTest("testDataSet")
fun `test input against response for endpoint`(testData) {

	setupEverything()
	
	val input = createInputObject(testData.input)
	val expected = createExpectedObject(testData.expected)
	
	val response = callEndpoint(input)	
	val output = extractOutputObject(response)
	
	assertEquals(expected, output)
}

fun testDataSet() {

    // each row is a pair of input, expected output
    val testData = [
		([a, b, c, d], [w, x, y, z]),
		...
		]
}

This has a single unit test that loops against rows of data (in testDataSet) and boilerplate is moved to standalone functions with intentful names.

Key to this for me is that the data appears as cleanly as possible in the test data set. I want to have only the stuff that varies in the test data and all the constants factored away. This means that once I understand the function being checked by the test, I can review the data alone and look for patterns or missing cases, and adding new cases is quick.

If I feel I need it, I can add names or comments as part of the data and dump it on failure. I might also use formatting to make the visualisation more helpful:

testData = [
		("customer C special case", [a, b, C], X),
		("vanilla case",            [a, b, _], Y),
		...
		]

If you're thinking that this looks like data-driven tests, you're right. One of the things I enjoy about frameworks like Karate is that they have a human-readable syntax for this kind of setup that removes even more boilerplate noise:

		| case                    | input     | expected|
		| customer C special case | [a, b, C] |     X   |
		| vanilla case            | [a, b, _] |     Y   |
		...

I think this is a healthy way to set up the unit tests when there are sets of similar cases that vary only in data.

But there's another benefit too: this is effectively a little test rig inside the tests and I can use it to explore the feature that I am testing. See The Love of a Loop and a List for more on that. This means that part of my calculation about whether it's worth performing the refactoring is the extent to which I think it might help me to test right now. 

If I decide that I can't pay the refactoring tax, but think it might still be useful to explore from the unit tests, I can parameterise one of the cases and write a for loop that calls it with my data. Yes, that sounds horrid but it's throwaway code, created to help me answer the questions that I want to ask the system under test.

The machinery should serve my needs, not trigger my rage.
Image: https://flic.kr/p/2mtwLRd
Syntax highlighting: Pinetools