Skimming a recent issue of Significance, the journal of the Royal Statistical Society, I saw a reference to a paper called We Need Both Exploratory and Confirmatory by John Tukey, published in the 1980 in The American Statistician. I hunted it down and found much that felt familiar. Here's a few quotes:
Exploratory data analysis is an attitude, a flexibility, and a reliance on display, NOT a bundle of techniques, and should be so taught. Confirmatory data analysis, by contrast, is easier to teach and easier to computerize.
... to implement the confirmatory paradigm properly we need to do a lot of exploratory work.
Neither exploratory nor confirmatory is sufficient alone. To try to replace either by the other is madness. We need them both
Science DOES NOT BEGIN WITH A TIDY QUESTION. Nor does it end with a tidy answer.
No catalogue of techniques can convey a willingness to look for what can be seen, whether or not anticipated. Yet this is at the heart of exploratory data analysis ... the picture-examining eye is the best finder we have of the wholly unanticipated.
... we have got to take seriously the need for steady progress toward teaching routine procedures to computers rather than people. That will leave the teachers of people with only things hard to teach, that is our proper fate.
Students who have never been exposed to CONFIRMATORY seem to learn EXPLORATORY more easily.I recognise that this stuff is preaching to my choir, but I continue to find it compelling that other disciplines are, and have been, talking about the kinds of things that I've come to think about ours through my own experience.
Image: Taylor and Francis