In 2024 I wrote a post, An AI Red Light , about a creative project I did with my daughter using both AI and "traditional" tooling. This was my conclusion: I think we were pragmatic in our choice of tools. Where our vision required control (video) we used a tool that gave us that control. Where we were happy to cede some control in exchange for output at a quality we could not produce ourselves (vocals) we used a different tool. I didn't set out to do some kind of AI experiment. Instead I set out to have some fun with my daughter and found tools that enabled us to produce something, some art, that we're proud of and I'm happy is our work. For sure, we were sometimes more creators and sometimes more curators, but at no point were we ever mere spectators. Two years down the road I have access to, and use, LLMs in my work. With them, I have been able to make things that I would not have done without them such as the user interface on my model-based dialog walker . ...
There was a time when testing was all about the mnemonics . Well, we had no AI back then so thinking about our human craft and how to share what we had learned using our human intelligence with other engaged humans for later recall in their human heads seemed important. But this post isn't about dumping on AI. It isn't about mnemonics either even if WTTWTH does look like something that'd fit well into that ancient world. No, this post is just a snappier version of what I said to my team this week when I was walking through some testing I'd done the day before. It was concerned with a change to a particular turn in the dialog system we're building where multiple variables are in play, some populated by an external call to an LLM service. I wanted to make the point that the bulk of the testing work was done in the research I did and the spreadsheet I made, not in the interaction with our product. That spreadsheet was the result of me looking in our service's cod...