How I Test Products I Can't Read the Code For
3 min read
Every product on this site got built by an AI tool while I watched. I can tell you what SchemaLens does with a connection string, and I can trace which n8n node hands data to which. What I can’t do is open a pull request and tell you whether a function is written well. Most testing advice assumes you can read the diff and reason about what could go wrong in it. Mine has to work without that.
Acting like the buyer, not the builder
The first pass is the simplest one and the one I skip most often when I’m in a hurry: I click through the whole install flow pretending I’ve never seen the product before. Not “does the feature work if I trigger it the way I know it works,” but “what happens if I follow only the setup guide, word for word, on a machine that’s never touched this before.” For a boilerplate that means a fresh Supabase project, not the one I built against. For a plugin it means a clean Obsidian vault. Half of what this catches isn’t even a bug. It’s a step I described from memory instead of from what’s actually on screen, which is the same gap I wrote about in the BYOK guide post: the part I control is easy to test, the part I don’t is where I skip steps without noticing.
Asking the tool to test itself
Since I can’t read the logic and spot the edge case by eye, I ask the same tool that wrote the code to write tests for it, then explain in plain language what each test is actually checking. That explanation step matters more than the test itself. If Claude Code or Cursor can’t tell me, in a sentence I understand, what a test proves, I don’t trust the test. For the n8n workflows this looks different: there’s no test suite, so I run the workflow with input that’s deliberately wrong, a missing field, a string where a number should be, and watch where it breaks instead of only running the clean example I built it against.
Where this still misses things
Two real bugs got past both of these passes and reached a buyer before I found them: a node that silently overwrote $json in one of the n8n workflows, and an Obsidian plugin bug that hid the upgrade button under specific settings. Both got caught by someone using the product, not by me testing it. The pattern in both cases is the same. I test the paths I can think to imagine, and a stranger’s real setup hits a path I didn’t think of, because I’ve never been that stranger. I don’t have years of seeing a hundred different ways software breaks in the wild, which is exactly the intuition this kind of testing usually depends on.
I wrote, back in the n8n packaging post, that the discipline is to test blind, pretending to be someone who’s never seen the setup before. I still believe that’s the right instinct. What I’d add now is that it’s a discipline I’m still building, not one I’ve actually finished. The gap between “I tested it” and “someone unfamiliar with it will get through it cleanly” is bigger than I want it to be, and closing it hasn’t gotten any faster just because I’ve shipped more products since I first noticed it.