3 saved prompts are the only reason i'd ever notice my AI tool quietly getting worse. that's the whole takeaway, the rest is story.
this week a model vendor got accused of secretly changing what their model could do, then changing it back. no notice either way - they didn't say a word. made me realize the model doing a real chunk of my build is an external dependency that can shift under me, and i had zero way to tell.
so i pulled 3 prompts i know cold - a notes-to-tasks pass and a spec breakdown - re-ran them against what they used to produce. two were fine. one dropped a caveat it always used to catch. might be noise, these things aren't deterministic, but i only saw it because i'd saved what good looked like.
took ten minutes and a text file. it's cheaper than shipping a release on a model that quietly degraded.
what would your 3 canary prompts be - the ones you'd actually notice breaking?