1
0 Comments

3 canary prompts caught my model drifting

3 saved prompts are the only reason i'd ever notice my AI tool quietly getting worse. that's the whole takeaway, the rest is story.

this week a model vendor got accused of secretly changing what their model could do, then changing it back. no notice either way - they didn't say a word. made me realize the model doing a real chunk of my build is an external dependency that can shift under me, and i had zero way to tell.

so i pulled 3 prompts i know cold - a notes-to-tasks pass and a spec breakdown - re-ran them against what they used to produce. two were fine. one dropped a caveat it always used to catch. might be noise, these things aren't deterministic, but i only saw it because i'd saved what good looked like.

took ten minutes and a text file. it's cheaper than shipping a release on a model that quietly degraded.

what would your 3 canary prompts be - the ones you'd actually notice breaking?

posted to Icon for group AI Tools
AI Tools
on June 13, 2026
Trending on Indie Hackers
6 weeks solo, 2 rejections, finally live but nobody told me marketing would be this hard User Avatar 139 comments I spent more time setting up cold email than actually selling. Here is what fixed it. User Avatar 36 comments I just wanted to taste AI coding tools. A week passed. User Avatar 24 comments I built a PDF API because every team I know has a haunted corner of their codebase they never want to open User Avatar 17 comments Building LinkCover – Day 3: Payment is live. No more building, time to sell. User Avatar 16 comments I built an AI that ruthlessly roasts your competitors based on their 1-star reviews. User Avatar 11 comments