agent-approval-gate

I spent 19 years in enterprise IT before I started putting AI agents into production, and the thing that kills pilots is almost never the one founders expect. It is very rarely accuracy. It is that nobody decided what happens after the agent is wrong.

Every team I talk to has an eval suite. That suite is a pre-launch gate: it proves the agent was right on the cases you thought of. Production is a post-launch problem. The agent will be wrong on a case you did not think of, and the only question that matters then is what your system does in the 20 minutes after that.

So before anything of mine goes live, I run 4 questions. They are boring and none of them are about the model.

1. Where does a wrong result come to rest? If the output lands in a log, you have time. If it writes to your CRM, sends a customer email, moves money, or closes a ticket, the wrong answer is now a fact that other systems will read as true. Most agents I look at write to a system of record and nobody drew that line on purpose.

2. Is the action reversible, and by whom? Reversible by an engineer with database access at 2am is not reversible. It means the person who owns the process can undo it themselves, from the interface they already use.

3. Who sees it first, and how late? Not "we have monitoring". Name the human, and name the gap between the wrong action and the moment that human notices. If the honest answer is "the customer", that gap is your incident process.

4. When the number changes between two runs, can you say which input moved it? This is the one almost nobody passes, and it is the one that fails a real review. If your agent scored something 34 last week and 61 today and you cannot point at the input that moved it, you do not have a system you can defend. You have a demo that got lucky twice.

What I actually see: teams pass 2 easily, pass 1 by accident, fail 3 because monitoring got confused with someone watching, and have never been asked 4.

None of this is an argument against shipping. I ship agents. It is an argument for knowing which of the 4 you are choosing to skip before you ship, so that when it breaks you are looking at a decision you made instead of a surprise.

The pattern I use for 1 and 2 is open source if it is useful to anyone: https://github.com/renezander030/agent-approval-gate

And the part I do not think is settled, which is why I am posting this: for question 4, what do you actually keep so you can attribute a change between two runs? I keep the raw provider payload plus the repair diff, and it is heavier than I want it to be. If you have something lighter that still survives someone asking "why did this number change", I would like to see it.