The thing that kills AI pilots is not accuracy.

August 31, 2026

I spent 19 years in enterprise IT before I started putting AI agents into production, and the thing that kills pilots is almost never the one founders expect. It is very rarely accuracy. It is that nobody decided what happens after the agent is wrong.

Every team I talk to has an eval suite. That suite is a pre-launch gate: it proves the agent was right on the cases you thought of. Production is a post-launch problem. The agent will be wrong on a case you did not think of, and the only question that matters then is what your system does in the 20 minutes after that.

So before anything of mine goes live, I run 4 questions. They are boring and none of them are about the model.

1. Where does a wrong result come to rest? If the output lands in a log, you have time. If it writes to your CRM, sends a customer email, moves money, or closes a ticket, the wrong answer is now a fact that other systems will read as true. Most agents I look at write to a system of record and nobody drew that line on purpose.

2. Is the action reversible, and by whom? Reversible by an engineer with database access at 2am is not reversible. It means the person who owns the process can undo it themselves, from the interface they already use.

3. Who sees it first, and how late? Not "we have monitoring". Name the human, and name the gap between the wrong action and the moment that human notices. If the honest answer is "the customer", that gap is your incident process.

4. When the number changes between two runs, can you say which input moved it? This is the one almost nobody passes, and it is the one that fails a real review. If your agent scored something 34 last week and 61 today and you cannot point at the input that moved it, you do not have a system you can defend. You have a demo that got lucky twice.

What I actually see: teams pass 2 easily, pass 1 by accident, fail 3 because monitoring got confused with someone watching, and have never been asked 4.

None of this is an argument against shipping. I ship agents. It is an argument for knowing which of the 4 you are choosing to skip before you ship, so that when it breaks you are looking at a decision you made instead of a surprise.

The pattern I use for 1 and 2 is open source if it is useful to anyone: https://github.com/renezander030/agent-approval-gate

And the part I do not think is settled, which is why I am posting this: for question 4, what do you actually keep so you can attribute a change between two runs? I keep the raw provider payload plus the repair diff, and it is heavier than I want it to be. If you have something lighter that still survives someone asking "why did this number change", I would like to see it.

René Zander

posted to

agent-approval-gate

Say something nice to determa…

Post Comment

1

This is exactly where the problem gets interesting, René. There’s another boundary after approval that we’ve been testing with OpsWatch: what happens when an action was validly authorised, but the authority changes after approval and before the consequential downstream action actually executes? The agent may still have technical permission. The approval may still exist in the audit trail. But neither necessarily proves the action remained authorised at the moment of consequence. That distinction becomes especially important in the examples you mentioned — moving money, customer communications, CRM writes and other actions that downstream systems then treat as fact. I’d be interested in how you handle that state change in your production architecture.

SuperMcG

·
14 hours ago
·
Reply
1. 1
  
  In the system I run for a client there's no gap. The tap on the approval card is the write, it goes out under the approver's own account, and the target and their access get re-checked right then, because a card that's sat in chat for a week can still get tapped. In most agents I look at, it's the service account that breaks this. If the agent writes with its own credential, the approver can lose access in between and the target system takes the write anyway.
  
  René Zander
  
  ·
  11 hours ago
  ·
  Reply
  1. 1
    
    That's a really useful distinction. Making the approval tap the write under the approver's own identity collapses most of the authority window because authorization and execution are effectively coupled.
    I think the service-account case exposes the harder boundary.
    If the agent executes under a credential whose authority survives independently of the human who approved the action, then the target can validate the credential perfectly and still accept an action whose underlying human authority has disappeared.
    That makes me wonder whether the missing primitive is not another approval check, but an action-specific authority artifact bound to the approver, target, parameters and current authority state, which the resource validates at write time.
    Then the service account proves who can technically write, while the authority artifact proves why this particular write is still authorised now.
    Have you seen anyone implement that separation cleanly in production?
    
    SuperMcG
    
    ·
    11 hours ago
    ·
    Reply
2

This resonates a lot. I’d add one distinction: preserving evidence from a run and being able to actually reproduce that run are slightly different problems. Even if you keep prompts, retrieved evidence, and model versions, the underlying data state may have changed since then. Curious whether you snapshot or otherwise bind the exact data state used for each decision as well.

Ho Bae

·
5 days ago
·
Reply
1. 1
  
  I gave up on reproducing the run. In a review nobody asks me to get 34 again, they ask why it is 61 now, and that is attribution rather than reproduction. So I bind identity and version instead of content: every retrieved item goes into the record as source id plus revision plus the retrieval parameters, and a delta between two runs is a set diff on the ids, then a version diff on whatever survived both. Sources with no version history are the exception, and those I do copy. The one that catches people is the index itself. A reindex or a new embedding model moves every neighbor without a single document changing, so the evidence set moves while every input looks untouched.
  
  René Zander
  
  ·
  4 days ago
  ·
  Reply
2

Question 4 feels like the hardest one to operationalize.
Curious whether keeping the raw provider payload + repair diff is mainly expensive in storage, or whether the bigger problem is deciding which inputs and intermediate state are actually worth preserving for attribution.

Aryan Sinh

·
10 days ago
·
Reply
1. 1
  
  Mostly the latter. Storage is the easy cost to see; deciding what represents causality is harder. A raw payload proves what the provider returned, but not which changed input moved the result. I currently preserve state at decision boundaries: retrieved evidence, normalized inputs, prompt/model/tool versions, and any repair that changed the output. The raw payload is fallback evidence. The boundary diff is what should answer "why did this number change?"
  
  René Zander
  
  ·
  10 days ago
  ·
  Reply
  1. 1
    
    That makes sense. The distinction between preserving evidence and preserving causality is the more interesting part here.
    
    Aryan Sinh
    
    ·
    9 days ago
    ·
    Reply
1

The “20 minutes after it’s wrong” point is where this gets really interesting. Have you encountered a production case where the action was legitimately approved, but something material changed between approval and execution? That seems like a different failure class again: the approval itself was valid, but the authority behind it may no longer have been current when the consequential action occurred. I’d be very interested in whether you’ve seen that with the enterprise agents you’ve deployed.

SuperMcG

·
2 days ago
·
Reply