The hard part of delegating to an AI agent is knowing what evidence would let you trust the stop point

This point about evidence generated by the same agent that did the work not being trustworthy verification is something I learned the hard way. I built a multi-agent strategy tool (feed it meeting notes, get SWOT/Porter's Five Forces/etc. back), and early on I let the analysis agent basically vouch for its own numbers — which meant confident-sounding fabrication slipped through. What ended up working was treating "no citation back to the original notes" as a hard stop, not a soft warning, and having a separate step whose only job is to check claims against the source material rather than trusting the generating agent's own confidence. Your reversibility framing is a nice way to think about when that kind of independent check is worth the friction vs. overkill.

The counterexample you asked for: every boundary I wrote up front was wrong, and every stop point that stuck got added the day after something went through that shouldn't have. So the brief isn't really the artifact, the amendment history is, and what decides whether people keep using it is how cheap it is to add one rule at 11pm while you're annoyed.

If authoring is beautiful and editing is a chore, it gets abandoned around week three.

https://gist.github.com/renezander030/9069db775e494ffd2cdd5a09adf83add

Full disclosure: I'm Avery Lin (avrlin). I've been packaging a Live Agent DFY offer ($97 — live pair/screen-share so an OpenClaw-style agent is actually running same-day / in ~24h) with AI assistance, so take this as adjacent interest, not neutral advice.

Hiring "an agent" often stalls on ops: secrets, first successful run, and what to delegate next. Docs alone leave a lot of founders half-installed; a short live install call usually closes that loop.

Curious — for OpenClaw / building a live agent, do buyers finish self-serve, or do they still need a same-day hand-hold to go live?

I think the useful boundary is often one step before execution. In a workflow I’m working on, AI can extract and organize the work, but the human still reviews the tasks, deadlines and priorities before anything is saved. That review point turned out to be important because the expensive mistake isn’t usually extraction — it’s acting confidently on the wrong interpretation.

One distinction that helped us was separating an execution receipt from an outcome receipt. A deploy log proves code was deployed; it does not prove the user-facing behavior works.

For each stop point, we now ask for five things: the exact claim, the source of truth, an independent check, the evidence timestamp/freshness, and what remains unproven. That final field matters because it prevents a narrow success from being reported as total completion.

A concrete failure: CI was green and the deployment job succeeded, but production was still serving an older bundle. Both logs were true; neither answered the user’s actual question. Only checking the live state did.

So I’d make the boundary two-dimensional: reversibility determines whether the agent must pause, while verifiability determines whether it may say “done.”

I’ve been operating this way with AI agents while building my SaaS. The boundary that finally worked was simple: agents can investigate, draft, test, and make reversible staging changes. Anything public, paid, destructive, or difficult to reverse comes back to me.

But the pause alone isn’t enough. I only trust it when I receive evidence I can independently check—an exact commit, test result, live URL, health response, or screenshot—not simply “done.”

What failed before was vague approval language. The agent either interrupted too often or inferred too much. Defining specific action classes and the evidence required at each stop made the workflow much more usable.

The real tell isn't whether the agent paused, it's whether the stop leaves you with a decision you can act on or just an obedient shrug. In the agentic workflows we build at Cloudify, the rule that actually held was that any evidence the agent cites to justify stopping has to be externally verifiable, not something it generated in that same run. Self-reported evidence is the leak: once an agent can point to its own log as proof, it can talk itself into or out of anything.

Really thoughtful framing, Yeye. Thanks for being explicit about what you still haven’t validated yet.

For me, the strongest approval boundary is usually based on reversibility. If an action is easy to undo, the agent can have more freedom. If it affects money, customers, permissions, or external systems, I’d want a clear pause plus evidence showing what it plans to do and why.

The harder problem is probably making those checkpoints informative without turning every workflow into constant approval fatigue.

The distinction that feels most useful to me is: “can the agent prove the state changed in the world, or is it only reporting that it followed the plan?”

For meaningful work, I’d want the stop point to include three things:

1. The exact action taken or drafted, not a summary.
2. The evidence source that confirms the result: URL, record ID, screenshot, log line, diff, metric, etc.
3. The next irreversible decision that still belongs to the human.

The dangerous middle ground is when the agent is allowed to keep moving because the task “sounds low risk,” but the evidence is still narrative. That creates fake confidence.

I’m working on a validation product, and this same pattern shows up with startup ideas too: the useful output is not “this seems good,” it’s “these are the assumptions, this is the evidence we have, this is what remains unproven, and this is the next test.”

So my answer: I trust a stop point when it separates claims from evidence and makes the next decision explicit.

The failure mode I'd add: an agent that reports success while having silently done nothing. That's worse than one that pauses too often, because there's no prompt to ignore — the run just looks clean.

Two from my own week, both in a data pipeline I run. First, a model call had its output budget set to 2500 tokens, but that model's internal reasoning consumes the same budget. Reasoning ate 2399 of it, the actual answer got 97 tokens and was cut off mid-heading — and the call still returned success. Second, a step that fetches a summary before deciding which sampling strategy to use started timing out on large inputs, and the timeout killed the entire source instead of falling through to the strategy that timeout was itself evidence for.

So my answer to your question about what makes me trust a stop point: I don't trust the agent's own report at all. What I trust is a separate check that runs after, written against the shape of the output rather than the status code. In my case that's a script asserting the result contains the sections it's supposed to, with the right fields filled. It catches exactly the class of failure the agent can't self-report, because an agent that thinks it succeeded will tell you it succeeded.

On your asymmetric-rule point from the other comment — agreed, and I'd extend it: the boundary should be about reversibility rather than risk. Risk is a judgement the agent has to make. "Can this be undone in one command" is a fact it can check.

One counterexample from running scheduled agents: an approval boundary that fires too often stops being a boundary. We had one asking before every low-risk action, and within a week I was approving without reading — the pause had turned into noise. What's held up better for me is an asymmetric rule: reads and drafts run unattended, anything that publishes or spends needs me, and the ask has to include the exact payload, not the agent's summary of it. If the approval prompt is a summary the agent wrote, I'm just approving its own description of the work.

One counterexample from running scheduled agents: an approval boundary that fires too often stops being a boundary. We had one asking before every low-risk action, and within a week I was approving without reading — the pause had turned into noise. What's held up better for me is an asymmetric rule: reads and drafts run unattended, anything that publishes or spends needs me, and the ask has to include the exact payload, not the agent's summary of it. If the approval prompt is a summary the agent wrote, I'm just approving its own description of the work.

the verification gap is wider than most people think. we assessed 412 people on AI skills (aisa.to/state-of-ai-fluency) and safety/verification scored lowest at 45/100 — everyone can get the agent to produce output, almost nobody has a systematic way to decide if they should trust it. your framing of "what evidence would let you trust" is exactly the right question.

The thread has the evidence question well covered in one direction, which is what the agent shows before it continues. What I'd add is a property of the evidence rather than its content: it has to come from a different implementation than the one that did the work.

Most of the mechanisms here are still the agent grading its own homework in a better format. A diff, a receipt, a dry-run list, all produced by the same code path that took the action, so a wrong assumption shared between doing and checking produces confident, well-structured, wrong evidence. The agent isn't lying in that case. It's reporting accurately on a rule it got wrong.

Concrete version from my own product. It generates PDFs that are meant to meet an accessibility standard, and the pipeline was certain it was doing that for weeks. The pass/fail actually comes from a separate open-source validator other people wrote against the spec, and when I widened the test inputs it rejected things I'd never have thought to doubt: bold and italic runs, lists with links in them. Nothing in my own code would ever have told me, because the bug was in what my code believed the rule was.

So for the boundary you're designing: "return evidence before continuing" is weaker than it sounds if the agent gets to choose the check. What makes it bite is a check the agent can't author and can't satisfy by reasoning about it.

The honest limit is that this only works where an independent checker exists. Plenty of business decisions have none, and then you're back to a human at the stop point, which is fine as long as nobody mistakes a well-formatted summary for verification.

Everyone here is answering "was the stop correct." There's a prior question that took me a while to see: was it a stop at all.

A process that died and an agent that paused look identical from outside. No error, no output, task incomplete, no summary. The laptop slept. The wifi dropped. The sandbox's idle reaper fired because nobody had typed in twenty minutes, which is exactly what a working unattended agent looks like to a timer.

This is the failure I ended up building infrastructure around, so I'm biased, but it changed how I read your question. tjgarage's rule that a stop must cite evidence the agent didn't produce is the right shape and I'd keep it. It assumes the agent survived to cite anything. The failure I'm describing produces no evidence in either direction, which makes it worse than a wrong stop: a wrong stop is at least an argument you can have.

What I do now is make the run write its own ending. The last line is either "stopped: waiting on approval, artifact is here" or "stopped: finished." If the record simply ends, that isn't a pause, it's a death, and it's a different investigation. For long runs, a heartbeat in the same file.

On your validation question, since you said you want counterexamples: if you count pauses in live runs, count the runs that ended with no pause record at all. They're invisible to any pause-quality metric, and they'll be more common than the judgment calls.

Which decisions stay mine: anything that leaves the machine. Everything inside it I audit after.

I think the review step is the key.
For small businesses, I’d rather start with a narrow AI task where the output is easy to verify, then expand only after the failure modes are clear.

mihir_kanzariya and omri_ben_shoham already have the shape of it — rule followed perfectly, outcome still wrong. You asked for counterexamples, so here's that shape pointed the other way: the boundary held, the agent stopped, and the stop was the failure.

Most of this thread guards one direction — whether the agent is allowed to continue. Receipt not reasoning, the number not the reason, the diff not the summary. ryanshrott's shadow mode is the one mechanism here that would catch both, and FounderFlow_57's question about too many pauses or too few points at the same gap. The second direction is the harder one.

A scheduled run of mine invented a safety check that was never in its brief. It listed the processes on the machine, found one that had been alive for eleven seconds, concluded a previous run was stuck, refused to do the work, and finished by asking me to pick between three options. It was scheduled. Nobody was at the keyboard. The process it found was itself.

Two things made that expensive. The evidence it stopped on was about its own state, so it was guaranteed to find some. And a wrong stop looks exactly like a clean finish from outside: no error, no output, nothing that reads as a failure.

On your first question: I now require a stop to cite evidence the agent did not produce. Its own process, its own log, its own account of what it was doing — none of those count. A file that exists, a record visible outside the run, a number computed elsewhere.

And on the live-run data you said you can't claim yet: a pause count alone won't tell you what you need. Record whether each pause left the user with something they could act on, or with nothing. A boundary stuck fully on produces obedient, well-explained pauses and no work — and against a "did it pause where it should" metric, that reads as a pass.

The payment/account-access edge case is where I would test this first. I don't think the stop point should be “the agent says it is done” or even only “the next action is irreversible”; it should be a guard on a state transition.

For anything externally visible, I would want the evidence to bind four things: the original request, the policy/approval that allowed it, the external receipt or state observed after the call, and the idempotency key used to resume after a lost response. Otherwise a retry can create the exact failure the pause was meant to prevent: duplicate spend, duplicate message, or double fulfillment.

For validation, shadow mode seems ideal: record every point where the agent would have paused, then score it later as correct pause, noisy pause, missed pause, or unrecoverable action prevented. That gives you a product metric beyond “the boundary felt sensible.”

This is fundamentally a measurement problem. The agent stops when it thinks it's done measuring success. The founder stops when they've measured what they actually needed. The mismatch between those two measurement systems is the invisible part.

mihir_kanzariya nailed it: the brief said "don't sit idle" (measuring N actions) when the real requirement was "meet the day's target" (measuring output vs goal). Both rules were followed perfectly, both measurements were satisfied, outcome was still wrong. That's not a boundary failure—it's a measurement visibility problem.

What makes evidence so powerful isn't that it lets the founder second-guess the agent. It's that it forces both parties to measure the same thing. The agent shows: "Here's the number I'm stopping on" instead of "Here's the reason I stopped." The reason can be generated and always sounds plausible. The number shows whether the agent was measuring what the founder needed measured.

The real evidence question: can you name, right now, what artifact or number would make you say "the task is complete" without needing to read more context? If you can't name it, the agent won't find it either. If you can name it, the agent's evidence problem becomes your measurement clarity problem, which is solvable.

The reversibility test in the top comment is the right instinct, but it broke for me in a way worth naming: you can't classify reversibility from the action, only from the target's state.

I ran a deploy that was a plain directory sync — about as routine and reversible-sounding as an operation gets. It used rsync --delete with a trailing slash on the source. The rule was followed exactly. What it destroyed was everything in the target that wasn't in the source tree: my .env.local, five .bak files sitting right next to it, and a week of generated content. Nothing about the operation looked irreversible. The irreversibility lived entirely in state the source tree had no knowledge of.

I got lucky on blast radius for a dumb reason — an --exclude on the build directory meant the old process kept running with secrets already loaded in memory, so checkout stayed up the whole time. That's not a safety property, that's a coincidence.

So on your evidence question: the artifact I'd want isn't a diff of what changed, it's a dry-run list of what's about to be deleted that the agent didn't create. Additions are cheap to review and mostly safe. Deletions of things outside the agent's own output are where the unrecoverable stuff hides, and they're exactly what a success summary won't mention.

Building Genie 007 (an AI agent that runs outreach, publishes content, manages engagement) has taught me one thing about this: the right stop signal is not risk level, it's reversibility.

Ask if risky fails in practice because the agent doesn't share your risk model. What I've found works better: pre-enumerate which action classes are irreversible, not which tasks are "risky." Drafting an email is reversible. Sending it isn't. Scheduling a post is reversible. Publishing it isn't. The agent owns everything up to the irreversible step. At that step, it hands back with evidence: here's what I prepared, here's what I checked, here's what I'm about to make permanent.

What broke with written briefs: anything phrased as a principle ("pause if uncertain") doesn't survive first contact with an actual task sequence. The agent evaluates uncertainty locally, not relative to your intent. What works: a decision register — a short list of specific actions that require explicit confirmation, everything else is delegated by default.

The second question you asked is the sharper one: "which must stay explicitly yours?" For me it's first contact and anything that creates an external record I can't take back. Everything between initiation and irreversibility is agent territory.

Have you found that the approval boundary breaks down differently for agents that take sequential actions vs. agents that work in parallel branches?

I'd test the boundary in shadow mode before letting it control live runs. Have the agent record every point where it would have paused, then compare those records with the moments you actually stepped in or had to redo work. That shows you false positives and missed stops without adding friction to the run. A week of that data would tell you more than asking users whether the rules feel sensible.

We built a support agent and the thing that helped most was dropping the word risky.

Risky is a judgement the agent has to make about itself. We replaced it with something we could measure. If the search over our own help docs came back with nothing close enough, it stopped and handed over. Not because the task was risky, because it had nothing to answer from.

On top of that there is a short list of topics that skip the agent no matter how confident it looks. Payments, account access, anything that can lose someone's data. A person gets those on the first message. Being right almost every time is fine until the exception is somebody's money.

Your evidence question is the one I would think about most. What told us the stop points were right was not the stops. It was how many conversations the agent closed that came back within 48 hours. Deflection goes up whether it helped or gave up, so it hides the failures. Reopens show them.

I have not used a written execution brief the way you describe, so I cannot tell you what breaks there.

I run scheduled agent jobs, and the only trust signal that survived contact with reality is the receipt, not the reasoning. I've had an agent report a task complete with a perfectly plausible summary while the comment it was supposed to post never actually went out. Now every delegated task ends with a check against real state (file exists, comment visible, diff present) instead of reading the log. On where to pause: anything irreversible and externally visible — money moving, a message going out, a deploy — waits for me; the rest runs and I audit after.

Non-technical founder here too, working with Claude on my product daily. The pattern that's worked for me: anything irreversible or public-facing — sending a message, posting, deploying — needs an explicit "ok" from me first, even when the draft is ready to go. Everything reversible (research, drafting, browsing) it just does without asking. The boundary isn't about how risky the task looks, it's about whether I can undo it after the fact.

The interesting distinction is between evidence that proves the agent stopped correctly and evidence that merely explains why it stopped. A fluent explanation can make a bad stop sound completely reasonable, especially when the founder already wants to trust the agent. The stronger product promise may therefore be less about “approval boundaries” and more about making the stopping condition independently verifiable. That changes the buyer question from “Can I trust the agent’s judgment?” to “Can I quickly verify the evidence myself?” I’d be curious whether framing the product around that verification moment resonates more strongly than framing it around risk boundaries.

This is the boundary problem in a nutshell. The rule that sounds solid in a doc ("ask when risky") falls apart in practice because "risky" isn't a property of the task, it's a property of the founder's tolerance that day. What's worked better for me: instead of a static risk threshold, tie the pause point to irreversibility and blast radius. If a wrong call is cheap and reversible, let it run and just log it. If it's expensive or affects someone outside your own systems (money moving, a message going out, a commitment being made), it pauses no matter how routine it looks. I'm working on this exact tradeoff for founder-facing AI, and the thing that surprised me is founders don't actually want fewer interruptions, they want to trust that the interruptions they do get are the right ones. Curious whether your beta users complained more about too many pauses or too few.

This is the gap I keep hitting too. "Ask before anything risky" sounds like control until you realize the agent and the human disagree about what counts as evidence that the task is done. For me the stop point only feels trustworthy when I can inspect the concrete change (the files touched, the diff, the test output), not a fluent summary that claims success. If you cannot name what artifact would make you say yes without reading more, the approval gate is just a polite speed bump.

the stop rules that actually fire are the ones tied to something countable. we run an agent for our own marketing ops off a written brief, and "ask when it's risky" has never once fired, because the agent doesn't feel risk, while "stop after N actions today" fires every time because it's arithmetic. what broke for us was quieter than the agent ignoring its boundary: the boundary held fine and it was measuring the wrong thing. the brief said don't sit idle, so the agent kept checking "is anything owed right now" when the question that mattered was "does today's output meet today's target", and both readings were true at the same time. nothing was owed, the day was badly short, rule followed perfectly, outcome still wrong. we swapped that sentence for a script that computes the comparison and prints DAY COMPLETE or DAY NOT COMPLETE, since a rule that describes a check isn't a check. on your evidence question, i trust a stop when the agent can show me the number it stopped on rather than the reason it stopped. reasons are generated and always sound plausible.

The distinction between “ask for approval when it’s risky” and defining exactly what the agent needs to show before continuing is really important. ~

The second feels much more actionable.

I also like the idea of treating the evidence as part of the handoff, rather than just adding another approval step. If the human can quickly see what happened, why the agent stopped, and what it plans to do next, the approval becomes much easier to trust.

It's a delicate balance between keeping the agent in check and giving it enough autonomy. If you stop it too frequently, it can be frustrating and disrupt the flow, but if you give it too much leeway, you might not catch issues until it's too late. Finding that sweet spot is crucial, where the agent has enough freedom to operate efficiently, but not so much that it causes problems that go unnoticed.

I’d be interested to see whether the right boundary ends up being different for different types of tasks. A financial action, for example, probably needs a very different level of evidence than a routine research task.

The distinction between a boundary that looks sensible and one people actually reuse in a live run is important.

How are you planning to get the first few real runs where you can observe that?