I had an agent send duplicate emails to the same user because of a retry. Nothing crashed. But it shouldn’t have happened. This gets worse in production when real money or user actions are involved.
It’s not a catastrophic failure. More like a slow, quiet leak that adds up over time.
Why it happens
Most agent frameworks retry failed or slow tool calls. That’s useful for reliability, but risky when the action isn’t reversible — sending emails, charging cards, writing data.
A retry just runs the same operation again.
The fix: idempotency (done properly)
A simple cache isn’t enough. You need a consistent way to ensure the same action only executes once across retries.
Here’s the core idea:
import { guard } from '[@keelstack](/keelstack)/guard';
// Agent calls sendWelcomeEmail(). Network issue → retry.
// Without guard → email sent twice
// With guard → email sent once
const result = await guard({
key: `send-welcome:${userId}`,
action: () => resend.emails.send({ to: user.email, subject: 'Welcome' }),
});
// result.status → "executed" | "replayed"
The action runs once per stable key. Retries just replay the result.
What turned out to be necessary in production
Idempotency alone wasn’t enough. Two more controls became important:
All together:
const result = await guard({
key: `agent-action:${userId}:${taskId}`,
action: () => stripe.charges.create({ amount: 9900, currency: 'usd' }),
budget: { id: userId, limitUsd: 50 },
extractCost: (res) => res.amount / 100,
risk: { level: 'irreversible', policy: 'log' },
});
The package is @keelstack/guard (MIT, zero deps).
Even if you don’t use it, the patterns matter:
Is anything critical missing from this pattern, or would you approach it differently?