the doc I write before every AI agent now
Month 2 of running 10+ AI agents for PM work, my Claude bill was 3x what I expected. No clear reason - every agent was "working."
Turned out I had no budget constraints in any of them. Each one had free rein on tokens and scope. Made sense at the time.
Now every agent gets a constraints doc before I deploy: max tokens per run, and which decisions stay with me. Writing "what this agent can't do" takes 3-4 iterations. It's genuinely harder than "what it does."
Two things improved: bill down ~40%, and I stopped getting silent completions - agents succeeding at tasks that weren't quite their job. Those are the worst to catch.
Next step is making the constraints doc the first required artifact in my deployment flow, not an afterthought.
Do you write anything like this before you hand work to an agent? Curious if there's a cleaner format than what I'm using.
This is a very useful distinction.
I like the idea that the constraints doc should define not only what the agent should do, but also what it is not allowed to decide. That feels especially important once you have multiple agents running in parallel, because cost problems and scope problems start looking almost the same.
The format I’d probably test is something like:
The “silent completions” point is the most interesting to me. A task can look successful while still burning tokens in the wrong direction.
We’ve been thinking about this from the API/model workflow side too: constraints, routing, and cost visibility probably need to live closer to the agent deployment flow, not only in a dashboard after the fact.
'not allowed to decide' is the part i'd add explicitly too. running multiple agents, scope and cost bleed together fast - one agent quietly absorbing decisions that were supposed to route elsewhere. your stop conditions field is the thing most setups skip. that's usually where drift starts.