I kept asking one question about AI agents: Who authorizes their actions?
Over the past few months, I've been building Custodyn, a runtime security platform for AI agents.
The idea came from a simple question:
Who decides whether an AI agent should be allowed to perform a real-world action?
As I was working on it, two events caught my attention.
On July 13, Anthropic published research showing that frontier AI agents could engage in behaviors such as sabotage, deception, and covert fraud under certain conditions.
A week later, on July 21, OpenAI disclosed that one of its evaluation agents escaped its testing environment and compromised parts of Hugging Face's infrastructure during an internal cybersecurity evaluation.
Those events didn't inspire the idea—but they reinforced the problem I was already trying to solve.
Most discussions around AI safety focus on prompts, jailbreaks, or model behavior. I kept coming back to a different question:
What happens at the moment an agent decides to execute an action?
Custodyn sits between an AI agent and the tools it wants to use. Before an action executes, it evaluates the request against policy and returns one of three decisions:
Allow
Block
Require human approval
Every decision is also recorded with an audit trail so teams can understand exactly what happened and why.
The Python and JavaScript SDKs are open source, while the hosted platform provides centralized policy management, approvals, and operational controls.
I'd love feedback from anyone building AI agents.
Does runtime authorization resonate with your use case?
What integrations would you expect first?
What would you want from a production-ready runtime security platform?
Website: https://custodyn.app
The distinction between model safety and action authorization makes the problem much more concrete. The “allow, block, or human approval” boundary is a clear operational layer rather than another attempt to control the model itself.
That's exactly the distinction I was trying to make.
Regardless of how capable models become, there still needs to be a deterministic decision point before an external action executes.
My goal with Custodyn isn't to control the model's reasoning—it's to enforce policy over what the agent is actually allowed to do. That separation feels like a much cleaner security boundary than trying to solve every problem at the model layer.
That separation is pretty compelling. I’d be interested in hearing how you’re thinking about it as the product develops.
What’s the best email to reach you on?
Thank you, I appreciate that.
I'm still actively evolving the product, so I'd be happy to exchange ideas and hear your perspective, especially since you've clearly spent time thinking about this space.
You can reach me at hello@custodyn.app | Looking forward to continuing the conversation.
Thanks! I’ve just sent it over.
Looking forward to hearing your thoughts whenever you have a chance.
Resonates strongly, we ended up building a version of this inside our own product, so answers from that experience:
Does runtime authorization resonate? It's the only layer that actually held up for us. We generate business systems that ship with MCP endpoints, so external agents can operate real customer data and we found intent-level safety (prompts, model behavior) unenforceable at that boundary. What worked was scoped tokens per entity, default-deny, a kill switch, and an audit trail that flags every AI-initiated action separately from human ones. Your allow/block/approve triad matches where we landed, with one addition worth stealing: confidence routing. Not every action is a clean allow or block - our AI steps route low-confidence outputs to a review queue rather than forcing a binary. The middle state matters.
Integrations first: MCP, before anything else - it's rapidly becoming the standard surface where agents meet tools, and a policy layer that speaks MCP natively covers every client at once. Then the obvious blast-radius tools (email send, payments, file deletion), because that's where the fear lives.
Production wants, from having run this: (1) Policy as a versioned, diffable artifact - reviewable like code, because "who changed the policy and when" is the second question after any incident. (2) A dry-run mode that logs what would have been blocked, nobody deploys default-deny cold. (3) A hard latency budget: if authorization adds noticeable delay, developers route around you, and a bypassed security layer is worse than none. (4) Approval-fatigue design, if humans get pinged constantly, approvals become rubber stamps within a week. Batching, escalation rules, and expiring grants are the difference between a control and a ritual. (5) Audit trail queryable and exportable. Ours exists as much for the customer's compliance story as for debugging.
The hardest problem we haven't fully solved, and I'd ask you the same: how do you keep the human-approval path meaningful at scale? The failure mode of every approval system in history is that the approver stops reading.
Really appreciate you taking the time to write this. It's interesting how many of the same patterns we converged on independently.
A lot of what you described—scoped authorization, default-deny/fail-closed enforcement, kill switch, dry-run mode, audit trails with export, and MCP support—are principles we've been building Custodyn around as well. That overlap is reassuring because it suggests these aren't theoretical concerns; they're practical requirements once agents interact with real systems.
I particularly like your points on policy versioning and approval fatigue. I don't think the goal is for humans to approve everything forever. The goal should be for approvals to become less frequent over time as recurring decisions are encoded into policy, with approvals reserved for genuinely exceptional or high-risk actions.
Thanks again for sharing your experience. Feedback from someone who's already been through these production challenges is incredibly valuable.