I built an “AI Agent Black Box” because debugging agents became impossible

Over the last few months, I’ve been building AI agents with tools, memory, retries, workflows, and multiple LLM calls.

At first everything looks simple.

Then production happens.

Suddenly you’re trying to answer questions like:

• Why did the agent call the wrong tool?

• Why did this run succeed yesterday but fail today?

• Why is latency randomly exploding?

• Which prompt actually caused the bad output?

• Why are logs completely unreadable once agents become multi-step?

I realized something:

Building AI agents today feels a lot like debugging distributed systems in the early cloud era.

We have incredible frameworks:

• Vercel AI SDK

• LangChain

• CrewAI

• OpenAI SDKs

…but once agents become autonomous and multi-step, debugging becomes chaos.

So I started building TracePilot AI.

The idea is simple:

Instead of digging through terminal logs and console spam, you can replay the full execution of an AI agent step-by-step.

TracePilot captures:

• LLM calls

• Tool executions

• Errors

• Latency

• Token usage

• Agent reasoning flow

• Retries

• Parent/child spans

• Full execution traces

Almost like:

“Chrome DevTools for AI agents”

Example

A real issue I hit recently:

An agent was intermittently failing in production.

Normal logs showed:

Tool execution failed

That was it.

With tracing, I could actually see:

1. The exact prompt sent to the model

2. The malformed tool arguments generated by the LLM

3. The retry loop

4. The latency spike

5. The downstream failure chain

The bug took 5 minutes to fix instead of hours.

What I’m focusing on right now

Current priorities:

• TypeScript SDK

• Easy integration with existing AI apps

• Trace replay UI

• Better observability for autonomous agents

• Production debugging workflows

Planned:

• LangChain integration

• CrewAI integration

• OpenTelemetry support

• Team collaboration

• Production analytics

Why I think this space matters

I honestly think “AI Observability” will become a massive category.

Right now most people focus on:

• prompts

• models

• agents

But once companies deploy agents into production, reliability becomes the real problem.

And reliability requires visibility.

I’d love feedback

Especially from developers building:

• AI agents

• multi-step workflows

• tool-calling systems

• autonomous AI apps

What’s the hardest debugging problem you’ve hit so far?

And what’s missing from current tooling?