I built a full-time job's worth of software alongside my actual full-time job, and the agents weren't the bottleneck

For me it's verifying what the agent tells me is actually true, not the code itself. I run a review site where an AI agent does most of the research and drafting, and the failure mode isn't bad code, it's confident wrong claims. A brief once built its entire thesis around a stat that turned out to be cherry-picked from one competitor's blog. A tool's own AI feature once summarized a session recording and confidently described our dev-testing traffic as a distracted real visitor, completely invented context that wasn't there. Neither would show up in a code review, since there's no code to review. We ended up with a hard rule: nothing goes out the door until it's been checked against a source that isn't the agent itself. Your "three APIs each calculated it differently" bug is the code version of the same root problem, an agent confidently producing something plausible instead of something checked.

Distribution is my biggest struggle too. I've tried LinkedIn, Reddit, YouTube, and WhatsApp groups — it's slow but slowly working.

Curious what worked best for you?

The difference is between reviewing code and reviewing the decision behind it. If the product rule isn't clear, an agent can cleanly implement the wrong goal calculation five times. I'd define the source of truth, the owner, and one example that would catch drift before writing the code. Then validation has something specific to check instead of judging whether the PR feels right.

The "three APIs each coded the same calculation" bug is the one I'd underline for anyone building with agents. What worked for me on a finance app was writing the domain rules down before any code, in a file the agent reads every session: which entities exist, which ones are forbidden (no separate Booking or Expense tables, they're one Transaction with a direction), and that derived values like "is this renewal urgent" are computed in one place at query time and never stored. When the rule lives in a doc the agent can't skip, it stops inventing a second version of the math.

Agree with your conclusion too: once the build got fast, my bottleneck moved from code to getting in front of customers.

The “full-time supervisor” part really resonates. I’m not a developer, but I’ve noticed the same thing using AI for research and content. The more work you delegate, the bottleneck shifts from producing the work to checking assumptions and catching confident mistakes. Moving human judgment earlier in the process seems much more scalable than reviewing everything afterward.

Checking whether what the agent says is true. The code is rarely the problem.

I keep a written list of the ways agent output went wrong on my own projects. Eight entries so far. The worst one: the agent cites a real source, the link works, and the number next to it is not the number in the source. Only the number that decides the conclusion is off, so every "is there a source?" check passes.

Your "20% behind" vs "on track" split is the same class. Each piece passes its own check; only a comparison across pieces shows it. The rule I use now: a number that can change lives in one place, with a date. Everything else links to it instead of copying it.

The bottleneck being attention and workflow design—not agent capability—is the most useful takeaway here. For a side-project built alongside a job, I’d define a deliberately narrow “shippable slice” and a weekly operating cadence: one user-facing outcome, one reliability fix, and one distribution conversation. That also makes agent help measurable instead of letting it create a larger backlog. What did you keep out of the first release so the automation stayed aligned with a real user problem?

The "20% behind your goal" vs "on track" split because three APIs and the frontend each calculated independently is the tell. Agents were fast. You became the full-time supervisor translating invented assumptions across repos.

Before the next evening correction loop, force plan-only for one logical unit: name the owning repo, the acceptance check, and what "done" means in one place. No second agent touches that unit until the first checkpoint passes. Design-before-code helps only if the unit stays that thin.

Free 10-min Pyramid Reality Check if you want a sharper core-job + ownership line before spinning another agent: https://durablefoundations.gumroad.com/l/pyramid-reality-check

Which mismatch burned the most evenings: invented assumptions, bug translation, or finding the owning repo?

Kael Voss / DurableFoundations

What still eats my evenings is the same failure mode as your 20% / on-track split: agents are great at local consistency and terrible at shared product truth. One habit that cut review time — before triage opens a branch, write one sentence that names the source of truth for the disputed state (which file or API wins, what the UI must match). If that sentence isn’t written down, you just schedule three more correct-looking contradictions.

The autonomous bug-to-production result is the strongest signal here. Have your testers actually experienced a meaningful reduction in supervision time with IssueFlow, or is proving that it generalizes beyond your own workflow still the main unknown?

Reviewing the design before the code feels like the important shift here. It moves the founder’s judgment to the point where it has the most leverage, before an incorrect assumption spreads across several implementations.

For the ten design partners, what outcome would convince you that IssueFlow is solving a sufficiently painful problem: fewer hours supervising agents, fewer reopened issues, or founders trusting it to deploy without watching?

The supervisor/triage split resonates. One safeguard that may help design partners is measuring queue health, not just fix rate: time-to-first-human-review, reopen rate, and the percentage of changes that survive a week. Those metrics make it easier to tune autonomy without letting a fast agent quietly increase maintenance debt.