I have one AI write my code and a second one review it. Yesterday the reviewer failed 2 changes out of 5. When do you stop and drop a change

The part that'd bug me isn't the two-round cap itself, it's that you don't have anything telling you whether two rounds is even the right number. If you never spot-check a handful of these yourself — read the actual diff on a change that passed clean, or one that got dropped after two failures — you're trusting the reviewer's calibration without ever independently checking it. Two AIs agreeing "this is fine" isn't the same as it being fine; it's just two models agreeing, which is closer to the reviewer grading its own homework than an actual check. Might be worth manually auditing 1 in 10 passes, just to see if the rule is catching what you'd actually want caught.

I use a hard stop after 2 review rounds on the same change. If the reviewer still fails the same acceptance check, I drop or rewrite that slice instead of retrying. Numbered criteria help, but looping the same failing item usually just burns tokens.

The failures landing on instruction changes isn't a coincidence — it's the signal. A change to written instructions fails review because the instructions were ambiguous, not because the execution was wrong; so a retry means running the same ambiguity through the same checklist again, which is why round two rarely helps there. Worth splitting the outcome three ways before deciding: spec-gap (fix the checklist — the change becomes a new change and the timer resets), execution-error (genuinely worth a retry), edge-case (add it to the checklist as a new line). Only one of those three is a retry. Your two-round rule still holds — it just shouldn't count spec rewrites as rounds.

the fact that your failures were on instruction changes, not code, is the interesting part. means your review criteria is actually working. for the drop-vs-retry call, I'd weight it by blast radius: user-facing flows always get a retry, internal docs or config get dropped. we see the same pattern assessing AI verification habits at aisa.to — the skill isn't knowing how to check, it's knowing what's worth checking.

The round count may be the wrong variable. What predicted "another round will help" for me was the kind of claim that failed.

Three kinds, and only one of them is worth a second round:

Claims about your own intent — the spec. More rounds don't help, as the comment above says. Rewrite the list.

Claims about the outside world — does this API exist, does this platform actually behave this way. Don't review these, verify them. I lost most of a morning this week to one: I fixed a "nothing happens when you tap the button" bug by routing the call through a method on a platform SDK. It compiled, it ran, and it did nothing, because that method was removed two major versions ago. My code caught the failure and fell through to a fallback, so it looked fine. No reviewer catches that. Opening the package's type definitions catches it in twenty seconds.

Claims about behavior you can't reproduce — a specific device, a store's throttling, a quota. Neither review nor rounds resolve these. The only move is to stop being clever and take the deterministic path.

That last one produced both of the real bugs I shipped this week, and they had the same shape: a call that reports success when nothing happened. The rating API returned "true" because it didn't throw, while the OS had silently decided not to show anything — so the fallback that would have saved it never ran.

The inverse is worth budgeting for too. Yesterday I wrote up a defect I was sure of, and a comment in the same file refuted it — the platform's limit was on displays, not calls, so the thing I "found" wasn't a thing. Reading the file's own comments before proposing the change would have cost two minutes.

So: before spending a second round, ask whether the claim is checkable outside the model. If it is, check it — that's cheaper than a round. Rounds are for judgment, and judgment is the smallest part of what fails.

(Context for the examples: I build SIGNUM HQ, a free US-market data app — options flow, dark pool share, GEX, max pain. iOS and Android: https://www.signumhq.com/app?from=indiehackers )

The most useful gate for me is whether a failure is actionable and observable: does the reviewer quote the violated acceptance criterion, point to a reproducible test, and distinguish spec ambiguity from an implementation defect? If a second pass gets the same evidence, drop the change or rewrite the requirement; if it exposes a new failing path, keep it. That makes the two-round budget a decision rule, not just a retry count.

Both failures landing on written instructions rather than app code is the part I would act on first. If the reviewer scores a change against a numbered list, and the change being scored is itself a change to that list, then the thing under test is your spec, and no number of writing rounds fixes a spec problem. What helped me was moving the review earlier: before anything gets written, hand the reviewer only the numbered list and ask it where the list is ambiguous. That catches the five out of six kind of failure while it still costs two minutes instead of a morning. Your two round rule can stay exactly as it is once the spec has survived a pass on its own.

What decides it for me is whether the objection names something I can go and look at: a file, a line, a value, a case it claims will break. Then another round is worth it, because the second pass has something concrete to check itself against, and it either holds or it does not.

If the objection is a judgment, which is most of what you get on prose rather than code, the second round just rerolls the dice. Two models will keep splitting on the same line until the line changes. That is DannieDan's underspecified-spec point from the other end.

Does yours quote the line it failed on, or just give the verdict?

The five-out-of-six instruction failure is a useful distinction from a six-out-of-ten miss. I would set the retry budget by blast radius and by whether the failure exposes a missing rule, then rewrite the numbered list before rerunning it. Do you track which failed changes later become bugs or support issues?

The thing I would change first is not the number of rounds, it is what the reviewer hands back. Right now a fail is one bit of information, and you are trying to make a judgment call out of one bit.

I spend a lot of my week around human code review, and the split that actually drives the decision is the reason, not the count:

Spec was ambiguous. The reviewer is really flagging your numbered list, not the change. A second pass against the same list fails the same way, so the work is fixing that line in the spec.

Writer was missing context. It never saw the file or the earlier decision. Cheapest retry there is, and the one with the highest hit rate.

Real defect. The hole you would not have spotted. Always worth another round.

Reviewer taste. It failed something your rubric does not actually require. That is not a failure, it is rubric drift, and it quietly inflates your failure rate until you fix the rubric.

The rule that falls out of that: never retry with the same inputs. If nothing changed between attempt one and attempt two except sampling, you are not iterating, you are rerolling the dice. Since your reviewer is already pass or fail on the whole thing, make it print the category and the exact spec line it violated. The triage becomes mechanical instead of a feel, and the two-sentence change stops eating a morning.

One cheap addition: keep a one line log of what you dropped, then note later if any of it came back as a bug or a support ticket. After a month that tells you whether two rounds is too generous or too tight. Right now that number is a guess, for you and for everyone answering you.

The bit that stuck with me is the hole you never would have spotted, sitting inside a failed instruction change, not the app. That one already paid for the review, even if the score was only five out of six. For the two-sentence change that eats a morning, I would stop the second you can describe that hole yourself. Another model pass after that is just credits.

One rule I use with AI builders that makes failed reviews easier to triage: make the writer read the existing code first and make only minimal edits. Then most fails are either scope drift or a real logic miss. Drift is cheap to fix by shrinking the diff instead of running another full round, so I wouldn't count it against a two-round cap.

Both failures were on written instructions, not app code. If the writer and the reviewer split on the same numbered line, another round is usually two models arguing over an underspecified spec. I'd drop that change, rewrite the one line they disagreed on, and give the writer a single new pass. Keep the two-round cap for Shopify code, where a miss is a bug. For the prompt, the split itself is the review.

The judgment problem is the hardest part of any AI-assisted workflow. The two-round rule sounds clean, but you're right that it breaks down when failure severity varies wildly. One heuristic that might help: weight by blast radius. A failed change to a shared utility or auth flow gets more rounds than a cosmetic fix. For solo Shopify devs, separating "did it break something real" from "did the AI miss a nuance" might save you from over-investing in low-stakes rollbacks.

The two-round rule is simple, but the failures have different consequences. Are you finding that the decision to retry should depend on the change’s impact, not just the review count?