Our AI workflow said “success.” It shipped nothing.

A workflow said “success.” The ledger recorded 14 permission denials and no artifact.

That happened on August 22 while running AI-assisted operations for Simple Memo, the iPhone app we develop. It is the reason our public report counts shipped work separately from successful workflow exits.

Here is the complete 23-day snapshot, from August 11 to September 2, 2026 (Japan time):

• 41 ledger records in total.
• 28 attempted runs: 19 shipped, 6 failed, 2 produced no artifact, and 1 was cancelled.
• 7 of those 28 attempts recorded human intervention.
• The remaining 13 records were gate skips or no-run records, excluded from those denominators.

A “shipment” here can be an article, a website update, or an operations change. It does not mean an App Store release. The intervention count records whether someone stepped in, not how many minutes they worked.

Three failures explain what the 67.9% shipment rate leaves out:

1. A green status was not proof of delivery.
The August 22 run reported success without producing an artifact. For an operations task, the evidence has to include the result: a file, a change, a deployment, or another output that the task actually required.

2. A fallback could block the primary route without shipping anything.
On August 29, the fallback claimed the day’s branch but produced neither an article nor a PR. The primary route then treated the claim as work in progress and skipped. A claim by itself was too weak a signal.

3. Two routes could fail against the same limit.
On August 30–31, the primary route hit an account usage limit. The August 31 fallback shared that account and also failed. Different schedules did not make the underlying dependency independent.

For a small team, I would start an operations ledger with the intended result, the observed result, the artifact link, the failure reason, and any human intervention. Keep skips visible, and decide what belongs in the denominator before reporting a success rate.

We published the report, all 41 CSV rows, definitions, and source snapshot. It is one app and a short, operator-maintained record; some entries were reconstructed from workflow history. It does not establish hours saved, revenue impact, or that the business runs without humans.

If you run agents on recurring work, what evidence do you require before marking a task as done?

Disclosure: I develop Simple Memo. This write-up was prepared with AI assistance from our published operations records.

Simple Memo Developer