The fix for "we don't know what happened" depends on one thing: can you ask?
Yesterday's post got two replies that moved the problem forward more than the post itself did.
James (building UtilitySEO) pointed out the same ambiguity shows up in web crawling: a 403 might mean forbidden, or might mean a CDN edge challenged the request for not looking like a browser. Seventeen pages came back undecided in one scan — every single one turned out fine on a second look. His framing: "rare enough to handle manually, until it isn't" — Stripe webhooks, phone automation, and crawling all eventually hit enough volume that manual review of the ambiguous cases stops being an option.
octyn asked the question I didn't have a good answer to: for a Stripe webhook, you can query Stripe's API afterward and ask "what's the actual status of this charge," independent of whether your own handler crashed. Is there an equivalent for StareBrain — can I ask the phone carrier or the recipient's device "did this call/text actually complete"?
Honest answer: sometimes, sometimes not. SMS delivery receipts exist in some pipelines. Phone calls mostly don't have an equivalent I can reach in real time. Web crawling, per James, doesn't either — you can't ask a CDN "was that a real forbid or an edge challenge," you can only retry and look harder like a browser.
So the actual shape of the fix isn't one thing. It's two different problems wearing the same symptom:
A source of truth exists elsewhere, and you're just not querying it. Fixable with more integration work, no new state needed. This is James's crawling case (retry looking more like a browser) and probably some fraction of Stripe cases.
No source of truth exists, full stop. The ambiguity is real and permanent from your side. This is most of StareBrain's phone-call case today. The only honest move is flagging it for a human, not chasing an API that doesn't exist.
Yesterday's post treated this as one gap. It's actually two, and knowing which one you're in changes whether you should be writing more integration code or building a better human-review flow. I was collapsing them together until these two comments forced the split.
Really solid approach — I'm juggling something similar myself (building Xstream4K on the side), what's been the hardest part for you so far?
The split into "a source of truth exists, you're just not querying it" versus "no source of truth exists" is really useful. Building on octyn's point about treating every manual resolution as data: one thing we've seen in sprint retros is that the undecided bucket grows quietly, because each case feels rare on its own. Tagging each manual resolution with which of your two buckets it fell into would also tell you where to spend effort: if most land in the first, it's integration work; if the second keeps growing week over week, it's time to invest in the review flow.
That's a concrete thing I could actually go build today, which is more than I can say for most of the ideas in this thread (mine included). Right now StareBrain doesn't tag anything when it flags an action - it just says "unresolved," full stop. Adding "which bucket" as a required field on every flag would cost almost nothing and would answer the exact question I don't have data for yet: is the phone-call case actually all bucket two, or is some fraction of it bucket one and I just haven't gone looking for the API that would resolve it.
The part I like most is that it turns "I think this is mostly unsolvable" from a guess into something I could disprove in a month by just looking at the tag counts.
Do you tag at the moment of resolution, or at the moment something first gets flagged - before anyone's actually looked at it and knows which bucket it's really in?
the tag is the part that closes the loop. the queue count tells you how much ambiguity you're carrying, but the tag tells you which direction it's drifting, and that's the decision that matters. and it costs the reviewer almost nothing, they're already resolving the case, the label just rides along.
the human-review flow has its own failure mode. a flag queue fills up with the same three causes dressed as new events, and the reviewer becomes the API you wished existed. the version that works treats every manual resolution as data: resolve enough of cause X and it graduates into a rule or a retry, and the queue actually shrinks. if the queue stays flat week over week, the review flow is just the old swamp with better labeling
That's the piece I was missing - I'd been treating "flag for a human" as the end state, and you're right that it's not, it's just a cheaper starting state. If nothing ever graduates out of the queue, you haven't solved the ambiguity problem, you've just paid a person to sit inside it instead of a machine.
One thing I'd want to be careful about though: graduating cause X into an automatic rule or retry is exactly the move that's safe for my crawling/webhook cases (case 1, where a real source of truth exists and you just need to query it right) but dangerous for the genuinely unresolvable ones (case 2, like StareBrain's phone call case) - because if you're wrong about which bucket cause X is actually in, you've just automated away the human check that was catching a real ambiguity. So the queue shrinking is good evidence the flow works, but only if there's also a way to catch a bad graduation before it quietly starts mis-resolving things the same way auto-merge would have.
Have you seen a queue where something got graduated too early, and how'd you find out?
we got the lucky version of this. for onchain actions the chain itself is the source of truth, so "did it happen" is always askable. the ambiguity just moves to "has the indexer caught up yet", and there the honest fix is the boring one: tell the user to check again in a minute
That's actually a third case I hadn't separated out yet - not "no source of truth" and not "source of truth exists, just need to query it," but "source of truth exists, and you can query it, it just might lie to you for a bit because it hasn't caught up." Different problem than either of mine, and honestly a nicer one to have, since at least you know it'll resolve itself given time instead of staying ambiguous forever.
Does "check again in a minute" ever get automated on your end - like the UI silently re-polls and updates once the indexer catches up - or is it genuinely just telling the user to go refresh manually?