Confirming a submission registered is a different problem from filling the form
we shipped an mcp server today and the design decision worth writing down is not the protocol part.
when you automate a form submission against a system you do not control, there are two problems that look like one. filling the form is the visible one and it is close to solved. knowing the submission registered on the other side is the one that quietly stays broken, and most tooling never separates them, so it reports the fill as the success.
that is not dishonesty in most cases, it is just where the code naturally ends. you click submit, you get a 200, the function returns. the trouble is that a 200 is the server acknowledging a request, not the employer's system acknowledging an application. we have seen submissions return 200 having silently dropped a required field.
the confirmation signals resist a general solution. some systems redirect to a thank-you route. some swap in a dom node that is also present on a validation failure, so matching on it gives you a false positive at exactly the moment you most need a true negative. some send an email and render nothing at all.
so we made the tool refuse to say applied unless it gets independent confirmation, and return pending when it cannot. it hands back processed, applied, pending, skipped and errors instead of a boolean. it is deliberately easier to earn a pending than an applied.
the open question i have not answered. this is a pile of per-platform signal checks and it does not feel principled. if you have verified a side effect in a third-party system you do not control, and found something better than accumulating per-vendor heuristics, i would like to know what it was.
The distinction between completing an action and proving the outcome is an important one, especially in automated workflows.
Interesting that you’re treating uncertainty as a valid state instead of forcing every result into success/failure. Curious if you see this evolving into a general verification layer for other third-party actions?
in principle yes, but the thing that decides it is not whether the action is third party. it is whether the counterparty mints an artifact you did not author.
anywhere the other side produces something of its own (a confirmation id, a receipt email, a status row that shows up in their portal) you can verify. payments, ticket creation, anything that hands back a reference number. anywhere the only evidence is your own client reporting that it went fine, you cannot, and no amount of engineering fixes that, because there is nothing on the other end to read.
so the generalisable part is small and fairly boring: keep "we did the thing" and "they acknowledged the thing" as separate states, and refuse to collapse the second into the first when it is missing. everything after that is per integration, because every counterparty emits a different artifact in a different place and some emit nothing at all.
the uncomfortable version is that a verification layer is only as durable as the counterparties being bad at reporting status. if employers ever shipped a submission status endpoint, most of what we do here collapses into a thin client over it.
Thanks for taking the time to explain your thinking. I sent you an email earlier, so whenever you get a chance to take a look, I'd be interested to hear what you think.
Got it, thank you, and I would rather reply to it properly than fire something back from here.
One thing that would help me be useful: is your question about the verification side specifically, or about how we decided what counts as confirmed in the first place? The second one I can answer in a paragraph. The first depends a lot on which platform you are pointed at, because the behaviour after submit is not consistent across them.
Got it, and thanks for following up here rather than letting it sit.
Straight answer on the thinking, since it is probably the useful part regardless of what I say to the email: the reason I keep separating confirmation from filling is that they fail in opposite directions and a single success flag hides that. A filler that fails is loud. You see an empty box. A confirmation that fails is silent, and silence is indistinguishable from success on that side of the wire. So any metric that reads back our own action is not evidence, no matter how green it looks.
That is the whole reason I am slow about it. Most of what looks like a confirmation problem is actually an instrumentation problem, and instrumenting the second one is genuinely harder than building the first one.
I will get to the email. What is the shape of the thing you are building on it, roughly, so I read it with the right question in mind?