Talk to customers is unfalsifiable in the hands of the person who wants the answer

Talk to customers is the standard answer and it is not wrong, it is unfalsifiable in the hands of the person who wants the answer. Here is the version I could actually hold myself to.

The failure is not the interview. You ask ten people whether they would pay, seven say something encouraging, and you now hold ten data points plus the freedom to decide afterwards which of them counted. You will decide in the direction you were already leaning, because you wrote the questions and you are the one who has to abandon eight months of work if the answer is no.

You cannot fix that by being more rigorous in the room. The bar moved because nothing was holding it in place.

So: pre-registration. Before the test runs, write down what would count — a number, a metric and a deadline. Then run it and compare. The ORDER is the entire trick, and it is the only reason a founder's own report about their own idea is admissible evidence at all.

The template I use, and if any line is blank the test is not pre-registered: the promise being tested, the named buyer, the channel, the observable action, the minimum result, the measurement window, and what I will do if the bar is missed.

Three rules that made the difference, all of them about which direction a number can move.

A blank is not a zero. If you did not report something, it grades as not reported rather than not met. Collapsing those makes an empty form read as evidence the idea failed, and I did not measure it is not that.

The window only runs one way. Hitting the bar faster passes. Taking longer does not, however good the number eventually is — otherwise the time-box is decorative.

And you cannot correct upward. The weakest report you file is the one that stands. Reporting a worse number takes effect immediately; a better one leaves the earlier grade in place. A bar written in advance is worth nothing if the answer can be rewritten after you see how it graded.

Now the part I had to accept about my own method. A landing-page test that goes well proves some specific things and is silent on others: in our own scorecard the widest single metric can evidence 55 percent of it, and 45 percent — support load, platform exposure, cash flow — it cannot reach at all. Traffic on its own evidences nothing. A channel is proven when it delivers a person, not attention.

The template and the full accounting: https://whittleos.com/guides/validate-startup-idea-without-talking-to-customers