I logged why an automated filter rejected 895 of 1,728 startup ideas
I build a tool that screens startup ideas for solo founders, and it keeps a record of every candidate it refuses and which check it failed. I had never looked at the whole pile at once. Last week I did, and the distribution surprised me.
33 runs, June to September. 1,728 candidates in, 895 dropped — 51.8%.
The single most common reason, by a distance: 496 ideas were dropped because I could not find a single documented complaint behind them. Not "the market is crowded", not "the economics don't work" — nobody had written the problem down anywhere I could find it. That is 55% of all refusals.
After that it gets more specific:
270 — too much hands-on support for one person
193 — leans too hard on one platform
148 — just an AI wrapper with no product underneath
129 — can't charge enough
109 — hard for a solo founder to sell or run
And the part I keep thinking about: 69% of the dropped ideas failed more than one check. A weak idea is rarely weak in one place. The worst one failed four.
Three things this is not, before anyone asks:
It is not an accuracy claim. I have no idea whether the 895 would have sold. Nobody does. All I can say is that the gate was applied consistently and wrote down its reasons.
It is not a market sample. These runs are overwhelmingly my own accounts — I have had one paying customer. This is a record of a machine's behaviour, not of demand.
The checks are my opinions, encoded. If you think "no documented problem" is a bad reason to drop an idea, that is the conversation I actually want.
The data is public and reusable (CC BY 4.0) if anyone wants to argue with it or reuse it: https://whittleos.com/benchmark — the JSON is at https://whittleos.com/gate-census.json
The question I cannot answer from my own data: for those of you who shipped something that worked — was there a written complaint behind it before you started, or did you find the problem by living it?
Thanks for sharing the numbers, that makes it much easier to follow.
Solid lesson. Which channel has worked best for you so far?
One channel has ever produced a paying customer: PeerPush, a small directory. Everything else — search, the listings, the writing — has produced readers and no money. And "customer" is singular: one person, who paid, ran a lot of analyses in an evening, hit a slow batch, and never came back. That's the honest state after two months, and I'd rather say it than describe a funnel.
Good write-up. What would you do differently if you started again?
I'd start the distribution work on day one instead of after the product was good. I spent months making the thing defensible and then discovered the hard part isn't the product. Would I recommend the approach? Only the honesty part — building the thing that refuses to flatter the user is worth it and it does earn respect in threads. It has not yet earned revenue, so treat anything I say about it as a report from the middle, not a lesson from the end.
How did you decide this was worth building in the first place?
Badly, by my own current standards. I had a problem I'd personally hit and I built for it without doing the validation my own tool now insists on. The tool exists partly because I did it the expensive way first.
Helpful post. How did you get your first bit of traction?
One channel has ever produced a paying customer: PeerPush, a small directory. Everything else — search, the listings, the writing — has produced readers and no money. And "customer" is singular: one person, who paid, ran a lot of analyses in an evening, hit a slow batch, and never came back. That's the honest state after two months, and I'd rather say it than describe a funnel.
Interesting take. Would you still recommend this approach to someone starting today?
I'd start the distribution work on day one instead of after the product was good. I spent months making the thing defensible and then discovered the hard part isn't the product. Would I recommend the approach? Only the honesty part — building the thing that refuses to flatter the user is worth it and it does earn respect in threads. It has not yet earned revenue, so treat anything I say about it as a report from the middle, not a lesson from the end.
Great breakdown. What feedback have you had from early users?
The useful feedback has all come from threads like this one rather than from users, because there aren't enough users yet to be a source. Several things shipped this month came straight out of comment replies. As for measuring: I wrote down in advance what would count as a product versus a toy, with a date, so I can't move the line later. The honest caveat is that nearly every channel is sitting at zero, so the measurement hasn't really been tested yet — a bar you can't fail isn't a bar.
Interesting. How are you measuring whether it is working?
The useful feedback has all come from threads like this one rather than from users, because there aren't enough users yet to be a source. Several things shipped this month came straight out of comment replies. As for measuring: I wrote down in advance what would count as a product versus a toy, with a date, so I can't move the line later. The honest caveat is that nearly every channel is sitting at zero, so the measurement hasn't really been tested yet — a bar you can't fail isn't a bar.
Curious how long it took before you saw the first real results?
Badly, by my own current standards. I had a problem I'd personally hit and I built for it without doing the validation my own tool now insists on. The tool exists partly because I did it the expensive way first.
One channel has ever produced a paying customer: PeerPush, a small directory. Everything else — search, the listings, the writing — has produced readers and no money. And "customer" is singular: one person, who paid, ran a lot of analyses in an evening, hit a slow batch, and never came back. That's the honest state after two months, and I'd rather say it than describe a funnel.
Interesting approach. What was the hardest part to get right?
Making the system comfortable returning nothing. Every instinct — mine and the model's — is to produce something when asked. A market that honestly yields no ideas has to come back empty, after the user has paid, and the page has to explain why that's the correct output rather than a refund conversation. That was harder than any of the analysis.
Thanks for writing this up. Bookmarking it for later.
Thanks for writing this up. Bookmarking it for later.
Thanks. If you get to it — the bit I'd like a second opinion on is whether
"no documented problem" (55% of the refusals) is the gate being right, or me
over-weighting written evidence. Did the problem behind your last project exist
in writing before you started?