From scoring forecasters to scoring whales: a 10-day pivot
10 days ago I posted "The Forecaster's Test" here, a 10-question calibration quiz that scores you on a Brier score. It hit the HN front page for ~6 hours and got a decent IH bump too. Final numbers:
- 1,588 quiz takers
- 10 signups
- 5 activated (logged a real decision)
- 0 paid
A 0.6% signup rate and a 0% paid rate is not a funnel problem you A/B-test your way out of. It's a product-market-fit signal. The quiz worked. People enjoyed it. Many shared their score. But the gap between "I want a number that says I'm sharp" and "I will pay to track my own forecasts every week for a year" was a canyon.
What I had after 10 days was 1,588 Brier scores from real humans on identical questions. That's a proprietary dataset most competitors would pay six figures to assemble.
The pattern that jumped out: most quiz-takers wanted the validation but not the work. They wanted to know if they were good. They didn't want to log a decision a week for the next year to find out empirically.
Meanwhile I'd been sitting on a much larger dataset on the side. 8,656 Polymarket wallets with full per-trade history. Same fundamental question (who is actually skilled), but answerable without the user doing any work, because the trades are public on-chain.
So I pivoted. Instead of selling people a tool to score themselves, I built a tool that scores the wallets they're already watching, and a public leaderboard anyone can browse for free.
Shipped in the 10 days since:
- Polymarket wallet analyzer (paste any address, get an Edge Score)
- V1 methodology paper. Spearman +0.148 between our score and out-of-sample profit. Top 1% of wallets capture 36.2% of all signed profit.
- Cross-venue extension (V1-M) covering Polymarket, Kalshi, and Manifold. Documented a -8.9 percentage-point concentration delta on a within-user sweepcash sub-sample (n=333).
- Public Truth Leaderboard with daily refresh.
- 24-hour follow-up email so analyzer users who don't sign up still see where their wallet score sits.
The takeaway: every funnel is a forecast about what users want. Mine returned a clear signal. The 1,588 takers told me the quiz wasn't the product. The 8,656 wallets told me what the product is.
Anyone here use their own conversion data as a research dataset that pointed to a different product? Curious how other founders read their funnels for product direction rather than for conversion-rate optimization.
convexly.app
This is a really sharp read of the funnel.
The part that stands out is the difference between “people want the result” and “people want to do the work required to get the result.”
That feels like a huge product lesson.
The quiz proved there was interest in the score, but the activation data showed that manual tracking was the wrong behavior to ask for.
Turning the funnel data itself into product research is a great way to frame it.
Curious how you’re thinking about monetization now that the product is moving toward wallet analysis and public leaderboards.
I know a few indie founders who've definitely used conversion data to pivot their product direction. I'm happy to ask them if they'd answer some questions you have about their experiences.
This is a great read — you didn’t just look at conversion, you interpreted behavior, which most people miss 👍
The key insight feels spot on:
→ people want validation without effort
→ not a system that requires discipline over time
That gap kills a lot of “self-improvement” products.
Your pivot makes sense because:
→ you removed effort
→ kept the core desire (who’s actually good?)
→ and used existing data instead of asking users to generate it
That’s a big shift from tool → insight layer.
One thing that stands out:
Your current product is strong on:
→ analysis
→ credibility
→ data depth
But the next unlock might be:
→ “what do I do with this?”
For example:
→ follow top wallets
→ alerts when high-edge wallets act
→ simple “copy / mirror signals” layer
Right now it answers:
→ “who is good?”
Next layer:
→ “how do I benefit from that?”
Also interesting:
you basically used your funnel as a research instrument, not a sales pipeline.
That’s rare — and powerful.
Curious:
→ are users spending more time exploring leaderboard
or checking specific wallets?
That’ll tell you where the real pull is.
Also, I’m running a small project (Tokyo Lore) where we surface ideas like this to a focused builder group and see what actually sticks in real usage.
Since you’ve already gone through a clean pivot driven by data, this could be a strong fit to test further direction.
Happy to share more if you’re open 👍