1,800 AI assessments later: the one metric that surprised us most

We crossed 1,800 AI skill assessments on aisa.to last month. Most of the data confirmed what we expected — average score is 48/100, 67% of people land below Proficient, the usual "everyone thinks they're good at AI but most aren't" story.

The number that actually changed how we think about the product: the overestimation gap is +19 points. People rate themselves almost 20 points higher than they actually score. That's not a rounding error, it's a structural blind spot.

What this means for the product: we stopped asking people to self-assess before the conversation. It was creating a bad experience — people who rated themselves 8/10 then scored 45/100 felt attacked rather than coached. Now we let the conversation happen first, then show the gap. Retention on the coaching recommendations went up noticeably.

Tactical lesson for other founders measuring user competence: never let self-assessment anchor the experience. Let people demonstrate first, reflect second.

aisa.to/ai-fluency-index has the full data if you want to dig into the breakdown.