The metric that actually moved our retention (and it wasn't what we expected)

Building aisa.to (AI skills assessment through conversation). Quick tactical update on something we learned this month.

We kept optimising for assessment completion rate. Makes sense, right? More completions = more data = better product. Got it from 71% to 84% over three months.

But retention didn't move. People would finish one assessment and never come back.

The thing that actually changed behaviour: showing people their specific skill gaps compared to their peer group. Not just "you scored 52/100" but "your verification skills are in the bottom 30% of people in your role."

Turns out nobody cares about an abstract score. Everyone cares about where they stand relative to people like them. We added role-based benchmarking from our dataset (1,000+ assessments now) and repeat assessment rate jumped.

The lesson for other builders: if your product generates any kind of score or result, the raw number is almost worthless for engagement. The comparison is everything.

Full data on AI skills patterns: https://aisa.to/state-of-ai-fluency