Executives score higher than individual contributors on AI fluency — but not where you'd expect
We just published something from our AISA data that surprised me.
We looked at 2,000+ AI skills assessments broken down by seniority level. Executives and senior leaders averaged 51.4 out of 100 on practical AI fluency. Individual contributors? 46.3.
The gap isn't huge, but the shape of it is interesting. Leaders consistently outperformed on two dimensions: Workflow & Application (how deeply AI is woven into daily work) and Critical Thinking (evaluating AI output, understanding limitations). They scored lower on Technical Understanding and Prompting.
My read: leaders who've adopted AI have done it strategically — they've integrated it into their decision-making process, not just their to-do list. But they're often working with a smaller technical vocabulary and less prompt craft. They don't know why something works, but they've figured out when to trust it.
For us as a product, this is reshaping how we think about the assessment itself. We initially designed AISA assuming technical depth would correlate with seniority. It doesn't. A VP of Marketing who uses Claude Projects as a thinking partner for every strategic decision scores higher on the dimensions that actually matter than a developer who only uses Copilot for autocomplete.
Full breakdown is on our blog: https://aisa.to/blog/ai-skills-for-executives
Building in public because this kind of finding changes your roadmap. We're now adding role-specific benchmarks to the reports so companies can see how their team stacks up against peers in similar roles, not just against the overall population.
Curious if anyone else is seeing this pattern — leaders who are more AI-fluent in practice but less technical about it?
Fascinating dataset, Ozan! As someone based in Japan, I’d love to add a bit of context from a Japanese organizational perspective:
In Japan, due to an aging management layer, overall AI adoption among executives is generally lower. On the flip side, digital-native junior employees are eager to use AI, though they sometimes face friction from traditional managers who are hesitant about these tools.
However, when looking specifically at capable and high-performing leaders, your findings ring totally true. Regardless of their technical knowledge, effective leaders seamlessly integrate AI into their strategic workflows.
It seems the ability to leverage AI as a thinking partner comes down to business literacy and critical thinking, not just age or technical jargon!
Nice progress. What is the next thing you are focusing on?
The Technical Understanding result might be measuring something other than knowledge.
An IC's AI output gets checked. It goes into a PR, a test suite, a spreadsheet someone reconciles. When it's wrong they find out within the hour and their name is on it. An executive's AI output mostly goes into a doc, a memo, a talking point, where wrongness surfaces in a quarter or never. So ICs get a tight feedback loop on failure modes, and leaders get a tight feedback loop on usefulness.
That would produce exactly the shape you found, but it flips the reading of the Critical Thinking score. A high critical-thinking score among leaders could just as easily reflect confident evaluation of output that nobody has checked yet.
One test would separate the two: score critical thinking against a set of outputs where some are subtly wrong and you know which ones. Describing a good evaluation process and actually detecting a plausible-looking error are very different measurements, and the gap between them is usually the whole story.
On the technical-vocabulary point, though: the piece of technical understanding I'd most want a leader to have isn't prompt craft. It's knowing which limits belong to the model and which belong to the vendor. Two that come up constantly:
Context window. The number on a model card is a property of the weights. The window you actually get is whatever your provider configured, and most hosted endpoints serve well under the advertised figure. It fails silently, no error, so the symptom people report is "the AI forgot what I told it," which sounds like a model limitation and isn't one.
Refusals. When a tool says it can't do something, that is almost always policy rather than capability. The model knows the answer. A leader who can't tell those apart writes off entire categories of work as "AI can't do that yet," which is a strategic error that quietly costs a lot and is the one I'd fix first.
Neither of those needs prompt craft, and both are about fifteen minutes to teach. I'd be curious whether your assessment distinguishes them at all, because in the data I'd expect them to sit under Technical Understanding while behaving much more like Critical Thinking.
Disclosure: I build Grunz, an AI chat and coding agent on open weights, so the second point is a professional interest as much as an observation.