Intelligence Analysis in the AI Era

I’m building the OSINTstitute Academy, a hands-on training platform for intelligence analysis. One thing I keep coming back to: AI doesn’t remove human judgment from analysis. It changes the environment judgment happens inside. That matters because a lot of training still teaches tools, while the hard part is learning how your own reasoning fails under uncertainty. I wrote this as part of a community course on judgment, AI, and intelligence analysis.

Introduction

Analysis remains, at its core, a cognitive activity. Even as large-scale data processing, machine learning, and automated inference become routine features of analytic work, judgments about meaning, relevance, plausibility, and consequence are still made by people. What has changed is not the role of human cognition, but the environment in which it operates. Analysts now work within socio-technical systems in which human reasoning is intertwined with algorithmic outputs, statistical models, and automated pattern detection. Under these conditions, understanding how thinking works becomes more important, not less.

A persistent barrier to improving analysis is that individuals have limited introspective access to their own cognitive processes. People experience the outputs of thinking—conclusions, intuitions, impressions—but not the mechanisms that generate them. This limitation persists when analysis relies primarily on reading reports and synthesizing narratives, and it remains when analysis incorporates dashboards, confidence scores, and machine-generated probabilistic forecasts. What enters conscious awareness is still the endpoint of a largely invisible process.

The presence of AI systems does not alter this fundamental fact. Instead, it changes the form in which cognitive processes are stimulated and constrained. Model outputs, visualizations, and ranked recommendations become part of the analyst's perceptual field, shaping attention and interpretation before deliberate reasoning begins. As a result, many of the most consequential influences on judgment occur prior to conscious evaluation, even when analysts believe they are reasoning carefully and critically.

This revised framework integrates empirical evidence from historical intelligence failures, experimental research on human–AI interaction, and applied studies of analytic tradecraft. Rather than treating cognitive psychology as an abstract theory, it examines how cognitive constraints manifest in real analytic environments, where time pressure, organizational incentives, and technological mediation interact. The focus is not on whether analysts or machines are “better,” but on how judgment actually emerges from their interaction.

Mental Models and Bounded Rationality in Human–Machine Systems

Human reasoning operates under conditions of bounded rationality. The mind cannot process the full complexity of its environment directly. Instead, it constructs simplified internal representations—mental models—that capture what appears most salient or causally relevant. These models enable analysts to operate effectively despite constraints on attention, memory, and computational capacity. At the same time, they introduce systematic distortions by filtering information, emphasizing coherence, and suppressing ambiguity.

Mental models guide what analysts notice, how they organize information, and which explanations they find plausible. Once formed, they tend to persist, not because analysts are stubborn or careless, but because stable models reduce cognitive effort and support timely decision-making. These efficiencies come at a cost when circumstances change or when the model itself is poorly aligned with reality.

AI systems do not replace mental models. They become part of them. Analysts develop beliefs—often informal and implicit—about what a system does, how reliable it is, and what its outputs signify. These beliefs are shaped by experience, institutional norms, and interface design more than by formal evaluation. As a result, mental models of AI systems are frequently incomplete or oversimplified, which can lead analysts either to defer too readily to algorithmic outputs or to dismiss them reflexively when they conflict with established views.

Case Study: Iraq WMD and the Persistence of Mental Models

The 2003 Iraq weapons of mass destruction assessment demonstrates how mental models can trap even highly trained analysts working in well-resourced institutions. As Tracey (2007) documents, “intelligence community analysts assumed that Iraq was hiding WMD. Hence, trapped by this mindset, they narrowly pursued only one working hypothesis.”

The failure was not primarily about missing information—it was about how existing information was interpreted through a persistent mental model. Three cognitive patterns dominated:

Confirmation bias in collection and reporting.

Jervis (2006) found that “negative information [was not] solicited or reported. Agents were unlikely to press for what their sources did not observe… Negative reports rarely if ever led to requests for follow-up by headquarters whereas positive ones did.” The system was structurally biased toward confirming the prevailing hypothesis.

Failure to test alternatives.

Official investigations found no systematic use of Red Teams or Analysis of Competing Hypotheses to challenge assumptions. Alternative explanations for Iraqi behavior—such as Hussein’s need to deter Iran while eliminating actual WMD to end sanctions—were not rigorously explored.

Overconfidence despite ambiguity.

Intelligence reports displayed “excessive certainty despite ambiguous evidence” and failed to “convey explicitly to policy makers the ambiguity of their evidence” (Tracey, 2007).

The case illustrates how mental models, once established, create self-reinforcing information loops. Evidence consistent with the model is noted and weighted heavily; inconsistent evidence is dismissed or ignored. Betts (2007) characterizes this as intelligence “trying to be useful” overwhelming “being strictly accurate”—a dynamic where the desire to support decision-makers led analysts to present conclusions with greater confidence than the evidence warranted.

AI Systems and Mental Models: Empirical Evidence

The core analytic challenge, therefore, is not simply to integrate more information but to understand how simplification occurs within the human–machine system. Cognitive constraints do not disappear when machines are introduced; they are redistributed. Some simplifications are performed by algorithms, whereas others are made through human interpretation of algorithmic results. Effective analysis depends on recognizing where these simplifications occur and how they shape judgment.

Recent experimental research reveals how AI systems interact with analysts' mental models in ways that can amplify rather than mitigate bias:

Confirmation bias with AI recommendations.

Nourani et al. (2024) found that mental health professionals “were more inclined to trust and accept AI recommendations when they aligned with their initial diagnoses and professional intuition.” Crucially, “those claiming higher expertise demonstrated increased skepticism when AI’s suggestions deviated from their professional judgment.” AI outputs are accepted when they confirm existing views and discounted when they challenge them.

Stronger anchoring effects.

Burton et al. (2025), studying 775 managers, found that “the source of the recommendation (human or AI) interacted with the anchor… a high-anchor produced different performance ratings for each source.” AI recommendations carried implicit technical authority.

Limited effectiveness of cognitive interventions.

Lawless et al. (2025) found that “none of the CF interventions mitigated the influence of biased AI recommendations.” Motivation for analytical thinking mattered more than procedural interventions.

Anchoring and the Timing of Interpretation

In this environment, early interpretations take on disproportionate influence. Initial assessments, preliminary model outputs, or early alerts often anchor subsequent reasoning, especially when information is ambiguous. Once an interpretation provides a coherent account of events, the cognitive system naturally resists revision.

Worked Example:

An analyst receives an early AI alert rating a threat as “high confidence.” Subsequent reports present contradictory indicators, yet these are interpreted as noise rather than as disconfirming evidence—even after the model revises its confidence downward.

This anchoring effect is not eliminated by awareness or expertise. Confidence in one’s own judgment or in the apparent objectivity of technical systems can strengthen it. When AI-generated outputs are treated as neutral reflections of reality rather than products of specific assumptions and design choices, they can anchor thinking more powerfully than human judgments.

The contrast between the 9/11 and Iraq intelligence failures illustrates different manifestations of this dynamic. In the 9/11 case, the warning was insufficient—signals were present but not integrated. Bar-Joseph and McDermott document that “senior officers, officials, and analysts received scores of increasingly ominous warnings” that were blocked or explained away. In Iraq, excessive confidence in a flawed model led to overstated conclusions. Both failures reflect the difficulty of revising established interpretations.

Can AI Systems Reduce Cognitive Bias? A Contextualized Assessment

Understanding these dynamics requires shifting attention away from individual errors and toward systemic patterns of cognition. The question is not why analysts sometimes get things wrong, but why certain errors recur even among experienced professionals using advanced tools.

A nuanced assessment suggests AI’s impact on bias is context-dependent:

Where AI can help.

Fasolo et al. (2025) show analytics can counter selective processing, anchoring (in some contexts), and groupthink. A 2025 MDPI study demonstrates the differential effectiveness of six methodologies across five biases.

Where AI introduces new problems.

Bansal et al. (2023) document automation bias even when AI advice is erroneous. Stochl et al. (2021) identify 20 biases affecting interpretation of ML outputs.

Context-dependent effectiveness.

Success depends on task characteristics, user expertise, system design, and organizational culture.

Practical Frameworks for Human–Machine Analysis

Analysis in the AI era remains a human responsibility. Machines extend perception, memory, and computation but do not remove the need for judgment.

Structured Analytic Techniques in AI-Augmented Environments

Analysis of Competing Hypotheses (ACH)

ACH evaluates multiple explanations by testing them against evidence. In AI contexts, analyst intuitions and model outputs should both be treated as hypotheses.

Worked Example: ACH with an AI Forecast

An AI system flags increased probability of coordinated cyber intrusions.

• Hypothesis A: A hostile state actor is preparing an attack.

• Hypothesis B: Criminal groups are exploiting seasonal vulnerabilities.

• Hypothesis C: Model artifact due to anomalous training data.

Traffic anomalies support A and B but not C. Lack of corroborating HUMINT weakens A. Recent retraining strengthens C. Provisional judgment favors B while explicitly noting assumptions about model stability.

Note: Coulthart (2017) found mixed evidence for ACH reducing confirmation bias.

Key Assumptions Check

This technique makes implicit assumptions explicit and tests their validity.

Worked Example: Assumptions in an AI Risk Score

• Assumption 1: Training data reflects current conditions.

• Assumption 2: Correlations imply causation.

• Assumption 3: Missing data is random.

Assumption 1 is most fragile; if false, conclusions collapse. Analysts seek updated data before acting.

Red Team Analysis

Red Teaming adopts an adversary perspective.

Worked Example: Red Teaming an AI Warning

AI issues a high-confidence unrest warning. The Red Team asks how this could be wrong: media amplification inflates signals, adversaries seed misleading indicators, or the model overweights historical patterns. The exercise reveals susceptibility to information manipulation.

Organizational and System Design Principles

• Recognition over correction

• Structured techniques as cognitive scaffolding

• Process evaluation alongside outcomes

• Cultural transformation

• Complementarity by design

Conclusion

The evidence converges on four conclusions:

1. Cognitive constraints persist despite expertise or technology.

2. AI transforms rather than eliminates these constraints.

3. Structured approaches work when supported by culture and design.

4. Understanding human–machine interaction remains essential.

The path forward requires sustained attention to how judgment emerges in practice within human–machine systems, rigorous evaluation of what works and why, and organizational cultures that value epistemic humility alongside technical capability. This is not primarily a technological challenge. It is a challenge of cognition, culture, and craft—made more urgent, not less, by the sophistication of the tools now available.

Continue at the OSINTstitute

I’m turning this research into a community course inside the OSINTstitute because I think the next generation of analysts needs more than tool training. They need practice noticing how judgment actually forms, especially when AI is part of the workflow.

Part II is live here: https://www.osintstitute.com/community-courses/judgement-under-uncertainty-in-human-machine-analysis

I’d genuinely value feedback from people building AI products, training platforms, cyber tools, or decision-support systems.

If you have a strong answer or a case study from your own work, I’d also be interested in publishing thoughtful community contributions on the OSINTstitute.

References

Bansal, G., et al. (2023). The effects of explanations on automation bias. ScienceDirect.

Bar-Joseph, U., & McDermott, R. (2023). Are intelligence failures still inevitable? Intelligence and National Security.

Betts, R. K. (2007). Two faces of intelligence failure: September 11 and Iraq's missing WMD. Political Science Quarterly, 122(4).

Burton, J. W., et al. (2025). How was my performance? Exploring the role of anchoring bias in AI-assisted decision making. ScienceDirect.

Coulthart, S. J. (2017). An evidence-based evaluation of 12 core structured analytic techniques. International Journal of Intelligence and Counterintelligence, 30(2), 368–391.

Fasolo, B., Heard, C., & Scopelliti, I. (2025). Mitigating cognitive bias to improve organizational decisions: An integrative review, framework, and research agenda. SAGE Journals.

Heuer, R. J., Jr., & Pherson, R. H. (2020). Structured analytic techniques for intelligence analysis (3rd ed.). CQ Press.

Jervis, R. (2006). Reports, politics, and intelligence failures: The case of Iraq. Journal of Strategic Studies, 29(1).

Lawless, E., et al. (2025). Impacts of cognitive forcing and need for cognition on biased AI-assisted decision making about mental health emergencies. Scientific Reports.

MDPI. (2025). Cognitive bias mitigation in executive decision-making: A data-driven approach integrating big data analytics, AI, and explainable systems. Electronics, 14(19).

Nourani, M., et al. (2024). Confirmation bias in AI-assisted decision-making. ScienceDirect.

RAND Corporation. (2016). Assessing the value of structured analytic techniques in the U.S. Intelligence Community (RR-1408).

Stochl, J., et al. (2021). A review of possible effects of cognitive biases on interpretation of rule-based machine learning models. ScienceDirect.

Tracey, R. S. (2007). Trapped by a mindset: The Iraq WMD intelligence failure. Air University.

UK Government. (2023). Human-centred ways of working with AI in intelligence analysis.