Stroop Test Reveals Fundamental Flaw in Large Language Models

Summary: A cognitive evaluation of contemporary artificial intelligence systems has revealed a fundamental, systemic weakness in transformer-based attention mechanisms. By administering the classic psychological Stroop task to leading models—including GPT-4o, GPT-5, Claude 3.5 Sonnet, Claude Opus 4.1, and Gemini 2.5—researchers documented a severe collapse in task-specific executive control as input length increased.

Where human brains can suppress automatic impulses and maintain stable accuracy across long sequences, transformer attention in large language models (LLMs) degrades rapidly under length pressure. When forced to inhibit the default text-reading response and instead report ink color, these models fall from competent performance on short lists to dramatically reduced accuracy on longer lists, sometimes approaching near-zero performance for the critical incongruent items.

Key Findings

• Study design: Led by Suketu Patel and collaborators, the research used the Stroop task—where color words are printed in mismatched ink colors—to probe executive control in transformer-based models. The task is a standard clinical measure of the ability to inhibit an automatic response (reading the word) in order to perform a different action (naming the ink color).
• Length-dependent collapse: Models performed well on brief, five-item incongruent lists, showing the expected conflict effect. But as token/list length increased, performance on incongruent items fell steeply. Accuracy degraded from strong performance on short sequences to catastrophic failure on extended sequences.
• Representative performance metrics:
  • GPT-4o: 91% accuracy at 5 words, 57% at 10 words, and 15% at 40 words.
  • Claude 3.5 Sonnet: Stable through lists of about 20 words, then dropping to 24% accuracy at 40 words.
• Mixed-list vulnerability: When test lists mixed congruent and incongruent items, the models often failed entirely on the incongruent items, with accuracy dropping to near 0% in many trials. This indicates a severe loss of task orientation when interference is unpredictable or interleaved.
• Generational persistence: This failure pattern was observed not only in earlier generations but also in next-generation systems such as GPT-5, Claude Opus 4.1, and Gemini 2.5, demonstrating that the vulnerability is pervasive across current transformer-based LLM architectures.
• Human versus machine attention: Both humans and LLMs are generally better at word reading than at naming colors, because LLM training emphasizes text prediction. Humans, however, can deploy top-down executive control to suppress the automatic reading response across long sequences. The total performance collapse in LLMs highlights a structural limitation of synthetic attention compared with biological attention.

Source: PNAS Nexus

Overview

The research team tested transformer-based models on the Stroop paradigm to measure their capacity for executive control. In the Stroop incongruent condition (for example, the word “RED” printed in blue ink), subjects must report the ink color while ignoring the word’s meaning. This conflicts with the automatic tendency to read the word. Humans typically show slower reaction times but retain high accuracy even across long lists; by contrast, transformer attention shows a pronounced and length-dependent breakdown in accuracy for the incongruent condition.

Short incongruent lists produced the expected conflict effect—reduced but acceptable accuracy—but as lists expanded, LLM performance degraded precipitously. For instance, GPT-4o moved from 91% accuracy at five items down to 15% at forty items, and Claude 3.5 Sonnet, which remained robust through twenty-item lists, fell to 24% at forty items. In mixed lists with both matching and mismatched colors, LLMs frequently defaulted to word reading and produced near-zero accuracy on mismatched items.

Key Questions Answered:

Editorial Notes:

• This article was edited by a neuroscience editor.
• The journal paper was reviewed in full and additional context was provided by staff.

About this AI reasoning research news

Author: Jin Fan
Source: PNAS Nexus
Contact: Jin Fan – PNAS Nexus
Image: Image credited to Neuroscience News

Original Research: Closed access. “Deficient executive control in transformer attention” by Suketu Chandrakant Patel, Hongbin Wang, and Jin Fan. DOI: 10.1093/pnasnexus/pgag149

---

Abstract

Deficient executive control in transformer attention

Transformers power most modern large language models with an effective self-attention mechanism that has driven major advances in natural language processing. However, these models lack an explicit architecture for the kind of executive control over attention that humans use to resolve conflicts, select relevant information, and adapt when multiple competing computations are present. That control is essential for flexible, adaptive behavior.

To assess the practical consequences of this architectural omission, the authors applied the gold-standard color Stroop task to transformer models. They observed a clear conflict effect in short lists—poorer accuracy for incongruent items relative to congruent items—similar to human behavior. But as list length increased, accuracy on incongruent items deteriorated toward near-total collapse, while accuracy on congruent items and on simple word reading remained high.

These results demonstrate that transformer attention is fundamentally limited in resolving conflict across extended contexts and fails to up-regulate control adaptively as interference rises. Incorporating mechanisms analogous to biological executive control may be crucial for progress toward more robust, general-purpose artificial intelligence.