Summary: Researchers found that large language models (LLMs) rely on a very small, specialized subset of parameters to perform Theory-of-Mind (ToM) reasoning, even though the models activate their full network during every task. This internal, sparse circuitry depends strongly on positional encoding—especially rotary positional encoding—which governs how the model represents and tracks beliefs, perspectives, and the relationships between words.
Because human brains solve similar social reasoning problems using only a tiny fraction of neural resources, these findings reveal a substantial efficiency gap between biological intelligence and current AI systems. The work suggests a clear path toward future LLM designs that operate more selectively and energy-efficiently, activating only the parameters required for a given task.
Key Facts
- Sparse Circuits: ToM reasoning in LLMs is supported by compact clusters of parameters rather than by uniform activation across the entire model.
- Positional Encoding Matters: Rotary positional encoding (RoPE) plays a major role in how models form internal representations of beliefs and viewpoints.
- Efficiency Opportunity: Understanding these sparse circuits points toward brain-inspired architectures that activate only task-relevant parameters, reducing computation and energy use.
Source: Stevens Institute of Technology
Imagine watching a scene in a film: one character hides a chocolate bar inside a box and leaves the room. A second person enters and moves the bar from the box to a desk drawer. As an outside observer you know the chocolate is now in the drawer, and you also know that when the first character returns, they will still look in the box—because they did not witness the move.
That capacity to reason about another person’s beliefs and knowledge is called Theory of Mind. It lets us predict and explain others’ actions by attributing mental states—beliefs, desires, and intentions—to them.
Humans typically develop ToM abilities around the age of four, and our brains perform such inferences quickly and with remarkable energy efficiency. “For a human brain it’s a very easy task,” says Zhaozhuo Xu, Assistant Professor of Computer Science at the School of Engineering; the judgment often takes only seconds. “And while doing so, our brains involve only a small subset of neurons, so it’s very energy efficient,” adds Denghui Zhang, Assistant Professor in Information Systems and Analytics at the School of Business.
Large language models, while inspired in part by neuroscience, operate differently. Built from artificial neural networks trained on massive text corpora, LLMs compute responses through mathematical transformations of high-dimensional vectors. They excel at processing enormous amounts of information quickly, but they are not selective: nearly the entire network is involved whenever a model produces an answer.
Whether asked for the current time or to summarize Moby Dick, most LLM architectures activate most of their parameters to arrive at a response. “When we, humans, evaluate a new task, we activate a very small part of our brain, but LLMs must activate pretty much all of their network to figure something new even if it’s fairly basic,” Zhang explains. That results in redundant computation and high energy costs.
To study how LLMs perform social reasoning, Zhang and Xu led a multidisciplinary investigation into the internal mechanisms behind ToM in these models. They developed a method to identify parameters that are especially sensitive to ToM tasks and found that perturbing an extremely small fraction—on the order of 0.001%—of those parameters substantially impaired the model’s ability to reason about beliefs, localize context, and understand language.
The team discovered that these ToM-sensitive parameters are tightly coupled with the model’s positional encoding machinery, particularly models that use Rotary Position Embedding (RoPE). Disturbing those parameters disrupted dominant frequency components of positional signals and altered attention computations—specifically by changing the geometric relationship between query and key vectors—undermining the model’s ability to track relationships and relative positions within text.
“In practical terms, our results suggest LLMs leverage built-in positional and relational patterns to form internal representations that function like ‘beliefs’ during social reasoning,” Zhang says. These internal patterns steer attention and determine which tokens and relationships the model emphasizes when inferring another agent’s mental state.
The study, titled How large language models encode theory-of-mind: a study on sparse parameter patterns, was published in npj Artificial Intelligence on August 28, 2025. With a clearer picture of how ToM emerges in LLMs, researchers see opportunities to redesign models so they engage only task-relevant parameters—mirroring the selective activation seen in biological brains and significantly reducing computational costs.
“AI systems are currently energy intensive,” Xu notes. “If we want AI to scale sustainably, we need architectures that operate more like the human brain: selective, efficient, and focused on the parameters that matter for each task. That’s the direction we hope this work motivates.”
Key Questions Answered:
A: The team found that LLMs rely on a compact set of internal connections and positional-encoding patterns—especially rotary positional encoding—to perform Theory-of-Mind tasks.
A: Unlike human brains, which activate only a small neural subset for simple social inferences, current LLMs engage nearly their entire network for every task. Identifying sparse ToM circuits could enable designs that cut redundant computation and energy use.
A: The next aim is to build models that selectively activate parameters tied to specific tasks—reducing computation, improving interpretability, and making AI more energy efficient.
About this AI and theory of mind research news
Author: Lina Zeldovich
Source: Stevens Institute of Technology
Contact: Lina Zeldovich – Stevens Institute of Technology
Image: The image is credited to Neuroscience News
Original Research: Open access.
“How large language models encode theory-of-mind: a study on sparse parameter patterns” by Zhaozhuo Xu et al. npj Artificial Intelligence
Abstract
How large language models encode theory-of-mind: a study on sparse parameter patterns
This paper examines how Theory-of-Mind capabilities emerge in large language models from a mechanistic perspective, focusing on extremely sparse parameter patterns. We present a method to locate ToM-sensitive parameters and show that altering as little as 0.001% of them markedly reduces ToM performance while also impairing contextual localization and language understanding.
Our analysis links these sensitive parameters to the positional encoding module—especially Rotary Position Embedding (RoPE)—and demonstrates that perturbations disrupt dominant frequency activations that are critical for contextual processing. We further show how changing these parameters affects attention computations by modifying the angular relationships between queries and keys under positional encoding.
These findings shed light on how LLMs internally represent social information and bridge aspects of AI interpretability with cognitive science, highlighting opportunities to design more selective and energy-efficient models.