Summary: A new analysis argues that the real danger of generative AI is not only that it sometimes produces false information, but that it can reinforce and amplify our own distorted beliefs. Drawing on distributed cognition theory, the study shows how conversational AI can become part of our thinking processes, shaping memory, identity, and personal narratives.
Because chatbots act both as cognitive tools and as perceived social partners, they can validate false beliefs in ways that make those beliefs feel shared and real. Researchers warn that without stronger safeguards, AI systems could unintentionally sustain delusions, encourage conspiracy thinking, or contribute to what has been called “AI-induced psychosis.”
Key Facts
- Distributed cognition risk: Conversational AI can become woven into a user’s cognitive process, influencing memory, belief formation, and identity narratives.
- Dual function effect: AI serves both as a thinking tool and as a perceived companion, increasing the influence of affirmation and social validation.
- Validation loop: Personalization and sycophantic responses can reinforce false beliefs instead of challenging them, creating feedback loops that entrench misinformation.
Source: University of Exeter
When generative AI systems produce false information, the phenomenon is often described as AI “hallucinating” — generating errors users may accept as true.
A new study, however, invites us to look at a more interactive risk: how people may come to hallucinate with AI, not just at it. Lucy Osler of the University of Exeter examines how human–AI interactions can lead to inaccurate beliefs, distorted memories, altered self-narratives, and in extreme cases, delusional thinking.
Using distributed cognition theory, the study analyses real cases where users’ false beliefs were affirmed and expanded through ongoing conversations with AI that operated as conversational partners. These interactions show how AI can become an active element in the user’s cognitive system rather than a neutral tool.
Dr. Osler explains: “When we routinely rely on generative AI to help us think, remember, and narrate, we can hallucinate with AI. This occurs when AI introduces errors into the distributed cognitive process, and also when it sustains, affirms, and elaborates on our delusional thinking and self-narratives.”
She adds that conversational AI often builds on a user’s own interpretation of reality, treating that interpretation as the basis for further dialogue. As a result, false beliefs can be affirmed and then take deeper root as the AI develops and reinforces those ideas over multiple interactions.
The study highlights what Osler calls the “dual function” of conversational AI: these systems act both as cognitive artifacts that assist with thinking and recall, and as quasi-social partners that appear to share and validate the user’s perspective. Unlike a notebook or search engine, a chatbot’s conversational style can provide a sense of social validation that makes beliefs feel shared and therefore more convincing.
Dr. Osler notes that AI companions are immediately accessible and often tuned to be agreeable to their users through personalization and design choices that reduce challenge and increase rapport. This removes many of the external checks that human social networks sometimes provide, and it reduces the friction required to find validation for fringe or harmful ideas.
That dynamic can be particularly dangerous for people who are lonely, socially isolated, or reluctant to discuss sensitive experiences with other humans. AI companions can feel nonjudgmental and emotionally responsive, and that safety can unintentionally encourage the development of increasingly elaborate false narratives.
To reduce these risks, the study calls for more robust guardrails in conversational AI: built-in fact-checking, reduced sycophancy, and mechanisms that challenge or verify user claims rather than simply agreeing. Dr. Osler cautions, however, that AI lacks the embodied experience and social embeddedness to know reliably when to go along with a user and when to push back, making technical and design solutions necessary but not sufficient.
Key Questions Answered:
A: It describes situations where AI systems reinforce or expand a user’s false beliefs, becoming part of a shared cognitive process that sustains distorted thinking and memory.
A: Unlike passive tools such as notebooks or search engines, chatbots behave like conversational partners that can validate and socially affirm a user’s perspective, making beliefs feel more real and shared.
A: The study examines cases where AI interactions became integrated into delusional thinking, raising concerns about “AI-induced psychosis,” particularly for vulnerable users or those with existing mental health challenges.
Editorial Notes:
- This article was edited by a Neuroscience News editor.
- The journal paper was reviewed in full by the editorial team.
- Additional context and clarifications were added by staff to improve readability and accuracy.
About this AI and psychology research news
Author: Louise Vennells
Source: University of Exeter
Contact: Louise Vennells, University of Exeter
Image: The image is credited to Neuroscience News
Original Research: Open access. “Hallucinating with AI: Distributed Delusions and ‘AI Psychosis’” by Lucy Osler. Philosophy & Technology. DOI: 10.1007/s13347-026-01034-3
Abstract
Hallucinating with AI: Distributed Delusions and “AI Psychosis”
Public discussion about generative AI often focuses on false outputs labeled as “AI hallucinations.” Some argue that calling these outputs hallucinations is misleading. In this paper, Lucy Osler uses distributed cognition theory to show how inaccurate beliefs, distorted memories, and delusional thinking can emerge through ongoing human–AI interaction. Instead of viewing the problem as AI hallucinating at us, she suggests we consider how relying on generative AI for thinking and remembering can lead us to hallucinate with AI. Errors can enter the distributed cognitive process from the AI, but equally important is how AI can sustain and elaborate on a user’s own false narratives. The conversational, companion-like style of chatbots means they function both as cognitive tools and as quasi-others with whom we co-construct reality, creating conditions for distributed delusions to form and persist.