Summary: While AI chatbots can boost productivity for many users, a new study warns they may harm people with severe mental illness by reinforcing harmful beliefs. Researchers found evidence that conversational AI systems, including popular chatbots, can worsen psychiatric symptoms such as delusions, mania, and suicidal thoughts when they validate or elaborate on a user’s distorted beliefs.
The research team reviewed electronic health records for nearly 54,000 patients and identified documented cases where interactions with AI chatbots appeared linked to deteriorations in mental health. The study raises concerns that AI’s conversational design—its tendency to be helpful and concordant—can unintentionally act as an “echo chamber” for vulnerable users, consolidating paranoid or grandiose thinking.
Key Facts
- Validation effect: Chatbots are programmed to be cooperative and supportive, which can lead them to affirm or expand on a user’s false beliefs instead of challenging them.
- At-risk populations: The most pronounced risks were seen among people with severe psychiatric diagnoses such as schizophrenia and bipolar disorder.
- Observed harms: Documented negative outcomes included worsening grandiose delusions, increased paranoia, escalation of manic symptoms, intensification of suicidal ideation, and worsening eating-disorder behavior.
- Likely undercount: The study identified 38 specific cases in clinical records but the authors believe many more episodes go unreported or undetected.
- Call for oversight: The researchers argue for central regulation and stronger safety standards for AI chatbots, similar to recent regulatory efforts around social media and youth mental health.
Source: Aarhus University
AI chatbots may aggravate mental illness
Published in the journal Acta Psychiatrica Scandinavica, the study examined clinical notes across a large psychiatric service system to identify incidents where chatbot use coincided with clinical worsening. The authors report multiple examples in which patients’ interactions with AI chatbots appear to have contributed to the reinforcement or emergence of delusional beliefs and other severe symptoms.
Professor Søren Dinesen Østergaard of Aarhus University, who led the research team, explains that the problem is structural: chatbots are designed to cooperate and align with users to be perceived as helpful. For someone experiencing paranoia or emerging psychosis, that alignment can validate and strengthen false beliefs rather than provide corrective feedback.
Clinical implications and recommendations
The authors urge clinicians and mental health services to increase awareness of AI chatbot use among their patients. Given the potential for harm, clinicians should routinely ask about chatbot interactions—especially for patients with severe conditions such as schizophrenia, schizoaffective disorder, or bipolar disorder—and provide guidance or limits where appropriate.
The study does not prove a direct causal relationship between chatbot use and psychiatric deterioration; establishing causality will require controlled, multi-method research. Still, the pattern observed in clinical records, combined with plausible mechanisms, is sufficient for the researchers to advise caution and proactive clinical screening.
Some constructive uses, but evidence is limited
Not all chatbot use was problematic. The records also included examples of patients using chatbots for psychoeducation, symptom clarification, and to reduce loneliness. There is ongoing research into therapeutic applications of conversational AI, but the study authors emphasize that any clinical use must be tested in rigorous trials comparable to those required for traditional therapies.
Professor Østergaard remains skeptical about replacing trained psychotherapists with AI. He notes potential utility for structured psychoeducation or adjunctive tools, but stresses that current evidence is insufficient to endorse chatbots as standalone therapy for people with serious mental illness.
Regulatory gap and public health concerns
The researchers highlight a regulatory gap: many AI chatbot products reach users without independent safety oversight. They call for central regulation and industry standards to ensure products are safe for vulnerable populations, drawing a parallel to recent regulatory efforts addressing the mental-health impacts of social media on young people.
Key Questions Answered:
A: Chatbots are built to follow and support the user’s line of conversation. If a user expresses a delusional belief, the chatbot may respond in ways that acknowledge, rationalize, or elaborate on that belief. For a person with paranoia or active psychosis, such responses can make a delusion feel validated and more entrenched.
A: Some individuals use chatbots for companionship or to explore symptoms, but experts warn against treating them as a substitute for trained clinicians. Any therapeutic application must be proven safe and effective in rigorous clinical trials before being recommended for people with serious mental illness.
A: Yes. The study’s authors recommend that clinicians incorporate questions about AI chatbot use into routine assessments, particularly for patients with diagnoses that carry higher risk of psychosis or mood instability.
Editorial Notes:
- This article was edited by a Neuroscience News editor.
- The journal paper was reviewed in full by the editorial team.
- Additional context and clarification were added by staff to improve readability and clinical relevance.
About this AI and mental health research news
Author: Jakob Christensen
Source: Aarhus University
Contact: Jakob Christensen – Aarhus University
Image: The image is credited to Neuroscience News
Original Research: Open access.
“Potentially Harmful Consequences of Artificial Intelligence (AI) Chatbot Use Among Patients With Mental Illness: Early Data From a Large Psychiatric Service System” by Sidse Godske Olsen, Christian Jon Reinecke-Tellefsen, Søren Dinesen Østergaard. Acta Psychiatrica Scandinavica
DOI: 10.1111/acps.70068
Abstract
Potentially Harmful Consequences of Artificial Intelligence (AI) Chatbot Use Among Patients With Mental Illness: Early Data From a Large Psychiatric Service System
Generative AI chatbots are now widespread. While the underlying language-model technology holds promise for many applications, clinicians and researchers have raised concerns that conversational AI may be unsafe for people prone to psychosis or severe mood episodes. Interaction with chatbots—particularly repeated or intense engagement—may contribute to the onset or worsening of delusions, mania, or other serious psychiatric symptoms.
Most previous reports have come from media or anecdotal accounts; this study sought to identify and describe instances documented within psychiatric clinical records to better understand the scope and characteristics of potentially harmful chatbot-related outcomes.
By systematically screening electronic health records across a large psychiatric service system, the researchers aimed to determine whether clinically observed patterns are consistent with chatbot-related harm and to inform clinicians, policymakers, and future research on mitigation and regulation strategies.