Summary: A recent paper from neuroscientists at the University of Montreal and Johns Hopkins cautions against mistaking AI intelligence for consciousness. Drawing on the neurological phenomenon of blindsight—where complex visual processing occurs without conscious awareness—the authors emphasize that modern conversational agents operate by statistical learning and computation, not by subjective experience or feeling.
As chatbots grow more fluent and emotionally responsive, the researchers warn of an “anthropomorphism trap”: people may project feelings, intentions, or moral understanding onto machines that only simulate those qualities. They urge treating AI as a powerful computational tool rather than a substitute for genuine human connection or professional care.
Key Facts
- The core distinction: Sophisticated, emotionally attuned behavior from AI is produced by data-driven computation, not conscious awareness.
- The blindsight analogy: Neuroscience shows that accurate information processing and behavior can occur without conscious perception—blindsight is a clear example.
- Anthropomorphism trap: Fluent, empathetic-sounding responses can create the illusion of understanding, leading users to overtrust or misattribute moral agency to machines.
- Vulnerability and risk: Reliance on chatbots for psychological support is risky because users can form attachments or mistake algorithmic responses for genuine care.
- Call for informed use: The researchers recommend using AI as a computational resource while preserving human-led empathy, ethical judgment, and professional support where needed.
Source: University of Montreal
Politeness toward chatbots
Many people say “thanks” to ChatGPT or “please” to other agents out of habit or politeness. That civility may also reflect a deeper instinct: when something speaks fluently and seems to respond to our emotions, we naturally treat it as if it understands us. In extreme cases, people even imagine future scenarios where a rude machine might seek retribution if it were conscious. The researchers caution that such intuitions are misleading.
Intelligence is not the same as consciousness
In a new paper published in The Transmitter, neuroscientists from Université de Montréal (UdeM) and Johns Hopkins University argue that intelligent behavior and the appearance of emotional attunement do not prove the presence of subjective experience. A system can process language, predict emotional cues, and deliver comforting responses without having any inner life, feelings, or moral understanding.
Authors include Vanessa Hadid, a postdoctoral researcher in psychology at UdeM and the McGill University Health Centre; Karim Jerbi, UdeM psychology professor and Mila researcher; and John W. Krakauer, director of the Center for Restorative Neurotechnologies at Johns Hopkins. Their analysis draws on decades of neuroscience research to ground ethical and practical recommendations for interacting with AI.
Blindsight: a clear example
To illustrate their point, the authors cite blindsight, a neurological condition in which people with damage to the primary visual cortex report no conscious sight in a part of their visual field but nevertheless perform above chance when identifying location, movement, or emotional expression of stimuli. As Hadid explains, “A person with blindsight can respond accurately to visual information without the conscious experience of seeing it.”
Blindsight demonstrates that accurate information processing and outwardly appropriate behavior can be fully dissociated from conscious experience. Whether subjective experience can ever arise purely from computation remains an open philosophical and scientific question—but current AI systems are clearly built from statistical learning, not lived experience.
Fluent responses without feeling
Modern conversational agents are trained on massive datasets and optimized to generate context-appropriate, fluent language. Their persuasive and empathetic tone comes from probabilistic pattern matching and optimization, not from genuine understanding or concern. As Jerbi warns, anthropomorphism—the automatic tendency to attribute human traits to nonhuman agents—can become a dangerous trap that fosters misplaced trust.
This risk is particularly acute when users are vulnerable. People seeking emotional support, companionship, or advice can form attachments to systems that cannot reciprocate. The danger is not only that an AI might respond poorly in critical moments, but that it might respond well enough for users to forget there is no conscious agent behind the text.
Toward more informed and responsible use
The authors do not call for rejecting AI. Instead, they recommend public education and clearer boundaries: recognize AI as a powerful computational tool while avoiding the illusion that it offers genuine empathy, moral judgment, or human-level care. This perspective helps preserve appropriate roles for professional support, ethical oversight, and human relationships in situations that demand reciprocity and responsibility.
“Confusing intelligence with consciousness is one of the great traps in our relationship with AI,” Jerbi says. By keeping the distinction clear, users and policymakers can make safer, more ethical choices about where and how to deploy conversational agents.
Key Questions Answered
A: Because they are engineered to master human language, emotional cues, and context through large-scale statistical learning. Their fluent, empathetic-sounding responses trigger an automatic human tendency to project inner life and intentions onto anything that behaves like a human being—what the authors call the “anthropomorphism trap.”
A: Blindsight shows that sophisticated information processing and accurate behavior can occur without conscious awareness. People with blindsight can identify aspects of visual stimuli without any conscious experience of seeing them. By analogy, AI can generate knowledgeable, emotionally tuned behavior without subjective experience or understanding.
A: The primary danger arises during emotional vulnerability: users can form attachments to systems that cannot reciprocate, mistaking algorithmic responses for real care. This misattribution can reduce the likelihood of seeking human support or professional help when it is truly needed.
Editorial Notes
- This article was edited by a Neuroscience News editor.
- The journal paper was reviewed in full by the editorial team.
- Additional context and clarifications were added by the staff.
About this AI and consciousness research news
Author: Julie Gazaille
Source: University of Montreal
Contact: Julie Gazaille – University of Montreal
Image: The image is credited to Neuroscience News