Summary: A new study cautions against equating AI intelligence with consciousness. Drawing on the neurological phenomenon known as “blindsight”—which shows that complex information processing can occur without conscious awareness—the researchers emphasize that AI systems operate through statistical learning and computation, not subjective feeling or lived experience.
As conversational agents grow more fluent and emotionally attuned, the team warns of an “anthropomorphism trap”: people may mistakenly treat AI as an empathetic partner or a moral agent instead of a sophisticated tool. The researchers urge users to rely on human connection and professional care when needed, and to treat AI as a powerful but non-conscious aid.
Key Facts
- Core distinction: Intelligent or emotionally responsive behavior in AI reflects statistical learning and computation, not subjective awareness or conscious experience.
- Blindsight parallel: Neuroscience shows that accurate, complex behavior can occur without conscious perception — a phenomenon exemplified by blindsight.
- Anthropomorphism risk: Fluent, humanlike responses can encourage people to project emotions, intentions, and consciousness onto AI systems.
- Vulnerability and harm: Relying on chatbots for psychological support can be risky when users mistake algorithmic responses for genuine care or reciprocal understanding.
- Informed use: The authors recommend treating AI as a computational tool and avoiding its use as a substitute for real human empathy or professional help.
Source: University of Montreal
Politeness toward chatbots
Have you ever thanked ChatGPT or said “please” to another conversational model? Such courtesy can feel natural. People may also worry—half in jest—that if machines ever became conscious, those who were rude might be punished. But behind fluent, sympathetic replies lies no inner life; these systems generate responses by statistical patterns learned from data.
With more people using chatbots for advice, comfort, or company, misunderstanding the nature of these systems can lead to real consequences. A team of neuroscientists from Université de Montréal and Johns Hopkins University reminds us of a crucial distinction: intelligence is not the same as consciousness.
A system can behave convincingly and respond to emotional cues without understanding them, feeling empathy, or having any subjective experience. The more convincing these agents become, and the more we rely on them, the more important it is to recognize their limits.
Decades of neuroscience
To illustrate their point, the authors draw on decades of research in neuroscience. One key example is blindsight: after damage to the primary visual cortex, some people report no conscious vision in part of their visual field but still perform above chance when asked to locate or identify visual stimuli, including emotional expressions.
“A person with blindsight can respond accurately to visual information without the conscious experience of seeing it,” said Vanessa Hadid, a postdoctoral researcher in psychology at the University of Montreal and the McGill University Health Centre. She co-authored the paper with Karim Jerbi (Université de Montréal and Mila) and John W. Krakauer (Johns Hopkins).
Blindsight underscores an important lesson: sophisticated information processing alone does not prove conscious experience. Whether consciousness could ever emerge from computation remains an open philosophical and scientific question, the authors note, but current AI systems do not possess subjective experience.
Fluent, but without feeling
Modern conversational agents are engineered to produce coherent, context-sensitive language by learning statistical relationships in vast datasets. Their apparent empathy and sensitivity are emergent effects of that training, not signs of inner life.
Karim Jerbi warns of the anthropomorphism trap: “Attributing emotions, intentions, or consciousness to something that behaves like a human can create an illusion of being understood and lead to misplaced trust.” This danger is especially acute when users are vulnerable; people can form attachments to systems that cannot reciprocate or offer real moral judgment.
Vanessa Hadid adds: “In contexts of psychological support, the risk is not only that AI may respond poorly, but that it may respond well enough for us to forget that there is no one behind the answer.” Jerbi emphasizes, “Current AI systems do not feel anything and do not have conscious experience. The more fluently they speak, the easier it becomes to forget that.”
Toward more informed use
The authors do not call for rejecting AI. Instead, they advocate for informed, responsible use: recognize these systems as powerful computational tools and avoid treating them as replacements for human empathy, moral judgment, or professional care. Maintaining clear expectations will help people benefit from AI while avoiding emotional dependence and potential harm.
Key Questions Answered:
A: They appear conscious because they are trained to master human language, syntax, and emotional context through massive statistical learning. Fluent, empathetic-sounding replies prompt an automatic projection of inner life—what researchers call the “anthropomorphism trap.”
A: Blindsight shows that accurate, complex behavior can occur without conscious perception. People with blindsight can identify locations or emotions of visual stimuli without consciously seeing them. Likewise, an AI can process information and behave intelligently without any subjective experience.
A: The primary danger arises when people in vulnerable states form attachments to systems that cannot reciprocate. Users may mistake algorithmic comfort for real care, potentially delaying or avoiding human support or professional help. The problem is that an AI may respond well enough to obscure the absence of a real person behind the interaction.
Editorial Notes:
- This article was edited by a Neuroscience News editor.
- The referenced journal paper was reviewed in full.
- Additional context was added by editorial staff.
About this AI and consciousness research news
Author: Julie Gazaille
Source: University of Montreal
Contact: Julie Gazaille – University of Montreal
Image: Image credit: Neuroscience News