Summary: Researchers have presented the first direct human evidence that brain-controlled hearing technology can help a listener focus on a single voice in a noisy, multi-speaker environment. The system functions as a neural extension: it reads real-time brain activity to determine which speaker a person is attending to and then automatically amplifies that voice while suppressing competing sounds.
This advance addresses the long-standing “cocktail party problem,” a situation where conventional hearing aids often fail because they amplify all incoming sounds and cannot selectively enhance one speaker among many.
Key Research Findings
- The Brain-First Approach: Instead of amplifying every sound indiscriminately, the system uses the brain’s natural attentional filtering to decide which voice to enhance.
- Real-Time Identification: Machine-learning algorithms analyze the timing of peaks and troughs in brain waves and relate them to the rhythmic patterns of individual speakers to determine the listener’s focus.
- Direct Human Evidence: The study involved epilepsy patients who already had implanted electrodes. The system identified their attentional target and adjusted audio levels instantly, improving speech intelligibility and reducing the cognitive effort of listening.
- Dynamic Flexibility: The technology worked both when participants were instructed to attend to a particular speaker and when they made spontaneous choices—reflecting how attention shifts in everyday conversations.
- Practical Prototype: These results move brain-controlled selective hearing from experimental demonstration toward a working prototype that provides immediate perceptual benefit in real time.
Source: Columbia University
Scientists at Columbia University’s Zuckerman Institute have produced the first direct human evidence that brain-controlled hearing technology can help people single out a voice in noisy settings.
The early results indicate a path toward hearing augmentation devices that could overcome the limitations of conventional hearing aids in crowded, noisy environments.
The research appears online in Nature Neuroscience.
“We built a system that serves as a neural extension for the user, using the brain’s own ability to sift through complex acoustic scenes and isolate the conversation they want to hear,” said senior author Nima Mesgarani, PhD, principal investigator at Columbia’s Zuckerman Institute and associate professor of electrical engineering at Columbia’s Fu Foundation School of Engineering and Applied Science.
“This moves the field beyond traditional amplification toward restoring the selective listening ability of the human brain,” Dr. Mesgarani added.
The study recruited patients undergoing neurosurgery for epilepsy who had temporary intracranial electrodes placed to localize seizure origins. With those electrodes, the researchers measured high-resolution brain activity as participants listened to two overlapping conversations and focused on one.
The system automatically detected which conversation each patient was attending to and adjusted the audio mix in real time—amplifying the attended speaker and quieting the other. Participants reported striking experiences: one volunteer initially suspected the researchers were secretly changing the volume, and others imagined the potential benefit for relatives with hearing loss.
Existing hearing aids can amplify speech and reduce steady background noise like traffic, but they generally cannot isolate and enhance a single voice among many. As a result, wearers still struggle to focus on a single talker in social settings.
A device that mimics the brain’s ability to lock onto one speaker—the so-called cocktail party effect—would address this limitation. In earlier work, Dr. Mesgarani and colleagues showed that specific patterns of brain activity align with the temporal structure of attended speech: the timing of neural peaks and troughs corresponds to the rhythm of a conversation, enabling identification of the attended stream.
Over the past decade the team and others have developed algorithms capable of separating multiple simultaneous voices and linking each separated audio stream to the listener’s brain signals. The current study asked whether such a closed-loop, brain-controlled system could operate fast and reliably enough to give a real-time perceptual advantage.
“The big question was whether brain-controlled hearing could move from incremental laboratory advances to a prototype that helps someone hear better in the moment,” said Vishal Choudhari, the paper’s first author, who led the system’s development and evaluation while completing his PhD in Dr. Mesgarani’s lab. “For the first time, we show that a system reading brain signals to enhance selected conversations can deliver a clear real-time benefit.”
Researchers collaborated with physicians and volunteers at multiple medical centers. They implemented fast, real-time machine-learning models that compared the brain’s activity patterns to separately recovered voice streams and determined which speaker the listener intended to follow. The system worked when attention was directed by the experimenters and when listeners shifted attention on their own.
The team found that the system reliably identified the attended speaker, significantly improved speech intelligibility, lowered listening effort, and was consistently preferred by participants over unassisted listening. Volunteers described the potential for such technology to restore more peaceful social experiences for people with hearing loss.
More than 430 million people worldwide live with disabling hearing loss, and many face the greatest challenges in noisy, multi-talker environments. Untreated hearing loss is also a major modifiable risk factor for dementia and contributes to depression and social isolation.
The study lays groundwork for future wearable systems that could combine noninvasive or minimally invasive brain sensing with advanced audio processing to assist people with hearing impairment and reduce listening fatigue for anyone in challenging acoustic settings—restaurants, classrooms, workplaces, and family gatherings.
The researchers emphasize that substantial work remains before a wearable, minimally invasive version is available for everyday use. Future tests will need to show robust performance in more complex, real-world acoustic scenes.
“These results are an important step toward a new generation of brain-controlled hearing technologies that align with a listener’s intent and could transform how people navigate noisy, multi-speaker environments,” said Dr. Choudhari.
Authors: Vishal Choudhari, Maximilian Nentwich, Sarah Johnson, Jose L. Herrero, Stephan Bickel, Ashesh D. Mehta, Daniel Friedman, Adeen Flinker, Edward F. Chang, and Nima Mesgarani.
Funding: Supported by grants from the Marie-Josee and Henry R. Kravis Foundation and the U.S. National Institute on Deafness and Other Communication Disorders.
Key Questions Answered:
A: This study used surgical electrodes to obtain precise signals, but the research goal is to create wearable, minimally invasive systems that combine brain sensing with audio processing for daily use.
A: Each voice follows a unique pattern of sound and silence. When you attend to a speaker, your brain activity synchronizes with that speaker’s rhythm. The algorithms match the listener’s brainwave patterns to the separate voice streams to identify the attended talker.
A: Disabling hearing loss affects hundreds of millions of people and is linked to increased risk of dementia, depression, and social isolation. Technology that restores effective social listening could reduce cognitive strain and improve quality of life for many.
Editorial Notes:
- This article was edited by a Neuroscience News editor.
- The journal paper was reviewed in full.
- Additional contextual material was added by the editorial staff.
About this auditory neuroscience and neurotech research news
Author: Nima Mesgarani, PhD (contact via Columbia University)
Source: Columbia University
Image: Image credit: Neuroscience News
Original Research: Open access. “Real-time brain-controlled selective hearing enhances speech perception in multi-talker environments” by Vishal Choudhari, Maximilian Nentwich, Sarah Johnson, Jose L. Herrero, Stephan Bickel, Ashesh D. Mehta, Daniel Friedman, Adeen Flinker, Edward F. Chang & Nima Mesgarani. Nature Neuroscience. DOI: 10.1038/s41593-026-02281-5
Abstract
Real-time brain-controlled selective hearing enhances speech perception in multi-talker environments
Many people find it difficult to follow speech in noisy settings because current hearing aids amplify all incoming sounds rather than isolating the talker of interest.
Auditory attention decoding (AAD) aims to use a listener’s brain signals to identify and amplify the attended speaker, but whether this approach can deliver perceptual benefits in real time has been uncertain.
Using high-resolution intracranial electroencephalography in patients undergoing neurosurgery, the researchers implemented a closed-loop system capable of decoding attention with the fidelity required to dynamically amplify the attended talker.
Across multiple experiments, the system improved speech intelligibility, reduced listening effort, was preferred by participants, and successfully tracked both instructed and self-initiated attention shifts.
These findings provide direct evidence that a real-time, brain-controlled hearing system can enhance perception, establishing a performance benchmark for future auditory brain–computer interfaces and advancing AAD from concept to a validated approach for personalized assistive hearing.