Summary: Researchers have produced the first direct human evidence that brain-controlled technology can help a listener isolate a single voice in a noisy, multi-speaker environment. The system functions as a neural extension, using real-time brain signals to identify which speaker a person is attending to and automatically amplifying that voice while suppressing competing speech.
This development targets the longstanding “cocktail party effect” problem—why conventional hearing aids often fail to separate overlapping conversations in crowded settings.
Key Research Findings
- The Brain-First Approach: Instead of simply amplifying all incoming sound, the system leverages the brain’s natural filtering ability to focus on the talker the listener intends to hear.
- Real-Time Identification: Machine-learning algorithms analyze the timing of brainwave peaks and troughs and match these patterns to the rhythms of individual speakers to determine which voice is attended.
- Direct Human Evidence: The study tested patients with implanted electrodes and showed the system could correctly identify the listener’s focus and adjust audio levels instantly, improving speech intelligibility and reducing listening effort.
- Dynamic Flexibility: The technology worked both when participants were instructed to attend to a specific speaker and when they freely chose which conversation to follow, reflecting everyday social behavior.
- Practical Application: The research moves brain-controlled hearing from theory to a working prototype that delivers immediate, measurable benefits in real time.
Source: Columbia University
Scientists at Columbia University’s Zuckerman Institute report the first direct human evidence that brain-controlled hearing can enable listeners to single out a voice in a crowd.
The early results suggest a path toward hearing augmentation devices that overcome the limitations of conventional hearing aids in noisy environments.
Their work was published today in Nature Neuroscience.
“We built a system that acts as a neural extension of the user, using the brain’s inherent ability to filter sounds in complex environments to dynamically isolate the conversation they want to hear,” said senior author Nima Mesgarani, PhD, principal investigator at Columbia’s Zuckerman Institute and associate professor of electrical engineering at Columbia’s Fu Foundation School of Engineering and Applied Science.
“This approach points beyond conventional hearing aids that simply amplify sound, toward technology that restores the selective hearing functions of the brain,” Dr. Mesgarani added.
The study was carried out with epilepsy patients who already had electrodes implanted as part of clinical care to localize seizure sources. With consent, these patients participated in experiments where two overlapping conversations were played simultaneously while the system recorded their brain activity.
Using the implanted electrodes, the system tracked brain responses as participants focused on one of the two speakers. In real time, the algorithms detected which conversation each participant attended and adjusted audio levels—raising the target voice and lowering the competing talker.
Participants had strong reactions: one volunteer was convinced the researchers were secretly changing volumes, while others imagined how the technology could help loved ones with hearing loss.
Modern hearing aids are effective at amplifying speech and filtering some types of background noise like traffic, but they typically cannot single out a particular voice among many. They amplify every voice that reaches the microphone, which makes it hard to focus on one talker amid multiple speakers.
A promising alternative is a device that mimics the brain’s ability to identify and focus on a single speaker in a crowd—the so-called cocktail party effect. Building on earlier work, researchers have shown that specific brain signals synchronize with the temporal features of a speaker’s voice, and those patterns can indicate which conversation a person is attending.
Over the past decade, Dr. Mesgarani and colleagues developed methods to separate the audio streams of multiple speakers and match each stream to the listener’s brain activity. The current study integrates those advances into a closed-loop, real-time system that can dynamically amplify the attended talker.
“The central question was whether brain-controlled hearing could move from incremental progress to a prototype that improves perception in real time,” said Vishal Choudhari, the paper’s first author, who led system development and evaluation. “For the first time, we show that a brain-reading system that selectively enhances conversations provides clear, immediate benefits.”
Clinical collaborators and volunteers at several medical centers contributed to the study. The team developed fast, reliable machine-learning decoders that examined brainwaves and matched them to individual voice streams. The system detected attention both when participants were instructed to attend and when they freely shifted attention, as in natural conversation.
Researchers report the system accurately identified attended conversations, substantially improved speech intelligibility for listeners, reduced listening effort, and was consistently preferred over unassisted audio. Volunteers described imagining a future where such technology could make social settings less stressful and more accessible.
Worldwide, more than 430 million people live with disabling hearing loss, and many struggle most in noisy social environments. Untreated hearing loss is a modifiable risk factor for dementia and contributes to depression and social isolation. Scientists say this research lays a critical foundation for wearable systems that combine brain sensing with advanced audio processing to assist people with hearing loss and reduce listening fatigue for anyone in challenging acoustic settings like restaurants, classrooms, or busy workplaces.
The team cautions that significant work remains before a minimally invasive, wearable version can operate reliably in complex real-world conditions. Future studies will test performance in more natural listening environments and work to shrink or replace implanted sensors with less invasive alternatives.
“These results represent an important step toward brain-controlled hearing technologies that align with user intent and could transform how people navigate noisy, multi-talker situations,” Dr. Choudhari said.
Author list: Vishal Choudhari, Maximilian Nentwich, Sarah Johnson, Jose L. Herrero, Stephan Bickel, Ashesh D. Mehta, Daniel Friedman, Adeen Flinker, Edward F. Chang, and Nima Mesgarani.
Funding: Supported by the Marie-Josee and Henry R. Kravis Foundation and the National Institute on Deafness and Other Communication Disorders (NIH).
Key Questions Answered:
A: The current study used implanted electrodes for precision, but the long-term aim is to create wearable, minimally invasive systems that combine brain sensing with audio processing for everyday use.
A: Each voice has a distinct rhythm of sounds and silences. When you focus on a speaker, your brain activity synchronizes with that speaker’s rhythm. The system compares detected voice streams to the listener’s brainwave patterns and matches the attended voice.
A: Disabling hearing loss affects hundreds of millions of people and contributes to cognitive decline, depression, and social isolation. Technology that restores selective hearing could reduce listening fatigue and improve social engagement and quality of life.
Editorial Notes:
- This article was edited by a Neuroscience News editor.
- The journal paper was reviewed in full.
- Additional context was added by staff.
About this auditory neuroscience and neurotech research news
Author: Nima Mesgarani, PhD
Source: Columbia University
Contact: Nima Mesgarani, PhD – Columbia University
Image: Image credit: Neuroscience News
Original Research: Open access. “Real-time brain-controlled selective hearing enhances speech perception in multi-talker environments” by Vishal Choudhari, Maximilian Nentwich, Sarah Johnson, Jose L. Herrero, Stephan Bickel, Ashesh D. Mehta, Daniel Friedman, Adeen Flinker, Edward F. Chang & Nima Mesgarani. Nature Neuroscience
DOI: 10.1038/s41593-026-02281-5
Abstract
Real-time brain-controlled selective hearing enhances speech perception in multi-talker environments
Many people find it difficult to understand speech in noisy settings because current hearing aids amplify all sounds rather than the specific talker of interest. Auditory attention decoding (AAD) offers a solution by using a listener’s brain signals to identify and enhance the attended speaker, but until now it was uncertain whether that approach could deliver real-time perceptual benefits.
Using high-resolution intracranial electroencephalography in patients undergoing neurosurgical preparation, the researchers implemented a closed-loop system with the decoding fidelity needed to dynamically amplify the attended talker. Across multiple experiments, the system improved speech intelligibility, reduced listening effort, and was preferred by subjects. It tracked both instructed and self-initiated shifts in attention.
By demonstrating that a real-time, brain-controlled hearing system can enhance perception, this work establishes a performance benchmark for future auditory brain–computer interfaces and advances AAD from a theoretical concept toward a validated, personalized assistive hearing solution.