Summary: The brain constantly generates predictions about incoming sensory information, and when those expectations are violated—whether in sight, sound, or both—neurons emit distinctive prediction error signals. New experiments in mice show that the auditory cortex produces strong error signals when expected sounds are missing, and simultaneous visual and auditory mismatches produce an amplified, non-linear response. Early human tests using virtual reality (VR) and EEG suggest the same principles apply to people and point toward possible clinical biomarkers for psychiatric conditions.
Our perception of the world relies heavily on prediction. As we move, the brain anticipates the sensory consequences of those movements, combining past experience with current motor commands to forecast visual and auditory feedback. When actual input deviates from these forecasts, specialized neurons report the discrepancy as a prediction error—an important signal for updating internal models of the world.
- Cross-Modal Prediction Errors: Concurrent mismatches in vision and sound trigger larger brain responses than mismatches in a single modality.
- Human-Brain Parallels: Preliminary VR and EEG recordings in humans show prediction error signals resembling those found in mice.
- Clinical Potential: Abnormal prediction error responses could become objective, brain-based biomarkers to help diagnose or monitor psychiatric conditions such as psychosis.
Research background
Previous work from the Keller laboratory at the Friedrich Miescher Institute (FMI) showed that in mice, pausing the visual flow while the animal runs through a virtual tunnel elicits a robust prediction error in visual cortex. It was unclear whether this phenomenon was unique to vision or represented a general function of sensory cortices.
To investigate auditory prediction errors, Magdalena Solyga designed experiments in which mice ran down a dark corridor while sound amplitude scaled with running speed. Occasionally the sound was unexpectedly muted, creating an audiomotor mismatch between the mouse’s expected and received auditory feedback. Neurons in the auditory cortex responded strongly to these omissions, demonstrating that prediction error signaling extends beyond vision and is present in auditory sensory areas.
Solyga then tested what happens when both visual and auditory feedback are tied to movement and are sometimes disrupted simultaneously. When the visual flow and sound were paused together, neural responses grew substantially larger than the sum of responses to single-modality mismatches. Some neurons were selective for this combined mismatch, suggesting that the brain performs non-linear integration of prediction errors across sensory modalities rather than treating them independently.
These mouse findings motivated adaptation of the paradigm for humans. In pilot studies, healthy volunteers navigated a VR environment while EEG recorded brain activity. When the visual scene froze unexpectedly while participants continued to move, EEG signals revealed a prediction error response similar to the visuomotor mismatch observed in mice. The research team is now expanding human tests to probe combined visual and auditory mismatches.
One important motivation is clinical translation. If individuals with psychiatric disorders—particularly those experiencing psychosis—show abnormal mismatch responses, these brain signals could help provide objective diagnostic or treatment-monitoring tools. Unlike subjective symptom reports, brain-based biomarkers could offer measurable indices of how sensory prediction and error processing are altered in specific conditions.
Translating these insights into clinical practice poses technical challenges. Recording reliable EEG during active movement is demanding because motion introduces electrical and muscle artifacts that mask the signals of interest. So far, the team has recorded data from 17 healthy adults and plans to recruit up to 50 participants to achieve robust, generalizable results. Practical factors such as hairstyle, electrode contact, and movement artifacts can affect signal quality, underscoring the need for larger samples and careful preprocessing.
The experiments also raise fundamental scientific questions. It remains unknown whether amplified multimodal prediction error responses arise from direct cross-talk between sensory areas, from a higher-order region that monitors mismatches across modalities, or from another circuit mechanism. Answering this will require targeted circuit interventions and recordings from multiple brain regions during multimodal perturbations.
Overall, this research demonstrates that prediction error computation is a multimodal cortical function and that interactions between sensory modalities can be non-hierarchical and non-linear. These findings broaden our understanding of how the brain monitors and updates sensory expectations and open avenues for both basic neuroscience and translational work aimed at objective markers of psychiatric dysfunction.
About this neuroscience research news
Author: Magdalena Solyga
Source: FMI
Contact: Magdalena Solyga – FMI
Image: The image is credited to Neuroscience News
Original Research: Open access. “Multimodal mismatch responses in mouse auditory cortex” by Magdalena Solyga et al. eLife. DOI: 10.7554/eLife.95398.3
Abstract
Multimodal mismatch responses in mouse auditory cortex
Movements generate predictable, often multimodal sensory feedback. Primary sensory cortical areas compute sensorimotor prediction errors when actual input deviates from these predictions. How prediction errors in one modality influence errors in another is not well understood. Using a virtual environment that couples running to both auditory and visual feedback, two-photon microscopy revealed that layer 2/3 neurons in mouse auditory cortex respond to sounds, visual stimuli, and running onsets. Audiomotor mismatch responses resembled visuomotor mismatch responses in visual cortex, and audiomotor mismatches were amplified when paired with concurrent visuomotor mismatches. These results indicate that multimodal and non-hierarchical interactions shape prediction error responses in cortical layer 2/3.