Summary: Our brains do more than store separate sights and sounds; they bind them into unified memories. New research shows that when visual and auditory speech cues are synchronized, they evoke rhythmic brain activity that supports stronger memory formation. When those cues are out of sync, that rhythmic activity is reduced and memories are weaker. This work highlights how precise timing across senses shapes the vividness and retrievability of memories that combine sight and sound.
Everyday memories—like recalling a friend telling a joke—blend how that person looked while speaking with how their voice sounded. The new study led by Emmanuel Biau at the University of Liverpool investigated how the timing between visual and auditory speech features influences the brain processes that form and later replay these audiovisual memories.
Key findings
- Synchronized audiovisual input strengthens memory: When lip movements and speech sounds were aligned, participants showed rhythmic brain activity associated with stronger memory encoding and later recall.
- Timing matters: Delaying auditory speech relative to lip movements weakened those memory-related oscillations during both viewing and subsequent memory retrieval.
- Phase alignment of neural rhythms: The results suggest that successful audiovisual memory binding depends on sensory inputs arriving in the same phase of ongoing brain oscillations, particularly theta rhythms in neocortex and hippocampus.
Study overview
Participants watched short movie clips of people speaking while researchers recorded brain activity using magnetoencephalography (MEG). The clips were presented in two conditions: synchronous, where auditory and visual onsets matched naturally, and asynchronous, where the auditory onset was delayed so speech sounds and lip movements were out of phase with the brain’s theta rhythm observed during the original clip.
When visual and auditory cues were synchronized, theta-band oscillations emerged in both neocortical and hippocampal regions during viewing. Crucially, those same oscillatory patterns tended to reappear when participants later recalled the clips, indicating a neural replay linked to successful memory. By contrast, when the audio lagged behind the visual signal, theta oscillations diminished during initial perception and were less likely to reemerge during recall, corresponding with poorer memory performance.
According to Biau, if auditory and visual inputs reach the brain simultaneously, they are more likely to be associated because they fall into the same phase of neural activity. Asynchronous inputs fail to align in phase, making it harder for the brain to bind them into a single cohesive memory.
Implications
These results suggest that theta oscillations in neocortex and hippocampus support the integration and later replay of audiovisual speech information. The study adds evidence that precise temporal alignment across senses is a key factor in forming integrated memories. It also points to a potential neural mechanism—phase alignment of theta rhythms—by which the brain binds sights and sounds together.
About this audiovisual memory research news
Author: SfN Media
Source: SfN
Contact: SfN Media – SfN
Image: The image is credited to Neuroscience News
Original Research: Closed access.
“Neocortical and Hippocampal Theta Oscillations Track Audiovisual Integration and Replay of Speech Memories” by Emmanuel Biau et al., Journal of Neuroscience. DOI: 10.1523/JNEUROSCI.1797-24.2025
Abstract summary
Neocortical and Hippocampal Theta Oscillations Track Audiovisual Integration and Replay of Speech Memories
The study examined how fine-grained synchrony between visual and auditory speech features affects brain oscillations during the formation of multisensory speech memories. Using a memory task with short synchronous or asynchronous movie clips of speakers while recording MEG, researchers compared conditions where natural audiovisual alignment was preserved versus conditions where the auditory onset was delayed to create antiphase relationships relative to theta oscillations.
Results showed that theta oscillations in neocortex and hippocampus were modulated by the degree of synchrony between lip movements and syllables during perception. When lip movements and auditory envelopes were out of phase, the accuracy of subsequent theta reinstatement during memory recollection was reduced. The authors conclude that neural theta oscillations play a pivotal role in both audiovisual integration and memory replay for speech.