Summary: What does a mouse actually see? For the first time, researchers have reconstructed ten-second video clips from the neural activity recorded in the mouse visual cortex. By measuring the firing of individual neurons while mice watched natural movies and using an algorithm to translate those signals back into moving images, the team produced clear, time-accurate reconstructions that reveal how the brain represents visual scenes.
This advance uses single-cell two-photon calcium imaging—offering much higher spatial resolution than fMRI—to show that neural representations are not simple photographic copies of the world. Instead, the brain warps and prioritizes information, producing a faithful but interpretive view of visual input.
Key Facts
- Neural decoding at single-cell resolution: The study used microscopic imaging to detect calcium transients in individual neurons, identifying which cells fired in response to specific visual stimuli.
- Pixel-by-pixel reconstruction: An iterative algorithm began with a blank video and updated pixels based on the mismatch between predicted and observed neural responses, gradually producing the reconstructed movie.
- Ten-second movie reconstructions: The researchers reliably reconstructed 10 s clips at 30 Hz, even for videos that were not included in the model’s training set.
- Data scale matters: Reconstruction quality improved substantially as more neurons were included, highlighting the value of comprehensive, high-resolution neural recordings.
- Interpretation, not error: Lead author Dr. Joel Bauer emphasizes that differences between the reconstructed and original videos reflect how the brain represents and prioritizes sensory information—the deviation is a feature of perception rather than a flaw.
Source: UCL
Researchers at University College London have reconstructed videos from mouse brain activity, revealing how the visual cortex represents moving scenes.
Published in eLife, the study demonstrates a powerful approach to studying visual processing by combining precise single-cell recordings with a dynamic neural encoding model. This method enables researchers to infer what an animal saw from neural activity alone, offering a direct window into the neural code for vision and opening new opportunities to compare perception across species.
Previous work has attempted to decode visual experience from human fMRI signals, but fMRI lacks the cellular detail available with two-photon calcium imaging. The current study leverages those cellular signals to reconstruct natural movies shown to mice, yielding higher-fidelity results than earlier efforts based on population-level measures.
Lead author Dr. Joel Bauer (Sainsbury Wellcome Centre at UCL) explained: “We sought a better way to investigate how the brain interprets what we see. Existing methods often fail to generalize beyond specific test conditions, so we developed an approach that captures what is represented in the brain and compares that representation to reality.”
The team used a state-of-the-art dynamic neural encoding model—initially developed for the 2023 Sensorium Competition—that predicts individual neuron activity from video input while accounting for the animal’s motion and pupil diameter. They refined this model by computing the difference between predicted neural activity for a blank screen and the actual activity recorded during movie presentation. That difference guided an optimization process that adjusted pixels in a candidate video until its predicted neural responses matched the recorded responses.
Once trained, this optimization produced ten-second reconstructions from single-trial neural recordings of previously unseen movies. The researchers quantified reconstruction fidelity using pixel-wise correlation between the original and reconstructed videos. They report a pixel-level correlation of 0.57 for single-trial reconstructions—substantially higher than earlier static-image reconstructions from awake mouse V1, which achieved about 0.24 over a comparable retinotopic area.
Timing alignment between original and reconstructed videos was close, with only minimal temporal differences. The team plans to improve both spatial resolution and coverage of reconstructions by collecting data with larger cortical coverage and higher-resolution imaging, which should further boost accuracy.
Future work will use this tool to probe how visual representations diverge from the physical scene—how the brain selectively emphasizes, compresses, or distorts information. As Dr. Bauer notes, these deviations are meaningful: they reflect how the visual system prioritizes cues and constructs a useful internal model of the world rather than producing a verbatim copy of the retinal image.
Key Questions Answered:
A: Not yet. This study reconstructs real-time visual input, but the underlying decoding principles could eventually be applied to internal imagery such as memories or dreams. Significant technical and conceptual challenges remain before that goal can be achieved.
A: In broad terms, yes: animals and humans extract many of the same visual features. However, the brain’s internal representation is selectively filtered and reshaped—certain cues are enhanced while others are deemphasized. That “warping” reflects the organism’s way of prioritizing useful information.
A: Quite accurate for current standards. The team measured reconstruction quality with pixel correlation and found strong temporal alignment. Accuracy improved with the number of neurons recorded and with model ensembling, supporting the idea of a neural “livestream” of visual experience when sufficient data are available.
Editorial Notes:
- This article was edited by a Neuroscience News editor.
- Journal paper reviewed in full.
- Additional context provided by editorial staff.
About this neuroscience and neurotech research news
Author: Chris Lane
Source: UCL
Contact: Chris Lane – UCL
Image: The image is credited to Neuroscience News
Original Research: Open access. “Movie reconstruction from mouse visual cortex activity” by Joel Bauer, Troy W. Margrie, and Claudia Clopath. eLife. DOI: 10.7554/eLife.105081.3
Abstract
Movie reconstruction from mouse visual cortex activity
Reconstructing the visual content represented by the brain offers a direct and intuitive window into perception. While image reconstruction from human fMRI has attracted attention, single-cell recordings provide a more direct measure of represented information. Here, the authors present high-quality reconstructions of natural movies shown to mice, derived from activity in visual cortex recorded with two-photon calcium imaging.
Using video optimization via backpropagation through a dynamic neural encoding model, the team reconstructed 10 s movies at 30 Hz from single-trial calcium imaging data. They report a pixel-level correlation of 0.57 between ground-truth movies and reconstructed videos, substantially exceeding prior reconstructions based on awake mouse V1 responses to static images. The results show that the number of neurons sampled and model ensembling are critical for high-quality reconstructions and point toward broad applications for studying visual processing phenomena.