Summary: Researchers introduce a new computational framework that uses advanced artificial intelligence to clarify how perception and memory interact in the human brain.
Source: Stanford
Neuroscience remains a young science compared with the physical sciences. While physics has detailed accounts of how complex phenomena arise from atomic and subatomic interactions, we still lack a similarly precise, mechanistic understanding of how intelligent behavior emerges from neural circuits.
To make progress, neuroscientists rely on intuitive, descriptive concepts such as “perception” and “memory.” These concepts help organize experiments and interpret behavior, but they are broad labels that can obscure the underlying neural computations that produce those behaviors.
For example, in primates the ventral visual stream (VVS) is widely associated with visual perception, while the medial temporal lobe (MTL) is classically linked to memory. Yet translating these conceptual distinctions into precise statements about neural function has proven difficult.
This conceptual gap fuels an ongoing and seemingly intractable debate: where does perception end and memory begin in the brain? Do neural systems respect the same boundaries implied by our everyday language? The answer matters not only for theory but also for clinical applications—for instance, better understanding how memory and perception interact could inform treatments for memory disorders, such as post-traumatic stress disorder.
For decades, researchers have interpreted the same experimental data in different ways. One group argues that the MTL contributes to both memory and certain perceptual processes; another maintains the MTL’s role is restricted to memory. These disagreements have persisted in part because experiments were evaluated using informal, varying intuitions about what counts as a perceptual or mnemonic task.
To confront this debate, Tyler Bonnen, a Stanford doctoral candidate in psychology, worked with Daniel Yamins, assistant professor of psychology and computer science and member of the Stanford Institute for Human-Centered Artificial Intelligence, and Anthony Wagner, professor of psychology and director of The Memory Lab at Stanford. Their study, published in Neuron, presents a computational strategy that uses modern AI tools to disentangle perception and memory by modeling neural-level behavior directly from experimental stimuli.
A fresh computational approach
The team leveraged advances in computer vision—particularly task-optimized convolutional neural networks (CNNs)—which are among the most successful AI models for predicting primate visual cortex responses. Unlike earlier hand-crafted models, these deep learning systems can take raw image pixels as input and produce behavior and neural activation patterns that closely match those measured in visual cortex.
“These models are more than just effective at image recognition,” Bonnen explains. “They also outperform prior neuroscience-specific models at predicting neural responses in primate visual areas, which allows us to use them as a biologically informed proxy for the ventral visual stream.”
Using these models, the researchers recreated the exact stimuli and task conditions from 30 previously published visual discrimination experiments that had been used to argue both for and against MTL involvement in perception. They fed the same images, in the same order, into the computational models and measured model performance on the same tasks. Then they compared those results directly to human behavioral data, including data from participants with MTL lesions.
The findings were revealing. Across experiments, the CNN-based VVS model predicted the behavioral performance of participants with MTL lesions. In contrast, participants with intact MTLs consistently outperformed the model. This pattern suggests that many behaviors previously labeled as perceptual actually require computations supported by MTL structures, resolving apparent inconsistencies in the literature.
Bonnen is careful about labels. “Our results align with the idea that MTL contributes to behaviors often called perceptual,” he says. “But our main aim is not to argue over terminology. Instead, we want to use computational models that solve the same tasks humans face—starting from pixels—to reveal how the brain carries out those tasks.”
The methodological advance is central: by asking AI systems to perform the same problems presented to human subjects and comparing model outputs to lesion and electrophysiological data, the team places behavioral, lesion, and neural results within a shared, testable computational framework.
Resolving debates and opening directions
This work demonstrates the limits of conceptual labels when they are not grounded in computational mechanisms and shows how modern AI can provide those mechanisms. By formalizing what the ventral visual stream can and cannot accomplish on its own, the research clarifies when additional computations—implemented by MTL structures such as the perirhinal cortex—are needed to support demanding visual discrimination tasks.
Beyond settling a longstanding debate, the approach offers a proof of principle: biologically informed deep learning models can extend neuroscientific inquiry beyond canonical visual cortex and help illuminate how other brain systems contribute to complex behaviors. For the MTL, this could advance both basic understanding of memory-perception interactions and applied research aimed at ameliorating memory-related disorders.
The authors note an important caveat. While vision-focused deep learning models are now well developed, algorithms that capture affective and memory-related computations are less mature. Developing such models—ideally aligned with the underlying biology—remains a crucial next step. Even so, this study shows that AI can formalize and test intuitions about neural function, providing a powerful bridge between computational models and empirical neuroscience.
About this artificial intelligence research news
Source: Stanford
Contact: Shana Lynch – Stanford
Image: The image is in the public domain
Original Research: Closed access. “When the ventral visual stream is not enough: A deep learning account of medial temporal lobe involvement in perception” by Tyler Bonnen et al., Neuron.
Abstract
When the ventral visual stream is not enough: A deep learning account of medial temporal lobe involvement in perception
The medial temporal lobe (MTL) supports a range of memory-related behaviors, but its role in perceptual processing has been controversial. This debate focuses on the perirhinal cortex (PRC), an MTL structure positioned at the apex of the ventral visual stream (VVS).
The authors use a deep learning framework that models visual behaviors supported by the VVS alone—i.e., without PRC contributions. Applying this approach retroactively to 30 published visual discrimination experiments, they find a strong correspondence between model-predicted performance and behavior of PRC-lesioned participants, while PRC-intact participants outperform both the model and PRC-lesioned groups.
A new experiment comparing PRC-intact human performance to electrophysiological recordings from macaque high-level visual cortex further supports these findings: human participants with intact PRC outperform a linear readout of visual cortical activity. By bringing lesion, electrophysiological, and behavioral data into a unified computational framework, the study resolves decades of seemingly conflicting results about PRC involvement in perception.