Summary: Researchers have identified how working memory is formatted, showing that visual memory can be stored in a flexible, abstract form.
Source: NYU
A team of scientists has uncovered how working memory is “formatted,” advancing our understanding of how visual information is stored and used.
“For decades researchers have sought to understand the neural representations that support working memory,” says Clayton Curtis, professor of psychology and neural science at New York University and senior author of the study published in the journal Neuron.
In this work, Curtis and co-author Yuna Kwak used experimental tasks together with new analytical approaches to reveal how the brain encodes visual memories.
Working memory—the ability to hold information briefly for immediate use—underpins many higher cognitive processes and is implicated in psychiatric and neurological conditions when it malfunctions. Yet despite its central role, the precise nature of how the brain stores working memory representations has remained unclear.
“We can often decode the contents of someone’s working memory from brain activity, but what those activity patterns actually represent has been opaque,” Curtis explains.
The researchers proposed that the brain not only discards irrelevant features but also re-codes relevant features into memory formats that are efficient and distinct from the original sensory input. There is already evidence that visual information such as letters and numbers can be recoded into phonological, sound-based forms for verbal working memory—for example, remembering the sound of a phone number rather than its visual appearance. But that observation does not explain the specific format of visual working memory representations.
To investigate, the team measured brain activity using functional magnetic resonance imaging (fMRI) while participants completed visual working memory tasks. On each trial, participants briefly viewed a stimulus, retained it for a few seconds, and then made a precise, memory-based judgment. Some trials used a tilted grating; others used a cloud of moving dots. After a delay, participants reported the exact angle of the grating’s tilt or the motion direction angle of the dot cloud.
Despite the different sensory inputs, the researchers found that patterns of activity in visual cortex and parietal cortex—regions involved in visual processing and memory—were interchangeable during memory. A decoder trained to predict motion direction from activity could also predict grating orientation, and vice versa.
This interchangeability suggested that the brain extracts the task-relevant feature (angle or direction) and represents it in a shared, abstract format rather than preserving the full sensory details. Curtis and colleagues hypothesized that this shared format resembles a simple, line-like pattern angled to match either the grating orientation or the dot-motion direction.
To test that idea, the team developed a novel visualization method. They used models of each cortical population’s receptive field to project the memory-related activity patterns onto a two-dimensional map of the visual display. This projection produced a screen-space representation of cortical activity and allowed the researchers to inspect the spatial structure of memory representations directly.
Using this approach, the team observed line-like patterns of activity across topographic maps in the cortex, with angles that corresponded to both the direction of motion and the orientation of the grating. In other words, the same abstract line-like representation captured the behaviorally relevant feature of two distinct stimulus types.
Conceptually, a single line—like a pointer or arrow—encoded the direction of motion (for example, up and left) and the orientation of the grating (the same up-and-left tilt). The tasks required participants to remember the summary feature (motion direction or grating angle), not every pixel or detail such as spatial frequency or contrast. That selectivity allowed the method to separate the storage of relevant information from irrelevant visual details.
“Our visual memory is flexible,” Curtis concludes. “It can store abstracted representations of what we see—formats that are more efficient and more closely tied to the behaviors they guide.”
About this memory research news
Author: Press Office
Source: NYU
Contact: Press Office – NYU
Image: The image is in the public domain
Original Research: Closed access.
“Unveiling the abstract format of mnemonic representations” by Clayton E. Curtis et al., Neuron.
Abstract
Unveiling the abstract format of mnemonic representations
Highlights
- Revealed the neural nature of abstract working memory representations
- Distinct visual stimuli (orientation and motion) were recoded into a shared abstract memory format
- Memory representations for orientation and motion direction took the form of a line-like pattern
- These abstract formats are more efficient and closely linked to the behaviors they support
Summary
Working memory bridges perception and behavior by maintaining information for immediate use. The authors propose that working memory stores abstractions of low-level perceptual features. Using fMRI and receptive-field-based projections, they show that oriented gratings and moving dots are flexibly recoded into the same mnemonic format in visual and parietal cortex when that format is useful for guiding behavior. Behaviorally relevant features—orientation and motion direction—are extracted and represented as an abstract, line-like pattern. The findings indicate that mnemonic representations are efficient abstractions of percepts that are proximal to the actions they support.