Summary: Using electrocorticogram (ECoG) recordings to capture brain-wave patterns, researchers demonstrated that the semantic category of a mentally imagined image can be inferred from those patterns, even when the imagined content conflicts with what the person is actually viewing.
Source: Osaka University
Researchers in Japan report that a person’s mental image can be decoded from brain activity, revealing the semantic meaning of what they imagine even when it differs from their ongoing visual input.
A study published in Communications Biology by a team at Osaka University shows that electrocorticogram recordings—direct measurements of cortical electrical activity—contain enough information to distinguish the semantic category of images a person is imagining. Crucially, this remains possible even while subjects view different, potentially conflicting images.
When we perceive a scene, our visual cortex produces characteristic patterns of electrical activity. Those patterns can be captured with electrocorticography and analyzed to infer aspects of the perceived content. But perception is dynamic: attention and mental imagery can alter those neural representations. The new research asked whether deliberate imagery could actively reshape neural representations while a person looked at an image from another category.
Lead author Ryohei Fukuma explains that while attention is already known to modulate neural representations of perceived images, it was unclear whether imagining a different image could produce measurable changes in those representations during conflicting visual stimulation.
To explore this, the team worked with patients undergoing clinical monitoring for epilepsy who had subdural electrodes implanted for electrocorticogram recording. The researchers built a closed-loop system that decoded semantic information from the intracranial recordings in real time using a visual-semantic space. The system displayed an inferred image on a monitor in front of the participant, and the participant was instructed to imagine a different category—such as a landscape, a human face, or a word—while viewing various images.
Senior author Takufumi Yanagisawa notes that the experiment clarified how brain activity differs when people look at images versus when they actively imagine different images. The ECoG readouts tied to imagery were distinguishable from the signals evoked by the actual images on the screen. Moreover, providing participants with real-time feedback made those imagery-driven signals even more distinct, suggesting that feedback can enhance voluntary control over semantic neural representations.
The time required for imagery to produce a clear shift in neural representations varied by category: generating a distinct representation for a word versus a landscape took different amounts of time. This likely reflects that different semantic categories engage partially distinct cortical networks and processing dynamics.
Fukuma highlights a practical implication: because observers can infer the semantic content a subject is imagining from ECoG readouts, this approach could form the basis of communication systems that rely on visual imagery rather than motor responses. Such systems might be especially valuable for people with severe paralysis—like advanced amyotrophic lateral sclerosis (ALS)—whose motor control is compromised more rapidly than visual cortical function. An imagery-based communication device could, in principle, offer an alternative channel for expressing intentions or selecting messages.
About this neuroscience research news
Author: Press Office
Source: Osaka University
Contact: Press Office – Osaka University
Image: The image is credited to the researchers
Original Research: Open access.
“Voluntary control of semantic neural representations by imagery with conflicting visual stimulation” by Ryohei Fukuma et al., Communications Biology
Abstract
Voluntary control of semantic neural representations by imagery with conflicting visual stimulation
Visual perception and mental imagery share overlapping neural representations, but it has been unclear how imagery influences those representations when perceived and imagined content are semantically inconsistent. The authors hypothesized that deliberately imagining an image would activate a corresponding neural representation even while perceiving a different, conflicting image.
To test this, the research team developed a closed-loop experimental setup that translated electrocorticogram signals into a visual-semantic space and displayed the decoded content back to participants in real time. Participants attempted to shift the decoded semantic vector toward an instructed imagined category while watching images from other categories.
Results showed that participants could successfully modulate the feedback images: the semantic vector inferred from ECoG signals moved closer to the vector of the imagined category despite concurrent viewing of different categories. The modulation effect depended on the combination of perceived and imagined categories and exhibited asymmetric influences across categories. Overall, the study demonstrates that mental imagery can selectively and voluntarily influence semantic neural representations under conflicting visual stimulation, with implications for brain-based communication interfaces and our understanding of how perception and imagery interact in the human brain.