Summary: A new study reveals how the brain represents and responds to emotionally charged scenes, showing that the occipital temporal cortex (OTC) encodes both the semantic category and the emotional intensity of visual stimuli to guide behavior.
Researchers report that the OTC does more than register simple “approach” or “avoid” signals. Instead, it distinguishes stimulus types (people, animals, objects, landscapes) and sorts them by emotional valence and arousal, supporting a richer range of behavioral choices. These findings clarify how visual and affective information combine in the brain and could inform future work on neurological and psychiatric conditions that affect emotion processing.
Key Facts:
- Brain representation: The occipital temporal cortex differentiates emotional stimuli by category and intensity.
- Behavioral guidance: OTC tuning supports nuanced responses beyond simple approach/avoid decisions.
- Research implications: Results provide a framework for studying emotional processing in neurological and psychiatric disorders.
Source: TCD
Why recognizing emotionally salient scenes matters
The ability to recognize and respond appropriately to emotionally important situations is fundamental to survival and social functioning. A new paper published in Nature Communications examines how the human brain represents natural emotional scenes and whether those representations are organized to support adaptive behavioral choices.
The research team, led by Prof. Sonia Bishop (Trinity College Dublin) with contributions from Samy Abdel-Ghaffar (formerly a PhD student in Prof. Bishop’s lab at UC Berkeley, now at Google) and collaborators across multiple institutions, investigated how visual semantic categories and affective features are co-represented in the occipital temporal cortex.
Prof. Bishop explains that while many neuroscience studies reduce motivated behavior to simple approach or avoidance, real-world emotional scenes demand a broader repertoire of responses. For example, fleeing from a threatening bear is a very different avoidance behavior than keeping distance from a sick animal; approaching a potential mate differs from approaching an infant. The authors asked whether OTC representations reflect these meaningful distinctions.
To answer this, the researchers presented volunteers with a large set of natural images—over 1,600 scenes that included interactions, animals, objects, buildings and landscapes—while collecting functional MRI data. Participants judged each image for valence (positive, negative, neutral) and arousal (emotional intensity). A separate group rated which behaviors best matched each scene.
Using voxel-wise modeling and modern machine learning, the team examined activity patterns in fine-grained brain volumes (voxels under 3 mm3). They found that regions within the OTC encode both the semantic category of a stimulus (for example, single human, couple, crowd, reptile, mammal, food, object, building, landscape) and its affective properties (valence and arousal). Importantly, these dimensions interacted: different combinations of animacy, valence and arousal were represented in distinct OTC subregions.
When the researchers used these neural tuning patterns to predict the behavioral responses selected by the second group of participants, the OTC-derived dimensions outperformed models built directly from image features alone. This indicates that the OTC does not simply mirror image statistics but transforms visual and affective features into representations that are well suited to guide behavior.
Samy Abdel-Ghaffar highlights that the voxel-wise modeling approach—combining large image sets, encoding models and machine learning—provides finer-grained insights into how OTC subregions represent the conjunction of semantic and emotional attributes. The approach reveals stable tuning profiles across voxels that map onto behaviorally relevant distinctions.
Prof. Bishop adds that because the paradigm involves passive viewing rather than a demanding task, it is well suited to future studies examining how individuals with neurological or psychiatric conditions process emotional natural stimuli. By identifying where and how affective and semantic features are combined in OTC, researchers can better investigate atypical patterns of emotion perception and behavioral selection.
Study methods (brief)
The team used a dataset of 1,620 emotional natural images and collected more than 3,800 functional MRI volume acquisitions while adult participants viewed the images and provided valence and arousal ratings. Voxel-wise modeling at a spatial scale of approximately 2.4 × 2.4 × 3 mm revealed that OTC voxels were tuned to stimulus animacy, arousal, and interactions between animacy and valence/arousal. These neural dimensions predicted behavior selections for new participants better than models based directly on image properties.
About this neuroscience and emotion research news
Author: Fiona Tyrrell
Source: Trinity College Dublin (TCD)
Contact: Fiona Tyrrell – TCD
Image: Image credited to Neuroscience News
Original Research: Open access. “Occipital-temporal cortical tuning to semantic and affective features of natural images predicts associated behavioral responses” by Samy Abdel‑Ghaffar et al., published in Nature Communications.
Abstract (summary):
People must respond appropriately to a wide range of emotional stimuli in daily life. This study demonstrates that human occipital-temporal cortex (OTC) co-represents the semantic category and affective content of visual stimuli. Voxel-wise modeling across 1,620 emotional images shows widespread tuning to semantic and affective features: the principal components of OTC responses encoded animacy, arousal, and interactions of animacy with valence and arousal. At modest dimensionality, OTC tuning patterns predicted behavioral responses associated with each image more accurately than regressors derived directly from image features, suggesting that OTC organizes semantic and affective information in a manner suited to guide behavior.