Summary: A recent study examined how people and the language model ChatGPT understand color metaphors, revealing important differences between experience-based human cognition and language-only AI. The results show that having typical color vision is not necessary to grasp color-based metaphors, but hands-on experience with color—such as that of painters—improves understanding of novel expressions. ChatGPT reliably reproduced common, culturally informed color associations yet struggled with unfamiliar or inverted metaphors, underscoring limits of text-only models in capturing embodied meaning.
Researchers tested four groups—people with normal color vision, colorblind participants, painters who work directly with pigments, and ChatGPT—on tasks that included assigning colors to abstract words, interpreting familiar metaphors (for example, “on red alert”), and judging novel phrases (for example, “a very pink party”). Participants were also asked to explain their reasoning. The study finds both convergence and divergence across groups, pointing to complex interactions between language exposure and real-world sensory experience.
Key findings:
- Color vision not required: Colorblind and color-seeing participants produced similar color associations, suggesting that visual perception is not strictly necessary to understand many color metaphors.
- Experience enhances nuance: Painters outperformed other human participants on novel metaphors, indicating that tactile and visual familiarity with pigments enriches conceptual reasoning about color.
- AI strengths and limits: ChatGPT produced consistent, culturally informed color mappings and often cited emotional or cultural connections when explaining choices, but it faltered on novel or inverted metaphors and offered fewer embodied explanations than people.
Color metaphors—phrases like “feeling blue” or “seeing red”—are frequent in English and therefore appear widely in text sources used to train large language models. ChatGPT leverages massive amounts of written language to learn statistical patterns and generate plausible answers. Still, unlike humans, the model has no sensory experience of seeing a blue sky or touching red paint. The researchers asked whether real-world sensory experience matters beyond the statistical regularities present in language.
The study, published in Cognitive Science and led by Professor Lisa Aziz-Zadeh at USC, assembled an interdisciplinary team from multiple universities and industry. Aziz-Zadeh, director of the USC Center for the Neuroscience of Embodied Cognition, studies how brain systems for perception, action, and emotion contribute to language and higher-level reasoning. The research was partially funded by a Google Faculty Gift and by campus fellowships; the funders did not influence the study design, analysis, or reporting.
How the experiments worked and what they revealed
Participants were given abstract words and asked to pick colors that best matched those concepts. They then evaluated both conventional metaphors (widely used expressions) and novel metaphors (creative or unusual color uses), and explained their choices in free text. Colorblind and color-seeing adults showed surprisingly similar patterns when mapping colors to abstract terms and novel words, suggesting that exposure to language and social-cultural context gives people robust tools for interpreting color-related language.
Painters showed distinct advantages on novel metaphors: their frequent, hands-on interaction with pigments appears to deepen conceptual links between color and meaning, enabling better interpretation when expressions depart from conventional usage. This supports the idea that embodied experience—direct sensory and motor engagement—contributes qualitatively to how people reason about color in language.
ChatGPT produced consistent associations and often justified choices by citing emotional or cultural connotations—explaining, for instance, that pink commonly evokes happiness, warmth, or friendliness when asked about “a very pink party.” Yet the model provided fewer embodied explanations than human participants and more often failed when asked to invert associations (for example, naming the opposite of a color association) or to interpret novel, unconventional metaphors (for example, “the meeting made him burgundy”).
These mixed outcomes highlight that statistical learning from text can replicate many human-like patterns, but may not capture the full range of embodied meanings people draw upon. Integrating multimodal sensory data into AI might help bridge some gaps, but the study emphasizes a persistent difference between mimicking semantic regularities and reasoning grounded in firsthand, sensorimotor experience.
Funding and authorship
The project was led by Lisa Aziz-Zadeh and involved researchers across institutions including UC San Diego, Stanford, Université de Montréal, the University of the West of England, and Google DeepMind. Funding sources included a Google Faculty Gift to Aziz-Zadeh, a Barbara and Gerson Bakar Faculty Fellowship, and support from the Haas School of Business at UC Berkeley. Google had no involvement in study design, data collection, analysis, or publication decisions.
About this AI and LLM research news
Author: Leigh Hopper
Source: USC
Contact: Leigh Hopper – USC
Image: Image credit: Neuroscience News
Original Research: Closed access. “Statistical or Embodied? Comparing Colorseeing, Colorblind, Painters, and Large Language Models in Their Processing of Color Metaphors” by Lisa Aziz-Zadeh et al., Cognitive Science. DOI and journal citation provided in the original publication.
Abstract
Statistical or Embodied? Comparing Colorseeing, Colorblind, Painters, and Large Language Models in Their Processing of Color Metaphors
This study asks whether metaphorical reasoning that involves sensory experiences—here, color perception—can be acquired from language alone. Prior work shows that people who are colorblind can reason about color metaphors, raising the possibility that linguistic exposure supplies much of the necessary information. To test this idea more directly, the researchers compared colorseeing adults, colorblind adults, painters, and large language models on tasks that required associating colors with abstract words and interpreting both conventional and novel color metaphors.
Colorblind and colorseeing adults produced highly similar and replicable color associations for abstract terms. A popular LLM also generated consistent associations, but its mappings often differed from those of human participants and it frequently failed to produce coherent explanations when prompted to invert associations or to interpret creative, novel metaphors. Painters, by contrast, were more likely to use embodied reasoning and to succeed on novel metaphor interpretation. The results suggest that embodied experience contributes importantly to conceptual connections for color and metaphorical reasoning beyond what language statistics alone can provide.