Summary: Researchers report that when the brain decodes a sentence in English or Portuguese, the neural activation patterns are essentially the same.

Same Neural Signatures for Sentence Meaning Across English and Portuguese

An international research team led by Carnegie Mellon University has demonstrated that the brain represents the meaning of sentences in English and Portuguese using the same neural patterns. Published in NeuroImage, the study is the first to show a cross-language commonality in neural representations for describing everyday events and scenes.

The researchers combined functional magnetic resonance imaging (fMRI) with a machine-learning model to link sentence semantics to brain activation. They trained a computational model on English-language data and then tested whether that English-based model could predict brain activation for sentences read in Portuguese. The model successfully recognized Portuguese sentence meaning from brain activation patterns, supporting the idea of a shared neural basis for sentence comprehension across these two languages.

Study design and computational model

Fifteen native Portuguese speakers took part in the experiment; eight were bilingual in Portuguese and English. While in an fMRI scanner, each participant read 60 Portuguese sentences. The Carnegie Mellon computational model—based on earlier work with English speakers—relies on a set of 42 concept-level semantic features (Neurally Plausible Semantic Features) and six markers that capture thematic roles within a sentence, such as agent or action.

Using the mapping from semantic features to voxel activation learned from English data, combined with brain-location clusters identified from factor analysis of English-language activation, the model predicted activation for individual Portuguese sentences. The model achieved 67 percent accuracy in predicting which of the 60 Portuguese sentences a person was reading, substantially above chance.

Shared organization of semantic information

Analysis of the brain activation patterns revealed that sentence meaning is organized in similar cortical locations and with similar intensity profiles across languages. The activation patterns clustered into four semantic categories that reflected the focus of each sentence: people, places, actions and feelings. These clusters were highly consistent across English and Portuguese, reinforcing the conclusion that semantic information is represented in common neural locations irrespective of the language used to express it.

Quotations and implications

“This tells us that, for the most part, the language we happen to learn to speak does not change the organization of the brain,” said Marcel Just, D.O. Hebb University Professor of Psychology at Carnegie Mellon and a pioneer in using brain imaging and machine learning to decode thought. He noted that semantic information appears in the same brain regions and similar intensity patterns for different speakers, suggesting that brain-to-brain or brain-to-computer interfaces could be designed to work across language communities.

Ying Yang, a postdoctoral associate in psychology at Carnegie Mellon and the study’s first author, added: “The cross-language prediction model captured the conceptual gist of the events or states described in the sentences, rather than relying on language-specific features. It demonstrated a meta-language prediction capability from neural signals across people, languages and bilingual status.”

Broader significance

These results have practical implications for several fields. A cross-language understanding of neural sentence representation could improve machine translation, enable brain decoding tools that work across languages, and inform approaches to second-language instruction. The study also contributes to a broader research agenda at Carnegie Mellon that leverages strengths in neuroscience, computer science and psychology to understand how brain structure and activity give rise to complex cognitive behaviors.

Funding, source and publication

Funding: The research was funded by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via the U.S. Air Force Research Laboratory (AFRL).

Source: Carnegie Mellon University.

Original research: “Commonality of neural representations of sentences across languages: Predicting brain activation during Portuguese sentence comprehension using an English-based model of brain function” by Ying Yang, Jing Wang, Cyntia Bailer, Vladimir Cherkassky and Marcel Adam Just, published in NeuroImage. DOI: 10.1016/j.neuroimage.2016.10.029.

Abstract summary

The study tested the cross-language generalizability of a predictive model that maps sentence meaning onto neural activation patterns. The model uses a set of 42 semantic features plus six thematic role markers to predict voxel-level activations. It transferred two types of information from English-based data: the mapping weights from semantic features to voxel activations, and the brain locations identified by factor analysis as representing core semantic dimensions. These meta-language locations corresponded to four main dimensions—people, places, actions and feelings. The cross-language model reliably predicted activation for novel Portuguese sentences, with performance above chance and unaffected by participants’ bilingual status. Error patterns indicated that the model captured conceptual gist rather than language-specific idiosyncrasies, supporting a shared neural representation of everyday events and states across languages.

Feel free to share this research summary.