Summary: A new study from Keio University shows that the “sight-over-sound” effect—where visual information can outweigh audio when people judge musical performances—depends strongly on the listener’s musical background. Using recordings from Japanese high school brass band competitions, researchers found that general musicians often relied more on visual cues, while brass band specialists evaluated performances more accurately by sound alone. Non-musicians showed no clear visual or auditory bias. These results clarify how expertise shapes auditory-visual integration and have practical implications for music education and competition judging.
People commonly assume that sound is the primary basis for evaluating music; however, visual elements such as stage presence and body language can influence judgments in subtle but powerful ways. This phenomenon, known as the sight-over-sound effect, has produced mixed findings across prior studies—partly because earlier experiments used varied camera angles, different musical pieces, and inconsistent definitions of musical experience among evaluators.
To address prior limitations, the research team led by Associate Professor Shinya Fujii of Keio University’s Neurosciences and Music Laboratory designed a tightly controlled study. Co-author Tomohiro Samma and colleagues used video and audio recordings of award-winning ensembles from Japanese high school brass band competitions, ensuring all selected bands had received gold awards to minimize performance-quality differences. Each comparison set presented the same musical piece filmed from consistent camera angles to reduce visual variability.
The experiment tested 301 adult participants divided into three groups by musical background: 171 brass band musicians (BMs) with direct experience in brass ensembles, 78 non-brass band musicians (NBMs) with broader musical training but no brass band experience, and 52 non-musicians (NMs) with no formal musical training. Participants evaluated performances in three conditions—audio-only, visual-only, and audio-visual—and selected the band they thought would perform best in competition.
When the entire participant pool was analyzed, the study did not find a significant overall sight-over-sound effect. This indicates that careful control of video angles, repertoire, and performance quality can reduce visual dominance in musical evaluation. However, subgroup analyses revealed that the effect depends on the evaluator’s experience.
- Visual bias in non-brass musicians: NBMs showed a clear sight-over-sound effect, judging winners more accurately from visual-only information than from audio alone.
- Auditory expertise in brass musicians: BMs showed no visual dominance; in fact, they were more accurate in audio-only assessments, suggesting that genre-specific auditory training helps them prioritize sound over visuals.
- No consistent bias in non-musicians: NMs did not display a reliable sight-over-sound effect, implying that lack of specialized training can lead to inconsistent use of visual cues.
These results emphasize that sensory weighting in music evaluation is not uniform: specialized training alters how listeners integrate visual and auditory information. As Mr. Samma explains, the sight-over-sound effect appeared only in non-brass musicians, highlighting the importance of specific musical experience in shaping evaluative strategies. Dr. Fujii adds that the findings speak to broader issues in social psychology and cognitive science: domain expertise can change the relative importance people assign to different sensory inputs.
Practically, the study suggests several actionable points for music educators, performers, and competition organizers. Judges and evaluators with varying backgrounds may weigh audio and visual cues differently, so awareness and training are important to promote fairness. For researchers, the absence of a universal sight-over-sound effect in the full sample underscores the need for rigorous stimulus control when studying multisensory influences.
Published in PLOS One on April 29, 2025, the study—co-authored by Tomohiro Samma and led by Shinya Fujii—provides a clearer picture of when and for whom visual cues dominate musical judgments. Future research can build on these findings to refine evaluation protocols in musical contexts and to deepen our understanding of how sensory integration develops with training.
About this music, auditory, and visual neuroscience research news
Author: Tomohiro Samma
Source: KEIO University
Contact: Tomohiro Samma – KEIO University
Image: The image is credited to Neuroscience News
Original Research: Open access.
“Sight-over-sound effect depends on interaction between evaluators’ musical experience and auditory-visual integration: An examination using Japanese brass band competition recordings” by Tomohiro Samma et al. PLOS One
Abstract
Sight-over-sound effect depends on interaction between evaluators’ musical experience and auditory-visual integration: An examination using Japanese brass band competition recordings
The sight-over-sound effect—where visual information can override auditory cues during music evaluation—challenges the assumption that sound alone determines perceived musical quality. Yet its reliability across genres and contexts has been disputed. This study examined the phenomenon using carefully controlled recordings from Japanese brass band competitions.
A total of 301 adults (mean age 34.81 ± 11.71 years) were grouped by experience: 171 brass band musicians (mean age 33.08 ± 11.57), 78 non-brass band musicians (mean age 35.39 ± 13.03), and 52 non-musicians (mean age 39.15 ± 8.73). Participants judged which band would win under audio-only, visual-only, and audio-visual conditions.
Results showed the sight-over-sound effect in non-brass band musicians (Kruskal–Wallis test: p < 0.001) but not in brass band musicians (p = 0.48) or non-musicians (p = 0.37). These findings indicate that specific musical training—particularly auditory training in brass band musicians—can reduce visual influence, while non-musicians do not consistently favor visual information. The study advances our understanding of multisensory integration and suggests that educational and evaluative practices should account for how expertise shapes sensory priorities.