How Social Brain Neurons Decide Dominance and Submission

Summary: Scientists have identified specific brain cells that determine how animals respond to social defeat, providing new insight into the biology of dominance and submission. In male mice, cholinergic interneurons in the dorsomedial striatum were shown to control the “loser effect,” where past defeats reduce an animal’s future social rank.

When these neurons were selectively removed, mice no longer showed submissive behaviors after losing contests, while their tendency to capitalize on wins remained. The study demonstrates that separate neural circuits underlie winning and losing, offering clues about how social status and flexible behavior are encoded in the brain.

Key Facts:

  • Neural basis of losing: Lesioning cholinergic interneurons in the dorsomedial striatum disrupted the “loser effect” in male mice.
  • Separate circuits for outcomes: Different neural pathways appear to drive dominance gained from victories and submission following defeats.
  • Relevance to humans: Because the involved brain structures are conserved across mammals, these results may help explain aspects of human social behavior and adaptability.

Source: OIST

Social hierarchies are ubiquitous — from school cliques where athletes are often shown as popular, to corporate structures where executives make major decisions. These patterns occur across many species: dominant animals typically eat first, mate more, and occupy the best territories. Although experience—winning or losing—has long been known to influence an individual’s rank, the neural mechanisms that translate past outcomes into social strategy have been unclear.

In a study published in iScience, researchers at the Okinawa Institute of Science and Technology (OIST) explored how specific neurons shape social hierarchy dynamics in male mice. Their experiments reveal which cells contribute to changes in dominance after competitions and how loss and victory are processed differently in the brain.

“Dominance is not determined solely by physical traits like size,” said Professor Jeffery Wickens, head of OIST’s Neurobiology Research Unit and co-author of the study. “Our data suggest past experience strongly influences whether an animal takes a dominant or submissive role, and the underlying circuitry is conserved between mice and humans.”

How social rank was assessed

The team used the dominance tube test to map social order within groups. In this test, two mice enter a narrow tube from opposite ends; the mouse that forces the other to retreat is scored as dominant. Repeating these encounters over days allowed the researchers to identify consistently dominant and subordinate animals within each cage.

The experiment then paired animals from different cages: dominant mice faced other dominants, and subordinates faced subordinates. Results from these cross-cage contests altered the mice’s positions in their home-cage hierarchies depending on whether they won or lost — demonstrating clear winner and loser effects.

“Winning tends to increase confidence and future dominance, while losing tends to reduce it,” explained Dr. Mao-Ting Hsu, lead author. “We found that the loser effect in particular depends on specific neurons in the dorsomedial striatum known as cholinergic interneurons.”

The neuroscience behind the loser effect

The basal ganglia, a brain region often discussed in relation to movement disorders like Parkinson’s disease, also plays a key role in behavioral flexibility — the ability to change actions based on context. Within the basal ganglia, the dorsomedial striatum contains cholinergic interneurons implicated in adjusting decisions when situations change.

To probe their function, researchers selectively lesioned these cholinergic interneurons and retested social dominance. Mice lacking these cells failed to display the usual drop in social rank after defeat: the loser effect was diminished. Importantly, their winner effect — the tendency for winners to become more dominant — remained intact, indicating that winning and losing are mediated by distinct neural mechanisms.

These findings suggest the winner effect may depend more on reward-based learning processes, while the loser effect likely involves context-dependent decision-making and changes in synaptic plasticity governed by cholinergic signaling.

Implications for human social behavior

Although this work was performed in male mice, it offers a framework for thinking about human social dynamics. Social rank in people is situation-dependent: an individual may hold authority at work but be less influential elsewhere. The circuits identified here point to biological mechanisms that could underlie such flexibility.

“Human social interactions are more complex, but the conserved brain architecture means studies in mice can provide valuable clues,” Dr. Hsu noted. “Understanding how experience reshapes decision-making circuits could inform how confidence, dominance, and adaptability emerge across contexts.”

Key Questions Answered:

Q: What did the study discover about the neuroscience of social hierarchies?

A: It identified cholinergic interneurons in the dorsomedial striatum as key regulators of how losing affects subsequent social standing, revealing a neural basis for the “loser effect.”

Q: How do winning and losing differ in the brain?

A: The evidence suggests the winner effect is linked to reward-based learning mechanisms, while the loser effect involves flexible decision-making processes and context-dependent neural adjustments.

Q: Why might this matter for humans?

A: Because the relevant brain structures are similar in mice and humans, these results may help explain how prior experience influences confidence, dominance, and adaptability across different social situations.

About this social neuroscience research news

Author: Tomomi Okubo
Source: OIST
Contact: Tomomi Okubo – OIST
Image: Image credit: Neuroscience News

Original Research: Open access. “Cholinergic interneurons of the dorsomedial striatum mediate winner-loser effects on social hierarchy dynamics in male mice” by Jeff Wickens et al., published in iScience.


Abstract

Cholinergic interneurons of the dorsomedial striatum mediate winner-loser effects on social hierarchy dynamics in male mice

Forming a new social hierarchy requires flexible choices about whether to behave dominantly or submissively in competitive encounters. The neural circuits that support this flexibility are not fully defined, though prior work implicates cholinergic interneurons in the dorsomedial striatum.

Using a series of dominance tube tests, the study showed that dominant mice defeated by similarly ranked animals from other cages subsequently dropped in rank within their home groups, while initially subordinate mice that won increased in rank. Selective lesions of dorsomedial striatal cholinergic interneurons reduced the loser effect in dominant mice.

The authors propose that losing alters the activity of these interneurons and thereby modulates synaptic plasticity in circuits responsible for flexible decision-making and reinforcement, producing changes in social behavior after defeat.