Summary: Serotonin neurons in the brainstem do not operate in isolation as once believed. New research shows these neurons form interacting networks that compete and cooperate to shape when and where serotonin is released across the brain, influencing decision-making and responses to threat.
This finding challenges the long-standing notion of a uniform serotonin signal and has important implications for understanding mood disorders and the neural mechanisms behind binary choices. The team mapped a circuit involving the lateral habenula that appears to help compute “go” versus “don’t go” decisions when animals evaluate potential threats.
Key Facts:
- Serotonin networks: Serotonin neurons form interconnected ensembles that regulate dynamic, region-specific serotonin release.
- Decision circuits: A newly identified pathway links lateral habenula activity to raphe serotonin ensembles, shaping binary behavioral outcomes.
- Behavioral impact: This refined view of serotonergic function may guide development of targeted treatments for mood disorders such as depression.
Source: University of Ottawa
Everyday decisions often come down to a simple choice between two options. But how does the brain compute these binary outcomes?
A study led by the University of Ottawa Faculty of Medicine and published in Nature Neuroscience offers new insight into this question by examining the midbrain origin of the central serotonin (5-HT) system. Serotonin influences a broad range of cognitive and behavioral functions, yet the local circuitry that organizes its release has remained unclear.
“The prevailing model held that individual 5-HT neurons act independently. While prior work suggested these neurons might be connected, direct evidence was lacking. Our experiments demonstrate that 5-HT neurons form recurrent, functionally meaningful connections,” says Dr. Jean-Claude Béïque, full professor in the Department of Cellular and Molecular Medicine and co-director of the uOttawa Brain and Mind Research Institute’s Centre for Neural Dynamics and Artificial Intelligence.
The international team combined electrophysiology, cellular imaging, optogenetics, behavioral testing, mathematical modeling and computer simulations to reveal how these circuits operate.
Forging advances
What does it mean that serotonin neurons clustered in the dorsal raphe are interconnected rather than independent? The researchers found distinct groups of 5-HT neurons—ensembles—with their own activity patterns, each controlling serotonin release to particular brain regions. These ensembles can interact: highly active “winning” ensembles suppress serotonin release from less active “losing” ensembles.
“This network implements a nonlinear, competitive dynamic—akin to a winner-take-all process—but with extra complexity. The interactions are slow, stochastic and strongly facilitating, which creates flexible rules for when and where serotonin is released,” explains Dr. Michael Lynn, the study’s first author and a former member of Dr. Béïque’s lab. Dr. Lynn completed his PhD in Neuroscience at the University of Ottawa and is now a postdoctoral fellow at the University of Oxford.
These findings revise the idea of a uniform serotonergic signal and suggest a richer anatomical and functional organization that could inform targeted interventions for disorders linked to serotonin dysfunction, including major depressive disorder.
Decisions, decisions
The team also traced how the lateral habenula—a brain region activated by negative outcomes and implicated in depression—modulates raphe serotonin ensembles. Habenular neurons are thought to encode perceived threat levels in the environment or as a consequence of actions.
“Consider everyday choices: do I jump from a high diving board or the low one? Do I walk down a dark alley or turn back? The brain must evaluate threat features of the environment and produce a binary output: go or don’t go,” says Dr. Béïque. The study identifies a circuit that appears to perform this computation by gating serotonergic ensemble activity, thereby influencing action selection when threat evaluation is required.
Next steps
Moving forward, the team plans to extend these discoveries into more naturalistic behavioral assays using mouse models. “So far, the behavioral readouts for the computation were somewhat artificial. We are now testing whether similar network dynamics emerge when animals navigate more realistic environments,” Dr. Béïque says.
The multidisciplinary team includes computational neuroscientist Dr. Richard Naud and Sean Geddes, director of Innovation and Partnerships at Ottawa, among others who contributed to the Nature Neuroscience paper.
About this serotonin and decision-making research news
Author: Paul Logothetis
Source: University of Ottawa
Contact: Paul Logothetis – University of Ottawa
Image: Image credited to Neuroscience News
Original Research: Open access. “Nonlinear recurrent inhibition through facilitating serotonin release in the raphe” by Jean-Claude Béïque et al., Nature Neuroscience. DOI: 10.1038/s41593-025-01912-7
Abstract
Nonlinear recurrent inhibition through facilitating serotonin release in the raphe
Serotonin (5-HT) neurons in the dorsal raphe nucleus (DRN) receive a variety of long-range inputs, but the local circuit organization and computational rules of this nucleus have been unclear. Using lateral habenula inputs to probe DRN processing, the authors uncovered 5-HT1A receptor-mediated recurrent connections between 5-HT neurons, challenging classical autoinhibition models.
Cellular electrophysiology and imaging with a genetically encoded serotonin sensor showed that these recurrent inhibitory connections span the raphe, are slow and stochastic, strongly facilitate with repeated activation, and gate spike output. These properties confer highly nonlinear dynamics on the network, producing excitation-driven inhibition and winner-take-all computations.
In vivo optogenetic activation of lateral habenula inputs to the DRN at frequencies predicted to engage these computations transiently disrupted a learned reward-conditioned response in an auditory conditioning task. Together, these data reveal a core computation supported by a previously unrecognized slow serotonergic recurrent inhibitory network.