Summary: A new machine learning model reveals that serotonin can accelerate learning rates in the brain.
Source: Sainsbury Wellcome Centre.
UCL researchers using data from the Champalimaud Centre for the Unknown have developed a computational model showing that serotonin, a widespread neuromodulator, can speed up learning.
Serotonin is a key chemical messenger in the brain implicated in many aspects of cognition and decision-making. It is a primary target of selective serotonin reuptake inhibitors (SSRIs), which are commonly prescribed for psychiatric conditions such as depression, obsessive-compulsive disorder and anxiety. Despite its clinical importance, serotonin’s computational role in learning and choice has remained difficult to pin down because it is involved in diverse processes including reward processing, punishment sensitivity and patience.
New findings published in Nature Communications bring clarity to one of serotonin’s functions. Kiyohito Iigaya and Peter Dayan, based at the Gatsby Computational Neuroscience Unit and the Max Planck–UCL Centre for Computational Psychiatry and Ageing Research, analyzed behavioural and neural data collected by collaborators Madalena Fonseca, Masayoshi Murakami and Zachary Mainen at the Champalimaud Centre for the Unknown in Portugal.
In the experimental task, mice chose repeatedly between two targets to receive water rewards. The environment was non-stationary: reward probabilities changed unpredictably, so animals had to continually update which option was better. The team used optogenetics to transiently increase serotonin neuron activity on some trials in genetically modified mice, allowing the researchers to directly assess the effect of serotonin release on learning.
Iigaya implemented a computational account of the mice’s behaviour based on reinforcement learning (RL), a framework widely used in machine learning and artificial intelligence to model how agents learn from reward feedback. The key quantity in these models is the learning rate, which determines how rapidly outcomes change future choices. Comparing trials with and without serotonin stimulation, the model showed that optogenetic boosts in serotonin significantly increased the learning rate: mice updated their value estimates faster when serotonin neurons were stimulated.
Crucially, the study revealed two distinct decision systems operating in parallel. On trials where choices occurred in quick succession, animals typically used a simple “win-stay, lose-switch” heuristic—repeating a choice after a reward and switching after no reward. Serotonin stimulation had little or no effect on these rapid, heuristic decisions. By contrast, when animals waited longer between trials, their choice behaviour reflected integration of outcomes over many past trials and was well described by the reinforcement learning model. Serotonin stimulation selectively boosted the learning process in this slower, integrative system.
Further analysis showed that the slow reinforcement-learning system continued to track outcomes even when the animal acted quickly using the win-stay strategy. This means serotonergic effects on the slow system could be hidden or “masked” by the fast decision mechanism and therefore only become apparent on trials with long intertrial intervals. The coexistence of these decision systems helps explain why previous studies have produced mixed or puzzling findings about serotonin’s role in learning and choice.
The authors summarize their findings by proposing that serotonin boosts neural plasticity by modulating learning rates. This view aligns with clinical observations that combining SSRIs with cognitive behavioural therapy (CBT) often enhances treatment outcomes: SSRIs may increase the brain’s capacity to relearn patterns of thought and behaviour promoted during therapy, making CBT more effective.
Clinical research has long shown that SSRIs and CBT can work synergistically, but the mechanisms behind their interaction have been unclear. By demonstrating that serotonergic stimulation selectively increases the pace of reinforcement learning in a slow decision system, this study provides a plausible mechanistic link between pharmacological treatment and behavioural therapy. Serotonin’s role in accelerating the updating of value estimates could help patients adopt new, healthier habits more readily during therapy.
Funding: The research received support from the Gatsby Charitable Foundation, the Joint Initiative on Computational Psychiatry and Ageing Research between the Max Planck Society and UCL, the Japan Society for the Promotion of Science, the European Research Council (grants 250334 and 671251), Fundação para a Ciência e a Tecnologia, and the Champalimaud Foundation.
Source and Publisher: April Cashin-Garbutt, Sainsbury Wellcome Centre. Published on NeuroscienceNews.
Original research: Iigaya, K., Fonseca, M. S., Murakami, M., Mainen, Z. F., & Dayan, P. “An effect of serotonergic stimulation on learning rates for rewards apparent after long intertrial intervals.” Nature Communications, published June 26, 2018.
Abstract
An effect of serotonergic stimulation on learning rates for rewards apparent after long intertrial intervals
Serotonin exerts broad modulatory effects on learning and cognition that are computationally unclear. This study examined how optogenetic stimulation of dorsal raphe serotonin neurons affects decision-making in mice performing a changing reward task. Two choice strategies emerged: after short intertrial intervals (ITIs), choices followed a win-stay-lose-switch rule based solely on the last outcome; after long ITIs, choices reflected reward history across multiple trials and matched reinforcement learning models. Optogenetic stimulation increased the learning rate associated with trial outcomes, but this effect appeared only on choices following long ITIs. These results suggest serotonin modulates reinforcement learning rates and that its influence can be masked by alternate, fast decision mechanisms. The findings offer insight into serotonin’s role in neural plasticity and its therapeutic relevance for psychiatric disorders.