Summary: The anterior cingulate cortex plays a central role in how the brain simulates the outcomes of different actions and selects the best choice.
Source: Zuckerman Institute
Our brains help us decide by imagining the future and predicting the consequences of our actions. For example, when you try to find a new restaurant near your home, your brain constructs a mental map of the neighborhood and plans the route to get there.
A new study in mice, published in Neuron, identifies the anterior cingulate cortex (ACC) as a crucial structure for using such mental models to guide learning and flexible decision making. The findings reveal detailed neural mechanisms that allow the brain to simulate likely outcomes of different actions and choose accordingly.
“The neurobiology of model-based learning is still poorly understood,” said Thomas Akam, PhD, a researcher at Oxford University and lead author of the study. “We were able to identify a brain region involved in this behavior and show that its activity encodes multiple aspects of the decision process.”
Understanding how the brain builds and uses mental models is essential for explaining flexible behavior—such as what you do when a familiar route to that new restaurant is blocked by construction and you must plan an alternative.
“These results were very exciting,” said senior author Rui Costa, DVM, PhD, Director and CEO of Columbia’s Zuckerman Institute, who initiated this work while at the Champalimaud Centre for the Unknown, where most of the experiments took place. “The data show that the anterior cingulate cortex is a key region for model-based decision making, specifically predicting what will happen if we choose one action over another.”
Model-based versus model-free learning
A major challenge in this field is separating model-based learning—where the brain builds explicit predictions of future states—from model-free learning, which relies on cached values from past rewards without simulating consequences. Model-free strategies are efficient and habitual: when you drive to a familiar restaurant, you may follow the habitual route without mentally simulating each turn.
To dissociate these approaches, the researchers designed a two-step decision task for mice that forces the animals to learn both the structure of the task and which actions lead to rewards. In each trial, a mouse first chooses one of two central holes. That choice probabilistically determines which of two side holes becomes available, and each side hole has its own probability of delivering a water reward.
“As in everyday life, subjects must perform extended action sequences with uncertain outcomes to obtain desired results,” Dr. Akam said. To solve the task optimally, mice must infer two things: which side hole is more likely to provide water, and which central choice tends to lead to that side hole. If they rely on model-based learning, they will predict the likely state transitions and choose actions that give the best expected outcome. If they use model-free learning, they will favor choices that have been rewarded in the past, regardless of the underlying state transitions.
To test flexibility, the researchers periodically changed the task structure: sometimes the side port with higher reward probability switched, and sometimes the mapping between central and side ports reversed. Analyzing roughly 230,000 individual choices, they found that mice employed both model-based and model-free strategies in parallel. This validated the task as a means to study the neural basis of these distinct learning systems.
How ACC encodes model-based information
The investigators then recorded activity from neurons in the anterior cingulate cortex while mice performed the task. Prior work implicated ACC in action selection and suggested it might contribute to model-based predictions, but activity of single neurons had not been examined in a task designed to distinguish the two learning modes.
They found a strong link between ACC neural activity and behavior. Patterns of activity across neuronal populations could predict which central hole the mouse was about to choose and whether the mouse received a reward. Beyond representing the animal’s current task state, ACC neurons encoded the state likely to follow a chosen action.
“This provides direct evidence that ACC predicts the specific consequences of actions, not just whether those actions are good or bad,” Dr. Akam explained. ACC neurons also signaled whether outcomes matched expectations or were surprising, a signal that could drive updates to the internal model when predictions fail.
To test causality, the team used optogenetic techniques to transiently silence ACC neurons during decision-making. When ACC activity was inhibited, mice lost the ability to adapt their choices after changes in state transitions—indicating impaired model-based behavior—while basic reward-driven behavior remained intact. This suggests ACC is necessary for using predicted state transitions to guide flexible decisions but not for simple reinforcement learning based on reward history alone.
These results offer a clearer neural map of model-based learning and position the anterior cingulate cortex as a central node in the circuitry that predicts future states and updates expectations based on surprising outcomes.
“Our study is among the first to demonstrate that complex elements of decision making, such as planning and sequential action selection, can be studied in mice at the single-cell level,” said Dr. Akam. “These findings create a foundation for mechanistic investigations into flexible decision making and how neural circuits implement model-based control.”
About this neuroscience research news
Source: Zuckerman Institute
Contact: Zuckerman Institute
Image: The image is credited to Thomas Akam / Rui Costa / Champalimaud Centre for the Unknown
Original Research: Open access. “The Anterior Cingulate Cortex Predicts Future States to Mediate Model-Based Action Selection” by Rui Costa et al. (Neuron). DOI: 10.1016/j.neuron.2020.10.013
Abstract
The Anterior Cingulate Cortex Predicts Future States to Mediate Model-Based Action Selection
Highlights
- • A novel two-step task separates model-based and model-free reinforcement learning in mice
- • ACC encodes the full task state space, and rewards are interpreted in the context of state
- • ACC predicts future states given chosen actions and signals state prediction surprise
- • Temporarily inhibiting ACC prevents state transitions—rather than rewards—from influencing choice
Summary
Behavioral control is not unitary; it arises from parallel systems that produce both flexible, model-based behavior and habitual, model-free behavior. Model-based decision making depends on predicting the specific consequences of actions, but the neural implementation of these predictions has been unclear. Using calcium imaging and optogenetics in a sequential decision task for mice, the authors demonstrate that the anterior cingulate cortex predicts the states that actions will lead to and monitors whether outcomes match those predictions. ACC represents the full task state space, with reward signals that depend strongly on the state in which reward occurs but show little dependence on the preceding choice. Consequently, ACC is required for updating model-based strategies but is not necessary for basic reward-driven reinforcement. These results identify ACC as a critical node in model-based control, with a specific role in predicting future states given chosen actions.