Summary: Researchers update a longstanding theory of human learning and emphasize its value as a guiding framework for building more capable artificial intelligence systems.
Source: Cell Press.
Recent advances that produced artificial agents able to outperform humans in complex games trace back to neural network ideas inspired by the brain. In a Review published June 14 in Trends in Cognitive Sciences, scientists from Google DeepMind and Stanford University revisit and expand a theory originally proposed to explain how humans and other mammals learn, and they highlight how this framework can help design better AI.
The complementary learning systems (CLS) theory, first articulated in 1995, posits that effective learning depends on two distinct but interacting learning systems. One system slowly extracts structured knowledge from repeated exposure to experience; the other rapidly encodes the details of individual episodes so they can be replayed and integrated into the slow system. This idea built on earlier insights from computational neuroscience and on contemporary advances in neural network methods.
“The evidence seems compelling that the brain has these two kinds of learning systems, and the complementary learning systems theory explains how they complement each other to provide a powerful solution to a key learning problem that faces the brain,” says James McClelland, Professor of Psychology at Stanford, lead author of the 1995 paper and senior author of the current Review.
In the biological implementation described by CLS theory, the slow-learning system is associated with the neocortex. This system resembles modern deep neural networks: multiple layers of processing units whose learned knowledge is encoded in connection weights and that gradually tune those connections through experience. Such structured neocortical representations support complex abilities such as object recognition, speech perception, language understanding and production, and decision-making based on acquired knowledge.
Those slow-learning networks face a core dilemma when confronted with new information. If connections are changed rapidly to incorporate new items, previously learned knowledge can be disrupted—a phenomenon sometimes called catastrophic interference. If learning is too slow, the system cannot take advantage of new experiences immediately. CLS theory resolves this tension by positing a second system that stores recent, detailed experiences without immediately overwriting consolidated knowledge.
In mammals, that fast-learning system corresponds to the hippocampus. By initially storing new experiences in hippocampal memory traces, the brain makes those experiences instantly available for behavior while preserving the stability of neocortical knowledge. Over time, the hippocampus can replay those stored episodes, interleaving them with existing cortical patterns so the cortex slowly integrates and reorganizes information into structured knowledge.
“That’s where the complementary learning system comes in,” McClelland explains. “By initially storing information about the new experience in the hippocampus, we make it available for immediate use and we also keep it around so that it can be replayed back to the cortex, interleaving it with ongoing experience and stored information from other relevant experiences.”
DeepMind cognitive neuroscientist Dharshan Kumaran, first author of the Review, notes that several neural network architectures that achieved human-level performance in video games were inspired by CLS concepts. Those systems use a memory buffer, analogous to the hippocampus, that records recent episodes of gameplay and replays them during training. This interleaving increases the effective use of experience data and prevents particular sequences of events from dominating learning.
Kumaran has collaborated with both James McClelland and DeepMind co-founder Demis Hassabis, who is also a co-author on the Review. Their joint work extends the role of the hippocampus within the original CLS formulation, examining how episodic replay, goal-directed prioritization of memories, and recurrent pattern activation contribute to generalization and memory-based reasoning.
“In my view,” Hassabis says, “the extended version of the complementary learning systems theory is likely to continue to provide a framework for future research, not only in neuroscience but also in the quest to develop artificial general intelligence, our goal at Google DeepMind.” The Review emphasizes that understanding biological learning mechanisms offers practical principles for machine learning systems, particularly when designing replay buffers, experience prioritization, and architectures that balance rapid adaptation with long-term stability.
Funding: The research cited in the Review received support from the New York State Stem Cell Science program and the Starr Foundation, with additional support in part from the National Institutes of Health and the National Cancer Institute.
Source: Joseph Caputo – Cell Press.
Image credit: Kumaran et al./Trends in Cognitive Sciences.
Original research: The Review “What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated” by Dharshan Kumaran, Demis Hassabis, and James L. McClelland was published online June 14, 2016 in Trends in Cognitive Sciences (doi:10.1016/j.tics.2016.05.004).
Abstract
What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated
This Review updates complementary learning systems (CLS) theory, which argues that intelligent agents benefit from two distinct learning systems, as embodied in mammals by the neocortex and hippocampus. The cortical system gradually acquires structured representations, while the hippocampal system rapidly stores individual experiences. The updated theory expands the role of hippocampal replay, showing how replay enables goal-dependent weighting of experience, supports interleaved integration into cortical representations, and can be modulated by reward or novelty. The authors address challenges to the original theory and extend it by describing how recurrent activation of hippocampal traces can support forms of generalization and how cortical learning can sometimes be rapid when new information aligns with known structure. Finally, the Review highlights the relevance of CLS principles to artificial intelligence, drawing connections between neuroscience findings and modern machine learning practices.
Key points:
- Discovering structure across many experiences relies on interleaved learning processes implemented in biological neocortex and in artificial neural networks.
- Once structured knowledge exists, new consistent information can be integrated rapidly into that structure.
- Both biological and artificial systems gain from a second, fast-learning mechanism that stores specific episodes—centred on the hippocampus in mammals.
- Replay of stored experiences supports interleaved learning and can be prioritized by reward or novelty to align learning with the agent’s goals.
- Recurrent activation within an instance-based memory system can reveal links among memories, supporting generalization and memory-based reasoning.