Summary: Simulations of spiking neural networks became unstable after prolonged unsupervised learning. Introducing activity patterns similar to biological slow-wave sleep restored stability in the models.
Source: DOE/Los Alamos National Laboratory
Artificial neural systems may need rest periods similar to sleep to remain stable, according to new research from Los Alamos National Laboratory.
Researchers at Los Alamos studied spiking neural networks—models that learn and communicate using brief, neuron-like spikes—because they more closely resemble biological brains than conventional deep learning systems. The team observed that networks undergoing extended unsupervised learning gradually drifted into unstable regimes. To address this, they tested a range of interventions and found that exposing the networks to patterns of activity analogous to slow-wave sleep in biological brains restored stable dynamics.
“We work with spiking neural networks, which learn in ways that mirror biological brains,” explained Los Alamos computer scientist Yijing Watkins. “Training neuromorphic processors in a manner similar to how humans and animals learn during development was a major motivation for this work.”
The instability problem emerged during unsupervised dictionary learning, a process where the network forms internal feature representations without labeled examples. During long stretches of this self-directed learning, the simulated cortical networks began to exhibit runaway activity and degraded representations, compromising their ability to classify and encode inputs reliably.
To stabilize the systems, the researchers experimented with different kinds of input perturbations. They compared various noise sources—random signals that can prevent the system from becoming locked into pathological states—and found that not all noise was equally effective. The most successful stabilizing input took the form of structured Gaussian noise that resembled the broad-spectrum, slow fluctuations observed during biological slow-wave sleep. When applied periodically, these sleep-like activity patterns returned the networks to a balanced operating point where meaningful feature learning resumed.
“It was as if we were giving the neural networks the equivalent of a good night’s rest,” Watkins said. The results indicate that slow-wave-like activity can act as a homeostatic mechanism for maintaining neural stability, preventing the kinds of spurious or hallucinatory activity that can arise when plasticity runs unchecked.
Garrett Kenyon, a coauthor on the study and also a computer scientist at Los Alamos, emphasized that this stability challenge is specific to biologically realistic, spiking neuromorphic processors or to investigations aimed at understanding biological learning. “Most mainstream machine learning and deep learning research does not encounter this issue because those artificial systems rely on global numerical operations that implicitly regulate network activity,” Kenyon said. By contrast, spiking networks and neuromorphic hardware lack such global normalization unless it is implemented through biologically inspired mechanisms.
The team describes their decision to trial sleep-like input as almost a last-resort approach after more conventional noise injections produced limited benefit. They drew an analogy between the Gaussian, broadband noise they applied and the slow oscillatory input the cortex receives during deep sleep. Their working hypothesis is that slow-wave activity helps biological cortical neurons maintain synaptic balance and prevents runaway excitation that could lead to hallucinations or other dysfunctional states. In the simulated networks, the slow-wave-like noise effectively reset synaptic dynamics and restored stable learning.
Looking ahead, the group plans to implement the sleep-restoration algorithm on Intel’s Loihi neuromorphic chip. Their objective is to determine whether periodic, sleep-like interventions will permit Loihi to process sensory streams—such as inputs from a silicon retina camera—in stable, real-time operation. Demonstrating sleep-like requirements on physical neuromorphic hardware would strengthen the case that future intelligent machines and androids may benefit from scheduled downtime that mimics biological sleep.
Watkins will present these findings at the Women in Computer Vision Workshop on June 14 in Seattle, where the team will discuss experimental details, network architectures, and the implications for neuromorphic engineering and biologically inspired machine learning.
About this machine learning research article
Source:
DOE/Los Alamos National Laboratory
Media Contacts:
James Riordon – DOE/Los Alamos National Laboratory
Image Source:
The image is credited to Los Alamos National Laboratory.
Original Research: The study will be presented at the Women in Computer Vision Workshop.
Sharing: This summary is intended to inform readers about recent advances in neuromorphic computing, spiking neural networks, and the potential role of sleep-like processes in maintaining stable, long-term learning in artificial brains.