Summary: A new study shows that learning in neural networks is driven mainly by adjusting the strengths of existing connections rather than by continuously expanding or reorganizing the underlying network structure.
Researchers trained artificial neural networks on language tasks while gradually increasing the amount of training data. Although model performance improved substantially with more data, the trained networks retained the ability to lose a large fraction of their synaptic connections without noticeable loss of function.
These results point to a shared computational principle across artificial and biological systems: effective learning and functional optimization arise from refining synaptic weights and improving cooperation among existing components rather than from ongoing topological expansion.
Key Facts
- Weight modification rather than topological growth: Experimental evidence indicates that performance gains come from changing internal connection strengths (synaptic weights) instead of adding new nodes or substantially reconfiguring network topology.
- Robustness to pruning: Trained networks tolerate removal of a significant proportion of synapses without measurable performance decline, demonstrating high redundancy and efficiency in their learned representations.
- Consistency with biology: In adult biological brains, neuron numbers remain largely constant, and there is no established mechanism for continuous, large-scale rewiring of global network architecture during normal learning.
- Convergence of artificial and biological intelligence: The work supports the idea that both brains and artificial systems achieve intelligence primarily by optimizing interactions among preexisting components.
- Implications for efficiency: Showing that architectural expansion is unnecessary for complex learning encourages development of more compact, energy-efficient AI designs and offers a physical perspective on biological cognitive reserve.
Source: Bar-Ilan University
How does the brain learn? Does it gain new abilities by creating new neural pathways, or by strengthening the connections it already has?
A study from Bar‑Ilan University provides evidence favoring the latter: learning appears to depend mainly on adjusting the strengths of existing neural connections instead of expanding the architecture of the network.
Published in Physica A, the paper by Prof. Ido Kanter and colleagues at Bar‑Ilan University’s Department of Physics and the Gonda Multidisciplinary Brain Research Center examined this question using artificial neural networks trained on language-processing tasks.
As researchers increased the size of the training datasets, the models’ ability to learn improved. Remarkably, these improved models could still lose a comparable fraction of their synaptic connections without any meaningful drop in performance. This indicates that better learning did not require creating more complex network structures. Instead, it resulted from more effective coordination among the network’s fixed set of components.
In practical terms, the study finds that learning is achieved primarily through changes in synaptic weights—how strongly units are connected—rather than by continuous rewiring or enlargement of the network.
“This result is especially notable when considering biological brains, where the number of neurons remains roughly constant,” said Yanir Harel, M.Sc., the study’s first author. “Currently, there is no evidence for a biological mechanism that would support large-scale, continuous reconfiguration of brain-wide network topology during typical learning.”
Collectively, the findings suggest a unifying principle: intelligence may emerge less from adding components and more from improving coordination among the components already present. Both human brains and modern AI systems might owe much of their learning power to continuous refinement of internal connections rather than to ever-expanding architectures.
Key Questions Answered:
A: They trained artificial neural networks on language tasks while increasing available data, then evaluated model performance after removing varying fractions of synaptic connections. The models improved with more data but remained resilient to heavy pruning, demonstrating that gains came from refined weights rather than added structural complexity.
A: In mature human brains the total number of neurons is essentially stable, and there is no established mechanism for continuous, large-scale structural rewiring across the whole brain during ordinary learning. The study supports the biological view that learning predominantly relies on synaptic plasticity—changes in connection strength among existing neurons.
A: Rather than endlessly increasing model size and parameter counts, AI research can prioritize optimizing weight interactions and intelligent pruning of unnecessary connections. This strategy could yield more compact, energy-efficient models that retain high performance.
Editorial Notes:
- This article was edited by a Neuroscience News editor.
- The journal paper was reviewed in full.
- Additional explanatory context was added by staff.
About this learning and neuroscience research news
Author: Elana Oberlander
Source: Bar‑Ilan University
Contact: Elana Oberlander – Bar‑Ilan University
Image: Image credited to Neuroscience News
Original Research: Open access. “NLP models: Capacity per weight—capacity per label” by Yanir Harel, Tal Halevi, Ronit D. Gross, Shalom Rosner, and Ido Kanter. Physica A: Statistical Mechanics and its Applications. DOI: 10.1016/j.physa.2026.131855
Abstract
NLP models: Capacity per weight—capacity per label
The study investigates the maximal capacity per weight (CpW) for feedforward neural networks and its relevance to natural language processing (NLP) tasks. Simulations indicate that CpW is finite and approaches the one-step replica symmetry-breaking solution, remaining effectively independent of the number of hidden layers. However, CpW declines as a power law with the number of output units, which raises doubts about its usefulness for characterizing NLP complexity.
Direct simulations of pre-training for deep NLP architectures produced two notable observations. First, deep networks can undergo substantial pruning without harming accuracy per token (APT). Second, the maximal level of pruning that preserves APT does not depend on the size of the training dataset (when datasets contain similar token counts), indicating that CpW may not capture task complexity for NLP. Similar patterns were observed for fine-tuning on classification problems.
To address this shortcoming, the authors propose an alternative measure, capacity per label (CpL), and support its relevance through a toy model built from archetype input vectors. In that model, each archetype defines a class for all noisy patterns generated from it. Simulations show that the maximal dilution preserving accuracy decreases with the number of output labels, yet remains independent of training dataset size and largely independent of the number of hidden layers.
The results include experiments performed on BERT-style architectures pre-trained on small subsets of Wikipedia and subsequently fine-tuned on the FewRel classification task, supporting the broader conclusion that improved performance arises from refining weights and cooperation among existing units rather than structural expansion of the network.