Summary: Researchers at Kobe University have developed an AI algorithm that predicts whether a mouse is moving or resting by analyzing whole-cortex functional imaging data. The end-to-end deep learning approach reaches approximately 95% accuracy and operates without conventional preprocessing steps. Remarkably, the system can make reliable predictions from as little as 0.17 seconds of imaging data, a capability that could accelerate the development of near real-time, non-invasive brain-machine interfaces.
In addition to high-speed classification, the team created a method to determine which parts of the imaging data were most influential in each prediction. This technique improves interpretability by giving researchers a clearer view of the neural signals that drive the AI’s decisions, helping to reduce the “black box” problem typical of deep learning models in neuroscience.
Key Facts:
- High prediction accuracy: The AI model classifies behavioral state—moving or resting—based on cortex-wide imaging with about 95% accuracy, without the need for noise removal or pre-defined regions of interest.
- Fast, individualized predictions: The model can make accurate decisions using only 0.17 seconds of imaging data and maintains performance across multiple individual mice, demonstrating robustness to individual variation and potential for personalized, near real-time applications.
- Improved interpretability: By systematically removing portions of the input data and measuring the impact on performance, the researchers identified cortical regions most critical for behavior classification, offering insight into what the neural network uses to make predictions.
Source: Kobe University
An AI image-recognition algorithm can predict whether a mouse is in motion by analyzing cortex-wide calcium imaging. The researchers also developed a method to identify the specific input data driving each prediction, increasing transparency into the AI’s decision process and advancing tools for brain-machine interface research.
Understanding how patterns of brain activity relate to actions is a central requirement for building functional brain-machine interfaces. This process, known as neural decoding, has traditionally relied on electrical recordings from electrodes implanted in brain tissue. Functional imaging methods, such as fMRI and calcium imaging, can monitor activity across the entire brain, but their adoption for neural decoding has been limited by challenges in data preprocessing and generalization.
Calcium imaging offers faster temporal resolution and finer spatial detail than many other functional imaging techniques, making it an attractive but underused source for decoding behavior. Conventional approaches often require noise reduction, region-of-interest selection, or other preprocessing, which can make it hard to develop generalizable decoders for diverse behaviors.
To address these limitations, Kobe University medical student Ajioka Takehiro worked with neuroscientist Takumi Toru and an interdisciplinary team to explore an end-to-end deep learning strategy that skips manual preprocessing. “Our experience with VR-based real-time imaging, motion tracking for mice, and deep learning techniques led us to pursue end-to-end models that learn directly from raw cortex-wide data without hand-crafted features,” Ajioka explains.
The team combined convolutional neural networks (CNNs) to extract spatial patterns with recurrent neural networks (RNNs) to capture temporal dynamics. They trained the CNN–RNN model on whole-cortex calcium imaging obtained while mice rested or ran on a treadmill. The trained model achieved near-perfect classification accuracy—around 95%—in predicting true behavioral state without denoising or preselecting regions of interest.
Equally important, the model achieved this performance on sub-second time scales: 0.17 seconds of imaging data was sufficient for accurate classification, enabling near real-time operation. The decoder also generalized across five different animals, showing robustness against individual differences and supporting its potential for broader applications.
To make model decisions interpretable, the researchers developed an input-perturbation analysis: they removed parts of the image data and observed how the model’s accuracy changed. Large drops in performance identified image regions that strongly influenced the network’s output. This analysis revealed that forelimb and hindlimb areas in the somatosensory cortex significantly contributed to behavior classification.
“The ability to pinpoint key cortical regions used by the decoder helps open the lid on deep learning’s black box,” Ajioka says. This interpretability is essential for translating AI-based neural decoders into reliable tools for neuroscience and clinical applications.
Taken together, the Kobe University team presents a generalizable, interpretable end-to-end approach for classifying behavioral states from cortex-wide calcium imaging. Their work lays important groundwork for developing non-invasive, near real-time brain-machine interfaces that rely on whole-brain functional signals.
Funding: This research was supported by the Japan Society for the Promotion of Science (grants JP16H06316, JP23H04233, JP23KK0132, JP19K16886, JP23K14673 and JP23H04138), the Japan Agency for Medical Research and Development (grant JP21wm0425011), the Japan Science and Technology Agency (grants JPMJMS2299 and JPMJMS229B), the National Center of Neurology and Psychiatry (grant 30-9), and the Takeda Science Foundation. The project was conducted in collaboration with researchers from the ATR Neural Information Analysis Laboratories.
About this AI and movement research news
Author: Daniel Schenz
Source: Kobe University
Contact: Daniel Schenz – Kobe University
Image: Image credited to Neuroscience News
Original Research: Open access. “End-to-end deep learning approach to mouse behavior classification from cortex-wide calcium imaging” by Takumi Toru et al., PLOS Computational Biology
Abstract
End-to-end deep learning approach to mouse behavior classification from cortex-wide calcium imaging
Deep learning offers a powerful framework for neural decoding in systems neuroscience and clinical studies. Interpretable and transparent decoders that explain which features of brain activity drive behavioral predictions are essential for identifying meaningful neural signals. In this study, we evaluate an end-to-end deep learning pipeline that classifies mouse behavioral states from mesoscopic, cortex-wide calcium imaging data.
Our convolutional neural network (CNN) combined with a recurrent neural network (RNN) achieves high accuracy and demonstrates robustness to individual differences on temporal scales of sub-seconds. Using this CNN–RNN decoder and an input-perturbation analysis, we identify forelimb and hindlimb areas in the somatosensory cortex as significant contributors to behavior classification. These results suggest that an end-to-end approach can be both accurate and interpretable, offering unbiased visualization of critical brain regions for neural decoding.