Summary: Researchers have created a mixed-reality game called Pop’Balloons designed to help children on the autism spectrum improve motor skills and social coordination.
Source: University of Montreal
Demand for mental health evaluation and intervention is rising sharply, and existing services are often centralized, overburdened, and difficult for many families to access. Children with autism spectrum disorder (ASD) are especially affected by these limitations.
To help fill this gap, therapists and researchers are increasingly turning to “serious games”—interactive applications designed with therapeutic goals in mind. One of the early examples is Pop’Balloons, a mixed-reality game developed to support motor skill development in autistic children. The game builds on years of laboratory research into social interaction, motor coordination, and human-machine systems.
The project was led by Guillaume Dumas, a professor of computational psychiatry at the Université de Montréal Faculty of Medicine and director of the Precision Psychiatry and Social Physiology Laboratory at the CHU Sainte-Justine Research Center. Dumas also holds the IVADO Chair in AI and Mental Health and is affiliated with Mila, the Quebec Artificial Intelligence Institute.
Traditional tools for measuring brain activity—such as scanners and electroencephalography—were developed to study one person at a time. Consequently, early neuroscience research into social processes often relied on recorded or simulated interactions rather than genuine, live social exchanges.
The emergence of hyperscanning changed that by allowing simultaneous recording of multiple brains, making it possible to study real-time social interactions and how people’s brains synchronize during those interactions.
Brains that sync during interaction
Using hyperscanning in 2010, Dumas found that people’s neural activity can synchronize during social interaction, a finding that supports the intuitive sense that people can be “on the same wavelength.” Building on that insight, his team developed systems to study human behavior with dynamic virtual partners.
During a postdoctoral period in the United States, Dumas and colleagues created an interactive system in which a virtual avatar responded to a human participant in real time. The avatar’s responses were not rigidly scripted but adapted to the human’s behavior in milliseconds. This human-machine interface—termed the Human Dynamic Clamp—was able both to mirror human behavior and to elicit new behaviors from participants. Early tests focused on neurotypical adults.
Research suggests that sensory-motor differences in people with ASD may contribute to social communication and interaction challenges. The Human Dynamic Clamp was capable of extracting precise measures of motor coordination and interpersonal synchronization, so Dumas’ team adapted these capabilities to develop a machine-assisted method for neuropsychological evaluation of ASD. That assessment method has been clinically tested on more than 155 subjects and peer-validated.
Because the system also showed potential for teaching new behaviors, the team proceeded to create a version aimed at improving motor skills in children with autism.
Designing a child-friendly game
Early testing in Paris produced frank feedback from children: the initial research prototype was functional but not engaging. “Children are blunt and they told me straight out: ‘Your video game sucks!’” Dumas recalled. Taking that feedback to heart, the team collaborated with students at the École Centrale de Paris to build a more appealing prototype and later worked with Act’image and the Pasteur Institute to produce a 3D iteration.
A partnership with Microsoft’s HoloLens enabled a mixed-reality version. Unlike virtual reality, which replaces the user’s surroundings with an immersive virtual scene, mixed reality overlays holograms onto the real environment. In Pop’Balloons, holographic balloons appear in a child’s real room and the objective is simple: pop the balloons. The mechanic is intuitive—the faster and more accurately children pop balloons, the higher their score. Children can explore their environment at their own pace and repeat sessions as needed, which supports gradual skill-building.
With funding from the Orange Foundation, an open-access version of the game was developed, allowing Dumas to continue refinement after joining the Université de Montréal.
From game development to precision medicine
Further collaboration with the Canadian studio Eidos and technical input from Mila enabled more advanced machine learning and gameplay improvements. IVADO’s collaborative research teams helped bridge academic and industry expertise to produce a more engaging and accessible serious game.
Dumas explains that the project has moved beyond simple gameplay to enable clinical stratification: analyzing gameplay data to identify subgroups within ASD populations for more targeted, precision-medicine trials in mental health. A paper describing preliminary results from stratification based on game data was recently accepted to NeurIPS, a major conference on machine learning and computational neuroscience.
Accessibility and inclusion
Creating games that are comfortable and effective for neurodiverse players requires careful validation. “The first step is to validate the accessibility parameters,” said Romain Trachel, a machine-learning specialist at Eidos-Montréal. Sensory perception and preferences can differ in autistic individuals, so designers must confirm that the device, pacing, visuals, and interactions are appropriate and engaging.
Mariem Hafsia, a master’s student at Université de Montréal, emphasizes user comfort and comprehension: will children be at ease with the headset or device, understand the game concept, and find the activity meaningful? A simple object like a balloon may carry different associations for children with atypical development, so testing and iteration are essential.
The goal is for Pop’Balloons to serve as both a therapeutic tool and a model for more inclusive game design. In time, such tools could be used for neuropsychological screening or monitoring in non-clinical settings, reducing pressure on overstretched health systems.
“The game wouldn’t necessarily be used to make diagnoses that require a psychologist or physician,” Dumas noted. Instead, it could support clinicians by automating parts of the assessment process and saving valuable time. Clinicians might then focus on higher-level care while the game collects repeatable, standardized behavioral data.
Looking ahead, an adaptive, cloud-connected version could enable personalized telemedicine: clinicians could monitor progress via a dashboard, consult performance history, and remotely adjust therapeutic levels. The game could autonomously guide players to the most appropriate challenges to promote improvement, functioning as an adaptive psycho-educational intervention under clinician oversight.
About this ASD research news
Author: Press Office
Source: University of Montreal
Contact: Press Office – University of Montreal
Image: The image is credited to University of Montreal