Summary: Researchers have developed a stretchable, rechargeable sticker-style wearable that detects genuine emotional states by monitoring physiological signals such as heart rate, skin temperature, humidity and blood oxygen. The flexible patch transmits real-time, wireless data to mobile devices and cloud services, enabling clinicians to assess emotional well-being remotely even when facial expressions are misleading.
This multimodal device combines independent on-skin sensors with facial expression analysis to improve emotion recognition while preserving user privacy. Powered by an AI model trained on synchronized physiological and facial data, the system shows strong accuracy for both posed and spontaneous emotional responses, offering promise for telemedicine, early mental health intervention and continuous emotional monitoring.
Key Facts:
- Multimodal sensing: The patch measures skin temperature, humidity, biaxial strain (facial movement), heart rate and blood oxygen (SpO2) using independently operated sensors to avoid cross-interference.
- AI-driven emotion recognition: The trained model classified acted facial expressions with 96.28% accuracy and identified real emotional responses with 88.83% accuracy.
- Remote monitoring for care: Data are transmitted wirelessly to mobile devices and cloud platforms to support telemedicine and earlier detection of mental health issues.
Source: Penn State
Understanding the challenge
People often mask their true feelings, and relying solely on facial expressions can mislead clinicians and caregivers. Bottled-up emotions contribute to anxiety, panic attacks and other mental health problems. To provide a clearer, more objective window into a person’s emotional state, a multidisciplinary team led by Huanyu “Larry” Cheng at Penn State developed a skin-adherent, stretchable electronic patch that captures multiple physiological indicators linked to emotion.
The BandAid-sized patch is built from ultra-thin, flexible layers of metals such as platinum and gold patterned into wavy geometries so sensors remain sensitive even when stretched or twisted on skin. Temperature- and humidity-sensitive materials change electrical current flow with thermal shifts, while hollow carbon nanotube structures absorb moisture to track humidity. The design also integrates a rechargeable flexible circuit with wireless powering and data transmission.
Decoupled sensing for accurate readings
A key engineering challenge in multimodal wearable systems is signal coupling—one sensor’s output interfering with another’s measurement. The team addressed this by arranging and layering materials so each sensor functions independently. For instance, a rigid sublayer supports temperature and humidity elements to shield them from the mechanical stretching that facial movement sensors experience. A waterproof layer protects temperature and strain sensors from moisture. These design choices reduce cross-talk and improve the clarity and reliability of the physiological signals.
Combining physiology and facial analysis
The device pairs on-skin physiological data with facial expression tracking to distinguish performed expressions from authentic emotional responses. Researchers trained a neural network using data from pilot participants who repeated six common expressions—happiness, surprise, fear, sadness, anger and disgust—100 times each while wearing the patch. The model was subsequently tested on additional participants and performed strongly at recognizing posed expressions (96.28% accuracy) and real emotional reactions elicited by video clips (88.83% accuracy).
During spontaneous emotional responses, the sensors recorded physiological patterns consistent with established links between emotion and body response—for example, rises in skin temperature and heart rate during surprise or anger—supporting the device’s ability to identify genuine affective states.
Privacy and remote care
The system transmits only physiological signal data, not personal identifying information, by design. This feature helps protect user privacy while enabling clinicians to evaluate patients remotely. The wireless capability makes the patch especially relevant for telemedicine, offering clinicians early indicators of anxiety, depression or other mental health conditions and enabling timely intervention.
Researchers also see broader clinical potential: improved assessment of nonverbal patients, detection of behavioral and psychological symptoms of dementia, potential recognition of opioid overdose events, chronic wound monitoring and applications in disease management, neurodegenerative disease tracking and athletic performance monitoring. While these applications require further validation, the flexible multimodal platform could support a range of AI-powered diagnostics and therapeutics.
Research and funding
The study describing this stretchable, rechargeable, multimodal hybrid electronics system and its decoupled sensing approach was published in Nano Letters. Contributors include Huanyu “Larry” Cheng and co-authors from Penn State, with additional contributions from Hongcheng Xu of Xi’an Jiaotong University and Libo Gao of Xiamen University. Funding for the Penn State team’s work was provided by the U.S. National Institutes of Health and the U.S. National Science Foundation.
About this emotion and neurotech research news
Author: Adrienne Berard
Source: Penn State
Contact: Adrienne Berard – Penn State
Image credit: Neuroscience News
Original Research: Closed access. “Stretchable, Rechargeable, Multimodal Hybrid Electronics for Decoupled Sensing toward Emotion Detection” by Huanyu “Larry” Cheng et al., Nano Letters (DOI listed in the original publication).
Abstract
This work presents a fully integrated, stretchable, rechargeable multimodal hybrid device that combines decoupled sensors with a flexible wireless powering and transmission module for emotion recognition. Through optimized structural design and material selection, the system provides continuous, real-time decoupled monitoring of biaxial strain, temperature, humidity, heart rate and SpO2. A stacked bilayer architecture for sensors and flexible circuitry reduces device footprint and improves wearer comfort. A neural network enables high-precision facial expression recognition, and real-time data transmission to mobile devices and the cloud allows healthcare professionals to remotely evaluate psychological health and provide emotional support when needed.