Summary: A new machine learning algorithm trained on echocardiogram videos can accurately predict which patients are likely to die within a year.
Source: Geisinger Health System
Researchers at Geisinger have developed a deep-learning model that analyzes echocardiogram videos to predict one-year all-cause mortality with high accuracy.
The algorithm—an application of machine learning and artificial intelligence (AI)—consistently outperformed commonly used clinical risk estimators, including pooled cohort equations and the Seattle Heart Failure Score.
These findings were published in Nature Biomedical Engineering.
“We were excited to find that machine learning can leverage unstructured medical images and videos to improve clinical risk prediction across a range of settings,” said Chris Haggerty, Ph.D., co-senior author and assistant professor in the Department of Translational Data Science and Informatics at Geisinger.
Medical imaging produces large, information-rich datasets that are increasingly part of the electronic health record (EHR). For example, a single cardiac ultrasound exam can generate roughly 3,000 frames. Clinicians have limited time to review these images alongside lab values, clinical notes, and other diagnostics. That creates an opportunity for AI and deep learning to help synthesize visual information and support clinician decision-making.
To build and evaluate the model, the research team trained a convolutional neural network on 812,278 echocardiogram videos collected from 34,362 Geisinger patients over a ten-year period. In total, the project leveraged nearly 50 million image frames, making it one of the largest medical imaging datasets reported to date. The trained model was tested against multiple clinical predictors and compared to cardiologists’ assessments gathered through structured surveys.
In a subsequent evaluation, cardiologists who reviewed echocardiograms with the model’s assistance improved their sensitivity for predicting one-year mortality by 13 percentage points while maintaining specificity. This indicates that AI support can enhance clinician judgment without increasing false positive rates.
Beyond beating standard risk scores, the neural network also outperformed a machine learning approach that relied on 58 human-derived echocardiographic variables combined with 100 clinical variables from the EHR. In an independent validation set that focused on patients with heart failure—2,404 individuals who underwent 3,384 echocardiograms—the model exceeded the predictive performance of the Seattle Heart Failure Score.
“Our goal is to develop algorithms that improve patient care,” said Alvaro Ulloa Cerna, Ph.D., senior data scientist and co-author. “This model shows promise as a clinical tool because it helps cardiologists better identify high-risk patients, which can inform treatment decisions and follow-up planning.”
Clinical implications
Accurate, timely predictions of short-term mortality can guide clinical conversations, inform the intensity of monitoring and intervention, and prioritize patients for more aggressive management or palliative discussions. Because echocardiography is widely used across cardiology and other specialties, an automated, image-based risk predictor could be integrated into clinical workflows to provide real-time decision support.
However, successful clinical implementation will require careful evaluation of generalizability, workflow integration, interpretability, and ethical considerations. The study demonstrates technical feasibility and potential benefit, but additional prospective testing and multi-center validation would be necessary before routine deployment.
Funding: The research received support from the Pennsylvania Department of Health and the Geisinger Health Plan and Clinic.
About this artificial intelligence research news
Source: Geisinger Health System
Contact: Ashley Andyshak Hayes – Geisinger Health System
Image: The image is in the public domain
Original Research: Closed access.
“Deep-learning-assisted analysis of echocardiographic videos improves predictions of all-cause mortality” by Chris Haggerty et al. Nature Biomedical Engineering
Abstract
Deep-learning-assisted analysis of echocardiographic videos improves predictions of all-cause mortality
Machine learning can help clinicians predict mortality and other outcomes by learning complex patterns from historical data, such as longitudinal electronic health records and imaging studies. In this work, a convolutional neural network trained on raw pixel data from 812,278 echocardiographic videos collected from 34,362 patients achieved superior predictions of one-year all-cause mortality. The model outperformed pooled cohort equations and the Seattle Heart Failure Score (evaluated in an independent dataset of 2,404 patients with heart failure who underwent 3,384 echocardiograms), as well as a machine learning model built from 58 echocardiography-derived variables and 100 clinical variables from the EHR. When cardiologists were provided the model’s output, their sensitivity for predicting one-year mortality increased by 13% without loss of specificity. These results suggest that large, unstructured imaging datasets can enable deep learning to enhance many clinical prediction tasks.