Summary: A surgical robot trained on real procedure videos autonomously completed a critical phase of gallbladder removal, adapting to unexpected situations and following spoken guidance from the surgical team. This achievement highlights how artificial intelligence can combine precise instrument control with the flexibility required for real-world surgery.
Using a machine-learning architecture related to large language models, the robot demonstrated expert-level performance across variable conditions. Researchers describe this as an important step toward reliable, autonomous surgical systems that can assist surgeons or complete operations independently.
Key Facts:
- Autonomous adaptability: The system adjusted in real time to anatomical variation, unexpected events, and spoken corrections.
- Imitation learning: The robot trained by observing videos of human surgeons, with task captions to provide context.
- Surgical milestone: It successfully executed a complex, 17-step phase of gallbladder removal with performance comparable to expert surgeons.
Source: Johns Hopkins University
A team led by researchers at Johns Hopkins University developed a surgical robot that performed a prolonged, clinically meaningful portion of a cholecystectomy (gallbladder removal) on lifelike tissue models without human intervention. During the procedure the robot responded to and learned from voice commands from the surgical team, much like a trainee who receives verbal guidance from a mentor.
Funded by federal grants, the project represents a major advance in surgical robotics: a system that combines mechanical precision with human-like adaptability and an ability to interpret and act on language. The lead research team emphasizes that the system is intended to operate robustly in the messy, unpredictable context of real clinical care, not only in highly controlled laboratory conditions.
“This work moves us beyond robots that simply execute fixed tasks to machines that understand and adapt within surgical procedures,” said medical roboticist Axel Krieger. He noted the distinction is essential for developing clinically viable autonomous systems that can function reliably when conditions change unexpectedly.
The new system, called Surgical Robot Transformer-Hierarchy (SRT-H), employs a hierarchical imitation-learning framework. It pairs a high-level planner that issues language-based task instructions with a low-level controller that generates precise instrument trajectories. The architecture draws on principles used in contemporary language models, enabling the robot to plan, to issue corrective guidance in language form, and to accept spoken instructions during operation.
Lead author Ji Woong “Brian” Kim, formerly a Johns Hopkins postdoctoral researcher and now at Stanford University, said the project tackles core barriers to real-world autonomous surgery. “Our results show that AI models can be trained to perform reliably enough to consider surgical autonomy as a practical possibility,” he said.
Previously, Krieger’s group demonstrated autonomous suturing and other short tasks with their Smart Tissue Autonomous Robot (STAR), but those efforts required marked tissue and tightly controlled environments. SRT-H was trained on unmarked videos of surgeons performing cholecystectomies on pig cadavers and reinforced with textual captions describing each step. After training, the robot completed the 17-step gallbladder dissection sequence with full accuracy in their ex vivo trials.
The procedure is substantially more complex than prior demonstrations: it requires identifying ducts and arteries, precise grasping, strategic clip placement, and careful cutting. Although SRT-H performed the tasks more slowly than an experienced human surgeon, outcome measures and task completion quality were comparable to expert performance.
The system also proved robust to variability and deliberate perturbations. It maintained successful performance across non-uniform anatomy, changes in the robot’s starting position, and modifications to visual appearance such as the addition of bloodlike dye. The robot reacted to spoken corrections—commands like “grab the gallbladder head” or “move the left arm a bit to the left”—and incorporated that feedback into subsequent actions.
Johns Hopkins surgeon Jeff Jopling, a co-author, likened the development to the staged training of surgical residents: mastering different portions of an operation progressively. The team views imitation learning and a hierarchical design as a modular path to broader autonomy, where systems can be extended step by step to cover full procedures.
Future work will expand training and validation across more procedure types and aim to scale the system toward complete autonomous surgeries. The authors stress that additional testing, safety validation, and regulatory review would be required before clinical deployment.
Author contributions include members from Johns Hopkins University, Stanford University, and Optosurgical. Key contributors listed are Juo-Tung Chen, Pascal Hansen, Lucy X. Shi, Antony Goldenberg, Samuel Schmidgall, Paul Maria Scheikl, Anton Deguet, Brandon M. White, Chelsea Finn, De Ru Tsai, and Richard Cha.
About this robotics and AI research news
Author: Jill Rosen
Source: Johns Hopkins University
Contact: Jill Rosen – Johns Hopkins University
Image: The image is credited to Neuroscience News
Original Research: Closed access. “SRT-H: A Hierarchical Framework for Autonomous Surgery via Language-Conditioned Imitation Learning” by Axel Krieger et al., published in Science Robotics.
Abstract
SRT-H: A Hierarchical Framework for Autonomous Surgery via Language-Conditioned Imitation Learning
Research on autonomous surgery has typically emphasized short, well-defined tasks in controlled settings. Real-world surgical practice, however, demands dexterous manipulation sustained over longer periods and the ability to generalize to the variability of biological tissue and anatomy.
Conventional rule-based systems and standard end-to-end learning approaches struggle to meet these requirements. To address this gap, the authors present a hierarchical framework that separates high-level planning from low-level motion control. The high-level component operates in a language space to generate task-level or corrective instructions, while the low-level policy translates those instructions into safe, precise instrument trajectories.
The framework was validated in ex vivo cholecystectomy experiments and through ablation studies that assessed critical system components. In these trials the method achieved a 100% success rate across eight ex vivo gallbladders, operating autonomously without human intervention. The hierarchical design enhanced the system’s ability to recover from suboptimal states, an important capability for deployment in realistic surgical environments.
These results demonstrate step-level autonomy in a surgical procedure and represent a milestone toward the eventual clinical deployment of autonomous surgical systems.