As AI agents take on more responsibilities in healthcare, Stanford researchers are working to ensure these systems are safe and practical.
The Safety, Trust, Evaluation and Watchdog for Embedded Agentic Care Delivery (STEWARD) is an independent, three-stage AI supervisory agent that evaluates the accuracy of medical recommendations of other clinical agents deployed in cardiovascular care.
The system first ensures that patients’ information and the agent’s recommendation are comparable to clinical trial data. When data is not available, STEWARD relies on past decisions and existing guidelines. It then verifies the information behind those recommendations, including patients’ medical history, medication dosage and drug interactions.
In stage three, STEWARD conducts a thorough evaluation of agents’ reasoning and treatment decisions, then provides feedback to improve system accuracy over time.
Researchers hope STEWARD will help physicians provide those with limited access to medical services the care they need.
The demand for healthcare exceeds physician capacity, said radiology professor Akshay Chaudhari. When simpler cases are managed autonomously or efficiently, he noted that physicians could focus their efforts on patients with more complex needs.
Chaudhari co-leads the initiative with medicine professor Euan Ashley and biomedical data science professor Roxana Daneshjou in collaboration with partners including OpenAI and the University of Washington.
Stanford Medicine announced the funding of the STEWARD initiative on Sept. 30. with the research team receiving $14.9 million from Advanced Research Projects Agency for Health (ARPA-H) to develop a regulatory-grade monitoring system.
The initiative falls under their Agentic AI-Enabled Cardiovascular Care Transformation (ADVOCATE) program, which has a broader objective of creating the world’s first FDA-authorized clinical agentic AI system, bringing 24/7 holistic cardiovascular care. For patients with cardiovascular issues, this could open up the possibility of more thorough and responsive care.
One possible application would be in changing drug doses, Ashley said. Treatment recommendations often vary, he said, since most decisions are made based on how different patients respond. As such, STEWARD’s responsibility is not to find a single answer but rather to eliminate incorrect ones generated by the clinical agent.
“There [are] multiple potential correct answers,” Ashley said. “But our focus is on the smaller number of potential wrong answers and detecting those.”
While applying a layer of oversight to the decisions of clinical AI agents might improve accuracy in their recommendations, researchers don’t plan for STEWARD to automate the medical process. Instead, it is meant to be an assistive tool that allows physicians to spend more time on specialized decisions, Ashley said.
Before the agent is implemented, the team must work with the Food and Drug Administration (FDA) and conduct clinical studies to test the system’s reliability. A major milestone for the project would be to achieve Medical Device Development Tool (MDDT) status, which would allow STEWARD to serve as a generic framework for other monitoring systems, facilitating broader use, Chaudhari said.
Even as the effort remains in its early stages, researchers are optimistic about its potential.
“It’s usually harder to be an artist than to be the critic, right?” Chaudhari said of the oversight role. “We are mostly being the critics. Someone else is creating that art already, which is the hard part.”