This Soc(AI)ety Seminars session is presented by Lucy Family Institute for Data and Society & Human-centered Analytics Lab (HAL).
The Soc(AI)ety Seminars, hosted by the Lucy Family Institute for Data & Society, are a collection of talks with a vision for AI’s present and future impact on society. Each session is meant to inspire a dialogue on ethical and socially responsible Data & AI innovation. For more information, and to view previous Soc(AI)ety Seminars sessions, please visit the Soc(AI)ety Seminars webpage.
Description:
AI systems are often evaluated by whether they produce the correct answer. An AI agent may arrive at an appropriate recommendation through incomplete or clinically invalid reasoning, limiting its reliability and usefulness in practice. In healthcare, how a system reaches an answer is as important as the quality of the answer. This talk presents Differential Reasoning Learning, a framework and methodology to identify and correct gaps in the reasoning of clinical AI agents. The approach compares a small model-based agent’s reasoning with trusted references, such as a more capable large language model and physician-authored rationales and clinical guidelines. It represents reasoning processes as structured graphs and identifies missing, unnecessary, or incorrectly connected reasoning steps. These discrepancies are then converted into reusable natural-language instructions. When the small model-based agent encounters a new case, the most relevant instructions are retrieved and provide the required context to help address likely reasoning gaps.
We evaluate the framework on a consequential decision – a prediction task in the hospital emergency department pertaining to whether a patient will return to the emergency room in 9 days. The approach improves both final-answer accuracy and the clinical validity of the reasoning produced by the small model agent. Ablation studies and clinician review provide additional support for the framework’s design. The results suggest how smaller models can learn not only from whether their answers are correct, but also from how their reasoning differs from clinically grounded reasoning. This may support more accessible, reliable, interpretable, and deployable clinical decision-support agents across diverse healthcare settings.
Guest Speaker Bio:
Rema Padman is Trustees Professor of Management Science and Healthcare Informatics in the Heinz College of Information Systems and Public Policy at Carnegie Mellon University, and Adjunct Professor in the Department of Biomedical Informatics at the University of Pittsburgh School of Medicine. Her research, funded by the NIH, CDC, Veterans Affairs, and several foundations, investigates predictive, prescriptive and generative analytics, informatics and operations for data-driven decision support in the context of clinical, public health and consumer-facing IT interventions in healthcare delivery and management. More recently, she has also been focusing on Artificial Intelligence + Operations Research approaches for investigating patient safety issues in medication management and for addressing health literacy challenges. She is a Distinguished Alumnus of IIT/Kanpur, India, recipient of the IBM Faculty Award, multiple Best Paper awards and Teaching Excellence awards, inaugural recipient of the Bufalini Award for AI in Medicine in Italy, and an elected Fellow of the American Medical Informatics Association. More information is available at https://www.heinz.cmu.edu/faculty-research/profiles/padman-rema/.