Using AI-generated suggestions from ChatGPT to optimize clinical decision support
Authors Siru Liu, PhD and Aileen Wright, MD, discuss this month’s JAMIA Journal Club selection.
Tian Kang, Yingcheng Sun, Jae Hyun Kim, Casey Ta, Adler Perotte, Kayla Schiffer, Mutong Wu, Yang Zhao, Nour Moustafa-Fahmy, Yifan Peng, Chunhua Weng. EvidenceMap: a three-level knowledge representation for medical evidence computation and comprehension. J Am Med Inform Assoc. 2023;ocad036, doi:10.1093/jamia/ocad036
Watch the Recording
Presenters
Siru Liu, PhD
Dr. Siru Liu is a postdoctoral research fellow in the Department of Biomedical Informatics at Vanderbilt University Medical Center. Her research interests include optimizing electronic health record features and functions, with an emphasis on clinical decision support, leveraging machine learning and natural language processing techniques to improve healthcare quality. She holds a Ph.D. in Biomedical Informatics from the University of Utah School of Medicine. Dr. Liu received an NLM K99/R00 in using explainable AI to optimize clinical decision support. She is recognized as a 2022 Google Cloud Research Innovator. She is a fellow in the NCI's Multilevel Intervention Training Institute, a 2022-23 American Association of University Women International Fellow, and a scholar in the Women in AMIA (WIA) Leadership Program.
Aileen Wright, MD
Dr. Wright is a board-certified internist. She received her MD from Yale University and completed a residency in Internal Medicine at Brigham and Women’s Hospital and an informatics fellowship at Vanderbilt University Medical Center (VUMC). She is currently a practicing primary care physician and Instructor in the Departments of Medicine (DOM) and Biomedical Informatics at VUMC. She leads the DOM Physician Builder group at VUMC. Her research lies at the intersection of Health IT, quality and patient safety, including projects on predicting hypoglycemia, increasing statin prescribing, and the application of large language models to healthcare.
Manager
Lu He, BS
Statement of Purpose
Alert fatigue is a pressing issue. In this study, we evaluated the feasibility of using ChatGPT to generate suggestions for improving the specificity of alert logic. The suggestions generated by AI were found to offer unique perspectives and were evaluated as highly understandable and relevant, with moderate usefulness, low acceptance, bias, inversion, redundancy, and low ability to improve clinical workflow. Therefore, these AI-generated suggestions could be an important complementary part of optimizing CDS alerts, can identify potential improvements to alert logic and support their implementation, and may even be able to assist experts in formulating their own suggestions for CDS improvement. Overall, ChatGPT shows great potential for using large language models and reinforcement learning from human feedback to improve CDS alert logic and potentially other medical areas involving complex, clinical logic, a key step in the development of an advanced learning health system.
Learning Objectives
- Understand the potential benefits and limitations of using artificial intelligence tools, such as ChatGPT, for generating suggestions to improve clinical decision support (CDS) logic.
- Learn about the potential of AI-generated suggestions to assist experts in formulating their own suggestions for CDS improvement.
- Develop an understanding of how large language models and reinforcement learning can be used to improve CDS alert logic and other medical areas.
Format
- 35-minute presentation by article author(s) considering salient features of the published study and its potential impact on practice
- 25-minute discussion of questions submitted by listeners via the webinar tools and moderated by JAMIA Student Editorial Board members.
Accreditation Statement
The American Medical Informatics Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.
Commercial Support
No commercial support was received for this activity.
Disclosures for this Activity
The following planners and staff who are in a position to control the content of this activity disclose that they have no financial relationships with commercial interests/ineligible entities:
- Presenter: Siru Liu, PhD and Aileen Wright, MD
- JAMIA Journal Club Planners: Lu He; Sanya Taneja; Kirk Roberts
- AMIA Staff: Eileen Bailey