Allison Beck McCoy

Vanderbilt University | Vander Bilt · Department of Biomedical Informatics

Clinical decision support (CDS) is essential for delivery of high-quality, cost-effective, and safe healthcare. The authors sought to evaluate the CDS capabilities across electronic health record (EHR) systems. We evaluated the CDS implementation capabilities of 8 Office of the National Coordinator for Health Information Technology Authorized Certi...

Alerting systems, a type of clinical decision support, are increasingly prevalent in healthcare, yet few studies have concurrently measured the appropriateness of alerts with provider responses to alerts. Recent reports of suboptimal alert system design and implementation highlight the need for better evaluation to inform future designs. The author...

We describe a novel, crowdsourcing method for generating a knowledge base of problem-medication pairs that takes advantage of manually asserted links between medications and problems. Through iterative review, we developed metrics to estimate the appropriateness of manually entered problem-medication links for inclusion in a knowledge base that can...

Background: Many healthcare providers are adopting clinical decision support (CDS) systems to improve patient safety and meet meaningful use requirements. Computerized alerts that prompt clinicians about drug-allergy, drug-drug, and drug-disease warnings or provide dosing guidance are most commonly implemented. Alert overrides, which occur when cl...

Objective: To quantify the percentage of records with matching identifiers as an indicator for duplicate or potentially duplicate patient records in electronic health records in five different healthcare organisations, describe the patient safety issues that may arise, and present solutions for managing duplicate records or records with matching i...

Background: The Vanderbilt Clinical Informatics Center (VCLIC) is based in the Department of Biomedical Informatics (DBMI) and operates across Vanderbilt University Medical Center (VUMC) and Vanderbilt University (VU) with a goal of enabling and supporting clinical informatics research and practice. VCLIC supports several types of applied clinical...

Objective This study aims to investigate the feasibility of using Large Language Models (LLMs) to engage with patients at the time they are drafting a question to their healthcare providers, and generate pertinent follow-up questions that the patient can answer before sending their message, with the goal of ensuring that their healthcare provider r...

OBJECTIVE To develop and validate a predictive model for postpartum hemorrhage that can be deployed in clinical care using automated, real-time electronic health record (EHR) data and to compare performance of the model with a nationally published risk prediction tool. METHODS A multivariable logistic regression model was developed from retrospect...

Objective Positive antinuclear antibodies (ANAs) cause diagnostic dilemmas for clinicians. Currently, no tools exist to help clinicians interpret the significance of a positive ANA in individuals without diagnosed autoimmune diseases. We developed and validated a risk model to predict risk of developing autoimmune disease in positive ANA individual...

Objective This study aimed to develop and assess the performance of fine-tuned large language models for generating responses to patient messages sent via an electronic health record patient portal. Materials and Methods Utilizing a dataset of messages and responses extracted from the patient portal at a large academic medical center, we developed...

Importance Chronic kidney disease (CKD) affects 37 million adults in the United States, and for patients with CKD, hypertension is a key risk factor for adverse outcomes, such as kidney failure, cardiovascular events, and death. Objective To evaluate a computerized clinical decision support (CDS) system for the management of uncontrolled hypertens...

Objectives To evaluate the capability of using generative artificial intelligence (AI) in summarizing alert comments and to determine if the AI-generated summary could be used to improve clinical decision support (CDS) alerts. Materials and Methods We extracted user comments to alerts generated from September 1, 2022 to September 1, 2023 at Vander...

Objective To develop and evaluate a data-driven process to generate suggestions for improving alert criteria using explainable artificial intelligence (XAI) approaches. Methods We extracted data on alerts generated from January 1, 2019 to December 31, 2020, at Vanderbilt University Medical Center. We developed machine learning models to predict us...

BACKGROUND Numerous pressure injury prediction models have been developed using electronic health record data. Yet, hospital-acquired pressure injuries (HAPIs) are increasing, which demonstrates the critical challenge of implementing these models in routine care. OBJECTIVE To help bridge the gap between development and implementation, we sought to...

Background Numerous pressure injury prediction models have been developed using electronic health record data, yet hospital-acquired pressure injuries (HAPIs) are increasing, which demonstrates the critical challenge of implementing these models in routine care. Objective To help bridge the gap between development and implementation, we sought to...

Objectives Geocoding, the process of converting addresses into precise geographic coordinates, allows researchers and health systems to obtain neighborhood-level estimates of social determinants of health. This information supports opportunities to personalize care and interventions for individual patients based on the environments where they live....

Background Early detection of clinical deterioration among hospitalized patients is a clinical priority for patient safety and quality of care. Current automated approaches for identifying these patients perform poorly at identifying imminent events. Objective Develop a machine learning algorithm using pager messages sent between clinical team mem...

Objective: This study aimed to develop and assess the performance of fine-tuned large language models for generating responses to patient messages sent via an electronic health record patient portal. Methods: Utilizing a dataset of messages and responses extracted from the patient portal at a large academic medical center, we developed a model (CLA...

Background Chronic kidney disease (CKD) is common and associated with adverse clinical outcomes. Most care for early CKD is provided in primary care, including hypertension (HTN) management. Computerized clinical decision support (CDS) can improve the quality of care for CKD but can also cause alert fatigue for primary care physicians (PCPs). Compu...

Objectives: This literature review summarizes relevant studies from the last three years (2020-2022) related to clinical decision support (CDS) and CDS impact on health disparities and the digital divide. This survey identifies current trends and synthesizes evidence-based recommendations and considerations for future development and implementation...

Objective: To develop and validate an approach that identifies patients eligible for lung cancer screening (LCS) by combining structured and unstructured smoking data from the electronic health record (EHR). Methods: We identified patients aged 50-80 years who had at least one encounter in a primary care clinic at Vanderbilt University Medical C...

Objective To determine if ChatGPT can generate useful suggestions for improving clinical decision support (CDS) logic and to assess noninferiority compared to human-generated suggestions. Methods We supplied summaries of CDS logic to ChatGPT, an artificial intelligence (AI) tool for question answering that uses a large language model, and asked it...

Objective: To determine if ChatGPT can generate useful suggestions for improving clinical decision support (CDS) logic and to assess noninferiority compared to human-generated suggestions. Methods: We supplied summaries of CDS logic to ChatGPT, an artificial intelligence (AI) tool for question answering that uses a large language model, and asked i...

Background: Chest pain (CP) is the hallmark symptom for acute coronary syndrome (ACS) but is not reported in 20-30% of patients, especially women, elderly, non-white patients, presenting to the emergency department (ED) with an ST-segment elevation myocardial infarction (STEMI). Methods: We used a retrospective 5-year adult ED sample of 279,132...

Objective: To develop and test an accurate deep learning model for predicting new onset delirium in hospitalized adult patients. Methods: Using electronic health record (EHR) data extracted from a large academic medical center, we developed a model combining long short-term memory (LSTM) and machine learning to predict new onset delirium and com...

Objectives To improve clinical decision support (CDS) by allowing users to provide real-time feedback when they interact with CDS tools and by creating processes for responding to and acting on this feedback. Methods Two organizations implemented similar real-time feedback tools and processes in their electronic health record and gathered data over...

Objectives Complex interventions with multiple components and behavior change strategies are increasingly implemented as a form of clinical decision support (CDS) using native electronic health record functionality. Objectives of this study were, therefore, to (1) identify the proportion of randomized controlled trials with CDS interventions that w...

Objective To quantify initial tidal volume (VT) during neonatal volume-targeted ventilation (VTV) and to characterize the agreement of initial VT with the limited-evidence available. Study design We performed a multi-center retrospective observational cohort study in two Neonatal Intensive Care Units evaluating 313 infants who received VTV as the...

Importance: Understanding the differences and potential synergies between traditional clinician assessment and automated machine learning might enable more accurate and useful suicide risk detection. Objective: To evaluate the respective and combined abilities of a real-time machine learning model and the Columbia Suicide Severity Rating Scale (...

Background: Patients taking high doses of opioids, or taking opioids in combination with other central nervous system depressants, are at increased risk of opioid overdose. Coprescribing the opioid-reversal agent naloxone is an essential safety measure, recommended by the surgeon general, but the rate of naloxone coprescribing is low. Therefore, w...

Objective: We describe the Clickbusters initiative implemented at Vanderbilt University Medical Center (VUMC), which was designed to improve safety and quality and reduce burnout through the optimization of clinical decision support (CDS) alerts. Materials and methods: We developed a 10-step Clickbusting process and implemented a program that in...

Developing a diverse informatics workforce broadens the research agenda and ensures the growth of innovative solutions that enable equity-centered care. The American Medical Informatics Association (AMIA) established the AMIA First Look Program in 2017 to address workforce disparities among women, including those from marginalized communities. The...

Purpose: The purpose of this study was to evaluate the current state of problem list maintenance at an academic medical center. Summary: We included problem list data for patients who had at least 2 face-to-face encounters at Vanderbilt University Medical Center or its clinics between January 1, 2018, and December 31, 2019. We used the frequency...

Objective Accurate and robust quality measurement is critical to the future of value-based care. Having incomplete information when calculating quality measures can cause inaccuracies in reported patient outcomes. This research examines how quality calculations vary when using data from an individual electronic health record (EHR) and longitudinal...

Objective Clinical decision support (CDS) can contribute to quality and safety. Prior work has shown that errors in CDS systems are common and can lead to unintended consequences. Many CDS systems use Boolean logic, which can be difficult for CDS analysts to specify accurately. We set out to determine the prevalence of certain types of Boolean logi...

Pneumonia is the most frequent cause of infectious disease-related deaths in children worldwide. Clinical decision support (CDS) applications can guide appropriate treatment, but the system must first recognize the appropriate diagnosis. To enable CDS for pediatric pneumonia, we developed an algorithm integrating natural language processing (NLP) a...

Objective: To quantify and contextualize the risk for coronavirus disease 2019 (COVID-19)-related hospitalization and illness severity in type 1 diabetes. Research design and methods: We conducted a prospective cohort study to identify case subjects with COVID-19 across a regional health care network of 137 service locations. Using an electronic...

Streamlining medication review and reconciliation Overview/Background Medication lists are fundamental to patient care delivery through the electronic medical record (EHR). Medication lists change over time as patients receive care through multiple encounters across different providers in distinct care settings. ThusTherefore, obtaining an accurate...

Overview/Background: Vanderbilt University Medical Center (VUMC) is an academic, tertiary care medical center in Nashville, TN with a 1000-bed general medical and surgical facility and nearly 2,000,000 patient visits annually. Similar to most other organizations, VUMC has implemented medication alerting (e.g., drug-allergy, drug-drug, and drug-dose...

Purpose: Kidney stone recurrence rates vary between patients. A patient's risk informs the frequency and intensity of preventative interventions. Clinicians routinely use clinical experience to estimate risk. We sought to compare clinician estimated recurrence risk with the recurrence of kidney stones (ROKS) nomogram. Materials and Methods: We sur...

Effect of a Clinical Decision Support Alert Encouraging Prescribing of Naloxone for Patients at High Risk of Opioid Overdose

In the past 2 decades, the United States has seen widespread adoption of electronic health records (EHRs) and a transition from mostly locally developed EHRs to commercial systems. However, most research on quality improvement and safety interventions in EHRs is still conducted at a single site, in a single EHR. Although single-site studies are imp...

Purpose: The problem list was introduced to the patient chart in the 1960s to improve the efficiency of patient review. Since then, the use of the problem list has expanded and regulatory agencies have required problem list maintenance in the health record; however, questions have been raised related to its accuracy and completeness. The literature...

What might the attendee be able to do after being in your session? Attendees will learn how to plan and implement decision support for intravenous (IV) to oral (PO) conversion of inpatient medications at their institutions. Topics include medication selection analysis, clinical decision support (CDS) implementation, and cost savings analysis. Descr...

Objective Within the National Pediatric Cardiology Quality Improvement Collaborative (NPC-QIC), a learning health network developed to improve outcomes for patients with hypoplastic left heart syndrome and variants, we assessed which centers contributed to reductions in mortality and growth failure. Study design Centers within the NPC-QIC were div...

To overcome limitations of previously developed scientific productivity ranking services, we created the Biomedical Informatics Researchers ranking website (rank.informatics-review.com). The website is composed of four key components that work together to create the automatically updating ranking website: 1) list of biomedical informatics researche...

The problem list was introduced to the patient chart in the 1960s to improve the efficiency of patient review. Since then, the use of the problem list has expanded to clinical decision support (CDS), problem-oriented charting, reporting, research, and even billing. Subsequently, regulatory agencies have required problem list maintenance in the heal...

There is a critical need need for multi-institutional, large-scale, international applied clinical informatics research, given the global, widespread use of commercially-available electronic health records with different designs, capabilities, configurations, and implementation strategies. The Clinical Informatics Research Collaborative (CIRCLE) ai...

Objective: The study sought to determine availability and use of structured override reasons for drug-drug interaction (DDI) alerts in electronic health records. Materials and methods: We collected data on DDI alerts and override reasons from 10 clinical sites across the United States using a variety of electronic health records. We used a multi...

Objective We identified the methods used and determined the roles of electronic health records (EHRs) in detecting and assessing adverse drug events (ADEs) in the ambulatory setting. Methods We performed a systematic literature review by searching PubMed and Google Scholar for studies on ADEs detected in the ambulatory setting involving any EHR use...

Objective: Developing effective and reliable rule-based clinical decision support (CDS) alerts and reminders is challenging. Using a previously developed taxonomy for alert malfunctions, we identified best practices for developing, testing, implementing, and maintaining alerts and avoiding malfunctions. Materials and methods: We identified 72 in...

Background Value-based payment for care requires the consistent, objective calculation of care quality. Previous initiatives to calculate ambulatory quality measures have relied on billing data or individual electronic health records (EHRs) to calculate and report performance. New methods for quality measure calculation promoted by federal regulati...

Objective: We assessed changes in the percentage of providers with positive perceptions of electronic health record (EHR) benefit before and after transition from a local basic to a commercial comprehensive EHR. Methods: Changes in the percentage of providers with positive perceptions of EHR benefit were captured via a survey of academic health...

Objective The United States Office of the National Coordinator for Health Information Technology sponsored the development of a “high-priority” list of drug-drug interactions (DDIs) to be used for clinical decision support. We assessed current adoption of this list and current alerting practice for these DDIs with regard to alert implementation (pr...

Purpose of Study: Electronic health records (EHR) have the potential to improve health care delivery and enhance patient safety. We sought to identify factors associated with clinician perception of improved patient safety following the transition from a basic, locally-developed EHR to a comprehensive, commercially-available EHR. Methods Used: We...

We developed the Biomedical Informatics Researchers ranking website (rank.informatics-review.com) to overcome many of the limitations of previous scientific productivity ranking strategies. The website is composed of four key components that work together to create an automatically updating ranking website: (1) list of biomedical informatics resear...

Clinical knowledge bases of problem-medication pairs are necessary for many informatics solutions that improve patient safety, such as clinical summarization. However, developing these knowledge bases can be challenging. We sought to validate a previously developed crowdsourcing approach for generating a knowledge base of problem-medication pairs i...

Background: Therapy for certain medical conditions occurs in a stepwise fashion, where one medication is recommended as initial therapy and other medications follow. Sequential pattern mining is a data mining technique used to identify patterns of ordered events. Objective: To determine whether sequential pattern mining is effective for identify...

To measure performance by eligible health care providers on CMS's meaningful use measures. Medicare Electronic Health Record Incentive Program Eligible Professionals Public Use File (PUF), which contains data on meaningful use attestations by 237,267 eligible providers through May 31, 2013. Cross-sectional analysis of the 15 core and 10 menu measur...

Background: Correlation of data within electronic health records is necessary for implementation of various clinical decision support functions, including patient summarization. A key type of correlation is linking medications to clinical problems; while some databases of problem-medication links are available, they are not robust and depend on pr...

Clinical databases may contain several records for a single patient. Multiple general entity-resolution algorithms have been developed to identify such duplicate records. To achieve optimal accuracy, algorithm parameters must be tuned to a particular dataset. The purpose of this study was to determine the required training set size for probabilisti...

The field of clinical informatics has expanded substantially in the six decades since its inception. Early research focused on simple demonstrations that health information technology (HIT) such as electronic health records (EHRs), computerized provider order entry (CPOE), and clinical decision support (CDS) systems were feasible and potentially be...

Automated summarization tools that create condition-specific displays may improve clinician efficiency. These tools require new kinds of knowledge that is difficult to obtain. We compared five problem-medication pair knowledge bases generated using four previously described knowledge base development approaches. The number of pairs in the resulting...

The Strategic Health IT Advanced Research Projects (SHARP) program seeks to conquer well-understood challenges in medical informatics through breakthrough research. Two SHARP centers have found alignment in their methodological needs: (1) members of the National Center for Cognitive Informatics and Decision-making (NCCD) have developed knowledge ba...

Introduction Clinical databases require accurate entity resolution (ER). One approach is to use algorithms that assign questionable cases to manual review. Few studies have compared the performance of common algorithms for such a task. Furthermore, previous work has been limited by a lack of objective methods for setting algorithm parameters. We co...

In a prior study, we developed methods for automatically identifying associations between medications and problems using association rule mining on a large clinical data warehouse and validated these methods at a single site which used a self-developed electronic health record. To demonstrate the generalizability of these methods by validating them...