Evaluating alert fatigue over time to EHR-based clinical trial alerts: findings from a randomized controlled study

Departments of Biomedical Informatics and Internal Medicine, College of Medicine, The Ohio State University, Columbus, Ohio, USA.
Journal of the American Medical Informatics Association (Impact Factor: 3.93). 04/2012; 19(e1):e145-e148. DOI: 10.1136/amiajnl-2011-000743
Source: PubMed

ABSTRACT OBJECTIVE: Inadequate participant recruitment is a major problem facing clinical research. Recent studies have demonstrated that electronic health record (EHR)-based, point-of-care, clinical trial alerts (CTA) can improve participant recruitment to certain clinical research studies. Despite their promise, much remains to be learned about the use of CTAs. Our objective was to study whether repeated exposure to such alerts leads to declining user responsiveness and to characterize its extent if present to better inform future CTA deployments. METHODS: During a 36-week study period, we systematically documented the response patterns of 178 physician users randomized to receive CTAs for an ongoing clinical trial. Data were collected on: (1) response rates to the CTA; and (2) referral rates per physician, per time unit. Variables of interest were offset by the log of the total number of alerts received by that physician during that time period, in a Poisson regression. RESULTS: Response rates demonstrated a significant downward trend across time, with response rates decreasing by 2.7% for each advancing time period, significantly different from zero (flat) (p<0.0001). Even after 36 weeks, response rates remained in the 30%-40% range. Subgroup analyses revealed differences between community-based versus university-based physicians (p=0.0489). DISCUSSION: CTA responsiveness declined gradually over prolonged exposure, although it remained reasonably high even after 36 weeks of exposure. There were also notable differences between community-based versus university-based users. CONCLUSIONS: These findings add to the limited literature on this form of EHR-based alert fatigue and should help inform future tailoring, deployment, and further study of CTAs.

Download full-text


Available from: Peter J Embi, Jul 03, 2015
  • Applied Clinical Informatics 01/2012; 3(3):334-6. DOI:10.4338/ACI-2012-06-LE-0025 · 0.39 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Clinical research informatics is the rapidly evolving sub-discipline within biomedical informatics that focuses on developing new informatics theories, tools, and solutions to accelerate the full translational continuum: basic research to clinical trials (T1), clinical trials to academic health center practice (T2), diffusion and implementation to community practice (T3), and 'real world' outcomes (T4). We present a conceptual model based on an informatics-enabled clinical research workflow, integration across heterogeneous data sources, and core informatics tools and platforms. We use this conceptual model to highlight 18 new articles in the JAMIA special issue on clinical research informatics.
    Journal of the American Medical Informatics Association 04/2012; 19(e1):e36-e42. DOI:10.1136/amiajnl-2012-000968 · 3.93 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Healthcare Electronic Syndromic Surveillance (ESS) is the systematic collection, analysis and interpretation of ongoing clinical data with subsequent dissemination of results, which aid clinical decision-making. To evaluate, classify and analyze the diagnostic performance, strengths and limitations of existing acute care ESS systems. All available to us studies in Ovid MEDLINE, Ovid EMBASE, CINAHL and Scopus databases, from as early as January 1972 through the first week of September 2012. Prospective and retrospective trials, examining the diagnostic performance of inpatient ESS and providing objective diagnostic data including sensitivity, specificity, positive and negative predictive values. Two independent reviewers extracted diagnostic performance data on ESS systems, including clinical area, number of decision points, sensitivity and specificity. Positive and negative likelihood ratios were calculated for each healthcare ESS system. A likelihood matrix summarizing the various ESS systems performance was created. The described search strategy yielded 1639 articles. Of these, 1497 were excluded on abstract information. After full text review, abstraction and arbitration with a third reviewer, 33 studies met inclusion criteria, reporting 102,611 ESS decision points. The yielded I2 was high (98.8%), precluding meta-analysis. Performance was variable, with sensitivities ranging from 21% -100% and specificities ranging from 5%-100%. There is significant heterogeneity in the diagnostic performance of the available ESS implements in acute care, stemming from the wide spectrum of different clinical entities and ESS systems. Based on the results, we introduce a conceptual framework using a likelihood ratio matrix for evaluation and meaningful application of future, frontline clinical decision support systems.
    Applied Clinical Informatics 01/2013; 4(2):212-24. DOI:10.4338/ACI-2012-12-RA-0053 · 0.39 Impact Factor