Evaluating the utility of syndromic surveillance algorithms for screening to detect potentially clonal hospital infection outbreaks

Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee, USA.
Journal of the American Medical Informatics Association (Impact Factor: 3.5). 07/2011; 18(4):466-72. DOI: 10.1136/amiajnl-2011-000216
Source: PubMed


The authors evaluated algorithms commonly used in syndromic surveillance for use as screening tools to detect potentially clonal outbreaks for review by infection control practitioners.
Study phase 1 applied four aberrancy detection algorithms (CUSUM, EWMA, space-time scan statistic, and WSARE) to retrospective microbiologic culture data, producing a list of past candidate outbreak clusters. In phase 2, four infectious disease physicians categorized the phase 1 algorithm-identified clusters to ascertain algorithm performance. In phase 3, project members combined the algorithms to create a unified screening system and conducted a retrospective pilot evaluation.
The study calculated recall and precision for each algorithm, and created precision-recall curves for various methods of combining the algorithms into a unified screening tool.
Individual algorithm recall and precision ranged from 0.21 to 0.31 and from 0.053 to 0.29, respectively. Few candidate outbreak clusters were identified by more than one algorithm. The best method of combining the algorithms yielded an area under the precision-recall curve of 0.553. The phase 3 combined system detected all infection control-confirmed outbreaks during the retrospective evaluation period.
Lack of phase 2 reviewers' agreement indicates that subjective expert review was an imperfect gold standard. Less conservative filtering of culture results and alternate parameter selection for each algorithm might have improved algorithm performance.
Hospital outbreak detection presents different challenges than traditional syndromic surveillance. Nevertheless, algorithms developed for syndromic surveillance have potential to form the basis of a combined system that might perform clinically useful hospital outbreak screening.

Download full-text


Available from: Karen C Bloch
  • Source
    • "There was a special section focused on CRI papers in the December 2011 supplement issue. Much of the increase can be attributed to publications from awardees of the CTSA, since publication rate is related to funding.38 JAMIA publications acknowledging CTSA funding rose from three in 200939–41 to four in 201014 42–44 and 15 in 2011.15 17 19 36 45–55 Some of the articles were not exclusively focused on CRI, but were directly related, covering many different topics that are highly relevant to CRI: data models and terminologies,27 56–68 natural language processing (NLP),16 50 61 69–99 surveillance systems,48 65 80 100–110 and privacy technology and policy.33 111–117 This 2012 CRI supplement adds 18 new publications to this growing field. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Clinical research informatics is the rapidly evolving sub-discipline within biomedical informatics that focuses on developing new informatics theories, tools, and solutions to accelerate the full translational continuum: basic research to clinical trials (T1), clinical trials to academic health center practice (T2), diffusion and implementation to community practice (T3), and 'real world' outcomes (T4). We present a conceptual model based on an informatics-enabled clinical research workflow, integration across heterogeneous data sources, and core informatics tools and platforms. We use this conceptual model to highlight 18 new articles in the JAMIA special issue on clinical research informatics.
    Full-text · Article · Apr 2012 · Journal of the American Medical Informatics Association
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Infections and outbreaks of antimicrobial-resistant bacteria, such as methicillin-resistant Staphylococcus aureus (MRSA) and vancomycin-resistant enterococcus (VRE), have been increasing. Detection methods for antimicrobial-resistant bacteria have been changed from traditional culture methods to chromogenic media culture and molecular methods. Strain-typing methods using various molecular technologies are essential tools for epidemiologic surveillance. Furthermore, outbreak detection, using syndromic surveillance as well as passive and active surveillance, has been applied. However, it is difficult to establish effective and robust guidelines and systems for using these various methods to control antimicrobial-resistant bacteria. Therefore, clinical microbiologists and policy makers must possess expertise in the control of antimicrobial resistant bacteria, discuss the issue sufficiently, and, finally, create a system to accomplish this control.
    Full-text · Article · Jan 2013
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Real-time alerting systems typically warn providers about abnormal laboratory results or medication interactions. For more complex tasks, institutions create site-wide 'data warehouses' to support quality audits and longitudinal research. Sophisticated systems like i2b2 or Stanford's STRIDE utilize data warehouses to identify cohorts for research and quality monitoring. However, substantial resources are required to install and maintain such systems. For more modest goals, an organization desiring merely to identify patients with 'isolation' orders, or to determine patients' eligibility for clinical trials, may adopt a simpler, limited approach based on processing the output of one clinical system, and not a data warehouse. We describe a limited, order-entry-based, real-time 'pick off' tool, utilizing public domain software (PHP, MySQL). Through a web interface the tool assists users in constructing complex order-related queries and auto-generates corresponding database queries that can be executed at recurring intervals. We describe successful application of the tool for research and quality monitoring.
    Preview · Article · Nov 2013 · Journal of the American Medical Informatics Association