A systematic review to evaluate the accuracy of electronic adverse drug event detection

Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada.
Journal of the American Medical Informatics Association (Impact Factor: 3.5). 01/2012; 19(1):31-8. DOI: 10.1136/amiajnl-2011-000454
Source: PubMed


Adverse drug events (ADEs), defined as adverse patient outcomes caused by medications, are common and difficult to detect. Electronic detection of ADEs is a promising method to identify ADEs. We performed this systematic review to characterize established electronic detection systems and their accuracy.
We identified studies evaluating electronic ADE detection from the MEDLINE and EMBASE databases. We included studies if they contained original data and involved detection of electronic triggers using information systems. We abstracted data regarding rule characteristics including type, accuracy, and rationale.
Forty-eight studies met our inclusion criteria. Twenty-four (50%) studies reported rule accuracy but only 9 (18.8%) utilized a proper gold standard (chart review in all patients). Rule accuracy was variable and often poor (range of sensitivity: 40%-94%; specificity: 1.4%-89.8%; positive predictive value: 0.9%-64%). 5 (10.4%) studies derived or used detection rules that were defined by clinical need or the underlying ADE prevalence. Detection rules in 8 (16.7%) studies detected specific types of ADEs.
Several factors led to inaccurate ADE detection algorithms, including immature underlying information systems, non-standard event definitions, and variable methods for detection rule validation. Few ADE detection algorithms considered clinical priorities. To enhance the utility of electronic detection systems, there is a need to systematically address these factors.

Download full-text


Available from: Alison Jennings
    • "A challenge for pharmacovigilance is that ADEs are heav­ ily underreported [4], both in spontaneous reporting systems -wherein ADE case reports submitted voluntarily by patients and clinicians are collected -and in EHRs, wherein ADEs can be encoded by a limited set of diagnosis codes. To address the underreporting problem, alerting systems that can automati­ cally detect ADEs in EHRs are potentially very valuable, and much research has been conducted to that end. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Adverse drug events (ADEs) are heavily underreported in electronic health records (EHRs). Alerting systems that are able to detect potential ADEs on the basis of patient-specific EHR data would help to mitigate this problem. To that end, the use of machine learning has proven to be both efficient and effective; however, challenges remain in representing the heterogeneous EHR data, which moreover tends to be high- dimensional and exceedingly sparse, in a manner conducive to learning high-performing predictive models. Prior work has shown that distributional semantics – that is, natural language processing methods that, traditionally, model the meaning of words in semantic (vector) space on the basis of co-occurrence information – can be exploited to create effective representations of sequential EHR data of various kinds. When modeling data in semantic space, an important design decision concerns the size of the context window around an object of interest, which governs the scope of co-occurrence information that is taken into account and affects the composition of the resulting semantic space. Here, we report on experiments conducted on 27 clinical datasets, demonstrating that performance can be significantly improved by modeling EHR data in ensembles of semantic spaces, consisting of multiple semantic spaces built with different context window sizes. A follow-up investigation is conducted to study the impact on predictive performance as increasingly more semantic spaces are included in the ensemble, demonstrating that accuracy tends to improve with the number of semantic spaces, albeit not monotonically so. Finally, a number of different strategies for combining the semantic spaces are explored, demonstrating the advantage of early (feature) fusion over late (classifier) fusion. Semantic space ensembles allow multiple views of (sparse) data to be captured (densely) and thereby enable improved performance to be obtained on the task of detecting ADEs in EHRs.
    No preview · Conference Paper · Oct 2015
  • Source
    • "They included "studies if they contained original data and involved detection of electronic triggers using information systems"[15]. "They abstracted data regarding rule characteristics including type, accuracy, and rational "[15]. Honigman et al. also developed a program that combines four computer search methods, including text searching of the electronic medical record, to detect ADEs in outpatient settings[16]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: BackgroundAntibiotics are the widely prescribed drugs for children and most likely to be related with adverse reactions. Record on adverse reactions and allergies from antibiotics considerably affect the prescription choices. We consider this a biomedical decision-making problem and explore hidden knowledge in survey results on data extracted from a big data pool of health records of children, from the Health Center of Osijek, Eastern Croatia.ResultsWe applied and evaluated a k-means algorithm to the dataset to generate some clusters which have similar features. Our results highlight that some type of antibiotics form different clusters, which insight is most helpful for the clinician to support better decision-making.ConclusionsMedical professionals can investigate the clusters which our study revealed, thus gaining useful knowledge and insight into this data for their clinical studies.
    Full-text · Article · May 2014 · BMC Bioinformatics
  • [Show abstract] [Hide abstract]
    ABSTRACT: We present and test the intuition that letters to the editor in journals carry early signals of adverse drug events (ADEs). Surprisingly these letters have not yet been exploited for automatic ADE detection unlike for example, clinical records and PubMed. Part of the challenge is that it is not easy to access the full-text of letters (for the most part these do not appear in PubMed). Also letters are likely underrated in comparison with full articles. Besides demonstrating that this intuition holds we contribute techniques for post market drug surveillance. Specifically, we test an automatic approach for ADE detection from letters using off-the-shelf machine learning tools. We also involve natural language processing for feature definitions. Overall we achieve high accuracy in our experiments and our method also works well on a second new test set. Our results encourage us to further pursue this line of research.
    No preview · Article · Jan 2012 · AMIA ... Annual Symposium proceedings / AMIA Symposium. AMIA Symposium
Show more