A systematic review to evaluate the accuracy of electronic adverse drug event detection

Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada.
Journal of the American Medical Informatics Association (Impact Factor: 3.5). 01/2012; 19(1):31-8. DOI: 10.1136/amiajnl-2011-000454
Source: PubMed


Adverse drug events (ADEs), defined as adverse patient outcomes caused by medications, are common and difficult to detect. Electronic detection of ADEs is a promising method to identify ADEs. We performed this systematic review to characterize established electronic detection systems and their accuracy.
We identified studies evaluating electronic ADE detection from the MEDLINE and EMBASE databases. We included studies if they contained original data and involved detection of electronic triggers using information systems. We abstracted data regarding rule characteristics including type, accuracy, and rationale.
Forty-eight studies met our inclusion criteria. Twenty-four (50%) studies reported rule accuracy but only 9 (18.8%) utilized a proper gold standard (chart review in all patients). Rule accuracy was variable and often poor (range of sensitivity: 40%-94%; specificity: 1.4%-89.8%; positive predictive value: 0.9%-64%). 5 (10.4%) studies derived or used detection rules that were defined by clinical need or the underlying ADE prevalence. Detection rules in 8 (16.7%) studies detected specific types of ADEs.
Several factors led to inaccurate ADE detection algorithms, including immature underlying information systems, non-standard event definitions, and variable methods for detection rule validation. Few ADE detection algorithms considered clinical priorities. To enhance the utility of electronic detection systems, there is a need to systematically address these factors.

Download full-text


Available from: Alison Jennings,
149 Reads
  • Source
    • "They included "studies if they contained original data and involved detection of electronic triggers using information systems"[15]. "They abstracted data regarding rule characteristics including type, accuracy, and rational "[15]. Honigman et al. also developed a program that combines four computer search methods, including text searching of the electronic medical record, to detect ADEs in outpatient settings[16]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: BackgroundAntibiotics are the widely prescribed drugs for children and most likely to be related with adverse reactions. Record on adverse reactions and allergies from antibiotics considerably affect the prescription choices. We consider this a biomedical decision-making problem and explore hidden knowledge in survey results on data extracted from a big data pool of health records of children, from the Health Center of Osijek, Eastern Croatia.ResultsWe applied and evaluated a k-means algorithm to the dataset to generate some clusters which have similar features. Our results highlight that some type of antibiotics form different clusters, which insight is most helpful for the clinician to support better decision-making.ConclusionsMedical professionals can investigate the clusters which our study revealed, thus gaining useful knowledge and insight into this data for their clinical studies.
    BMC Bioinformatics 05/2014; 15(Suppl 6):S7. DOI:10.1186/1471-2105-15-S6-S7 · 2.58 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: We present and test the intuition that letters to the editor in journals carry early signals of adverse drug events (ADEs). Surprisingly these letters have not yet been exploited for automatic ADE detection unlike for example, clinical records and PubMed. Part of the challenge is that it is not easy to access the full-text of letters (for the most part these do not appear in PubMed). Also letters are likely underrated in comparison with full articles. Besides demonstrating that this intuition holds we contribute techniques for post market drug surveillance. Specifically, we test an automatic approach for ADE detection from letters using off-the-shelf machine learning tools. We also involve natural language processing for feature definitions. Overall we achieve high accuracy in our experiments and our method also works well on a second new test set. Our results encourage us to further pursue this line of research.
    AMIA ... Annual Symposium proceedings / AMIA Symposium. AMIA Symposium 01/2012; 2012:1030-9.
  • [Show abstract] [Hide abstract]
    ABSTRACT: AIM: This review examines the effectiveness of detection methods in terms of their ability to identify and accurately determine medication-related problems in hospitals. METHODS: A search was conducted of databases from inception to June 2012. The following keywords were used in combination: medication error or adverse drug event or adverse drug reaction, comparison, detection, hospital and method. Seven detection methods were considered: chart review; claims data review; computer monitoring; direct care observation; interviews; prospective data collection; and incident reporting. RESULTS: Forty relevant studies were located. Detection methods that were better able to identify medication-related problems compared with other methods tested in the same study included chart review, computer monitoring, direct care observation and prospective data collection. However, only small numbers of studies were involved in comparisons with direct care observation (n=5) and prospective data collection (n=6). There was little focus on detecting medication-related problems during various stages of the medication process, and comparisons associated with the seriousness of medication-related problems were examined in 19 studies. Only 17 studies involved appropriate comparisons with a gold standard, which provided details about sensitivities and specificities. CONCLUSIONS: In view of the relatively low identification of medication-related problems with incident reporting, use of this method in tracking trends over time should be met with some scepticism. Greater attention should be placed on combining methods, such as chart review and computer monitoring in examining trends. More research is needed on the use of claims data, direct care observation, interviews and prospective data collection as detection methods.
    British Journal of Clinical Pharmacology 11/2012; 76(1). DOI:10.1111/bcp.12049 · 3.88 Impact Factor
Show more