Automated detection of harm in healthcare with information technology: a systematic review

Dartmouth-Hitchcock Medical Center, One Medical Center Drive, Lebanon, NH 03756, USA.
Quality and Safety in Health Care (Impact Factor: 2.16). 10/2010; 19(5):e11. DOI: 10.1136/qshc.2009.033027
Source: PubMed

ABSTRACT To improve patient safety, healthcare facilities are focussing on reducing patient harm. Automated harm-detection methods using information technology show promise for efficiently measuring harm. However, there have been few systematic reviews of their effectiveness.
To perform a systematic literature review to identify, describe and evaluate effectiveness of automated inpatient harm-detection methods.
Data sources included MEDLINE and CINAHL databases indexed through August 2008, extended by bibliographic review and search of citing articles. The authors included articles reporting effectiveness of automated inpatient harm-detection methods, as compared with other detection methods. Two independent reviewers used a standardised abstraction sheet to extract data about automated and comparison harm-detection methods, patient samples and events identified. Differences were resolved by discussion.
From 176 articles, 43 articles met inclusion criteria: 39 describing field-defined methods, two using natural language processing and two using both methods. Twenty-one studies used automated methods to detect adverse drug events, 10 detected general adverse events, eight detected nosocomial infections, and four detected other specific adverse events. Compared with gold standard chart review, sensitivity and specificity of automated harm-detection methods ranged from 0.10 to 0.94 and 0.23 to 0.98, respectively. Studies used heterogeneous methods that often were flawed.
Automated methods of harm detection are feasible and some can potentially detect patient harm efficiently. However, effectiveness varied widely, and most studies had methodological weaknesses. More work is needed to develop and assess these tools before they can yield accurate estimates of harm that can be reliably interpreted and compared.

  • Source
    American Journal of Respiratory and Critical Care Medicine; 01/2013
  • [Show abstract] [Hide abstract]
    ABSTRACT: Many systematic reviews exist on interventions to improve safe and effective medicines use by consumers, but research is distributed across diseases, populations and settings. The scope and focus of such reviews also vary widely, creating challenges for decision-makers seeking to inform decisions by using the evidence on consumers' medicines use.This is an update of a 2011 overview of systematic reviews, which synthesises the evidence, irrespective of disease, medicine type, population or setting, on the effectiveness of interventions to improve consumers' medicines use. To assess the effects of interventions which target healthcare consumers to promote safe and effective medicines use, by synthesising review-level evidence. Search methods: We included systematic reviews published on the Cochrane Database of Systematic Reviews and the Database of Abstracts of Reviews of Effects. We identified relevant reviews by handsearching databases from their start dates to March 2012. Selection criteria: We screened and ranked reviews based on relevance to consumers' medicines use, using criteria developed for this overview. Data collection and analysis: We used standardised forms to extract data, and assessed reviews for methodological quality using the AMSTAR tool. We used standardised language to summarise results within and across reviews; and gave bottom-line statements about intervention effectiveness. Two review authors screened and selected reviews, and extracted and analysed data. We used a taxonomy of interventions to categorise reviews and guide syntheses. We included 75 systematic reviews of varied methodological quality. Reviews assessed interventions with diverse aims including support for behaviour change, risk minimisation and skills acquisition. No reviews aimed to promote systems-level consumer participation in medicines-related activities. Medicines adherence was the most frequently-reported outcome, but others such as knowledge, clinical and service-use outcomes were also reported. Adverse events were less commonly identified, while those associated with the interventions themselves, or costs, were rarely reported.Looking across reviews, for most outcomes, medicines self-monitoring and self-management programmes appear generally effective to improve medicines use, adherence, adverse events and clinical outcomes; and to reduce mortality in people self-managing antithrombotic therapy. However, some participants were unable to complete these interventions, suggesting they may not be suitable for everyone.Other promising interventions to improve adherence and other key medicines-use outcomes, which require further investigation to be more certain of their effects, include:· simplified dosing regimens: with positive effects on adherence;· interventions involving pharmacists in medicines management, such as medicines reviews (with positive effects on adherence and use, medicines problems and clinical outcomes) and pharmaceutical care services (consultation between pharmacist and patient to resolve medicines problems, develop a care plan and provide follow-up; with positive effects on adherence and knowledge).Several other strategies showed some positive effects, particularly relating to adherence, and other outcomes, but their effects were less consistent overall and so need further study. These included:· delayed antibiotic prescriptions: effective to decrease antibiotic use but with mixed effects on clinical outcomes, adverse effects and satisfaction;· practical strategies like reminders, cues and/or organisers, reminder packaging and material incentives: with positive, although somewhat mixed effects on adherence;· education delivered with self-management skills training, counselling, support, training or enhanced follow-up; information and counselling delivered together; or education/information as part of pharmacist-delivered packages of care: with positive effects on adherence, medicines use, clinical outcomes and knowledge, but with mixed effects in some studies;· financial incentives: with positive, but mixed, effects on adherence.Several strategies also showed promise in promoting immunisation uptake, but require further study to be more certain of their effects. These included organisational interventions; reminders and recall; financial incentives; home visits; free vaccination; lay health worker interventions; and facilitators working with physicians to promote immunisation uptake. Education and/or information strategies also showed some positive but even less consistent effects on immunisation uptake, and need further assessment of effectiveness and investigation of heterogeneity.There are many different potential pathways through which consumers' use of medicines could be targeted to improve outcomes, and simple interventions may be as effective as complex strategies. However, no single intervention assessed was effective to improve all medicines-use outcomes across all diseases, medicines, populations or settings.Even where interventions showed promise, the assembled evidence often only provided part of the picture: for example, simplified dosing regimens seem effective for improving adherence, but there is not yet sufficient information to identify an optimal regimen.In some instances interventions appear ineffective: for example, the evidence suggests that directly observed therapy may be generally ineffective for improving treatment completion, adherence or clinical outcomes.In other cases, interventions may have variable effects across outcomes. As an example, strategies providing information or education as single interventions appear ineffective to improve medicines adherence or clinical outcomes, but may be effective to improve knowledge; an important outcome for promoting consumers' informed medicines choices.Despite a doubling in the number of reviews included in this updated overview, uncertainty still exists about the effectiveness of many interventions, and the evidence on what works remains sparse for several populations, including children and young people, carers, and people with multimorbidity. This overview presents evidence from 75 reviews that have synthesised trials and other studies evaluating the effects of interventions to improve consumers' medicines use.Systematically assembling the evidence across reviews allows identification of effective or promising interventions to improve consumers' medicines use, as well as those for which the evidence indicates ineffectiveness or uncertainty.Decision makers faced with implementing interventions to improve consumers' medicines use can use this overview to inform decisions about which interventions may be most promising to improve particular outcomes. The intervention taxonomy may also assist people to consider the strategies available in relation to specific purposes, for example, gaining skills or being involved in decision making. Researchers and funders can use this overview to identify where more research is needed and assess its priority. The limitations of the available literature due to the lack of evidence for important outcomes and important populations, such as people with multimorbidity, should also be considered in practice and policy decisions.
    Cochrane database of systematic reviews (Online) 04/2014; 4(4):CD007768. DOI:10.1002/14651858.CD007768.pub3 · 5.94 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Background Adverse events are associated with significant morbidity, mortality and cost in hospitalized patients. Measuring adverse events is necessary for quality improvement, but current detection methods are inaccurate, untimely and expensive. The advent of electronic health records and the development of automated methods for encoding and classifying electronic narrative data, such as natural language processing, offer an opportunity to identify potentially better methods. The objective of this study is to determine the accuracy of using automated methods for detecting three highly prevalent adverse events: a) hospital-acquired pneumonia, b) catheter-associated bloodstream infections, and c) in-hospital falls.Methods/designThis validation study will be conducted at two large Canadian academic health centres: the McGill University Health Centre (MUHC) and The Ottawa Hospital (TOH). The study population consists of all medical, surgical and intensive care unit patients admitted to these centres between 2008 and 2014. An automated detection algorithm will be developed and validated for each of the three adverse events using electronic data extracted from multiple clinical databases. A random sample of MUHC patients will be used to develop the automated detection algorithms (cohort 1, development set). The accuracy of these algorithms will be assessed using chart review as the reference standard. Then, receiver operating characteristic curves will be used to identify optimal cut points for each of the data sources. Multivariate logistic regression and the areas under curve (AUC) will be used to identify the optimal combination of data sources that maximize the accuracy of adverse event detection. The most accurate algorithms will then be validated on a second random sample of MUHC patients (cohort 1, validation set), and accuracy will be measured using chart review as the reference standard. The most accurate algorithms validated at the MUHC will then be applied to TOH data (cohort 2), and their accuracy will be assessed using a reference standard assessment of the medical chart.DiscussionThere is a need for more accurate, timely and efficient measures of adverse events in acute care hospitals. This is a critical requirement for evaluating the effectiveness of preventive interventions and for tracking progress in patient safety through time.
    Implementation Science 01/2015; 10(1):5. DOI:10.1186/s13012-014-0197-6 · 3.47 Impact Factor