The Effects of On-Screen, Point of Care Computer Reminders on Processes and Outcomes of Care

Director, University of Toronto Centre for Patient Safety, Sunnybrook Health Sciences Centre, Room D474, 2075 Bayview Avenue, Toronto, Ontario, Canada, M4N 3M5.
Cochrane database of systematic reviews (Online) (Impact Factor: 6.03). 02/2009; 3(3):CD001096. DOI: 10.1002/14651858.CD001096.pub2
Source: PubMed


The opportunity to improve care by delivering decision support to clinicians at the point of care represents one of the main incentives for implementing sophisticated clinical information systems. Previous reviews of computer reminder and decision support systems have reported mixed effects, possibly because they did not distinguish point of care computer reminders from e-mail alerts, computer-generated paper reminders, and other modes of delivering 'computer reminders'.
To evaluate the effects on processes and outcomes of care attributable to on-screen computer reminders delivered to clinicians at the point of care.
We searched the Cochrane EPOC Group Trials register, MEDLINE, EMBASE and CINAHL and CENTRAL to July 2008, and scanned bibliographies from key articles.
Studies of a reminder delivered via a computer system routinely used by clinicians, with a randomised or quasi-randomised design and reporting at least one outcome involving a clinical endpoint or adherence to a recommended process of care.
Two authors independently screened studies for eligibility and abstracted data. For each study, we calculated the median improvement in adherence to target processes of care and also identified the outcome with the largest such improvement. We then calculated the median absolute improvement in process adherence across all studies using both the median outcome from each study and the best outcome.
Twenty-eight studies (reporting a total of thirty-two comparisons) were included. Computer reminders achieved a median improvement in process adherence of 4.2% (interquartile range (IQR): 0.8% to 18.8%) across all reported process outcomes, 3.3% (IQR: 0.5% to 10.6%) for medication ordering, 3.8% (IQR: 0.5% to 6.6%) for vaccinations, and 3.8% (IQR: 0.4% to 16.3%) for test ordering. In a sensitivity analysis using the best outcome from each study, the median improvement was 5.6% (IQR: 2.0% to 19.2%) across all process measures and 6.2% (IQR: 3.0% to 28.0%) across measures of medication ordering. In the eight comparisons that reported dichotomous clinical endpoints, intervention patients experienced a median absolute improvement of 2.5% (IQR: 1.3% to 4.2%). Blood pressure was the most commonly reported clinical endpoint, with intervention patients experiencing a median reduction in their systolic blood pressure of 1.0 mmHg (IQR: 2.3 mmHg reduction to 2.0 mmHg increase).
Point of care computer reminders generally achieve small to modest improvements in provider behaviour. A minority of interventions showed larger effects, but no specific reminder or contextual features were significantly associated with effect magnitude. Further research must identify design features and contextual factors consistently associated with larger improvements in provider behaviour if computer reminders are to succeed on more than a trial and error basis.

22 Reads
  • Source
    • "Such automatic systems combine medical evidence with patient-specific data from the Electronic Patient Record (EPR), which supports clinical decision making [9-11]. According to a Cochrane Review of 28 studies, computer reminders achieved a median improvement in process adherence of 4.2% [12]. Focused computer-generated reminders and alerts work well in a variety of single conditions [13-16] and in preventive care [17]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Computer-based decision support systems are a promising method for incorporating research evidence into clinical practice. However, evidence is still scant on how such information technology solutions work in primary healthcare when support is provided across many health problems. In Finland, we designed a trial where a set of evidence-based, patient-specific reminders was introduced into the local Electronic Patient Record (EPR) system. The aim was to measure the effects of such reminders on patient care. The hypothesis was that the total number of triggered reminders would decrease in the intervention group compared with the control group, indicating an improvement in patient care. From July 2009 to October 2010 all the patients of one health center were randomized to an intervention or a control group. The intervention consisted of patient-specific reminders concerning 59 different health conditions triggered when the healthcare professional (HCP) opened and used the EPR. In the intervention group, the triggered reminders were shown to the HCP; in the control group, the triggered reminders were not shown. The primary outcome measure was the change in the number of reminders triggered over 12 months. We developed a unique data gathering method, the Repeated Study Virtual Health Check (RSVHC), and used Generalized Estimation Equations (GEE) for analysing the incidence rate ratio, which is a measure of the relative difference in percentage change in the numbers of reminders triggered in the intervention group and the control group. In total, 13,588 participants were randomized and included. Contrary to our expectation, the total number of reminders triggered increased in both the intervention and the control groups. The primary outcome measure did not show a significant difference between the groups. However, with the inclusion of patients followed up over only six months, the total number of reminders increased significantly less in the intervention group than in the control group when the confounding factors (age, gender, number of diagnoses and medications) were controlled for. Computerized, tailored reminders in primary care did not decrease during the 12 months of follow-up time after the introduction of a patient-specific decision support system.Trial registration: NCT00915304.
    Implementation Science 01/2014; 9(1):15. DOI:10.1186/1748-5908-9-15 · 4.12 Impact Factor
  • Source
    • "Knowledge translation (KT) strategies including workshops [14], mentoring [15], outreach visits [16], audit and feedback [17] and reminders and memos [18] aim to embed research into practice and lead to small to moderate changes in health professionals’ behavior. Even though KT is an emergent science, it is known that KT strategies should be tailored to be context specific, and planned in response to a thorough assessment of barriers and facilitators [19,20]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: It is difficult to foster research utilization among allied health professionals (AHPs). Tailored, multifaceted knowledge translation (KT) strategies are now recommended but are resource intensive to implement. Employers need effective KT solutions but little is known about; the impact and viability of multifaceted KT strategies using an online KT tool, their effectiveness with AHPs and their effect on evidence-based practice (EBP) decision-making behavior. The study aim was to measure the effectiveness of a multifaceted KT intervention including a customized KT tool, to change EBP behavior, knowledge, and attitudes of AHPs. This is an evaluator-blinded, cluster randomized controlled trial conducted in an Australian community-based cerebral palsy service. 135 AHPs (physiotherapists, occupational therapists, speech pathologists, psychologists and social workers) from four regions were cluster randomized (n = 4), to either the KT intervention group (n = 73 AHPs) or the control group (n = 62 AHPs), using computer-generated random numbers, concealed in opaque envelopes, by an independent officer. The KT intervention included three-day skills training workshop and multifaceted workplace supports to redress barriers (paid EBP time, mentoring, system changes and access to an online research synthesis tool). Primary outcome (self- and peer-rated EBP behavior) was measured using the Goal Attainment Scale (individual level). Secondary outcomes (knowledge and attitudes) were measured using exams and the Evidence Based Practice Attitude Scale. The intervention group's primary outcome scores improved relative to the control group, however when clustering was taken into account, the findings were non-significant: self-rated EBP behavior [effect size 4.97 (95% CI -10.47, 20.41)(p = 0.52)]; peer-rated EBP behavior [effect size 5.86 (95% CI -17.77, 29.50)(p = 0.62)]. Statistically significant improvements in EBP knowledge were detected [effect size 2.97 (95% CI 1.97, 3.97(p < 0.0001)]. Change in EBP attitudes was not statistically significant. Improvement in EBP behavior was not statistically significant after adjusting for cluster effect, however similar improvements from peer-ratings suggest behaviorally meaningful gains. The large variability in behavior observed between clusters suggests barrier assessments and subsequent KT interventions may need to target subgroups within an organization.Trial registration: Registered on the Australian New Zealand Clinical Trials Registry (ACTRN12611000529943).
    Implementation Science 11/2013; 8(1):132. DOI:10.1186/1748-5908-8-132 · 4.12 Impact Factor
  • Source
    • "Recent studies have shown that physician adherence to evidence based and standardized medical care results in achieving adequate blood pressure control among hypertensive patients [10,11]. Clinical guidelines and algorithms at the point of health care delivery, in the form of decision making aids to the health care providers, such as clinical and or computerised decision support systems (DSS) are a possible way to improve the standard of care delivery, more so, in a resource constrained primary health care setting [11,12]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Hypertension remains the top global cause of disease burden. Decision support systems (DSS) could provide an adequate and cost-effective means to improve the management of hypertension at a primary health care (PHC) level in a developing country, nevertheless evidence on this regard is rather limited. Development of DSS software was based on an algorithmic approach for (a) evaluation of a hypertensive patient, (b) risk stratification (c) drug management and (d) lifestyle interventions, based on Indian guidelines for hypertension II (2007). The beta testing of DSS software involved a feedback from the end users of the system on the contents of the user interface. Software validation and piloting was done in field, wherein the virtual recommendations and advice given by the DSS were compared with two independent experts (government doctors from the non-participating PHC centers). The overall percent agreement between the DSS and independent experts among 60 hypertensives on drug management was 85% (95% CI: 83.61 - 85.25). The kappa statistic for overall agreement for drug management was 0.659 (95% CI: 0.457 - 0.862) indicating a substantial degree of agreement beyond chance at an alpha fixed at 0.05 with 80% power. Receiver operator curve (ROC) showed a good accuracy for the DSS, wherein, the area under curve (AUC) was 0.848 (95% CI: 0.741 - 0.948). Sensitivity and specificity of the DSS were 83.33 and 85.71% respectively when compared with independent experts. A point of care, pilot tested and validated DSS for management of hypertension has been developed in a resource constrained low and middle income setting and could contribute to improved management of hypertension at a primary health care level.
    PLoS ONE 11/2013; 8(11):e79638. DOI:10.1371/journal.pone.0079638 · 3.23 Impact Factor
Show more