Impact of an Educational Intervention Designed to Reduce Unnecessary Recall during Screening Mammography
ABSTRACT The aim of this study was to describe the impact of a tailored Web-based educational program designed to reduce excessive screening mammography recall.
Radiologists enrolled in one of four mammography registries in the United States were invited to take part and were randomly assigned to receive the intervention or to serve as controls. The controls were offered the intervention at the end of the study, and data collection included an assessment of their clinical practice as well. The intervention provided each radiologist with individual audit data for his or her sensitivity, specificity, recall rate, positive predictive value, and cancer detection rate compared to national benchmarks and peer comparisons for the same measures; profiled breast cancer risk in each radiologist's respective patient populations to illustrate how low breast cancer risk is in population-based settings; and evaluated the possible impact of medical malpractice concerns on recall rates. Participants' recall rates from actual practice were evaluated for three time periods: the 9 months before the intervention was delivered to the intervention group (baseline period), the 9 months between the intervention and control groups (T1), and the 9 months after completion of the intervention by the controls (T2). Logistic regression models examining the probability that a mammogram was recalled included indication of intervention versus control and time period (baseline, T1, and T2). Interactions between the groups and time period were also included to determine if the association between time period and the probability of a positive result differed across groups.
Thirty-one radiologists who completed the continuing medical education intervention were included in the adjusted model comparing radiologists in the intervention group (n = 22) to radiologists who completed the intervention in the control group (n = 9). At T1, the intervention group had 12% higher odds of positive mammographic results compared to the controls, after controlling for baseline (odds ratio, 1.12; 95% confidence interval, 1.00-1.27; P = .0569). At T2, a similar association was found, but it was not statistically significant (odds ratio, 1.10; 95% confidence interval, 0.96 to 1.25). No associations were found among radiologists in the control group when comparing those who completed the continuing medical education intervention (n = 9) to those who did not (n = 10). In addition, no associations were found between time period and recall rate among radiologists who set realistic goals.
This study resulted in a null effect, which may indicate that a single 1-hour intervention is not adequate to change excessive recall among radiologists who undertook the intervention being tested.
- [Show abstract] [Hide abstract]
ABSTRACT: In February 2010, our radiology department adopted the use of the Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 criteria for newly diagnosed oncology patients. Prior to staff used RECIST 1.1, we hypothesized that education and feedback interventions could help clarify differences between RECIST 1.0 and the newly adopted RECIST 1.1 guidelines and result in appropriate and accurate utilization of both reporting systems. This study evaluates the effect of education and feedback interventions on the accuracy of computed tomography (CT) reporting using RECIST criteria. Consecutive CT scan reports and images were retrospectively reviewed during three different periods to assess for compliance and adherence to RECIST guidelines. Data collected included interpreting faculty, resident, type, and total number of errors per report. Significance testing of differences between cohorts was performed using an unequal variance t-test. Group 1 (baseline): RECIST 1.0 used; prior to adoption of RECIST 1.1 criteria. Group 2 (post distributed educational materials): Following adoption of RECIST 1.1 criteria and distribution of educational materials. Group 3 (post audit and feedback): Following the audit and feedback intervention. The percentage of reports with errors decreased from 30% (baseline) to 28% (group 2) to 22% (group 3). Only the difference in error rate between the baseline and group 3 was significant (P = .03). The combination of distributed educational materials and audit and feedback interventions improved the quality of radiology reports requiring RECIST criteria by reducing the number of studies with errors.Academic radiology 03/2013; 20(3):351-7. DOI:10.1016/j.acra.2012.12.002 · 2.08 Impact Factor
- [Show abstract] [Hide abstract]
ABSTRACT: To evaluate appropriate utilization rates for computed tomography (CT) pulmonary angiography (CTPA) in a tertiary center emergency department (ED), before and after a health care provider educational intervention. Institutional Review Board-approved retrospective study. Records for 100 consecutive CTPA studies ordered by the ED were retrieved from a radiology database. Appropriateness rates for the studies were determined using information from existing literature (clinical decision rules and society guidelines). Where pretest probability was not performed, it was calculated by the authors. After ED health care provider education regarding appropriateness guidelines through a dedicated lecture and question-and-answer session, appropriateness rates for another 100 consecutive CTPA ordered by the ED were calculated. In the preeducational intervention, 1% of patients had Wells scores performed, 65% were women, and 29% were age <40 years. Before CTPA, 40% patients had d-dimer testing, 15% of patients had a "negative" d-dimer, 17% had alternative explanations for chest pain, and 76% had low or intermediate pretest probability. Appropriateness rates for CTPA was 7%, and 8% of studies were positive. Postintervention, no Wells scores were performed, 59% were women, and 34% <40 years. Before CTPA, 32% of patients had d-dimer, 16% had a "negative" d-dimer, 22% had alternative explanations for chest pain, and 84% had low or intermediate pretest probability. The appropriateness rate for CTPA was 6% and 10% of studies were positive. A single educational intervention had no effect on appropriate utilization rates for CTPA. Repeated and sustained educational interventions may help improve imaging ordering pathways through the ED and other departments.Academic radiology 09/2013; 20(9):1107-14. DOI:10.1016/j.acra.2013.05.005 · 2.08 Impact Factor
- [Show abstract] [Hide abstract]
ABSTRACT: OBJECTIVE. The objective of our study was to conduct a randomized controlled trial of educational interventions that were created to improve performance of screening mammography interpretation. MATERIALS AND METHODS. We randomly assigned physicians who interpret mammography to one of three groups: self-paced DVD, live expert-led educational seminar, or control. The DVD and seminar interventions used mammography cases of varying difficulty and provided associated teaching points. Interpretive performance was compared using a pretest-posttest design. Sensitivity, specificity, and positive predictive value (PPV) were calculated relative to two outcomes: cancer status and consensus of three experts about recall. The performance measures for each group were compared using logistic regression adjusting for pretest performance. RESULTS. One hundred two radiologists completed all aspects of the trial. After adjustment for preintervention performance, the odds of improved sensitivity for correctly identifying a lesion relative to expert recall were 1.34 times higher for DVD participants than for control subjects (95% CI, 1.00-1.81; p = 0.050). The odds of an improved PPV for correctly identifying a lesion relative to both expert recall (odds ratio [OR] = 1.94; 95% CI, 1.24-3.05; p = 0.004) and cancer status (OR = 1.81; 95% CI, 1.01-3.23; p = 0.045) were significantly improved for DVD participants compared with control subjects, with no significant change in specificity. For the seminar group, specificity was significantly lower than the control group (OR relative to expert recall = 0.80; 95% CI, 0.64-1.00; p = 0.048; OR relative to cancer status = 0.79; 95% CI, 0.65-0.95; p = 0.015). CONCLUSION. In this randomized controlled trial, the DVD educational intervention resulted in a significant improvement in screening mammography interpretive performance on a test set, which could translate into improved interpretative performance in clinical practice.American Journal of Roentgenology 06/2014; 202(6):W586-96. DOI:10.2214/AJR.13.11147 · 2.74 Impact Factor