Using a Tailored Web-based Intervention to Set Goals to Reduce Unnecessary Recall

Departments of Family Medicine and Public Health and Preventive Medicine, Oregon Health & Science University, Portland, 97239-3098, USA.
Academic radiology (Impact Factor: 1.75). 04/2011; 18(4):495-503. DOI: 10.1016/j.acra.2010.11.017
Source: PubMed


To examine whether an intervention strategy consisting of a tailored web-based intervention, which provides individualized audit data with peer comparisons and other data that can affect recall, can assist radiologists in setting goals for reducing unnecessary recall.
In a multisite randomized controlled study, we used a tailored web-based intervention to assess radiologists' ability to set goals to improve interpretive performance. The intervention provided peer comparison audit data, profiled breast cancer risk in each radiologist's respective patient populations, and evaluated the possible impact of medical malpractice concerns. We calculated the percentage of radiologists who would consider changing their recall rates, and examined the specific goals they set to reduce recall rates. We describe characteristics of radiologists who developed realistic goals to reduce their recall rates, and their reactions to the importance of patient risk factors and medical malpractice concerns.
Forty-one of 46 radiologists (89.1%) who started the intervention completed it. Thirty-one (72.1%) indicated they would like to change their recall rates and 30 (69.8%) entered a text response about changing their rates. Sixteen of the 30 (53.3%) radiologists who included a text response set realistic goals that would likely result in reducing unnecessary recall. The actual recall rates of those who set realistic goals were not statistically different from those who did not (13.8% vs. 15.1%, respectively). The majority of selected goals involved re-reviewing cases initially interpreted as Breast Imaging Reporting and Data System category 0. More than half of radiologists who commented on the influence of patient risk (56.3%) indicated that radiologists planned to pay more attention to risk factors, and 100% of participants commented on concerns radiologists have about malpractice with the primary concern (37.5%) being fear of lawsuits.
Interventions designed to reduce unnecessary recall can succeed in assisting radiologists to develop goals that may ultimately reduce unnecessary recall.

Download full-text


Available from: Diana Miglioretti
  • [Show abstract] [Hide abstract]
    ABSTRACT: Mammography quality assurance programs have been in place for more than a decade. We studied radiologists' self-reported performance goals for accuracy in screening mammography and compared them to published recommendations. A mailed survey of radiologists at mammography registries in seven states within the Breast Cancer Surveillance Consortium (BCSC) assessed radiologists' performance goals for interpreting screening mammograms. Self-reported goals were compared to published American College of Radiology (ACR) recommended desirable ranges for recall rate, false-positive rate, positive predictive value of biopsy recommendation (PPV2), and cancer detection rate. Radiologists' goals for interpretive accuracy within desirable range were evaluated for associations with their demographic characteristics, clinical experience, and receipt of audit reports. The survey response rate was 71% (257 of 364 radiologists). The percentage of radiologists reporting goals within desirable ranges was 79% for recall rate, 22% for false-positive rate, 39% for PPV2, and 61% for cancer detection rate. The range of reported goals was 0%-100% for false-positive rate and PPV2. Primary academic affiliation, receiving more hours of breast imaging continuing medical education, and receiving audit reports at least annually were associated with desirable PPV2 goals. Radiologists reporting desirable cancer detection rate goals were more likely to have interpreted mammograms for 10 or more years, and >1000 mammograms per year. Many radiologists report goals for their accuracy when interpreting screening mammograms that fall outside of published desirable benchmarks, particularly for false-positive rate and PPV2, indicating an opportunity for education.
    No preview · Article · Nov 2011 · Academic radiology
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: The aim of this study was to describe the impact of a tailored Web-based educational program designed to reduce excessive screening mammography recall. Radiologists enrolled in one of four mammography registries in the United States were invited to take part and were randomly assigned to receive the intervention or to serve as controls. The controls were offered the intervention at the end of the study, and data collection included an assessment of their clinical practice as well. The intervention provided each radiologist with individual audit data for his or her sensitivity, specificity, recall rate, positive predictive value, and cancer detection rate compared to national benchmarks and peer comparisons for the same measures; profiled breast cancer risk in each radiologist's respective patient populations to illustrate how low breast cancer risk is in population-based settings; and evaluated the possible impact of medical malpractice concerns on recall rates. Participants' recall rates from actual practice were evaluated for three time periods: the 9 months before the intervention was delivered to the intervention group (baseline period), the 9 months between the intervention and control groups (T1), and the 9 months after completion of the intervention by the controls (T2). Logistic regression models examining the probability that a mammogram was recalled included indication of intervention versus control and time period (baseline, T1, and T2). Interactions between the groups and time period were also included to determine if the association between time period and the probability of a positive result differed across groups. Thirty-one radiologists who completed the continuing medical education intervention were included in the adjusted model comparing radiologists in the intervention group (n = 22) to radiologists who completed the intervention in the control group (n = 9). At T1, the intervention group had 12% higher odds of positive mammographic results compared to the controls, after controlling for baseline (odds ratio, 1.12; 95% confidence interval, 1.00-1.27; P = .0569). At T2, a similar association was found, but it was not statistically significant (odds ratio, 1.10; 95% confidence interval, 0.96 to 1.25). No associations were found among radiologists in the control group when comparing those who completed the continuing medical education intervention (n = 9) to those who did not (n = 10). In addition, no associations were found between time period and recall rate among radiologists who set realistic goals. This study resulted in a null effect, which may indicate that a single 1-hour intervention is not adequate to change excessive recall among radiologists who undertook the intervention being tested.
    Full-text · Article · Jun 2012 · Academic radiology
  • [Show abstract] [Hide abstract]
    ABSTRACT: Purpose: To examine radiologists' screening performance in relation to the number of diagnostic work-ups performed after abnormal findings are discovered at screening mammography by the same radiologist or by different radiologists. Materials and methods: In an institutional review board-approved HIPAA-compliant study, the authors linked 651 671 screening mammograms interpreted from 2002 to 2006 by 96 radiologists in the Breast Cancer Surveillance Consortium to cancer registries (standard of reference) to evaluate the performance of screening mammography (sensitivity, false-positive rate [ FPR false-positive rate ], and cancer detection rate [ CDR cancer detection rate ]). Logistic regression was used to assess the association between the volume of recalled screening mammograms ("own" mammograms, where the radiologist who interpreted the diagnostic image was the same radiologist who had interpreted the screening image, and "any" mammograms, where the radiologist who interpreted the diagnostic image may or may not have been the radiologist who interpreted the screening image) and screening performance and whether the association between total annual volume and performance differed according to the volume of diagnostic work-up. Results: Annually, 38% of radiologists performed the diagnostic work-up for 25 or fewer of their own recalled screening mammograms, 24% performed the work-up for 0-50, and 39% performed the work-up for more than 50. For the work-up of recalled screening mammograms from any radiologist, 24% of radiologists performed the work-up for 0-50 mammograms, 32% performed the work-up for 51-125, and 44% performed the work-up for more than 125. With increasing numbers of radiologist work-ups for their own recalled mammograms, the sensitivity (P = .039), FPR false-positive rate (P = .004), and CDR cancer detection rate (P < .001) of screening mammography increased, yielding a stepped increase in women recalled per cancer detected from 17.4 for 25 or fewer mammograms to 24.6 for more than 50 mammograms. Increases in work-ups for any radiologist yielded significant increases in FPR false-positive rate (P = .011) and CDR cancer detection rate (P = .001) and a nonsignificant increase in sensitivity (P = .15). Radiologists with a lower annual volume of any work-ups had consistently lower FPR false-positive rate , sensitivity, and CDR cancer detection rate at all annual interpretive volumes. Conclusion: These findings support the hypothesis that radiologists may improve their screening performance by performing the diagnostic work-up for their own recalled screening mammograms and directly receiving feedback afforded by means of the outcomes associated with their initial decision to recall. Arranging for radiologists to work up a minimum number of their own recalled cases could improve screening performance but would need systems to facilitate this workflow.
    No preview · Article · Jun 2014 · Radiology