Evaluation of an audit with feedback continuing medical education program for radiation oncologists
ABSTRACT Meta-analyses demonstrate audit with feedback (AWF) is effective continuing medical education (CME). However, efficacy varies between specialties, with little published radiation oncologist (RO)-specific evidence. We evaluated an AWF CME intervention for ROs determining efficacy, cost-effectiveness, and participant satisfaction.
CME program: The CME incorporated fortnightly random patient chart audit, scoring management adequacy via a checklist. Scores were presented at a same-day institutional meeting, and case management discussed. Senior peers provided individualized, educational feedback.
Changes in behavior and performance were evaluated by chart review of new patients seen by ROs in the 2 months before commencement of AWF (T0), and at months 13-14 of the program (T1). Behavior and performance were evaluated with a validated, reproducible, 19-item instrument. Criteria for each case audited included 10 targeted and 3 nontargeted behavior items and 6 performance items; each scored 1 point if deemed adequate (maximum score 19). Cost-effectiveness was reported as cost to the institution per item point gained. The mean score (out of 5) of a 14-item questionnaire evaluated program perception.
A total of 113 and 118 charts were evaluated at T0 and T1, respectively. Mean score of targeted behavior improved between T0 and T1 (from 8.7 to 9.2 out of 10, P = .0001), with no significant improvement of nontargeted behavior/performance items. Annual costs and cost-per-point gained were US 7,897 dollars and 15 dollars. Participant satisfaction was positive, increasing after efficacy result distribution (P = .0001).
Audit with comparative, individualized, educational feedback is cost-effective and positively perceived CME, significantly improving targeted RO behavior. Oncologists' CME design and evaluation require further research.
[Show abstract] [Hide abstract]
ABSTRACT: Purpose Physician peer review seeks to improve the quality of care through the evaluation of physician performance, specifically medical decision making and technical expertise. To establish current peer review practice patterns, evaluate interest in recommendations for peer review, and establish a framework for future recommendations, the American Society for Radiation Oncology (ASTRO) surveyed its physician members. Methods and materials A radiation oncology-specific peer review survey instrument was developed, formally tested, and found to meet established levels of reliability and validity. The final instrument was delivered using a web-based survey platform including reminders. All ASTRO physician-members and members-in-training worldwide were invited by email to participate. Results A total of 5674 physicians were contacted starting in January 2013. A total of 572 physicians participated (10%) yielding a ± 4% margin of error. Those responding were split evenly between academic providers and private practice and others. The median time since training = 16 years, median number of new patients per year = 215, and median practice size = 6 physicians; 83% of respondents were involved in peer review and 75% were comfortable with their program. Of those involved, 65% report doing some review before radiation begins. Of patients treated by these physicians, 56% are reviewed before treatment. Peer review elements reviewed include overall treatment strategy (86%), dose and fractionation (89%), contouring (59%), and isodose or dose-volume histogram (75%). Ninety percent of physicians have changed radiation plans because of peer review. These providers make changes in 7%-10% of cases. Seventy-four percent of physicians agree that ASTRO should make formal peer review recommendations, with 7% in opposition. Conclusions This survey suggests that peer review in radiation oncology is common and leads to changes in management in a meaningful fraction of cases. There is much variation in the manner of conducting, and reported utility of, peer review. The majority of ASTRO physician members support formal recommendations and guidance on peer review.06/2014; 5(1). DOI:10.1016/j.prro.2014.04.004
[Show abstract] [Hide abstract]
ABSTRACT: Although the value of peer review is increasingly recognized, there is little research documenting its impact in the setting of stereotactic body radiation therapy (SBRT) for lung cancer. This study determines the dosimetric effect of peer review of tumor and normal tissue contouring in lung SBRT planning. Forty anonymized lung SBRT plans were retrospectively evaluated post treatment. Each plan was independently reviewed by two to three radiation oncologists using established institutional guidelines. For each structure, reviewers recorded recommendations for "no change," "minor change," "major change," or "missing contour" and provided a modified or new contour as needed. Dose-volume histograms were analyzed for dosimetric violations. Among 472 contoured structures evaluated, recommendations from peer review were 107 major change (23%), 176 minor change (37%), 157 no change (33%), and 32 missing (7%). Common major changes involved the skin (n = 20), heart (n = 18), and proximal bronchial tree (n = 15). Dose constraints were not achieved for 25 new or recontoured structures (5%), of which 17 involved the planning target volume (PTV). Among cases with PTV violations, the mean prescription dose coverage to the modified PTVs was 90%, compared with the protocol standard of greater than or equal to 95% coverage. The remaining violations involved the ribs (n = 5), spinal canal (n = 2), and heart (n = 1). Peer review of structure contouring resulted in significant changes in lung SBRT plans. Recontouring of several plans revealed violations of dose limits, most often involving inadequate PTV coverage. Peer review, especially of target volume delineation, is warranted to improve consistency and quality in lung SBRT planning.Journal of thoracic oncology: official publication of the International Association for the Study of Lung Cancer 04/2014; 9(4):527-33. DOI:10.1097/JTO.0000000000000119 · 4.55 Impact Factor
[Show abstract] [Hide abstract]
ABSTRACT: This study describes the process and outcomes of breast radiotherapy (RT) quality assurance (QA) rounds, seeking to identify variables associated with plan modifications. Real-time data were prospectively collected over 2years. Descriptive statistics determined the proportion of cases requiring no (A), minor (B), or major (C) modifications, which were then subjected to univariate and multivariate analyses. A total of 2223 breast cancer QA cases were reviewed; 47 cases (2.1%) underwent a minor, and 52 cases (2.3%) required a major modification. Common changes included boost, volume, seroma, and bolus. On univariate analysis, regional nodal irradiation (RNI), tumour size, and axillary node dissection were significantly associated with major modifications. Upon multivariate analysis, the only independent predictor was RNI (OR 2.12, p=0.0075). For patients with no RNI, <2cm tumours, no axillary lymph node dissection, and no boosts (n=420); the likelihood of category C was only 1.4%. It is feasible to conduct QA review for all breast cancer cases prior to commencing RT. Patients undergoing RNI had a higher likelihood of plan modifications; a group with low risk of modification was identified, which could direct future re-structuring of QA rounds. Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.Radiotherapy and Oncology 12/2014; 114(1). DOI:10.1016/j.radonc.2014.11.042 · 4.86 Impact Factor