The Validity and Reliability of the Affective Competency Score to Evaluate Death Disclosure Using Standardized Patients

ArticleinJournal of Palliative Medicine 9(2):361-70 · May 2006with7 Reads
DOI: 10.1089/jpm.2006.9.361 · Source: PubMed
To explore the validity and reliability of the affective competency score (ACS), compared to a global rating measure to predict overall competency to perform a death disclosure in a standardized patient exercise and to investigate useful thresholds of the ACS. Thirty-seven fourth-year students underwent standardized patient training in death disclosure during a fourth-year emergency medicine clerkship. Students were evaluated using a checklist, an ACS, and a global rating assessment. ACS interrater reliability, interitem reliability, item-total reliability, and split-half reliability were calculated. Area under the curve (AUC) measurements were used to establish criterion validity. For the ACS, item-total correlations ranged from 0.76 to 0.85, 0.76 to 0.93, and 0.42 to 0.87; the split-half reliability was 0.82 (p = 0.0001), 0.86 (p = 0.0001) and 0.55 (p = 0.0007) for the standardized patient (SP), the faculty and the medical students, respectively. Interitem correlations were adequate. A moderate interrater correlation of the ACS was observed between the faculty observer and the SP (r = 0.47; p = 0.04); however, the medical students' self evaluation did not correlate significantly with either the SP (r = -0.04; p = 0.79), or the faculty observer (r = 0.00; p = 0.99). The AUC for was 0.98 (95% confidence interval [CI] 0.94 to 1.00), 0.87 (95% CI 0.73 to 0.99), and 0.74 (95% CI 0.53 to 0.95) for the faculty, SP, and medical student, respectively. The ACS may be a valid, reliable, and useful measure to assess communication skills by faculty or SPs in this setting. At an ACS score of 16, 19, and 21 points for faculty, SPs, and medical students, respectively, there is 100% specificity for the detection of competency assessed on a global rating. However, the ACS appears to have limited reliability and validity when used by medical students.
    • "It is necessary to train teachers in learning and assessment of clinical communicaton skills. In the clinical setng, assessment by colleagues and other professionals who share care work with students such as colleagues, nurses, doctors or medical assistants (360 º assessment) can be of enormous educatonal value (Quest, Ander & Ratclif, 2006). Its inclusion in the summatve assessment requires the design and validaton of assessment scales to decrease variability and to increase valuaton objectvity (Norcini & Burch 2007). "
    [Show abstract] [Hide abstract] ABSTRACT: Traditional learning and assessment systems are overwhelmed when it comes to addressing the complex and multi-dimensional problems of clinical communication and professional practice. This paper shows results of a training program in clinical communication under Problem Based Learning (PBL) methodology and correlation between student self-assessment and teachers assessment. This involves a qualitatve-quantitative cross-sectional study in usual practice in the 2nd year of the degree in Medicine. Teaching methodology is PBL, including 15 associate professors and 90 students. Educational tools for learning:PBL cases and seminars (video recorded, theoretical-practical lectures). Assessment tools: Tutorials on those cases worked on PBL (40%), knowledge test (30%), assessment of a case with PBL methodology (20%) and video recording report (10%). Communication skills are evidenced by CICCA-D scale (Connect-Understand-Identfy-Agree-Help-Decision). Variables: academic performance, score on CICCA-D and academic methodological assessment. The analysis is carried out using descriptive statistics, calculating the intra-class correlation coefcients and weighted Kappa index with quadratic weights. 92.2% of students passed the course on the frst round. In a range between 0 and 34 points students' self-assessment scored 13 (SD ± 5) points and teachers' 16 (SD ±7). A weak (21% - 41%) or poor (< 20%) correlation was obtained between teachers and students for all questions on CICCA-D. The authors suggest a summative assessment using diferent instruments and techniques to assess clinical communication skills from the frst year onwards, and highlight the key role of self-assessment, peer assessment and the use of video recording techniques along with feedback in formative assessment.
    Full-text · Article · Jul 2014
  • [Show abstract] [Hide abstract] ABSTRACT: Delivering bad news to patients is an essential communication skill for physicians. Educators commonly use standardized patient (SP) encounters to train physicians in the delivery of bad news. It is expensive to use actors, for logistical reasons such as travel and scheduling, and there are limits to the characters and conditions an actor can portray in teaching encounters. Thus we studied the feasibility of creating SP avatars in a virtual world for the task of training medical trainees to deliver bad news. The SP avatars are easily customized for different scenarios and amenable to distance learning. We recruited 10 medical trainees to interact with a standardized female avatar in a three-dimensional simulated clinic, where the trainee was to inform the avatar of her newly diagnosed breast cancer. The trainee evaluated his or her self-efficacy in delivering bad news via an affective competency score (ACS) before and after the encounter. Two palliative care specialists evaluated each trainee's performance using the Bad News Assessment Schedule and the performance ACS. The self-efficacy ACS scores of the trainees improved overall: before, 20 ± 4, versus after, 24 ± 3, p = 0.001 (maximum score = 30). All participants considered the experience positive and commended the novel approach, although noting that the avatars were not able to portray body language cues. Participants viewed the avatar-mediated training as an excellent approach for learning how to deliver bad news but believed it could not substitute for real patient interactions. However, participant self-efficacy improved, which suggests that avatar-mediated training in a virtual world is a viable educational approach for skill training in delivering bad news.
    Article · Dec 2010
  • [Show abstract] [Hide abstract] ABSTRACT: To investigate changes of different domains of breaking bad news (bbn) competences after a teaching module for medical students, and to collage the results generated by different approaches of evaluation. Rating of medical student-SP interactions by means of a global rating scale and a detailed checklist used by SPs and independent raters. Students improved their breaking bad news competency. However, the changes vary between the different domains of bbn competency. In addition, results generated by different evaluation instruments differ. This study serves as a stimulus for further research on the training of specific elements of bbn and different approaches of evaluating bbn competency. In light of the different facets of bbn competency, it is important to set priorities regarding the teaching aims and to provide a consistent approach.
    Article · May 2011
Show more