The validity and reliability of the affective competency score to evaluate death disclosure using standardized patients

Department of Emergency Medicine, Emory University School of Medicine, Atlanta, GA 30303, USA.
Journal of Palliative Medicine (Impact Factor: 2.06). 05/2006; 9(2):361-70. DOI: 10.1089/jpm.2006.9.361
Source: PubMed

ABSTRACT To explore the validity and reliability of the affective competency score (ACS), compared to a global rating measure to predict overall competency to perform a death disclosure in a standardized patient exercise and to investigate useful thresholds of the ACS.
Thirty-seven fourth-year students underwent standardized patient training in death disclosure during a fourth-year emergency medicine clerkship. Students were evaluated using a checklist, an ACS, and a global rating assessment. ACS interrater reliability, interitem reliability, item-total reliability, and split-half reliability were calculated. Area under the curve (AUC) measurements were used to establish criterion validity.
For the ACS, item-total correlations ranged from 0.76 to 0.85, 0.76 to 0.93, and 0.42 to 0.87; the split-half reliability was 0.82 (p = 0.0001), 0.86 (p = 0.0001) and 0.55 (p = 0.0007) for the standardized patient (SP), the faculty and the medical students, respectively. Interitem correlations were adequate. A moderate interrater correlation of the ACS was observed between the faculty observer and the SP (r = 0.47; p = 0.04); however, the medical students' self evaluation did not correlate significantly with either the SP (r = -0.04; p = 0.79), or the faculty observer (r = 0.00; p = 0.99). The AUC for was 0.98 (95% confidence interval [CI] 0.94 to 1.00), 0.87 (95% CI 0.73 to 0.99), and 0.74 (95% CI 0.53 to 0.95) for the faculty, SP, and medical student, respectively.
The ACS may be a valid, reliable, and useful measure to assess communication skills by faculty or SPs in this setting. At an ACS score of 16, 19, and 21 points for faculty, SPs, and medical students, respectively, there is 100% specificity for the detection of competency assessed on a global rating. However, the ACS appears to have limited reliability and validity when used by medical students.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Traditional learning and assessment systems are overwhelmed when it comes to addressing the complex and multi-dimensional problems of clinical communication and professional practice. This paper shows results of a training program in clinical communication under Problem Based Learning (PBL) methodology and correlation between student self-assessment and teachers assessment. This involves a qualitatve-quantitative cross-sectional study in usual practice in the 2nd year of the degree in Medicine. Teaching methodology is PBL, including 15 associate professors and 90 students. Educational tools for learning:PBL cases and seminars (video recorded, theoretical-practical lectures). Assessment tools: Tutorials on those cases worked on PBL (40%), knowledge test (30%), assessment of a case with PBL methodology (20%) and video recording report (10%). Communication skills are evidenced by CICCA-D scale (Connect-Understand-Identfy-Agree-Help-Decision). Variables: academic performance, score on CICCA-D and academic methodological assessment. The analysis is carried out using descriptive statistics, calculating the intra-class correlation coefcients and weighted Kappa index with quadratic weights. 92.2% of students passed the course on the frst round. In a range between 0 and 34 points students' self-assessment scored 13 (SD ± 5) points and teachers' 16 (SD ±7). A weak (21% - 41%) or poor (< 20%) correlation was obtained between teachers and students for all questions on CICCA-D. The authors suggest a summative assessment using diferent instruments and techniques to assess clinical communication skills from the frst year onwards, and highlight the key role of self-assessment, peer assessment and the use of video recording techniques along with feedback in formative assessment.
  • [Show abstract] [Hide abstract]
    ABSTRACT: Abstract Introduction. Emergency medical services (EMS) personnel are rarely trained in death notification despite frequently terminating resuscitation in the field. As research continues to validate guidelines for the termination of resuscitation (TOR) and reputable organizations such as NAEMSP lend support to such protocols, death notification in the field will continue to increase. We sought to test the hypothesis that a learning module, GRIEV_ING, which teaches a structured method for death notification, will improve the confidence, competency, and communication skills of EMS personnel in death notification. Methods. The GRIEV_ING didactic session consisted of a 90-minute education session composed of a didactic lecture, small group breakout session, and role-plays. This was both preceded and followed by a 15-minute case role-play using trained standardized survivors. To assess performance we used a pre-post design with 3 quantitative measures: confidence, competency, and, communication. Paramedics from the local EMS agency participated in the education as a part of continuing education. Pre-post differences were measured using a paired t-test and McNemar's test. Results. Thirty EMS personnel consented and participated. Confidence and competency demonstrated statistically significant improvements: confidence (percent change in scores = 11.4%, p < 0.0001) and competency (percent change in scores = 13.9%, p = 0.0001). Communication skill scores were relatively unchanged in pre-post test analysis (percent change in scores = 0.4, p = 0.9). Conclusion. This study demonstrated that educating paramedics to use a structured communication model based on the GRIEV_ING mnemonic improved confidence and competence of EMS personnel delivering death notification. Key words: death; education; emergency medical services; emergency medical technician.
    Prehospital Emergency Care 06/2013; DOI:10.3109/10903127.2013.804135 · 1.86 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: When patients die in the emergency department (ED), emergency physicians (EP) must disclose the bad news to family members. The death is often unexpected and the act of notification can be difficult. Many EPs have not been trained in the skill of communicating death to family members. This article reviews the available literature regarding ED death notification training and proposes future directions for educational interventions to improve physician communication in ED death disclosure.
    The western journal of emergency medicine 03/2013; 14(2):181-185. DOI:10.5811/westjem.2012.10.14193