The use of standardized patients to evaluate family medicine resident decision making

Wilson Family Practice Residency, Johnson City, NY 13790, USA.
Family medicine (Impact Factor: 1.17). 04/2007; 39(4):261-5.
Source: PubMed


This study was intended to assess the usefulness of standardized patients (SPs) in the evaluation of family medicine residents' clinical decision-making skills in ambulatory settings.
A pool of SPs was trained about the symptoms of one of three clinical conditions (depression, headache, or irritable bowel syndrome). These patients were then surreptitiously incorporated into the office hours of 11 residents on one occasion during their second year and once during their third year of training. A different SP was used at each encounter, but the same clinical case was presented each year for that particular resident. The SPs were given a questionnaire after each office visit to evaluate whether the resident completed elements of history, physical exam, clinical decision making specific to their case, and to report their satisfaction with the encounter. In addition, the assessment component of the resident's progress notes for each SP visit was reviewed by a single examiner to determine if the correct diagnosis was made by the resident.
The residents showed a statistically significant decrease in the percentage of checklist items completed for all clinical cases from the second year (82.70%) to the third year (75.55%). However, the average patient satisfaction was unchanged, as was the number of correct diagnoses, even though fewer questions were asked.
The use of SPs is a feasible and potentially useful method for evaluating family medicine resident decision making. Several factors may account for the differences in resident performance with SP scenarios.

Download full-text


Available from: Gary D James,
  • [Show abstract] [Hide abstract]
    ABSTRACT: Effective communication skills for clinical practice are essential for all advanced practice nurses. However, competence in these skills is not necessarily intuitive. Advanced communication skills should be taught in a way that is similar to the way other psychomotor skills in advanced nursing practice programs are taught. To develop a patient communication simulation laboratory for the acute care nurse practitioner program at a major university and to evaluate students' perceived confidence and communication effectiveness before and immediately as well as 4 months after completion of the laboratory. The communication simulation laboratory was developed in collaboration with faculty from the schools of nursing and medicine. Students participated in a didactic session and then completed a 2-hour communication simulation in the laboratory. Content and simulation concentrated on breaking "bad news," empathetic communication, motivational interviewing, and the "angry" patient. Students' self-reported confidence and perceived skill in communication were measured via a Likert scale before, immediately after, and 4 months after completion of the laboratory simulation. Students also evaluated the experience by responding to open-ended questions. Compared with baseline findings (before the lecture and simulation), students' confidence in initiating difficult conversations increased significantly both immediately (P<.001) and 4 months after (P=.001) the laboratory simulation. Students' self-ratings of overall ability to communicate were also significantly greater immediately (P<.001) and 4 months (P=.001) after the simulation. Overall, students rated the laboratory simulation experience highly beneficial. The content and methods used for the simulation improved students' confidence and perceived skill in communication in potentially difficult acute care situations.
    American Journal of Critical Care 08/2008; 17(4):364-72. · 2.12 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Effective communication skills and professionalism are critical for physicians in order to provide optimum care and achieve better health outcomes. The aims of this study were to evaluate residents' self-assessment of their communication skills and professionalism in dealing with patients, and to evaluate the psychometric properties of a self-assessment questionnaire. A modified version of the American Board of Internal Medicine's (ABIM) Patient Assessment survey was completed by 130 residents in 23 surgical and non-surgical training programs affiliated with a single medical school. Descriptive, regression and factor analyses were performed. Internal consistency, inter-item gamma scores, and discriminative validity of the questionnaire were determined. Factor analysis suggested two groups of items: one group relating to developing interpersonal relationships with patients and one group relating to conveying medical information to patients. Cronbach's alpha (0.86) indicated internal consistency. Males rated themselves higher than females in items related to explaining things to patients. When compared to graduates of U.S. medical schools, graduates of medical schools outside the U.S. rated themselves higher in items related to listening to the patient, yet lower in using understandable language. Surgical residents rated themselves higher than non-surgical residents in explaining options to patients. This appears to be an internally consistent and reliable tool for residents' self-assessment of communication skills and professionalism. Some demographic differences in self-perceived communication skills were noted.
    BMC Medical Education 02/2009; 9(1):1. DOI:10.1186/1472-6920-9-1 · 1.22 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Four- or five-option multiple choice questions (MCQs) are the standard in health-science disciplines, both on certification-level examinations and on in-house developed tests. Previous research has shown, however, that few MCQs have three or four functioning distractors. The purpose of this study was to investigate non-functioning distractors in teacher-developed tests in one nursing program in an English-language university in Hong Kong. Using item-analysis data, we assessed the proportion of non-functioning distractors on a sample of seven test papers administered to undergraduate nursing students. A total of 514 items were reviewed, including 2056 options (1542 distractors and 514 correct responses). Non-functioning options were defined as ones that were chosen by fewer than 5% of examinees and those with a positive option discrimination statistic. The proportion of items containing 0, 1, 2, and 3 functioning distractors was 12.3%, 34.8%, 39.1%, and 13.8% respectively. Overall, items contained an average of 1.54 (SD = 0.88) functioning distractors. Only 52.2% (n = 805) of all distractors were functioning effectively and 10.2% (n = 158) had a choice frequency of 0. Items with more functioning distractors were more difficult and more discriminating. The low frequency of items with three functioning distractors in the four-option items in this study suggests that teachers have difficulty developing plausible distractors for most MCQs. Test items should consist of as many options as is feasible given the item content and the number of plausible distractors; in most cases this would be three. Item analysis results can be used to identify and remove non-functioning distractors from MCQs that have been used in previous tests.
    BMC Medical Education 02/2009; 9(1):40. DOI:10.1186/1472-6920-9-40 · 1.22 Impact Factor
Show more