The use of standardized patients to evaluate family medicine resident decision making

Wilson Family Practice Residency, Johnson City, NY 13790, USA.
Family medicine (Impact Factor: 1.17). 04/2007; 39(4):261-5.
Source: PubMed


This study was intended to assess the usefulness of standardized patients (SPs) in the evaluation of family medicine residents' clinical decision-making skills in ambulatory settings.
A pool of SPs was trained about the symptoms of one of three clinical conditions (depression, headache, or irritable bowel syndrome). These patients were then surreptitiously incorporated into the office hours of 11 residents on one occasion during their second year and once during their third year of training. A different SP was used at each encounter, but the same clinical case was presented each year for that particular resident. The SPs were given a questionnaire after each office visit to evaluate whether the resident completed elements of history, physical exam, clinical decision making specific to their case, and to report their satisfaction with the encounter. In addition, the assessment component of the resident's progress notes for each SP visit was reviewed by a single examiner to determine if the correct diagnosis was made by the resident.
The residents showed a statistically significant decrease in the percentage of checklist items completed for all clinical cases from the second year (82.70%) to the third year (75.55%). However, the average patient satisfaction was unchanged, as was the number of correct diagnoses, even though fewer questions were asked.
The use of SPs is a feasible and potentially useful method for evaluating family medicine resident decision making. Several factors may account for the differences in resident performance with SP scenarios.

Download full-text


Available from: Gary D James, Oct 02, 2015
12 Reads
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Effective communication skills and professionalism are critical for physicians in order to provide optimum care and achieve better health outcomes. The aims of this study were to evaluate residents' self-assessment of their communication skills and professionalism in dealing with patients, and to evaluate the psychometric properties of a self-assessment questionnaire. A modified version of the American Board of Internal Medicine's (ABIM) Patient Assessment survey was completed by 130 residents in 23 surgical and non-surgical training programs affiliated with a single medical school. Descriptive, regression and factor analyses were performed. Internal consistency, inter-item gamma scores, and discriminative validity of the questionnaire were determined. Factor analysis suggested two groups of items: one group relating to developing interpersonal relationships with patients and one group relating to conveying medical information to patients. Cronbach's alpha (0.86) indicated internal consistency. Males rated themselves higher than females in items related to explaining things to patients. When compared to graduates of U.S. medical schools, graduates of medical schools outside the U.S. rated themselves higher in items related to listening to the patient, yet lower in using understandable language. Surgical residents rated themselves higher than non-surgical residents in explaining options to patients. This appears to be an internally consistent and reliable tool for residents' self-assessment of communication skills and professionalism. Some demographic differences in self-perceived communication skills were noted.
    BMC Medical Education 02/2009; 9(1):1. DOI:10.1186/1472-6920-9-1 · 1.22 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Four- or five-option multiple choice questions (MCQs) are the standard in health-science disciplines, both on certification-level examinations and on in-house developed tests. Previous research has shown, however, that few MCQs have three or four functioning distractors. The purpose of this study was to investigate non-functioning distractors in teacher-developed tests in one nursing program in an English-language university in Hong Kong. Using item-analysis data, we assessed the proportion of non-functioning distractors on a sample of seven test papers administered to undergraduate nursing students. A total of 514 items were reviewed, including 2056 options (1542 distractors and 514 correct responses). Non-functioning options were defined as ones that were chosen by fewer than 5% of examinees and those with a positive option discrimination statistic. The proportion of items containing 0, 1, 2, and 3 functioning distractors was 12.3%, 34.8%, 39.1%, and 13.8% respectively. Overall, items contained an average of 1.54 (SD = 0.88) functioning distractors. Only 52.2% (n = 805) of all distractors were functioning effectively and 10.2% (n = 158) had a choice frequency of 0. Items with more functioning distractors were more difficult and more discriminating. The low frequency of items with three functioning distractors in the four-option items in this study suggests that teachers have difficulty developing plausible distractors for most MCQs. Test items should consist of as many options as is feasible given the item content and the number of plausible distractors; in most cases this would be three. Item analysis results can be used to identify and remove non-functioning distractors from MCQs that have been used in previous tests.
    BMC Medical Education 02/2009; 9(1):40. DOI:10.1186/1472-6920-9-40 · 1.22 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Increasing numbers of medical schools are providing videos of lectures to their students. This study sought to analyze utilization of lecture videos by medical students in their basic science courses and to determine if student utilization was associated with performance on exams. Streaming videos of lectures (n = 149) to first year and second year medical students (n = 284) were made available through a password-protected server. Server logs were analyzed over a 10-week period for both classes. For each lecture, the logs recorded time and location from which students accessed the file. A survey was administered at the end of the courses to obtain additional information about student use of the videos. There was a wide disparity in the level of use of lecture videos by medical students with the majority of students accessing the lecture videos sparingly (60% of the students viewed less than 10% of the available videos. The anonymous student survey revealed that students tended to view the videos by themselves from home during weekends and prior to exams. Students who accessed lecture videos more frequently had significantly (p < 0.002) lower exam scores. We conclude that videos of lectures are used by relatively few medical students and that individual use of videos is associated with the degree to which students are having difficulty with the subject matter.
    BMC Medical Education 02/2009; 9(1):6. DOI:10.1186/1472-6920-9-6 · 1.22 Impact Factor
Show more