The use of "war games" to enhance high-risk clinical decision-making in students and residents.

Abstract

Initial studies found that residents and students performed poorly in simple clinical scenarios. We hypothesized that repeated simulations in the "war games" format would improve performance.
Participants included medical students and residents on the trauma and surgical intensive care unit (SICU) services. Subjects were given a nursing report of an unstable patient and asked to verbalize management of the situation. Responses were transcribed and graded.
Eighty subjects and 5 experts participated in 227 simulations. Naive medical students, postgraduate year (PGY)-1, and PGY-2+ subjects performed worse than experts (P <.05). After participation in >/=3 war games sessions, trainees' scores were similar to experts. Subjects with the least amount of clinical experience demonstrated the most improvement.
We have designed an educational system that rapidly enhances the cognitive performance of students and residents. This may represent an important tool in assessing and enhancing the competencies of medical trainees in a safe environment.

Full text

Page 1

Surgical Education
The use of “war games” to enhance high-risk clinical
decision-making in students and residents
Traci L. Hedrick, M.D., Jeffrey S. Young, M.D.*
University of Virginia Cognitive Research Laboratory, University of Virginia Health System, Charlottesville, VA 22906-
0709, USA
Abstract
BACKGROUND: Initial studies found that residents and students performed poorly in simple clinical
scenarios.Wehypothesizedthatrepeatedsimulationsinthe“wargames”formatwouldimproveperformance.
METHODS: Participants included medical students and residents on the trauma and surgical inten-
sive care unit (SICU) services. Subjects were given a nursing report of an unstable patient and asked
to verbalize management of the situation. Responses were transcribed and graded.
RESULTS: Eighty subjects and 5 experts participated in 227 simulations. Naive medical students,
postgraduate year (PGY)-1, and PGY-2? subjects performed worse than experts (P ?.05). After
participation in ?3 war games sessions, trainees’ scores were similar to experts. Subjects with the least
amount of clinical experience demonstrated the most improvement.
DISCUSSION: We have designed an educational system that rapidly enhances the cognitive per-
formance of students and residents. This may represent an important tool in assessing and enhancing
the competencies of medical trainees in a safe environment.
© 2008 Elsevier Inc. All rights reserved.
KEYWORDS:
Decision-making;
Cognition;
Medical errors;
Simulation;
Surgical education
With the advent of work-hour restrictions, many training
programs are struggling to evaluate the education and com-
petence of trainees in more exacting ways.1,2The traditional
role of the “house officer” consisted of extended hours in
the hospital and minimal cross-coverage. This system has
since evolved into shift work (ie, night float) where resi-
dents are caring for larger numbers of patients whom they
may not be familiar with. In this new environment, the
ability of the clinician to rapidly assess and intervene in a
situation, with little previous knowledge, is becoming more
important. Additionally, the ability of junior clinicians to
recognize the need for help is imperative.3
“Failure to rescue” is a term applied to clinical issues
that, if unrecognized or improperly treated, lead to adverse
outcomes (eg, respiratory distress, hypotension, altered
mental status).4–6Previously, young clinicians learned the
components of rescue through real-life scenarios under the
close supervision of senior house officers. Most facilities
have consolidated their in-house backup such that fewer
senior clinicians are covering larger populations of patients.
Given these limitations in coverage, simulation is a poten-
tial remedy for decreased clinical experience.7–11
Simulation can assess and increase resident competency
in a controlled environment. However, there is no objective
manner for clinicians to document their decision-making
processes. Therefore, it is difficult to demonstrate that sim-
ulated learning improves the clinical efficiency and perfor-
mance of trainees. As a result, teachers may be unable to
diagnose significant defects in cognitive performance of
trainees until it is too late.12
We previously examined the cognitive components of
* Corresponding author. Tel.: ?1-434-982-3549; fax: ?1-434-924-
2260.
E-mail address: jsy2b@virginia.edu
Manuscript received March 29, 2007; revised manuscript June 13, 2007
0002-9610/$ - see front matter © 2008 Elsevier Inc. All rights reserved.
doi:10.1016/j.amjsurg.2007.06.032
The American Journal of Surgery (2008) 195, 843–849

Page 2

rescue through the use of a “war games” simulator format.13
We created war games to provide a controlled, reproducible
environment for assessing and improving clinical decision-
making in trainees. Our initial studies found that junior
residents and medical students performed poorly in these
relatively simple clinical scenarios.14Therefore, we sought
to devise a reproducible simulation where performance
could be objectively evaluated. We hypothesized repeated
simulations in the war games format would improve per-
formance in medical students and residents.
Methods
Subjects
Approval was obtained from the Institutional Review
Board for Social and Behavioral Sciences of the University
of Virginia. This prospective study was conducted at the
University of Virginia Health System between July 1, 2004
and July 31, 2006. Participants included medical students
and residents rotating on the trauma and surgical intensive
care unit (SICU) services. The SICU team is composed of 1
to 2 fourth-year medical students and 3 residents (2 surgical
postgraduate year [PGY]-1s, 1 anesthesia PGY-3) rotating
on a 1-month period. The trauma service includes 1 to 2
third-year medical students, 1 fourth-year medical student, 1
surgical PGY-1, 1 emergency medicine PGY-1, 1 surgical
PGY-2, 1 emergency medicine PGY-3, and 1 surgical
PGY-4 rotating on 1- to 3-month periods. Subjects partici-
pated in war games sessions, which were conducted twice
per week for 1 hour. Following the completion of their
rotations, the residents and students were sent electronic
mail asking if they would agree to have their de-identified
responses used in our research. If the response was “no,” or
if they declined, their responses were deleted from the
database.
Conduct of sessions
Biweekly war games sessions are held in our cognitive
research laboratory within the constraints of the 80-hour
work week. Each session is interactive and conducted by a
physician examiner. There are 10 participant stations in the
cognitive research laboratory with a microphone routed to a
Dell XPS computer at each station (Dell Computers, Round
Rock, TX). A digital transcription program (Express Dic-
tate, NCH Swift Sound, Sydney, Australia) is used to record
all audio input and prepare it for transcription.
Subjects are asked to sign in and provide the following
data: name, year of residency or medical school, number of
ICU weeks completed, and number of previous war games
sessions completed; the data are recorded in the database. A
list is provided by the research assistant of all previous cases
completed by each subject to assure that they will always
receive a different case at each session. Individual subjects
are given an initial nursing report of an unstable patient
and asked to verbalize management of the situation while
other subjects in the group listened. Naive subjects are
always tested first, beginning with the most junior sub-
ject. We then proceed through subjects from most junior
to most senior.
At each session the examiner is provided with a list of all
subjects who had attended previous sessions, and the cases
they had been tested on. This was done to insure that each
subject was tested on a different case at each session. No
participant was tested on the same case twice.
Cases
Cases consist of basic floor situations including the clin-
ical conditions listed in Table 1. In these cases, the subject
is given a “stem,” which is the phone call from the nurse on
the floor. The subject is then told to proceed as if they were
the physician on call for this patient. Subject responses are
audio-recorded, transcribed, and graded by 2 independent
reviewers. A grading template (Table 2), is used to grade the
responses in an objective fashion. During most of this time
period, the grading was performed by 2 physicians; how-
ever, as more sessions were performed, performances were
graded in real-time by the physician examiner and the re-
search assistant.
The grading template includes laboratory, physical ex-
amination, and study results for each case. These data will
only be presented if requested by the subject. The case
proceeds until the subject has no further orders or action. No
immediate debriefing is given so that other subjects are not
“taught” during the session. Following the completion of
cases for all subjects, naive subjects are given a presentation
consisting of 36 slides on suggested strategies for the care of
these types of patients.
Analysis
Scores were assigned for clinical actions ordered as a
ratio of actual actions performed divided by suggested ac-
tions on the grading sheet. Scores were assigned by 2
examiners, and the average of the scores in each area was
Table 1
Cases
Atrial fibrillation
Acute myocardial infacrtion
Aspiration with respiratory failure
Asthmatic exacerbation
Cerebrovascular accident
Acute dyspnea from rib fractures
Hypotension post-surgery
Hypotensive blunt trauma patient
Narcotic overdose
Pulmonary embolism
Pneumonia
Sepsis
844The American Journal of Surgery, Vol 195, No 6, June 2008

Page 3

used in statistical analysis. Each grading sheet contains
scores in four sections: initial evaluation, secondary evalu-
ation, diagnosis, and follow-up.
● Initial evaluation consisted of actions such as going to see
the patient, performing a physical examination, gathering
data from the chart, and evaluating/correcting problems
in airway, breathing, and circulation.
● Secondary evaluation consisted of ordering actions based
on findings in the initial evaluation (eg, labs, radiographs,
intravenous fluids).
● Diagnosis focused on the subject’s ability to evaluate the
situation and intervene appropriately after the subject
completed their secondary evaluation and received the
results of the secondary data they requested.
● Follow-up consisted of vital actions in the efficiency of
patient care. These included detailing that the subject
would continually re-evaluate the patient until their con-
dition completely stabilized, ordering a critical care bed,
ordering appropriate consults (neurosurgery or neurology
for an obvious stroke, cardiology for a confirmed myo-
cardial infarction, etc), and contacting the senior resident
or attending to inform them of the events that had
transpired.
The scores for each section are recorded, as well as a
total score. The possible scores for each section and for the
entire case are recorded and the individual’s performance is
expressed as a percentage of possible correct responses in
each section and for the problem as a whole.
Experts
The subjects were recruited by the principle investigator.
The criteria for inclusion were attending or fellow physi-
cians who regularly participate in critical care or acute care
emergencies (surgical critical care, emergency medicine,
medical critical care). The experts were asked if they had
attended a war games session, read a transcript, or discussed
the content of the sessions with previous participants. If they
answered yes to these questions they were not recruited.
Thus, we attempted to include in the expert group only
physicians beyond their training who had no exposure to the
war games format. Two surgical critical care attendings, an
Table 2
Grading template
Respiratory arrest on floorDataPoints
Stem: 58 year-old female 3 days s\p admission for partial SBO.
Called by nurse for difficulty breathing
Initial evaluation
Go to room
Arrange information and help
Airway
1
1
1 Vomitus in mouth, labored respirations,
unarousable
Yes Intubation required
Etomidate
Succinylcholine
Suction
7.0 ET tube
Check tube position and breath sounds
Anesthesia backup
IV access
Vital signs
Secondary actions
ICU bed
CBC
Lytes
Troponin
EKG
CXR
ABG
Diagnosis
Aspiration
MI
Follow-up
Indicates where patient will go
Notifies senior resident
1
.5
.5
.5
.5
1
.5
1
1 120/90, P95, T37, Sats 100
1
1
1
1
1
1
1
HCT 42, WBC 18
Bicarb 19
0.5
ST changes laterally
RLL infiltrate
7.34/40/200
1
1
1
1
MaxTotal
Total
20.5
s/p ? status post; Sbo ? small bowel obstruction; ET ? endotracheal; IV ? intravenous; ICU ? intensive care unit; P ? pulse; T ? temperature;
HCT ? hematocrit; WBC ? white blood cell count; CBC ? complete blood count; Lytes ? electrolytes; EKG ? electrocardiogram; ABG ? arterial blood
gas; Bicarb ? bicarbinate; ST ? ST; RLL ? right lower lobe; MI ? myocardial infarction.
845T.L. Hedrick and J.S. YoungUse of “war games” to enhance trainee performance

Page 4

emergency medicine attending, and 2 pulmonary/critical
care fellows were separately presented with all cases during
a single session. Their responses were graded in the usual
fashion and used as our validation (expert group).
Statistical analysis
Values are expressed as the mean ? SD. Student t test
and analysis of variance were used to compare groups of
subjects to each other and to the expert performance. Due to
the fact that we were comparing discrete groups of subjects
performance, these tests were felt to be adequate for our
statistical consultant. A P value of less than .05 was con-
sidered significant. The subjects were grouped into third-
year medical students (MS3), fourth-year students (MS4),
first-year residents (PGY-1), residents beyond their first
year (PGY2?), and experts.
Results
There were 20 subjects in each group and 5 experts (n ?
85). In total, 227 simulations were performed and graded
(Table 3). This did not include the 1 subject who declined to
allow his/her responses to be used in this study. Figure 1
represents the subjects’ scores for each individual category
based on the number of prior war games sessions compared
to the experts. Naive MS3, MS4, PGY-1, and PGY-2?
subjects performed significantly worse than experts in all
categories (P ?.05). However, after participation in war
games sessions, subject scores improved with each addi-
tional session (Figure 1). After completion of 3 or more war
games sessions, trainees’ scores did not differ significantly
from the experts in any category or total score.
Individual cohorts were examined separately according
to the postgraduate year of the trainee and the amount of
Table 3
Trainee and expert performance
Naive (n ? 77) 1–2 prior sessions (n ? 73)
? 3 prior sessions (n ? 17) Expert (n ? 60)
Initial evaluation
Secondary evaluation
Diagnosis
Follow-up
Total score
.552 ? .18*
.367 ? .20*
.461 ? .29*
.371 ? .29*
.453 ? .12*
.621 ? .17*
.553 ? .26*
.431 ? .29*
.400 ? .31*
.551 ? .13*
.715 ? .136
.640 ? .20
.647 ? .30
.576 ? .35
.661 ? .13
.712 ? .11
.720 ? .18
.742 ? .26
.691 ? .29
.710 ? .09
*P ? .05 vs expert group.
Figure 1
in evaluation and treatment of the simulated patient expressed as a function of repeated war games sessions.
Performance of individual groups in the initial evaluation, secondary evaluation, diagnosis, follow-up, and total tasks required
846The American Journal of Surgery, Vol 195, No 6, June 2008

Page 5

prior ICU experience (Figure 2). Subjects with the least
amount of experience (PGY-1 and those with ?4 weeks of
ICU experience) demonstrated the most improvement
through participation in the war games sessions. We further
examined whether there was a difference in expert perfor-
mance in areas outside of the participant’s primary area of
specialization. We found no significant differences between
experts when we analyzed their performance by type of
clinical problem.
Comments
Education in medicine is poorly understood with regard
to what is known about medical expertise. Since the days of
Halsted, competence in surgery was granted by turns of the
calendar year. When a trainee progressed from medical
school through senior residency, graduated responsibility
was conferred with only minimal attempts to quantitatively
assess competence. We have previously demonstrated that
postgraduate year is not nearly as important as experience
(weeks of ICU training) in gaining cognitive reasoning
skills.13–15In response to political and economic pressures,
decreased work hours, increased cross coverage, and the
increased use of physician extenders, change is needed for
surgical education.
The 1999 Institute of Medicine study, To Err is Human,
reported that between 44,000 and 98,000 deaths annually in
American hospitals could be attributed to medical errors.16
More specifically, the majority of these errors could be
traced to deficits in cognitive appraisal and clinical deci-
sion-making. Following the release of this report, which
called for a 50% reduction in medical errors over 5 years,
there has been substantial interest in methods to improve the
quality of care in the medical industry.17–21Based on the
success afforded to the aviation industry, simulation models
are increasingly being employed to train medical profes-
sionals.7–12,22–26
Prior studies of simulation demonstrate improved perfor-
mance in the achievement of technical skills in multiple
disciplines.10,22,27Additionally, simulation is being used
increasingly to replace traditional lecture-based teaching
methods in the clinical setting.2,28–32As familiarity with
simulation evolves, simulation-centered educational curric-
ula are beginning to emerge.28,31,33Within the field of
surgery, most simulation models focus on performance of
invasive procedures. However, development of cognitive
ability in high-risk clinical scenarios is of equal importance.
Figure 2
repeated war games sessions. (B) Percentage improvement of individual groups in total task required in the evaluation and treatment of the
simulated patient after 3 or more war games sessions. P ?.05: PGY-1, MS, ICU ?4 naive subjects vs subjects following 3 sessions.
(A) Performance of individual groups in total tasks required in the evaluation and treatment of the simulated patient through
847T.L. Hedrick and J.S. YoungUse of “war games” to enhance trainee performance