A model for evaluating resident education with a focus on continuity of care and educational quality.
- [show abstract] [hide abstract]
ABSTRACT: To describe the activities of attending physicians in a residency-based continuity clinic and to examine factors that affect their teaching of, supervision of, and interaction with residents. Six full-time board-certified faculty members (three internal medicine, three internal medicine-pediatrics) in an urban residency program participated in a descriptive observational time-motion study. The attending faculty were directly observed by "shadow" technique for 30 half-day sessions from April 1994 through September 1994. Each activity was measured by a trained research assistant using a digital stopwatch. The observed activities were assigned to one of 16 subcategories. 6,389 minutes of activities were observed. Activities were distributed among four general categories: direct contact with residents (43.1%), clinic operations (33.7%), personal and/or professional activities (18.0%), and miscellaneous time (5.2%). Attending physicians spent the most time in direct contact with residents when the patient-to-attending ratio was 10-14:1. The activities of the clinic's attending physicians were quite varied. Less than half of their time in the clinic was spent in contact with residents. This contact time may be significantly increased by changes to clinic policies, such as optimizing the patient-to-faculty ratio and increasing administrative support for the clinic. These findings can be used as a reference point for studies of attending physicians' activities since the federally mandated rules changes regarding their responsibilities for supervising residents.Academic Medicine 12/2000; 75(11):1138-43. · 3.29 Impact Factor
- [show abstract] [hide abstract]
ABSTRACT: Surgical resident education is entering a critical era of achieving core competencies despite work hour restrictions. An assessment of on-call activity is needed to maximize educational merit. A time-motion study of resident on-call activity was performed at a university medical center and an urban affiliate hospital. Residents were followed by "shadow" residents who concurrently recorded resident activity. Activities of daily living and patient evaluation comprised the majority of on-call activity. Residents slept a median of 200 minutes per night. Cross-coverage activities accounted for 41% of pages and 19% of patient evaluation. Direct patient contact comprised only 7% of call night duties. Communication activity occupied 15% of total minutes, and a mean of 16 pages were received nightly. Significant differences in activities existed between resident levels and hospitals. Call activity consists primarily of activities of daily living, patient evaluation, and communication. Sleep accounts for nearly one third of all on-call activity. These data may be useful in improving both patient care and resident call experience.The American Journal of Surgery 10/2004; 188(3):225-9. · 2.52 Impact Factor
- Impact of the 80-hour work week on resident emergency opera-tive experience. 968-972..
2010 APDS SPRING MEETING
A Model for Evaluating Resident Education
with a Focus on Continuity of Care and
Heron Rodriguez, MD, Jonathan P. Turner, MS, Paul Speicher, BA, Mark S. Daskin, PhD,
and Debra DaRosa, PhD
Feinberg School of Medicine, Northwestern University, Chicago, Illinois
COMPETENCY: Medical Knowledge, Professionalism, Sys-
tem Based Practice
Surgical residency program directors continue to be challenged
with balancing duty-hour requirements established in July
2003 by the Accreditation Council on Graduate Medical Edu-
cation (ACGME), Residency Review Committee (RRC) core
competency and case requirements, and patient-care needs. It
generally is presumed that both continuity of care and patient
part, by the decreased presence of residents in the hospital.1
Although residents now have more time off, the amount and
increased. Similarly, achieving the required number of surgical
cases, with residents spending less time in the hospital, has
become more problematic. This issue is particularly true for
programs with specialty fellowships that directly compete for
our institution, a traditionally strong vascular surgery fellow-
ship consistently exceeds the required operative experiences,
often at the expense of general surgery residents.
residents from various specialties spend their time while on duty.
types and duration of resident activities as observed,2,3whereas
others asked residents to self-report their activities either in retro-
studies and disciplines, but most activities included specific activi-
ties categorized under “Patient care” and “Nonpatient care” or
signed an alternative internal medicine training rotation for resi-
dents and compared it with a control group.6The overall satisfac-
proportion of time spent in meaningful learning activities. CoC
was not measured in this study.
more difficult to achieve CoC.7–9As some of the activities of
patient care have been shifted to physician assistants and nurse
practitioners, the opportunities for CoC and to develop bond-
ing relationships with patients and their families have dimin-
ished.5Some investigators have shown that increases in time
that residents spend in clinic lead to increased CoC.9,10
increase CoC, it does not do so substantially.11
Several authors in the industrial engineering literature ap-
plied mathematical programming and computer algorithms to
optimize resident rotation assignments.12–18A common theme
across these studies is that residents are regarded as human
resources for supporting operational service requirements
rather than students trying to achieve defined educational goals
within an established clinical curriculum.
a baseline analysis of resident activities on a vascular surgery rota-
tion. The second aim was to employ mathematical modeling to
ties for CoC and meet RRC case requirements. More specifically,
our study addressed the following research questions: How do
residents spend their time while on the vascular rotation and are
they spending sufficient time in activities that facilitate objectives
achievement? How many RRC-defined category surgical experi-
ences are residents logging on their vascular rotations? Are they
rotation length on the opportunity to achieve CoC.
Correspondence: Inquiries to Heron E. Rodriguez, MD, Feinberg School of Medicine,
Northwestern University, 676 N St. Clair St., Suite 650, Chicago, IL 60611; fax; e-mail:
Webster Educational Innovation Grants Program of Northwestern University Feinberg
School of Medicine.
Journal of Surgical Education • © 2010 Association of Program Directors in Surgery
Published by Elsevier Inc. All rights reserved.
Subjects included a convenience sample of general surgery res-
idents (n ? 9) progressing through their vascular surgery rota-
tion at a large urban teaching hospital from June through Au-
gust 2009. This included postgraduate year (PGY) 1 (n ? 5),
PGY 2 (n ? 1), PGY3 (n ? 1), and PGY 4 (n ? 2) residents.
Institutional Review Board-approved, informed consent forms
were obtained from each subject. Vascular surgery was chosen
for 2 reasons. First, the vascular surgery rotation was consis-
tently the lowest rated rotation by general surgery residents.
Second, a strong fellowship program graduating 2 vascular sur-
geons every year directly competes for cases with the general
For the purposes of this article, CoC is defined as a resi-
dent who has participated in a patient’s operation and either
has completed the preoperative patient work up in the clinic
or the postoperative follow-up in clinic. The required num-
ber and type of cases were based directly on those stipulated
by the surgery RRC.
A resident activity card (RAC) was developed (Fig. 1) for 2
purposes. Its first purpose was to use as an instrument to record
how residents spend time during the rotation. The RAC also
on the rotation’s learning objectives. Activities on the RAC
were classified into 5 major categories, including educational
patient care, required education, noneducational patient care,
self study, other. Content validity of the RAC was accom-
plished through vascular faculty, fellow, and resident input and
consensus, as well as a review of published time and motion
studies relevant to surgery residents.19–21
Using the competency-based knowledge objectives provided
by the Association for Program Directors in Surgery,22the
learning objectives outlined by the Surgical Council on Resi-
review of the educational goals of the vascular surgery rotation
for residents in general surgery was carried out. The new goals
and objectives were defined and grouped according to resident
level and the ACGME core competencies.24Through consen-
sus of the vascular faculty with input from the Department of
to determine which of the RAC learning activities were most
likely to facilitate the objective’s achievement. Table 1 reflects a
associated with their achievement.
of residents on the vascular rotation, pager numbers, and times to
card between 1 and 4 times each weekday (as determined by the
Microsoft Excel Visual Basic for Applications pseudorandom
of the time that residents spent in different activities. These data
were analyzed using descriptive statistics.
To investigate opportunities for operating room experi-
ence, we analyzed all operative cases performed within the
2-month study period to determine the number of vascular
operations that were performed, the number that count to-
ward RRC-defined category requirements, and the number
in which vascular residents were involved. We measured the
length of time from preoperative visit to operation and from
operation to postoperative visit for each one of these cases.
We also looked at the number of cases vascular residents
logged with the ACGME for the 2008–2009 academic year
to determine averages and standard deviations across differ-
ent postgraduate years.
Calculus and probability theory were used to compute the
effects of rotation length on CoC as well as the likelihood that
an arbitrary patient’s preoperative visit (whether in outpatient
clinic or inpatient consultation), index operative procedure,
and outpatient follow-up visit can occur in the same fixed pe-
riod of a 1- or 2-month rotation. This analysis is based on the
assumption that the times between outpatient clinic diagnosis
and surgery and from surgery to outpatient clinic follow-up
come from exponential probability distributions with means
estimated from patient data.
Operative case experience was analyzed by determining how
FIGURE 1. Resident activity card.
2Journal of Surgical Education • Volume xx/Number x • Month 2010
to the list of defined category cases required by the RRC. This
ing the minimum goal of 44 vascular cases.
The designs of the project, informed consent forms, and
patient chart reviews were approved by the Institutional Re-
Figure 2 reflects the activities that the residents should spend
time on to achieve the learning objectives set for this rotation.
Three hundred and fifty-nine RACs were completed by the resi-
done by PGY3 and 4 residents. Figure 3 shows how residents spend
The surgical case logs for PGY1, PGY2, PGY3, and PGY4
residents for the 2008–2009 academic year are shown in Table
2. Additionally, for our 2-month study of surgical experience,
we found that 178 patients were seen, resulting in 210 proce-
dures (some patients had multiple procedures). Of these, 129
could be counted toward RRC-defined category requirements,
but residents only were involved in 66 of the surgeries.
TABLE 1. Example of Vascular Learning Objectives ? Activity
Patient careIn a patient with suspected venous
thromboembolism, the resident accurately
can assess the existence, probable
location, and extent of peripheral clots.
In a patient presenting with varicose veins,
the resident can recognize the clinical
signs and symptoms of chronic venous
disease without assistance.
Given a patient with compartment syndrome,
the resident can define the anatomy of the
compartments of the lower extremity
The resident can outline the indications for
operations for claudication, abdominal
aortic aneurysm, carotid stenosis, and
The resident can provide examples of Level 1
evidence used in clinical decision making
with regard to vascular surgery.
The resident can list the indications for
abdominal aneurysm repair and describe
clinical studies supporting these
The resident effectively can summarize and
communicate to the senior staff and faculty
clinical information needed to formulate
The resident effectively can communicate
with allied health professionals and other
physicians for consultations and follow-up.
The resident can demonstrate respect,
compassion, and integrity toward patients
The resident can become a role model for
junior residents and students in the different
academic and clinical activities of the
The resident can demonstrate an
understanding of the pivotal role of the
junior resident in the delivery of health
care to vascular patients.
The resident can demonstrate an ability to
lead a resident team.
Reading participating in
Reading pre-op care Clinic—diagnosis
Sign out post-op careConsults updating the
Consults post-op careUpdating the patient
ProfessionalismMeeting with attendings
Teaching or supervising
System-based practiceMeeting with attendings Clinic—follow up
System-based practiceTeaching or supervising
Journal of Surgical Education • Volume xx/Number x • Month 2010
The average time between preoperative diagnosis in the vas-
cular outpatient clinic and surgery was 16.7 days. The average
time between a preoperative evaluation for a vascular inpatient
and surgery was 1.9 days, and the time between surgery and
postoperative follow-up in the vascular outpatient clinic was
22.0 days. Seventy-one percent of these patients presented to
the outpatient clinic and 29% presented from within the hos-
pital. Based on these time periods, we estimated the likelihood
of a resident achieving CoC as a function of the length of his or
and in Fig. 5 for inpatients.
Duty-hour regulations continue to be a subject of great debate
with both advocates and opponents fiercely defending their
point of views. Instead of fueling this debate, surgical educators
need to prioritize resident activities by focusing on what resi-
dents do with their hours during a rotation more than merely
necessitate program directors to “tighten” their clinical curric-
ing residents’ activities on high-priority learning goals and ob-
jectives. Residency clinical curriculum structures require
change to optimize CoC and patient ownership, as well as to
achieve other important learning objectives. Findings reported
here are similar to other studies reported in the literature that
show that residents spend significant amounts of time in activ-
ities labeled as noneducational and not enough time in the
so-called “educational activities” that facilitate achievement of
learning objectives. When scrutinized against the activities pre-
determined to be relevant to objectives achievement, we found
that our residents do not spend enough time in clinic, confer-
ences, reading, doing consultations, and in the blood flow and
activities. Nevertheless, when specific educational objectives are
mapped to individual activities, the labels discriminating between
ing consultations often is viewed as an activity with little educa-
tional value and frequently relegated to physician extenders. Our
map of education objectives and activities finds the consultations
activity to achieve CoC. Simply delegating patient care to physi-
cian extenders not only critically compromises CoC and patient
ownership but it also prevents the resident from being exposed to
and updating the patient list, have the potential to accomplish
goals related to interpersonal and communication skills, profes-
FIGURE 2. Learning activites/ locations listed on the resident activity card and the percentage of time needed during the rotation to accomplish associated
4Journal of Surgical Education • Volume xx/Number x • Month 2010
the time spent in the operating room exceeded the amount
no longer valid to expect a valuable learning experience by simply
individual learning needs.
CoC is very unlikely—if not impossible—to be achieved with
the limitations imposed by a 1-month rotation. Similarly, in the
era of endovascular surgery, exposing general surgery residents to
part on the length of their rotations in this specialty. We propose
that the length of the rotation should be dictated by the time
needed to achieve specific learning objectives, by the number of
operative cases, and with CoC as a priority. Because of this study,
we have increased the rotation length of our residents in vascular
surgery from 1 month to 2 months.
This study has several limitations. The primary limitation is
that the study was conducted within 1 urban hospital with a
limited number of residents from 1 general surgery program.
The overall response rate of 43%, although low, is not signifi-
cantly different from prior published studies requesting resi-
FIGURE 3. Percent of time by activity as reported on the RAC tool.
TABLE 2. Average and Standard Deviation of Vascular Case
FIGURE 4. Limitations on CoC caused by length of rotation for
Journal of Surgical Education • Volume xx/Number x • Month 2010
dents to respond to inquiries about their use of time.4Never-
theless, we recognize it to be a potential source for bias and a
major limitation of the study. Although the generalizability of
the study requires multiinstitutional replication, we believe the
results of the activity cards reflect patterns typical of general
surgery resident experiences on a vascular surgery rotation. A
second limitation is the use of self-report. We attempted to
control by having the researcher on the project present at the
ing reminder of the importance of their compliance to study out-
comes. The paging system alert data-collection system is a recom-
mended self-report format reported to increase compliance and
time and does not need to rely on long-term recall.25Another
limitation to the study was the length of time data were collected.
Our results could be skewed by seasonable “business” or by resi-
dents’ personalities. Finally, limiting the study to the hours be-
tween6 AMand6 PMunderscoresthetime that residents spent in
self-directed activities that can be performed outside the hospi-
first reason is that we wanted to see how residents spent their
time while “at work” on the rotation. Second, we wanted to
them to participate in a study that would require their involve-
ment after hours.
In summary, faculty cannot expect that their clinical rota-
tions will enable accomplishment of stated learning objectives
using a “catch as catch can” organizational approach. Instead,
resident activities should be planned according to where they
most likely can achieve these objectives. Second, continuity of
to occur if residents do not attend clinic or take responsibility
for inpatient consults. Industrial engineers can apply mathe-
can suggest rotation lengths that can best facilitate CoC based
on the surgical specialty averages of time from preoperative to
operative to postoperative periods. We need to be aware of and
we need to balance busy clinical service needs and resident
learning needs. This concept requires understanding how resi-
dents spend time on a rotation so adjustments can be made to
increase the likelihood that they are engaged in activities mean-
ingful to a rotation’s goals and objectives.
1. Mann B. Creativity, simulation, and patient-safety: sem-
inal themes during surgical education week. Focus (ASE
Newsletter), Volume 26. Springfield, IL: Association for
Surgical Education; 2009.
2. Melgar T, Schubiner H, Burack R, Aranha A, Musial J. A
an internal medicine and internal medicine-pediatrics resi-
dent continuity clinic. Acad Med. 2000;75:1138-1143.
3. Innes GD, Stenstrom R, Grafstein E, Christenson JM. Pro-
load predictors. Can J Emerg MED. 2005;7:299-293.
4. Morton JM, Baker CC, Farrell TM. What do surgery
residents do on their call nights? Am J Surg. 2004;188:
5. Brasel KJ, Pierre AL, Weigelt JA. Resident work Hours:
what they are really doing. Arch Surg. 2004;139:490-494.
J. Evaluation of a redesign initiative in an internal-medicine
residency. N Engl J Med. 2010;362:1304-1311.
7. Laine C, Goldman L, Soukup JR, Hayes JG. The impact
of a regulation restricting medical house staff working
hours on the quality of patient care. JAMA. 1993;269:
8. Feanny MA, Scott BG, Mattox KL, Hirshberg A. Impact
of the 80-hour work week on resident emergency opera-
tive experience. Am J Surg. 2005;190:968-972.
9. McBurney PG, Gustafson KK, Darden PM. Effect of 80-
hour workweek on continuity of care. Clin Pediatr. 2008;
11. Bell NR, Szafran O. Continuity of care: opportunity for
residents to see repeat patients. Can Fam Physician 1995;
12. Franz LS, Miller JL, Scheduling. Medical residents to ro-
tations—solving the large-scale multiperiod staff assign-
ment problem. Oper Res. 1993;41:269-279.
FIGURE 5. Limitations on CoC caused by length of rotation for
6 Journal of Surgical Education • Volume xx/Number x • Month 2010
13. Ozkarahan I. A scheduling model for hospital residents.
J Med Syst. 1994;18:251-265.
14. Sherali HD, Ramahi MH, Saifee QJ. Hospital resident
scheduling problem. Product Plan Control 2002;13:
15. Cohn A, Root S, Esses J, Kymissis C, Westmoreland, N.
idents, Industrial and Operations Engineering Technical.
Report 06-06. Ann Arbor, MI: University of Michigan;
16. Day TE, Napoli JT, Kuo PC. Scheduling the resident
80-hour work week: an operations research algorithm.
Curr Surg. 2006;63:136-142.
17. Topaloglu S. A multi-objective programming model for
scheduling emergency medicine residents. Comput Ind
different seniority levels and an application in healthcare.
Eur J Oper Res. 2009;198:943-957.
19. Brasel KJ, Pierre AL, Weigelt JA. Resident work hours.
Arch Surg. 2004;139:490-494.
20. Westbrook JI, Ampt A, Kearney L, Rob MI. All in a day’s
work: an observational study to quantify how and with
whom doctors on hospital wards spend their time. MJA.
residents do on their night on call? Am J Surg. 2004;188:
22. Association of Program Directors in Surgical Education.
cessed April 15, 2010.
23. Surgical Council on Resident Education. Available at:
http://www.surgicalcore.org/index.html. Accessed April
24. General A, Competencies. Available at: http://www.acgme.
25. Stone A, Turkkan JS, Bachrach CA, Jobe JB, Kurtzman
HS, Cain VS, Lawrence E, eds. The Science of Self Report:
Implications for Research and Practice. Mahwah, NJ: Law-
rence Erlbaum; 2000.
Journal of Surgical Education • Volume xx/Number x • Month 2010