Collaborative Cross-Checking to Enhance Resilience

Abstract

Resilience, the ability to adapt or absorb disturbance, disruption, and change, may be increased by team processes in a complex, socio-technical system. In particular, collaborative cross-checking is a strategy where at least two individuals or groups with different perspectives examine the others' assumptions and/or actions to assess validity or accuracy. With this strat- egy, erroneous assessments or actions can be detected quickly enough to mitigate or eliminate negative con- sequences. In this paper, we seek to add to the under- standing of the elements that are needed in effective cross-checking and the limitations of the strategy. We define collaborative cross-checking, describe in detail three healthcare incidents where collaborative cross- checks played a key role, and discuss the implications of emerging patterns.

Full text

ORIGINAL ARTICLE
Collaborative cross-checking to enhance resilience
Emily S. Patterson ÆDavid D. Woods Æ
Richard I. Cook ÆMarta L. Render
Received: 1 October 2005 / Accepted: 1 October 2006 / Published online: 12 December 2006
Springer-Verlag London Limited 2006
Abstract Resilience, the ability to adapt or absorb
disturbance, disruption, and change, may be increased
by team processes in a complex, socio-technical system.
In particular, collaborative cross-checking is a strategy
where at least two individuals or groups with different
perspectives examine the others’ assumptions and/or
actions to assess validity or accuracy. With this strat-
egy, erroneous assessments or actions can be detected
quickly enough to mitigate or eliminate negative con-
sequences. In this paper, we seek to add to the under-
standing of the elements that are needed in effective
cross-checking and the limitations of the strategy. We
define collaborative cross-checking, describe in detail
three healthcare incidents where collaborative cross-
checks played a key role, and discuss the implications
of emerging patterns.
1 Introduction
There is a growing consensus that resilience, the ability
to adapt or absorb disturbance, disruption, and change,
may be increased by team processes in a complex, so-
cio-technical system (Hollnagel et al. 2006; Rasmussen
1990; Weick et al. 1999). The concept of resilience is
founded upon the belief that failures are breakdowns
in the normal adaptive processes necessary to cope
with the complexity of the real world and that success
relates to organizations, groups and individuals who
produce resilient systems that recognize and adapt to
variations, changes and surprises (Rasmussen et al.
1994; Cook et al. 2000; Woods and Shattuck 2002;
Sutcliffe and Vogel 2003).
This body of work has led to the emergence of
techniques for Resilience Engineering (Hollnagel et al.
2005; Cook and Rasmussen 2005). Resilience Engi-
neering focuses on what sustains or erodes the adaptive
capacities of the human–technical system in a changing
environment. The focus is monitoring organizational
decision making to assess the risk that the organization
is operating nearer to safety boundaries than it realizes
in response to pressures to be ‘‘faster better cheaper.’’
More generally, an organization has levels of adaptive
capacity that needs to meet or exceed a wide range
of environmental pressures and demands (Westrum
2006). Studies of how systems are resilient and how
E. S. Patterson (&)
VA Getting at Patient Safety (GAPS) Center,
Cincinnati VAMC, Institute for Ergonomics,
Ohio State University, 210 Baker Systems,
1971 Neil Avenue, Columbus, OH 43210, USA
e-mail: patterson.150@osu.edu
D. D. Woods
Institute for Ergonomics, Ohio State University,
210 Baker Systems, 1971 Neil Avenue, Columbus,
OH 43210, USA
R. I. Cook
Cognitive Technologies Laboratory,
Department of Anesthesia and Critical Care,
University of Chicago, 5841 S. Maryland Avenue MC4028,
Chicago, IL 60637, USA
M. L. Render
Department of Internal Medicine,
VA Getting at Patient Safety (GAPS) Center,
Cincinnati VAMC, University of Cincinnati,
3200 Vine Street, Cincinnati, OH 45220, USA
123
Cogn Tech Work (2007) 9:155–162
DOI 10.1007/s10111-006-0054-8

breakdowns in resilience occur have begun to show
how to measure and improve system safety (e.g., Car-
they et al. 2001; Patterson et al. 2002; Woods 2005;
Miller and Xiao 2006).
These studies point to collaborative cross-checking as
a critical component of resilience because erroneous
assessments or actions can be detected quickly enough
to mitigate or eliminate negative consequences. Studies
of problem detection in aviation and nuclear power
both found that mis-assessments of the situation at hand
were only corrected when fresh perspectives entered
the situation (Woods et al. 1987; Sarter 2000). Problem
detection is challenging because it often requires rec-
onceptualizing the situation from a different stance and
experience base (Klein et al. 2005; Klein 2006).
The findings regarding the effectiveness of collabo-
rative cross-checking are mixed in the literature. On the
positive side, the inclusion of pharmacists during phy-
sician rounds was associated with fewer adverse drug
events (Leape et al. 1999). Similarly, interdisciplinary
rounds with the entire care team in a Surgical Intensive
Care Unit (SICU) was associated with decreased mor-
tality, increased patient satisfaction, and increased
quality of work life (Uhlig et al. 2002). Cross-checking
through machine critiquers in blood banking also en-
hanced performance even in situations where both the
machine advisor and the human practitioner had great
difficulties performing alone (Guerlain et al. 1999). On
the negative side, routinized cross-checking, particu-
larly when effortful attention is required and the base
rate for problems is low, is rarely effective, often re-
ferred to as the ‘‘vigilance decrement’’ (Williams 1986).
A number of studies have found that having two
observers perform a monitoring task does not neces-
sarily lead to improved performance (e.g., Erev et al.
1995). Similarly, handoff strategies are an important
concern today because well-designed handoffs can
increase resilience whereas poorly designed handover
processes can contribute to adverse events (Cooper
et al. 1982; Patterson et al. 2004; Behera et al. 2005).
In this paper, we seek to add to the understanding of
the elements that are needed in effective cross-check-
ing and the limitations of the strategy. We define col-
laborative cross-checking, describe in detail three
healthcare incidents where collaborative cross-checks
played a key role, and discuss the implications of
emerging patterns.
2 Distinctions in collaborative cross-checking
Collaborative cross-checking is a strategy where at least
two individuals or groups with different perspectives
examine the others’ assumptions and/or actions to as-
sess validity or accuracy. These perspectives can differ
on a number of dimensions, including goals, responsi-
bilities, functions, authority, stance, expertise, re-
sources, methodologies, and knowledge or information
of various types. The primary intent of a collaborative
cross-check is to detect an erroneous assessment or
action, although a number of other benefits may be
realized. Collaborative cross-check interactions incur
costs such as interruptions, increased cognitive work,
increased complexity resulting from changes to plans,
diffusion of responsibility, and coordinative costs such
as identifying and scheduling appropriate individuals.
2.1 Scope of change following collaborative cross-
checking
A cross-checking interaction can result in changes to
activities and outcomes. The anticipated scope of
change stemming from a coordination event is largely
determined by the relationship between the partici-
pating individuals or teams. Distinctions in relation-
ships include:
(1) Asymmetric hierarchy and expertise with similar
perspective (e.g., attending physician and resi-
dent physician).
(2) Asymmetric expertise, non-hierarchical with
similar perspective (e.g., mentor and mentee).
(3) Symmetric expertise with similar perspective:
(a) Same time (e.g., resident physicians reporting to
the same attending physician with different pa-
tients during the same shift).
(b) Different times (e.g., handoff from resident
physician to resident physician).
(c) During escalating situations (e.g., nurse provides
aid to another nurse in ‘‘on-call’’ manner).
(4) Functionally distinct, overlapping roles:
(a) Anomalous situations (e.g., code team and
nurse).
(b) Situations requiring specialized expertise (e.g.,
cardiologist physician and internal medicine
physician).
(c) Nominal situations (e.g., physician and nurse
caring for the same patient).
(5). Weakly defined role relationships
(a) Integration resource (e.g., case manager).
(b) ‘‘Floater’’ resource (e.g., Clinical Nurse Special-
ist).
(c) Cross-cutting functional resource (e.g., patient
safety officer).
Although the extent of receptivity to changes to
ongoing and future plans following a cross-checking
156 Cogn Tech Work (2007) 9:155–162
123

interaction is somewhat predictable based on these
relationships, in practice there is immense variability.
People in leadership positions impact social norms for
receptivity expectations, whether or not they partici-
pate directly in collaborative cross-checks. In addition,
individual differences and social relationships contrib-
ute to variability.
Note that the actual change and perceived change,
often as portrayed in official documentation, following
a collaborative cross-check can differ. In many situa-
tions, changing a plan following an interaction is posi-
tively rewarded. In these cases, the scope of change
might be exaggerated in official documentation. In
other situations, a large scope of change might imply
that prior plans were erroneous, which might negatively
impact reputation. In these cases, the actual change
may be larger than what is officially documented.
2.2 Short-term and long-term benefits
of collaborative cross-checking
The primary intent of a collaborative cross-check is to
detect an erroneous assessment or action in the short
term. Nevertheless, a number of other benefits may be
realized, including improved plans and ‘‘second order’’
adaptations that improve system resilience over the
long term.
Regarding improvements to ongoing and future
plans, the following short-term benefits could poten-
tially be realized:
(1) reveal hidden assumptions;
(2) clarify goal trade-offs;
(3) explore new regions of a solution space to satisfy
competing goals;
(4) identify unintended consequences of actions
(‘‘side effects’’);
(5) identify exceptions and ‘‘boundary conditions’’;
(6) identify possible contingencies;
(7) identify information gaps;
(8) identify influential people who might support or
obstruct plan.
In addition, coordination might be improved by a
collaborative cross-check, even when no erroneous
assessments or actions are detected. Short-term coor-
dination benefits may include:
(1) Improved ability to distribute tasks among indi-
viduals or groups to reduce redundant efforts and
fill gaps in knowledge.
(2) Increased awareness of perspectives of other
individuals and groups (i.e., calibrated ‘‘common
ground’’).
(3) Increased awareness of others’ needs for infor-
mation to promote timely information updates
and better anticipate interaction requests.
(4) Improved ability to anticipate changes to others’
perspectives when situations change.
(5) Reduced coordination costs for future meetings,
such as by improved ability to anticipate sched-
uling conflicts.
Finally, collaborative cross-checks may prompt
investment in long-term system adaptation, ultimately
increasing system resilience over a longer period than a
single instance. Interestingly, these adaptations may
reduce the benefits of individual cross-checking inter-
actions by removing or reducing system-level con-
tributors to erroneous assessments and actions. For
example, a hospital might find that calls to rapid re-
sponse teams (RRTs) to identify which patients are
likely to suffer cardiac or respiratory arrest gradually
reduce over time. Rather than signaling a reduction in
system resilience, however, these reductions could in-
stead indicate a transfer of specialized knowledge from
the RRT to the floor nurses, who then require the RRT
less frequently. Long-term ‘‘second order’’ benefits of a
series of cross-checking interactions might include:
(1) transfer of knowledge across perspectives;
(2) increased ‘‘meta-knowledge’’ of others’ perspec-
tives;
(3) increased ‘‘team identity’’ across previously dis-
tinct individuals or groups;
(4) improve coordination across ‘‘stovepiped’’ groups
in an organization;
(5) identify ineffective cross-checking strategies;
(6) system and training redesign.
3 Three incidents employing collaborative
cross-checks
An extant corpus of approximately 30 voluntarily re-
ported healthcare incidents, collected to explore a
broad set of patient safety issues, was reviewed. Three
cases were selected on the basis of collaborative cross-
checks playing a key role and firsthand experience with
the data collection.
3.1 Overview of cases
3.1.1 Case 1: chemotherapy overdose
In the first case (Table 1), an oncology fellow physician
erroneously substituted the medication navelbine for
Cogn Tech Work (2007) 9:155–162 157
123

the intended etoposide during ordering. The patient
had a prolonged hospitalization with severe leuco-
penia.
3.1.2 Case 2: questionable heparin order
In the second case (Table 2), an order to initiate hep-
arin when a patient was planned to be discharged was
questioned, as well as the subsequent decision by an
on-call physician to verbally discontinue the order
without knowing the rationale for the original order.
There were no apparent negative impacts to the pa-
tient.
3.1.3 Case 3: erroneously labeled IV bag
In the third case (Table 3), a patient was administered
glucose via a bag that was labeled ‘‘bicarb’’. There
were several attempts to troubleshoot high glucose
levels, which eventually led to detection of the erro-
neous label. There were no apparent negative long-
term impacts to the patient.
Table 1 Cross-checking interactions during case 1
Cross-checking interaction Commentary
Attending cross-checks fellow physician (phone): Successful
Fellow: Plan is the same as an outpatient from the previous week
(administering navelbine on a weekly basis)
Possibly routine strategy for senior physician to first ask junior
physician for a plan, which is then critiqued
Attending: No, because the prior case was a non-small cell as
opposed to a small cell lung cancer case. Plan should be VP-16
(etoposide) 100 mg/m
2
on days 1, 2, 3 and Carboplatin 6 AUC
on day 1, to be repeated every 3–4 weeks
Possibly common misconception that small cell and non-small
cell cancers are treated similarly
Pharmacist cross-checks oncology fellow physician (phone): Not successful
Fellow: I will order navelbine at 100 mg/m
2
and Carboplatin 6
AUC to be given today
Embedded in hospital policy is a defense against cross-checking
by a pharmacist who does not specialize in chemotherapy
Pharmacist: The hospital policy does not allow initiating new
chemotherapy regimens on the weekends
Rationale for the hospital policy is not stated during interaction
Fellow: I discussed the case with the attending and it is critical to
administer the medications as quickly as possible
Fellow assumes that attending physician is aware of the policy:
therefore assumes that there is no new information from the
pharmacist (who has no specialized oncological expertise)
that would overturn decision
Pharmacist cross-checks fellow physician (phone): Not successful
Pharmacist: The package insert says that the standard dose is
30 mg/m
2
for navelbine. Are you sure that you want
100 mg/m
2
?
Pharmacist cross-checks using package insert information
because pharmacist with chemotherapy expertise
was not home
Fellow: Yes Package insert included that navelbine is administered with
Carboplatin
Pharmacist cross-checks fellow physician (phone): Not successful
Pharmacist: I am giving the medications to the nurse to hang. Are
you sure that you want 100 mg/m
2
?
Asking the same question again resulted in the same response
Fellow: Yes New information was not conveyed (many bags required for the
dose, nurse asked pharmacist to call again because she was
not very familiar with the medication)
Nurse cross-checks fellow physician (phone): Successful (somewhat)
Nurse: I administered the navelbine. The patient is diaphoretic,
short of breath, and hypertensive. Do you want to hold the
carboplatin?
New information and suggestion of a specific action (hold
medication) prompted a change of plan
Fellow: Yes, hold the carboplatin Change of plan was later judged poor because side effects are
usually managed rather than halting a chemotherapy
treatment prior to completion
Physician unaware that nurse is relatively inexperienced with
chemotherapy medication
Incoming fellow cross-checks fellow physician: Successful
Incoming fellow (reviewing patient record): Navelbine is not the
right medication for this diagnosis, it is VP-16 (etoposide)
Person with one additional year of experience detected
substitution while reviewing medications in preparation
for taking over
Fellow: Discusses impacts, how to respond, discusses recovery
plan with attending and incoming fellow, attending informs
family
Once detected, easy to recognize erroneous action
158 Cogn Tech Work (2007) 9:155–162
123

3.2 Collaborative cross-checks in case 1
The first case (Table 1) is a chemotherapy overdose
resulting primarily from erroneous substitution of a
medication name. The case was collected via a se-
quence of critical decision method interviews (Klein
1989) conducted individually with the oncology fellow,
oncology attending, pharmacist, and nurse. This case is
described in detail in Patterson et al. (2004).
3.3 Collaborative cross-checks in case 2
The second case (Table 2) involves a questionable
heparin order. This case was collected via direct
observation of an acute care nurse.
3.4 Collaborative cross-checks in case 3
The third case involves medication administration by a
nurse from an erroneously labeled IV bag. The IV bag
was mixed and erroneously labeled at the point of care
by a nurse from a previous shift because pharmacy
resources were overwhelmed. This case was collected
during a critical decision method interview with a
Clinical Nurse Specialist.
4 Discussion of emerging patterns for effective
cross-checking
Although no strong conclusions can be drawn from
three case studies, there are suggestive emerging pat-
terns:
1. Routinized collaborative cross-checks were not
very effective:
(a) All cases. Routinized physician and nurse veri-
fication of physician orders did not successfully
detect erroneous assessments or actions.
2. Some collaborative cross-checks helped to detect
and recover from erroneous assessments and ac-
tions, but specialized knowledge and interdisci-
plinary interactions were often involved:
(a) Case 1. Incoming fellow immediately detected
erroneous medication order when reviewing the
chart, but not the pharmacist or nurse who did
not have as much specialized knowledge of
oncology as other staff.
(b) Case 2. Patient detected that the order was
questionable, but the night nurse, the on-call
physician, the next nurse, and the nurse manager
could not adequately recover because they did
not access the physician who wrote the original
order in order to learn the intent.
Table 2 Cross-checking interactions during case 2
Cross-checking interaction Commentary
Pharmacy cross-checks physician order (software): Not successful
Physician order for starting heparin at 18 drops an
hour at 6 a.m. today was verified by pharmacy
and distributed to the ward
Routinized cross-checking strategy
Same strategy for every medication
High false alarm rate
Production pressure
Nursing cross-checks physician order (software): Not successful
Pharmacy-verified physician order for starting
heparin at 18 drops an hour at 6 a.m. today was
verified by Registered Nurse assigned as primary
caregiver to patient during the night shift
Routinized cross-checking strategy
Same strategy for every medication
High false alarm rate
Typically batch process verification of orders
Patient cross-checks physician order: Not clear
Patient did not want heparin started because he was
going to get a computed axial tomography (CAT)
scan and then go home that day. There is a hep-lock
and no order for a CAT scan. Night nurse did
not hang medication
No one knows intent behind heparin order
Hep-lock order indicates can get heparin
Patient reports CAT scan planned
No order for CAT scan
Nurse does not access physician who wrote order
Little support on night shift to answer questions
On-call physician cross-checks physician order: Not clear
Night nurse called on-call physician who said that he
did not know why heparin had been ordered and
that it’s OK not to give it (but did not cancel the
order in the software package)
On-call physician does not access physician who wrote order
Verbal decision not to give heparin does not match written order
Nurse manager cross-checks cancellation of order: Not clear
Nurse manager asks incoming shift of nurses
listening to audiotape: why are they starting
heparin on him today? No response from nurses
(resolution of discrepancy not observed; unknown
if patient should have received heparin)
Handoff from night nurse to day nurse
Nurse manager routinely listens to handoff updates (unusual)
Nurse manager asks error checking question
Unclear plan or responsibility for resolution
Cogn Tech Work (2007) 9:155–162 159
123

(c) Case 3. The incoming nurse during the handoff
suspected a problem based on the high glucose
level, but did not detect the erroneous label. The
Clinical Nurse Specialist suspected a problem
but was only able to detect the erroneous label
following the decision of an interdisciplinary
team conference knowledgeable about the spe-
cific patient to employ a strategy to replace all
medications.
3. Benefits of collaborative cross-checks were not
always immediately realized:
(a) Case 1. The fellow’s decision to hold the car-
boplatin medication was likely influenced by the
cumulative effect of three cross-checks from the
pharmacist and one from the nurse.
(b) Case 3. The incoming nurse during the handoff
suspected a problem based on the high glucose
level and had been considering asking for help
from the Clinical Nurse Specialist ‘‘all day,’’ but
the erroneous label was not detected until some
time later.
4. Knowledge of ‘‘typical mistakes’’ might aid error
detection as well as serve as ‘‘red herrings’’:
(a) Case 1. Fellow and attending knew that it was
easy for first year fellows to confuse small cell and
non-small cell cancer treatments.
(b) Case 1. Pharmacist suspected the wrong dose,
which was more common than the wrong drug.
(c) Case 2. CNS conducted literature search to verify
that insulin dose was higher than used in other cases.
Table 3 Cross-checking interactions during case 3
Cross-checking interaction Commentary
Incoming nurse cross-checks outgoing nurse: Not successful
Outgoing nurse (face to face report): Unstable burn patient with
care efforts focused on improving respiratory status from
smoke inhalation. Patient has been running high glucose levels
(~500) for the last 3 h. She has increased the level of insulin
about 10 units/h for 10 h to try to get below 400. IV bag is
hanging with hand-labeled ‘‘Bicarb’’
Focus on respiratory status, not glucose levels
Incoming nurse: Why are you mixing bags by hand? Unusual method for mixing IV medications detected
Outgoing nurse: Because pharmacy was unable to keep up, I have
been mixing by hand bicarbonate and insulin
Slow dose increases add up to large changes over time
Fresh perspective triggers suspicion about glucose levels, but
not acted on until questioned by CNS
Clinical Nurse Specialist (CNS) cross-checks pharmacists: Successful (somewhat)
Pharmacy: We are running out of insulin in the main pharmacy.
Please alert the units to the problem until they can get
additional supplies
Pharmacy recognizes unusual situation but does not detect
erroneous substitution
CNS: How did this happen? Pharmacy ‘‘unable to keep up’’
Pharmacy: High usage by the burn patient Unusual request triggers CNS to get more information
CNS cross-checks outside official role
CNS cross-checks nurse: Successful (somewhat)
CNS: What’s going on with your patient? CNS starts troubleshooting high glucose levels
Nurse: I have been thinking about calling you all day because of
the difficulty in managing the glucose levels. I did not call
because I saw progress in the glucose falling from 500 to 400 in
the last few hours and our care efforts were focused on
respiratory status due to smoke inhalation. Since we are not
getting progress from the usual protocol steps in managing
glucose, can you give me a written plan of action?
CNS not responsible for other time-critical tasks
CNS: OK. She does literature search to confirm that insulin
dosage was very high. Arranges team conference with nurse,
physicians, pharmacists, and dietitian
Team troubleshoots unexpected response to high insulin doses: Successful (somewhat)
Team conference arranged with CNS, nurse, physicians,
pharmacists, dietitian. Team cannot identify reason for
patient’s unexpectedly low response to high insulin dose.
Group decision to replace all IV medications and solutions.
With no other changes in treatment, patient’s glucose falls
from 400 to 100 by 6AM the next day. Decided that best
explanation was that glucose was erroneously labeled IV bag
during hand mixing
Specific error not immediately detected
Generic ‘‘restart’’ strategy for IV medications/solutions
Time lag to detect change in glucose levels
Handoff to next nurse before normal levels
160 Cogn Tech Work (2007) 9:155–162
123

(d) Case 3. It is unusual for nurses to mix and label
IV bags.
5. Cross-checks were employed by people outside
official roles and responsibilities, and who were not
consumed by production pressures:
(a) Case 1. The pharmacist called the hospital’s
pharmacist with expertise in chemotherapy even
though she was not working or on-call that day.
(b) Case 2. The nurse manager overheard the handoff
update that described the questionable order.
Although listening to handoff updates was a
routine strategy for this individual, it is an unu-
sual role for nurse managers in general. She
encouraged resolution of the questionable order.
(c) Case 3. The Clinical Nurse Specialist (CNS) con-
ducted a series of activities without being asked to
do so that ultimately detected the erroneous
labeling on the IV bag.
6. Strategies to render processes more observable
may increase resilience even when a specific erro-
neous assessment or action is not detected:
(a) Case 1. Senior physician first asked for the fellow
physician’s plan, critiqued it, and asked the fellow
to restate the new plan.
(b) Case 1. The incoming fellow physician reviewed
new medication orders for all patients after the
handoff.
(c) Case 2. Nurse manager overheard handoff up-
dates.
(d) Case 3. They ‘‘restarted’’ all medications when
the team could not explain the unusual patient
condition.
Overall, these findings confirm that collaborative
cross-checking can enhance system resilience. They
also confirm that routinized cross-checking will never
detect all erroneous assumptions and actions because
the reconceptualization of the problem from another
perspective is a cognitively effortful activity and the
benefits are not always realized by the person doing the
effort. Therefore, a strong reliance on routinized cross-
checking to detect erroneous assumptions and actions
should not be pursued.
Probably the most important contribution of this
research is the finding that specialized knowledge
might be required by the individual performing the
collaborative cross-check. This finding implies that
using automated software or relatively inexperienced
or less knowledgeable personnel to perform cross-
checks might not be a particularly effective approach.
In order to enhance resilience, these findings suggest
two approaches to increase adaptive capacity both
before and after a problem is detected. First, personnel
with weakly defined roles who are not consumed by
production pressures can support collaborative cross-
checking and other cognitively challenging sensemak-
ing functions. Second, processes can be rendered more
observable either by explicitly communicating the
rationale behind a plan and the intent behind an order
or by supporting the ability for people in loosely cou-
pled roles to ‘‘listen in’’ on planning discussions, such
as by having nurses or pharmacists attend physician
rounds.
Acknowledgments This research was supported by the
Department of Veteran’s Affairs, Veteran’s Health Adminis-
tration, Health Services Research and Development Service
(Cincinnati REAP Developing Center of Excellence) and the
Department of Defense (BAA-001-04). This work was also
carried out through participation in the Advanced Decision
Architectures Collaborative Technology Alliance sponsored by
the U.S. Army Research Laboratory under Cooperative Agree-
ment DAAD19-01-2-0009. A VA HSR&D Merit Review Entry
Program Award supported Emily Patterson. The views ex-
pressed in this article are those of the authors and do not nec-
essarily represent the view of the Department of Veterans
Affairs.
References
Behara R, Wears R, Perry S, Eisenberg E, Murphy L, Vander-
hoef M, Shapiro M, Beach C, Croskerry P, Cosby K (2005)
A conceptual framework for studying the safety of transi-
tions in emergency care. Adv Patient Safety 2309–2321
Brown JP (2005) Ethical dilemmas in healthcare. In: Patankar M,
Brown JP, Treadwell MD (eds) Ethics in safety. Cases from
aviation, healthcare, and occupational and environmental
health. Ashgate, Burlington VT
Carthy J, de Leval MR, Reason JT (2001) Institutional resilience
in healthcare systems. Qual Health Care 10:29–32
Cook RI, Render ML, Woods DD (2000) Gaps in the continuity
of care and progress on patient safety. Br Med J 320:791–794
Cook RI, Rasmussen J (2005) Going solid: a model of system
dynamics and consequences for patient safety. Qual Safety
Health Care 14:130–134
de Leval MR, Carthey J, Wright DJ, Farewell VT, Reason JT
(2000). Human factors and cardiac surgery: a multicenter
study. J Thorac Cardiovasc Surg 119(4 Pt 1):661–672
Cooper JB, Long CD, Newbower RS, et al (1982) Critical
incidents associated with intraoperative exchanges of anes-
thesia personnel. Anesthesiology 56(6):456–461
Dekker SWA (2005) Ten questions about human error: a new
view of human factors and system safety. Lawrence
Erlbaum, Hillsdale, NJ
Erev I, Gopher D, Itkin R, Greenshpan Y (1995) Toward a
generalization of signal detection theory to n-person games:
the example of two person safety problem. J Math Psychol
39:360–376
Guerlain S, Smith PJ, Obradovich JH, Rudmann S, Strohm P,
Smith J, Svirbely J (1996) Dealing with brittleness in the
design of expert systems for immunohematology. Immuno-
hematology 12:101–107
Hollnagel E (2004) Barriers and accident prevention. Ashgate
Publishing, Aldershot
Cogn Tech Work (2007) 9:155–162 161
123

Hollnagel E, Woods DD, Leveson N (2006) Resilience engi-
neering: concepts and precepts. Ashgate Publishing, Alder-
shot
Klein GA, Calderwood R, MacGregor D (1989) Critical decision
method for eliciting knowledge. IEEE Trans Syst Man
Cybern 19(3):462–472
Klein G (2006) The strengths and limitations of teams for
detecting problems. Cogn Technol Work (in press)
Klein G, Pliske R, Crandall B, Woods D (2005) Problem
detection. Cogn Technol Work 7(1):14–28
Leape L, Cullen DJ, Clapp MD, Burdick E, Demonaco HJ,
Erickson JI, Bates DW (1999) Pharmacist participation on
physician rounds and adverse drug events in the intensive
care unit. JAMA 282:267–270
Miller A, Xiao Y (2006) Multi-level strategies to achieve
resilience for an organisation operating at capacity: a case
study at a trauma centre. Cogn Technol Work (in press)
Patterson ES, Cook RI, Render ML (2002) Improving patient
safety by identifying side effects from introducing bar
coding in medication administration. J Am Med Inform
Assoc 9(5):540–553
Patterson ES, Roth EM, Woods DD, Chow R, Gomes JO (2004)
Handoff strategies in settings with high consequences for
failure: lessons for health care operations. Int J Qual Health
Care 16(2):125–132
Patterson ES, Cook RI, Woods DD, Render ML (2004)
Examining the complexity behind a medication error:
generic patterns in communication. IEEE Trans Syst Man
Cybern Part A 34(6):749–756
Rasmussen J (1990) The role of error in organizing behavior.
Ergonomics 33:1185–1199
Sarter N (2000) Error types and related error detection mech-
anisms in the aviation domain: an analysis of aviation safety
reporting system incident reports. Int J Aviat Psychol
10(2):189–195
Sutcliffe K, Vogus T (2003) Organizing for resilience. In:
Cameron KS, Dutton IE, Quinn RE (eds) Positive organi-
zational scholarship. Berrett-Koehler, San Francisco, pp 94–
110
Uhlig PN, Brown J, Nason AK, Camelio A, Kendall E (2002)
System innovation: concord hospital. Jt Comm J Qual
Improv 28(12):666–672
Weick KE, Sutcliffe KM, Obstfeld D (1999) Organizing for high
reliability: processes of collective mindfulness. Res Org
Behav 21:13–81
Westrum R (2006) A typology of resilience situations. In:
Hollnagel E, Woods DD, Leveson N (eds) Resilience
engineering. Ashgate, Aldershott, pp 55–67
Williams PS (1986) Processing demands, training, and the
vigilance decrement. Hum Factors 28:567–579
Woods DD (2005) Creating foresight: lessons for resilience from
Columbia. In: Starbuck WH, Farjoun M (eds) Organization
at the limit: NASA and the Columbia disaster. Blackwell
Woods DD, Shattuck LG (2000) Distant supervision—local
action given the potential for surprise. Cogn Technol Work
2:86–96
Woods DD, O’Brien J, Hanes LF (1987) Human factors
challenges in process control: the case of nuclear power
plants. In: Salvendy G (ed) Handbook of human factors/
ergonomics. Wiley, New York
162 Cogn Tech Work (2007) 9:155–162
123