ArticlePDF Available

Collaborative Cross-Checking to Enhance Resilience

  • Adaptive Capacity Labs


Resilience, the ability to adapt or absorb disturbance, disruption, and change, may be increased by team processes in a complex, socio-technical system. In particular, collaborative cross-checking is a strategy where at least two individuals or groups with different perspectives examine the others' assumptions and/or actions to assess validity or accuracy. With this strat- egy, erroneous assessments or actions can be detected quickly enough to mitigate or eliminate negative con- sequences. In this paper, we seek to add to the under- standing of the elements that are needed in effective cross-checking and the limitations of the strategy. We define collaborative cross-checking, describe in detail three healthcare incidents where collaborative cross- checks played a key role, and discuss the implications of emerging patterns.
Collaborative cross-checking to enhance resilience
Emily S. Patterson ÆDavid D. Woods Æ
Richard I. Cook ÆMarta L. Render
Received: 1 October 2005 / Accepted: 1 October 2006 / Published online: 12 December 2006
Springer-Verlag London Limited 2006
Abstract Resilience, the ability to adapt or absorb
disturbance, disruption, and change, may be increased
by team processes in a complex, socio-technical system.
In particular, collaborative cross-checking is a strategy
where at least two individuals or groups with different
perspectives examine the others’ assumptions and/or
actions to assess validity or accuracy. With this strat-
egy, erroneous assessments or actions can be detected
quickly enough to mitigate or eliminate negative con-
sequences. In this paper, we seek to add to the under-
standing of the elements that are needed in effective
cross-checking and the limitations of the strategy. We
define collaborative cross-checking, describe in detail
three healthcare incidents where collaborative cross-
checks played a key role, and discuss the implications
of emerging patterns.
1 Introduction
There is a growing consensus that resilience, the ability
to adapt or absorb disturbance, disruption, and change,
may be increased by team processes in a complex, so-
cio-technical system (Hollnagel et al. 2006; Rasmussen
1990; Weick et al. 1999). The concept of resilience is
founded upon the belief that failures are breakdowns
in the normal adaptive processes necessary to cope
with the complexity of the real world and that success
relates to organizations, groups and individuals who
produce resilient systems that recognize and adapt to
variations, changes and surprises (Rasmussen et al.
1994; Cook et al. 2000; Woods and Shattuck 2002;
Sutcliffe and Vogel 2003).
This body of work has led to the emergence of
techniques for Resilience Engineering (Hollnagel et al.
2005; Cook and Rasmussen 2005). Resilience Engi-
neering focuses on what sustains or erodes the adaptive
capacities of the human–technical system in a changing
environment. The focus is monitoring organizational
decision making to assess the risk that the organization
is operating nearer to safety boundaries than it realizes
in response to pressures to be ‘‘faster better cheaper.’’
More generally, an organization has levels of adaptive
capacity that needs to meet or exceed a wide range
of environmental pressures and demands (Westrum
2006). Studies of how systems are resilient and how
E. S. Patterson (&)
VA Getting at Patient Safety (GAPS) Center,
Cincinnati VAMC, Institute for Ergonomics,
Ohio State University, 210 Baker Systems,
1971 Neil Avenue, Columbus, OH 43210, USA
D. D. Woods
Institute for Ergonomics, Ohio State University,
210 Baker Systems, 1971 Neil Avenue, Columbus,
OH 43210, USA
R. I. Cook
Cognitive Technologies Laboratory,
Department of Anesthesia and Critical Care,
University of Chicago, 5841 S. Maryland Avenue MC4028,
Chicago, IL 60637, USA
M. L. Render
Department of Internal Medicine,
VA Getting at Patient Safety (GAPS) Center,
Cincinnati VAMC, University of Cincinnati,
3200 Vine Street, Cincinnati, OH 45220, USA
Cogn Tech Work (2007) 9:155–162
DOI 10.1007/s10111-006-0054-8
breakdowns in resilience occur have begun to show
how to measure and improve system safety (e.g., Car-
they et al. 2001; Patterson et al. 2002; Woods 2005;
Miller and Xiao 2006).
These studies point to collaborative cross-checking as
a critical component of resilience because erroneous
assessments or actions can be detected quickly enough
to mitigate or eliminate negative consequences. Studies
of problem detection in aviation and nuclear power
both found that mis-assessments of the situation at hand
were only corrected when fresh perspectives entered
the situation (Woods et al. 1987; Sarter 2000). Problem
detection is challenging because it often requires rec-
onceptualizing the situation from a different stance and
experience base (Klein et al. 2005; Klein 2006).
The findings regarding the effectiveness of collabo-
rative cross-checking are mixed in the literature. On the
positive side, the inclusion of pharmacists during phy-
sician rounds was associated with fewer adverse drug
events (Leape et al. 1999). Similarly, interdisciplinary
rounds with the entire care team in a Surgical Intensive
Care Unit (SICU) was associated with decreased mor-
tality, increased patient satisfaction, and increased
quality of work life (Uhlig et al. 2002). Cross-checking
through machine critiquers in blood banking also en-
hanced performance even in situations where both the
machine advisor and the human practitioner had great
difficulties performing alone (Guerlain et al. 1999). On
the negative side, routinized cross-checking, particu-
larly when effortful attention is required and the base
rate for problems is low, is rarely effective, often re-
ferred to as the ‘‘vigilance decrement’’ (Williams 1986).
A number of studies have found that having two
observers perform a monitoring task does not neces-
sarily lead to improved performance (e.g., Erev et al.
1995). Similarly, handoff strategies are an important
concern today because well-designed handoffs can
increase resilience whereas poorly designed handover
processes can contribute to adverse events (Cooper
et al. 1982; Patterson et al. 2004; Behera et al. 2005).
In this paper, we seek to add to the understanding of
the elements that are needed in effective cross-check-
ing and the limitations of the strategy. We define col-
laborative cross-checking, describe in detail three
healthcare incidents where collaborative cross-checks
played a key role, and discuss the implications of
emerging patterns.
2 Distinctions in collaborative cross-checking
Collaborative cross-checking is a strategy where at least
two individuals or groups with different perspectives
examine the others’ assumptions and/or actions to as-
sess validity or accuracy. These perspectives can differ
on a number of dimensions, including goals, responsi-
bilities, functions, authority, stance, expertise, re-
sources, methodologies, and knowledge or information
of various types. The primary intent of a collaborative
cross-check is to detect an erroneous assessment or
action, although a number of other benefits may be
realized. Collaborative cross-check interactions incur
costs such as interruptions, increased cognitive work,
increased complexity resulting from changes to plans,
diffusion of responsibility, and coordinative costs such
as identifying and scheduling appropriate individuals.
2.1 Scope of change following collaborative cross-
A cross-checking interaction can result in changes to
activities and outcomes. The anticipated scope of
change stemming from a coordination event is largely
determined by the relationship between the partici-
pating individuals or teams. Distinctions in relation-
ships include:
(1) Asymmetric hierarchy and expertise with similar
perspective (e.g., attending physician and resi-
dent physician).
(2) Asymmetric expertise, non-hierarchical with
similar perspective (e.g., mentor and mentee).
(3) Symmetric expertise with similar perspective:
(a) Same time (e.g., resident physicians reporting to
the same attending physician with different pa-
tients during the same shift).
(b) Different times (e.g., handoff from resident
physician to resident physician).
(c) During escalating situations (e.g., nurse provides
aid to another nurse in ‘‘on-call’’ manner).
(4) Functionally distinct, overlapping roles:
(a) Anomalous situations (e.g., code team and
(b) Situations requiring specialized expertise (e.g.,
cardiologist physician and internal medicine
(c) Nominal situations (e.g., physician and nurse
caring for the same patient).
(5). Weakly defined role relationships
(a) Integration resource (e.g., case manager).
(b) ‘‘Floater’’ resource (e.g., Clinical Nurse Special-
(c) Cross-cutting functional resource (e.g., patient
safety officer).
Although the extent of receptivity to changes to
ongoing and future plans following a cross-checking
156 Cogn Tech Work (2007) 9:155–162
interaction is somewhat predictable based on these
relationships, in practice there is immense variability.
People in leadership positions impact social norms for
receptivity expectations, whether or not they partici-
pate directly in collaborative cross-checks. In addition,
individual differences and social relationships contrib-
ute to variability.
Note that the actual change and perceived change,
often as portrayed in official documentation, following
a collaborative cross-check can differ. In many situa-
tions, changing a plan following an interaction is posi-
tively rewarded. In these cases, the scope of change
might be exaggerated in official documentation. In
other situations, a large scope of change might imply
that prior plans were erroneous, which might negatively
impact reputation. In these cases, the actual change
may be larger than what is officially documented.
2.2 Short-term and long-term benefits
of collaborative cross-checking
The primary intent of a collaborative cross-check is to
detect an erroneous assessment or action in the short
term. Nevertheless, a number of other benefits may be
realized, including improved plans and ‘‘second order’’
adaptations that improve system resilience over the
long term.
Regarding improvements to ongoing and future
plans, the following short-term benefits could poten-
tially be realized:
(1) reveal hidden assumptions;
(2) clarify goal trade-offs;
(3) explore new regions of a solution space to satisfy
competing goals;
(4) identify unintended consequences of actions
(‘‘side effects’’);
(5) identify exceptions and ‘‘boundary conditions’’;
(6) identify possible contingencies;
(7) identify information gaps;
(8) identify influential people who might support or
obstruct plan.
In addition, coordination might be improved by a
collaborative cross-check, even when no erroneous
assessments or actions are detected. Short-term coor-
dination benefits may include:
(1) Improved ability to distribute tasks among indi-
viduals or groups to reduce redundant efforts and
fill gaps in knowledge.
(2) Increased awareness of perspectives of other
individuals and groups (i.e., calibrated ‘‘common
(3) Increased awareness of others’ needs for infor-
mation to promote timely information updates
and better anticipate interaction requests.
(4) Improved ability to anticipate changes to others’
perspectives when situations change.
(5) Reduced coordination costs for future meetings,
such as by improved ability to anticipate sched-
uling conflicts.
Finally, collaborative cross-checks may prompt
investment in long-term system adaptation, ultimately
increasing system resilience over a longer period than a
single instance. Interestingly, these adaptations may
reduce the benefits of individual cross-checking inter-
actions by removing or reducing system-level con-
tributors to erroneous assessments and actions. For
example, a hospital might find that calls to rapid re-
sponse teams (RRTs) to identify which patients are
likely to suffer cardiac or respiratory arrest gradually
reduce over time. Rather than signaling a reduction in
system resilience, however, these reductions could in-
stead indicate a transfer of specialized knowledge from
the RRT to the floor nurses, who then require the RRT
less frequently. Long-term ‘‘second order’’ benefits of a
series of cross-checking interactions might include:
(1) transfer of knowledge across perspectives;
(2) increased ‘‘meta-knowledge’’ of others’ perspec-
(3) increased ‘‘team identity’’ across previously dis-
tinct individuals or groups;
(4) improve coordination across ‘‘stovepiped’’ groups
in an organization;
(5) identify ineffective cross-checking strategies;
(6) system and training redesign.
3 Three incidents employing collaborative
An extant corpus of approximately 30 voluntarily re-
ported healthcare incidents, collected to explore a
broad set of patient safety issues, was reviewed. Three
cases were selected on the basis of collaborative cross-
checks playing a key role and firsthand experience with
the data collection.
3.1 Overview of cases
3.1.1 Case 1: chemotherapy overdose
In the first case (Table 1), an oncology fellow physician
erroneously substituted the medication navelbine for
Cogn Tech Work (2007) 9:155–162 157
the intended etoposide during ordering. The patient
had a prolonged hospitalization with severe leuco-
3.1.2 Case 2: questionable heparin order
In the second case (Table 2), an order to initiate hep-
arin when a patient was planned to be discharged was
questioned, as well as the subsequent decision by an
on-call physician to verbally discontinue the order
without knowing the rationale for the original order.
There were no apparent negative impacts to the pa-
3.1.3 Case 3: erroneously labeled IV bag
In the third case (Table 3), a patient was administered
glucose via a bag that was labeled ‘‘bicarb’’. There
were several attempts to troubleshoot high glucose
levels, which eventually led to detection of the erro-
neous label. There were no apparent negative long-
term impacts to the patient.
Table 1 Cross-checking interactions during case 1
Cross-checking interaction Commentary
Attending cross-checks fellow physician (phone): Successful
Fellow: Plan is the same as an outpatient from the previous week
(administering navelbine on a weekly basis)
Possibly routine strategy for senior physician to first ask junior
physician for a plan, which is then critiqued
Attending: No, because the prior case was a non-small cell as
opposed to a small cell lung cancer case. Plan should be VP-16
(etoposide) 100 mg/m
on days 1, 2, 3 and Carboplatin 6 AUC
on day 1, to be repeated every 3–4 weeks
Possibly common misconception that small cell and non-small
cell cancers are treated similarly
Pharmacist cross-checks oncology fellow physician (phone): Not successful
Fellow: I will order navelbine at 100 mg/m
and Carboplatin 6
AUC to be given today
Embedded in hospital policy is a defense against cross-checking
by a pharmacist who does not specialize in chemotherapy
Pharmacist: The hospital policy does not allow initiating new
chemotherapy regimens on the weekends
Rationale for the hospital policy is not stated during interaction
Fellow: I discussed the case with the attending and it is critical to
administer the medications as quickly as possible
Fellow assumes that attending physician is aware of the policy:
therefore assumes that there is no new information from the
pharmacist (who has no specialized oncological expertise)
that would overturn decision
Pharmacist cross-checks fellow physician (phone): Not successful
Pharmacist: The package insert says that the standard dose is
30 mg/m
for navelbine. Are you sure that you want
100 mg/m
Pharmacist cross-checks using package insert information
because pharmacist with chemotherapy expertise
was not home
Fellow: Yes Package insert included that navelbine is administered with
Pharmacist cross-checks fellow physician (phone): Not successful
Pharmacist: I am giving the medications to the nurse to hang. Are
you sure that you want 100 mg/m
Asking the same question again resulted in the same response
Fellow: Yes New information was not conveyed (many bags required for the
dose, nurse asked pharmacist to call again because she was
not very familiar with the medication)
Nurse cross-checks fellow physician (phone): Successful (somewhat)
Nurse: I administered the navelbine. The patient is diaphoretic,
short of breath, and hypertensive. Do you want to hold the
New information and suggestion of a specific action (hold
medication) prompted a change of plan
Fellow: Yes, hold the carboplatin Change of plan was later judged poor because side effects are
usually managed rather than halting a chemotherapy
treatment prior to completion
Physician unaware that nurse is relatively inexperienced with
chemotherapy medication
Incoming fellow cross-checks fellow physician: Successful
Incoming fellow (reviewing patient record): Navelbine is not the
right medication for this diagnosis, it is VP-16 (etoposide)
Person with one additional year of experience detected
substitution while reviewing medications in preparation
for taking over
Fellow: Discusses impacts, how to respond, discusses recovery
plan with attending and incoming fellow, attending informs
Once detected, easy to recognize erroneous action
158 Cogn Tech Work (2007) 9:155–162
3.2 Collaborative cross-checks in case 1
The first case (Table 1) is a chemotherapy overdose
resulting primarily from erroneous substitution of a
medication name. The case was collected via a se-
quence of critical decision method interviews (Klein
1989) conducted individually with the oncology fellow,
oncology attending, pharmacist, and nurse. This case is
described in detail in Patterson et al. (2004).
3.3 Collaborative cross-checks in case 2
The second case (Table 2) involves a questionable
heparin order. This case was collected via direct
observation of an acute care nurse.
3.4 Collaborative cross-checks in case 3
The third case involves medication administration by a
nurse from an erroneously labeled IV bag. The IV bag
was mixed and erroneously labeled at the point of care
by a nurse from a previous shift because pharmacy
resources were overwhelmed. This case was collected
during a critical decision method interview with a
Clinical Nurse Specialist.
4 Discussion of emerging patterns for effective
Although no strong conclusions can be drawn from
three case studies, there are suggestive emerging pat-
1. Routinized collaborative cross-checks were not
very effective:
(a) All cases. Routinized physician and nurse veri-
fication of physician orders did not successfully
detect erroneous assessments or actions.
2. Some collaborative cross-checks helped to detect
and recover from erroneous assessments and ac-
tions, but specialized knowledge and interdisci-
plinary interactions were often involved:
(a) Case 1. Incoming fellow immediately detected
erroneous medication order when reviewing the
chart, but not the pharmacist or nurse who did
not have as much specialized knowledge of
oncology as other staff.
(b) Case 2. Patient detected that the order was
questionable, but the night nurse, the on-call
physician, the next nurse, and the nurse manager
could not adequately recover because they did
not access the physician who wrote the original
order in order to learn the intent.
Table 2 Cross-checking interactions during case 2
Cross-checking interaction Commentary
Pharmacy cross-checks physician order (software): Not successful
Physician order for starting heparin at 18 drops an
hour at 6 a.m. today was verified by pharmacy
and distributed to the ward
Routinized cross-checking strategy
Same strategy for every medication
High false alarm rate
Production pressure
Nursing cross-checks physician order (software): Not successful
Pharmacy-verified physician order for starting
heparin at 18 drops an hour at 6 a.m. today was
verified by Registered Nurse assigned as primary
caregiver to patient during the night shift
Routinized cross-checking strategy
Same strategy for every medication
High false alarm rate
Typically batch process verification of orders
Patient cross-checks physician order: Not clear
Patient did not want heparin started because he was
going to get a computed axial tomography (CAT)
scan and then go home that day. There is a hep-lock
and no order for a CAT scan. Night nurse did
not hang medication
No one knows intent behind heparin order
Hep-lock order indicates can get heparin
Patient reports CAT scan planned
No order for CAT scan
Nurse does not access physician who wrote order
Little support on night shift to answer questions
On-call physician cross-checks physician order: Not clear
Night nurse called on-call physician who said that he
did not know why heparin had been ordered and
that it’s OK not to give it (but did not cancel the
order in the software package)
On-call physician does not access physician who wrote order
Verbal decision not to give heparin does not match written order
Nurse manager cross-checks cancellation of order: Not clear
Nurse manager asks incoming shift of nurses
listening to audiotape: why are they starting
heparin on him today? No response from nurses
(resolution of discrepancy not observed; unknown
if patient should have received heparin)
Handoff from night nurse to day nurse
Nurse manager routinely listens to handoff updates (unusual)
Nurse manager asks error checking question
Unclear plan or responsibility for resolution
Cogn Tech Work (2007) 9:155–162 159
(c) Case 3. The incoming nurse during the handoff
suspected a problem based on the high glucose
level, but did not detect the erroneous label. The
Clinical Nurse Specialist suspected a problem
but was only able to detect the erroneous label
following the decision of an interdisciplinary
team conference knowledgeable about the spe-
cific patient to employ a strategy to replace all
3. Benefits of collaborative cross-checks were not
always immediately realized:
(a) Case 1. The fellow’s decision to hold the car-
boplatin medication was likely influenced by the
cumulative effect of three cross-checks from the
pharmacist and one from the nurse.
(b) Case 3. The incoming nurse during the handoff
suspected a problem based on the high glucose
level and had been considering asking for help
from the Clinical Nurse Specialist ‘‘all day,’’ but
the erroneous label was not detected until some
time later.
4. Knowledge of ‘‘typical mistakes’’ might aid error
detection as well as serve as ‘‘red herrings’’:
(a) Case 1. Fellow and attending knew that it was
easy for first year fellows to confuse small cell and
non-small cell cancer treatments.
(b) Case 1. Pharmacist suspected the wrong dose,
which was more common than the wrong drug.
(c) Case 2. CNS conducted literature search to verify
that insulin dose was higher than used in other cases.
Table 3 Cross-checking interactions during case 3
Cross-checking interaction Commentary
Incoming nurse cross-checks outgoing nurse: Not successful
Outgoing nurse (face to face report): Unstable burn patient with
care efforts focused on improving respiratory status from
smoke inhalation. Patient has been running high glucose levels
(~500) for the last 3 h. She has increased the level of insulin
about 10 units/h for 10 h to try to get below 400. IV bag is
hanging with hand-labeled ‘‘Bicarb’’
Focus on respiratory status, not glucose levels
Incoming nurse: Why are you mixing bags by hand? Unusual method for mixing IV medications detected
Outgoing nurse: Because pharmacy was unable to keep up, I have
been mixing by hand bicarbonate and insulin
Slow dose increases add up to large changes over time
Fresh perspective triggers suspicion about glucose levels, but
not acted on until questioned by CNS
Clinical Nurse Specialist (CNS) cross-checks pharmacists: Successful (somewhat)
Pharmacy: We are running out of insulin in the main pharmacy.
Please alert the units to the problem until they can get
additional supplies
Pharmacy recognizes unusual situation but does not detect
erroneous substitution
CNS: How did this happen? Pharmacy ‘‘unable to keep up’’
Pharmacy: High usage by the burn patient Unusual request triggers CNS to get more information
CNS cross-checks outside official role
CNS cross-checks nurse: Successful (somewhat)
CNS: What’s going on with your patient? CNS starts troubleshooting high glucose levels
Nurse: I have been thinking about calling you all day because of
the difficulty in managing the glucose levels. I did not call
because I saw progress in the glucose falling from 500 to 400 in
the last few hours and our care efforts were focused on
respiratory status due to smoke inhalation. Since we are not
getting progress from the usual protocol steps in managing
glucose, can you give me a written plan of action?
CNS not responsible for other time-critical tasks
CNS: OK. She does literature search to confirm that insulin
dosage was very high. Arranges team conference with nurse,
physicians, pharmacists, and dietitian
Team troubleshoots unexpected response to high insulin doses: Successful (somewhat)
Team conference arranged with CNS, nurse, physicians,
pharmacists, dietitian. Team cannot identify reason for
patient’s unexpectedly low response to high insulin dose.
Group decision to replace all IV medications and solutions.
With no other changes in treatment, patient’s glucose falls
from 400 to 100 by 6AM the next day. Decided that best
explanation was that glucose was erroneously labeled IV bag
during hand mixing
Specific error not immediately detected
Generic ‘‘restart’’ strategy for IV medications/solutions
Time lag to detect change in glucose levels
Handoff to next nurse before normal levels
160 Cogn Tech Work (2007) 9:155–162
(d) Case 3. It is unusual for nurses to mix and label
IV bags.
5. Cross-checks were employed by people outside
official roles and responsibilities, and who were not
consumed by production pressures:
(a) Case 1. The pharmacist called the hospital’s
pharmacist with expertise in chemotherapy even
though she was not working or on-call that day.
(b) Case 2. The nurse manager overheard the handoff
update that described the questionable order.
Although listening to handoff updates was a
routine strategy for this individual, it is an unu-
sual role for nurse managers in general. She
encouraged resolution of the questionable order.
(c) Case 3. The Clinical Nurse Specialist (CNS) con-
ducted a series of activities without being asked to
do so that ultimately detected the erroneous
labeling on the IV bag.
6. Strategies to render processes more observable
may increase resilience even when a specific erro-
neous assessment or action is not detected:
(a) Case 1. Senior physician first asked for the fellow
physician’s plan, critiqued it, and asked the fellow
to restate the new plan.
(b) Case 1. The incoming fellow physician reviewed
new medication orders for all patients after the
(c) Case 2. Nurse manager overheard handoff up-
(d) Case 3. They ‘‘restarted’’ all medications when
the team could not explain the unusual patient
Overall, these findings confirm that collaborative
cross-checking can enhance system resilience. They
also confirm that routinized cross-checking will never
detect all erroneous assumptions and actions because
the reconceptualization of the problem from another
perspective is a cognitively effortful activity and the
benefits are not always realized by the person doing the
effort. Therefore, a strong reliance on routinized cross-
checking to detect erroneous assumptions and actions
should not be pursued.
Probably the most important contribution of this
research is the finding that specialized knowledge
might be required by the individual performing the
collaborative cross-check. This finding implies that
using automated software or relatively inexperienced
or less knowledgeable personnel to perform cross-
checks might not be a particularly effective approach.
In order to enhance resilience, these findings suggest
two approaches to increase adaptive capacity both
before and after a problem is detected. First, personnel
with weakly defined roles who are not consumed by
production pressures can support collaborative cross-
checking and other cognitively challenging sensemak-
ing functions. Second, processes can be rendered more
observable either by explicitly communicating the
rationale behind a plan and the intent behind an order
or by supporting the ability for people in loosely cou-
pled roles to ‘‘listen in’’ on planning discussions, such
as by having nurses or pharmacists attend physician
Acknowledgments This research was supported by the
Department of Veteran’s Affairs, Veteran’s Health Adminis-
tration, Health Services Research and Development Service
(Cincinnati REAP Developing Center of Excellence) and the
Department of Defense (BAA-001-04). This work was also
carried out through participation in the Advanced Decision
Architectures Collaborative Technology Alliance sponsored by
the U.S. Army Research Laboratory under Cooperative Agree-
ment DAAD19-01-2-0009. A VA HSR&D Merit Review Entry
Program Award supported Emily Patterson. The views ex-
pressed in this article are those of the authors and do not nec-
essarily represent the view of the Department of Veterans
Behara R, Wears R, Perry S, Eisenberg E, Murphy L, Vander-
hoef M, Shapiro M, Beach C, Croskerry P, Cosby K (2005)
A conceptual framework for studying the safety of transi-
tions in emergency care. Adv Patient Safety 2309–2321
Brown JP (2005) Ethical dilemmas in healthcare. In: Patankar M,
Brown JP, Treadwell MD (eds) Ethics in safety. Cases from
aviation, healthcare, and occupational and environmental
health. Ashgate, Burlington VT
Carthy J, de Leval MR, Reason JT (2001) Institutional resilience
in healthcare systems. Qual Health Care 10:29–32
Cook RI, Render ML, Woods DD (2000) Gaps in the continuity
of care and progress on patient safety. Br Med J 320:791–794
Cook RI, Rasmussen J (2005) Going solid: a model of system
dynamics and consequences for patient safety. Qual Safety
Health Care 14:130–134
de Leval MR, Carthey J, Wright DJ, Farewell VT, Reason JT
(2000). Human factors and cardiac surgery: a multicenter
study. J Thorac Cardiovasc Surg 119(4 Pt 1):661–672
Cooper JB, Long CD, Newbower RS, et al (1982) Critical
incidents associated with intraoperative exchanges of anes-
thesia personnel. Anesthesiology 56(6):456–461
Dekker SWA (2005) Ten questions about human error: a new
view of human factors and system safety. Lawrence
Erlbaum, Hillsdale, NJ
Erev I, Gopher D, Itkin R, Greenshpan Y (1995) Toward a
generalization of signal detection theory to n-person games:
the example of two person safety problem. J Math Psychol
Guerlain S, Smith PJ, Obradovich JH, Rudmann S, Strohm P,
Smith J, Svirbely J (1996) Dealing with brittleness in the
design of expert systems for immunohematology. Immuno-
hematology 12:101–107
Hollnagel E (2004) Barriers and accident prevention. Ashgate
Publishing, Aldershot
Cogn Tech Work (2007) 9:155–162 161
Hollnagel E, Woods DD, Leveson N (2006) Resilience engi-
neering: concepts and precepts. Ashgate Publishing, Alder-
Klein GA, Calderwood R, MacGregor D (1989) Critical decision
method for eliciting knowledge. IEEE Trans Syst Man
Cybern 19(3):462–472
Klein G (2006) The strengths and limitations of teams for
detecting problems. Cogn Technol Work (in press)
Klein G, Pliske R, Crandall B, Woods D (2005) Problem
detection. Cogn Technol Work 7(1):14–28
Leape L, Cullen DJ, Clapp MD, Burdick E, Demonaco HJ,
Erickson JI, Bates DW (1999) Pharmacist participation on
physician rounds and adverse drug events in the intensive
care unit. JAMA 282:267–270
Miller A, Xiao Y (2006) Multi-level strategies to achieve
resilience for an organisation operating at capacity: a case
study at a trauma centre. Cogn Technol Work (in press)
Patterson ES, Cook RI, Render ML (2002) Improving patient
safety by identifying side effects from introducing bar
coding in medication administration. J Am Med Inform
Assoc 9(5):540–553
Patterson ES, Roth EM, Woods DD, Chow R, Gomes JO (2004)
Handoff strategies in settings with high consequences for
failure: lessons for health care operations. Int J Qual Health
Care 16(2):125–132
Patterson ES, Cook RI, Woods DD, Render ML (2004)
Examining the complexity behind a medication error:
generic patterns in communication. IEEE Trans Syst Man
Cybern Part A 34(6):749–756
Rasmussen J (1990) The role of error in organizing behavior.
Ergonomics 33:1185–1199
Sarter N (2000) Error types and related error detection mech-
anisms in the aviation domain: an analysis of aviation safety
reporting system incident reports. Int J Aviat Psychol
Sutcliffe K, Vogus T (2003) Organizing for resilience. In:
Cameron KS, Dutton IE, Quinn RE (eds) Positive organi-
zational scholarship. Berrett-Koehler, San Francisco, pp 94–
Uhlig PN, Brown J, Nason AK, Camelio A, Kendall E (2002)
System innovation: concord hospital. Jt Comm J Qual
Improv 28(12):666–672
Weick KE, Sutcliffe KM, Obstfeld D (1999) Organizing for high
reliability: processes of collective mindfulness. Res Org
Behav 21:13–81
Westrum R (2006) A typology of resilience situations. In:
Hollnagel E, Woods DD, Leveson N (eds) Resilience
engineering. Ashgate, Aldershott, pp 55–67
Williams PS (1986) Processing demands, training, and the
vigilance decrement. Hum Factors 28:567–579
Woods DD (2005) Creating foresight: lessons for resilience from
Columbia. In: Starbuck WH, Farjoun M (eds) Organization
at the limit: NASA and the Columbia disaster. Blackwell
Woods DD, Shattuck LG (2000) Distant supervision—local
action given the potential for surprise. Cogn Technol Work
Woods DD, O’Brien J, Hanes LF (1987) Human factors
challenges in process control: the case of nuclear power
plants. In: Salvendy G (ed) Handbook of human factors/
ergonomics. Wiley, New York
162 Cogn Tech Work (2007) 9:155–162
... Therefore, a signaling mechanism is needed to decide when and how to increase or decrease participation (Maguire, 2020;Reynolds, 2020). Crosschecking, where two or more members of the team with different perspectives examine each other's assumptions and actions, can functionally supply this signal; however, the cross-checks must be carefully architected to produce the intended diversity instead of a routinized redundancy (Patterson et al., 2007). ...
... As a two-way relationship between agents, effective collaborative cross-checking must be both well-delivered and well-received to beneficially impact the system. To deliver crosschecks, personnel must have spare capacity to engage in challenging sensemaking functions (Patterson et al., 2007) and possess deep, overlapping competencies (with the receiver) that surpass their own individual stated responsibilities. To receive cross-checks, personnel must explicitly communicate the rationale and intent behind actions and decisions (Patterson et al., 2007), make their actions and decisions openly observable to others (Klein, Feltovich, et al., 2004;Patterson et al., 2007), and actively invite others to cross-check (Rayo et al., 2013). ...
... To deliver crosschecks, personnel must have spare capacity to engage in challenging sensemaking functions (Patterson et al., 2007) and possess deep, overlapping competencies (with the receiver) that surpass their own individual stated responsibilities. To receive cross-checks, personnel must explicitly communicate the rationale and intent behind actions and decisions (Patterson et al., 2007), make their actions and decisions openly observable to others (Klein, Feltovich, et al., 2004;Patterson et al., 2007), and actively invite others to cross-check (Rayo et al., 2013). ...
Full-text available
Any clinical decision support (CDS) design project integrating computational technologies with clinician workflows will require the merging of multiple perspectives and fields of expertise in multidisciplinary teams. Much like the tools these teams aim to create, the team itself will need to continuously build, monitor, and repair a mutually beneficial relationship between each of its members. From our experience during the early development stages of an AI-enabled CDS tool for hospital-acquired infection (HAI) prevention, we abstract three central tenets of a symbiotic design process we have found to be vital for aligning goals, priorities, mental models, and techniques among a multidisciplinary team: (1) recurrent bottom-up feedback, (2) continual model (re-)alignment, and (3) openness to co-direction. With regards to these tenets, we discuss the successes and challenges our team has faced during the symbiotic design process through a series of vignettes and how these experiences coalescing diverse human design teams can influence the design of human-machine teams.
... The analysis was applied to all publications equally. Difficulties encountered during the process of classifying some publications into categories were resolved using the collaborative cross-checking strategy [38]. ...
Full-text available
During the COVID-19 pandemic, online media were the most widely used sources of scientific information. Often, they are also the only ones on science-related topics. Research has shown that much of the information available on the Internet about the health crisis lacked scientific rigor, and that misinformation about health issues can pose a threat to public health. In turn, millions of Catholics were found to be demonstrating against vaccination against COVID-19 based on "false" and misleading religious arguments. This research analyses publications about the vaccine in Catholic online media with the aim of understanding the presence of information (and misinformation) in this community. An algorithm designed for each media outlet collected COVID-19 vaccine-related publications from 109 Catholic media outlets in five languages. In total, 970 publications were analysed for journalistic genres, types of headlines and sources of information. The results show that most publications are informative and most of their headlines are neutral. However, opinion articles have mostly negative headlines. Furthermore, a higher percentage of the opinion authors come from the religious sphere and most of the sources cited are religious. Finally, 35% of the publications relate the vaccine to the framing issue of abortion.
A review of scientific publications aimed at determining the basis for the formation of the resili-ence paradigm in the security sphere is provided. The main stages of the evolution of the resili-ence paradigm in the context of security are considered, including its origin, development and multifactorial impact on the security of critical systems and infrastructures at different levels. The definitions, concepts, and key ideas underlying the paradigm are examined in detail, high-lighting the fundamental principles that contributed to its emergence. Special attention is paid to the constructs underlying the resilience paradigm in the security domain. Emphasis is placed on their practical implementation in frameworks and international legislation.
Full-text available
The pressures of an everchanging world have impacted the ways in which service-based systems operate, along with their forms and boundaries. Resilience and survivability have been treated interchangeably when readying a system to remain true to its functions despite disturbances. Some situations prove the concepts may not always be the equivalent of the other, not even the consequence of the other. There may come scenarios where system components fail to adhere to certain predefined thresholds and cross a breaking point. It is therefore proposed in this study that systems can be survivable, instead of resilient, when they comply in time with the resurgence property. This property signifies the systematic behavior of overcoming a certain stagnation period and, after a time range, return as a transformed system with new functions and challenges. Through this study, it was detected that the symmetries between resilience and survivability are only superficial if systems suffer breakages after misconceiving the true causes of failure. Still, a lack of consensus among scientists and practitioners remains an issue when applying resilience and survivability in their own problems. Although workful, pushing to achieve a greater consensus would signify optimal performance in multifaceted systems involving technical, social, and economic challenges.
Full-text available
Numerous social changes and the dynamic development of information and communication technology require constant improvement of the edu- cational system, which should rapidly adapt to all changes in modern so- ciety. These changes also cause numerous difficulties and transformations of the educational system. Resilience is an important process that helps the educational system to adapt to new challenges and changes. In this paper, we focused on educational systems that successfully adapt to such changes, wanting to show why and in what way they are more successful than others, and how they evaluate student competencies.
This chapter explores the prescription and practice of safe work method statements (SWMS) to understand whether they enhance or hinder resilience engineering (RE). SWMS are a regulated construction safety strategy in Australia but their specific role in safety is unknown. In RE reconciling the gap between prescription and practice of work plays an important role in achieving safety; however, the specific links between SWMS, RE and safety have not been empirically investigated. Semi-structured interviews with managers, supervisors, and workers showed across three construction projects showed that SWMS as prescribed are a cognitive artefact, act as a form of control, involve a process, and act as a tool, while SWMS in practice were expected to provide protection and act as a tool. Findings suggest that SWMS, if used flexibly, will enhancing RE a construction safety strategy. This research provides empirical evidence on the utility of SWMS for improving construction safety, introduces an integrative framework for RE for investigating work-as-imagined and work-as-done, and provides additional insights of how social interactions initiated through SWMS can enhance safety and organizational behaviour. The research will be useful for developing and deploying SWMS deployed more effectively in construction projects.KeywordsConstruction safetyResilience engineeringSafe work method statementsConstruction project management
Managing complexity in safety–critical sociotechnical systems requires practitioners to adapt continuously to new challenges. However, adaptation has been subjected to a variety of interpretations depending on the particular research paradigm, which makes it difficult for practitioners to understand and apply to safety management. We use a systematic review of the literature on human adaptation in order to derive a model of team adaptation to guide further research and application to new circumstances by both researchers and practitioners. The cybernetic model and its associated taxonomy of adaptation strategies have been used in a retrospective analysis of human performance data. All elements of the taxonomy are explained in the context of an emergency scenario of Quanta’s flight QF32. Practical implications from this study are highlighted for modern Resilience and Safety II approaches to safety management.
Full-text available
Como compreender os mecanismos ou os processos que, empresas retalhistas independentes de Lisboa, criadas ao longo do século XIX e algumas até anteriores, conseguiram dobrar o século XX e chegar aos dias de hoje ainda saudáveis, lucrativas e com potencial para o futuro? Tal situação só pode ser explicada pela fantástica capacidade de resiliência demonstrada na sua atividade ao longo de quase dois séculos. Fatores como a transição familiar intergeracional e a ligação da atividade retalhista a uma indústria não são determinantes, sendo mesmo os fatores com menor potencial de resiliência encontrados, facto que, em certa medida, face à perceção pacífica existente, constitui uma surpresa deste estudo. Em contrapartida, os fatores internos da atividade comercial como, a localização, a natureza ultra especializada e a qualidade do serviço e do atendimento constituem-se como fortes fatores potencializadores da resiliência do retalho independente centenário de Lisboa.
We used incident reports to improve treatment planning in radiation oncology to advance patient safety. First, a cluster of related reports enabled discovery of communication issues for setup instructions stemming from a text box with a limited character size. A redesigned process used structured data and image formats to reduce the proportion of inaccurate setup instructions. An upward trend in inaccurate setup instructions identified that some rotating staff were unaware of which equipment to use. Implementing default disease site setups ensured that when a patient was to commence treatment for a specific disease site, it was known which immobilizing equipment should be utilized as the basis before customization. Second, inappropriate plans were identified for proactive improvement. The process resilience was enhanced with targeted validations of generated treatment plans. The automated checks incorporated data automatically pulled from the electronic health record. Useful methods included clustering reports into categories to proactively see patterns in reported issues, collaborating with clinical experts from different perspectives, surveillance monitoring to identify new risks as conditions change, improving communications across disciplines through health information technology, and increasing system resilience with targeted validation of known gaps and variation in plans. The implementation of ILS frameworks, careful review of events and categorization, which culminate with interventions targeting the most frequent or highest severity incidents reported, increased patient safety within our department.
Full-text available
Foundations of a New Discipline.
Full-text available
Healthcare services are examples of organisa- tions that operate frequently at capacity, as reflected by periods of high demand and hospital overcrowding. Using the safe operating envelope framework (Qual Saf Health Care 14:130-134, 2005), this study identifies the strategies hospital staff use to respond to high patient demand pressures. A surgical unit (SU) in a dedicated trauma hospital provides the context for the study. Re- sults are based on the outcomes of structured, critical decision method and contextual interviews involving six participants selected according to their roles in relation to resource allocation within the SU. The study's central findings are (1) that temporally nested patterns of emergency patient admissions are the dominant influ- ence on three levels of management decision making in the SU and (2) that compensatory buffers are actively planned at multiple levels of work organisation. These results are discussed in terms of their theoretical impli- cations and implications for technological design. The methodological limitations of the research are also dis- cussed.
High Reliability Organizations (HROs) have been treated as exotic outliers in mainstream organizational theory because of their unique potentials for catastrophic consequences and interactively complex technology. We argue that HROs are more central to the mainstream because they provide a unique window into organizational effectiveness under trying conditions. HROs enact a distinctive though not unique set of cognitive processes directed at proxies for failure, tendencies to simplify, sensitivity to operations, capabilities for resilience, and temptations to overstructure the system. Taken together these processes induce a state of collective mindfulness that creates a rich awareness of discriminatory detail and facilitates the discovery and correction of errors capable of escalation into catastrophe. Though distinctive, these processes are not unique since they are a dormant infrastructure for process improvement in all organizations. Analysis of HROs suggests that inertia is not indigenous to organizing, that routines are effective because of their variation, that learning may be a byproduct of mindfulness, and that garbage cans may be safer than hierarchies.
Two possible sources of vigilance decrement are inadequate training of operators and taxing information-processing demands. This study attempted to compensate for both of these sources. Although compensation for training was effective, processing demands may have been set too high, as detection sensitivity declined. This result suggests that an event rate much less than the current threshold of 24/min is necessary to stabilize sensitivity. Response latency increased progressively during the task and was greater for “unsure” as compared with “sure” decisions. Consistent with an adaptation-level hypothesis, reduced sensitivity was compensated by a general negative shift in response bias. This adaptive process may account for the absence in this study of the usual form of vigilance decrement: a positive shift in bias, which reduces both detection and false-alarm scores.
Human error is considered a contributing factor in 70% to 80% of all aviation accidents. Because errors can never be eliminated completely, a further reduction of the already low accident rate in this domain will require investments in better support for error management. In particular, a better understanding of the nature and effectiveness of error detection mechanisms is needed. With this goal in mind, NASA Aviation Safety Reporting System incident reports were analyzed in terms of the formal characteristics of underlying errors, the cognitive stage, and the performance level at which these errors occurred, and with respect to the processes that led to their detection and, thus, prevented these incidents from turning into accidents. The majority of incidents involved lapses (i.e., failures to perform a required action) or mistakes, such as errors in intention formation and strategy choice. These errors were most often detected based on routine checks and the observed outcome of an action, respectively. Most slips appear to have been discovered by the crew before they could lead to a problem worth reporting. Our findings suggest a need for more effective feedback in support of data-driven monitoring, especially in the case of errors of omission and for shared knowledge of intent between airborne and ground-based operators to promote the more timely and reliable detection of mistakes.