Content uploaded by N. Dahlstrom
Author content
All content in this area was uploaded by N. Dahlstrom
Content may be subject to copyright.
RULE- AND ROLE-RETREAT: AN EMPIRICAL
STUDY OF PROCEDURES AND RESILIENCE
J. Bergström1*, N. Dahlström2,R. Van Winsen3, M. Lützhöft4,
S. Dekker5and J.Nyce6
Received 13 March 2009; received in revised form 17 March 2009; accepted 30 March 2009
ABSTRACT:
To manage complex and dynamic socio-technical systems places demands on teams
to deal with a range of more and less foreseeable situations.Three groups of partici-
pants with different maritime experiences were studied using the same simulation
of a ship to better understand the role of generic competencies (e.g. information
management, communication and coordination, decision making, and effect con-
trol) play in such high-demand situations. Groups with moderate maritime experi-
ence were able to balance contextual knowledge with use of generic competencies to
successfully manage unexpected and escalating situations.Novices, lacking contex-
tual knowledge, performed less well. Groups with the most maritime expertise
remained committed to presumed procedures and roles and did not perform as well
as the other two groups.The results suggest that training to operate complex socio-
technical systems safely and effectively should go beyond procedures and include
development of generic competencies. This could provide operators with better
tools to enhance organizational resilience in unexpected and escalating situations.
Keywords: resilience,procedures,emergency management, training,simulation.
JOURNAL OF MARITIME RESEARCH 75
Journal of Maritime Research, Vol. VI. No. 1, pp.75-90,2009
Copyright © 2009. SEECMAR
Printed in Santander (Spain). All rights reserved
ISSN: 1697-4840
1PhD Student, Lund University, johan.bergstrom@brand.lth.se, +46462884506, PO Box 118, 221 00 Lund,
Sweden. 2Researcher PhD, Lund University School of Aviation,nicklas.dahlstrom@tfhs.lu.se,+46435445400,Drot-
tningvägen 5, 260 70 Ljungbyhed,Sweden. 3Human Factors researcher, Lund University School of Aviation, rdvan-
winsen@hotmail.com, +31647049272, De Weiden 59, 2361VX, Warmond, The Netherlands. 4Asst Professor,
Chalmers University of Technology, margareta.lutzhoft@chalmers.se, +46317721464, no street 412 96 Göteborg,
Sweden. 5Asst Professor,Department of Anthropology Ball State University, jnyce@bsu.edu, 7657898793, 2000 W
University Ave 47306, Muncie Indiana United States. 6Professor, Lund University School of Aviation,
idney.dekker@tfhs.lu.se,+46435445434, Drottningvägen 5, 260 70 Ljungbyhed,Sweden.
* *Corresponding author: johan.bergstrom@brand.lth.se, +46462884506.
6 abril 2009.qxp 15/04/2009 21:02 PÆgina 75
INTRODUCTION
Procedures & resilience
Teams managing complex and dynamic systems have to be able to deal with a range
of more and less foreseeable situations. In unexpected and escalating situations
demands on such abilities can increase rapidly, for example when an operational
problem of unknown origin becomes more serious or more intense. A fire aboard a
ship could be a relevant example, such as the blaze on the Scandinavian Star, a pas-
senger ferry on the route between Oslo and Copenhagen, on 7 April 1990, in which
158 people died when fire broke out and could not be contained until the next day
(NOU, 1991). Unexpected and escalating situations create a range of cognitive and
coordinative demands. As the tempo of operations rises and becomes more (if not
entirely) event-driven, there is more to consider, communicate and coordinate, i.e.
more information to process,distribute and act upon (e.g.Woods, Patterson & Roth,
2002). Goals can multiply, diversify and compete more steeply, and risk can increase
dramatically.
A common regime in many operational worlds for absorbing those increased
demands is proceduralization - the matching of situational symptoms with prepared
scripts of coordinated action. Proceduralization is meant not only to help teams
accomplish the sorts of actions necessary to further diagnose the situation, reduce
uncertainty in it, and deal with its effects. It also supports the prioritization of cer-
tain work in the face of time pressure and resource constraints.Proceduralization can
also be indivisibly connected with role assignments that govern who does what in
dealing with the problem,and who double-checks or follows up the accomplishment
or effect of specified actions. Proceduralization has provided operators in complex
socio-technical systems with solutions on how to resolve normal and emergency sit-
uations and thus increased the reliability of operational activities. In the aviation
industry in particular, proceduralization has been regarded as the most important
system component to achieve increased operational safety and this has inspired mar-
itime, nuclear and chemical industry as well as recently medicine.Shipping has rela-
tively unthinkingly adopted this practice without reflection on the consequences,
often perceived by seafarers as “counteracting the use of common sense, experience,
and professional knowledge epitomized in the concept of seamanship” (Knudsen,
2009). Over time proceduralization has become more than an answer of how to
increase safety in modern socio-technical systems, it may have become the answer.
Scientific management gave the original impetus to the development of proce-
dures as ways of specifying action (see Taylor, 1947).It assumed that order and sta-
bility in operational systems can be achieved rationally, mechanistically, and that
control is implemented vertically.These strategies persist. For instance, shortly after
a fatal shootdown of two US Black Hawk helicopters over Northern Iraq by US
RULE- AND ROLE-RETREAT: AN EMPIRICAL STUDY OF PROCEDURES AND RESILIENCE
Volume VI. Number 1. year 200976
6 abril 2009.qxp 15/04/2009 21:02 PÆgina 76
fighter jets ‘‘higher headquarters in Europe dispatched a sweeping set of rules in
documents several inches thick to ‘absolutely guarantee’ that whatever caused this
tragedy would never happen again”(Snook,2000, p.201). In the maritime regulatory
domain, one sees the direct influence of singular accidents in new rules, often to the
point where a ship can be identified by name.In many operational worlds, the strong
influence of information-processing models on human factors has reinforced the
idea of procedures as IF-THEN rule following, where procedural prompts serve as
input signals to the human information processor (Wright & McCarthy,2003).Pro-
cedures, however, are inevitably incomplete specifications of action. They contain
abstract descriptions of objects and actions that relate only loosely to particular
objects and actions that are encountered in the actual situation, and a procedure
often requires a whole set of cognitive and coordinative tasks to be executed that the
procedure itself cannot specify or call for (Suchman, 1987).Therefore it will remain
difficult to construct global and prescriptive rules intended to guarantee certain out-
comes of human behavior in complex socio-technical systems. Consequently, an
over-reliance on procedures for safe operation may add additional layers of complex-
ity rather than guide action and become part of the problem rather than the solution
on to how to resolve a situation. One example is the fatal accident of Swissair 111
that crashed into the Atlantic ocean after that the presence of smoke in the cockpit
was responded to promptly by the crew by adhering to the relevant checklist (Trans-
portation Safety Board of Canada, 2003).However,while following established pro-
cedures for this situation (aiming at finding the source of the smoke rather than
extinguishing any fire or putting the aircraft on safe ground), the fire engulfed the
aircraft. In this case following the procedures turned out to be the problem rather
than the solution (see Burian & Barshi, 2003).
The literature has recognized the limits and costs of a procedural approach to
management of high-demand situations (e.g. Vicente, 1999, ; Snook, 2000,; Burian
& Barshi, 2003,;Dismukes,Berman & Loukopoulos, 2007),but it has remained rel-
atively mute on the mechanisms that translate that cost into real performance losses.
Whereas the literature has shed light on the difficulty of processes of sensemaking in
demanding situations that lie beyond procedural reach (Weick, 1993) it has not
developed a detailed understanding of those aspects of performance that enhance or
debilitate a team’s adaptive capacity; its ability to migrate into a different regime for
handling high demands.These are situations where novel behaviors can emerge,new
and different resources are brought to bear, a regime where the stretched capacities
of its constituent members actually open up opportunities for novel initiatives, inter-
actions or role adoptions.
The ability to adapt is one of the key aspects of making a team work as a resilient
organization, where resilience is defined as the ability to recognize, absorb and adapt
to disruptions that fall outside a system’s design base (Woods, 2006). The design
base incorporates soft and hard aspects that went into putting the system together
J. Bergström, N. Dahlström, R. Van Winsen, M. Lützhöft, S. Dekker and J. Nyce
JOURNAL OF MARITIME RESEARCH 77
6 abril 2009.qxp 15/04/2009 21:02 PÆgina 77
(e.g., equipment, people, training, and indeed procedures).The question is whether
we can expect this kind of resilience from operators, experts as they may be, who are
trained to follow the rules, adhere to checklists, and who have perhaps been profes-
sionally indoctrinated to believe that there is a procedure for every occasion.
In this paper we report on a series of empirical studies in which we explored the
basis for the kind of adaptive capacity that could enhance organizational resilience in
high-demand situations. Previous micro-world experiments (see Dörner,1996) have
illuminated the importance of generic competencies in handling dynamically esca-
lating situations. Based on previous research on team decision making and human
behavior, when handling unexpected and escalating events, as well as case studies
from various domains, Bergström,Dahlström and Petersen (2008) suggested a theo-
retical framework to describe such generic competencies. The framework contains
the four categories information management, communication and coordination,
decision making,and finally effect control.
Here we ask the question whether a gap in such generic competencies can lead
to over-proceduralization, over- reliance in role behavior and real operational losses.
Without generic competencies to lean on, an increase in demands could actually
accelerate rule- and role-retreat: a fall-back into, and an increasing rigidity of,
rehearsed roles and rules (procedures). Possession, articulation and rehearsal of
generic competencies, in contrast, could enhance a team’s resilience in the face of
accumulating demands.
The M/S Antwerpen simulation
The M/S Antwerpen simulation is part of a two-day emergency management train-
ing course developed at the University of Bamberg in Germany (Strohschneider &
Gerdes, 2004). The course consists of first one simulator session, followed by
instructions, extensive debriefings and theoretical training on emergency manage-
ment, and concludes the following day with a second simulation session. However,
apart from its training purposes, the simulation can be used as a research tool for
providing data on group action and interaction in escalating events.
The simulation is designed for a group of five to seven participants who act as
the ship’s officers. Each participant takes on a specified role, namely: captain, chief
officer, chief and main engineer, chief steward, ship’s doctor and navigation officer.
Initially each participant is provided with general information which describes the
features of the ship as well role specific information. The information emphasizes
the conditions for the simulation with extreme clarity, e.g. stresses not to make
assumptions about the ship or its status but rather use the information that is avail-
able. It is the participants’ task to safely navigate the ship through a stormy night in
the North Atlantic. Due to the adverse conditions, and because the ship has been
poorly maintained, the crew is forced to deal with a number of passenger-related
RULE- AND ROLE-RETREAT: AN EMPIRICAL STUDY OF PROCEDURES AND RESILIENCE
Volume VI. Number 1. year 200978
6 abril 2009.qxp 15/04/2009 21:02 PÆgina 78
problems and technical failures that towards the end of the simulation result in a
state of emergency.
To sail the ship and handle these events, the participants have a wide variety of
options and actions available to them.They have control over the technical facilities
of the ship,including maintenance and repairs. Furthermore,they are presented with
an abundance of information regarding the ship provide by regular printer outputs.
By filling out order sheets they direct the crew and give various orders relating to the
passengers (including, e.g. sending misbehaving passengers to their cabins,closing or
evacuating sections of the ship, and using life boats and rafts to abandon the ship).
Participants are not provided with a prescribed list of possible responses. Instead
they have to plan and execute actions as a team and deal with the possible conse-
quences and side-effects of their actions.The participants, therefore, find themselves
in a dynamically developing situation, one that has a high level of uncertainty. More-
over,they have to deal with all this under the threat of all the conceivable emergen-
cies that come with navigating a poorly maintained ship in bad weather.
The effect of different levels of procedural-experience
In this study, three different types of groups participated in the M/S Antwerpen
training program. Each type of group differed in their familiarity with maritime
operations. We were interested in the effects of the different levels of knowledge,
experience and availability of procedures on concepts regarding team interaction and
management of unexpected and escalating situations. Each type of group was repre-
sented by three groups of participants which were observed during the M/S Antwer-
pen training program.
The first type of participants was made up of novices in maritime settings,
namely civil aviation student pilots. This type of group had minimal knowledge of
maritime concepts, operations, and procedures. The second type of group consisted
of maritime students with limited experience of maritime operations. These partici-
pants had some professional experience, however, being students, they did not have
years of practical firsthand experience with the procedures and practices on large
ships. The third group consisted of experienced seafarers, who had multiple years of
experience on large ships.This type of group possessed practical expertise in regards
to maritime concepts as well as with normal and emergency procedures onboard
such ships.
Our main point of interest was whether the availability of procedures would
enhance or impair the groups’use of generic competencies and therefore their ability
to be resilient. When observing the three groups in order to assess their generic
competencies, we categorised statements made and strategies chosen into the four
categories information management, communication and coordination, decision
making,and effect control (see Bergström, Dahlström & Petersen,2008).
J. Bergström, N. Dahlström, R. Van Winsen, M. Lützhöft, S. Dekker and J. Nyce
JOURNAL OF MARITIME RESEARCH 79
6 abril 2009.qxp 15/04/2009 21:02 PÆgina 79
Regarding the teams’ information management-strategies we specifically
recorded whether or not the participants: openly stated their personal and group’s
goals, whether or not these goals were discussed, and whether or not decisions were
based upon them. The simulation ensures that the groups are required to find a way
to manage the continuous printer outputs.Therefore strategies for receiving and dis-
tributing incoming information to the group or its individual members were exam-
ined. We were also interested in how the groups would establish strategies,like pri-
oritize incoming information based on the team goals, to cope with the data over-
load problem.
Secondly we focused our attention on the category communication and coordi-
nation; mainly how team members dealt with workload and role definition.We were
interested to see to what extent participants shared their tasks and workload (flexible
structure), or whether they rigidly remained within the work constraints of their pre-
arranged roles (robust structure).How the groups dealt with the idle time during the
simulation was a further area of interest. Idle time refers to the periods during the
simulation in which there is hardly any work to do for the group. Idle time, inten-
tionally produced by the simulation, provided the groups with the opportunity to
think ahead, reflecting upon team processes, and construct team strategies accord-
ingly, instead of just waiting for the next crisis to come along.
Thirdly we observed and recorded the decision making process, specifically at
where the decision making process took place (distributed, hierarchical, etc.). We
were interested in observing whether or not group goals were used to build up shared
mental models and shared expectations of the problems at hand, and also in how
team members shared information about decisions made.
Finally we examined the groups’overall capabilities to step back and on a meta-
level analyze, discuss, and adjust their approach and team functioning during the
simulation.
It seems that the different levels of (procedural) experience described above give
rise to different ways of managing emergencies, in particular unexpected and escalat-
ing situations. In analyzing the performance of the groups, we distinguished
between the two perspectives of “process” and “outcome”. Outcome relates to the
quantitative results of the simulation; primarily numbers of injuries and casualties,
and damage to the ship. As a performance measure, however, the outcome of the
simulation is dependent on the interaction between the participants and the facilita-
tors of the simulation and therefore renders it a less reliable measure of performance.
“Process” refers to qualitative aspects of how the group managed situations encoun-
tered in the simulator sessions, e.g. the generic competencies outlined above,in rela-
tion to accepted and recommended practices for emergency management. Reporting
of results are mainly focused on “process”.
RULE- AND ROLE-RETREAT: AN EMPIRICAL STUDY OF PROCEDURES AND RESILIENCE
Volume VI. Number 1. year 200980
6 abril 2009.qxp 15/04/2009 21:02 PÆgina 80
RESULTS: DESCRIP TIONS OF THE THREE T YPES OF GROUPS’
PERFORMANCES DURING THE M/S ANTWERPEN SIMULATION
Group type 1: Civil aviation student pilots
The first type of group, in general, performed poorly in the first simulator session in
regards to both process and outcome. Most of these groups lost the ship and barely
saved any passengers on their first trips. This began directly with the group getting
immersed in the simulation, literally just starting and going wherever events would
take them, without any clearly communicated goal or strategy,thus becoming locked
in the dynamics of events almost immediately. This was manifested in the group’s
performance; they soon became overwhelmed by the amount of information they
received. At no time did they discuss whether their initial approach of handling the
information (printouts that soon formed piles and stacks of paper) was successful or
even functional. Actions were almost exclusively reactive, i.e. in response to emerging
problems, and normally based on urgency, not priority. Tasks were not shared in the
group outside their original role descriptions, which led to a very high workload for
some participants while others had few tasks to perform. The group did not monitor,
discuss or change the group processes in any way, even when the workload was rec-
ognized within the group as becoming impossible to manage.
In the second session this type of groups did better regarding the outcome of the
simulation: all groups still lost the ship,however this time they managed to save the
majority of the passengers and crew.On a process level,the improvements were even
more evident. The group showed great improvements in competencies. Although
this improvement is somehow expected due to the participants’ experience from the
first trip and the subsequent theoretical training on emergency management, it was
surprising how these groups were able to successfully transfer this to improved
behavior in the second session.Tasks were explicitly distributed and redistributed to
manage workload within the group.A role with low workload took the role of mod-
erator, who then was in charge of monitoring the group’s processes and dynamics.
This meant that the captain was able to maintain an overview of the current situa-
tion and focus on issues of priority (rather than of urgency). Measures to ensure
effective information sharing were taken, such as attempts at establishing routines
for regular briefings and other forms of presenting the current situation of the ship,
personnel, and problems to establish a shared mental model of the situation (use of
blueprints, whiteboard, log notes etc.). They also followed up on more orders to
ensure that they were carried out as intended (i.e. “effect control”) and were much
more cautious when making assumptions about a situation when there were gaps in
their knowledge.The groups took more precautions in regards to various threats and
were more proactive during this trip,for example by checking that rescue equipment
was operative. The differences between the two sessions, in terms of team processes
and generic competencies, are outlined in table 1.
J. Bergström, N. Dahlström, R. Van Winsen, M. Lützhöft, S. Dekker and J. Nyce
JOURNAL OF MARITIME RESEARCH 81
6 abril 2009.qxp 15/04/2009 21:02 PÆgina 81
Table 1.Summary of the established processes to manage the simulation in the groups of type one.
Overall these groups, being student pilots, were hampered by a lack of under-
standing of maritime concepts which made it challenging for them to extrapolate
from available knowledge in order to invent creative solutions for solving the prob-
lems they faced. Nevertheless, clever alternatives were devised for a number of prob-
lems, e.g. as the general alarm was defective an effective notification was devised for
passengers to tell them that they should go to the life boats.
Group type 2: Maritime students
During the first trip, the second type of group performed, in regards to final out-
come, moderately better then the first group, however,most groups still lost the ship
and the majority of the passengers and the crew.These groups devised creative solu-
tions which were difficult for the first type of group to come up with, since a basic
understanding of ships is required to be able to devise such solutions. An example of
this was when a group of this second type used a stream anchor to maneuver the ship
after a breakdown of the steering engine. However, despite their creative problem
solving capabilities, in comparison with the first type of group this type of group
only performed marginally better in regards to aspects of process. Similarly to the
previous groups, almost all of the second groups’ actions were reactive; discussions
and action were prompted exclusively by the information that came out of the print-
er.There was hardly any analysis of the occurred events, revision of current situation
RULE- AND ROLE-RETREAT: AN EMPIRICAL STUDY OF PROCEDURES AND RESILIENCE
Volume VI. Number 1. year 200982
Session one Session two
Information
handling
No explicit goals
No prioritizing of incoming
information
Information overload
Clearer formulation of goals
Captain able to prioritize tasks
Communication
and
coordination
No assignment of tasks outside
role descriptions
Robust rather than flexible
environment
Distribution and redistribution of some
tasks
Not overruling the predefined roles
Moderator responsible for monitoring
group processes and dynamics
Decision-making No sharing of goals
No sharing of expectations
Little sharing about decisions
made
Regular briefings to share information
about the latest decisions made
Using blueprints, whiteboards to share
information
Clearer orders
Effect control No following-up on decisions made
No reflections about, or updates of,
the tasks assigned to each role
Redistribution of tasks based on the
situation
Follow-ups on decisions made and
orders given
6 abril 2009.qxp 15/04/2009 21:02 PÆgina 82
and strategies, or discussions on which potential tasks and risks may be ahead of
them.There were also no structural solutions to deal with the information overflow.
After reflecting on the first trip and receiving the theoretical training, these
groups performed very well in the second trip; saving the ship and its passengers.
They were still applying their creative problem solving skills, but proactive thinking
was added to that. They questioned their initial approaches to problems, and what
became apparent during their second trips was their anticipation of potential prob-
lems. Examples of improved effect control were evident in the groups’ continuous
verification of the results of all their actions. There were numerous examples of
proactive actions and planning in regards to potential threats to the safety of the
ship.During a large fire all these groups sent people to higher decks, anticipating the
spread of the fire.Their ability to foresee contingencies and consequences lead them
to react quickly and forcefully.They prioritized effectively and responded in this way
to every event that threatened safety.They were able to rank actions,given the situa-
tion, and defer the least important of them until a normal status of operation was
regained.They were also reluctant to make any sort of assumptions which could not
be sustained by facts.They were also able to dynamically adjust their strategies,given
the conditions encountered. An example of this was their discovery, prior to any
emergency situation, that the instructions did not include specific references to
muster stations and the resulting creation of an alternative approach to evacuation.
The differences between the two sessions, in terms of team processes and generic
competencies, are outlined in table 2.
J. Bergström, N. Dahlström, R. Van Winsen, M. Lützhöft, S. Dekker and J. Nyce
JOURNAL OF MARITIME RESEARCH 83
Session one Session two
Information
handling
Everyone’s responsibility to handle
incoming information
No explicit goal formulations
Clear goals
Prioritizing and ranking potential threats
Communication
and
coordination
No assignment of tasks outside
role descriptions
Robust rather than flexible
environment
Resisting confirmation bias
Clear roles, but flexible distribution of
tasks
Decision-making Creative solutions to maritime-
related problems
No discussions about potential
tasks and risks
Creative solutions to maritime-related
problems
Formulation of potential problems guided
a distributed decision making processes
Quick reactions to contingencies
Effect control Hardly any analysis of occurred
events
No revision of current situations
or strategies
Continuous verification of the results of
actions
Dynamically adjusting strategies
Table 2. Summary of the established processes to manage the simulation in the groups of type two.
6 abril 2009.qxp 15/04/2009 21:02 PÆgina 83
These groups performed better than the first type of groups mainly because they
had contextual knowledge that supported a more effective use of generic competen-
cies. They devised creative solutions based on extrapolations of their nautical knowl-
edge,e.g.reducing roll angle by course change and using auxiliary engines proactively.
Group type 3: Experienced seafarers
This type of group consisted of participants with first hand practical experience in
direct relation to their roles in the simulation (the captain, chief officer and chief
engineer all had experience in these respective professional roles). In the first trip a
steep hierarchy of command was quickly established among the participants. This
hierarchy seemed comparable to that of real vessels,with the captain being clearly in
charge of everything. This hierarchical role division and task distribution worked
effectively during this session as it provided structure to both normal tasks and
unclear situations. All group members reported to the captain,which resulted in the
captain being in control of all goals and problems, and he gave orders accordingly.
He then relied on his crew to execute the given orders. The captain and the rest of
the group relied heavily on many types of procedures that they would also apply in
real-life situations, and these proved useful on most occasions. These procedures,
drawn from practical experience, led to the participants checking a number of
parameters. This also proved useful as it supported a structured plan for manage-
ment of emergencies. However,the group did not fully integrate and act on informa-
tion given to them even though this issue was heavily emphasized in the instructions
they had received. This was manifested on many occasions, including a fire where a
captain ordered the crew to go to their muster stations and only then it was discov-
ered that these were not specified in the simulation.The participants were however
unable to solve this problem by inventing an alternative to the concept of muster sta-
tions; instead there were repeated questions about them in the group as well as com-
plaints to the facilitators about the lack of them. Overall the procedures, often
unspoken and simply assumed, did not always work as anticipated, because the sce-
nario of the simulation did not match entirely with their expectations based on their
previous experiences. As a result the group was not able to advance much beyond
their silent consensus on how things “should be”. There were also expectations on
the behavior of the crew in the simulation which did not prove valid.The amount of
expectations and inability to break out of them resulted in unanticipated failures and
losses. The hierarchical team structure worked smoothly initially. It did however
start to erode as soon as the workload increased; the captain became buried in infor-
mation and tasks. For example, during a large fire the captain was in charge of both
the fire fighting and evacuation of passengers. In practice, this meant that only one
of these two tasks could be adequately managed. The final outcome was moderately
successful because, based on his experience, the captain decided at an early stage of
RULE- AND ROLE-RETREAT: AN EMPIRICAL STUDY OF PROCEDURES AND RESILIENCE
Volume VI. Number 1. year 200984
6 abril 2009.qxp 15/04/2009 21:02 PÆgina 84
the emergency that they were not going to be able to fight the fire effectively and
therefore he decided to evacuate.
Surprisingly, after the relatively good first trip (keeping in mind the flaws of
relying solely on procedures mentioned above) and subsequent training, this group
showed only marginal improvement in the second session after receiving the train-
ing that the other types of groups received. The group preserved their reliance on
roles and procedures that proved less than ideal during the first trip.In many aspects
the mistakes during the second session were identical to those committed during the
first session. The serial approach to problems used in the first session, which was
ineffective at times when the captain was preoccupied, again in the second session
lead to standstills until the captain was free to approach the next problem. With the
exception of maintenance of machinery,practically no action was executed in paral-
lel. An example of this was seen as the group was attempting to investigate potential
water penetration at a time when a bomb threat was received,resulting in a switch of
full focus to the bomb threat and a return to the potential water penetration only
when the bomb threat had been investigated. By not approaching these two prob-
lems in parallel, for which there were more than enough resources available, the
lower decks could have been flooded before both problems had been addressed.The
group members stayed precisely within the boundaries of their roles and task
descriptions (relying on other “roles” to do “their own” tasks). In one case it took a
“hero”,one of the most experienced group members, a chief engineer, to break out of
J. Bergström, N. Dahlström, R. Van Winsen, M. Lützhöft, S. Dekker and J. Nyce
JOURNAL OF MARITIME RESEARCH 85
Session one Session two
Information
handling
All group members reporting to
the captain
The captain in control of possible
goals
All group members reporting to the
captain
The captain in control of possible goals
Communication
and
coordination
Steep hierarchy
Robust rather than flexible
environment
Steep hierarchy
Robust rather than flexible environment
Decision-making Decisions made by the captain
Reliance on perceived procedures
Not acting on the information given
No information sharing except for
reporting to the captain
Tasks not performed in parallel
Decisions made by the captain
The Captain overloaded with tasks
Reliance on perceived procedures
No information sharing except for
reporting to the captain
Tasks not performed in parallel
Effect control Unable to revise the belief in per-
ceived roles and procedures
No revision of roles and tasks
Few follow-ups on orders given
Unable to revise the belief in perceived
roles and procedures
Few revisions of roles and tasks
Few follow-ups on orders given
Table 3. Summary of the established processes to manage the simulation in the groups of type three.
6 abril 2009.qxp 15/04/2009 21:02 PÆgina 85
this structure.During an emergency he stepped out of the hierarchy to suggest to the
captain that he should take charge of the evacuation as long as the captain was pre-
occupied with fire fighting. Due to this initiative, the evacuation was taken care of
effectively and again the outcome was relatively successful (although not nearly as
successful regarding process as session two for the type 2 groups). The differences
between the two sessions, in terms of team processes and generic competencies, are
outlined in table 3.
DDIISSCCUUSSSSIIOONN
Group type 1 & 2 vs. type 3: The limiting effect of procedural experience
Overall, the third type of group did relatively well in terms of group process and out-
come. Despite that they did not perform optimally under increasing stress they did
have a clear solution to the information flow problem and a clear distribution of tasks,
as well as a systematic approach to most of the situations encountered. However, as
soon as events developed beyond the “design base”of their procedures, the work struc-
ture started to disintegrate rapidly.This did not occur in the other two types of groups
– ones that were not as reliant on presupposed procedures.These two groups used the
generic competencies and the knowledge they gained from their experiences in the
simulation as well as from the training in-between the two scenarios. Furthermore, in
contrast to the other two types of groups, the third type of group very rigidly held
onto their original hierarchical group structure, even when this proved ineffective.
The other groups were able to manage a more dynamic group structure in order to
deal with unexpected situations and the different phases of escalation.
Although procedures may prove effective to some degree, as shown by the rela-
tive success of the third group during their first trip, as soon as the specific condi-
tions for a procedure did not precisely apply, control was lost and competence
degraded. As not all scenarios or situations can be foreseen, relying on procedural
knowledge or procedures may not always be enough. However, trained crews tend to
react to emergency events with the use of pre-prescribed procedures even when these
do not match the problem. In other words, for someone equipped with a hammer all
problems will look like nails, and this is where the danger lies. In particular in emer-
gency situations individuals and groups need to be able to review their situation and
assess whether their “standard” approach is appropriate (exemplified by the fatal
accident of Swissair 111, outlined above).
Relying on procedural knowledge can limit alternatives and may prevent poten-
tially powerful non-presupposed solutions from being considered. This may mean
that not even reviewing or reframing of a situation may occur. In short, relying on
procedural knowledge can severely limit crews’options to be resilient.
Moreover, armed with the knowledge that they have a procedure for every sce-
nario,the experienced participants (group type 3) felt more secure than the inexperi-
RULE- AND ROLE-RETREAT: AN EMPIRICAL STUDY OF PROCEDURES AND RESILIENCE
Volume VI. Number 1. year 200986
6 abril 2009.qxp 15/04/2009 21:02 PÆgina 86
enced groups (group type 1 and 2).This sense of security seemed to cause overconfi-
dence. In the simulation this was manifested by the following: giving non-specific
orders, not following up on orders and making unfounded assumptions in many sit-
uations. This may be a functional approach in extremely stable and reliable condi-
tions. However, this was not the case in the simulation and is rarely the case in the
transportation industry, where crew composition and operational conditions may
change frequently and equipment often is used at or close to the limits prescribed by
the manufacturer.
Despite long periods of operation during which nothing dangerous ever happens,
it is important that operators go to work and are prepared to expect the unexpected
(Dekker,2006).In the simulation sessions, in particular during the second trip,group
types 1 and 2 showed caution regarding their situation; institutionalizing effect con-
trol, proactively checking the status of equipment (e.g. technical and rescue equip-
ment) and simply contemplating what might go wrong next. Multiple groups of type
1, for example, checked whether all the life-boats were in good condition,whereas the
experienced seafarers assumed that they were, based on the claim that such matters
are regularly tested (even though they admitted that there has been a number of acci-
dents at sea where rescue equipment has been defective).This summarizes exactly the
issues that the experienced crews faced (group type 3); they assumed that things were
as they should have been,that situations would proceed as expected or as ordered,and
all of this would occur in accordance with procedures. Experience makes people
expect certain things regarding quality of equipment, action sequences performed by
crews, crew reactions in emergencies, passenger reactions and behavior etc. Proce-
dures create reliability,i.e. expected events.Procedures are in place to fight foreseeable
problems, whereas caution, forethought and inspection can give rise to resilience, to
the ability to adapt to unexpected and escalating events (see McDonald, 2006).
A point of criticism may be that the experienced crew did not do as well as the
other type of groups during the simulation because the simulation did not exactly
mirror real world conditions.This is however to some degree the main point that the
participants of the simulation were meant to take home; in an emergency situation
not everything will go according to plan, e.g.not everybody will report back and not
all orders will be carried out (Dahlström, van Winsen, Dekker, & Nyce, 2008). But
under those circumstances groups need to find and mobilize resources and compe-
tences that will enable them to remain functioning. When procedures limit options
they need to be able to find alternative ways to solve problems.
Group 1 vs. group 2: Difference between generic competencies,
domain knowledge and procedural knowledge
If relying entirely on procedures is not appropriate, we can argue that having only
generic competencies without having any domain specific knowledge (group type 1)
J. Bergström, N. Dahlström, R. Van Winsen, M. Lützhöft, S. Dekker and J. Nyce
JOURNAL OF MARITIME RESEARCH 87
6 abril 2009.qxp 15/04/2009 21:02 PÆgina 87
may have the same effect. Maritime knowledge should facilitate the generation of
solutions to the problems encountered in the simulation.The student pilots (group
type 1) were not likely to come up with alternative solutions for navigation at sea.
The second group type did have knowledge of maritime concepts in addition to the
general competencies learned during the training phase of the program; even when
the steering engine broke down they explored actions in order to make the bow of
the ship turn into the wind, thereby reducing the dangerous roll angle of the ship.
However they were not limited to following established procedures,even when there
were explicit procedures available. To argue that even in a simulation appropriate
contextual and domain competence “counts” seems self-evident. But this is not the
point we wish to make here. What we believe this simulations suggests is that the
role generic competencies can play for safety needs to be reconsidered for unexpect-
ed and escalating situations.
On a process level, it proved easier for groups with a certain level of domain
knowledge (group type 2), as compared to groups with only generic competencies
(group type 1), to cooperate.The reason for this lies in the more appropriate match
between their common knowledge and the situation. The type 1 groups improved
their generic competencies between the two trips and therefore performed more
effectively during the second trip. Despite this improvement, it proved harder for
these groups to apply their skills in an unfamiliar setting. Moreover, for participants
without previous domain-specific knowledge, it proved impossible to step out of
their roles, in order to assist or even overrule other roles. The reason for this lays in
the role-specific information being the only information on which to base their
behavior. With no alternative knowledge to use in these unfamiliar settings they
remain “too loyal”to these roles.
There is a fine line that distinguishes the second group’s (theoretical) domain
knowledge, which enabled them to come up with more creative solutions, from the
first group who lacked maritime concepts, and the third group’s procedural experi-
ence which seemed to limit them to following procedures.The student pilots had no
tools with which to deal with a maritime emergency, they had to rely solely on
generic knowledge and competencies. The experienced crews relied on tools and
procedures they normally used. The maritime students on the other hand were able
to look into the toolbox and select an appropriate (but not necessarily prescribed)
tool for the situation.
CONCLUSIONS
If operators are to be able to resolve normal and emergency situations, they do need
to be trained in established procedures. Many industries, including the maritime,
have tended to equate training and education with the acquisition of procedural
knowledge. This has led to a regime of control, a discourse, in which every conceiv-
RULE- AND ROLE-RETREAT: AN EMPIRICAL STUDY OF PROCEDURES AND RESILIENCE
Volume VI. Number 1. year 200988
6 abril 2009.qxp 15/04/2009 21:02 PÆgina 88
able variation away from normal operation can be “anticipated” or controlled addi-
tively by the creation of new or more procedures and rules—a trend that may be
larger than the safety-critical situations studied for this paper (see Foucault, 1977).
However, the intractability of human action and design compromises inherent in
any complex socio-technical system can lead to operational situations which require
action and response that extends beyond any set of established procedures no matter
how elaborate or detailed. Observations of emergencies in these ship simulations
suggest that procedural knowledge or guidance can lead operators in unexpected and
escalating situations to act in ways that are irrelevant or detrimental given the situa-
tion at hand. The issue here is not whether procedural knowledge be part of any
operator’s training. It is to suggest that operator training should not be limited to
this. Apart from being reliable in the sense of following predetermined procedures,
an organization also has to be able to “recognize and adapt to handle unanticipated
perturbations that call into question the model of competence, and demand a shift of
processes, strategies and coordination” (Woods, 2006, p.22). McDonald (2006) sug-
gests that being able to successfully resolve this apparent contradiction is a charac-
teristic of a resilient organization. While unanticipated, it is industry’s strong com-
mitment to and investment in safety and procedure that has left operators less able
to respond to unexpected and escalating situations. Next to the training of proce-
dures, operator training needs to stress as well the development of generic compe-
tencies that add up to resilience in the face of unexpected and escalating situations.
This would provide operators with the tools they need to manage situations that go
beyond what can be anticipated. Only then can the kinds of counterproductive rule-
and role-retreat behavior observed in these shipboard simulations (and often report-
ed in accident reports) be avoided.
REFERENCES
Bergström, J., Dahlström,N. and Petersen, K.(2008): Securing Organizational Resilience in
Escalating Situations: Development of skills for crisis and disaster management, Proceed-
ings of the third Resilience Engineering Symposium, 28-30 October, Antibes-Juans Les
Pins, France, pp.11-17 .
Burian, B. and Barshi, I. (2003): Emergency and Abnormal Situations: A Review of ASRS
Reports. the 12th International Symposium on Aviation Psychology, Dayton, OH,
Wright State University Press.
Dekker,S. (2006): The field guide to understanding human error.Aldershot: Ashgate Publishing
Company.
J. Bergström, N. Dahlström, R. Van Winsen, M. Lützhöft, S. Dekker and J. Nyce
JOURNAL OF MARITIME RESEARCH 89
6 abril 2009.qxp 15/04/2009 21:02 PÆgina 89
Dismukes, R. K., Berman, B. A. and Loukopoulos, L. D. (2007): The limits of expertise:
Rethinking pilot error and the causes of airline accidents. Aldershot: Ashgate Publishing
Company.
Dörner, D. (1996): The Logic of Faliure. New York: Metropolitan Books.
Foucault, M. (1977): Discipline and punish: The birth of the prison (A. Sheridan, Trans.).New
York: Vintage (Original work published 1975).
Knudsen, F. (2009): Paperwork at the service of safety? Workers’ reluctance against written
procedures exemplified by the concept of ‘seamanship’.Safety Science,47, 295-303.
McDonald, N. (2006): Organisational Resilience and Industrial Risk. In E. Hollnagel, D.
Woods and N. Leveson (Ed) Resilience Engineering, Concepts and Precepts, pp. 155-180.
Aldershot, Ashgate Publishing Company.
NOU (1991): 1A Scandinavian Star: Ulykken 7 april 1990 Hovedrapport, Oslo
Snook, S. A. (2000): Friendly Fire, The accidental shootdown of U.S. Black Hawks over northern
Iraq.Princeton: Princeton University Press.
Strohschneider, S. and Gerdes, J. (2004): MS: ANTWERPEN: Emergency management
training for low-risk environments. SIMULATION & GAMING 35 (3), 394-413.
Suchman, L. A. (1987): Plans and situated actions: The problem of human-machine communica-
tion. Cambridge,UK:Cambridge University Press.
Taylor, F. W. (1947): Scientific Management.New York: Harper & Brothers Publishers.
Transportation Safety Board of Canada (2003): Aviation Investigation Report: In-flight fire
leading to collision with water, Swissair Transport Limited, McDonnell Douglas MD-11
HB-IWF, Peggy’s Cove, Nova Scotia 5 nm SW, 2 September 1998, (Report number
A98H0003), Catinau, QC
Weick, K. (1993): The Collapse of Sensemaking in Organizations: The Mann Gulch Disas-
ter. Administrative Science Quarterly,38(4), 628-652.
Vicente, K. (1999): Cognitive work analysis: Toward safe, productive, and healthy computer-based
work.Mahwah,NJ: Lawrence Erlbaum Associates.
Woods, D. D. (2006): Essential Characteristics of Resilience. In E. Hollnagel, D. D. Woods
and N. Leveson (Ed) Resilience Engineering - Concepts and Precepts, pp. 21-34. Aldershot,
Ashgate Publishing Company.
Woods, D.D., Patterson, E.S. and Roth, E. M. (2002): Can we ever escape from data over-
load? A cognitive systems diagnosis. Cognition,Technology & Work 14, 22-36.
Wright,P. C.and McCarthy,J.(2003): Analysis of procedure following as concerned work.In
E. Hollnagel (Ed) Handbook of cognitive task design, pp. 679-700. Mahwah, NJ,
Lawrence Erlbaum Associates.
RULE- AND ROLE-RETREAT: AN EMPIRICAL STUDY OF PROCEDURES AND RESILIENCE
Volume VI. Number 1. year 200990
6 abril 2009.qxp 15/04/2009 21:02 PÆgina 90
