Content uploaded by Petter Almklov
Author content
All content in this area was uploaded by Petter Almklov on Jan 11, 2016
Content may be subject to copyright.
ORIGINAL ARTICLE
What can possibly go wrong? Anticipatory work in space
operations
Jens Petter Johansen
1
•Petter Grytten Almklov
1
•Abdul Basit Mohammad
1
Received: 16 June 2015 / Accepted: 31 October 2015
Springer-Verlag London 2015
Abstract This paper explores how different forms of
anticipatory work contribute to reliability in high-risk
space operations. It is based on ethnographic field work,
participant observation and interviews supplemented with
video recordings from a control room responsible for
operating a microgravity greenhouse at the International
Space Station (ISS). Drawing on examples from different
stages of a biological experiment on the ISS, we demon-
strate how engineers, researchers and technicians work to
anticipate and proactively mitigate possible problems.
Space research is expensive and risky. The experiments are
planned over the course of many years by a globally dis-
tributed network of organizations. Owing to the inacces-
sibility of the ISS, every trivial detail that could possibly
cause a problem is subject to scrutiny. We discuss what we
label anticipatory work: practices constituted of an entan-
glement of cognitive, social and technical elements
involved in anticipating and proactively mitigating every-
thing that might go wrong. We show how the nature of
anticipatory work changes between planning and the
operational phases of an experiment. In the planning phase,
operators inscribe their anticipation into technology and
procedures. In the operational phase, we show how trou-
bleshooting involves the ability to look ahead in the
evolving temporal trajectory of the ISS operations and to
juggle pre-planned fixes along these trajectories. A key
objective of this paper is to illustrate how anticipation is
shared between humans and different forms of technology.
Moreover, it illustrates the importance of including con-
siderations of temporality in safety and reliability research.
Keywords Anticipation Space operations Distributed
cognition Procedures Resilience Control room Voice
loop
Don’t plan to be ready—be ready to plan
1 Introduction
From its control room in the basement of a university
building in Trondheim, Norway, the Norwegian User Sup-
port and Operations Centre (N-USOC) operates a micro-
gravity research laboratory on board the International Space
Station (ISS). With the possibility of exposing plant seeds to
different light and gravity conditions, the facility provides a
unique research infrastructure with which to study cellular
mechanisms in plants. The experiments are proposed by
external scientists, and it is the N-USOC’s responsibility to
translate the scientists’ ideas into functional experiment
designs to be executed at the ISS. This involves everything
from the initial planning to the execution of these experi-
ments, in close collaboration with a vast network of orga-
nizational nodes in the National Aeronautics and Space
Administration (NASA) and the European Space Agency
(ESA). Even though ISS crew safety may not be at stake,
failure of these experiments would mean years of planning
and millions of dollars lost. Thus, ensuring the success of
operations is the main concern of the operators, and every
possible error is subject to scrutiny.
In both the planning phase of the project and the oper-
ational phase (when the plant is actually growing at the
ISS), anticipatory work practices, staying ahead of possible
&Jens Petter Johansen
johansen.jensp@gmail.com
Petter Grytten Almklov
petter.almklov@samfunn.ntnu.no
1
NTNU Social Research, 7491 Trondheim, Norway
123
Cogn Tech Work
DOI 10.1007/s10111-015-0357-8
problems, are a key to reliability. We label these work
practices ‘‘anticipatory work.’’ Our discussion will separate
between anticipatory work practices in two different
stages: long term and real time.
In the planning of an experiment, anticipatory work is
institutionalized. Everything that can possibly go wrong,
down to the last small detail is scrutinized and solved in
advance if possible. In this stage that can last for years, the
meticulous anticipatory work of the vast research organi-
zation results in changes in hardware and software, pro-
cedures and protocols. Through individual work, meetings,
simulation and training, every detail is attended to, and
every possible thing that might go wrong is captured and
closed. Still, even when the best planned experiment is
implemented, skillful actions of control room operators are
necessary. In the dynamic execution of the experience,
with time constraints and parallel activities added to the
mix, another form of anticipatory work is crucial. The
improvisation performed in the ‘‘real-time’’ phase, when
the experiment is running and plants are growing, depends
on extensive knowledge of procedures and protocols. Here,
anticipatory work is a matter of monitoring the experiment
execution and other situational factors and staying ahead of
upcoming issues. It also involves navigating situational
time constraints in order to implement the solutions to
problems as they occur.
Improvisation in the real-time setting is improvisation
with procedures. Plans do not represent action; they are
resources employed in specific situations (Suchman 1987).
In our case, plans are highly detailed; however, the
dimension of time and an unstoppable operation cannot be
fully accounted for in advance. This article highlights the
importance of temporality to the understanding of the
relationship between procedures and situated action.
Secondly, we also argue for a socio-technical under-
standing of knowledge, by showing how knowledge and
anticipation are inscribed in different ways into procedures,
protocols and technological artifacts and how exercising
this knowledge is a joint venture of people and technology.
This means: anticipation is not only a cognitive, nor social
process, but also involves externalizations and tools. It is,
in Hutchins (1995; Hutchins and Klausen 1996) terminol-
ogy, socio-technically distributed.
Our data are based on ethnographic fieldwork at the
N-USOC, from the planning phase to execution of a specific
experiment Gravi-2. We followed the N-USOC operators
through planning meetings, simulations, tests, training and in
the writing up of procedures to study the extensive long-term
anticipatory work leading up to the experiment. We also
observed and video recorded the execution of Gravi-2 in
order to study how the operators work when their plans meet
the reality of uncertain conditions at the ISS. In this paper, we
will open the ‘‘black box’’ of a seemingly trivial event which
occurred during the execution of the Gravi-2 experiment.
There was need to reboot a computer owing to an error that
had been anticipated in advance. Several similar examples
(alike, yet with their own peculiarities) could be chosen, as
these issues are common at N-USOC. The example case
study shows how a small error anticipated through years of
planning with verified work-arounds still caused challenging
problems. By exploring this case in depth, we will show how
control room operators coordinate efforts and constantly
anticipate future events in real time.
2 Theoretical background
The first part of our analysis draws on a relational under-
standing of technology and representation. Essentially, this
means that our analysis is built on a particular interest in
how the mind, material objects and representations con-
stitute a joint system. One example of such a view is how
people use inscriptions and notes on paper, not as repre-
sentations of their thought processes, but as something with
which they think (e.g., Giere and Moffat 2003). Several
research strands, particularly in Science and Technology
Studies, forward such an understanding of technology.
Technology Studies, forward such an understanding of
technology. Stanton et al. (2006), drawing on Hutchins
(1995), argue that situational awareness is not only a
cognitive process, but also a process that is distributed
between the mind and technological artifacts.
1
. Hutchins
and Klausen (1996) show how pilots delegate parts of their
memory and attention to artifacts in the cockpit. Humans
enroll representations and tools in their thought process.
Since they are tools, representations and technologies
should be understood from a pragmatic perspective. They
are inscriptions that do something. This, the agency of
representations and artifacts is a key aspect of Latour’s
philosophy of technology. A policeman by the roadside can
prevent speeding cars. His task can also be delegated to a
speed bump (Latour 1999). The hotel manager, tired of
forgetful customers, can attach a too-big-for-your-pockets
key chain to the room keys to avoid customers taking the
keys with them when they leave (Latour 1990). This body
of research is large and contains nuances into which we
need not delve for the present argument.
2
In our analysis,
1
Interestingly, Haavik (2014a) argues that the theoretical frame-
works Normal Accidents Theory (NAT) and High Reliability
Organizations Resilience Engineering are relationally oriented in
their initial conceptions.
2
There have been debates between Hutchins and Latour on whether
or not cognitive explanations are necessary (Giere and Moffatt 2003).
Also, Latour’s insistence that the agency of technology must be
understood as symmetrical with the agency of humans is
controversial.
Cogn Tech Work
123
we will show how the research team delegates the task of
preventing these problems in different ways. They seek to
create speed bumps, to borrow from Latour’s (1999)
analogy.
3
We also draw on this literature to argue that
when they do this, adjustments to procedures, software and
hardware are a part of the teams distributed knowledge.
Their knowledge is indivisible from these externalizations.
It is in Østerlie et al.’s words (2012: 103) ‘‘inextricably
entwined’’ with them.
Within microsociological studies of work, Suchman’s
(1987) classic book, Plans and situated action, proposes a
similar argument. Plans are not simply prescriptions of
action, but also resources employed in a specific situational
context. In this paper, we discuss how different inscriptions
of answers to the question of ‘‘what can possibly go wrong’’
are generated and used in specific situations. The interest in
details and the importance of situational adaptation, repre-
sented by Suchman and others (e.g., Orr 1996; Haavik
2014b; and our own work, Almklov 2008; Almklov and
Antonsen 2014; Almklov et al. 2014), is of particular
importance when conceptualizing ad hoc situational coor-
dinative work found in the real-time phase of an experiment.
The idea that improvisation, situational adaptation or
more generally human activities beyond what is described
in procedures are necessary components of successful
work, not only sources of error to be stamped out, is
increasingly recognized by researchers in safety science as
well.
4
Most prominently, this is seen in the Resilience
Engineering strand of research and its discussions of ‘‘work
as imagined’’ and ‘‘work as done’’ (Dekker 2006; see also
Hollnagel 2015) By focusing on work as actually done, this
literature lifts the importance of situational improvisation,
experience and tacit knowledge as sources of resilience.
There are also authors that seek to move beyond viewing
rules, procedures and plans and situated practice simply as
opposites, and rather explore the nuances in how they are
interconnected and may inform each other (Nathanael and
Marmaras 2006). Hollnagel (2015) discusses the differ-
ences between work as imagined and work as done and
suggests that they can be realigned through organization
learning processes. Haavik (2014b), in a discussion of
drilling operations, details how work fluctuates between
formalized and informal ways of working.
Resilience Engineering (Hollnagel et al. 2006)isbyno
means the only strand of safety research addressing this
issue. Skillful improvisation is also a core component in
the literature on high reliability organizations (LaPorte and
Consolini 1991; see also Weick 1993; Weick and Sutcliffe
2001). More recently and more closely linked to our con-
text, Hayes (2012) discusses how control room operators
and air traffic controllers make situation-specific experi-
ence-based decision criteria in abnormal situations. Also
from control rooms operating in time-critical settings, Roe
and Schulman (2008; see also Schulman et al. 2004)
studied how control room operators operate creatively and
with little support of procedures, to control a complex
electricity grid. To operate it, they need to be able to bal-
ance loads with production (power plants). They creatively
explore their system, testing options and trying to stay
ahead of possible problems by identifying possible redun-
dancies and other solutions to problems that might occur.
Staying ahead of the situation, the cognitive orientation
toward future problems and the readiness to deal with them
is a central component in our notion of anticipatory work.
‘‘Projection of future status’’ was already in the early
descriptions of situation awareness theory identified as an
aspect of situation awareness (Endsley 1995: 37). Simi-
larly, Rosness et al. (2015) employ the concept of
prospective sensemaking, ‘‘sensemaking processes where
the attention and concern of people is primarily directed at
events that may occur in the future’’ to discuss how pro-
fessionals (in their case surgeons) work to stay ahead of the
situation and of upcoming problems. While both these
concepts would work well to describe cognitive and social
processes in the N-USOC control room, ‘‘anticipatory
work’’ is more oriented toward practices. In particular, the
operator’s intense focus on contingency plans also in the
operational phase invites a concept that highlights practice
more than thought. And they are activities in which tech-
nology and inscriptions from the planning phase, as argued
above, are an inextricable component. In sum, then, we aim
in this paper to show the dynamics between procedures and
situated practice in space operations, and in particular, we
explore the role of time in these dynamics.
3 Case—operating a biological laboratory in space
3.1 The Norwegian User Support and Operations
Centre (N-USOC) and the Gravi-2 experiment
The International Space Station (ISS) is a multinational
program for the construction, maintenance and utilization
of a habitable space station in low-earth orbit as a micro-
gravity and space environment research laboratory. As part
of the ISS operations’ ground network, the N-USOC is the
main control center responsible for a scientific facility
named the European Modular Cultivation System (EMCS).
The EMCS is an automated greenhouse used for biology
experiments in space. The focal point of this paper is one
3
Latour’s examples are trivial, but pedagogical. Consult Ribes et al.
(2013) for a more empirically relevant discussion of delegation (viz. a
networked organization managing a computing grid).
4
See Hale and Borys (2013) for a comprehensive review. See also
the volume edited by Bieder and Bourrier (2013) and Antonsen et al
(2008).
Cogn Tech Work
123
experiment, Gravi-2, that was executed within the EMCS
in May 2014. Gravi-2 is a 96-h-long biological experiment,
whereby lentil seeds are grown in the EMCS under con-
trolled atmospheric, light and gravity conditions. The
experiment consists of two experiment runs that must be of
equal length for comparison purposes. Small errors or
delays in critical phases can lead to loss of science, which,
in a worst-case scenario, could rend a multimillion dollar
experiment and years of planning worthless. Thus, an
ongoing main concern of the N-USOC operators is to
successfully complete the experiment without loss of sci-
ence or adverse effects to the research facility and without
compromising the safety of the crew.
3.2 Experiment flow and time constraints
As the prime responsible party for the Gravi-2 experiment,
the N-USOC is involved in everything from planning the
experiment (integration phase) to its execution (operation
phase).
The integration phase, which often lasts years, involves
designing the flow of experiment activities based on the
biology researcher’s ideas. The experiment design consists
of sequenced activities, from hydration of the seeds and
inserting the seed cassettes, followed by a growth period
with changing conditions of gravity and light settings,
before the seed cassettes are fixated
5
and removed for
storage (Fig. 1).
The experiment activities are regulated by strict time
constraints in order to replicate the two experiment runs in
Gravi-2. As it involves living biological systems, the
experiment must follow an uninterrupted course and
atmospheric conditions, the hydration, the growth period
until experiment stop and fixation of the seeds, must occur
at specific times. The working conditions at the ISS also
impose time constraints. There are several activities for the
crew to perform in different parts of the station and only a
dedicated amount of time for each experiment. Several
experiments and activities are run in parallel on the ISS,
and this imposes time and resource constraints. Another
point to note regarding time is that the ISS irregularly
moves in and out of the communication shadow because of
its orbit. When the ISS is in the shadow,
6
the EMCS cannot
be operated from the ground and contact with the astro-
nauts is lost. Therefore, one of the main concerns is to align
the experiment activities with these time constraints. In the
planning phase, this is a puzzle involving the whole dis-
tributed organization, but it is especially challenging when
anomalies occur and the solutions must be coordinated
with the real-time situation in the ISS.
3.3 Distributed responsibilities
In the integration phase, the N-USOC operators are
involved in writing detailed procedures for both the ISS
crew and ground personnel, and aligning the scientists’
experiment with the various constraints imposed by the
NASA and ESA regulations and conditions at the ISS. In
the operation phase, which starts a few days before the
execution of the experiment, the N-USOC is responsible
for conducting the experiment by commanding the EMCS
from the ground, interpreting and responding to telemetry,
and assisting the astronauts when crew activities are
performed.
In order to achieve this, they rely on a vast ground
infrastructure of organizations where responsibilities are
distributed among different teams and roles. NASA and
ESA have a number of organizational nodes that have
dedicated responsibilities. Some nodes are responsible for
authority in the network, having an overall picture of the
activities at the ISS and giving authority to the executing
nodes, such as the N-USOC, to perform activities. Other
nodes are responsible for data flow and the delegation of
access to send commands to the ISS, whereas others
maintain other functions, such as communicating with
astronauts. The EMCS is placed within a computer rack
which is operated by NASA; thus, the Payload Operations
and Integration Center (POIC) in Huntsville is responsible
and is the main point of contact regarding issues with the
EMCS (Table 1).
However, the computer rack is placed within the
Columbus module at the ISS, which is operated by ESA.
This requires that the Gravi-2 activities must also be
coordinated with ESA. Engineers from the technology
developer of the experiment equipment are also involved in
this work, assisting the N-USOC.
3.4 Coordination tools
To coordinate the different functions in the ISS network
during the ‘‘operations’’ phase, they rely on several coor-
dination tools. The voice-loop system represents the ‘‘front
room’’ of the ISS infrastructure (Watts et al 1996).
7
The
different organizational nodes have designated console
5
Fixating seeds means injecting chemicals into the seed cassettes
which stops the biological mechanisms within the seeds in order to
study them on the ground from the time of fixation.
6
The terminology being used for communication shadow ISS is loss
of signal (LOS) and acquisition of signal (AOS) when the connection
is good. Availability of S-band and KU-band is also commonly used
to describe communication windows.
7
Further details can be found in Mohammad et al. (2014). Watts
et al.’s (1996) rather short overview also illustrates the key features of
the voice-loop system in space operations and some of the important
ways in which it contributes to robustness.
Cogn Tech Work
123
positions within the voice-loop system which enables them
to listen in on the various talk groups to achieve an over-
view of the activities at the ISS. Permissions and technical
barriers enable or disable the various console positions
with regard to speaking on the channels, in order to avoid
disturbance and for effective communication. Communi-
cation is regulated by voice protocols with strict norms for
talking to avoid misunderstandings. The back room work,
such as mid- to long-term planning, is reserved for separate
conference calls and several asynchronic coordination
tools, such as message boards, e-mail, console logs and
daily reports. The main objective of this structure is to
remove all back room work from the front room, which is
dedicated to optimize the flow of activities and perform
efficient troubleshooting.
In addition, this work system relies on several coordi-
nation tools to achieve and overview of the ISS activities
such as the OSTPV (Fig. 2). This contains the daily plan
for each astronaut, as well as an overview of key resources,
activities and command windows. The view is customiz-
able. Notice the vertical orange line, illustrating the actual
time with upcoming events on the right. The horizontal red
and green lines above these plans are the communication
windows for data traffic and voice communication.
Looking at these displays and communicating on (and
listening to) the voice-loop system, the operators scan the
immediate future for upcoming events.
3.5 Rules, protocols and procedures
Experiments at the ISS rely heavily on rules, protocols and
procedures which regulate the actions of operators, the
design of experiment activities and the room for improvi-
sation. Safety and flight rules are non-negotiable and set
limits as to which actions are allowed in order to ensure the
safety of personnel and equipment. Protocols for commu-
nication structure, the content and the style of communi-
cation on the voice-loop system are also regulated. When
manual work by the astronauts is needed in the experi-
ments, they operate according to detailed ‘‘recipes.’’ These
crew procedures describe how the activities are to be
accomplished step by step in a ‘‘Tayloristic’’ way (or like a
computer code). These procedures are easy accessible and
shared so that all of the functions in the distributed orga-
nization follow the astronaut movements on video link
from the ISS. This ensures that the procedures are per-
formed according to plan. Other procedures function as the
shared memory of the organization, by describing previous
anomalies, possible consequences and how these have been
solved in the past. The organization is pervaded by pro-
cedures with different functions. These procedures also
serve to ensure accountability within the organization, in
order to verify that all actions are tested and aligned to the
different constraints imposed by conditions at the ISS.
Fig. 1 Schematic experiment flow of Gravi-2, second run. Copied from experiment planning documentation. The blue colored boxes indicate
activities where astronauts (crew) are involved; orange boxes show activities with commanding from ground (color figure online)
Table 1 Description of roles and console positions used in the article narrative
Role Description
EMCS Operator Responsible for the EMCS. Situated at the N-USOC
POD (Payload Operations
Director)
Manages day-to-day operations of NASA payloads at the ISS such as the EMCS. Provides authority for
commanding. Situated at the POIC (NASA)
PRO (Payload Rack Officer) Responsible for coordination and configuration of systems resources to the rack the EMCS is connected to.
Provides access for commanding. Situated at the POIC (NASA)
OC (Operations Controller) Responsible for daily planning and ensure that scheduled research activities are accomplished safely and on time.
Situated at the POIC (NASA)
FD (Flight Director) Overall responsibility for NASA operations at the ISS. Top of the hierarchy. Situated in Houston (NASA)
Engineering Support Technology developers of the EMCS. Situated at the N-USOC during operations
Cogn Tech Work
123
4 Our research methods and data
Gravi-2 was planned (and delayed several times) over a time
span of 6 years. Our data collectionstarted at the beginning of
2013 when we followed the last year of planning the experi-
ment at the N-USOC. Our project group consists of two social
scientists and one N-USOC operator, thus giving us both
insider and outsider views of the organization. The firstauthor,
one of the social scientists, also completed the N-USOC
training program in this period in order to become an operator,
thus affording us competence with which to interpret the more
complex technological and operational issues. Applying an
ethnographical approach, we followed planning meetings,
simulations of the experiment, debriefings and document
studies of procedures, as well as the writing up of these pro-
cedures and testing at the astronaut center, which all led up to
the real-time experiment. In November 2013, we conducted
interviews with all of the six operators and a former manager
to reflect upon the work done at the N-USOC prior to the
Gravi-2 experiment. We had access to all logs and docu-
mentation of both the planning phase and execution.
During the execution of Gravi-2 in May 2014, we
observed the critical phases of the experiment and took
field notes. To complement these, we performed a video
analysis of the entire experiment.
8
By analyzing the
operators’ console logs and our own field notes as a point
of departure, we were able to frame specific events and
analyze these in workshops among the authors. From this
work, two representative events have been picked out as
examples for this article. The video recordings enabled us
to more accurately reconstruct these events by transcribing
the dialog between the N-USOC and other parties in the
ISS ground infrastructure. Five hours of video has been
analyzed and fully transcribed to account for the two
examples. By comparing these data with the interviews of
the operators planning these specific activities, we achieved
a unique understanding of planned action and action
unfolded. Thus, this paper is based on different sources of
data used to reconstruct the work performed by the
N-USOC operators in the planning (integration) phase of
the experiment, and how these actions unfolded during its
real-time (operation) phase.
5 Distributed anticipation
Anticipatory work can be seen in different modes in the
integration phase of the experiment and in the operation
phase when unexpected events occur. For the purposes of
this paper, we will describe how one specific anomaly was
anticipated in the planning stages and solutions were
inscribed into the technology and procedures. We will then
Fig. 2 Screenshot of the timeline tool ‘‘onboard short-term plan viewer’’ (not from Gravi-2)
8
Mohammad et al. (2014) provides a thorough description of the
methodological audiovisual set up.
Cogn Tech Work
123
turn to the real-time execution of the experiment and see
how the same anomaly was resolved at two occasions.
5.1 Integration phase: planning of Gravi-2
In the planning or integration phase of Gravi-2, the oper-
ators tried to anticipate possible anomalies or events which
could compromise the scientific output from an experi-
ment. The N-USOC has to consider what could go wrong
with respect to the different stages of the experiment’s
activities, and account for them in ‘‘what-if scenarios.’’ The
responsibility for this work is dedicated to the person with
overall responsibility for the integration of the experiment
at the N-USOC. The scenarios are briefed to all operators
in planning meetings, where the other operators are asked
to provide input on other potential scenarios that could
compromise the experiment:
Interviewer: [The scenarios] are they just previously
experienced problems or things you imagine
can go wrong?
Responder: It is everything. First of all, it is things that
have happened before and we know can
happen again. After that we just sit and
think ‘‘what if that happens even though it
looks impossible?’’, so we start to think very
negatively, that works well, and we write
what-if scenarios. N-USOC Operator.
This process is institutionalized locally at the N-USOC
through planning meetings, but also within the ISS ground
network by a regulated review process of the experiment’s
relevant anomalies. These scenarios are inscribed into
experiment-specific documents which are shared and
reviewed in the NASA/ESA infrastructure. This is an
iterative process, whereby anticipation, reviews and addi-
tional input are inscribed into documents through feedback
loops. Thus, the procedures are a product of the operators’
anticipatory work, inscribed into documents which act as
coordination tools in the organization and a knowledge
base (Table 2).
These documents are formal accounts of their prepara-
tion process, whereby the intention is both to verify that
N-USOC has thought through its experiment, and also to
ease real-time coordination since recommended action
points are accessible if these scenarios happen. Thus,
anticipation work in the planning phase is a collective
process during which experience from the operators and
other partners in the distributed network is inscribed into
shared documents that are accessible within the organiza-
tion. Moreover, as the operators themselves contribute to
this process, these documents become a central part of their
personal and social knowledge.
Foreseen or unforeseen problems or incidents occur in
every experiment. Equipment breaks down; tests from the
ground are not representative of the conditions at the ISS,
or unforeseen parallel activities interfere with nominal
procedures or work-arounds. Previous experiments have
failed because of seemingly trivial errors. The misinter-
pretation of air flow and humidity because of weak sensor
data, the development of mold caused by non-sterilized
water and the incorrect assembly of hardware have previ-
ously caused limited scientific output, and in one case, a
ruined experiment. All previously experienced problems
and potential what-if scenarios with potential impact and
prescribed work-arounds are logged (Table 3).
There is an extreme focus on anticipating everything
that can go wrong before the experiment starts. In the
following, we give a detailed picture of how one specific
anomaly ‘‘unstable telemetry’’ was anticipated in the
planning phase and inscribed into the technology and
procedures.
5.2 Anticipating ‘‘telemetry loss’’
The N-USOC operators, together with the technology
developers, previously experienced an anomaly that caused
unstable or loss of telemetry data. In these events,
telemetry cannot be received or sent from ground. The
operators are left blind as to what is going on in the EMCS
and are unable to control it. In itself, the anomaly is
harmless in nominal situations when the seeds are growing.
However, if this anomaly occurs in critical situations in
which atmospheric conditions must be altered, it can lead
to delays, which, in a worst-case scenario, can lead to loss
of scientific output. Also, if this anomaly is accompanied
by other errors, the operators will not be able to notice or
correct it.
The source of this problem is uncertain, but a plausible
explanation is that a buildup of telemetry packages over
time leads to a miscommunication between the EMCS
computer (Standard Payload Computer) and a local com-
puter [Rack Interface Computer (RIC)] at the ISS (Fig. 3).
In a previous experiment, this error had to be fixed on
the fly. Because of this experience, the N-USOC operators,
together with the technology developers of the EMCS,
have discussed what to do if this happens again and come
up with possible work-arounds.
‘‘We have made a schedule [a command script] which
basically tells the EMCS to stop generating data-packages
so that the buffer and queue can loosen up. Then telemetry
will disappear for 30 s, and the script tells the EMCS to
generate as normal again. Then we are back to the status
quo and nominal state. So that’s how we solve it’’—N-
USOC operator.
Cogn Tech Work
123
As it is not possible to solve this problem permanently,
the N-USOC and technology developers have inscribed a
‘‘quick fix’’ into a schedule; a computer script containing a
sequence of commands that will temporarily stop telemetry
packages from being sent. This will not fix the problem
permanently, but it is considered to be a satisfactory work-
around since the source of the problem is not known. Short
dropouts of telemetry do not significantly impact the sci-
entific output, but longer periods of this anomaly can
implicate that images of the plant growth may be lost.
Moreover, the unresolved nature of this anomaly means
that it reoccurs during most of their experiments.
This work-around is not the end of the anticipatory work
for the telemetry loss anomaly. As this work-around may
not fix the problem, alternative action points were also
devised. The second action point is to restart the RIC. This
action must be performed by the PRO function at NASA
which is responsible for the RIC. In addition, this must be
done in coordination with ESA since the location of the
RIC is within ESA domain. The second work-around step
has been done before when the script has not worked
properly. If RIC reboot does not work, the third work-
around step is to reboot the surrounding rack giving power
to the EMCS. This is a critical action which implies that
Table 2 Example scenarios of critical time constraints, impact, rationale and work-around for Gravi-2
No. Contingency/
scenario
Science impact Rationale Contingency response, work-
around
6 Delay in post-
processing
activities
(demounting:
storage)
Impact on quality of fixed samples.
Degradation of samples
Due to other activities/operations delay in
Gravi-2 post-processing activities
(demounting, storage)
Both runs should be of equal duration
Fixation should be performed just after
gravistimulation, and ECs should stay
on EMCS rotors (rotating if applicable)
for 1–2 h. Cold stowage of fixed
samples at 4required
This activity is time critical
and should be performed as
soon as possible according
to the timeline
7 Gravi-2 run 2
cannot start
24 h after run 1
Impact on quality of fixatives/fixed
samples
Run 2 should start asap due to the life
time of the fixatives and fixative
residues in Gravi-2 handler. It is
recommended to perform run 2 within
24 h after end of run 1. Cleaning of the
handler is not possible
Troubleshooting and re-
planning of the activities of
run 2 asap are required
8 Early end of the
experiment due
to issues
outside EMCS
Impact on science return if experiment
aborted after less than 30 (±5) h:
Analyses affected because of
insufficient root material due to
insufficient root growth
Impact on science return/loss of science if
run 2 affected and run 2 does not have
the same duration as run 1
Experiment run length should be 30 (±5)
h to ensure sufficient root growth
Both runs should be of equal duration
Ask for crew availability for
an early sample processing
Table 3 Example scenarios of ‘‘things that can go wrong,’’ impact, rationale and work-around for the EMCS
No. Contingency/scenario Science impact Rationale Work-around
3 ER3 smoke detector
failure
Experiment cannot be
started or has to be
aborted
Due to safety issues, ER3
cannot be powered without
smoke detector
4 No telemetry from
EMCS
Experiment cannot be
started or has to be
aborted if problem
persists
No commanding possible.
Ground operators cannot
monitor important
parameters
Schedule to re-establish network connection to ER3
Perform command sequence as developed per SPR-
193 to re-establish nominal TM rate
It is suggested to start the EMCS timeline well
before the crew hydrates the CCs: in order to
detect any EMCS anomalies before the
experiment is started
5 Indication of EMCS
error (warning light
‘‘EMCS error’’ is lit)
Experiment cannot be
started or has to be
aborted
Error on EMCS subsystem Check of telemetry by ground operators. Try to find
the cause of the problem and work on
troubleshooting steps
Cogn Tech Work
123
atmospheric conditions, such as gravity and light settings in
the EMCS, will be temporarily turned off, which in some
stages of the experiment will reduce the scientific output.
These action points are inscribed into a repository system
as a Payload Anomaly Report (PAR) and tagged with a
case number (EMCS-SW-0016) (Table 4).
Signatures for the anomaly,
9
the technical and scientific-
associated impact and the preferred work-around are
written into the procedure. This will save the operators’
time if it happens again. They do not have to find the
individual commands and can follow the procedure if the
signature shows this anomaly. Also, they will have an easy
way of discussing the anomaly with others by referring to
the procedure. This gives the other nodes in the organiza-
tion easy access to the argumentation behind the work-
around in case it occurs in real time.
5.3 Operation phase: execution of Gravi-2
The situation at the ISS is different from that at the labo-
ratories on the ground. Despite being a closed and regu-
lated system, conditions at the ISS are ever changing, with
different experiment activities, different equipment and
unexpected events in other parts of the station that can
affect experiment activities. Resources, such as crew time,
storage place and commanding windows, are also limited.
Experiments are planned to a tight schedule, with several
activities, which implies that the surrounding conditions for
one experiment will change to the next. In the following,
we will see how the telemetry loss anomaly, together with
situational factors, interfered with the anticipated action
points and initiated real-time anticipation in the distributed
network.
5.4 Dealing with ‘‘telemetry loss’’ in real time:
example 1
At 01:23 GMT during the first night shift of the experi-
ment, the screens at the N-USOC suddenly start blinking
between green and blue. The ISS is within a good coverage
window, which means this is a clear indication of unsta-
ble telemetry. After studying the signatures for a few
minutes and looking through the folder with experiment
documents, the EMCS operators call POD, the first point of
contact up the hierarchy, to give a status and to get
authorization to implement the first work-around step: the
command script. The POD loop is monitored by all posi-
tions in POIC. Table 5shows an excerpt of the transcripts
(refer Table 1for description of roles):
The EMCS Ops call POD using the correct communi-
cation protocol and provide a short status of the anomaly
and refer to the specific PAR; this anomaly is documented
(EMCS-SW-16). In order to achieve the necessary
authority to proceed with the corrective commanding, POD
needs to get a go further up the hierarchy. The conversation
continues (Table 6) on the flight director loop (literally a
second after the previous).
By referring to that the anomaly is documented in a
PAR, no further questions are asked and the flight director
gives a go to continue. In addition to perform this work-
around, there is a new critical element—time. The com-
mand should be sent before the upcoming communication
shadow. PRO, which is the function that monitors infor-
mation flow between the computer racks and the payloads
and provides access for commanding, has been listening in
on the loop and jumps on the POD loop to inform that they
only have 3 min left until there is a break in communica-
tion with the ISS. POD gives the EMCS operator the
necessary ‘‘go’’ to start the work-around, and PRO gives
the necessary access by enabling the EMCS position for
commanding.
The PAR serves different functions, it provides a guide
for the EMCS operator for what to do in this scenario, but it
also serves as a point of reference to get it through the
hierarchy fast. This makes it easier for the organization to
make quick decisions and achieve the necessary authority
and access to command in a quick manner.
5.5 Dealing with ‘‘telemetry loss’’ in real time:
example 2
The ‘‘unstable telemetry’’ anomaly reoccurred several
times during Gravi-2. Closing in on the critical fixation
stage on the last day of Gravi-2, the N-USOC operators
noticed that they were losing telemetry packages again.
After consulting with the NASA entities responsible for the
data flow, system technicians and the technology
Table 4 Examples of PAR
EMCS-SW-0009 Automatic subsystems shutdown due to lost communication
between the SPLC and the TCS
2008/5/17 138 GMT OPEN
EMCS-SW-0015 Drop in power consumptions 2011/7/22 203 GMT OPEN
EMCS-SW-0016 EMCB unstable telemetry 2011/7/25 206 GMT OPEN
EMCS-SW-0016 is the PAR for unstable telemetry
9
Signatures refer to telemetry parameters, color-coded visual signs
for errors or sensor interpretation, which indicate that the system is in
a nominal or off-nominal state.
Cogn Tech Work
123
developers (engineers flown in from Germany during the
project execution), it was reasoned that in all probability,
this was a reoccurrence of the unstable telemetry anomaly.
The EMCS operator communicated this to the NASA
partners, referring to PAR EMCS-SW-0016, and got a
‘‘go’’ for starting on the work-around script. After first
trying the work-around by running the script to fix this
problem, nothing happened. The next step in this procedure
is to reboot the RIC at the ISS. However, because of par-
allel crew activities using this computer for another
experiment (Veggie), a reboot was not possible. The par-
allel activity had priority because of manual crew opera-
tions. The EMCS operator documented the episodes in the
operator log during the event (Table 7).
Because of the telemetry loss, the N-USOC is not
receiving images which are important for the scientific
output from the experiment. Instead, they have started a
video recording with a camera that is not being downlinked
through the broken telemetry chain. In addition, this time
the EMCS shows slightly different signatures than expec-
ted with this anomaly. Instead of seeing unstable telemetry
flow, the telemetry completely disappears. Other partners
in the network also read slightly different signatures. Fol-
lowing this realization, there were hectic activities at the
N-USOC to assess how they could fix this issue and what
else that could possibly go wrong. The operators and
technology developers tried to get answers as to whether
this anomaly could be fixed. The two engineers in the room
try to relate the signatures of the event to previous expe-
riences in the PAR (Table 8).
At 12:14 (GMT), the N-USOC is closing in on a shift
handover and EMCS Ops 2 enters the room. He is
responsible for the critical fixation steps and has worked on
the timing on these commands which he discusses with the
engineering support (Table 9).
The EMCS Ops explain the plan and timing for the
upcoming commanding. First, he has to send preparation
commands (stop events and observation loop) to be ready
for the critical fixation of the seed which is scheduled for
14:55. For the fixation activity (needle movement), they
need 2 h and must be finished before another critical time
constraint when the astronauts have dedicated time to
open the EMCS and remove the seed cassettes. The time
14:15 (or as early as 14:00) is pointed out as the most
appropriate time to perform the first commands, since
delays from here can lead to delays in the following
EMCS activities which have critical time constraints
(Table 10).
Table 5 Discussion on POD loop
Who Voice loop Message
EMCS Ops POD POD EMCS your loop
POD POD EMCS Ops this is POD go ahead
POD Yes sir, I am experiencing unstable telemetry from the EMCS, I believe this is related to the PAR
EMCS-SW-16. We have a fix telemetry schedule that usually fix this problem so I would like
to command this schedule to begin
POD POD Alright so you have unstable telemetry signature which you have seen before and you think
you have a corrective, which is to start a schedule?
EMCS Ops POD That’s affirmative
POD POD Ok, when you say unstable is it sort of like dropping in and out?
POD POD That’s affirmative, its dropping in and out
POD POD One second
Table 6 Discussion continues on Flight Directors loop
POD FD POD flight for EMCS
Flight director FD Go
POD FD We have some drop outs in telemetry, they have seen the signatures before and documented
in a PAR, and they have a corrective commanding and want to put that in work
Flight director FD Yes you have a go
POD FD Copy
PRO POD PRO copies POD and we only have \3 min
POD POD You go EMCS, see if you can get it in
EMCS Ops POD Copy
PRO POD And EMCS you are enabled for commanding
Cogn Tech Work
123
At 12:30 (GMT), EMCS Ops 2 prints out a sheet of
paper and moves to the back of the room to the two
engineers. He has created a table of the critical time win-
dows where certain procedure steps must be performed.
They have done this several times to account for LOS
periods and time constraints.
All procedures and commands have been planned in the
integration phase. The issues to be solved on the fly are the
time windows where these can be implemented. When
searching for opportunities to implement solutions, the
visual artifact shown such as the OSTPV (Fig. 2) is pivotal.
Due to the situational factors, the EMCS Ops have decided
to go for ‘‘Plan C.’’ The EMCS operator has also noted
down which commands are necessary to do manually and
which are automated—to be certain exactly which of the
steps that are critical to be within a communication
window.
At 12:54 GMT, there is 1 hour to the commanding
sequence must be started to prepare for the critical fixation
of the seeds. This is important in order to replicate the first
run of the experiment and be ready in time for crew
activities. This is time critical for the experiment, which is
described in a ‘‘what-if scenario’’ for post-processing
activities (Table 2, Scenario 6). Before this, the RIC reboot
must be completed in order for the EMCS Ops to start the
commanding. The EMCS operators fear a delay with the
RIC reboot, and the EMCS Ops call up the planner OC on
his loop to get attention on the time constraints (Table 11).
The EMCS Ops communicate the criticality of the 14:15
to the other cooperating partners. After 15 min, the EMCS
operators listen in on a conversation between OC and the
operator responsible for the Veggie experiment. There
seems to be some delay in this activity, and OC calls the
EMCS Ops with a status. The EMCS operator now uses
every opportunity to press on for the RIC reboot by
referring to the time constraints (Table 12).
At 13:47 (GMT), the EMCS operators have tried to get
hold of the other partners at POIC for some time now, but
Table 7 EMCS operator log (11:10–11:35 GMT)
11:10 GMT TM drops
11:18 GMT Sent FIX_TM several times, command did not went through, difficult due to TM drops
11:31 GMT RIC reboot not possible at the moment due to Veggie
11:31 GMT Talked to POD explained the fixation commanding that are really important for science and that we are losing images. Veggie will
need 1 h to finished, and we can do the RIC reboot
11:35 GMT During Veggie activities, we will record video
Table 8 Discussion at the N-USOC between engineers (11:58 GMT)
Engineer 1 PRO was saying that this is something different. That it was this data pulling…I think I understood. Then POD was asking him will
the RIC reboot fix it and he said probably yes
Engineer 2 But he also mentioned that he found this also in the PAR, this was one of the former cases
Engineer 1 Definitely. I think it is coming from their side from Express Rack. Of course he cannot assess it. He can only see what Express Rack
3 is doing. This maybe is something different. The signatures are similar for us, we get sometimes the messages here [Engineers
points to screen showing messages automated from the EMCS] with the type…Even if they don’t mean anything other than its re-
established connection. It started with going in and out and it went to a static loss. So this is quite similar from my point of view
for what we had yesterday […]
Table 9 Discussion at the N-USOC between EMCS Ops and engineering support (12:14 GMT)
EMCS Ops 2 First we stop the events. That’s just stopping and starting the observation loop
Engineer Which time?
EMCS Ops 2 Which time…I need to look at my computer. We looked at the numbers yesterday what was the optimal. I think we decided for
14:15 GMT something
Engineer 14:15…
EMCS Ops 2 Maybe as early as 14:00. We aimed for that AOS period. And then the needle movement is predicted for, I think it was 14:55. I
can double check after
Engineer So we need 2 h for it
EMCS Ops 2 Yeah
EMCS Ops 1 Yeah I will talk to OC as soon as the loops are calm again because now it’s like Veggie, veggie, veggie
Cogn Tech Work
123
this is a critical period for other experiments as well. POD
and both PROs are busy, and the EMCS Ops keep pushing
the OC whether they will meet the 14:00 mark for the
reboot (Table 13).
PRO is the console position that will actually perform the
RIC reboot, and both PROs have been busy working on
other payloads. At 13:51 (GMT), OC jumps on the POD
loop on his own initiative to get attention on the RIC reboot.
EMCSOpslisteninontheconversation(Table14).
The EMCS has finally been prioritized, and the partners
at the NASA side are ready for the RIC reboot. However, a
RIC reboot requires coordination with ESA. Due to its
placement within the ESA owned Columbus module at the
ISS, ESA partners need to perform a caution suppression
before PRO can start the reboot. After some hectic minutes
of coordination, they are finally able to begin. At 14:12
(GMT), the word ‘‘Telemetry!’’ rings through the room.
Telemetry is back, but the EMCS operator is still waiting
on the final ‘‘go’’ to start commanding. The time is pushing
toward a new LOS window, and the EMCS operator starts
examining the OSTPV yet again. The two experts from the
engineering team approach the console, and a discussion
breaks out regarding timing of the following commanding
(Table 15).
The scheduled activity was aligned to fit the windows of
communication window where the ISS is within satellite
Table 10 Discussion at the N-USOC between the EMCS Ops and the engineering support (12:30 GMT)
EMCS Ops 2 EMCS Ops 2 shows the printed sheet he has been working on: These are the numbers we agreed on last night so the time we
spoke we were discussing whether the crew activities would start 17:15 or 17:35
EMCS Ops 2 EMCS Ops 2 starts pointing at a timeslot in the OSTPV: I think it is this grey space they wanted to do the clean up. So start prep
here means I will stop the observation loop first and the extract the air needles. And once we start at 2G everything will be
automated from that point. It is just wait time between the critical steps
Engineer 1 Ok. Plan C is now the one we are going to do?
EMCS Ops 2 Yes plan C was the one. We looked at the other options and put the yellow to indicate where it is a conflict between LOS and
things like that
Table 11 Voice-loop and in-room discussions (12:54 GMT)
Who Voice loop Message
EMCS Ops 1 OC OC EMCS Ops on your loop
OC OC Go ahead EMCS Ops
EMCS Ops 1 OC I just want to get through the information that this fixation command
and procedure that we are going to do later today, I found out that actually
we need to send some preparation commands just before the fixation. And
we need to be in a good configuration before that. And it should be before 15:00 GMT
The EMCS Ops 2 comes over to console with the timetable showing the command procedures
OC OC We need to do a RIC reboot on that rack…[noise on voice loop]
EMCS Ops 2 In room We need it by 14:15 because of good coverage. Just in case
EMCS Ops 2 In room Ok
OC OC EMCS OC
EMCS Ops 1 OC Go ahead
OC OC Are you saying you want to do some unscheduled commanding before 15:00?
EMCS Ops 1 OC This is not unscheduled, this is only scheduled commanding. It is preparation
commanding before the fixation. And due to the KU-band [communication
coverage] we actually need to have good configuration before 14:15
EMCS Ops 2 In room The RIC reboot should be finished by…
OC OC Ok I see that
EMCS Ops 1 OC It is about the RIC reboot. The RIC reboot should be before that for us
to be in a configuration to start our commanding
OC OC All right, I understand
EMCS Ops are looking more closely at the notes from EMCS Ops2
EMCS Ops 1 In room Jesus, you have planned a lot
EMCS Ops 2 In room Yes, we have considered a lot of options
Cogn Tech Work
123
range, but the delayed start-up implicated that the activities
were no longer aligned with the reoccurring communica-
tion shadow. This led to real-time puzzling work for the
operator, who tried to determine the time frame for each
command, what could be sent before and after the next
communication shadow, in order to configure the EMCS in
time for the critical fixation checkpoint. He also assesses
whether he can skip one of the image sets (observation
loop) since it is more critical to be ready for the fixation
than securing another set of images. At 14:37, the EMCS
operator was finally able to send the necessary commands,
finishing minutes before the checkpoint before the com-
munication is lost again (Table 16).
The telemetry loss anomaly is an example of how years
of planning cannot fully account for the real-time situation
and its situational factors. They key distinguishing element
is time. We have shown in this section how well made
plans must be aligned with the evolving trajectory of the
experiment.
5.6 Analysis
Anticipating everything is impossible—even within a rel-
atively closed and controlled system such as the ISS. Thus,
the operator teams are organized to expect as much as
possible and to deal with the unexpected. Our example, one
of many in our observations, shows two modes of antici-
pation: long-term anticipatory work and anticipatory work
in real-time operations, and how they should be seen to be
intertwined.
5.7 Long-term anticipatory work: What can
possibly go wrong?
Long-term anticipatory work prepares the operators to
‘‘expect as much as possible.’’ This is a collective process
in the ISS network and occurs both locally in the nodes of
the organization and being distributed through the for-
malized review processes, testing, training and simulations.
Based on the N-USOC operators’ and cooperative partners’
experience and imagination of ‘‘what can possibly go
wrong,’’ the organization tries to anticipate future events.
This way of institutionalizing ‘‘negative thinking’’ in the
planning phase is a powerful tool in priming operators to
expect as much as possible.
Anticipated anomalies are described and possible solu-
tions are inscribed into computer scripts and procedures.
The organization revolves around achieving reliability
through testing and verification. Solutions to experienced
or potential problems are thoroughly tested and the asso-
ciated implications taken into account.
Our discussion of the role of artifacts in anticipatory
work is inspired by the notion of delegation from Science
and Technology Studies (Latour 1999; Ribes et al. 2013)
and Hutchins’s (1995,1996) notion of distributed cogni-
tion. When remedies for possible problems are attributed to
technological fixes, scripts, procedures and check lists, this
also transforms the humans involved. The remaining work
is altered, and the cognitive knowledge of the workers
always stands in relation to these inscriptions. In our case,
we discussed a known, but not completely understood
problem, and how to create a step-by-step work-around.
This work-around, written on paper (digitally), is an inte-
grated part of their knowledge. They know where it comes
from and why they need it. And their key task is to know
how to utilize it in specific situations. The knowledge of
the EMCS operators and engineers becomes interwoven
with the procedures that are made to represent it. It is the
Table 12 EMCS operator log (13:12 GMT)
13:12 GMT OC repeated that Veggie is their main focus at the
moment. I let him note I try to be patient and hoped
that the RIC reboot be finished by 14:00
Table 13 EMCS operator log (13:47 GMT)
13:47 GMT Tried to contact PRO, but OC got me to his loop. Both
PROs are real busy with repowering external
payloads. I asked OC if PRO would be able to meet
the 14:00 mark for the RIC reboot. We should give
them a few more minutes and then get attention on it
Fig. 3 Data flow from the EMCS to the N-USOC operators
Cogn Tech Work
123
distributed system that learns and develops in the long-term
anticipatory process, not only individuals.
The procedures serve several functions. They are a
source of information containing knowledge on how to
discover signatures of anomalies, their impact and veri-
fied work-arounds. Operators with different knowledge
levels with regard to the technical properties of the
system have easy access to summaries of the anomalies.
Buttheyarenotmerelyaninformation bank containing
step-by-step guidelines, they are also tools employed by
the operators to enable the necessary authority and
access needed within the ISS network. To achieve the
necessary authority to perform work-arounds, the
responsible parties for the overall situation at the ISS
must be absolutely certain that the chosen work-around
will not impact on the parallel activities. The procedures
act as translators, translating complex problems in the
EMCS into what the specific impact of the work-arounds
could be. They also provide accountability, in the sense
that the work-arounds have been assessed, tested and
verified.
However, long-term anticipation work focuses on
potential problems and how to work-around them. It is
impossible to account for the temporal dimension at the
ISS due to concurrent activities and situational factors.
5.8 Real-time anticipatory work: When can we
do it?
Unexpected events or signatures regularly occur during
operations, and small interacting details often lead to sit-
uations that the operators have not experienced before. In
such situations, the control room is on high alert, with
people streaming into it, swearing, sighing and holding
heated discussions, followed by intense activity on the
voice-loop system (however, according to protocol, always
calm and concise) in order to gather information, assess the
situation and anticipate any potential impact and upcoming
events. Even in situations such as ‘‘unstable telemetry’’
where the error is seen before, the situation in which it
must be resolved, in this case the parallel and delayed
Veggie activity is always unique.
The seemingly trivial case of ‘‘rebooting of the com-
puter’’ described above is an example of how errors in a
tightly coupled system can have potentially cascade to
Table 14 Discussion on the POD loop (13:51 GMT)
Who Loop Message
OC POD POD OC
POD POD Go OC
OC POD Yes sir, this may be a good window for PRO1 to go ahead and reboot RIC in rack 3. We kind of get to the time when we
need that
POD POD Ok, I don’t know. PRO do you have time for RIC reboot in Express Rack 3?
PRO POD I have finished AMS, PRO2 is doing the ELC, so I can do Rack 3 RIC reboot
POD POD Ok, if you don’t mind that will be good, if you need a few minutes to get prepared for that it is fine too
PRO POD I will but it won’t take long
POD POD Ok I appreciate it
EMCS
Ops
POD EMCS copies all, thank you guys
Table 15 Discussion at the N-USOC between the EMCS Ops and engineering support (14:16 GMT)
Engineer Can you start immediately after you get a go with the needle injection or?
EMCS Ops The problem is this S-band [communication coverage] here so…the commands I have is also…depending on where the
observation loop is. I’m not sure if I should…Ideally I should wait for the observation schedule to finish as well. So if we’re
lucky we can stop it…We will have to see. I think there is time afterward
Engineer Ok
EMCS Ops This one [points at a timeslot in the OSTPV] starts at 14:55 or I would say 14:57 with good coverage. So that should be more than
sufficient time. And fixation begin yeah 15:15. That should be enough time
Table 16 Discussion at the N-USOC between EMCS Ops and
engineering support (14:37 GMT)
EMCS Ops And we have 1 min and 30 s to LOS
Engineer Hehe
EMCS Ops Well timed!
Cogn Tech Work
123
devastating effects. The work-around and scientific impli-
cations of this issue are known from the planning phase and
shared in the network through the procedures. The RIC
must be restarted in order to successfully fixate the seeds in
order to replicate the experiment runs and not lose scien-
tific output. In this case, the previous anticipation work
could not take into consideration how parallel activities
using the RIC, which are outside N-USOC’s responsibility,
could interfere with their second action point: RIC reboot.
The planned solutions focus only on solving the specific
problem without possibility to consider interconnection
with other activities and situational constraints at the ISS.
The procedures and work-arounds are not sufficient to
solve the problem in the operational phase. This left it to be
solved real time to align the temporal trajectories of par-
allel activities and time constraints. Assessing what-if sit-
uations in the upcoming minutes and hours, the time
constraints imposed by the experiment requirements, the
satellite coverage, parallel activities, and how long each
activity and specific command script will take, reflects a
complicated puzzle with temporal trajectories. The exam-
ples show how the operators stay ahead of the situation
hours before the critical time constraints and continuously
plan for several different trajectories as the situation
evolves.
By employing the anomaly procedure, the operators
work to establish a shared understanding of the scientific
impact of this error and work-around in the network. But
they also have to establish a shared understanding of the
time constraints which are not prescribed in the procedures.
This is almost ‘‘nagging’’ behavior with using every
opportunity to make the other functions aware of the time
constraints is a key part of the operators work. When time
was of essence, both the technical inscriptions and referring
to procedures save time and avoid potential misunder-
standing, as well as the need to double check action points.
Protocols for communication enable efficient communica-
tion between the relevant nodes in the network.
During experiment execution, there is no time to per-
form an institutionalized procedure and review process.
Rather, decisions are communicated in real time with the
other nodes in the network. Through the voice-loop system,
the N-USOC operators listen in on ongoing conversations
about parallel activity in order to assess how much time it
will take and to stay ahead of possible upcoming coordi-
nation and synchronization issues. This is a crucial way of
anticipating upcoming problems and opportunities, and
keeping track of other activities real time. In our case, we
saw how listening to the activities concerning another
experiment (Veggie) was an important input for their
contingency planning for the upcoming hours. Another key
aid for the contingency planning is the prominent visual
timeline of activities and communication shadows
(OSTPV). In the integration phase, this is used to describe
the planned activity on the ISS, but in real time this is also
used as a tool to improvise.
In these specific examples, we show how improvisation
can take the form of alignment of solutions with other
activities and time constraints. As one of the N-USOC
operators state, improvisation can take somewhat different
forms in situations when the problem is unknown.
‘‘If something unexpected happens, and you see a wrong
signature that you haven’t seen before…how do you
approach it and start to retrieve information to understand
what’s going on? What the impact will be and how to plan
ahead? That kind of improvisation is necessary, and it’s
Table 17 Conceptual model of long-term and real-time anticipation work for ISS experiments
Long-term anticipatory work Real-time anticipatory work
Plan until ready Plan while executing
Identify what can possibly go wrong and its impacts. Anticipated
anomaly signatures are identified and mapped
Signatures are constantly monitored and compared to previous
situations. Voice loops are monitored to anticipate interacting
activities
Plans are designed to solve the specific problem Stay ahead of unexpected events, interaction effects and situational
factors
Identify what can be done about it Identify which and when solutions and work-arounds can be
performed. Establish a shared understanding of problem, work-
around and time constraints
Solutions and work-arounds to anticipated problems are inscribed into
procedures. This list of procedures is the shared memory of
experienced and anticipated anomalies in the distributed
organization. Delegation between operator’s knowledge and
technology
Procedures are also tools to establish a situation awareness real time.
Necessary in order to get the authority and access to perform
procedures. Referring to an anomaly procedure makes the
organization ready to respond swiftly to contingencies
Carefully planned work flows. All resources, procedures in use and
known constraints are mapped and inscribed into tools (such as the
OSTPV and PARs). Not dominated by time or situational factors
Improvisation by aligning temporal trajectories between various
constraints. OSTPV is an important tool to anticipate the near future
and change plans
Cogn Tech Work
123
difficult. It requires knowledge about the experiment and
facility’’—N-USOC system operator.
Although other forms of improvisation may occur when
signatures are unknown, improvisation with procedures
aligned with temporal trajectories is a key situation in this
work system. Our case shows two different modes of
anticipatory work in the integration and operation phase of
ISS experiments (Table 17).
5.9 Implications
5.9.1 Temporality and operations
There is a marked difference between the preparatory
stages of the experiment and the real-time phase with
regard to time. As soon as the seeds are watered, the
experiment has commenced and time is critical. Experi-
ments at the ISS are a work system with several particular
features such as the possibility for long-term planning, near
to unlimited resources for risk analysis and a closed system
where every parameter is almost constantly monitored.
Implications can, however, be drawn for other work sys-
tems. Forms of long-term and real-time anticipatory work
could be studied elsewhere. In particular, we believe that
the difference between planning and operations with regard
to time is relevant to other work contexts, especially where
operations are unstoppable (i.e., critical infrastructures and
petroleum processing plants). Our case serves to illustrate
that even in situations with extensive resources available
for planning and preparation, plans need to be situated in
the temporal flows of practice. That there is a difference
between ‘‘work as imagined’’ and ‘‘work as done’’ (Dekker
2006; Hollnagel 2015) or plans and situated action
(Suchman 1987) is established knowledge. Our case
explores how temporality affects this relationship. In the
integration phase, possible problems and contingencies are
methodically identified and studied, and risks are analyzed
and mitigated in a formal process. As far as possible, plans
and future work processes are aligned. During the opera-
tional phase, however, new contingencies must be inter-
preted and solved as they occur, and here improvisation is
necessary. In our example, the problem itself (the telemetry
error) and even its solution had been anticipated in
advance. Improvisation was still needed to align plans with
other situational factors, such as other experiments and the
irregularities of the communication shadow. This high-
lights one aspect of procedures that is often forgotten.
Procedures are detached from the temporality of actual
work practices.
The telemetry error itself did not require much analysis
in the real-time phase, as it had been anticipated in
advance. This is not always the case for the experiments at
the ISS, and sometimes the analysis of the problem itself
can require much creativity and improvisation in real-time
situations. This is quite similar to descriptions of control
room operators (e.g., Roe and Schulman 2008; Hayes
2012) and others that have to make sense of vast amounts
of data and make decisions on the fly to keep the system
running. Handling complex information and making deci-
sions under time pressure often necessitate improvisation.
Our case shows that also when the problem and the solu-
tion are known, improvisation may be necessary. When
solving the telemetry issue, extensive work was performed
to embed the prescribed interventions in the evolving
temporal flow within the ISS network. There are two
aspects of temporality at play here. The first is the obser-
vation that sometimes control room operators do not have
time to use procedures and operate in a rule-based manner
when faced with complex and unpredicted situations. The
second is a bit more intricate: improvisation is necessary to
place procedures into the evolving trajectories of events.
When an experiment is well planned and runs smoothly,
this temporal alignment is still one of the key tasks of the
N-USOC operators.
Similar types of temporal juggling also occur in other
contexts. For example, in the same way as the biological
experiment is ticking away, operational work on many
critical infrastructures is characterized by its unstoppable
nature. Almklov and Antonsen (2014) describe the situated
work of operators’ with managing water and electricity
grids in operation and argue that the handling of temporal
trajectories is an undervalued part of this and that tempo-
rality is poorly reflected in procedures and task descrip-
tions. Kongsvik et al. (2015) discuss how operators at
petroleum processing plants juggle work permits on a shift
into a workable (and safe) temporal trajectory avoiding
resource conflicts and dangerous interaction effects
between activities.
At the N-USOC, everything is as far as possible planned
and controlled in advance. What remains uncontrollable in
advance and subject to skilled real-time improvisation is
the articulation of temporal trajectories. Real-time antici-
patory work is the activities conducted to stay ahead of
events during operations in order to align tasks and
resources toward the unstoppable river of time.
5.9.2 Rules, procedures and resilience
Hale and Borys’ (2013) review, ‘‘Working to rule or
working safely?’’ summarized parts of a discussion which
has been important in safety research for the last few years.
Are rules, procedures and compliance the way to go or
should we aim for competent improvisation? Of course,
most researchers end up in middle-ground positions in this
debate. Our paper could add some nuances to this topic.
First of all, it is a clear observation in our paper that
Cogn Tech Work
123
procedures, formal protocols and rules are indispensable
artifacts in N-USOC’s work. They are everywhere. The
improvisation conducted in the real-time phase depends on
an extensive and intimate relationship with these formal
documents. Our socio-technical perspective highlights the
procedures as an integral part of the knowledge on which
improvisation is carried out. Our case is special, though, as
our informants’ procedures are partly developed by them-
selves and are constantly revised and closely aligned with
practice. The process by which learning from operational
experience is inscribed in procedures is institutionalized,
and the operators themselves contribute to it. As noted by
Antonsen et al. (2008), procedures may serve several
purposes, some of which make them less useful as tools in
work execution.
10
Thus, while formal rigid procedures
(developed elsewhere to serve purposes of management,
accountability and control) are often in opposition to situ-
ated improvisation and expert judgement, this dichotomy is
less relevant in cases where the procedures are an inte-
grated part of the workers’ ‘‘distributed cognition’’
(Hutchins 1995), as in our case. For the N-USOC opera-
tors, plans and procedures are tools that they use also when
they improvise.
6 Conclusion
In this paper, we explored anticipatory work in space
operations. By inspecting one specific technical issue in
depth, we observed how control room operators work for
the purposes of anticipating, as far as possible, what can go
wrong and how they delegate their understanding to pro-
cedures and technologies. In real-time situations, these
artifacts are more than guidelines, and they serve as tools to
align the understanding of a situation in a distributed
organization. By studying the tight connection between the
planning process and real-time work, we see the contours
of two modes of anticipatory work. While long-term
anticipatory work is focused on potential problems and
fixes, the main concern in real time is to align temporal
trajectories and stay ahead of upcoming problems. In this
setting, both are imperative for safe and successful exper-
iments. In the case we have studied, rules and procedures
are intrinsic parts of the anticipatory work of our infor-
mants. Arguably a somewhat ‘‘special’’ case, our obser-
vations of the dynamics between procedures and
improvisation still has relevance for other work systems.
Rather than discussing rules versus improvisation in
operational work, maybe our case could be an inspiration
to discuss what makes rules and procedures resources in
real-time situations and what does not. The close worker
involvement in procedure development and the strict
operational focus (versus the focus on accountability and
administrative issues sometimes seen other places) of the
procedures might be parts of the explanation. Another
implication of our paper is the importance of time for the
ways in which safety is achieved. Our organization goes
from a project phase to an operational phase, where an
unstoppable clock is ticking. The nature of work, and the
ways one should support it to make it as safe as possible, is
radically different in these two situations. We have stressed
the importance of temporal coordination in real time for
resilience and shown some strategies to obtain it, but here
we believe there could be other interesting discussion to be
pursued regarding procedures as a part of distributed cog-
nition and more generally how the temporal dimension of
situated work relates to plans and procedures.
References
Almklov PG (2008) Standardized data and singular situations. Soc
Stud Sci 38(6):873–897
Almklov PG, Antonsen S (2014) Making work invisible: new public
management and operational work in critical infrastructure
sectors. Public Adm 92(2):477–492
Almklov PG, Østerlie T, Haavik TK (2014) Situated with infrastruc-
tures: interactivity and entanglement in sensor data interpreta-
tion. J Assoc Inf Syst 15(5):263–286
Antonsen S, Almklov P, Fenstad J (2008) Reducing the gap between
procedures and practice–lessons from a successful safety inter-
vention. Saf Sci Monit 12(1):1–16
Bieder C, Bourrier M (eds) (2013) Trapping safety into rules: an
introduction. Trapping safety into rules. How desirable or
avoidable is proceduralization. Ashgate Publishing, Farnham
Dekker S (2006) Resilience engineering: chronicling the emergence
of confused consensus. In: Hollnagel E, Woods DD, Leveson N
(eds) Resilience engineering: concepts and precepts. Ashgate,
Hampshire
Endsley MR (1995) Toward a theory of situation awareness in
dynamic systems. Hum Fact J Hum Fact Ergon Soc 37(1):32–64
Giere RN, Moffatt B (2003) Distributed cognition: where the
cognitive and the social merge. Soc Stud Sci 33(2):301–310
Haavik TK (2014b) On the ontology of safety. Saf Sci 67:37–43
Haavik TK (2014c) Sensework. J Comput Support Cooper Work
23(3):269–298
Hale A, Borys D (2013) Working to rule, or working safely? Part 1: a
state of the art review. Saf Sci 55:207–221
Hayes J (2012) Use of safety barriers in operational safety decision
making. Saf Sci 50(3):424–432
Hollnagel E (2015) Why is work-as-imagined different from work-as-
done. Resilience in everyday clinical work. Ashgate, Farnham,
pp 249–264
Hollnagel E, Woods DD, Leveson N (2006) Resilience engineering:
concepts and precepts. Gower Publishing Company, Aldershot
Hutchins E (1995) Cognition in the wild. MIT Press, Cambridge
Hutchins E, Klausen T (1996) Distributed cognition in an airline
cockpit. In: Engestro
¨m Y, Middleton D (eds) Cognition and
10
For example, they can be ways for management to show the
authorities that a lesson has been learned from the incident, or to
assign responsibility (or blame) for specific issues.
Cogn Tech Work
123
communication at work. Cambridge University Press, Cam-
bridge, pp 15–34
Kongsvik T, Almklov P, Haavik T, Haugen S, Vinnem JE, Schiefloe
PM (2015) Decisions and decision support for major accident
prevention in the process industries. J Loss Prev Process Ind
35:85–94
LaPorte TR, Consolini PM (1991) Working in practice but not in
theory: theoretical challenges of ‘‘high-reliability organiza-
tions’’. J Public Adm Res Theory 1(1):19–48
Latour B (1990) Technology is society made durable. Sociol Rev
38(S1):103–131
Latour B (1999) Pandora’s hope: essays on the reality of science
studies. Harvard University Press, Cambridge
Mohammad AB, Johansen JP, Almklov P (2014) Reliable operations
in control centers, an empirical study safety, reliability and risk
analysis: beyond the horizon: proceedings of the European safety
and reliability conference, ESREL 2013, Amsterdam, The
Netherlands, 29 Sept–2 Oct 2013, CRC Press
Nathanael D, Marmaras N (2006) The interplay between work
practices and prescription: a key issue for organizational
resilience. In: Proceedings of the 2nd resilience engineering
symposium, pp 229–237
Orr JE (1996) Talking about machines: an ethnography of a modern
job. Cornell University Press, Ithaca
Østerlie T, Almklov PG, Hepsø V (2012) Dual materiality and
knowing in petroleum production. Inf Organ 22(2):85–105
Ribes D, Jackson S, Geiger S et al (2013) Artifacts that organize:
delegation in the distributed organization. Inf Organ 23(1):1–14
Roe E, Schulman PR (2008) High reliability management: operating
on the edge. Stanford Business Books, Stanford University Press,
Stanford
Rosness R, Evjemo TE, Haavik TK, Wærø I (2015) Prospective
sensemaking in the operating theatre. In: Review for cognition,
technology and work 1–17. doi:10.1007/s10111-015-0346-y
Schulman P, Roe E, Eeten MV, Bruijne MD (2004) High reliability
and the management of critical infrastructures. J Conting Crisis
Manage 12(1):14–28
Stanton NA, Stewart R, Harris D et al (2006) Distributed situation
awareness in dynamic systems: theoretical development and
application of an ergonomics methodology. Ergonomics
49(12–13):1288–1311
Suchman L (1987) Plans and situated actions. Cambridge University,
New York
Watts JC, Woods DD, Corban JM et al (1996) Voice loops as
cooperative aids in space shuttle mission control. ACM confer-
ence on computer-supported cooperative work
Weick KE (1993) The collapse of sensemaking in organizations: the
Mann Gulch disaster. Adm Sci Q 38(4):628–652
Weick KE, Sutcliffe KM (2001) Managing the unexpected. Assuring
high performance in an age of complexity, San Francisco
Cogn Tech Work
123
A preview of this full-text is provided by Springer Nature.
Content available from Cognition Technology and Work
This content is subject to copyright. Terms and conditions apply.
