PreprintPDF Available

Resiliency Trade Space Study: The Interaction of Degraded C2 Link and Detect and Avoid Autonomy on Unmanned Aircraft

Preprints and early-stage research may not have been peer reviewed yet.


Content may be subject to copyright.
Resiliency Trade Space Study:
The Interaction of Degraded C2 Link and
Detect and Avoid Autonomy on Unmanned Aircraft
David D. Woods and E. A. Balkin
The Ohio State University
June 27, 2018
All material Copyright © 2018 by David D. Woods, All rights reserved
Resiliency Trade Space Study:
The Interaction of Degraded C2 Link and Detect and Avoid Autonomy on Unmanned
David D. Woods and E. A. Balkin
The Ohio State University
Detect and Avoid Autonomy on Unmanned Aircraft
The Unmanned Aerial System industry would like to authorize broader use of the
autonomous capabilities of the Detect and Avoid (DAA) system onboard unmanned aircraft
(UA) in a wider range of operational conditions. Under current operational plans for handling
a potential loss of well clear encounter during UA flights, the DAA system provides the
ground pilot or remote pilot in command (RPIC) with a complete set of conflict free maneuver
options that the human ground pilot then selects from and executes (see RTCA, NASA).
Under some conditions, automated DAA systems hold the potential to permit the UA to act
with a greater degree of autonomy than would otherwise be possible. DAA systems could
automatically execute an avoidance maneuver to prevent loss of well clear criteria during an
encounter, and then inform the RPIC and other parts of the aviation system about its actions
as necessary afterwards. In such a scenario, UA would take actions on its own without direct
proximal instruction. This would represent a significant divergence from current operational
standards and would therefore, necessitate that stakeholders are willing to delegate authority
for actions to handle potential loss of well clear encounters to DAA automation with minimal
oversight and coordination.
Degraded Communication Link
One such context for delegating authority to the onboard DAA system for staying well clear
of other traffic occurs when there is a failure or performance degradation in the of the
command and control (C2) link between the ground control station (GCS) and the unmanned
aircraft (UA). The satellite communication (SAT-COM) C2 link envisioned is subject to
performance limits and failures. Specifically, intermittent connection drop-outs, variable
latencies, including latency times exceeding the two-second round-trip limit have all been
observed (see Minimum Operational Performance Standards for Detect and Avoid Systems
or DAA MOPS and RTCA results). C2 link degradation will challenge the ability of a pilot in the
loop to effectively navigate or manage the aircraft, especially during a potential loss of well
clear encounter when the the UA’s operational situation is changing rapidly and swift action
may be required. In order for maintain safe operations, the UA would need the capacity to act
autonomously during periods of C2 link degradation or loss (long latency or failure).
Robust and Resilient System Performance
Communication reliability has a number of consequences for integrated UA system
performance. First, studies of the effects of deploying new autonomous capabilities show
design for coordination is a critical requirement for robust and resilient system performance
when anomalies and surprises occur (Johnson et al., 2014). Second, the Defense Science
Board Task Force on Autonomy in 2012 recommended that trade space studies be included
when new deployments of autonomous capabilities are planned (Murphy and Shields, 2012;
see also Buchanan et al., 2018). Third, Resilience Engineering has found that autonomous
machines are brittle, the brittleness of machines can lead to new forms of failure, and there
are systems engineering interventions that build resilience (Woods, 2006; Woods, 2016).
Resilience Engineering, as a field of inquiry has, for many years, been developing the
theoretical and conceptual foundation for resiliency trade space studies (Hoffman and
Woods, 2011).
These concerns defined the context for this project to carry out a Resiliency Trade Space
Study that examined the resilience of the integrated system when degraded communications
occur. The integrated system for this context includes interactions between C2 link reliability,
autonomous DAA capabilities, delegation of authority to onboard DAA software/sensors/
hardware, remote pilot in command’s (RPIC) responsibility for safe operation of the UA, and
the need for coordination across roles in the air transport systems when potential loss of well
clear encounters occur in the national airspace system.
This project consisted of two parts: (a) developing a Resiliency Trade Space Analysis
approach for envisioned deployments of autonomous capabilities, and (b) using the
approach to provide an initial analysis for the specific context of degraded C2 link and DAA
autonomy. The latter part was commissioned to provide initial results on a fast track as an
input to RTCA deliberations in the summer of 2017.
The advances in Resilience Engineering provide only general guidance for resiliency trade
space analysis (Alderson and Doyle, 2010; Hoffman and Woods, 2011). As a result, the
guidance needed to be adapted for deployments of autonomy (Murphy and Shields, 2012;
Woods, 2016). Second, the analysis technique needs to be formulated for the particular case
of unmanned aircraft operating in controlled airspace with other air traffic. The result is a
method for Resiliency Trade Space Analysis based on principles of Resilience Engineering
(Hollnagel et al., 2006) and relevant to assess the impact of new autonomous capabilities. The
method is designed to identify how an integrated system of multiple human and machine
roles will perform in the face of anomalies, disturbances, and surprises, given design
decisions about the competencies under development for engineered systems. This
technique examines the factors that will enhance or undermine resilient performance when
challenging ‘edge’ events occur. The method was then used for the case of degraded
communications between UA and remote pilot (RPIC) including C2 link performance, DAA
capabilities, traffic complexity in controlled airspace, ground pilot roles, and ATC
responsibilities. Note the method and its application to DAA autonomy focuses on
envisioned performance and builds on previous work on envisioning the impact of new
technology on complex settings such as aviation (Dekker and Woods, 1999a; 1999b; Woods
and Dekker, 2000).
Assessing the Potential for Resiliency
System Complexity and Resilience
Recent national advisory reports on autonomous systems (e.g., Defense Science Board, 2012;
National Research Council, 2014) have consistently noted a key barrier to the development
and deployment of increasingly autonomous systems in aviation.
Barrier Statement: Increasingly autonomous capabilities create a more complex
aviation system, with new interdependencies and new relationships among various
operational elements. This will likely reduce the resilience of the civil aviation system:
disturbances in one portion of the system could cause the performance of the entire
system to degrade precipitously.
The Defense Science Board (DSB) Task Force on Autonomy (2012) noted that as increasingly
autonomous platform capabilities continue to grow, the critical focus for design and testing is
the larger multi-role/multi-echelon system in which the vehicle operates. For example, the
report states, “a key challenge presented by the complexity of software is that the design
space and tradeoffs for incorporating autonomy into a mission are not well understood and
can result in unintended operational consequences (p.11).” The Networking and Information
Technology Research and Development (NITRD) report on Grand Challenges (NITRD, 2006)
noted the opportunity to “understand how people, (software) agents, robots, and sensors
(PARS) contribute to a collaboration” and the difficulties to “understand the structural
complexity of PARS collaborations (for example, teams, networks, or hierarchies into which
the PARS components can self-organize).” Echoing the DSB findings, the NITRD report called
for R&D on architectures for adaptive layered networks (see also Alderson and Doyle 2010;
Doyle and Csete, 2011). A 2012 NITRD workshop directly addressed the complexity issues
and noted that we do not have the technical means to design and test complex engineered
networked systems (NITRD, 2012). The report points to the need to develop new designs of
networks, networks composed of increasingly autonomous vehicles as well as human roles,
that dependably meet criteria of flexibility, robustness, resilience, extensibility as the
environment, technology, and uses change over time (see also Woods, 2016). Adaptive
networks including increasingly autonomous platforms need to be capable of increasing
optimality on some specific target dimensions, while possessing increasing robustness for
well modeled disturbances, and while still retaining the ability to adapt and evolve its
mechanisms to be resilient in the face of surprises and change (Woods, 2015).
The DSB report noted that “current designs of autonomous systems, and current design
methods for increasing autonomy, can create brittle platforms, and have led to missed
opportunities and new system failure modes when new capabilities are deployed. Brittle
autonomous technologies result in unintended consequences and unnecessary performance
trade-offs, and this brittleness, which is resident in many current designs, has severely
retarded the potential benefits that could be obtained by using advances in autonomy” (p.
7-8, highlighting in original). The report recommends the “development of measures and
models of the dimensions of system resilience/brittleness that can be used early in systems
development as well as later in T&E” (test and evaluation, p.11). The problem of brittleness of
autonomous systems has been noted for years in human-machine systems research (Roth et
al., 1987; Smith et al., 1997; Perry et al., 2005; Smith 2018), and overcoming brittleness is a
critical goal that guides the development of the growing field of Resilience Engineering
(Hollnagel et al., 2006; Spero et al., 2014).
The barrier of complexity arises because current designs disproportionately focus on adding
on-board autonomous capabilities and minimize the need for supporting coordination in a
geographically and temporally distributed and layered system (Johnson et al., 2014).
Resilience Engineering takes a broader perspective that designs a multi-role, multi-echelon
system which includes both human and various computational and robotic roles, and tests
how well the network can synchronize the distributed activities to keep up with the changing
pace and tempo of dynamic situations. !Such systems are distributed with multiple layers and
highly adaptive - changing the relationships across roles and echelons to be able to match
changing demands and situations — so as to meet robustness/resilience goals as well as cost/
productivity goals for the integrated system.!
Together these sources indicate:!
(a) The growth of increasingly autonomous systems establishes the need for new ways
to assess, measure, and test unintended consequences from deployments of
autonomous capabilities into risk critical, distributed systems.
(b)There remain unsolved technical challenges.
(c) There are several promising trends in interdisciplinary research such as in
Resilience Engineering to anticipate and overcome complexity penalties.
Resiliency Trade Space Analysis for Human-Machine Complex & Adaptive Systems
This new method examines the interactions of three general factors:
• Envelope of Engineered Competencies — the envelope of capabilities of engineered
systems to perform in the intended environments. For example, ability of a UA to see
traffic nearby given radar range and other capabilities and parameters.
• Potential for Resilient Performance — the capabilities of all various agents in the
airspace system to adapt and coordinate to handle disruptions to plans, including
contingency plans. Two capabilities are key — anticipation to keep pace with
potentially cascading effects of disruptions and coordination across roles and
echelons in these challenge events. This step asks how will coordination and
adaptation occur at the boundaries of engineered competencies when events
challenge the competence envelope. Edge scenarios are used to assess brittleness at
boundaries, revealing how the system will prove to be brittle and what adaptations will
emerge to defuse this threat. These results identify requirements for resilient
performance when events challenge the engineered competence envelope.
• Risk of Consequences Associated with Brittleness. These risks address performance
penalties, efficiency loses, and likelihood of accidents from brittle breakdowns in
coordination and adaptive capacities for the operational context.
Table 1 captures the three aspects for Resiliency Trade Space Analysis.
Challenges from Edge Cases
The project focused on #2 above, which required gathering information about the
competence envelope for DAA and C2 systems on UAs from NASA studies, RTCA MOPS, and
meetings with NASA (at NASA Ames for DAA systems and at NASA Glenn for C2 systems) and
participation in RTCA SC-228 working group meetings (May 22 – 26, Ft. Collins, CO and July
10 – 14, Washington, D.C.).
Testing the potential for resiliency uses edge cases to assess the brittleness of autonomous
capabilities proposed for development or deployment. Developing edge cases uses
information about assumptions and engineering trade-off decisions in defining standards for
DAA and C2 systems, and then uses this information to begin to see how events can
challenge integrated system performance.
Edge cases present anomalies that challenge the system’s competency envelope. Assessing
resilience explores how the different interdependent roles can coordinate as the anomalies
develop over time. The specific anomalies built into each edge case instantiate general kinds
of challenges and difficulties.
How the integrated system would respond to the cognitive and coordination challenges
embedded in that anomalous situation is explored through cognitive walkthroughs.
Walkthroughs can be carried out at different levels of fidelity depending on the resources
available, the time pressure for results, and the degree of rigor required. In this project, input
from the trade space study was desired quickly.
Assembling the data on how agents would respond to the evolving anomaly provide the
basis for assessing how the system is brittle and how people will adapt to overcome potential
brittleness. These results then inform options for changes that will reduce brittleness and
improve the potential for resilient performance when challenges occur. The results can be
combined with assessments of the potential consequences from brittleness driven
performance breakdowns using familiar methods for estimating risk.
The potential for resiliency assessment asked the question:
How can the UA/DAA act as a cooperative agent in shared airspace when disruptions
occur and standard communication channels are compromised?
The assessment first examines the interactions across the full set of agents and roles who
need to coordinate throughout the disruption — from initial triggering event through
responses and return to normal planned traffic flow. This steps considers all of the roles and
agents involved in the integrated system of UA/DAA, PIC, ATC, and organized traffic flows
(Fern et al., 2015; Fern, 2016). Second, the assessment looks at how the different roles and
agents will coordinate when disruptions occur. Third, the disruptions requiring coordination
occur in combination with a degraded or lost C2 link.
The result is the basic form for edge cases that challenge the UA to act as a cooperative agent
in shared airspace when disruptions occur and standard communication channels are
1) Well-structured traffic flows given previous decisions, trade-offs, and criteria by
ATC for the airspace context including UA flight plans and contingencies.
2) Triggering event occurs which disrupts traffic flow.
3) Surprise — the event produces surprises and uncertainty about the ongoing
behavior and interactions with respect to maintaining separation and safety as
the disruption spawned by the triggering are handled.
4) Adjustment/re-configuration as the various parties act and coordinate to handle
the cascading effects of the disruption(s).
5) Return to structured flow as the situation is resolved.
Edge Scenarios
The basic challenge can be instantiated in many different edge scenarios (See Table 2). One
example is:
• Typical traffic flows are moving through well-structured (non-terminal) airspace;
• Having completing its pipeline survey mission, UA begins climbing to a higher
• While climbing, UA encounters extreme wake turbulence and is unable to maintain
level flight or proper orientation;
• Intermittent satcom loss due fuselage masking;
• While losing altitude, UA violates well-clear volumes and intended flight paths of
other sector aircraft;
• UA maneuvers to stay well-clear
• ATC manages other aircraft in the sector given UA behavior
• ATC begins to re-establish traffic flow given performance goals.
In this particular case, the triggering event that disrupts traffic flows is the UA behavior given
some UA vehicle’s susceptibility to wake turbulence. The UA’s behavior also initiates a
mechanism that degrades the C2 link — fuselage masking. The scenario is based on one kind
of mission planned for unmanned aircraft, but one that can interact with controlled airspace.
The competence envelope is relevant as a source of information about what the UA systems
can do and the limits on its performance, given the missions that the UA may perform.
The analysis built a set of challenge scenarios and several variations on each scenario. These
are summarized in Table 2.
Cognitive Walkthrough
Cognitive Walkthrough is a well established method in human-machine systems (Polson et al.,
1992; Lewis and Wharton, 1997). Cognitive Walkthroughs can be carried out at different
levels of fidelity and are especially useful to address envisioned situations and technologies
(for examples in aviation and air traffic control see Dekker and Woods, 1999a; 1999b; Smith
et al., 1997; Smith, 2018).
In this project, cognitive walkthrough method was utilized: various people with professional
experience in air traffic management were asked individually or in groups to walkthrough
some of the scenarios (depending on their time availability). As they moved step by step
through the scenario, they were asked: (a) what would you expect the UA to do next? Given
your answer to (a), how would/should ATC respond?
Robust? and Resilient?!
There is a fundamental ambiguity regarding the status and value of edge cases. Each edge
case can be used as test of the system as designed — to test, reform and/or redefine the
competence envelope of a specific system. When testing for robustness, weaknesses
identified in walkthroughs identify specific design changes so that the engineered system can
perform successfully should it encounter the specific wrinkles of each scenario. The typical
test/re-design cycle for ‘use cases’ in human-machine system and software development is an
example. Test using edge cases in this way enhance (and more clearly define) the
competence envelope of the engineered system as it is planned to operate by making the
system more robust to the specific factors built into each specific challenge case.
This is a typical and useful way to use challenge cases. However, robustness and resilience
are different, and the standard approach only helps build robustness (Doyle et al., 2005;
Woods, 2006; Alderson and Doyle, 2010; Woods, 2015). This is because a complexity
penalty arises even when systems are designed to be robust as well more optimal on some
criteria, and this is a fundamental finding for all complex adaptive systems. Something extra
is needed to use scenarios to test the integrated system’s resilience/brittleness.
Complexity penalties arise from changes to increase optimality and robustness of a system
since these changes also lead to an emergent susceptibility to sudden performance collapses
and failures. The higher performance on some criteria comes at the cost of increased
brittleness of the system (Alderson and Doyle, 2010; Woods, 2015). The effort invested to
improve fitness leads to systems “ which are robust to perturbations they were designed to
handle, yet fragile to unexpected perturbations and design flaws” (Carlson and Doyle, 2000,
p. 2529). The network will look more and more fit to its environment on some criteria, while,
the same processes produce severe brittleness when events occur that challenge the design
envelope. Ormerod and Colbaugh (2006) summarize results: “as the connectivity of a
network increases, we observe an increase in the average fitness of the system. But at the
same time, there is an increase in the proportion of failure/extinction events which are
extremely large.” Simply put, the pursuit of optimality increases brittleness (Woods, 2006).
This is a fundamental trade-off (Woods, 2018). But some systems demonstrate the ability to
continue to adapt to changing environments, stakeholders, demands, contexts, and
constraints. These systems are ‘poised to adapt’ (Woods, 2015; Cook and Woods, 2016).
This means each scenario provides two tests in parallel:
• how/where to add robustness?
!how/where to add resiliency?
For the former, the walkthrough and analysis explores how engineers can improve/assure
system performance for the specific difficulties in that specific scenario. In this way
robustness is assessed and the competence envelope is adjusted. For the latter, the analysis
assumes (1) the scenario instantiates general challenges to competence believed to have
been engineered into the system, and (2) the next challenge case the system will face will be
different than the scenario used in the test.
Fundamental to the usefulness of this technique is the development of scenarios which stress
the system in particular ways — anomalies that arise at the edges of the competence
envelope. Specifically, the instantiation of disruptions which create the types of loads and
challenges to the system which test the system’s ability to anticipate crunches or bottlenecks
ahead and examine it’s ability to synchronize activities across roles as tempo varies or
increases (Woods and Branlat, 2011). In this way walkthroughs the set of scenarios reveal
general capabilities needed for resilient performance across a range of potential surprise
events which challenge the competence envelope (Dekker and Woods, 1999b).
The key difference between testing for robustness and testing for resilience is the former
improves the system for the set of specific situations used in the tests, while the latter uses the
general challenges instantiated in specific scenarios to test how the system will respond to
unanticipated situations that stress performance (Woods, 2015).
The unanticipated stress or surprise situations arise because of the limits and trade-offs made
in defining or developing the basic competencies of the UA system. The RTCA committees
on DAA and C2 work hard to cope with engineering trade-offs on cost, weight, performance,
as well as a multitude of other factors (e.g., antenna characteristics). All the while, such teams
are making judgements about future, envisioned operations when the anticipated value of
UAs will spawn new kinds of missions and vehicles with diverse performance characteristics
and constraints. The future viable missions, vehicles, and emergent system which will emerge
are all inherently underspecified and subject to change as new opportunities develop and
others directions are discarded (Woods and Dekker, 2000). The limits, assumptions, and
trade-offs made to specify US capabilities in MOPs are just a particular instance of the
fundamental constraints on all adaptive and layered networks in complex environments
(Alderson and Doyle, 2010; Doyle and Csete, 2011; Woods, 2015).
The constraints identified in work on complexity penalties highlight there are always inherent
limits in these systems that make anomalies, exceptions, surprises which challenge the
competence envelope inevitable. In addition the research shows that design perspectives
will always overestimate the competence envelope and be overconfident that surprises have
been minimized (Woods, 2018).
Results from Walkthroughs
A wide range of challenge scenarios were developed and used in the cognitive
walkthroughs. The results across all of the scenarios and walkthroughs were very consistent.
Finding 1. The combination of a triggering event that disrupted structured traffic flows plus
degraded or lost SAT-COM for the UA involved in the disruption led to a significant increase
in uncertainty about UA intent and future behavior.
The increase in uncertainty stood out in the discussions triggered by the scenarios. This
occurred across all of the cases in Table 2. In many walkthroughs, the participants
highlighted this and felt it unnecessary to linger on all of the details in each scenario or set of
variations on a scenario. For them, the driving factor in how the scenario would play out and
be handled all revolved around handling the uncertainty about UA intent and future behavior
created by the degraded communications. The key questions about autonomous system
behavior first identified by Wiener in 1989 with respect to cockpit automation arose
repeatedly: what is it doing now? and even more critically, what is it going to do next? The
limits on their ability to know and anticipate what the UA will do next in a disrupted
environment was the key factor noted over and over.
Finding 2. The uncertainty regarding future UA behavior led to a consistent projection about
how ATC would respond across the scenario set. To compensate for the uncertainty about
UA behavior — inability or limited ability to anticipate what the UA will do next — the
participants projected that ATC would adapt by moving the other manned traffic further away
from UA with degraded SAT-COM. Uncertainty led to tactical maneuvering of other manned
aircraft to increase separation away from the UA.
The projected response was based on ATC’s fundamental responsibility to maintain
separation. As many commented, “for safety, it is easier to maneuver everything else.” They
would expand the dynamic ‘bubble’ of space around the UA, keeping other traffic further
away. In other words, the respondents felt they could not delegate the responsibility for
maintaining separation to the on-board DAA system of sensors and software (trust and
responsibility issues in part; Hoffman et al., 2009; Murphy and Woods, 2009). A variety of
consequences emerged from the walkthroughs: cascading effects could produce anomalies
in adjacent sectors, compromise the efficiency of flight profiles of sector traffic, risk of fast
onset of sector instability, how increased uncertainty removes degrees of freedom for tactical
This finding is similar to findings from other studies of other deployments of increasingly
autonomous capabilities — when automation is opaque and difficult to direct, responsible
human roles work around or outside of the automated system to meet key goals (Wiener
1989; Sarter et al., 1997). In this case, the UA as an actor in shared airspace becomes more
opaque and difficult to direct relative to the larger traffic context when the SAT-COM link is
degraded. The result is that adaptations to manage uncertainty will lead to reduced sector
capacity. The tactical actions to keep other traffic away from UA with degraded C2
capabilities would also undercut the efforts to optimize network capacity and efficiency, most
directly by reducing throughput. How much of a reduction of throughput would result
depends on analyses that assess the size/frequency of the throughput loss to estimate traffic
capacity and financial consequences (factor 3, Risk of Consequences Associated with
Brittleness, in a full Resiliency Trade Space analysis).
Finding 3. There was a high potential for cascading effects in the scenarios which led to new
difficulties for air traffic management for all of the aircraft within ATC’s scope of responsibility.
The difficulties described or highlighted, depending on specific scenarios, included:
(a) increase in the tempo of events, challenging the ability of controllers to
keep pace with the changing situation,
(b) sudden increases in workload for specific ATC positions (workload spikes
risk undermining performance),
(c) handoffs and cross sector transitions become more difficult.
The walkthroughs revealed a number of difficulties that would challenge ATC performance.
The difficulties increase the risk of poor ATC performance as tempo management, situation
awareness, managing workload peaks, managing handoffs & transitions all become more
challenging activities.
Overall, the cognitive walkthroughs portion of the Resiliency Analysis, revealed how roles will
adapt to anomalies and challenges to the competence envelope being developed for UA
operations in controlled airspace. In terms of factors relevant to resilient performance, the
results identified:
(1) The key driver is uncertainty about what the UA will do next when SAT-COM
are degraded or lost,
(2) ATC will adapt prior to DAA action and not rely on DAA based self-
separation in these situations,
(3) Situations of disrupted traffic plus UA with degraded SAT-COM will
introduce significant cognitive and coordination difficulties for responsible air
traffic controllers.
Supporting Resiliency
The above is only one part of Resiliency Analysis. The second phase uses information
gathered in the first phase to identify key factors needed to enhance systemic resiliency. In
this way a Resiliency Trade Space Study should identify ways to improve or sustain resilient
performance in the future. To that end, two capabilities are key — anticipation to keep pace
with potentially cascading effects of disruptions and coordination across roles and echelons
(Woods and Branlat, 2011; Woods, 2015). For this portion of the study the key question was:
How to maintain synchronization in coordinated activity when standard channels for
information sharing are no longer available?
To synchronize, first, all potentially interdependent roles have to know or signal to others that
standard channels for information sharing are no longer available. The difficulty is the degree
to which the C2 link is required to in order to share information about the health of the C2
link. It is not at all obvious how the different roles will share timely information or alerts
regarding SAT-COM status across a wide range of contexts. How does the UA itself know
about degraded/lost SAT-COM link? How does the ground pilot (remote pilot in command or
RPIC) know of the degraded/lost SAT-COM link? How does ATC know one aircraft has
degraded/lost SAT-COM link? How is that information shared with other relevant players?
The first design direction to build resiliency is to find ways to address the recognition
problem that standard channels for information sharing are no longer available or functioning
As the different players understand the UA has degraded SAT-COM, there is a transition to
alternate means to share anticipatory information (information about intent and what the UA
will do next) and to new ways to support coordination?
The second design direction to build resiliency is to support the transition to alternate means
to share anticipatory information about what the UA will do next in order for other roles to
synchronize and coordinate around the UA should disruptions arise when communication is
Central to supporting resilient performance in the event of a degraded C2 link will be
providing a means to communicate with high semantic content but using only means with
low bandwidth/bit (e.g., low character-count). The requirements for effective coordination
starts with anticipatory information that signals intent — what the UA will do or is likely to do
next. For example, what diversion plan will the UA with lost C2 link follow and how long after
the link is lost? This information can be provided in a single bit or 2 if the receiving roles also
share or have access to knowledge about the contingency plans the vehicle will use. Data on
aircraft speed, direction, and altitude can then be interpreted relative to the plan the vehicle
is following — is the UA operating on plan or off plan? It is a well documented finding that
highly trained and adaptive human teams signal intent with a few highly coded signals — high
semantic content but low bit communication — based on shared experience, training and
knowledge, and that this skill is essential to highly coordinated and adaptive responses when
anomalies occur.
Anticipatory information can be signaled in many forms and through different channels. This
means there are many design possibilities that could provide signals with high semantic
content about intent despite loss or degradation of SAT-COM link. The requirements for
anticipatory information include but are not limited to:
• low bit signals with high semantic content about future plans and behavior;
• diverse channels for signaling intent to other roles when the standard
communication channels are compromised;
!shared models of contingency plans / behavior following disruptions across all of
the interdependent roles;
• greater on-board machine intelligence: (a) about how to act as a cooperative agent
in shared airspace in different situations, and (b) context-aware contingencies;
• ability to provide ATC updates about context-aware contingencies regularly
throughout a flight and prior to disrupting events.
During this study, a specific example of applying the above criteria occurred. In a briefing of
preliminary results, the first two of the above requirements were presented — low bit signals
with high semantic content about future plans and behavior and diverse channels for
signaling intent to other roles when the standard communication channels are compromised.
Almost immediately, a modification to existing MOPs was identified that required no new or
special equipment. Existing ADS-B technology can be used to provide a diverse
communication in today’s system when there is degraded or lost communication. The
restricted character count of ADS-B messages is no obstacle when the communication should
convey a large volume of semantic content via only a few bits. Using the ADS-B channel does
require the development and use of a specific ADS-B message set which would allow the UA
to signal its intent and future actions to other roles across the integrated system even in the
event of a lost C2 link or general communications equipment failure. Developing this
mechanism provides a way for the UA to provide anticipatory information and act as a
cooperative member of shared airspace as the disrupting event is managed and traffic
returns to structured flows.
The value of investing in engineering the capability for reliable high semantic signaling is the
boost in resilient performance in future difficult situations that are difficult to anticipate
precisely. The benefits of this investment apply to the integrated system for managing
unmanned aerial systems across a wide spectrum of conditions that go beyond the
combination investigated here, especially as the percentage of UA operating in shared
airspace grow as new missions and vehicles come into use.
Summary, Limits, and Next Steps
This study results reveal how roles are likely to adapt to anomalies and events that challenge
the competence envelope being developed for UA operations in controlled airspace. The
study looked at a specific combination — a disruption to structured traffic flow that included
degraded or lost C2 link. In this way the study began to reveal multiple unintended
consequences that can follow deployment of new capabilities. The study identifies directions
that can support resilient performance when disruptions occur and standard communication
channels are compromised. The criteria for resilient performance can be implemented
through a wide range of design possibilities, some of which include enhanced machine
intelligence. The study results were communicated via briefing to SC-228/other stakeholders
(July 10 – 14, 2017) and through this report which includes the set of edge scenarios
The study was tasked to assess the possibility that increasing the on-board capacity of a UA
for self-control and automated decision-making can be offered as a compensatory
mechanism for less-reliable C2 links, thereby permitting the operation of a UA in situations
where communications link consistency is unpredictable. The study answer is this simple
trade of autonomy and communication reliability is too simple and insufficient to handle
challenging situations when one examines the integrated system. Complexity penalties
identified include sudden workload peaks and difficulty keeping pace with the tempo of
events when disruptions that involve the UA occur. The ATC system is likely to adapt to
degraded UA C2 link by increasing separation with other aircraft within their scope of
responsibility without relying on the DAA self-separation. This adaptive response will reduce
sector throughput and reduce performance relative to other relevant criteria. The UA with
degraded or lost C2 will not be able to act as a cooperative participant in a shared airspace.
The study also shows how the complexity penalties can be offset by supporting the ability for
resilient performance of the integrated system that includes UAs. There are mechanisms that
can be used to maintain synchronization in coordinated activity when standard channels for
information sharing are no longer available. The study provides several requirements for
developing such mechanisms.
The set of edge scenarios with anomalies developed for this study can be used in other
studies to assess the brittleness of proposed autonomous capabilities and to evaluate the
benefits of design for resiliency.
The study provides an initial method for carrying out resiliency trade space studies. The
method is an innovative approach based on research results from resilience engineering.
Given the purpose and timeline to provide results, the initial method was carried out rapidly
at the cost of lower rigor. The good news is that the method can provide quick results cost
effectively as in this case. These results can then be followed up in different ways that
increase rigor and explore the effects of alternative design possibilities.
The method as used in this study falls short of another critical need. Resiliency trade space
methods need to be able to assess and estimate performance/resource trade-offs. How much
does an investment needed to provide a capability result in enhanced resilient performance?
A follow on project to extend the current method to estimate performance/resource trade-
offs is important but would require further work to identify several potential design
alternatives to use in order to build quantitative estimates.
Multiple organizations have recommended research on methods to test the integrated
system effects of deployments of increasingly autonomous capabilities as a high national
priority. The current study provides a candidate method that can be further developed.
Follow up work using DAA capabilities could extend the current method to provide a more
quantitative analysis of impact on resilient performance relative to resource investments
needed across several potential design possibilities.
Alderson, David L. and John Doyle. “Contrasting Views of Complexity and Their Implications
For Network-Centric Infrastructures.!IEEE Transactions on Systems, Man, and Cybernetics -
Part A: Systems and Humans!40 (2010): 839-852.
Carlson, J. M., and John Doyle. "Highly Optimized Tolerance: Robustness and Design in
Complex Systems."!Physical Review Letters!84, no. 11 (2000): 2529-2532.
Doyle, J. C., D. L. Alderson, L. Li, S. Low, M. Roughan, S. Shalunov, R. Tanaka, and W. Willinger.
"The "robust Yet Fragile" Nature of the Internet."!Proceedings of the National Academy of
Sciences102, no. 41 (2005): 14497-4502. doi:10.1073/pnas.0501426102.
Doyle, J. C. and Csete, M. E. (2011). Architecture, constraints, and behavior. Proceedings of
the National Academy of Science USA 2011; 108 (Suppl. 3):S15624–30.
Dekker, S. W. A., and D. D. Woods. "To Intervene or Not to Intervene: The Dilemma of
Management by Exception."!Cognition, Technology & Work1, no. 2 (1999a): 86-96. doi:
Dekker, S. W. A. and Woods, D.D. (1999b). Extracting Data from the Future: Assessment and
Certification of Envisioned Systems. In S. Dekker and E. Hollnagel (Eds.), Coping with
Computers in the Cockpit, Ashgate, p. 131-143.
Fern, Lisa, R. Conrad Rorie, Jessica Pack, Jay Shively, and Mark Draper. "An Evaluation of
Detect and Avoid (DAA) Displays for Unmanned Aircraft Systems: The Effect of
Information Level and Display Location on Pilot Performance."!15th AIAA Aviation
Technology, Integration, and Operations Conference, 2015. doi:10.2514/6.2015-3327.
Fern, Lisa "A Cognitive Systems Engineering Approach to Developing Human Machine
Interface Requirements for New Technologies." Electronic Thesis or Dissertation. Ohio
State University, 2016.
Hoffman, R. R., Lee, J. D., Woods, D. D., Shadbolt, N., Miller, J. and Bradshaw, J. (2009). The
Dynamics of Trust in Cyberdomains. IEEE Intelligent Systems, 24(6), November/
December, p. 5-11.
Hoffman, Robert R., and David D. Woods. "Beyond Simons Slice: Five Fundamental Trade-Offs
That Bound the Performance of Macrocognitive Work Systems."!IEEE Intelligent
Systems26, no. 6 (2011): 67-71. doi:10.1109/mis.2011.97.
Hollnagel, Erik, Woods, D.D. and Leveson, N., Eds.!Resilience Engineering: Concepts and
Precepts. Aldershot: Ashgate, 2006.
Johnson, Matthew, Jeffrey M. Bradshaw, Robert R. Hoffman, Paul J. Feltovich, and David D.
Woods. "Seven Cardinal Virtues of Human-Machine Teamwork: Examples from the DARPA
Robotic Challenge."!IEEE Intelligent Systems, 29, no. 6 (2014): 74-80. doi:10.1109/mis.
Lewis, C., & Wharton, C. (1997). Cognitive walkthroughs. In M. Helander, T. K. Landauer, & P.
Prabhu (Eds.), Handbook of human-computer interaction (2nd ed., pp. 717-732).
Amsterdam: Elsevier Science.
Polson, P. G., Lewis, C., Rieman, J. & Wharton, C. (1992) Cognitive Walkthroughs: A Method for
Theory-Based Evaluation of User Interfaces. International Journal of Man-Machine Studies,
36 (5), 741-773.
Minimum Operational Performance Standards (MOPS) for Detect and Avoid (DAA) Systems
(RTCA Paper No. 261-15/PMC-1400) Version 3.9 Prepared by: SC-228. December 09,
Murphy, Robin, and David D. Woods. "Beyond Asimov: The Three Laws of Responsible
Robotics."!IEEE Intelligent Systems24, no. 4 (2009): 14-20. doi:10.1109/mis.2009.69.
Murphy R. R. and Shields, J.. The Role of Autonomy in DoD Systems, Defense Science Board
Task Force Report. Office of the Secretary of Defense, July 2012.
National Research Council. !Autonomy Research for Civil Aviation: Toward a New Era of
Flight.!Washington DC:!!National Academies Press, (2014),!
The Networking and Information Technology Research and Development Program (NITRD),
Workshop Report on Complex Engineered Networks, 2012 Washington, DC.
The Networking and Information Technology Research and Development Program (NITRD).
Grand Challenges: Science, Engineering, and Societal Advances Requiring Networked
Information Technology Research and Development. November, 2006 Washington, DC.
Ormerod, Paul. "Cascades of Failure and Extinction in Dynamically Evolving Complex
Systems."!Noise and Stochastics in Complex Systems and Finance, 2007. doi:
Perry, Shawna J., Robert L. Wears, and Richard I. Cook. "The Role of Automation in Complex
System Failures."!Journal of Patient Safety1, no. 1 (2005): 56-61. doi:
Roth, E.M., K. Bennett, and D.D. Woods. "Human Interaction with an ‘intelligent’
Machine."!International Journal of Man-Machine Studies.27 (1987): 479-525.
Rorie, R. Conrad, and Lisa Fern. "The Impact of Integrated Maneuver Guidance Information on
UAS Pilots Performing the Detect and Avoid Task."!Proceedings of the Human Factors and
Ergonomics Society Annual Meeting59, no. 1 (2015): 55-59. doi:
Sarter, N., Woods, D.D. and Billings, C. Automation Surprises. In G. Salvendy, editor,
Handbook of Human Factors/Ergonomics, second edition, Wiley, New York, pp.
1926-1943, 1997.
Spero, Eric, Miochael P. Avera, Pierre E. Valdez, and Simon R. Goerger. "Tradespace
Exploration for the Engineering of Resilient Systems."!Procedia Computer Science28
(2014): 591-600.
Smith, P.j., C.e. Mccoy, and C. Layton. "Brittleness in the Design of Cooperative Problem-
solving Systems: The Effects on User Performance."!IEEE Transactions on Systems, Man,
and Cybernetics - Part A: Systems and Humans27, no. 3 (1997): 360-71. doi:
Smith, P.J. Making Brittle Technologies Useful. In Smith, Philip J., and Robert R.
Hoffman.!Cognitive Systems Engineering: The Future for a Changing World. Boca Raton,
FL: CRC Press, Taylor & Francis Group, 2018.
Wiener, E. L. Human factors of advanced technology ("glass cockpit") transport aircraft. NASA
Contractor Rep. No. 177528,1989. Moffett Field, CA. NASA-Ames Research Center.
Woods, D.D. "Essential Characteristics of Resilience for Organizations." In!Resilience
Engineering: Concepts and Precepts, p. 21-34. Aldershot, UK: Ashgate, 2006.
Woods, D. D. Four Concepts of Resilience and the Implications for Resilience Engineering.
Reliability Engineering and Systems Safety, 141, 5-9, 2015. published online: 3 APR 2015 |
Woods, David D. "The Risks of Autonomy: Doyle’s Catch ."!Journal of Cognitive Engineering
and Decision Making10, no. 2 (2016): 131-33. doi:10.1177/1555343416653562.
Woods, D. D. The Theory of Graceful Extensibility. Environment Systems and Decisions,
(2018), in press.
Woods, David, and Sidney Dekker. "Anticipating the Effects of Technological Change: A New
Era of Dynamics for Human Factors."!Theoretical Issues in Ergonomics Science1, no. 3
(2000): 272-82. doi:10.1080/14639220110037452.
U.S. Department of Transportation. Federal Aviation Administration. Integration of Civil
Unmanned Aircraft Systems (UAS) in the National Airspace System (NAS) Roadmap.
Washington, DC: 2013.
Table 1. Resiliency Trade Space Analysis
Engineering Competence
Potential for Resiliency
Consequences of Breakdown
Specifies the competence envelope of
the device or system as engineered
Explores how the larger distributed
system adapt when challenges and
disruptions occur
Predicts likelihood and potential
results of poor performance and
Parameters, limits, operational
standards, and assumptions in the
design of engineered systems
Behavior in edge events, particularly
those with cascading eects requiring
coordination over interdependent
roles and levels
Probabalistic / statistical
Calibrated robustness against known
(well-modeled) challenges
Adaptive capacity when faced with
Visual Analogy
© 2017 David D. Woods and E. Asher Balkin. All rights reserved.
Table 2. Edge Cases Used in Cognitive Walkthroughs
Double Decker
3 Variations
One aircraft departs and begins to climb from one runway, while second aircraft
preforms a missed approach and go-around on a near-by runway. This produces a
condition wherein two aircraft are in a close proximity while both are in critical
phases of flight. ATC issues new heading directives to both aircraft with.
Coordination with other craft in the local airspace must be maintained to assure
that all well-clear volumes are maintained.
Wake Turbulence and Cascade
5 Variations
Having completing its low-altitude survey mission, an aircraft climbs to a higher
altitude. While climbing, the aircraft encounters extreme turbulence and is unable
to maintain level flight it proper orientation. While losing altitude, the craft violates
the well-clear volumes of other craft in the vicinity.
Overtake on Climb
5 Variations
Two aircraft depart in rapid succession from the same runway. The leading aircraft is
traveling more slowly than anticipated and as a result, the following aircraft begins
to catch-up; separation is lost and well-clear is breached. Variations include ATC
recognizing the potential for harm and issuing new headings/altitudes, TCAS
activation, and ATC issues new instructions, leading to significant disruptions to the
structure of terminal airspace.
Push from the Rear
2 Variations
Multiple aircraft are in line on final approach an aircraft in second position travels
faster than the craft in front of it (first position). The difference in travel speed
causes a breach of wake-turbulence separation between the aircraft in first and
second positions. ATC issues a go-around order to the aircraft in the first position
(leading craft) causing the first-position craft to accelerate and gain altitude, thus
regaining separation.
Push from the Rear
2 Variations
Multiple aircraft are in line on final approach an aircraft in second position travels
faster than the craft in front of it (first position). The difference in travel speed
causes a breach of wake-turbulence separation between the aircraft in first and
second positions. ATC issues a go-around order to the aircraft in the first position
(leading craft) causing the first-position craft to accelerate and gain altitude, thus
regaining separation.
Intermittent Loss of Data-Link While Crossing Sector
3 Variations
Frequent, but unpredictable interruptions in RPIC/UAS connectivity lead RPICs to
conclude they have unreliable control over the UAS. Variations include a desire to
change the mission/flight plan and attempting to do so during a period brief
period of connectivity (with success/ without success/with limited success and
dangerous consequences), and loss of data link means no voice comms between
Loss of UAS/ATC Communications
3 Variations
GCS/UAS data link remains operational, however, UAS/ATC voice link is non-
functional. Variations include UAS maintaining its pre-shared flight plan thereby
limiting the need for ATC to RPIC via UAS communications, ATC concerned that
lack of voice comms indicates other problems leading to larger sector clearance
and propagating disruption, or UAS signal kinetic intent to ATC limiting ATC
concerns over its capacity to cooperate.
Transit of Structured Airspace Due to In-Flight Change of
Priorities of Operator Organization
2 Variations
Mid-mission, the UAS is re-tasked and alters course. ATC attempts to clear a path
for UAS travel to its new mission site causing a disruption to well-structured
airspace. Variations include the cascading effects of the requisite reorganization of
manned aircraft on the sector and the clearance of large sections of airspace as a
result of the failure of the RPIC to properly alert ATC to the change, and the
resultant confusion.
© 2017 David D. Woods and E. Asher Balkin. All rights reserved.
Loss of GCS/UAS Comm-Link Post-Tactical Direction
3 Variations
RPIC issues a series of direct commands which take the UAS off its flight plan.
Without a new/updated flight plan, at least three variations could be considered:
UAS continues on last given heading, UAS recognizes lost-link condition and
preforms lost link procedure, UAS continues following last given instruction which
included a descend order without a terminal altitude - UAS continues descending
until terrain avoidance takes over preventing collision - UAS behavior after
terrestrial collision avoidance is unpredictable.
Loss of GCS/UAS Data Link Mid-GCS Hand-Off
3 Variations
The extended endurance of the UAS leads to the need for the transfer of
operational control authority from one GCS (ground control station) to another. In
the transition, the link to the UAS is lost. Variations include neither GCS being able
to recover the link causing the UAS to abort its mission and preform a lost-link
procedure, continuing on its mission plan, and a third version wherein each GCS,
attempting to reconnect, issues conflicting directions to the UAS leading to
unpredictable behavior.
ResearchGate has not been able to resolve any citations for this publication.
Full-text available
Tradespace exploration supports the Systems Engineering Technical Management Process of Decision Analysis by identifying compromises, revealing opportunities, and communicating the impacts of decisions across a system's development lifecycle. Critical program decisions are made based on the outcomes of trades; trades being performed with multiple types and quantities of data coming out of tools and methods employing qualitative and quantitative analyses. Tradespace exploration for Engineered Resilient Systems (ERS) is envisioned to coalesce pertinent information tuned to specific decision makers, at the appropriate time, presenting a holistic view of decision impacts on required system capabilities. This study provides an ERS view of tradespace exploration, which reveals that having a valid set of attributes, and an understanding of how a cross-section of tools can satisfy them, is insufficient – what is needed is a deeper understanding of how these tools are used and, more importantly, how they can be used when performing tradespace exploration in support of the Decision Analysis Process. Gaining this understanding will enable users to better assess if they possess the appropriate tradespace exploration tools. A holistic view of 81 candidate tradespace exploration tools is provided. This study seeks to address a fundamental aspect of tradespace exploration by assembling a “best common practice” process for their requirements, identifying a set of attributes that defines an ideal tradespace exploration tool, and surveying existing tools that satisfy these attributes. In this way, a set of tools can be selected to enable the ERS tradespace vision on a particular project. A paradigm shift towards common tradespace methods, tools, cost models, and steps is emphasized.
Full-text available
This article plays counterpoint to our previous discussions of the “seven deadly myths” of autonomous systems. The seven deadly myths are common design misconceptions to be acknowledged and avoided for the ills they breed. Here, we present seven design principles to be understood and embraced for the virtues they engender. The cardinal virtues of classical antiquity that were adopted in Christian tradition included justice, prudence, temperance, and fortitude (courage). As we’ll show in this essay, in effective human-machine teamwork we can also see virtues at play—namely clarity, humility, resilience, beneficence (helpfulness), cohesiveness, integrity, and thrift. As we unfold the principles that enable these virtues to emerge, it will become clear that fully integrating them into the design of intelligent systems requires the participation of a broad range of stakeholders who aren’t always included in such discussions, including workers, engineers, operators, and strategic visionaries developing research roadmaps. The principles aren’t merely for the consumption of specialists in human factors or ergonomics. We illustrate these principles and their resultant virtues by drawing on lessons learned in the US Defense Advanced Research Projects Agency (DARPA) Robotics Challenge (DRC).
The cognitive walkthrough (CW) is a method for evaluating user interfaces by analyzing the mental processes required of users. To perform a CW an analyst chooses a specific task from the suite of tasks that the interface is intended to support and determines one or more correct sequences of actions for that task. The cognitive walkthrough is used to identify problems with a user interface and to suggest reasons for these problems. The chapter begins by presenting a brief overview of the cognitive walkthrough and locates it in the space of interface evaluation methods. Following this, the chapter discusses cognitive walkthrough method in more detail, with practical instructions for its use. The chapter then describes related methods that have been developed from the cognitive walkthrough idea. It also reviews published accounts of experience with these methods.
Nine active unmanned aircraft system (UAS) pilots were tasked with flying a simulated UAS in civil airspace and instructed to maintain safe separation (i.e., well clear) from surrounding traffic. Pilots’ task of maintaining separation (referred to here as ‘Detect-and-Avoid’, or DAA) was facilitated by four different traffic displays, each differing in the level of maneuver guidance they presented to the pilot. Pilots were found to spend the least amount of time implementing a maneuver when provided with an integrated form of directive guidance, but were found to subjectively prefer a maneuver guidance tool that allowed them to test self-derived maneuver options and then receive feedback as to that option’s predicted safety level. The results of this study are related back to previous research and to the task of identifying the minimum information requirements for UAS pilots performing the DAA task. Limitations and future research are also discussed.