Content uploaded by David E. Wilkins
Author content
All content in this area was uploaded by David E. Wilkins on Sep 10, 2014
Content may be subject to copyright.
Integrating Planning and Reactive Control*
David E. Wilkins Karen L. Myers
Abstract
Agents situated in dynamic and unpredictable envi-
ronments require several capabilities, including syn-
thesizing and executing plans while continuing to be
responsive to the world. The Cypress system is a
domain-independent framework for defining persistent
agents with this full range of behavior. Cypress has
been used for several applications, including military
operations and fault diagnosis on the Space Shuttle.
Introduction
Our research is developing persistent agents that can
achieve complex tasks in dynamic and uncertain envi-
ronments. We refer to such agents as taskable, reactive
agents. An agent of this type requires a number of
capabilities. The ability to execute complex tasks ne-
cessitates the use of strategic plans for accomplishing
tasks; hence, the agent must be able to synthesize new
plans at run time. The dynamic nature of the envi-
ronment requires that the agent be able to deal with
unpredictable changes in its world. As such, agents
must be able to react to unanticipated events by taking
appropriate actions in a timely manner, while contin-
uing activities that support current goals. The unpre-
dictability of the world could lead to failure of plans
generated for individual tasks. Agents must have the
ability to recover from failures by adapting their activ-
ities to the new situation, or replanning if the world
changes sufficiently. Finally, the agent should be able
to perform in the face of uncertainty.
Many domains of interest require problem-solving
agents with the capabilities described above. Military
0. This paper will appear in Proceedings of the Third In-
ternational Symposium on Artificial Intelligence, Robotics,
and Automation for Space, Jet Propulsion Laboratory,
Pasadena, CA, October 1994.
SRI International
Artificial Intelligence Center
333 Ravenswood Ave.
Menlo Park, CA 94025
phone: 415-859-2057 fax: 415-859-3735
{myers, wilkins}@ai.sri.com
and space operations provide good examples. Cer-
tainly one would not engage in an undertaking such as
Desert Storm or repairing the Hubble Space Telescope
without first formulating a strategic mission plan. Re-
active response and failure recovery are necessary be-
cause unexpected equipment failures, weather condi-
tions, enemy actions, and other events may require
changes to the overall strategic plan.
The Cypress system, described here, provides a
framework for creating taskable, reactive agents. Sev-
eral features distinguish our approach: (1) the gen-
eration and execution of complex plans with parallel
actions, (2) the integration of goal-driven and event-
driven activities during execution, (3) the use of ev-
idential reasoning for dealing with uncertainty, and
(4) the use of replanning to handle run-time execution
problems.
Our model for a taskable, reactive agent has two
main intelligent components, an executor and a plan-
ner. The two components share a library of possible
actions that the system can take. The library en-
compasses a full range of action representations, in-
cluding plans, planning operators, and executable pro-
cedures such as predefined standard operating proce-
dures (SOPs). These three classes of actions span mul-
tiple levels of abstraction.
The executor is always active, constantly monitor-
ing the world for goals to be achieved or events that
require immediate action. In accord with its current
beliefs and goals, the executor takes actions in resp.onse
to these goals and events. Appropriate responses in-
clude applying SOPs stored in the action library, in-
voking the planner to produce a new plan for achieving
a goal, or requesting that the planner modify a pre-
vious plan during execution. The planner should be
capable of synthesizing sophisticated action sequences
161
From: AAAI Technical Report FS-94-01. Compilation copyright © 1994, AAAI (www.aaai.org). All rights reserved.
that include parallel actions, conditional actions, and
resource assignments. The planner plans only to a cer-
tain level of detail, with the executor taking that plan
and expanding it at run time by applying appropriate
library actions at lower levels of abstraction.
Cypress
Cypress constitutes a framework in which to define
taskable, reactive agents based on the above model.
The architecture of Cypress is depicted in Figure 1.
The motivation for Cypress was to build a heuris-
tically adequate system for use in practical applica-
tions. To this end, Cypress relies on mature, power-
ful planning and execution technologies, namely the
SIPE-2 generative planner (Wilkins et al. to appear)
and the PrtS-CL reactive execution system (Wilkins et
al. to appear). We have applied Cypress to a number
of demanding problems, including real-time tracking,
fault diagnosis on the Space Shuttle, production-line
scheduling, and military operations (Wilkins et al. to
appear).
PRS-CL is a framework for constructing persistent,
real-time controllers that perform complex tasks in dy-
namic environments while responding in timely fashion
to unexpected events. It has been used to monitor the
Reaction Control System (RCS) of the Space Shuttle
(Wilkins et al. to appear). This application illustrates
the use of multiple agents, and has been used to detect
and recover from most of the possible malfunctions of
the RCS, including sensor faults, leaking components,
and regulator and jet failures. The system demon-
strated guaranteed response, support for asynchronous
inputs, interrupt handling, continuous operation, and
handling of noisy data.
SIPE-2 is a partial-order AI planning system that
supports planning at multiple levels of abstraction. It
has the properties required by our agent model, includ-
ing the ability to generate plans that include parallel
actions, conditional actions, resource assignments, and
the ability to modify previously generated plans. In
contrast to most AI planning research, heuristic ade-
quacy has been a primary design goal of SIPE-2.
PRS-CL and SIPE-2 employ their own internal rep-
resentations for plans and actions for efficiency. For
this reason, Cypress supports the use of an interlin-
gua called the ACT formalism (Wilkins et al. to ap-
pear) that enables these two systems to share informa-
tion. ACT provides a language for specifying actions
and plans for both planners and executors. Cypress
includes translators that can automatically map Acts
onto SIPE-2 and PRS-CL structures, and one that can
map SIPE-2 operators and plans into Acts. Using the
ACT interlingua, PRS-CL can execute plans produced
by SIPE-2 and can invoke SIPE-2 in situations where
run-time replanning is required. The ACT-Editor sub-
system supports the graphical creation and display of
Acts. Gister-CL (Wilkins et al. to appear) implements
a suite of evidential reasoning techniques that can be
used to analyze uncertain information about the world
and possible actions. For example, Gister-CL can be
used to reason about uncertain information in order to
choose among candidate Acts in either the planner or
executor.
In contrast to many other agent architectures, plan-
ning and execution operate asynchronously in Cypress,
in loosely coupled fashion. This approach makes it
possible for the two systems to run in parallel, even on
different machines, without interfering with the actions
of each other. In particular, PP~S-CL remains respon-
sive to its environment during plan synthesis. While
the subsystems of Cypress can function independently,
Cypress is used most advantageously as an integrated
framework that supports a wide range of planning and
execution activities.
Applications
An example from military operations planning
(Wilkins & Desimone 1994) is currently the only imple-
mented application that illustrates the use of all sub-
systems of Cypress, but it is similar to a space mission.
The most advantageous use of Cypress in space appli-
cations will most likely be in situations that do not
directly involve humans. A planetary rover will cer-
tainly need the combination of plan-directed behavior
with reactive response to the environment provided by
Cypress, and can build directly on our use of Cypress
modules to control an indoor mobile robot. Other ap-
propriate space applications include control of a satel-
lite or probe, controlling experiments on the shuttle
or space station, and providing an assistant to astro-
nauts to handle routine malfunctions and alert them of
important events that affect the overall mission plan.
The military application domain knowledge includes
approximately 100 plan operators, 500 objects with 15
to 20 properties per object, and 2200 initial predicate
instances. Plans range in size from several dozen to
162
ACT->Sipe, Sipe->ACT
translators
SIPE-2
-generative planning
-replanning
operator selection
object selection
situation assessment
ACT-Editor /
-common interface
-customize graphics
replanning ,,
Gister-CL
-reasoning about uncertainty
PRS-CL
-reactive execution
-replanning overseer
~|~lra:J~l lu r~ti:~ection
I
situation assessment
Figure 1: The Architecture o~’ Cypress
200 actions, including many that are to be executed in
parallel (Wilkins & Desimone 1994).
The scenario begins with a goal request for deter-
ring several military threats. SIPE-2 uses a set of
Acts previously input to the system to generate a plan
with many threads of parallel activities. During the
planning process, Gister-CL assists SIPE-2 in choos-
ing appropriate military forces for particular missions,
by analyzing uncertain information about the situa-
tion. Throughout the planning process, PRS-CL mon-
itors the world for additional goals and events that
might require immediate action. PRS-CL executes the
plan by applying appropriate Acts to refine the plan to
lower levels of abstraction, eventually bottoming out in
actions that are executable in the world.
PRS-CL responds to many unexpected events by ap-
plying Acts representing SOPs. Sometimes an event
causes an execution failure that cannot be repaired by
any defined Acts (e.g., if transit approval is rescinded
for air space that is being used). PR.S-CL then invokes
a second PRS-CL agent to issue a replanning request
to SIPE-2. Meanwhile, the first agent continues exe-
cution of parallel threads of the plan not affected by
the failure. The planner modifies the plan by elimi-
nating actions that use the air space in question and
replacing them with an alternative mobilization. The
actions in the new plan are selected so as not to inter-
fere with the continuing execution of other actions in
the original plan. The new plan is sent to to the first
agent, which integrates the new plan with its current
activities and continues.
In a similar fashion, a Cypress agent controlling a
planetary rover would have the executor handle unex-
pected obstacles in its path, and call the planner to
modify the plan when progress can no longer be made
in the desired direction. On a satellite, the executor
could continue to monitor spacecraft systems while re-
questing the planner to modify the plan for transmit-
ting pictures back to earth after a failure in one of the
transmitters.
163
Conclusion
Cypress is a powerful framework in which to define
agents that must accomplish complex goals in dynamic
and unpredictable environments. The application of
Cypress to the military domain and to the Space Shut-
tle’s RCS (only the PRS-CL subsystem is used) attests
to the system’s usefulness.
The asynchronous replanning facility constitutes one
important technological advance, providing flexible
plan execution that can adapt to significant unex-
pected changes in the world. An interesting techni-
cal problem that had to be solved was the design of
ACT as a common representation for both executors
and planners. PRS-CL had to be extended in numer-
ous ways to support the execution of plans employing
constructs not found in the domain procedures defined
for previous PRS-CL applications.
Several characteristics distinguish Cypress from
other systems that provide both planning and reac-
tive execution. Many systems do not use general-
purpose planning and so cannot generate plans of suffi-
cient complexity for interesting applications. Previous
work in run-time replanning has either been limited
to synchronous approaches (Laird 1990) or focuses
local, adaptive modifications to rule sets, rather than
employing the full look-ahead reasoning of a planner
(Lyons & Hendricks 1992; Firby 1987). The ability
modify a complex, parallel plan at run time and adapt
execution activity to the new plan is, to our knowledge,
a new accomplishment.
Acknowledgments
This research was supported by the ARPA and
Rome Laboratory Planning Initiative (ARPI) un-
der Contract F30602-90-C-0086. The implementation
of the military operations domain was done under
ARPA/Rome Laboratory Contract F30602-90-C-0086
by Marie Bienkowski, Marie desJardins, and Roberto
Desimone. The RCS application was supported by
NASA Ames Research Center and was done by a team
led by Michael Georgeff. SIPE-2, PRS-CL, Gister-CL,
Grasper-CL, and Cypress are trademarks of SRI Inter-
national.
References
Firby, R. J. 1987. An investigation into reactive plan-
ning in complex domains. In Proceedings of the lg87
AAAI, 202-206.
Laird, J. 1990. Integrating planning and execution in
Soar. In Proceedings of the AAAI Spring Symposium.
Lyons, D. M., and Hendricks, A. J. 1992. A practical
approach to integrating reaction and deliberation. In
First International Conference on AI Planning Sys-
tems, 153-162.
Wilkins, D. E., and Desimone, R. V. 1994. Applying
an AI planner to military operations planning. In
Fox, M., and Zweben, M., eds., Intelligent Scheduling.
Morgan Kaufmann, San Mateo, CA. 685-709.
Wilkins, D. E.; Myers, K. L.; Lowrance, J. D.; and
Wesley, L. P. to appear. Planning and reacting in
uncertain and dynamic,environments. Journal of Ex-
perimental and Theoretical AL
164
