ArticlePDF Available

Abstract

Agents situated in dynamic and unpredictable environments require several capabilities, including synthesizing and executing plans while continuing to be responsive to the world. The Cypress system is a domainindependent framework for defining persistent agents with this full range of behavior. Cypress has been used for several applications, including military operations and fault diagnosis on the Space Shuttle. 1 Introduction Our research is developing persistent agents that can achieve complex tasks in dynamic and uncertain environments. We refer to such agents as taskable, reactive agents. An agent of this type requires a number of capabilities. The ability to execute complex tasks necessitates the use of strategic plans for accomplishing tasks; hence, the agent must be able to synthesize new plans at run time. The dynamic nature of the environment requires that the agent be able to deal with unpredictable changes in its world. As such, agents must be able to react to unanticipate...
Integrating Planning and Reactive Control*
David E. Wilkins Karen L. Myers
Abstract
Agents situated in dynamic and unpredictable envi-
ronments require several capabilities, including syn-
thesizing and executing plans while continuing to be
responsive to the world. The Cypress system is a
domain-independent framework for defining persistent
agents with this full range of behavior. Cypress has
been used for several applications, including military
operations and fault diagnosis on the Space Shuttle.
Introduction
Our research is developing persistent agents that can
achieve complex tasks in dynamic and uncertain envi-
ronments. We refer to such agents as taskable, reactive
agents. An agent of this type requires a number of
capabilities. The ability to execute complex tasks ne-
cessitates the use of strategic plans for accomplishing
tasks; hence, the agent must be able to synthesize new
plans at run time. The dynamic nature of the envi-
ronment requires that the agent be able to deal with
unpredictable changes in its world. As such, agents
must be able to react to unanticipated events by taking
appropriate actions in a timely manner, while contin-
uing activities that support current goals. The unpre-
dictability of the world could lead to failure of plans
generated for individual tasks. Agents must have the
ability to recover from failures by adapting their activ-
ities to the new situation, or replanning if the world
changes sufficiently. Finally, the agent should be able
to perform in the face of uncertainty.
Many domains of interest require problem-solving
agents with the capabilities described above. Military
0. This paper will appear in Proceedings of the Third In-
ternational Symposium on Artificial Intelligence, Robotics,
and Automation for Space, Jet Propulsion Laboratory,
Pasadena, CA, October 1994.
SRI International
Artificial Intelligence Center
333 Ravenswood Ave.
Menlo Park, CA 94025
phone: 415-859-2057 fax: 415-859-3735
{myers, wilkins}@ai.sri.com
and space operations provide good examples. Cer-
tainly one would not engage in an undertaking such as
Desert Storm or repairing the Hubble Space Telescope
without first formulating a strategic mission plan. Re-
active response and failure recovery are necessary be-
cause unexpected equipment failures, weather condi-
tions, enemy actions, and other events may require
changes to the overall strategic plan.
The Cypress system, described here, provides a
framework for creating taskable, reactive agents. Sev-
eral features distinguish our approach: (1) the gen-
eration and execution of complex plans with parallel
actions, (2) the integration of goal-driven and event-
driven activities during execution, (3) the use of ev-
idential reasoning for dealing with uncertainty, and
(4) the use of replanning to handle run-time execution
problems.
Our model for a taskable, reactive agent has two
main intelligent components, an executor and a plan-
ner. The two components share a library of possible
actions that the system can take. The library en-
compasses a full range of action representations, in-
cluding plans, planning operators, and executable pro-
cedures such as predefined standard operating proce-
dures (SOPs). These three classes of actions span mul-
tiple levels of abstraction.
The executor is always active, constantly monitor-
ing the world for goals to be achieved or events that
require immediate action. In accord with its current
beliefs and goals, the executor takes actions in resp.onse
to these goals and events. Appropriate responses in-
clude applying SOPs stored in the action library, in-
voking the planner to produce a new plan for achieving
a goal, or requesting that the planner modify a pre-
vious plan during execution. The planner should be
capable of synthesizing sophisticated action sequences
161
From: AAAI Technical Report FS-94-01. Compilation copyright © 1994, AAAI (www.aaai.org). All rights reserved.
that include parallel actions, conditional actions, and
resource assignments. The planner plans only to a cer-
tain level of detail, with the executor taking that plan
and expanding it at run time by applying appropriate
library actions at lower levels of abstraction.
Cypress
Cypress constitutes a framework in which to define
taskable, reactive agents based on the above model.
The architecture of Cypress is depicted in Figure 1.
The motivation for Cypress was to build a heuris-
tically adequate system for use in practical applica-
tions. To this end, Cypress relies on mature, power-
ful planning and execution technologies, namely the
SIPE-2 generative planner (Wilkins et al. to appear)
and the PrtS-CL reactive execution system (Wilkins et
al. to appear). We have applied Cypress to a number
of demanding problems, including real-time tracking,
fault diagnosis on the Space Shuttle, production-line
scheduling, and military operations (Wilkins et al. to
appear).
PRS-CL is a framework for constructing persistent,
real-time controllers that perform complex tasks in dy-
namic environments while responding in timely fashion
to unexpected events. It has been used to monitor the
Reaction Control System (RCS) of the Space Shuttle
(Wilkins et al. to appear). This application illustrates
the use of multiple agents, and has been used to detect
and recover from most of the possible malfunctions of
the RCS, including sensor faults, leaking components,
and regulator and jet failures. The system demon-
strated guaranteed response, support for asynchronous
inputs, interrupt handling, continuous operation, and
handling of noisy data.
SIPE-2 is a partial-order AI planning system that
supports planning at multiple levels of abstraction. It
has the properties required by our agent model, includ-
ing the ability to generate plans that include parallel
actions, conditional actions, resource assignments, and
the ability to modify previously generated plans. In
contrast to most AI planning research, heuristic ade-
quacy has been a primary design goal of SIPE-2.
PRS-CL and SIPE-2 employ their own internal rep-
resentations for plans and actions for efficiency. For
this reason, Cypress supports the use of an interlin-
gua called the ACT formalism (Wilkins et al. to ap-
pear) that enables these two systems to share informa-
tion. ACT provides a language for specifying actions
and plans for both planners and executors. Cypress
includes translators that can automatically map Acts
onto SIPE-2 and PRS-CL structures, and one that can
map SIPE-2 operators and plans into Acts. Using the
ACT interlingua, PRS-CL can execute plans produced
by SIPE-2 and can invoke SIPE-2 in situations where
run-time replanning is required. The ACT-Editor sub-
system supports the graphical creation and display of
Acts. Gister-CL (Wilkins et al. to appear) implements
a suite of evidential reasoning techniques that can be
used to analyze uncertain information about the world
and possible actions. For example, Gister-CL can be
used to reason about uncertain information in order to
choose among candidate Acts in either the planner or
executor.
In contrast to many other agent architectures, plan-
ning and execution operate asynchronously in Cypress,
in loosely coupled fashion. This approach makes it
possible for the two systems to run in parallel, even on
different machines, without interfering with the actions
of each other. In particular, PP~S-CL remains respon-
sive to its environment during plan synthesis. While
the subsystems of Cypress can function independently,
Cypress is used most advantageously as an integrated
framework that supports a wide range of planning and
execution activities.
Applications
An example from military operations planning
(Wilkins & Desimone 1994) is currently the only imple-
mented application that illustrates the use of all sub-
systems of Cypress, but it is similar to a space mission.
The most advantageous use of Cypress in space appli-
cations will most likely be in situations that do not
directly involve humans. A planetary rover will cer-
tainly need the combination of plan-directed behavior
with reactive response to the environment provided by
Cypress, and can build directly on our use of Cypress
modules to control an indoor mobile robot. Other ap-
propriate space applications include control of a satel-
lite or probe, controlling experiments on the shuttle
or space station, and providing an assistant to astro-
nauts to handle routine malfunctions and alert them of
important events that affect the overall mission plan.
The military application domain knowledge includes
approximately 100 plan operators, 500 objects with 15
to 20 properties per object, and 2200 initial predicate
instances. Plans range in size from several dozen to
162
ACT->Sipe, Sipe->ACT
translators
SIPE-2
-generative planning
-replanning
operator selection
object selection
situation assessment
ACT-Editor /
-common interface
-customize graphics
replanning ,,
Gister-CL
-reasoning about uncertainty
PRS-CL
-reactive execution
-replanning overseer
~|~lra:J~l lu r~ti:~ection
I
situation assessment
Figure 1: The Architecture o~’ Cypress
200 actions, including many that are to be executed in
parallel (Wilkins & Desimone 1994).
The scenario begins with a goal request for deter-
ring several military threats. SIPE-2 uses a set of
Acts previously input to the system to generate a plan
with many threads of parallel activities. During the
planning process, Gister-CL assists SIPE-2 in choos-
ing appropriate military forces for particular missions,
by analyzing uncertain information about the situa-
tion. Throughout the planning process, PRS-CL mon-
itors the world for additional goals and events that
might require immediate action. PRS-CL executes the
plan by applying appropriate Acts to refine the plan to
lower levels of abstraction, eventually bottoming out in
actions that are executable in the world.
PRS-CL responds to many unexpected events by ap-
plying Acts representing SOPs. Sometimes an event
causes an execution failure that cannot be repaired by
any defined Acts (e.g., if transit approval is rescinded
for air space that is being used). PR.S-CL then invokes
a second PRS-CL agent to issue a replanning request
to SIPE-2. Meanwhile, the first agent continues exe-
cution of parallel threads of the plan not affected by
the failure. The planner modifies the plan by elimi-
nating actions that use the air space in question and
replacing them with an alternative mobilization. The
actions in the new plan are selected so as not to inter-
fere with the continuing execution of other actions in
the original plan. The new plan is sent to to the first
agent, which integrates the new plan with its current
activities and continues.
In a similar fashion, a Cypress agent controlling a
planetary rover would have the executor handle unex-
pected obstacles in its path, and call the planner to
modify the plan when progress can no longer be made
in the desired direction. On a satellite, the executor
could continue to monitor spacecraft systems while re-
questing the planner to modify the plan for transmit-
ting pictures back to earth after a failure in one of the
transmitters.
163
Conclusion
Cypress is a powerful framework in which to define
agents that must accomplish complex goals in dynamic
and unpredictable environments. The application of
Cypress to the military domain and to the Space Shut-
tle’s RCS (only the PRS-CL subsystem is used) attests
to the system’s usefulness.
The asynchronous replanning facility constitutes one
important technological advance, providing flexible
plan execution that can adapt to significant unex-
pected changes in the world. An interesting techni-
cal problem that had to be solved was the design of
ACT as a common representation for both executors
and planners. PRS-CL had to be extended in numer-
ous ways to support the execution of plans employing
constructs not found in the domain procedures defined
for previous PRS-CL applications.
Several characteristics distinguish Cypress from
other systems that provide both planning and reac-
tive execution. Many systems do not use general-
purpose planning and so cannot generate plans of suffi-
cient complexity for interesting applications. Previous
work in run-time replanning has either been limited
to synchronous approaches (Laird 1990) or focuses
local, adaptive modifications to rule sets, rather than
employing the full look-ahead reasoning of a planner
(Lyons & Hendricks 1992; Firby 1987). The ability
modify a complex, parallel plan at run time and adapt
execution activity to the new plan is, to our knowledge,
a new accomplishment.
Acknowledgments
This research was supported by the ARPA and
Rome Laboratory Planning Initiative (ARPI) un-
der Contract F30602-90-C-0086. The implementation
of the military operations domain was done under
ARPA/Rome Laboratory Contract F30602-90-C-0086
by Marie Bienkowski, Marie desJardins, and Roberto
Desimone. The RCS application was supported by
NASA Ames Research Center and was done by a team
led by Michael Georgeff. SIPE-2, PRS-CL, Gister-CL,
Grasper-CL, and Cypress are trademarks of SRI Inter-
national.
References
Firby, R. J. 1987. An investigation into reactive plan-
ning in complex domains. In Proceedings of the lg87
AAAI, 202-206.
Laird, J. 1990. Integrating planning and execution in
Soar. In Proceedings of the AAAI Spring Symposium.
Lyons, D. M., and Hendricks, A. J. 1992. A practical
approach to integrating reaction and deliberation. In
First International Conference on AI Planning Sys-
tems, 153-162.
Wilkins, D. E., and Desimone, R. V. 1994. Applying
an AI planner to military operations planning. In
Fox, M., and Zweben, M., eds., Intelligent Scheduling.
Morgan Kaufmann, San Mateo, CA. 685-709.
Wilkins, D. E.; Myers, K. L.; Lowrance, J. D.; and
Wesley, L. P. to appear. Planning and reacting in
uncertain and dynamic,environments. Journal of Ex-
perimental and Theoretical AL
164
... (Pollack & Ringuette 1990) explore the relation between agent design and environmental factors. (Rosenschein & Kaelbling 1989) provide a situated automata theory highlighting the role of logic, complexity and information in situated agents. (Russell & Wefald 1991) offer an optimal design for a limited rational agent using utility-based search. ...
... Wilkins and Myers describe the CYPRESS system that provides the framework for the creation and control of taskable, reactive agents [19, 20]. Taskable, reactive agents have two main components: an executor and a planner. ...
Article
Full-text available
A new approach to military planning and execution has been proposed, called Anticipatory Planning, that seeks to merge planning and execution and replace reaction to events with anticipation of events. This paper describes a methodology for building an Anticipatory Planning Support System. A Plan Description is developed to manage the many tree-like branches that occur in planning and execution of an operation. A Planning Executive uses the differences between the plan and the actual operation to control the activities of Planners and Execution Monitors in anticipating future branches to the plan while constraining the planning space. At the heart of the system are inference mechanisms for determining branches in the plan and simulations for predicting future states. KEYWORDS Simulation, Planning, Decision Support
Article
This thesis describes an initial effort into creating a rule-based, reactive system for air combat simulation. This program uses the object-oriented extension of the expert system tool known as the C Language Integrated Production System (CLIPS). This effort rose out of the need for creating and integrating semi-autonomous forces for the Distributed Interactive System (DIS). This thesis describes the basic maneuvers a pilot uses in present air-to-air combat. The methodology includes the design decisions, knowledge-base development, phase architecture, and maneuver architecture development. The actual implementation of the selected architecture is described. This thesis also discusses the results of experimental runs with two agents maneuvering against one another.
Article
The traditional Military Decision Making Process (MDMP) focuses on developing a few friendly Courses of Action (COAs) against the "most-likely and most-dangerous" enemy COAs. Military planners need a way to incorporate the continuous feed of battle information into the planning process so that they achieve and maintain "option dominance". A new approach to military operations, called Anticipatory Planning and Adaptive Execution, treats planning and execution as a tightly coupled, single process, and replaces reaction to events with anticipation of events. This research develops the methodology for automating the Anticipatory Planning process. A prototype Anticipatory Planning Support System (APSS) has been designed and implemented to provide human planners with an interactive visual development system using simulations to build Plan Descriptions. The primary goals of this implementation are to provide a common representation of the plan, facilitate the planning process, anticipate the flow of the battle, and provide a means for stimulating planning systems.
Article
Agent theory in AI and related disciplines deals with the structure and behaviour of autonomous, intelligent systems, capable of adaptive action to pursue their interests. In this paper it is proposed that a natural reinterpretation of agent-theoretic intentional concepts like knowing, wanting, liking, etc., can be found in process dynamics. This reinterpretation of agent theory serves two purposes. On the one hand we gain a well established mathematical theory which can be used as the formal mathematical interpretation (semantics) of the abstract agent theory. On the other hand, since process dynamics is a theory that can also be applied to physical systems of various kinds, we gain an implementation route for the construction of artificial agents as bundles of processes in machines. The paper is intended as a basis for dialogue with workers in dynamics, AI, ethology and cognitive science. 1 Introduction Agent theory is a branch of artificial intelligence (Kiss, 1988). Its domain is...
Article
Full-text available
In order for a mobile robot to accomplish a non-trivial task, the task must be described in terms of primitive actions of the robot's actuators. Our contention is that the transformation from the high level description of the task to the primitive actions should be performed primarily at execution time, when knowledge about the environment can be obtained through sensors. Our theory is based on the premise that proper application of knowledge increases the robustness of plan execution. We propose to produce the detailed plan of primitive actions and execute it by using primitive components that contain domain specific knowledge and knowledge about the available sensors and actuators. These primitives perform signal and control processing as well as serve as an interface to high-level planning processes. In this work, importance is placed on determining what information is relevant to achieve the goal as well as determining the details necessary to utilize the sensors and actuators. Kn...
Article
Full-text available
In order for a mobile robot to accomplish a non-trivial task, the task must be described in terms of primitive actions of the robot's actuators. Our contention is that the transformation from the high level description of the task to the primitive actions should be performed primarily at execution time, when knowledge about the environment can be obtained through sensors. We propose to produce the detailed plan of primitive actions by using a collection of low-level planning components that contain domain specific knowledge and knowledge about the available sensors, actuators, and sensor/actuator processing. This collection will perform signal and control processing as well as serve as an interface to high-level planning processes. 1. Introduction To be useful in the real world robots need to be able to move safely in unstructured environments and achieve their given tasks despite unexpected changes in their environment or failures of some of their sensors. The variability of the wor...
Article
Full-text available
In order for a mobile robot to accomplish a non-trivial task, the task must be described in terms of primitive actions of the robot's actuators. We claim that the transformation from the high level task description to the primitive actions should occur primarily at execution time, when information about the environment can be obtained through sensors. Our claim is based on the premise that proper application of knowledge in the integration and utilization of sensors and actuators increases the robustness of execution. This paper addresses the issue of what knowledge needs to be known about sensors, actuators and processes in order to be able to integrate their usage, and control them during execution. The proposed method of execution works for any sensor/actuator existing on the robot when given such knowledge. I. KNOWLEDGE ABOUT UTILIZING SENSORS/ACTUATORS A very large part of our civilized world is unstructured and dynamic. This fact has been one of the leading obstacles to mass-qua...
Article
Full-text available
To be useful in the real world, robots need to be able to move safely in unstructured environments and achieve their given tasks despite unexpected environmental changes or failures of some of their sensors. The variability of the world makes it impractical to develop very detailed plans of actions before execution since the world might change before execution begins and thus invalidate the plan. We propose to generate the very detailed plan of actions needed to control a robot at execution time. This allows to obtain uptodate information about the environment through sensors and to use it in deciding how to achieve the task. Our theory is based on the premise that proper application of knowledge in the integration and utilization of sensors and actuators increases the robustness of execution. In our approach we produce the detailed plan of primitive actions and execute it by using an object-oriented approach, in which primitive components contain domain specific knowledge and knowledg...
Article
Full-text available
In order for a mobile robot to accomplish a non-trivial task, the task must be described in terms of primitive actions of the robot's actuators. Our contention is that the transformation from the high level description of the task to the primitive actions should be performed primarily at execution time, when knowledge about the environment can be obtained through sensors. Our theory is based on the premise that proper application of knowledge increases the robustness of plan execution. We propose to produce the detailed plan of primitive actions and execute it by using primitive components that contain domain specific knowledge and knowledge about the available sensors and actuators. These primitives perform signal and control processing as well as serve as an interface to high-level planning processes. In this work, importance is placed on determining what information is relevant to achieve the goal as well as determining the details necessary to utilize the sensors and actuators. I. ...
Article
Full-text available
Agents situated in dynamic and uncertain environments require several capabilities for successful operation. Such agents must monitor the world and respond appropriately to important events. The agents should be able to accept goals, synthesize complex plans for achieving those goals, and execute the plans while continuing to be responsive to changes in the world. As events render some current activities obsolete, the agents should be able to modify their plans while continuing activities unaffected by those events. The Cypress system is a domain-independent framework for defining persistent agents with this full range of behavior. Cypress has been used for several demanding applications, including military operations, real-time tracking, and fault diagnosis. ii Contents 1 Introduction 1 1.1 Research Strategy : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 2 1.2 A New Technology : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 2 2 Overview of C...
Article
Full-text available
This paper describes a prototype system for quickly developing joint military courses of action. The system, SOCAP (System for Operations Crisis Action Planning), combines a newly extended version of an AI planning system, SIPE--2 (System for Interactive Planning and Execution), with a color map display and applies this technology to military operations planning. This paper describes the Socap problem domain, how SIPE--2 was used to address this problem, and the strengths and weaknesses of our approach. i Contents 1 Introduction 1 2 Military Operations Crisis Action Planning 2 3 The SIPE-2 Planning System 4 3.1 Describing the Domain in SIPE-2 : : : : : : : : : : : : : : : : 6 3.2 The User Interface in SIPE-2 : : : : : : : : : : : : : : : : : : 7 3.3 Interactive Planning : : : : : : : : : : : : : : : : : : : : : : : 10 3.4 Scheduling : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10 4 SOCAP -- System for Operations Crisis Action Planning 12 4.1 Applying Operators :...
Article
A robot system operating in an environment in which there is uncertainty and change needs to integrate the ability to react with the ability to plan ahead. In this paper, we present a novel approach to this integration for a practical robot problem, the kitting robot. In our approach, planning is cast as the ongoing adaptation of a reactive system, incrementally bringing its behavior into line with a set of goals. A method is presented whereby the planner can make iterative improvements to the reactor so that it eventually converges on the ideal reactor (a concept related to the universal plan) for its environment and set of planner goals. To balance this theoretical work, our current implementation on a Puma 560 robot is overviewed.
Conference Paper
A model of purely reactive planning is proposed based on the concept of reactive action packages. A reactive action package, or RAP, can be thought of as an independent entity pursuing some goal in competition with many others at execution time. The RAP processing algorithm addresses the problems of execution monitoring and replanning in uncertain domains with a single, uniform representation and control structure. Use of the RAP model as a basis for adaptive strategic planning is also discussed.
Integrating planning and execution in Soar
  • J Laird
Laird, J. 1990. Integrating planning and execution in Soar. In Proceedings of the AAAI Spring Symposium.