Conference PaperPDF Available

A deliberative architecture for AUV control

  • Genesys, Daly City, San Francisco

Abstract and Figures

Autonomous Underwater Vehicles (AUVs) are an increasingly important tool for oceanographic research demonstrating their capabilities to sample the water column in depths far beyond what humans are capable of visiting, and doing so routinely and cost-effectively. However, control of these platforms to date has relied on fixed sequences for execution of pre-planned actions limiting their effectiveness for measuring dynamic and episodic ocean phenomenon. In this paper we present an agent architecture developed to overcome this limitation through on-board planning using Constraint- based Reasoning. Preliminary versions of the architecture have been integrated and tested in simulation and at sea.
Content may be subject to copyright.
Fig 1. An MBARI AUV at sea
Fig. 2: A 4-reactor T-REX agent.
Abstract—Autonomous Underwater Vehicles (AUVs) are an
increasingly important tool for oceanographic research
demonstrating their capabilities to sample the water column in
depths far beyond what humans are capable of visiting, and
doing so routinely and cost-effectively. However, control of
these platforms to date has relied on fixed sequences for
execution of pre-planned actions limiting their effectiveness for
measuring dynamic and episodic ocean phenomenon. In this
paper we present an agent architecture developed to overcome
this limitation through on-board planning using Constraint-
based Reasoning. Preliminary versions of the architecture have
been integrated and tested in simulation and at sea.
Oceanography has traditionally relied on ship-based
observations. These have recently been augmented by
robotic platforms such as Autonomous Underwater Vehicles
(AUV) [1-3], which are untethered powered mobile robots
able to carry a range of payloads efficiently over large
distances in the deep ocean. A common design relies on a
modular tube-like structure with propulsion at the stern and
various sensors, computers and batteries taking up the bulk
of the tube (Fig. 1). AUVs have demonstrated their utility in
oceanographic research in gathering time series data by
repeated water-column surveys [4], detailed bathymetric
maps of the ocean floor in areas of tectonic activity [5,6] and
performed hazardous under-ice missions [7].
Typically AUVs do not communicate with the support
ship or shore while submerged and rely on limited stored
battery packs while operating continuously for tens of hours.
Current AUV control systems [8] are a variant of the
behavior-based Subsumption architecture [9]. A behavior is
a modular encapsulation of a specific control task and
includes acquisition of a GPS fix, descent to a target depth,
drive to a given waypoint, enforcement of a mission depth
envelope etc. An operator defines each plan as a collection
of behaviors with specific start and end times as well as
maximum durations, which are scripted a priori using simple
mission planning tools. In practice, missions predominantly
consist of sequential behaviors with duration and task
specific parameters equivalent to a linear plan with limited
flexibility in task duration. Such an approach becomes less
effective as mission uncertainty increases. Further, the
architecture offers no support to manage the potentially
complex interactions that may result amongst behaviors,
pushing a greater cognitive burden on behavior developers
and mission planners. This paper describes an automated
onboard planning system to generate robust mission plans
using system state and desired goals. By capturing explicit
interactions between behaviors as plan constraints in the
domain model and
the use of goal-
commanding, we
expect this approach
to reduce the
cognitive burden on
AUV operators. Our
interest in the near
term is to incorporate decision-making capability to deal
with a range of dynamic and episodic ocean phenomenon
that cannot be observed with scripted plans.
The remainder of this paper is laid out as follows. Section
II lays out the architecture of our autonomy system, section
III details the experimental results to date, related work
follows in section IV with concluding remarks in section V.
T-REX (Teleo-Reactive EXecutive) is a goal-oriented
system, with embedded automated planning [14,15] and
adaptive execution. It encapsulates the long-standing notion
of a sense-deliberate-act cycle in what is typically
considered a hybrid architecture where sensing, planning
and execution are interleaved. In order to make embedded
planning scalable the system enables the scope of
deliberation to be partitioned functionally and temporally
and to ensure the current state of the agent is kept consistent
and complete during execution. While T-REX was built for
a specific underwater robotics application, the principles
behind its design are applicable in any domain where
deliberation and execution are intertwined.
Fig. 2 shows a conceptual view of a Teleo-Reactive
Agent. An agent is viewed as the coordinator of a set of
concurrent control loops. Each control loop is embodied in a
Teleo-Reactor (or reactor for short) that encapsulates all
Conor McGann, Frederic Py, Kanna Rajan, Hans Thomas, Richard Henthorn, Rob McEwen
Monterey Bay Aquarium Research Institute, Moss Landing, California
A Deliberative Architecture for AUV Control
details of how to accomplish its control objectives. Arrows
represent a messaging protocol for exchanging facts and
goals between reactors: thin arrows represent observations
of current state; thick arrows represent goals to be
accomplished. Reactors are differentiated in 3 ways:
Functional scope: indicating the state variables of concern
for deliberation and action.
Temporal scope: indicating the look-ahead window over
which to deliberate.
Timing requirements: the latency within which this
component must deliberate for goals in its planning
Fig. 2 for example, shows four different reactors; the
Mission Manager provides high-level directives to satisfy
the scientific and operational goals of the mission: its
temporal scope is the whole mission, taking minutes to
deliberate if necessary. The Navigator and Science Operator
manage the execution of sub-goals generated by the Mission
Manager. The temporal scope for both is in the order of a
minute even as they differ in their functional scope. Each
refines high-level directives into executable commands
depending on current system state. The Science Operator is
able to provide local directives to the Navigator. For
example if it detects an ocean front it can request the
navigation mode to switch from a Yo-Yo pattern in the
vertical plane to a Zig-Zag pattern in the horizontal plane, to
have better coverage of the area. Deliberation may safely
occur at a latency of 1 second for these reactors. The
Executive provides an interface to a modified version of the
existing AUV functional layer. It encapsulates access to
commands and vehicle state variables. The Executive is
reasonably approximated as having zero latency within the
timing model of our application since it will accomplish a
goal received with no measurable delay, or not at all; in
other words it does not deliberate.
T-REX has a central and explicit notion of time with all
reactors synchronized by an internal clock. The unit of time
is a tick, defined in external units on a per application basis;
tick boundaries signify when synchronization of all reactors
must occur while between ticks reactors may deliberate. The
agent-state is represented as a set of timelines, which capture
the evolution of a system state-variable over time. A
timeline is a sequence of tokens that are temporally qualified
assertions expressed as a predicate with start and end time
bounds defining the temporal scope over which it holds. The
minimum duration of a token is a tick giving a discrete
synchronous view of the state of the world. Token start and
end times can be defined as intervals to express temporal
Agent timelines are distributed across reactors depending
on their functional scope. Information exchange between
reactors, where necessary, is provided through the following
Explicit timeline ownership: Each timeline is owned by
exactly one reactor. Any reactor may request a new goal,
or replan such requests in the event of a change of plan;
but only the owner of the timeline can decide what goal to
Observations: capture the current value of a timeline.
Observations are asserted by the owner of a timeline.
Goals: express a desired future timeline value. They offer
a way to delegate a task to a reactor. Goals are requested
for expansion into sub-goals or commands and can be
recalled on plan changes when replanning is triggered.
Dispatch and notification rules: define when information
must be shared to ensure consistency and completeness of
agent state at the execution frontier and to allow sufficient
time for deliberation.
The mapping between reactors and timelines is the basis for
sharing information. If a reactor owns a timeline it is
declared internal to that reactor; if it uses a timeline to
observe values and/or express requirements it is declared
external to that reactor. Fig. 3 illustrates the flow of
information in a system containing 3 reactors: The Mission
Manager keeps track of science goals to give directives to
the Navigator using the Path external timeline. The
Navigator manages the navigation of the AUV with one
internal timeline and three external timelines. The
navigation route is used to select the appropriate commands
to send to the Executive as an internal timeline while
Position and Attitude timelines capture AUV navigation
data. A Command timeline captures the command state of
the Executive. These external timelines are internal to the
Executive in turn. The Command timeline values are the
actual commands that are managed by the AUV functional
layer. The content of this last timeline at the execution
frontier corresponds to the currently active behavior.
To ensure a complete and consistent view of system state,
the T-REX information exchange framework needs to
impose further restrictions on the way timelines,
observations and goals can be used:
No ’holes’ are allowed at the execution frontier i.e. all
timelines must have a value at the end of the current tick.
If no update is provided via an observation, and in the
absence of information to the contrary, a reactor assumes
the previous value(s) on the timeline is/are still valid. We
Fig. 4: The T-REX agent algorithm
synchronize (tick);
done = false;
while(!done && currentTick() == tick)
done = stepNextReactor(tick);}
refer to this as the Inertial Value Assumption since it
conveys some inherent inertia of current values.
Contradictory information can come from the model or
from a new observation. This has important implications
for reducing the cost of synchronization since observations
need only be published as timeline values change.
At the end of the current tick, all observations must be
consistent, by requiring all reactors to hold the same view
at the execution frontier.
The past is monotonic. All tokens that have finished or
that have started but have yet to finish (i.e. they span the
execution frontier) can only be restricted in time.
An observation received at a tick applies to that tick only.
It cannot refer to the past except by restricting the values
of a token that is actually running (i.e. with an end time in
the future). It cannot refer to the future, as it would then be
a goal, rather than observed reality.
The algorithm at the heart of a T-REX agent in Fig. 4 is
called at the start of every tick. There are three key steps in
the algorithm; first, all timelines are synchronized at the
current execution frontier which is followed by the dispatch
of goals. And finally, the remaining CPU time can be
allocated to reactors for deliberation in incremental steps.
Each of these component algorithms operates over the entire
set of reactors.
A. Synchronization
The goal of synchronization is to produce a consistent and
complete view of agent state at the execution frontier. All
reactors synchronize at the same rate – once per tick. While
this may seem onerous, the actual cost of synchronization is
based on how much information has actually changed. For
example, in Fig. 3. the Position timeline is relatively volatile
and will likely change on every tick. However, the Path
timeline may hold a single value for many ticks. In this case,
as a result of the Inertial Value Assumption, if no new
observation is received, the Path timeline will extend its
current value simply by incrementing the lower bound of the
end time of the current value.
The strict rules of timeline ownership enable a clear
policy for conflict resolution: observations dominate
expectations. For example, if the Navigator expected the
vehicle depth to be less than 0.3m in order in order to obtain
a GPS fix but the actual depth observed by the Executive is 1
meter, then the expected value is discarded. This may impact
plan feasibility and force the Navigator to find an alternative
solution by rejecting the current plan.
To ensure global consistency the agent undertakes local
synchronization of the reactors until quiescence. In principle,
this operation is equivalent to solving a planning problem
over the set of all internal timelines for a planning horizon
restricted to a tick. If a reactor has an external timeline, it
depends on its owner for such consistency. In this way the
reactors form a dependency graph which in practice we
require to be acyclic, allowing ordering of synchronization
for purposes of efficiency.
B. Dispatching Goals
Where observations are the driver for reaction, goals are
the driver for deliberation. The purpose of dispatching is to
task reactors with new goals in a timely manner. To
accomplish this, T-REX provides explicit parameters and
rules to govern dispatching.
λ - The latency of the reactor or the worst-case number of
ticks to deliberate over a request.
π - The planning horizon of the reactor quantifying the
look-ahead for deliberation.
τ - The execution frontier expressing the boundary
between the past and the future.
To understand the implications of the above parameters,
consider the example given in Fig. 5. To satisfy the goal
Go(31.73, -121.80, 100) in its Path timeline the Navigator
decides that it needs the vehicle to descend(100) at tick 10
for a duration between 50 and 55 ticks and then to achieve
waypoint(31.73,-121.80) on successful termination of
descend. Since the Executive is the owner of the Command
timeline, these two goals need to be dispatched by the
Navigator to the Executive so that the latter can resolve
them. The importance of λ is to ensure the Executive has
sufficient time to complete deliberation prior to starting the
requested goal. If the start-time for a goal dispatched to the
Executive at τ were necessarily less than τ+λExec the
Executive may be unable to deliberate to resolve the goal,
leading to a plan failure.
Since the planning window of the Executive is πExec, the
Executive should receive all goals that can start before
τ+λExec+πExec. This will enable the Executive to leverage as
much information as it can handle in making judicious
decisions on how to accomplish the goals requested. Sending
a goal with a start time strictly greater than τ+λExec+πExec
will not be considered by the Executive. Moreover, such
dispatch incurs a cost for transmission of information and
may over-commit the Navigator unnecessarily.
Fig. 5: Illustration of goal dispatching window
Fig. 7: A Deliberative reactor
class Path extends AgentTimel ine {
predicate At{Node location;}
predicate Go{Node from; Node To;}
class Position extends AgentTimeline {
predicate Holds{Node value};
Path::At {
met_by(Go g);
eq(, location);
contained_by(Position.Holds p);
eq(p.value, location);
Path::Go {
met_by(At p);
eq(p.location, from);}
Fig. 8. A Mission traversal graph
Therefore the general rule is that the dispatching window
for a timeline is a time window that depends on the latency
and the look ahead of the reactor owning the timeline. This
dispatch window, HD is therefore defined by:
HD = [τ + λ, τ + λ + π]
This implies that as soon as the start time of a goal on an
external timeline intersects HD, it is dispatched to the owner
of the timeline. This rule is necessary and sufficient to
ensure that each reactor has sufficient time (λ) and
information (π) to deliberate on goals provided by other
reactors. In our implementation, we have an Executive,
which is purely reactive and therefore λExec = πExec = 0
implying that the Executive does not plan beyond the
execution frontier.
C. Deliberation
The framework presented thus far makes the details of
deliberation an internal concern for each reactor even if it
has to capture different functional and temporal scope. Our
own implementation of T-REX uses a Constraint-based
Temporal Planning approach based on EUROPA-2 [10,11]
employing a declarative model-based paradigm. The model
describes state variables (e.g. position, battery level) and
actions (e.g. ascend, descend, getGPS, takeWaterSample) of
the system. Constraints can be specified to enforce
relationships between state variables. For example, it is
convenient to represent the vehicle as being at the surface, or
not, which can be captured with a boolean state variable (e.g
AtSurface). We define a relationship between this variable
and the deph of the vehicle as follows: if depth <= 0.3 then
AtSurface = true. The model also describes constraints
between states and actions. For example, the vehicle must be
at the surface during getGPS. A sample domain model is
shown in Fig. 6 with the Path timeline having two predicates
At and Go; the example rules in the parameter specification
express the constraint that to be at a location, the AUV needs
to go to that coordinate and the position must be maintained
for a temporal interval that is consistent with the rest of the
model. A T-REX agent uses a single model for control at
various levels of abstraction and at various speeds of
execution. Different reactors reference subsets of this model
according to their functional scope.
The Deliberative
reactor is a
specialization of a
utilizing models,
plans and planning to
accomplish reactive
and goal directed
control. Fig. 7
describes the main
components of this
reactor. The inward
pointing arrows
reflect the
invocations of the
agent control loop
dispatch and
deliberation. The
Database is a
source and sink for
observations and
goals based on the
semantics of
internal and external timelines and the rules of information
exchange. It is an extension of the EUROPA-2 plan
database, augmented for specialized buffering for efficient
access to timeline data for dispatch and synchronization and
manages state information. Model rules are applied
automatically through a combination of propositional
inference and constraint propagation [21], to check
consistency and prune infeasible elaborations of the plan
maintained in the database. The Synchronizer is a
specialized configuration of a EUROPA solver operating
over a 1-tick horizon. It accomplishes local consistency and
completeness. The database propagates the results of
synchronization to the future. The Dispatcher is a simple
algorithm that publishes goals to owner reactors of its
external timelines according to the dispatch semantics
previously defined. Finally, the Planner is yet another
instance of a EUROPA solver used to deliberate over the
specified temporal and functional scope of the reactor using
a heuristic based chronological backtracking search for
partial plan refinement. These entities together are used
under different configurations for the Mission Manager,
Science Operator and Navigator shown in the example in Fig
3. Further details on EUROPA can be found in [10,11].
Our experiments with T-REX at sea involved using two
onboard computers on our AUV: a main vehicle computer a
244 Mhz PC/104 stack running the QNX real-time operating
system running the functional layer, and a separate 367 MHz
EPIC EPX-GX500 AMD Geode stack running Linux and T-
REX. Communication between T-REX and the functional
layer computer was with a socket-based protocol allowing
the exchange of goals and state updates. For validation
purposes we initially ran experiments on a high-fidelity
AUV simulator based on [13] which captures vehicle
dynamics to validate our missions. Sea trials with T-REX
onboard an AUV
were in the
Monterey Bay,
California using
our support ship
the R/V Zephyr.
The tick duration
was set to 1
second. In this
section we
Fig. 6: A domain model in T-REX
Fig. 9. A mission to traverse to the South node
Fig. 10. A Token flexibility example
Fig. 11. CPU usage for the mission in Fig 9.
Fig. 12. T-REX plan with heading changes
discuss one such mission among many executed at sea,
where we focused on demonstrating nominal mission
scenarios where scientists orient observations along specific
One such set of legs was encoded as a graph located in
the northern end of the Bay (Fig. 8). Such a representation
has a number of distinct advantages; first it accurately
predicts lower bounds on traversals from one node in the
graph to another and thereby quantifies time and resources
towards goal achievement (or for shedding over-subscribed
goals). Second, it allows us to naturally deal with shortest
path computations using our planner’s existing constraint
network algorithms and representation. Finally it allows
scientists to clearly represent their requirements in a
compact representation not unlike existing transect patterns
with the important addition of specifying meta-level features
such as goal priorities without concern for how the AUV
would achieve these goals.
In Fig. 9, the goal of the mission was to head to the south
node of such a traversal graph. The straight-line transect
planned was repeatedly interrupted in-situ during
deliberation, with check-in windows forcing the vehicle to
surface every 100 seconds with at least 40 seconds at the
surface. The dynamics of the vehicle [24] resulted in the
vehicle to damp its downward decent by compensating on its
pitch axis prior to a straight and level move thru the water
column. This was soon followed by an ascend to the surface
for a GPS fix followed by a short burst by the AUV to
accelerate to depth. The mission goals are decomposed to
those on the navigation timeline as a series of Go(South)
followed by check-in tokens. Further decomposition of the
Go activity in turn, results in setpoint, descend and waypoint
tokens also within the Navigator. The waypoint token tries
to achieve reaching the South node; however the 100sec
check-in window constraint preempts the achievement of
this traversal making the AUV surface.These series of
actions are successively generated till an observation from
the executive determines that the vehicle is indeed at the
South node. Within each set of these setpoint, descend and
waypoint tokens there is an important issue T-REX has to
deal with in terms of execution uncertainty; in this case the
precise end time
for descending to
depth is
uncontrollable, i.e
only exogenous
conditions can
determine the
precise duration
of this activity. The waypoint token duration therefore is
limited by the durations of the Go, setpoint and descend
tokens and can only be executed when the descend token
finishes. Fig. 10 shows two examples where a descend
could take longer (a) or shorter (b); modeled durations of the
descend token however need to be able to reasonably
encapsulate such variations which in practice are already
considered when scripting plans a priori.
Fig. 11 shows the CPU usage and the impact of
synchronization and deliberation that lead to changes in
multiple reactors. The spikes shown correspond to a
dynamic plan repair associated with the insertion of a check-
in window. When the Executive terminates the waypoint
activity, an observation is returned comparing the Goto
location (South in metric units) with the (open-loop)
distance traversed by the vehicle. If the vehicle is not at its
desired Goto location, an additional Goto goal will be
generated to make up the difference. The most common
reason for waypoint terminating before reaching the target
destination is due to duration constraints imposed by a
check-in window. The Navigator inserts a check-in goal and
further decomposes the goal in-situ as mentioned above into
ascend, getgps and setpoint activities as needed. If on
completion of a waypoint the vehicle is within an expected
distance of its target location the Navigator will terminate
the higher-level navigation goal. An interesting feature of
the localization activity is the requirement that the AUV stay
on the surface for at least 40 secs. However, as shown in the
second check-in window in Fig. 11, when the vehicle is able
to obtain a GPS fix well under this time limit the planner
reactively inserts an Idle activity.
Fig. 12 illustrates a longer mission where T-REX received
the goals to be At the West node and then GoTo the North
node starting from the South node in the traversal graph. As
before, we see the Navigator refining these goals with an
interesting twist; when the AUV is at the West node, T-REX
realizes that it had sufficient time to start the new goal
before the next check-in and inserted a Go(North) token for
the remaining duration. Such opportunistic decision-making
is unrealizable with scripts designed a-priori and clearly
demonstrates advantages of onboard deliberation. Additional
data on T-REX test results can be found at [25].
T-REX is inspired from IDEA [16,17], which in turn is
based on ideas in the Remote Agent Experiment (RAX)
[14,15]. T-REX is similar to both in its formulation of a
timeline-based representation, and in its use of planning and
execution at its core. It is distinct from IDEA primarily in its
formulation for exchanging and synchronizing state between
reactors. The Autonomous Sciencecraft Experiment [18]
conceptually borrows from RAX. The CASPER planner is
not directly embedded in the execution as in T-REX.
Further, temporal flexibility within and deals only with
grounded plan representation. The 3-layered LAAS
architecture [19] provides decisional capabilities using a
constraint-based symbolic planner integrated with reactive
components. However its disparate components are
manipulating different representations using heterogeneous
modeling languages. Such an approach tends to make system
design and integration difficult [20]. In contrast, although T-
REX’s design leads to factoring of computation into layers,
in practice a hierarchical structure is not inherent, nor is
deliberation required or prohibited for any layer.
While a number of control architectures have been built
for AUV control [1,8] T-REX’s design philosophy is closest
to DAMN [22] and ORCA [23]. DAMN is a reactive
Subsumption based architecture with no inherent
deliberation. ORCA uses schemas within a case-based
planning framework; however the efficacy of ORCA’s
approach is unclear in terms of scalability in the number of
schemas since the literature does not indicate whether the
system was actually fielded on an AUV. Further there is no
indication that it reasons explicitly with time and resources.
Our results to date show that onboard planning and
execution within the T-REX framework can handle
uncertainty in the sub-sea domain gracefully well within the
computational capacity available on our AUV’s. Our
immediate next steps are to integrate resource constraints for
deliberation in goal selection and to demonstrate dynamic
re-planning onboard the vehicle to adapt to science
observations opportunistically to enable characterization of
dynamic and episodic phenomenon such as ocean Fronts and
Thin Layers.
This research was supported by the David and Lucile
Packard Foundation. We thank the crew of the R/V Zephyr
for their help in deployments and NASA Ames Research
Center for making the EUROPA planner available.
[1] Yuh, J., “Design and Control of Autonomous Underwater Robots: A
Survey” Autonomous Robots, 2000 8, 7–24, 2000.
[2] Blidberg D.R., “Autonomous Underwater Vehicles: A Tool for the
Ocean,” Unmanned Systems, vol. 9, pp. 10-15, Spring 1991
[3] Bellingham J. G., “A Second Generation Survey AUV” In IEEE
Conf on Autonomous Underwater Vehicles, Cambridge, MA, 1994.
[4] Ryan, J.P., “Physical-biological coupling in Monterey Bay,
California: topographic influences on phytoplankton”, Marine
Ecology Progress Series, Vol 287: 28-32 2005.
[5] Yoerger, D. R., “Surveying deep-sea hydrothermal vent plumes
with the Autonomous Benthic Explorer (ABE)”, Proc 12th UUST,
Durham, New Hampshire, August, 2001
[6] Thomas, H.,, “Mapping AUV Survey of Axial Seamount”, Eos
Trans. AGU, 87(52), Fall Meet. Suppl., Abstract V23B-0615, 2006
[7] Bellingham, J. G., “Field Results for an Arctic AUV Designed
for Characterizing Circulation and Ice Thickness” AGU, Fall 2002.
[8] Carreras M.,, “Behaviour Control of UUV’s” in Advances in
Unmanned Marine Vehicles, by G. N. Roberts, Robert Sutton Eds,
IEE Control Series, January, 2006.
[9] Brooks, R. A. “A robust layered control system for a mobile robot”
IEEE Journal of Robotics and Automation, RA-2:14–23, 1986.
[10] Jonsson, A., P. Morris, N. Muscettola, K Rajan, B Smith “Planning in
Interplanetary Space: Theory and Practice" AIPS 2000, Brekenridge.
[11] Frank, J., A. Jonsson, “Constraint-based attributes and interval
planning,” Journal of Constraints, vol. 8, Oct. 2003.
[12] Nilsson, N., “Teleo-Reactive Programs for Agent Control," Journal of
Artificial Intelligence Research, 1:139-158, January 1994.
[13] Gertler, M., Hagen, G.R. “Standard Equations of Motion for
Submarine Simulation” Naval Ship Research and Development Center
Report 2510, June 1967.
[14] Muscettola N., “Remote Agent: To Boldly Go Where No AI
System Has Gone Before” AI Journal 103(1-2):5-48, August 1998.
[15] Rajan K.,, “Remote Agent: An Autonomous Control System for
the Ne w Millennium,” PAIS, 14th ECAI, Berlin, 2000.
[16] Muscettola N.,, “IDEA: Planning at the core of autonomous
reactive agents”, in Proc IWPSS, Houston, October 2002.
[17] Finzi, A., F. Ingrand, N. Muscettola, “Model-based Executive Control
through Reactive Planning for Autonomous Rovers”, IROS 2004
[18] Chien, S., “Using Autonomy Flight Software to Improve Science
Return on Earth Observing One”, J. of Aerospace Computing,
Information Communication. April 2005
[19] Ingrand, F.,, “Decisional Autonomy of Planetary Rovers,”
Journal of Field Robotics, Vol 24, Issue 7, Pages 559 - 580, July 2007.
[20] D. Bernard,, “Remote Agent Experiment: Final Report,” NASA
Technical Report, Feb. 2000
[21] Dechter, R., I . Meiri, J. Perl “Temporal constraint networks”,
Artificial Intelligence, Volume 49, Issue 1-3, pp 61 – 95, 1991
[22] Rosenblatt, J., “A Behavior-based Architecture for Autonomous
Underwater Exploration” Intnl. J. of Info Sciences, vol. 145, 2002.
[23] Turner, R. M., Stevenson, R. A. G. “ORCA: An adaptive, context-
sensitive reasoner for controlling AUVs”. in Proc 7th UUST, 1991.
[24] McEwen, R and Streitlien, K “Modeling and Control of a Variable-
Length AUV” Proc 12th UUST, 2001.
... Then we conducted a series of experiments to formally evaluate these strategies with respect to a set of required properties. Second, we applied our approach to a real-life case study for a robot, namely, the DALA robot [16]. ...
... In this section, we propose to apply our approach to a concrete autonomous system, namely the DALA Robot [16]. In particular, we focus on failure detection and recovery in the navigation module at the functional level of the robot. ...
... In general, robotic architectures are organized into several levels, which correspond to various temporal requirements (e.g., TREX [16]) or various levels of abstraction of functionality, such as the LAAS architecture [50]. The functional level, of most complex systems and robotic architectures, includes all the basic, built-in action and perception capabilities. ...
Full-text available
The challenges of current software-intensive systems, large-scale information and computing systems environments, which are highly dynamic, heterogeneous, and unpredictable, have motivated the development of techniques that enhance these systems with autonomous behaviors. Even though different concerns about these systems have been deeply studied, their design is still considerably more challenging than traditional ones. Self-healing is one of the main features that characterize autonomic computing systems. Failure detection, recovery strategies, and reliability are of paramount importance to ensure continuous operation and correct functioning even in the presence of a given maximum amount of faulty components. Most existing research and implementations focus on architecture-specific solutions to introduce self-healing behaviors. This implies that users must tailor their software by taking into account architecture-specific fault tolerance features, which requires too much effort from developers and users. This paper proposes a distributed formal model for the specification, verification, and analysis of self-healing behaviors in autonomous systems, from failure-detection to self-recovery. Such a high-level model allows users to specify and apply the desired type of failure detection and recovery without requiring any knowledge about its implementation. Our model allows not only formal verification of different properties but also performance evaluation. We provide the verification of qualitative properties using state-space exploration tools, and quantitative properties are also validated through statistical model-checking. All these properties are preserved in actual implementation by ensuring that the deployed code is consistent with the validated model.
... Several types of marine operations are transitioning to higher levels of autonomy. Among them are military surveillance operations using unmanned surface vehicles, mine reconnaissance and scientific surveys with autonomous underwater vehicles [1,2], and transportation with autonomous surface vehicles (ASVs) [3]. Commercial cases of automatic crossing with ships were demonstrated by Wärtsilä and Rolls-Royce (later acquired by Kongsberg Maritime) in 2018, with limited scope. 1 Both tests included automatic transit and docking. ...
... • Blocked switching poses • Sensor failures • Actuator failures • Trajectory planners unable to find solutions Other methodologies that enable increased autonomy are available in the scientific literature. This includes T-REX from the Monterey Bay Aquarium Research Institute [2], HAL from the Norwegian Defence Research Establishment [22], CARACaS by the US Office of Naval Research [23], GenoM by Laboratory for Analysis and Architecture of Systems [24], and MOOS-IvP from the Massachusetts Institute of Technology [25]. These tools have advanced capabilities such as replanning, negotiating competing goals, and deliberative task-level planning, which would take us closer to unsupervised, autonomous operation. ...
... Therefore, AAL and robot task planning are critical aspects of providing the environment with the ability to execute complex activities. Initially, task planning techniques based on basic models of artificial intelligence [2]; proprietary architectures [3], based on configurable Markov Decision Process (MDP) planners [4]; and knowledge based planners [5] have been proposed. Systems with such planning techniques can execute task planning and establish a connection between the sensor-actuator level and the behavior level. ...
Full-text available
There are numerous problems concerning the concepts of the Internet of Robotic Things (IoRT). Two notions are particularly challenging to achieve: heterogeneity and interoperability. A new architecture is proposed here to solve these problems. The goal of this study is to examine the design and development of a completely new architecture that integrates a range of components and robots into an intelligent environment. The following elements of the architecture are described, the connection with the environment, monitoring, the planning system, and the knowledge base of the system. In the main part, the article analyzes how the architecture generates plans from pre-established knowledge through ontologies and how it prioritizes certain plans over others. In this context, the structure within the ontologies is detailed, as well as their operation, how they can include relevant user information, and their assistance in generating plans. For the purpose of evaluating the architecture, the outcomes of two cases are presented in a virtual scenario and through a series of activities whose performance is examined in terms of time and priority.
... Our work connects also with research on control design for AUVs for adaptive ocean sampling, Yoerger and Slotine (1985); Frazzoli et al. (2002); Low et al. (2009); Rudnick and Perry (2003); Yuh (2000); Frank and Jónsson (2003); Graver (2005); Barnett et al. (1996); Carreras et al. (2000); Ridao et al. (2000); Rosenblatt et al. (2002); Turner and Stevenson (1991); Whitcomb et al. (1999Whitcomb et al. ( , 1998McGann et al. (2008b), McGann et al. (2008a), McGann et al. (2008c, Slotine (1985)-McGann et al. (2008c). Applications of ocean sampling techniques for autonomous vehicles are discussed in Singh et al. (1997)-Eriksen et al. (2001. ...
Full-text available
Prediction and estimation of phenomena of interest in aquatic environments are challenging since they present complex spatio-temporal dynamics. Over the past few decades, advances in machine learning and data processing contributed to ocean exploration and sampling using autonomous robots. In this work, we formulate a reinforcement learning framework to estimate spatio-temporal fields modeled by partial differential equations. The proposed framework addresses problems of the classic methods regarding the sampling process to determine the path to be used by the agent to collect samples. Simulation results demonstrate the applicability of our approach and show that the error at the end of the learning process is close to the expected error given by the fitting process due to added noise.
... In contrast to promises, which we also intend as a model of constraints to be fulfilled, those expectations are used for discrepancy detection rather than multi-agent coordination. The Teleo-Reactive Executive (T-REX) architecture (McGann et al. 2008) is a goal-oriented system that employs multiple levels of reasoning abstracted in reactors, each of which operates in its own functional and temporal scope (from the entire mission duration to second-level operations). Reactors on lower levels manage the execution of subgoals generated at higher levels, working in synchronised timelines which capture the evolution of state-variables over time. ...
Full-text available
Reasoning and planning for mobile robots is a challenging problem, as the world evolves over time and thus the robot's goals may change. One technique to tackle this problem is goal reasoning, where the agent not only reasons about its actions, but also about which goals to pursue. While goal reasoning for single agents has been researched extensively, distributed, multi-agent goal reasoning comes with additional challenges, especially in a distributed setting. In such a context, some form of coordination is necessary to allow for cooperative behavior. Previous goal reasoning approaches share the agent's world model with the other agents, which already enables basic cooperation. However, the agent's goals, and thus its intentions, are typically not shared. In this paper, we present a method to tackle this limitation. Extending an existing goal reasoning framework, we propose enabling cooperative behavior between multiple agents through promises, where an agent may promise that certain facts will be true at some point in the future. Sharing these promises allows other agents to not only consider the current state of the world, but also the intentions of other agents when deciding on which goal to pursue next. We describe how promises can be incorporated into the goal life cycle, a commonly used goal refinement mechanism. We then show how promises can be used when planning for a particular goal by connecting them to timed initial literals (TILs) from PDDL planning. Finally, we evaluate our prototypical implementation in a simplified logistics scenario.
... Guzmán et al. (2012) presents PELEA, a domain-independent, online execution architecture which performs planning at two different levels, high and low, and is able to learn domain models, low-level policies and planning heuristics. McGann et al. (2008) proposes T-REX, an online execution system used to control autonomous underwater vehicles. This system partitions deliberation across a set of concurrent reactors. ...
In this work we propose a planning and acting architecture endowed with a module which learns to select subgoals with Deep Q-Learning. This allows us to decrease the load of a planner when faced with scenarios with real-time restrictions. We have trained this architecture on a video game environment used as a standard test-bed for intelligent systems applications, testing it on different levels of the same game to evaluate its generalization abilities. We have measured the performance of our approach as more training data is made available, as well as compared it with both a state-of-the-art, classical planner and the standard Deep Q-Learning algorithm. The results obtained show our model performs better than the alternative methods considered, when both plan quality (plan length) and time requirements are taken into account. On the one hand, it is more sample-efficient than standard Deep Q-Learning, and it is able to generalize better across levels. On the other hand, it reduces problem-solving time when compared with a state-of-the-art automated planner, at the expense of obtaining plans with only 9% more actions.
... While there are several architectures that support plan execution for AUV missions (McGann et al. 2008;Marques et al. 2017;Py et al. 2016), we use ROSPlan (Cashmore et al. 2015) which connects the widely used Robot Operating System (ROS) (Quigley et al. 2009) and PDDL2.1. ROSPlan allows different task planners to be embedded in a modular architecture, making it suitable for testing plan feasibility and quality while varying the underlying planning approach. ...
Conference Paper
Full-text available
Heterogeneous multi-robot systems offer the potential to support complex missions, such as those needed for persistent autonomy in underwater domains. Such systems enable each robot to be optimised for specific tasks to better manage dynamic situations. In this context, temporal planning can generate plans to support the execution of multi-robot missions. However, the task distribution quality in the generated plans is often poor due to the strategies that existing planners employ to search for suitable actions, which do not tend to op-timise task allocation. In this paper, we propose a new algorithm called the Decentralised Heterogeneous Robot Task Allocator (DHRTA) which enhances goal distribution by considering task spatial distribution, execution time, and the capabilities of the available robots. DHRTA is the first phase of our decentralised planning strategy which supports individual robot plan generation using temporal planners. Experiments illustrate the robustness of the approach and indicate improvements in plan quality by reducing the planning time, mission time and the rate of mission failures.
... Research on AUV operations has focused on mission and trajectory planning [93], [94], autonomy architectures [95], [96], and cooperative deployments [97]- [99]. Eichhorn et al. [100] developed a mission planning framework for gliders in dynamic ocean current scenarios. ...
The oceans have a major impact on the planet: they store 28% of the CO 2 pro- duced by humans, they act as the world’s thermal damper for temperature changes, and more than 17, 000 species call the deep oceans their home. Scientific drivers, like climate change, and commercial applications, like deep sea fisheries and underwater mining, are pushing the need to know more about oceans at depths beyond 1000 meters. However, the high cost associated with autonomous underwater vehicles (AUVs) capable of operating beyond the depth of 1000 meters has limited the study of the deep ocean. Traditional AUVs used for deep-sea navigation are large and typically weigh up- wards of 1000-kgs, thus requiring careful planning before deployment and multi- person teams to operate. This thesis proposes the use of a new vehicle design based around a low-cost oceanographic glass sphere as the main pressure enclosure to reduce its size and cost while maintaining the ability for deep-sea operation. This novel housing concept, together with a minimal sensor suite, enables environmental research at depths previously inaccessible at this price point. The key characteristic that enables the cost reduction of this platform is the removal of the Doppler velocity log (DVL) sensor, which is replaced by optical cameras. Cameras allow the vehicle to estimate its motion in the water, but also enable scientific applications such as identification of habitat types or population density estimation of benthic species. After each survey, images can be further processed to produce full, dense 3D models of the survey area. While underwater optical cameras are frequently placed inside pressure housings behind flat or domed viewports and used for visual navigation or 3D reconstructions, the underlying assumptions for those algorithms do not hold in the underwater domain. Refraction at the housing viewport, together with wavelength-dependent attenuation of light in water, render the ubiquitous pinhole camera model invalid. This thesis presents a quantitative evaluation of the errors introduced by underwater effects for 3D reconstruction applications, comparing low- and high-cost camera systems to quantify the trade-off between equipment cost and performance. Although the distortion effects created by underwater refraction of light have been extensively studied for more traditional viewports, the novel design proposed necessitates new research into modeling the lensing effect of this off-axis domed viewport. A novel calibration method is presented that explicitly models the effect of the glass interface on image formation based on the characterization of optical distortions. The method is capable of accurately finding the position of the camera within the dome and further enables the use of deconvolution to improve the quality of the taken image. Finally, this thesis presents the validation of the designed vehicle for optical surveying tasks and introduces a end-to-end ocean mapping pipeline to streamline AUV deployments, enabling efficient use of time and resources.
This paper presents CSTNU Tool , a Java library for representing and checking different kinds of temporal constraint networks. In particular, CSTNU Tool offers an optimized implementation of some constraint-propagation algorithms to check the dynamic consistency/controllability (DC) of Conditional Simple Temporal Networks (CSTNs), Conditional Simple Temporal Networks with Uncertainty (CSTNUs), and Flexible Simple Temporal Networks with Uncertainty (FTNUs). The optimization is with respect to the management of labeled values that are present in conditional and flexible networks. The library offers also a simple GUI application to build/manage and check temporal networks in an intuitive way, and some Java programs for generating random temporal networks according to some input parameters.
Full-text available
Physical-biological couplings impacting phytoplankton ecology are examined with synoptic, high-resolution observations of Monterey Bay, California. Influences of submarine canyon and shelf break topography on the physical-biological couplings are supported by 2 case studies. In the first case study, benthic-pelagic coupling was observed in southern shelf waters where a turbid plume extended from the bottom at ~60 m deep to the base of a phytoplankton layer centered at ~10 m deep. The alongshelf scale of the plume ranged from ~5 km near the bottom to ~1 km at its intersection with the phytoplankton layer. In situ and remote sensing data support the influence of Monterey Canyon on circulation forcing the benthic-pelagic coupling. In the second case study, a frontal zone and adjacent waters were rapidly surveyed over ~20 km 2 of the northern shelf. The front was associated with an isopycnal ridge/trough structure, surface slick, and frontal eddy
Conference Paper
Full-text available
This paper reports on the design and implementation of a real-time executive for a mobile rover that uses a model-based, declarative approach. The control system is based on the Intelligent Distributed Execution Architecture (IDEA), an approach to planning and execution that provides a unified representational and computational framework for an autonomous agent. The basic hypothesis of IDEA is that a large control system can be structured as a collection of inter-acting agents, each with the same fundamental structure. We show that planning and real-time response are compatible if the executive minimizes the size of the planning problem. We detail the implementation of this approach on an exploration rover (Gromit, an RWI ATRV Junior at NASA Ames) pre-senting different IDEA controllers of the same domain and comparing them with more classical approaches. We demon-strate that the approach is scalable to complex coordination of functional modules needed for autonomous navigation and exploration.
Full-text available
NASA's Earth Observing One Spacecraft (EO-1) has been adapted to host an advanced suite of onboard autonomy software designed to dramatically improve the quality and timeliness of science-data returned from remote-sensing missions. The Autonomous Sciencecraft Experiment (ASE) enables the spacecraft to autonomously detect and respond to dynamic scientifically interesting events observed from EO-1's low earth orbit. ASE includes software systems that perform science data analysis, mission planning, and run-time robust execution. In this article we describe the autonomy flight software, as well as innovative solutions to the challenges presented by autonomy, reliability, and limited computing resources.
Standard equations of motion are presented for use in submarine simulation studies being conducted for the U. S. Navy. The equations are general enough to simulate the trajectories and responses of submarines in six degrees of freedom resulting from various types of normal maneuvers as well as for extreme maneuvers such as those associated with emergency recoveries from sternplane jam and flooding casualties. Information is also presented pertaining to the hydrodynamic coefficients and other input data needed to perform simulation studies of specific submarine designs with the standard equations of motion.
In late August and early September 2006, the MBARI Mapping Autonomous Underwater Vehicle (AUV) was deployed for 5 missions on Axial Seamount during a NOAA NeMO cruise on the R/V Thompson. The objective of the survey was to determine the geologic history of the summit of Axial Seamount using high resolution multibeam, sidescan, and sub-bottom profiler data. The Mapping AUV is a torpedo-shaped, 6000 m rated vehicle designed and constructed by MBARI. The vehicle is equipped with a 200 kHz multibeam sonar, 110 kHz and 410 kHz chirp sidescan sonar, and a 2-16 kHz sweep chirp sub-bottom profiler. The multibeam provides a 120-degree swath with 0.94 degree by 0.94 degree beam resolution. The endurance of the AUV is eight hours at 3 knots. Navigation derives from an inertial navigation system (INS) incorporating a ring laser gyro aided by GPS at the surface and by velocity-over- ground observations from a Doppler velocity log (DVL) when within 130 m of the seafloor. A navigational precision of 0.05 percent of distance traveled is achieved with continuous DVL bottom lock. An acoustic modem allows surface aiding of navigation during deep descents. The AUV ran two types of missions: those on the rim of the caldera were run at 90 m altitude with a line spacing of 250 m and those on the caldera floor were run at 50 m altitude with a line spacing of 150 or 175 m. The surveys covered most of 1998 lava flow on the south rim of the caldera and northern part of the south rift zone, the southern region of the caldera floor where hydrothermal vents are common, the northeast rim of the caldera where volcaniclastic deposits related to caldera collapse drape the surface, the north rift zone, and the northern portion of the caldera floor. The low-altitude maps have a resolution of 1 m, so large individual lava pillars and hydrothermal chimneys can be seen, fissures stand out clearly, and the regions of collapsed lobate flows and lava channels are prominent. Many of the flows, including the 1998 flow, have drained lava ponds and channel systems that closely resemble on-land flow channels. Three ROV ROPOS dives explored the channel margins of lobate flows and their abundant pillars and located new sulfide chimneys based on the new maps. The northeast rim of the caldera is cut by numerous fissures, some of which served as eruptive vents for post-caldera flows that are on top of volcaniclastic deposits. The CHIRP sub-bottom profiler did not resolve the roughly 2-m thick deposits observed and sampled with a vibracorer during a recent ROV Tiburon dive. The north rift zone is a constructional ridge that is cut by numerous fissures within 1.6 km of the caldera. Northeast trending fissures are located about 1-1.5 km east of the north rift zone. A lobate flow along the base of the northeastern caldera wall has 9 collapse pits, 10-50 m across and 7-13 m deep, aligned like skylights along a tube system on subaerial flows. Nearby to the northwest, there is a flat lava pond with a bathtub ring of lava around it that is 3 m shallower than the pond surface. The 900 m wide flat surface and the rim are both tilted with the side closest to the center of the caldera about 1.5 m shallower than the northeastern corner of the pond. Fifteen round 5 m tall, 10-40 m diameter mounds of unknown origin are scattered on the flow surface. The AUV generates high-resolution maps for geologic analysis not possible using previously existing mapping systems. Such maps will facilitate the next generation of detailed geologic studies of the seafloor.
An Autonomous Underwater Vehicle designed for operation at high latitudes and under ice completed its first Arctic field tests from the USCGC Healy in fall of 2001. The ALTEX AUV has been under development since 1998, and is being created to provide: unprecedented endurance, ability to navigate at high latitudes, a depth rating of 1500 to 4500 meters depending on payload, and the capability to relay data through the ice to satellites via data buoys. The AUV's initial applications are focused on tracking the warm Atlantic Layer inflow - the primary source of seawater to the Arctic Ocean. Consequently the primary payloads are twin pumped CTD systems. Oxygen and nitrate sensors provide the ability to use NO as a tracer. An ice profiling sonar allows the AUV to estimate the ice thickness in real-time and is designed to generate high quality post-processed ice draft data comparable to that collected through the SCICEX program. The experiments in October aboard the USCGC Healy generated numerous water column and under-ice data sets. Traditional ship-based CTD operations were used to provide a comparison data set for AUV water column measurements. The post-processed ice draft results show reasonable ice profiles and have the potential, when combined with other science data collected, to shed some additional light on upper water column processes in ice-covered regions. Cruise results include: operating the AUV from the USCGC Healy in the ice pack, demonstrating inertial navigation system performance, obtaining oceanographic sections with the AUV, obtaining ice draft measurements with an AUV born sonar, and testing the data-buoy system. This work is supported by the National Science Foundation under grant NSF-OPP 9910290. The Packard Foundation and the Office of Naval Research have also provided support. The project was initiated under the National Ocean Partnership Program under contract N00014-98-1-0814.
During the 1990s, numerous worldwide research and development activities have occurred in underwater robotics, especially in the area of autonomous underwater vehicles (AUVs). As the ocean attracts great attention on environmental issues and resources as well as scientific and military tasks, the need for and use of underwater robotic systems has become more apparent. Great efforts have been made in developing AUVs to overcome challenging scientific and engineering problems caused by the unstructured and hazardous ocean environment. In the 1990s, about 30 new AUVs have been built worldwide. With the development of new materials, advanced computing and sensory technology, as well as theoretical advancements, R&D activities in the AUV community have increased. However, this is just the beginning for more advanced, yet practical and reliable AUVs. This paper surveys some key areas in current state-of-the-art underwater robotic technologies. It is by no means a complete survey but provides key references for future development. The new millennium will bring advancements in technology that will enable the development of more practical, reliable AUVs.