IEEE TRANSACTIONS ON ROBOTICS, VOL. 26, NO. 4, AUGUST 2010619
Evolutionary Trajectory Planner for Multiple UAVs
in Realistic Scenarios
Eva Besada-Portas, Luis de la Torre, Jes´ us M. de la Cruz, Member, IEEE, and Bonifacio de Andr´ es-Toro
Abstract—This paper presents a path planner for multiple un-
manned aerial vehicles (UAVs) based on evolutionary algorithms
(EAs) for realistic scenarios. The paths returned by the algorithm
fulfill and optimize multiple criteria that 1) are calculated based
on the properties of real UAVs, terrains, radars, and missiles and
2) are structured in different levels of priority according to the
selected mission. The paths of all the UAVs are obtained with the
multiple coordinated agents coevolution EA (MCACEA), which is
a general framework that uses an EA per agent (i.e., UAV) that
share their optimal solutions to coordinate the evolutions of the
EAs populations using cooperation objectives. This planner works
offline and online by means of recalculating parts of the original
path to avoid unexpected risks while the UAV is flying. Its search
space and computation time have been reduced using some special
operators in the EAs. The successful results of the paths obtained
in multiple scenarios, which are statistically analyzed in the paper,
and tested against a simulator that incorporates complex models
of the UAVs, radars, and missiles,make us believe that this planner
could be used for real-flight missions.
Index Terms—Aerial robotics, multiobjective evolutionary algo-
rithms (EAs), path planning for multiple mobile robot systems.
autonomously based on preprogrammed flight plans . The
autonomy level achieved by the UAVs depends on the method-
ology used to control the vehicle and to generate its routes. Al-
though both tasks are equally important (and become especially
NMANNED aerial vehicles (UAVs) are aircrafts without
onboard pilots that can be remotely controlled or can fly
Manuscript received December 1, 2009; revised March 22, 2010; accepted
April 13, 2010. Date of publication May 24, 2010; date of current version
August 10, 2010. This paper was recommended for publication by Associate
Editor G. Antonelli and Editor J.-P. Laumond upon evaluation of the review-
ers’ comments. This work was supported by the Community of Madrid under
Project “COSICOLOGI” S-0505/DPI-0391, by the Spanish Ministry of Educa-
tion and Science under Project DPI2006-15661-C02-01 and Project DPI2009-
14552-C02-01, and by the European Aeronautic Defense and Space Company
work of E. Besada-Portas was supported by the Spanish Postdoctoral Grant
EX-2007-0915 associated with the Prince of Asturias Endowed Chair of the
University of New Mexico, University of New Mexico, Albuquerque, NM. This
paper was presented in part at the Genetic Evolutionary Computation Confer-
ence, Atlanta, GA, 2008, and in part at the 8th International FLINS Conference,
Madrid, Spain, 2008.
tamento de Computadores y Autom´ atica, Universidad Complutense de Madrid,
Ciudad Universitaria, Madrid 28040, Spain (e-mail: email@example.com;
L. de la Torre is with the Department of Computer Sciences and Auto-
Digital Object Identifier 10.1109/TRO.2010.2048610
critical when the UAVs must operate in urban terrains ), this
paper focuses only on the second: the development of a planner
capable of solving the route-generation problem by obtaining
a near-optimal path in every possible situation that UAVs can
The advances in microcontroller design, optimization tech-
niques, and control theory increment the number of fields,
both civil and military, where UAVs are currently being used:
surveillance, reconnaissance, geophysical survey, environmen-
tal and meteorological monitoring, aerial photography, search-
and-rescue tasks, etc. Although all these missions seem differ-
ent, the optimality of a route for any of them can be defined
by different optimization planning criteria (such as minimal fly-
ing time and/or path length) and fulfillment of some mission
constraints (such as flying at a given altitude or visiting some
points). Besides, the physical characteristics of the UAVs and
the environment also restrict the feasibility of any route and
should be considered by a realistic planner.
type of threat and restricted zone. Therefore, the best routes are
imposed by the proposed mission, the physical characteristics
of the UAVs, and the environment. The original routes that are
obtained offline by the planner are not always valid in dynamic
environments where the position of all the threats is not known
beforehand. Therefore, the planner must be able to also work
online in order to propose a new path during the UAVs mission
when a pop-up (i.e., unknown threat) appears.
Finding the optimal solution to the route-planning problem is
as the size of the problem grows. The problem has been tackled
with different heuristics, such as mixed-integer linear program-
ming – and A* –, and with nonlinear program-
ming . The planners based on those techniques work with
a simplified version of the original problem, where the addi-
tion of new constraints or objective criteria is a difficult task.
They consider only point-mass dynamics, discretize the solu-
tion space, and, in some cases –, linearize the models.
Although the majority solve the single-UAV case, Richards and
How  and Raghunathan et al.  tackle the multi-UAV
Evolutionary algorithms (EAs) are versatile optimizers that
have already been used to solve different UAV path-planning
problems: Mittal and Deb , Nikolos et al. , ,
Hasircioglu et al. , Pehlivanoglu et al. , and Nikolos
1552-3098/$26.00 © 2010 IEEE
620IEEE TRANSACTIONS ON ROBOTICS, VOL. 26, NO. 4, AUGUST 2010
and Tsourvelouds  optimized the paths of UAVs flying over
a terrain, and Zheng et al.  and Zhang et al.  searched
for optimal UAV paths in military missions. All formulated the
problem as finding the trajectory that minimizes and fulfills a
set of optimization indexes and constraints, and while Mittal
and Deb , Nikolos et al. , Hasircioglu et al. ,
Pehlivanoglu et al. , and Zhang et al.  solved the single-
UAV problem, Nikolos et al. , Nikolos and Tsourvelouds
, and Zheng et al.  solved the multi-UAV one.
This paper presents our evolutionary planner to solve a
multi-UAV route-planning and cooperation problem. It extends
– and our previous works, which are presented in 
and , as follows.
1) Using complex models and new optimization criteria and
response of the planner, an extremely simple model of
the UAVs, terrain, and threats is used in –, and
the performance of the planner is tested against the same
model instead of a realistic complex simulator. However,
in , , and in this paper, the routes are evaluated
according to the properties of a realistic aircraft for the
UAVs, real maps of the world for the terrain, and the
characteristics of the radars and missiles of the complex
simulator used to test the results of our planner. Besides
the UAVs position that is considered in –, we also
the fuel consumption, and the risk of the trajectories. Not
only do we check that the UAVs do not collide against
the terrain, as in –, but we also use maps of any
part of the world to represent it, as in  and , and
to inhibit the radar detection’s capability when UAVs fly
hidden behind mountains. We calculate the probability to
detect and/or to destroy an UAV based on the range of
detection, the tracking probability, and the fire range of
the three types of radars and missiles considered in our
simulator. Our planner also supports the definition of pro-
hibited zones (i.e., no flying zones, NFZs), which UAVs
have to avoid due to mission restrictions. Finally, this is
the first of our papers that introduces the mathematical
models used in all our work.
2) Implementing a new codification for the trajectories. Al-
though 3-D trajectories can be defined pointwise as the
sian points (x, y, z), as in  and , smoother curves,
which are easier to follow by the UAVs, can be obtained
with other representations . Therefore, the list of
3-D points determines the B-spline curve followed by
each UAV in –, , and , a Bezier curve
in , and a cubic spline (which lets us include in the
trajectory all the fixed points that each UAV has to visit,
and not only one, as in ) in our study, i.e., in 
and . Besides, the EAs in – codify the 3-D
points with absolute Cartesian coordinates, while the EAs
in , , , and  use relative polar coordi-
nates for (x, y). Using relative polar coordinates signif-
icantly reduces the search space; however, the mutation
and crossover steps produce global changes in the trajec-
tory instead of the local ones of the absolute Cartesian
codification. Our new codification takes advantage of all
the approaches: We use a list of 3-D absolute Cartesian
polar coordinates, to define cubic-spline trajectories.
3) Using a general multiobjective evaluation method based
on goals and priorities. The evaluation method that com-
bines the different objectives and constraints used in
– does not provide an easy support to modify
the priorities of the objectives and their goals for different
ityused.Theplanners ofand ,andthispaper
use the generic multiobjective Pareto-evaluation function
with goals and priorities presented in , which easily
permits to change the levels of priority of the different
objectives as well as to include new objectives and con-
straints. This way, our planners can be used for different
types of missions very easily.
4) Developing a general evolutionary framework to optimize
the behavior of multiple cooperating agents. The multiple
path-generation problem is solved using, for each UAV,
an EA that optimizes its path based on the objectives and
constraints associated with that UAV and some coopera-
tion objectives related with the others. As the evaluation
of the cooperation objectives requires some knowledge of
the optimal solutions of all the UAVs, all the EAs need to
run simultaneously and send some information of the op-
timal solution they have obtained so far to the remaining
particular case of this work, means coordination between
UAVs) is obtained by introducing coordination objectives
and sharing information between the EAs. The result of
thisapproach isamultiplecoordinated agents coevolution
EA (MCACEA), which is a general evolutionary frame-
work to optimize some characteristics (i.e., the path in our
problem) of multiple cooperating agents. MCACEA can
obtain the path of a single UAV (as in –, and )
or the paths of several cooperating UAVs that have to be
flying simultaneously without colliding (as in –).
Although the core idea of MCACEA is shortly described
more challenging problems that illustrate its performance
specific features with standard efficient EA operators,
whose parameters are tuned after a statistical perfor-
our EAs also include some special features, such as the
option of forcing the UAVs to fly at constant altitude and
the initialization of the algorithm with previous solutions.
The substitution method based on hypercubes , which
is used in  and , is replaced by the more efficient
one that is included in the second version of the Non-
dominated Sorting Genetic Algorithm (NSGA-II) .
Besides, some of the EA parameters are selected using
the statistical performance analysis based on dominance
BESADA-PORTAS et al.: EVOLUTIONARY TRAJECTORY PLANNER FOR MULTIPLE UAVs IN REALISTIC SCENARIOS 621
statistical difference between the results of the optimiza-
that is used periodically over the best solutions in our pre-
vious work has made us eliminate it as well. Therefore,
the improved EAs that constitute our current multi-UAV
planner are computationally quicker and more standard.
and online. The offline paths are obtained by the multiple EAs
suddenly pops up, the EAs of the affected UAVs start working
online to find a new path that avoids the threat.
The rest of this paper is organized as follows. Section II de-
scribes the problem by means of the mathematical models used
to define the missions. Sections III–VI, which are, respectively,
related with points 2)–5) of this section, explain the different
parts of our EA planner. Section VII presents the results of our
planner in different scenarios, and Section VIII contains the fi-
nal discussion and conclusions. Finally, the Appendix contains
the most-used acronyms.
II. PROBLEM DESCRIPTION
The military missions of the optimization problem that are
solved by our EA planner are defined by a set of optimization
planning criteria and constraints, which include the minimiza-
tion of the risk of destruction of the UAVs as well as the restric-
tions imposed by the UAVs’ dynamics and the environment.
Currently, our problem considers 11 objectives. The first ten
Six are constraints, while the other four are values that should
be minimized in order to optimize the solution. The last, i.e., the
eleventh one, is a cooperation constraint that checks the paths
feasibility when all the UAVs are flying simultaneously.
Their values are calculated over the UAV trajectories, which
are cubic-spline curves obtained from the list of 3-D Cartesian
waypoints used to codify the solutions of each EA; for more
details, see Section III. To be able to evaluate their values in
the computer, the continuous-spline curves are discretized in
N points, whose values in the absolute Cartesian-coordinate
system are (xi,yi,zi). The evaluation process uses the dis-
the optimization scenarios: the terrain, prohibited flying zones,
radars, missiles, and UAVs. The characteristics of our radars
and missiles have been estimated after many simulations over
our final test bench. The properties of our UAVs are obtained
plete nonlinear dynamic model. Therefore, they are only valid
in standard flight conditions, such as UAV velocity close to
Their 11 mathematical models are presented next, with the
variables in the International System of Measurements.
1) Minimum turning radius: The UAV maneuverability is
constrained by its minimum turning radius Rmin
depends on the maximum load factor nmax
and velocity v. For each point of the trajectory, Rmin
, altitude zi,
are obtained as follows:
= 5.3809 × 10−9z2
i− 4.4291 × 10−4zi+ 6.1000
where g is the gravity. All the trajectory points, whose
turning radius Riis smaller than the minimum permitted
minimum value (i.e., zero) ensures the fulfillment of the
constraint. Riis calculated as the radius of the circumfer-
ence that is defined by the points i − 1, i, and i + 1
strained by its maximum climbing slope αiand its mini-
mum gliding slope βi, which, in our case, depend on its
altitude zi. These slope limits are calculated with
αi= −1.5377 × 10−10z2
i with c1
i− 2.6997 × 10−5zi+ 0.4211
βi= 2.5063 × 10−9z2
i− 6.3014 × 10−6zi− 0.3257.
The UAV slope at the ith point (i.e., Si) is obtained with
Similarly to the previous constraint, all the trajectory
points, whose slope is out of the permitted range, are pe-
nalized using the following expression, whose minimum
value (i.e., zero) is the optimal for this constraint:
3) Fuel: UAVs carry a limited quantity of fuel and they have
to reach their destination before consuming all of it.
The fuel consumption for each trajectory point is calcu-
lated based on the expressions of the following three basic
a) The fuel consumption when the UAV is flying hori-
zontally and straight (i.e., FCH
(xi+1− xi)2+ (yi+1− yi)2.
i with c2
βi< Si< αi
), which is dependent
b) The fuel consumption when the UAV is flying at max-
imum slope (i.e., FMS
), which is also dependent on
− 4.259 × 10−3zi+ 87.881.
= 1.6679 × 10−11z3
i− 2.4832 × 10−7z2
622IEEE TRANSACTIONS ON ROBOTICS, VOL. 26, NO. 4, AUGUST 2010
c) The fuel consumption when the UAV is flying with
maximum bank angle (i.e., FMT
+ 16.552 × nmax
The real cases are considered as a combination of the
previous cases, where only the main contributions that
appear in each type of maneuver are taken into account.
a) When the UAV is ascending, the real consumption
at constant height (i.e., FCH
), as well as on the discrepancy between the
actual and maximum slopes (i.e., Siand αi)
b) When the UAV is turning but not ascending, the fuel
tion at constant height (i.e., FCH
turning angle (i.e., FMT
), as well as on the ratio be-
tween the actual and maximum load factors (i.e., ni
where nican be obtained from (1), using Riand ni
instead of Rmin
c) Finally, when the UAV flies straight and descending,
the fuel consumption (i.e., FCCH
the one for horizontal and straight-flight conditions
. This assumption is conservative since in this
situation, the consumption is lower.
Considering that between two points of the trajectory, the
fuel consumption is constant, the total fuel consumption
is calculated as the summation of the product of the fuel
needed at point i (i.e., FCcase(i)
go from i to i + 1 (∆ti), which is estimated as follows:
), which is dependent
= −3.0435 × 10−1(nmax
i) depends basically on the fuel consumption
) and at maximum slope
) and at maximum
) is approximated by
) and the time needed to
(xi+1− xi)2+ (yi+1− yi)2+ (zi+1− zi)2
The total fuel consumption value is limited by the fuel
capacity of the UAV (i.e., Fuel). The following expres-
sion only penalizes the constraint when the consumption
exceeds the capacity. Its minimum value (i.e., zero) is ob-
tained when the fuel consumption is less than the fuel
capacity. A positive value implies that the consumption is
higher than the capacity. Therefore, the following expres-
sion calculates the extra fuel that is needed to follow the
with case(i) =
when UAV ascends
when UAV turns but
does not ascend
4) Terrain: A feasible path cannot go through the terrain
and has to avoid collisions with mountains. To ensure this
behavior, the algorithm penalizes the solutions that have
at least one point of the spline trajectory inside the terrain.
If map(xi,yi) is the function that returns the altitude of
the terrain at any point (xi,yi), the following expression,
which calculates the number of points that are inside the
terrain, is used to penalize this constraint:
i with c4
5) Map limits: To penalize the points of the trajectory that
are not inside the map limits, (16), shown below, is used,
for the “x” coordinate, and ym
lent for the “y” coordinate. Its minimum value (i.e., zero)
ensures the fulfillment of the following map constraint:
uare the map lower and upper limits
u are the equiva-
InMap(xi,yi) = (xm
l ≤ xi≤ xm
u) ∧ (ym
l ≤ yi≤ ym
6) Flight-prohibited zones (i.e., NFZs): The user can define
considered high-risk zones, unknown zones, etc.
They are defined as M rectangular regions constrained
by their external limits (xNFZ,j
, with j = 1:M). The N points of the trajectory
have to avoid them. To penalize this objective, for each of
the trajectory points that are inside a NFZ, we accumulate
the distance to the closest NFZ edge, with the purpose of
distinguishing between two solutions that have the same
of their frontiers. The following expression imposes the
selected penalization and the constraint is fulfilled with its
minimal value (i.e., zero):
InNFZ(i,j) = (xNFZ,j
with k = l,u ∧ a = x,y
≤ xi≤ xNFZ,j
≤ yi≤ yNFZ,j
7) Minimum path length: For military applications, shorter
are equal) because they require less time of flight and
BESADA-PORTAS et al.: EVOLUTIONARY TRAJECTORY PLANNER FOR MULTIPLE UAVs IN REALISTIC SCENARIOS 623
Fig. 1. ADU model. (a) Eagle eye view. (b) Cross-section view.
usually have a lower chance of finding an unknown threat
in a dynamic environment.
Instead of minimizing the actual path length, we mini-
mize its value normalized by the minimum flight distance
lmin, because both values are equivalent and the values
of the second represent ratios that are considered admis-
sible. The path length ratio (PLR) is calculated with the
selected by the user to define the mission: the start, inter-
mediate, and ending points of each UAV.
8) Minimum probability of kill: Trajectories that accumulate
lower probability of destruction are safer than those with
higher accumulated probability values.
The probability-of-kill (PKill) function depends on the
model used for the Air Defense Units (ADUs) that are
groups of radars and missiles. For each ith point of
the trajectory, the jth ADU only has a certain possi-
bility to destroy the UAV if the UAV is inside the re-
gion defined by the ADU’s maximum-risk distance (i.e.,
sented in Fig. 1, which also shows the maximum detec-
tion distance (Rj
objective. The three types of ADUs used in our scenar-
ios have different Rj
representation of the probability of destruction inside that
region, which is dependent on the orientation (θj
after many simulations against our test bench. Interpo-
lating their values with the function TabPK(θj
obtain the PKill of the ith point of the trajectory for the
jth ADU. The PKill accumulated along the trajectory due
to the different A active ADUs of the problem is calcu-
lated with the following expression, whose value has to be
PKmax) and minimum- and maximum-risk altitudes
max). The shape of that region is pre-
Dmax) that will be defined in the ninth
min, and Hj
max. A tabular
i) and al-
i) of the UAV relative to the ADU, is obtained
PKill = 1 −
(1 − PKj
InPK(i,j) = (Rij≤ Rj
PKmax) ∧ (Hj
9) Minimum probability of radar detection: When the radars
them (and, therefore, trajectories that accumulate lower
probabilities of detection are safer), but also, the flight of
the UAV and its mission are kept secret.
model used for the ADU’s radars. For each ith point of the
trajectory and jth radar, the detection-probability model
considers the radar cross section RCSij of the UAV, the
and some properties of the radar.
The RCSijdepends on the orientation of the UAV with
semiaxis “a,” “b,” and “c”:
(aαzβφ)2+ (bαzαφ)2+ (cβz)2
with αz= sin(aze
and the segment that joins the UAV and radar positions,
ijis the angle between the velocity of the UAV
ijis given by
ij= φij− arctan
with φijthe roll, elijthe elevation, and azijthe azimuth
between the UAV at position i and the jth ADU.
TheLoS between twopoints(i,j) of thespaceiscalcu-
lated with the function LoS(i,j) that considers the terrain
elevation at the line that joins the two considered points.
Although this function significantly increments the com-
the terrain inhibits the radar effects.
The PRD accumulated along the trajectory due to the
radars of the different A active ADUs is calculated with
(22), whose value should be minimized. Rj
maximum detection distance of the jth ADU, and ζj
624 IEEE TRANSACTIONS ON ROBOTICS, VOL. 26, NO. 4, AUGUST 2010
(i.e., 50 km)
2are parameters that are dependent on the model of its
PRD = 1 −
(1 − DPj
if Rij> Rj
1 + ζj
10) Minimum flight altitude: UAVs flying at low altitude can
benefit from the terrain-mask effect that will help them
to avoid unknown radars. Although this increases the fuel
bination of objectives 3) and 10) makes the planner search
low-altitude trajectories whose fuel needs are within the
To achieve this objective, we calculate the accumulated
difference between ziand the altitude of the terrain at that
point (i.e., map(xi,yi)) and divide it by the number of
points in the trajectory, i.e., N. Therefore, the algorithm
the elevation of the UAV over the sea level. The following
expression calculates this objective value:
if zi< map(xi,yi)
11) Avoiding UAVs collisions: When generating paths for sev-
eral UAVs, it is important to check whether two UAVs are
getting too close while following their respective paths.
For that, the planner has to test if two paths coincide in
space and time.
With this purpose, given the trajectory of the u UAV
and the v trajectory for the remaining UAVs, every spline-
curve point of the first trajectory is compared with every
point of the others. If the distance duv
point of the trajectory of the uth UAV and the jth point of
the trajectory of the vth UAV is smaller than a minimum
critical value dmin, it is checked if the arrival times to
those points (i.e., tu
risk, which happens if their differences are lower than the
minimum permitted time tmin. The parameters dminand
tmin, which are chosen by the user, specify the requested
safety level. Each dangerous pair of points is penalized
by (24), whose minimum value ensures that the trajectory
of the u UAV does not collide against any of the other v
UAVs trajectories. The indices in Nk, tk
ijbetween the ith
j) are too close to constitute a
l, and zk
are related with the the kth UAV and l trajectory point
ij< dmin∧ |tu
j| < tmin
III. SOLUTION CODIFICATION FOR THE EVOLUTIONARY
EAs work with populations of possible solutions that, in this
problem, codify the possible 3-D trajectories of each UAV.
Our codification is a list of 3-D points that are used to define
the cubic-spline curve  that each UAV has to follow. This
list, whose points are floating-point coded, contains some fixed
to visit and some undetermined points (i.e., waypoints) that the
EA planner calculates to find the feasible and optimal cubic-
spline trajectory for each UAV. The cubic-spline curve, which is
only used inour works,automatically ensures thatthe trajectory
visits all the points in the list.
The offline planner sets the number of waypoints that exists
between each pair of fixed points, which is based on their dis-
online planner adds some new waypoints to those lists whose
paths are affected by the pop-ups and recalculates the values of
all the waypoints placed after the UAVs current positions.
Each waypoint in the 3-D space is defined by three coor-
dinates, whose values can have different meaning depending
on the selected codification. For example, in –, ,
and , these coordinates are the absolute Cartesian “x,” “y,”
and “z,” while in  and , and ours, i.e.,  and ,
they belong to a relative polar-coordinate system, with absolute
“θ” (i.e., angle) relative to the previous point in the list. Fig. 2
shows both codifications for the same cubic-spline curve.
The absolute Cartesian codification generates a huge search
space in real scenarios since any point of the space can be
reached from the previous point of the list. The relative polar-
coordinate system dramatically reduces the search space, be-
cause it forces the EA to have a predefined order of points.
However, any change of the values of the waypoints in the rel-
ative polar codification modifies the whole shape of the part of
the spline curve that comes after it. In the absolute Cartesian
codification, the changes only modify the spline curve locally.
Our current codification takes advantage of both approaches.
We maintain a cubic-spline representation to force the visit of
all the points of the list while the waypoints are codified in
absolute Cartesian coordinates to ensure only local changes in
the crossover and mutation steps. Also, to dramatically reduce
the search space, the initial values of the absolute Cartesian co-
ordinates are obtained from a random process that creates the
radios, angles, and altitudes of the waypoints in relative polar
representation, i.e., the waypoints of a trajectory are created
BESADA-PORTAS et al.: EVOLUTIONARY TRAJECTORY PLANNER FOR MULTIPLE UAVs IN REALISTIC SCENARIOS 625
line) cubic-spline curve defined by the fixed points S, IF, and E and the
Cartesian coordinates) Each waypoint is represented with its (xWi,yWi,zWi)
values. (b) (Relative polar coordinates) Each waypoint is represented with its
(rWi,θWi,zWi) values. rWiis the distance from the previous list point to the
ith waypoint. θWiis the angle change with respect to the angle defined by the
segment that joins the two previous list points (i.e., dashed lines and vectors).
The rWiand θWivalues to any fixed point are fixed by that point and the
Absolute Cartesian and relative polar codifications for the same (solid
from scratch by the offline planner within the polar relative
representation and translated into its absolute Cartesian repre-
sentation, the absolute Cartesian waypoints are manipulated by
the EA operators, and the discretized spline, which is defined
by the fixed points and waypoints, is only used to determine
the goodness of each trajectory. The initialization in the po-
lar relative representation is carried out using uniform random
distributions for the floating-point-coded (r,θ,z) values. The
“z” limits constitute the minimal and maximal altitude that the
UAV can reach, the “r” limits represent the original permit-
ted separation between two consecutive waypoints, and the “θ”
limits are related with the UAV maneuverability. The absolute
Cartesian values that they are translated into are codified as
real floating-point-coded numbers. Finally, the objective values
are obtained with the expressions given in Section II, using the
N (xi,yi,zi) points in which the cubic spline is discretized.
N = p ∗ (NIF+ NW + 1) + 1, where NIFis the number of in-
termediate fixed points, NW the number of waypoints, and p a
user-selected parameter (which is 40 in this paper).
The process of initializing the lists of waypoints of the online
planner, which we will explain using the example presented in
Fig. 3, is different. When a flying UAV following the offline
optimal trajectory identifies a new pop-up, the online planner
starts working. First, the planner finds the points of the list (W3,
W4, W5, W6, E) of the offline trajectory situated after the UAV
position (black filled circle). It also finds the segment (limited
by asterisks) of the UAV-discretized spline trajectory that falls
inside the new pop-up maximum-risk-distance region (dashed
circle) and identifies the waypoints (i.e., W3) it bypasses. Then,
it creates four waypoint sublists (such as the one defined by A,
B, C, and D), whose values, which are generated in the polar
relative representation, define, when translated into the absolute
Cartesian one, a random U-shape around the pop-up Rj
area. Next, it creates the new lists combining in the appropriate
order an origin fixed point (i.e., O) close to the UAV position,
the U-shaped waypoint sublists, and the set of offline waypoints
intersected segments. This way of proceeding gives the online
waypoints) (W#), and start (S) and end (E) points that define the optimal
trajectory obtained by the offline planner, the areas defined by the (dotted
risk distance (i.e., Rj
position when the online planner starts working, and the (asterisks) intersection
of the discretized spline trajectory with the region defined by the maximum-risk
distance (i.e., Rj
of the online trajectory and a U-shaped four waypoint sublist (A, B, C, and D)
created randomly by the online planner to avoid the pop-up Rj
new online 3-D list is defined by (O, A, B, C, D, W4, W5, W6, and E).
Waypoint list online initialization process. (a) (Final optimal offline
PKmax) of a pop-up ADU, (black filled circle) the UAV
PKmax). (b) (Initial online waypoints) Original fixed point (O)
planner a high chance of creating lists (O, A, B, C, D, W4,
W5, W6, and E) that usually avoid the pop-ups to start with.
Finally, their not-fixed-point values are optimized to ensure that
avoiding that the UAV collides against others).
IV. EVALUATION FUNCTION
Our problem has the 11 objectives presented in Section II.
Therefore, the planner needs a function that combines their
values to obtain the fitness of each solution of the population.
method based on goals, priorities, and Pareto sets proposed
in , where the objectives are placed in levels of priority and
where limits (goals) can be imposed on each of them. With this
method, which is already used successfully in other complex
multicriteria constrained problems , , the EA compares
all pairs of solutions in the population, decides if one Pareto
by Goldberg .
To be able to set the priority levels of the 11 objective values,
(i.e., single-UAV objectives) and (II) the cooperation objectives
(multi-UAV objectives). Furthermore, both groups are divided
in two different types: a) constraints that the path of the UAV
has to satisfy completely due to its physical restrictions, the
terrain, or mission considerations and b) optimization indexes
that must be minimized according to the selected mission. The
rows Name and Eq. of Table I show this classification and the
equations that calculate them.
Based on this classification, the objective values used to mea-
objectives whose values must be zero. The objective indexes
to be minimized are located at lower priority levels organized
according to the goal of the mission. With this organization,
solutions that fulfill the constraints are better than the ones that
do not, and the latter are basically organized according to how
far the constraints have been optimized. The rows of Table I
Level, Min, and Max, show a typical election of priority levels
and limits for the objectives. The limits establish the accepted
626 IEEE TRANSACTIONS ON ROBOTICS, VOL. 26, NO. 4, AUGUST 2010
USUAL EVALUATION FUNCTION PRIORITIES AND LIMITS
interval of values for each objective and are tuned by the user
considering the mission goals.
Furthermore, as we are using a general method to evaluate
the fitness of the function given the constraints and objective in-
dexes, we can easily add or remove constraints or optimization
indexes as needed. For instance, we could consider other co-
ordinating constraints, such as the timing restrictions proposed
in  and , which are the simultaneous arrivals of two or
more aircrafts to a selected point, and tight/loose sequencing
(especially important if the mission considers on-air UAV re-
fueling). The autonomous aerial-refueling problem could also
be formulated here in terms of the four parameters presented
in : maximum waiting time, refueling time, return-to-field
optimization index (none is currently considered in our prob-
lem), such as the minimization of the distance between two
different UAVs in order to carry out a tracking mission, could
easily be included.
Although the missions presented in – are also de-
fined as multiobjective problems, their evaluation functions are
significantly less flexible than ours.
Finally, it is important to highlight that the time needed to
evaluate the solutions is especially problematic when the online
planner starts running due to the presence of a sudden pop-up.
In order to speed up the algorithm, the online planner uses a
simpler model of the problem and less optimization indexes.
This difference is justified because the critical objective of the
online planner is to avoid the threat. Therefore, minimizing the
use the same evaluation function, the seven constraints, and
different objective values. The differences in this respect are the
1) The offline planner uses the four optimization indexes
that we want to minimize. Table I shows the configuration
used in the offline planner in all the scenarios presented in
Section VII, except in scenario B1.
PKill and the PLR. PKill is more important than the PLR
when replanning due to the appearance of an unexpected
ADU and, therefore, in the online planner, PLR priority
level is reduced (i.e., set to 3).
V. MULTIPLE UNMANNED-AERIAL-VEHICLES
The previous sections show how to codify the solutions and
evaluate them. However, they do not explain how the planner
solves the single and multiple-UAV trajectory problem. This
section and the following contain that information.
Calculating the paths of several UAVs that are flying simul-
taneously is a complex task due to the space constraints that
one path imposes on the others and the cooperative objectives
of the set of UAVs that the planner has to optimize. The prob-
lem has already been tackled by nonlinear-programming and
mixed-integer linear-programming methods, and the results of
both techniques have been compared in .
Our approach, i.e., MCACEA, uses multiple EAs (one per
each UAV) that evolve their own populations to find the best
path for its associated UAV according to their individual and
cooperation constraints and objective indexes. Each EA is a
single-UAV planner that runs in parallel and that exchanges
information is needed to let each EA measure the coordination
taking into account the possible optimal paths of the remaining
related to the best solutions of the remaining single planners
before evaluating the cooperative objectives of each possible
solution of its own population.
As the cooperation objective values depend on the best solu-
tions of the other populations and the optimality of a solution
depends both on the individual and cooperation objectives, it
is not really possible to select and send the best solution of
each planner to the others. However, we can divide the evalu-
ation step inside each EA in three parts: In the first part, the
EA identifies the best solution considering only the individual
objective values and sends it to the others; in the second part,
the cooperation objective values of all the paths are calculated
taking into account the received information; and in the third
the individual and cooperation objective values. However, there
is another problem that also needs to be solved. Although each
UAV can only follow a unique optimal path, each EA maintains
a Pareto set of optimal paths and selects the unique optimal path
at the end, when the last population has already been obtained
(see Section VI). Therefore, to be able to determine a unique
optimal path according with the individual objectives in each
generation, the step in charge of selecting the final optimal path
is included in the evaluation step of each EA.
BESADA-PORTAS et al.: EVOLUTIONARY TRAJECTORY PLANNER FOR MULTIPLE UAVs IN REALISTIC SCENARIOS 627
EVALUATION STEP IN THE COOPERATING PLANNERS
Table II shows the complete evaluation step of the individual
the path of a single UAV, it only uses the first two steps of this
new evaluation process.
Note that in our evaluation process, the trajectory that is con-
sidered the best in accordance with the individual objectives
is not necessary the best after evaluating the coordination ob-
jectives. In other words, the EAs may sometimes evaluate the
coordination objectives against the “wrong” trajectories. This
could make the EAs have some initial difficulties to converge
to collision-free trajectories, although the results presented in
Section VII show that this problem is usually solved after a few
generations. Another possibility, which is considered in ,
is to send the best completely evaluated trajectory of the pre-
vious generation to the other EAs instead of our current best-
only individual objective evaluated one. Nevertheless, not only
does the approach in  not solve the problem, but, we be-
lieve, it worsens it because it introduces a bias toward outdated
the path of multiple UAVs without considering interdependent
To reduce the bandwidth of the distributed planner, the in-
formation exchanged by the EAs are the 3-D point lists instead
of the pointwise discretized cubic splines. The time used to ex-
change this information among the UAVs’ onboard computers
depends on the UAVs position, environment, and/or commu-
nication technology. The communication delays affect every
EA generation, and therefore, they are especially problematic
for the online planner since it needs to obtain the alternative
routes before the pop-ups destroy the UAVs. To minimize their
effects, when the UAVs are flying, the replanning process is
done sequentially: Only the EA of an UAV is run until it fin-
ishes to optimize its own trajectory considering the trajectories
of the other UAVs constant because their associated EAs are
not working. Therefore, only the final 3-D list of the u UAV is
broadcasted to the others when the u EA has finished after Tu
Which EAs need to be started is determined whenever a new
pop-up is identified, and their initialization order is calculated
based on the time, Tu
i, needed by each u UAV to intersect the
ing part of its current cubic spline. All the UAVs survive, when
0 on time, i.e, when Tv
it is extremely important to minimize Tu
the U-shaped inclusion of waypoints in the 3-D lists described
in Section III and the stop criterion presented in Section VI-F.
Section VII shows an illustrative example, where we analyze
Finally, MCACEA is not only valid for the UAV problem, but
can also be applied to other optimization problems with cooper-
ating agents. Although this way of proceeding may look similar
to the habitual parallelization of EAs (which is used in 
and  to solve UAV problems), in this case, instead of dis-
tributing the solutions of the whole problem between different
EAs that share their solutions periodically, we are dividing the
problem into smaller problems that are solved simultaneously
by each EA taking into account the solutions of the part of the
problems that the other EAs are obtaining.
r. To achieve it, we use
VI. EVOLUTIONARY OPERATORS AND OTHER RELEVANT
CHARACTERISTICS OF THE EVOLUTIONARY ALGORITHM
The basic characteristics of each EA, which are given by
the type of operators and functions implemented inside it, are
presented in this section. The selection, recombination, and mu-
tation methods differ from the ones used in our previous work:
the first two to make it closer to NSGA-II and the last to take
into account the new codification. Besides, the previously used
local-search procedure has been disabled.
The binary tournament method  is used to select Nspairs
of elements from the current population of Npsolutions.
A uniform distribution with crossover probability Pcis used
to decide if each pair of selected solutions (lists with NW way-
points) is mated to create two new solutions or left without
change. The crossing point Cpof each pair of mating solutions
is selected from the interval [1,NW], and the first new solution
is formed with the 1 : Cpwaypoints of the first parent and the
Cp+ 1 : NW of the second, while the second new solution has
the 1 : Cpwaypoints of the second parent and the Cp+ 1 : NW
of the first.
We increment the values of each of the 3 ∗ NW elements of
each solution (list with NW 3-D (xWi,yWi,zWi) waypoints)
with the values obtained from a zero-mean normal distribution
with a small Cmscovariance. Besides, some of the elements,
which are selected according to an uniform distribution with
level incremental mutation lets the EA explore any region of the
628 IEEE TRANSACTIONS ON ROBOTICS, VOL. 26, NO. 4, AUGUST 2010
space and escape from the confined region associated with the
relative polar-representation initialization.
(i.e., immigrants) is created using the random relative polar-
The NSGA-II recombination method  is used to create
the new population selecting a total of Npsolutions from the
and undergone the crossover and mutation processes, and the
immigrants. The population size Np is maintained by means
of preserving all the Pareto sets that fit into Np, with a good
Pareto-front distribution of the elements of the last fitted set.
F. Stop Criterium
Offline EAs stop when the generation limit is reached. On-
line EAs stop after 1) the routes are feasible (i.e., constraints
fulfilled), safe (PKill = 0), and short enough, or 2) the genera-
tion limit is reached.
G. Selection of the Final Solution
We also need a criterion to select at the end, among the
solutions belonging to the first Pareto set, the trajectory that the
of the evaluation function and some additional rules.
For example, for the offline planner and priority levels pre-
sented in Table I, we consider following two cases.
1) There is a feasible solution with PKill = 0. Therefore,
there is only one element in the first Pareto set because
from the two objectives in the secondary priority level
PKill has reached the minimum and PLR determines
which solution is the best.
2) For all the feasible solutions, PKill > 0. We first obtain
the relative PKilliof each ith solution in the first Pareto
solutions with PKreli≥ V (for example, with V = 1.05),
which makes the nondiscarded jth solutions have a PKillj
that is not significantly higher than the minimum. Finally,
we select the solution with minj(PLRj). Therefore, the
final solution has a PKill that is a bit higher than the
minimum, as well as a reasonable PLR.
H. Reuse of Solutions
When an offline EA returns the final solution, the planner
copies the waypoints that codify it to be able to use them for
future optimizations. This lets us reduce the computational time
when we want to find a trajectory over a scenario slightly differ-
also include the results of any algorithm that finds a suboptimal
solution of the problem in the first generation of the EA, as long
as it is codified as described in Section III.
Our planner is part of a complex system that tests the perfor-
mance of UAV planners in hostile environments. This system
lets us 1) set the known and pop-up ADUs positions and the
UAVs fixed points, 2) obtain their routes with a planner (in-
cluding ours), and 3) test them against a simulator that uses
real-world maps and complex UAV and ADU models.
important characteristics. Fig. 4 shows the scenarios (identifi-
able by a capital letter followed by the number of UAVs they
contain) and the final trajectories returned by the planner. The
big dashed gray circles mark each ADU’s maximum distance
of detection (i.e., Rj
resent their maximum-risk distance (i.e., Rj
each circle, respectively, represents the areas that can increment
PRD and PKill, they do not have to do it necessarily because
those values also depend on other factors (see Section II). The
label ADUiidentifies the ith known ADU, and POPithe ith
pop-up ADU. Unlabeled ADUs are always known. The brown
rectangles show the NFZs. Labels S, E, and IF identify the
start, end, and intermediate fixed points of the UAVs. The of-
fline planner trajectories are the solid lines that are not black or
red. Sometimes they are labeled close to its S point as UAVi.
The thick black and red lines in Fig. 4(i) show the alternative
routes calculated, during a simulation where pop-ups are iden-
tified, by the online planner. Note that some paths cover tens of
kilometers, while others hundreds, and therefore, the examples
show the good scalability of our planner. Besides, although we
only show the isometric view of E2 in Fig. 4(h), the planner is
always searching for 3-D routes.
We also present how some tunable parameters of the planner
affect the quality of the trajectories in the different scenarios. To
of parameter values, each planner is run Nrtimes with the same
obtain different trajectories in each execution. The comparison
is carried out using the statistical front-dominance-ranking pro-
of algorithms A and B are statistically significantly different ap-
plying the nonparametric Mann–Whitney rank tests  to the
number of times that each of the Nrfinal best Pareto fronts ob-
tained by algorithm A is dominated by each of the Nrfinal best
Pareto fronts obtained by algorithm B, and vice versa. We pick
SFDRP because it can be directly applied to the single-UAV
problem using the generic multiobjective evaluation method
based on goals, priorities, and Pareto sets (see Section IV) when
calculating the number of times the Pareto fronts are domi-
nated. To use it in the multi-UAV problem, we have to consider
the existence of a different best Pareto front for each UAV. To
overcome this difficulty, the Pareto fronts used by SFDRP are
created from the Cartesian product of the UAVs Pareto fronts.
Therefore, the multi-UAV statistical tests are carried out over
the space of all the possible trajectories for all the UAVs in the
scenario. Finally, it is important to highlight that although our
EAs incorporate a final step that selects a single best trajectory
Dmax), while the small red solid circles rep-
BESADA-PORTAS et al.: EVOLUTIONARY TRAJECTORY PLANNER FOR MULTIPLE UAVs IN REALISTIC SCENARIOS 629
(f) Scenario F3 (intermediate solutions). (g) Scenario D8. (h) Scenario E2 (isometric view). (i) Scenario F3 (final solution + pop-up alternative route).
UAV scenarios and solutions. (a) Scenario A1. (b) Scenario B1 (the priority level of PRD is 2). (c) Scenario C1. (d) Scenario D4. (e) Scenario E2.
for each UAV, SFDRP is carried out over the final best fronts,
because the EAs evolve its own population according to the
fronts and select the best at the end.
The EA parameters and number of waypoints used to obtain
the offline trajectories of Fig. 4 are shown in Tables1III–V. The
mean execution time (i.e., Time) of the planner in a Pentium
IV at 3.0 GHz is also included,2although it is highly dependent
on the scenario and trajectories that the EAs are evaluating. For
instance, evaluating only one point of a trajectory that passes
inside an ADU’s Rj
it is not due to the use of the LoS function. Therefore, the LoS
function is disabled in the online planner, which prevents the
Dmaxregion is 100 times slower than when
1The meaning of the acronyms in the tables is summarized in the Appendix.
2The code has been serialized to be run over a single computer and account
only for the total processing time without measuring communication delays.
UAVs from using the masking effect in the replanned part of the
trajectory, but evicts this costly time consumption.
Section II), which ensures that when PKill = 0 and there are
no pop-up ADUs, the UAV always reaches its final destination.
WhenPKill > 0,theUAVisassumingsomerisk,anddepending
on the simulation, it will succeed or fail with a probability close
to the value obtained by the EA.
A. Results in Single Unmanned-Aerial-Vehicle Scenarios
These scenarios, which are presented in the top row of Fig. 4,
show the planner’s ability to find no risky paths in static envi-
ronments with different number of ADUs. In scenario A1 [see
Fig. 4(a)], there is a little corridor (of 8 km) between ADU1and
ADU2, where PKill = 0, which the planner always finds. How-
ever, the UAV is detected, because PRD is in the third priority
630 IEEE TRANSACTIONS ON ROBOTICS, VOL. 26, NO. 4, AUGUST 2010
PLANNER-TUNABLE PARAMETERS UNDER TEST
level, while PLR is in the second, and therefore, the planner
primes shorter paths to nondetectable ones. Scenario B1 [see
Fig. 4(b)] minimizes PRD as well, as we increment the priority
level of this objective before running the planner. Therefore, the
planner now finds paths adjacent to the dashed gray circles. Fi-
nally, scenario C1 [see Fig. 4(c)] shows the planner’s ability to
find trajectories in areas with many ADUs, where the PKill = 0
area has a complex shape.
We use SFDRP in these scenarios to see how the values
of some tunable parameters of the EAs affect the solutions.
Table III shows the parameters under test (first row), which are
changed one at a time, their nominal value during all the op-
timizations except when they are being changed (second row),
their minimum value (third row), their maximum (fourth row),
the increment used to change them (fifth row), and the range of
values that are considered better after analyzing the results of
SFDRP (sixth row). For each scenario and set of parameter val-
ues, we run the planner Nr= 30 times during 800 generations
erations to be able to perform SFDRP at different stages of the
algorithm. The results of the test show, with a 5% confidence,
that there is no systematic statistical significant difference on
the solutions obtained by the algorithm at the same generation,
when we change Np. Therefore, we pick the smaller value. The
same happens when we change Pm, and therefore, we main-
tain the value used in our previous works. However, SFDRP
shows that using a Pc= 0.7 or Pc= 0.75, or a Ns≥ 12, lets
the EAs obtain fronts less times Pareto-dominated by the fronts
obtained with the other Pcor Nsvalues, and therefore, we pick
Pc= 0.75 (the EA has more crossing opportunities with big-
ger values) and Ns= 12 (smaller values reduce computational
SFDRP is also used to see how the scenario influence the
number of needed generations. In this test, the planner main-
tains the nominal tunable parameters and SFDRP compares the
Nr= 30 best fronts obtained at a given X generation with the
Nr= 30 best fronts obtained at another Y. Therefore, this test
[whose results for scenario A1 are presented in Fig. 5(a)] tells
us if we can (white) or cannot (gray) expect to obtain an overall
better solution when we run the planner once up to generation
Y than when we run it once up to generation X. As one expects,
duces better results up to a given point, which is identified as
GShereafter, after which, no better results should be expected.3
3For lack of space, we can include neither the SPDRP generation graphics
for all scenarios nor the SPDRP-tunable parameter graphics. However, we use
them to obtain the GSof Tables IV and V and the conclusions of Table III.
However, SFDRP cannot quantify the improvements. There-
fore, for each execution of the planner, we measure, in some
generations, the improvement of the PLR4with respect to an-
other generation 50 steps earlier, calculate its mean over the
Nr= 30 executions, and represent it, for the three scenarios, in
Fig. 5(b). These graphics let us identify a generation number,
i.e., GI, after which the planner cannot significantly improve its
actual solution. Therefore, GIis associated with improvements
smaller than a threshold inside one execution of the planner,
while GS is related with overall improvements, but does not
account for their size. Finally, to analyze the convergency of the
planner, we alsorepresent thePLR mean and standard deviation
(Std) values of all the solutions that appear in the best Pareto
fronts of the Nr= 30 optimizations in Fig. 5(c). The amplitude
of values of the ordinate axis is maintained; therefore, all PLR
Std values are presented in the same scale. Table IV also con-
tains the generation in which the constraints are fulfilled, i.e.,
GC; the mean execution time for 50 generations, i.e., Time; the
number of waypoints, i.e., NW; and the number of points of the
discretized spline curve, i.e., N.
Finally, we want to point out that we also use SFDRP to
decide if our new codification is better than the previous and to
disable the local-search procedure used in our previous works.
The results show that the new codification is usually better and
only punctually worst at generation 50, while the local-search
procedure rarely and punctually improves the results.
B. Results in Multiple Unmanned-Aerial-Vehicle Scenarios
The scenarios of the second and third rows of Fig. 4 show
the planner’s ability to find no risky paths in dynamic environ-
ments with multiple UAVs.We use Nr= 30planner executions
per scenario, with the nominal values of the tunable parameters
of Table III, and perform the same generation tests as in the
single-UAV case. The second row of Fig. 5 presents their mean
improvement and mean PLR graphics. Table V collects the of-
fline planner GS, GI, GC, and Time numbers, and the NW and
N of each UAV trajectories.
Scenarios D2, D4 [see Fig. 4(d)], D6, and D8 [see Fig. 4(g)]
are specially designed to analyze the difficulties of our planner
to converge initially to collision-free trajectories. They consist
of several pairs of UAVs with opposite starting and final points
that are confined in a small region and have to bypass the same
intermediate point flying at constant altitude.5In all cases, the
planner converges to free-collision (and completely feasible)
trajectories before generation 50. It also optimizes them consid-
least one path is straight, and the others bend as little as possible
to avoid collisions.
In scenario E2 [see Fig. 4(e) and (h)], we illustrate the ad-
vantage of using the terrain inhibition to find free risky paths
4We use PLR only because the constraints are fulfilled in the three scenarios
before generation 50, and PKill=0 in A1 and B1 before the generation 50 and
in C1 before the generation 100.
5We impose this restriction because it increments the challenge as the UAVs
cannot avoid each other using different altitudes. Each setup is the same as the
for lack of space, are special cases of D4 and D8.
BESADA-PORTAS et al.: EVOLUTIONARY TRAJECTORY PLANNER FOR MULTIPLE UAVs IN REALISTIC SCENARIOS 631
best pareto solutions. (d) Mean values of the PLR improvements. (e) Mean PLR values of the best pareto solutions. (f) Mean improvements and Mean PLR.
UAV scenarios graphic results. (a) SFDRP generation graphic for scenario A1. (b) Mean values of the PLR improvements. (c) Mean PLR values of the
PLANNER AND SOLUTIONS PROPERTIES FOR THE SINGLE-UAV SCENARIOS
PLANNER AND SOLUTIONS PROPERTIES FOR THE MULTI-UAV SCENARIOS
among the mountains. ADU1is placed between mountains, and
the UAVs find free risky paths inside the Rj
solid circle), where PKill=0 due to the mountains.
In scenario F3 [see Fig. 4(f) and (i)], the UAVs have to visit
sents the offline planner results after only 100 generations (as a
solution (see Section VI-H) and run it during 400 extra gener-
ations [see Fig. 4(i)]. Therefore, we run the planner during 500
generations, i.e., 100 initially and 400 later. The offline trajec-
tories of UAV1, UAV2, and UAV3, respectively, overfly POP2
detects POP2as soon as it enters the RDmaxarea, and as its
RPKillarea pop-up only intersects UAV1path, only UAV1EA
is run to obtain the UAV1black trajectory in 11.8 s. Later on,
the same happens with UAV2and POP3, and the UAV2black
trajectory is obtained in 1.8 s. Finally, UAV3 detects POP1,
whose RPKill area intersects first with UAV3 trajectory and,
later, with UAV1. Therefore, UAV3EA runs for 0.3 s to obtain
UAV3black trajectory, and once it is finished, UAV1EA runs
for 2.5 s to obtain UAV1red trajectory. We run the online EA
several times in this scenario, and observe that max(Tv
when the online EA run for the limited number of 100 gener-
ations. Considering that RDmax− RPKill= 30000 m and that
UAVs are flying at 250m/s, in the worst case, only five UAVs
could be entering pop-up detection regions simultaneously and
still be able to have their routes recalculated sequentially. How-
ever, the mean value of Tv
will not be a problem for our sequential approach.6
r) = 24 s
ris 5 s, and therefore, up to 24 UAVs
C. General Remarks
The results of Fig. 5, and the GS and GI values of
Tables IV and V show that although the number of needed
which illustrate the range of applicability of our sequential approach in this
scenario, are only orientative.
r), RDmax, and RPKilldepend on the scenario, these numbers,
632 IEEE TRANSACTIONS ON ROBOTICS, VOL. 26, NO. 4, AUGUST 2010
generations depends on the scenario (as one could expect), with
the correct number of generations the planner is able to find a
solution really close to the optimum in all the cases.7Although
the correct generation number for a specific scenario cannot be
ing the planner with a previous solution.
Besides, Gcshows that the constraints are always fulfilled in
a small number of generations. Moreover, the planner quickly
overcomes the initial instability free-collision issue (Gc< 50
in the multi-UAV problem), even in really crowed scenarios
with multiple UAVs. Additionally, the analysis of the scenario
with pop-ups shows that the sequential online approach is valid
for multiple UAVs entering pop-up detection regions simulta-
neously, although it can fail when the number of UAVs grows
and the online EAs need too much time.
a few waypoints NW.
Our current work solves the problem of finding the paths of a
set of cooperating UAVs in military missions. For this purpose,
we have created the MCACEA for cooperating agents, which
consists of a set of EAs that obtain the optimal behavior of each
agent. EAs run in parallel offline and in serial online, and com-
municate each other information about their optimal solution
(i.e., every iteration, offline; at the end, online). Each EA finds
the optimal path of its associated UAV, and the optimality of
the path for each UAV is measured, considering 1) each UAV
mission and 2) the cooperation of the UAVs.
The flexibility of this planner is based on the evaluation
method used to calculate the fitness of the solutions, which
easily permits adding new optimization indexes and changing
their goals and priorities. This characteristic is used to achieve
initial paths and when it works online to avoid pop-up threats.
In both cases, the EAs consider their UAV physical constraints,
the terrain, the properties of the known ADUs, the exclusion
zones, and the obligatory intermediate points. Speedups can be
obtained using a constant flight altitude and/or previously found
The results of the planner have been statistically analyzed to
see its ability to converge to the optimal regions of the space
and the influence of the scenarios in a set of parameters. The
7The small improvements and PLR Std of scenario A1 are due to the fact that
at generation 50, the planner finds a solution inside the corridor that is really
the planner is trying to fit them around the dashed gray circle. In scenario C1,
the big initial improvement and Std are due to the fact that at generation 50, the
solution of the planner in 20% of all executions are passing through PKill = 0
regions. After generation 100, the planner improves the solutions around the
optimum. Scenarios D# get harder as the number of UAVs grows, and therefore,
the improvements [see Fig. 5(d)] and mean PLR values [see Fig. 5(e)] decrease
slowly. In scenario E2, the bigger PLR Std of UAV1 [see Fig. 5(f)] is originated
by the high variety of free risky paths that UAV finds among the mountains.
Finally, the high PLR Std of UAV2 in scenario F3 is due to the fact that in 10%
of the execution, the trajectory after IF1 surrounds its closer NFZ by its left
scenarios and that the number of generations highly depends on
the problem. It is also able to find feasible and nearly optimal
solutions with small population sizes.
Our current work focuses on including the velocities of the
UAVs in the problem codification, incrementing the number of
cooperation objectives, and making the online planners work
in parallel or serial in spite of the communication delays. The
first idea will bring important improvements and support new
ways to solve collisions problems . It will also increment
the survival possibilities of the UAVs because the planner will
be able to raise the velocity near the ADUs to stay the shortest
time possible inside the dangerous areas. The planner will also
be able to slow down an UAV to save fuel or to facilitate its
maneuvers. The second idea will consider new ways of coordi-
nating the missions, such as making different UAVs to arrive at
the same (or different) point at similar times or using bait UAVs
to increment the survival possibilities of others. The third idea
will permit testing of the distributed planner in scenarios that
include communication delays.
Other UAV-planning tasks, such as target recognition ,
, tracking , cooperative search , and vehicle-routing
problem , are also being solved with EAs. Although our
research is currently focused in the problem of finding the best
path that avoids the risk, we are analyzing the possibilities of
widening the types of tasks that the EA planner can optimize.
Table VI contains the acronyms that were most used in this
The authors would like to thank the reviewers for all their
useful comments about this work. The EA that has been used to
Toolbox EVOCOM , .
BESADA-PORTAS et al.: EVOLUTIONARY TRAJECTORY PLANNER FOR MULTIPLE UAVs IN REALISTIC SCENARIOS 633
Vehicle (Library of Flight Series).
 T. Samad, J. S. Bay, and D. Godbole, “Network-centric systems for mil-
itary operations in urban terrain: The role of UAVs,” in Proc. IEEE, Jan.
2007, vol. 95, no. 1, pp. 92–107.
 J. A. Goldman, “Path planning problems and solutions,” in Proc. Nat.
Aerosp. Electron. Conf., 1994, pp. 105–108.
 R. J. Szczerba, “Threat netting for real-time, intelligent route planners,”
in Proc. IEEE Symp. Inf., Decis. Control, 1999, pp. 377–382.
 J. Bellingham, “Coordination and control of UAV fleets using mixed-
integer linear programming,” Ph.D. dissertation, Mass. Inst. Technol.,
Cambridge, MA, 2002.
 Y. Kuwata and J. P. How, “Three dimensional receding horizon control
for UAVs,” presented at the AIAA Guid., Navigat., Control Conf. Exhib.,
AIAA, Monterey, CA, 2002.
 A. Richards and J. P. How, “Aircraft trajectory planning with collision
avoidance using mixed-integer linear programming,” in Proc. Amer. Con-
trol Conf., 2002, pp. 1936–1941.
 J. J. Ruz, O. Ar´ evalo, J. M. de la Cruz, and G. Pajares, “Using MILP for
UAVs trajectory optimization under radar detection risk,” in Proc. 11th
IEEE Int. Conf. Emerging Technol. Factory Autom., 2006, pp. 1–4.
 P. Melchior, B. Orsoni, O. Lavialle, A. Poty, and A. Oustaloup, “Con-
sideration of obstacle danger level in path planning using A* and fast-
marching optimisation: Comparative study,” Signal Process., vol. 83,
no. 11, pp. 2387–2396, 2003.
 R. J. Szczerba, P. Galkowski, I. Glickstein, and N. Ternullo, “Robust
algorithm for real-time route planning,” IEEE Trans. Aerosp. Electron.
Syst., vol. 36, no. 3, pp. 869–878, Jul. 2000.
 K. Trovato, “A* planning in discrete configuration spaces of autonomous
systems,” Ph.D. dissertation, Amsterdam Univ., Amsterdam, The
 Y.Qu, Q. Pan,and J. Yan, “Flight path planningof UAV based on heuristi-
cally search and genetic algorithms,” in Proc. 31st Annu. Conf. IEEE Ind.
Electron. Soc., Nov. 2005, p. 5.
 A. Raghunathan, V. Gopal, D. Subramanian, L. T. Biegler, and T. Samad,
“Dynamic optimization strategies for 3d conflict resolution of multiple
aircraft,” AIAA J. Guid., Control Dyn., vol. 27, no. 4, pp. 586–594, Jul.–
 S. Mittal and K. Deb, “Three-dimensional offline path planning for UAVs
Comput., 2007, vol. 7, pp. 3195–3202.
 I. K. Nikolos, K. P. Valavanis, N. C. Tsourveloudis, and A. N. Kostaras,
“Evolutionary algorithm based offline/online path planner for UAV nav-
igation,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 33, no. 6,
pp. 898–912, Dec. 2003.
 I. Hasircioglu, H. R. Topcuoglu, and M. Ermis, “3-d path planning for the
in Proc. Genet. Evol. Comput. Conf., 2008, pp. 1499–1506.
 Y. V. Pehlivanoglu, O. Baysal, and A. Hacioglu, “Vibrational genetic
algorithm based path planner for autonomous UAV in spatial data based
environments,” in Proc. 3rd Int. Conf. Recent Adv. Space Technol., 2007,
vol. 7, pp. 573–578.
 I. K. Nikolos, N. C. Tsourveloudis, and K. P. Valavanis, “Evolutionary al-
gorithm based path planning for multiple UAV cooperation,” in Advances
in Unmanned Aerial Vehicles. Berlin, Germany: Springer-Verlag, Jan.
2007, pp. 309–340.
 I. K. Nikolos and N. C. Tsourvelouds, “Path planning for cooperating
unmanned vehicles over 3-d terrain,” Inf. Control, Autom. Robot., vol. 24,
pp. 153–168, Jan. 2009.
 C. Zheng, L. Li, F. Xu, F. Sun, and M. Ding, “Evolutionary route planner
for unmanned air vehicles,” IEEE Trans. Robot., vol. 21, no. 4, pp. 609–
620, Aug. 2005.
with multiple missions using an evolutionary algorithm,” in Proc. IEEE
3rd Int. Conf. Nat. Comput., 2007, pp. 1499–1506.
 J. M. de la Cruz, E. Besada-Portas, L. de la Torre, B. Andr´ es-Toro, and
J. A. Lopez-Orozco, “Evolutionary path planner for UAVs in realistic
environments,” in Proc. Genet. Evol. Comput. Conf., 2008, pp. 1447–
 J. M. de la Cruz, E. Besada Portas, L. de la Torre, and B. Andr´ es Toro,
“Multiobjective path planner for unmanned air vehicles (UAVs) based on
genetic algorithms,” presented at the 8th Int. FLINS Conf., Madrid, Spain,
Reston, VA: AIAA, 2004.
 G. Farin, Curves and Surfaces for Computer Aided Geometric Design. A
Practical Guide. New York: Academic, 1988.
 D. A. V. Veldhuizen and G. B. Lamont, “Evolutionary computation and
convergence to a pareto front,” in Proc. Genet. Program. Conf., 1998,
 C. M. Fonseca and P. J. Fleming, “Multiobjective optimization and mul-
tiple constraint handling with evolutionary algorithms—Part I: Unified
formulation,” IEEE Trans. Syst., Man, Cybern. A, Syst. Humans, vol. 18,
no. 1, pp. 26–37, Jan. 1988.
 C. A. Coello-Coello and M. Salazar Lechuga, “MOPSO: A proposal for
multiple objective particle swarm optimization,” presented at the Congr.
Evol. Comput., Honolulu, HI, 2002.
 K. Deb, S. Agrawal, A. Pratap, and T. Meyarivan, “A fast elitist non-
dominated sorting genetic algorithm for multi-objective optimization:
NSGA-II,” in Proc. Parallel Problem Solving Nat. VI Conf., Berlin,
Germany: Springer-Verlag, 2000, pp. 849–858.
 J. Knowles, L. Thiele, and E. Zitzler, “A tutorial on the performance
Lab., ETH Zurich, Zurich, Switzerland, Tech. Rep. TIK 214, Feb. 2006.
 P. T. Kabamba, S. M. Meerkov, and F. H. Zeitz, “Optimal path planning
for unmanned combat aerial vehicles to defeat radar tracking,” J. Guid.
Control Dyn., vol. 29, no. 2, pp. 279–288, Mar./Apr. 2006.
 B. Andr´ es-Toro, E. Besada-Portas, P. Fernandez-Blanco, J. A. Lopez-
Orozco, and J. M. de la Cruz, “Multiobjective optimization of dynamic
processes by evolutionary algorithms,” presented at the 15th Triennial
World Congr. IFAC, Barcelona, Spain, 2002.
 S. Esteban, B. Andr´ es-Toro, E. Besada-Portas, J. M. Gir´ on Sierra, and
J. M. de la Cruz, “Multiobjective control of flaps and t-foil in high speed
ships,” in Proc. 15th Triennial World Congr. IFAC, 2002, pp. 1–6.
 D. Goldberg, Genetic Algorithms in Search, Optimization, and Machine
Learning. Reading, MA: Addison-Wesley, 1989.
 T. W. McLain and R. W. Beard, “Coordination variables, coordination
functions and cooperative-timing missions,”
vol. 28, no. 1, pp. 150–161, Jan./Feb. 2005.
 R. Beard, T. McLain, M. Goodrich, and E. Anderson, “Coordinated target
assignment and intercept for unmanned air vehicles,” IEEE Trans. Robot.
Autom., vol. 18, no. 6, pp. 911–922, Dec. 2002.
 Z. Jin, T. Shima, and C. J. Schumacher, “Optimal scheduling for refueling
no. 4, pp. 682–693, Aug. 2006.
and NLP techniques for centralized trajectory planning of multiple un-
manned air vehicles,” in Proc. Amer. Control Conf., Jun. 2006, pp. 5763–
 D. Jian and J. Vagners, “Parallel evolutionary algorithms for UAV path
 M. A. Darrah, W. M. Niland, B. M. Stolarik, and L. E. Walp, “Increased
UAV task assignment performance through parallelized genetic algo-
rithms,” Air Force Res. Lab., Dayton, OH, Tech. Rep. AFRL-VA-WP-
TP-2006-339, Aug. 2006.
 D. Goldberg and K. Deb, “A comparative analysis of selection schemes
used in genetic algorithms,” in Foundations of Genetic Algorithms.
Mateo, CA: Morgan Kaufmann, 1991, pp. 69–93.
 H. B. Mann and D. R. Whitney, “On a test of whether one of two random
variables is stochastically larger than the other,” Ann. Math. Statist.,
vol. 18, pp. 50–60, Jan. 1947.
for real-time collision avoidance of multiple aerial robots in dynamic en-
vironments,” in Proc. 17th World Congr., Int. Federation Autom. Control,
Jul. 2008, pp. 1–6.
taneous tasks using genetic algorithms,” in Proc. AIAA Guid., Navigat.,
Control Conf. Exhib., Aug. 2005, pp. 15–18.
 J. Tian, L. Shen, and Y. Zheng, “Genetic algorithm based approach
for multi-UAV cooperative reconnaissance mission planning problem,”
in Lecture Notes in Computer Science, vol. 4203.
Springer-Verlag, 2006, pp. 101–110.
 D. Howden and T. Hendtlass, “Collective intelligence and bush fire spot-
ting,” in Proc. Genet. Evol. Comput. Conf., 2008, pp. 41–48.
 J. Tian, Y. Zheng, H. Zhu, and L. Shen, “A MPC and genetic algorithm
Computer Science, vol. 3801. Berlin, Germany: Springer-Verlag, 2005,
 M. Russel and G. Lamont, “A genetic algorithm for unmanned aerial
vehicle routing,” in Proc. GECCO, 2005, pp. 1499–1506.
J. Guid. Control Dyn.,
634 IEEE TRANSACTIONS ON ROBOTICS, VOL. 26, NO. 4, AUGUST 2010 Download full-text
 E. Besada-Portas, J. A. Lopez-Orozco, and B. Andres-Toro, “A versatile
toolbox for solving industrial problems with several evolutionary tech-
niques,” in Evolutionary Methods for Design, Optimization and Control.
Chapel Hill, NC: Univ. North Carolina Press, 2002, pp. 325–330.
box Ver. 2.0, Spanish Registro Propiedad Intelectual Comunidad Madrid,
Madrid, Spain, 16/2007/5293, 2004.
Eva Besada-Portas was born in Madrid, Spain, on
January 16, 1974. She received the M.Sc. degree in
from the Complutense University of Madrid (UCM)
in 1997 and 2004, respectively.
She was a Teaching Assistant at the UCM, where
she has been an Assistant Professor with the De-
partment of Computer Architecture and Automatic
Control since 2005. Since 2006, she has been a Post-
doctoral Visiting Researcher with the Department of
Computer Science, University of New Mexico, Al-
buquerque. Her current research interests include optimal control, evolutionary
23, 1983. He received the M.Sc. degree in physics
from the Complutense University of Madrid (UCM),
Madrid, Spain, with specialization in electronic de-
vices and automatic control, in 2008. He is currently
working toward the Ph.D. degree in systems engi-
neering and automatic control with Open University
of Spain (UNED), Madrid.
ment of Computer Sciences Automatic and Control,
UNED. In 2007, he joined the Department of Com-
puters Architecture and Automatic Control, UCM, as a Research Fellow. His
current research interests include evolutionary algorithms, unmanned aerial ve-
hicles, path planning, remote and virtual laboratories, and distance education.
Jes´ us M. de la Cruz (M’80) received the M.Sc. and
Ph.D. degrees in physics from the Physics Science
Faculty, Complutense University of Madrid (UCM),
Madrid, Spain, in 1979 and 1984, respectively.
From 1985 to 1990, he held teaching appoint-
ments at the UCM and at the Open University of
Spain (UNED), Madrid. From 1990 to 1992, he was
with the Department of Electronics, Cantabria Uni-
versity of Santander, Santander, Spain. In October
1992, he joined the Department of Computer Archi-
tecture and Automatic Control, UCM, where he was
the Dean from 1997 to 2001 and is currently a Full Professor and the Head of
the Automatic Control and Robotics Group. His current research interests in-
clude broad aspects of automatic control and its applications, real-time control,
optimization, statistical learning, and robotics.
on April 26, 1940. He received the M.Sc. and Ph.D.
degrees in physics from the Complutense University
of Madrid (UCM), Madrid, Spain, in 1971 and 1996,
In 1990, he joined the Department of Computer
turer, where he has been an Associate Professor since
2002. His current research interests include devel-
opment of evolutionary algorithms to solve complex
problems, such as the control of beer-fermentation
processes and path planning in unmanned vehicles.