Content uploaded by Wei Lin
Author content
All content in this area was uploaded by Wei Lin on May 30, 2019
Content may be subject to copyright.
JID:AESCTE AID:3022 /FLA [m5G; v 1.131; Prn:1/04/2014; 11:27] P.1 (1-9)
Aerospace Science and Technology ••• (••••)•••–•••
Contents lists available at ScienceDirect
Aerospace Science and Technology
www.elsevier.com/locate/aescte
Distributed UAV formation control using differential game approach
Wei Lin
Department of EECS, University of Central Florida, Orlando, FL 32816, USA
article info abstract
Article history:
Received 12 July 2013
Received in revised form 18 January 2014
Accepted 27 February 2014
Available online xxxx
Keywords:
Formation control
Differential games
Distributed control
This paper considers a formation control problem for a multiple-UAV (unmanned aerial vehicle) system
where each UAV is able to exchange information with other UAVs according to a fixed information graph.
In this paper, each UAV tries to minimize its own performance index which is chosen independently
based on its local information. Because of the UAVs’ different objectives, the formation control problem
is formulated and solved as a differential game problem. Realizing the incapability of the classical Nash
strategy approach in dealing with the distributed information, we propose a novel open-loop Nash
strategy design approach for each UAV to implement in a fully distributed manner through estimating its
terminal state. An illustrative example of a five-UAV formation control problem is solved under different
scenarios.
©2014 Elsevier Masson SAS. All rights reserved.
1. Introduction
A multiple-UAV (unmanned aerial vehicle) system is often char-
acterized by an environment with physical constraints such that
each UAV can only exchange information with neighboring ones.
Because of this constraint, the control design for each UAV uti-
lizing only the information available to it becomes a challenge.
An important application of this system is the multiple-aircraft
(including multi-UAV) formation control problem which is to de-
sign control inputs such that a prescribed formation is formed
among the aircrafts. In recent years, a variety of results on air-
craft formation control have emerged. Some of them are reviewed
as follows. In [21], the decentralized overlapping control was de-
sign to control a group of interconnected UAVs, where a feedback
controller was designed in the expanded space for each UAV and
then converted back to the original space. In [7], the aerodynam-
ics coupling effects of the formation flying system was studied
and the trajectory tracking control and formation keeping con-
trol were combined and designed using linear quadratic regulator
approach. In [10], the high-level formation control problem of or-
ganic air vehicles (OAVs) was considered using receding horizon
control approach. In [22], a unified optimal control approach in-
cluding formation control, trajectory tracking, and obstacle avoid-
ance was proposed for multiple-UAV coordination. In [23],thefuel
optimization of formation initialization problem for spacecraft was
considered and the optimization was convexified and solved as a
semidefinite program. In [11], the attitude synchronization prob-
lem of the spacecraft was considered and the decentralized control
E-mail address: weilin0929@gmail.com.
algorithm was developed based on nonlinear cooperative control
theory. A comprehensive review in larger scope on multi-agent
control systems including recent progress on aircraft formation
control can be found in [6]. Most of the recent results on for-
mation control problem utilize tools such as cooperative control
theory [17,16], optimal control theory [5,2], receding horizon con-
trol [15,4], etc., and all the aircrafts are usually assumed to pur-
sue a common goal of minimizing the total formation errors and
velocity differences among them. However, it is of practical inter-
est to have a more general setting where individual aircrafts can
have their own objectives. For example, one aircraft’s objectives
might be chosen based on its locally measured formation errors
and velocity differences. Therefore, given the aircrafts’ different
objectives, the formation control problem indeed becomes a dif-
ferential game problem [9]. However, only a few research works
have been done in this area. In [8], the formation control prob-
lem was formulated as a noncooperative differential game and the
receding horizon Nash equilibrium was solved. In [18], the con-
sensus problem as a special case of formation control problem
was formulated as a cooperative differential game and the Nash
bargain solution among the Pareto-efficient solutions was found
using linear matrix inequality (LMI) approach. In this paper, based
on the previous results, we consider the distributed Nash strategy
design approach for multiple-UAV formation control problem. We
will propose a novel approach that enables each UAV to implement
its Nash strategy only based on the information available to it only.
The rest of the paper is organized as follows: The problem
is formulated in Section 2. We derive the classical open-loop
Nash equilibrium in Section 3. The distributed Nash strategy de-
sign approach is provided in Section 4. An illustrative example of
http://dx.doi.org/10.1016/j.ast.2014.02.004
1270-9638/©2014 Elsevier Masson SAS. All rights reserved.
JID:AESCTE AID:3022 /FLA [m5G; v 1.131; Prn:1/04/2014; 11:27] P.2 (1-9)
2W. Lin / Aerospace Science and Technology ••• (••••)•••–•••
Fig. 1. UAV mo del.
a five-UAV formation control problem is solved in Section 5.The
paper is concluded in Section 6.
2. Problem formulation
2.1. UAV model
Since there exist various UAVs that are designed to complete
different real life tasks, it is impossible to have one universal
mathematical dynamic model to describe all the UAVs. This pa-
per only focuses on the high-level formation control design among
a group of UAVs and hence will adopt a representative UAV model
which has been commonly used in many literatures [13,16,22].
We consider a system of NUAVs with the following point-mass
model [13] as shown in Fig. 1:
˙
xi=Vicos γicos χi,(1)
˙
yi=Vicos γisin χi(2)
˙
hi=Vsin γi(3)
˙
Vi=Ti−Di
mi
−gsin γi(4)
˙
γi=Lcos Φi−migcos γi
miVi
(5)
˙
χi=Lisin Φi
miVicos γi
(6)
for i=1,...,N, where xiis the down-range displacement, yiis
the cross-range displacement, hiis the altitude, Viis the ground
speed which is assumed to be equal to the airspeed in this paper,
γiis the flight path angle, χiis the heading angle, Tiis the engine
thrust, Diis the drag, miis the UAV mass, gis the acceleration
due to gravity, Liis the lift, and Φiis the banking angle. The three
control inputs of UAV iis the banking angle Φi,lift Li, and engine
thrust Ti.
It is shown in [13] that the highly nonlinear UAV model in (2.1)
can be pre-linearized using feedback linearization to be
¨
xi=uxi,¨
yi=uyi,¨
hi=uhi (7)
where uxi ,uyi, and uhi are the virtual acceleration control inputs.
These virtual control inputs and the real control inputs are related
through the following equations
Φi=tan−1uyi cos χi−uxi sin χi
(uhi +g)cos γi−(uxi cos χi+uyi sin χi)sin γi(8)
Li=mi
(uhi +g)cos γi−(uxi cos χi+uyi sin χi)sin γi
cos Φi
(9)
Ti=mi(uhi +g)sin γi+(uxi cos χi+uyi sin χi)cos γi+Di
(10)
where tan χi=˙
yi/˙
xiand sin γi=˙
hi/Vi. Therefore, after the vir-
tual control inputs are designed based on the linear model (7),the
real control inputs can then be obtained by substituting the virtual
ones into (7). Expressing (7) in terms of state-space representation
yields
˙
zi=Azi+Bui,(11)
pi=Cpzi(12)
vi=Cvzi(13)
where zi=[pT
ivi]Tis the state vector, piis the position vector,
viis the velocity vector, ui=[uT
xi uT
yi uT
hi]Tis the virtual acceler-
ation control vector,
Ai=01
00
⊗I3,Bi=0
1⊗I3,
Cp=[10
]⊗I3,Cv=[01
]⊗I3,
I3∈R3×3is the identity matrix, and ⊗is the Kronecker product.
2.2. Information graph
Suppose that individual UAVs are able to communicate with
each other in a certain pattern to achieve the desired formation.
We define a time-invariant directed information graph G=(V,E)
to describe the information exchange pattern among them. Specif-
ically, node vi∈Vrepresents UAV iand edge eij∈Erepresents the
directional information transmission from UAV jto UAV i.Several
terminologies from graph theory are introduced as follows.
Definition 1. Nodeiis globally reachable in graph Gif there exists
a sequence of edges directed from vito vjfor all j=1,..., N,
j= i.
A globally reachable node is also known as a root node of
a spanning tree on the graph. Based on the definition of globally
reachable node, the connectivity of a graph is defined as follows.
Definition 2. Graph Gis connected if there exists at least one glob-
ally reachable node.
In this paper, to achieve the formation requirement, the con-
nectivity of the UAVs on the information graph must be assured.
Hence, we make the following assumption:
Assumption 1. The underlying information graph among the N
UAVs is connected.
A widely used mathematical tool in graph theory is the Lapla-
cian matrix L=[Lij]∈RN×Nwhich is defined as follows:
Lij =⎧
⎨
⎩
−lij if eij∈Efor j= i
0ifeij /∈Efor j= i
−N
q=1,q=iliq if j=i,
(14)
JID:AESCTE AID:3022 /FLA [m5G; v 1.131; Prn:1/04/2014; 11:27] P.3 (1-9)
W. Lin / Aerospace Science and Technology ••• (••••)•••–••• 3
where lij is a positive scalar represents the weight made by UAV i
on the information transmitted through eij∈E. In this paper, scalar
lij can be regarded as a weighting factor of UAV i’s willingness
to keep the desired displacement between itself and UAV jfor
eij∈E. A larger weight indicates a stronger willingness. Further-
more, a well known property of the Laplacian matrix is that all its
eigenvalues always have nonnegative real parts.
2.3. Performance indices and Nash equilibrium
The objective of this paper is to design control inputs for in-
dividual UAVs to form a prescribed formation. Hence, assuming
that the desired displacement vector pointing from UAV jto UAV
iis αij, the formation requirement can be expressed mathemat-
ically in terms of the following performance index for UAV ito
minimize:
Ji=
eij∈E
lij
2
pi(tf)−pj(tf)−αij
2+
vi(tf)−vj(tf)
2
+ri
2
tf
0
ui2dt ∀i=1,...,N,(15)
where · is the Euclidean norm, lij is the entry of the Laplacian
matrix defined in (14), and riis a positive scalar, tf>0istheter-
minal time. Performance index (15) means that UAV iwill try to
minimize a weighted sum of the terminal formation errors and ve-
locity errors according to the information graph while at the same
time minimizing its control effort made during the entire forma-
tion control process. The coefficients in the performance index (15)
represent the penalties on the terminal formation errors, velocity
errors, and control effort. The larger the coefficients are, the more
emphases are placed on the corresponding terms. Note that coef-
ficients lij and riin performance index (15) for different UAVs are
not necessarily the same because these coefficients reflect the real
situation. For instance, if UAV ihas sufficient fuel in its tank, it will
naturally choose a large value of lij and a small value of riin order
to keep the desired formation with others actively. On the contrary,
if UAV idoes not have much fuel left in its tank, it will naturally
choose a small value of lij and a large value of rito preserve its
energy or fuel cost. Therefore, since there exist different perfor-
mance indices for the NUAVs, the formation control problem in
fact becomes a differential game problem. In this paper, we con-
sider the Nash equilibrium solution for this game problem and the
definition of Nash equilibrium is as follows: For the N-UAV differ-
ential game defined by system (11)–(13) and performance indices
described in (15),thestrategiesu∗
1,...,u∗
Nform a Nash equilib-
rium if
Jiu∗
1,...,u∗
NJiu∗
1,...,u∗
i−1,ui,u∗
i+1,...,u∗
N
∀ui∈Ui(16)
hold for all i=1,...,N, where Uiis UAV i’s admissible strategy
set.
In other words, once the UAVs’ strategy form a Nash equilib-
rium, no one tends to deviate from the Nash equilibrium unilater-
ally. To derive the Nash equilibrium, the admissible strategy set Ui
must be clearly specified and it is closely related with the informa-
tion structure [3]. In this paper, we will seek the open-loop Nash
equilibrium for the UAVs where UAV i’s admissible strategy set Ui
contains strategy as a function of the initial states z1(0),...,zN(0)
and tonly.
3. Classical open-loop Nash equilibrium
In this section, the open-loop Nash equilibrium for the formu-
lated differential game is derived. To begin with, we define the
following new state vectors to simplify the problem:
spi(t)=[1tf−t]zi(t)and svi(t)=[01
]zi(t). (17)
Differentiating both sides of (17) with respect to tand recalling
system dynamics (11)–(13) yield
˙
spi =˜
Bpuiand ˙
svi =˜
Bvui,(18)
where ˜
Bp=(tf−t)I3and ˜
Bv=I3.Basedontheproperties
spi(tf)=pi(tf)and svi(tf)=vi(tf)for spi and svi defined in (17),
the performance indices in (15) can be rewritten as
Ji=
eij∈E
lij
2
spi(tf)−spj(tf)−αij
2+
svi(tf)−svj(tf)
2
+ri
2
tf
0
ui2dt ∀i=1,...,N.(19)
Before we present the result of open-loop Nash equilibrium, the
following lemma is introduced first.
Lemma 1. All the eigenvalues of matrix M defined by
M=I2N+W⊗R−1L⊗I3,(20)
have positive real parts, where Lis the graph Laplacian matrix defined
in (14),
W=wpp wpv
wvp wvv ,(21)
wpp =
tf
0
˜
Bp˜
BT
pdt =t3
f
3,wpv =
tf
0
˜
Bp˜
BT
vdt =t2
f
2(22)
wvp =
tf
0
˜
Bv˜
BT
pdt =t2
f
2,wvv =
tf
0
˜
Bv˜
BT
vdt =tf,(23)
R=diag{r1,...,rN},(24)
and diag{·} stands for “diagonal matrix”.
Proof. Firstly, since matrix Wis obviously positive define, all its
eigenvalues are positive. Secondly, since matrix Rin (24) is a
positive diagonal matrix, the product of (R−1L)becomes a new
weighted Laplacian matrix whose eigenvalues still have nonneg-
ative real parts. Thirdly, since the eigenvalues of matrices’ Kro-
necker product are the product of these matrices’ eigenvalues, all
the eigenvalues of matrix [W⊗(R−1L)]have nonnegative real
parts. Therefore, all the eigenvalues of Min (20) have positive real
parts. 2
The open-loop Nash equilibrium solution is now presented as
follows.
Theorem 1. Given the differential game among N UAVs with system dy-
namics (18) and performance indices (19),thestrategies
u∗
i=−1
ri
FiM−1sp(0)
sv(0)+Wαα+1
ri
˜
BT
pαi
∀i=1,...,N(25)
JID:AESCTE AID:3022 /FLA [m5G; v 1.131; Prn:1/04/2014; 11:27] P.4 (1-9)
4W. Lin / Aerospace Science and Technology ••• (••••)•••–•••
form an open-loop Nash equilibrium, where matrix M is defined in (20),
sp=sT
p1··· sT
pNT,sv=sT
v1··· sT
vNT,(26)
Fi=˜
BT
p˜
BT
vI2⊗dT
iL⊗I3,(27)
Wα=wpp
wvp ⊗R−1⊗I3,(28)
α=αT
1··· αT
NT,(29)
αi=
eij∈E
lijαij ∀i=1,...,N,(30)
Lis the Laplacian matrix defined in (14),d
i∈RNis a vector with the ith
entry equal to 1and the other entries equal to 0, and scalars w pp and
wvp are defined in (22) and (23), respectively.
Proof. We define the Hamiltonian for UAV ias
Hi=ri
2ui2+λT
pi ˜
Bpui+λT
vi ˜
Bvui
where vectors λpi and λvi are the Lagrangian multipliers. Accord-
ing to the well known Pontryagin’s minimum principle [14],the
necessary conditions for optimality are
˙
spi =∂Hi
∂λpi
=˜
Bpui,˙
svi =∂Hi
∂λvi
=˜
Bvui(31)
˙
λpi =−∂Hi
∂spi
=0,˙
λvi =−∂Hi
∂svi
=0,(32)
λpi(tf)=
eij∈E
lijspi(tf)−spj(tf)−αij,(33)
λvi(tf)=
eij∈E
lijsvi(tf)−svj(tf),(34)
∂Hi
∂ui
=riui+˜
BT
pλpi +˜
BT
vλvi =0,∂2Hi
∂u2
i
=ri>0.(35)
Conditions (32)–(34) indicate that λpi and λvi are constant vectors.
Substituting them into (35) yields
ui=−1
ri
˜
BT
pλpi −1
ri
˜
BT
vλvi
=−1
ri
˜
BT
p
eij∈E
lijspi(tf)−spj(tf)−αij
−1
ri
˜
BT
v
eij∈E
lijsvi(tf)−svj(tf).(36)
Substituting (36) into (31) and integrating both sides from 0 to tf
yield
spi(tf)+wpp
ri
eij∈E
lijspi(tf)−spj(tf)
+wpv
ri
eij∈E
lijsvi(tf)−svj(tf)=spi(0)+wpp
ri
αi(37)
And
svi(tf)+wvp
ri
eij∈E
lijspi(tf)−spj(tf)
+wvv
ri
eij∈E
lijsvi(tf)−svj(tf)=svi(0)+wvp
ri
αi(38)
where scalars wpp ,wpv,wvp ,wvv are defined in (22)–(23) and
αiis defined in (30). Combining (37) and (38) and stacking them
from i=1toi=Nyield
sp(tf)
sv(tf)=M−1sp(0)
sv(0)+wpp
wvp ⊗R−1⊗I3α(39)
where vectors spand svare defined in (26),matrixMis defined
in (20) and invertible according to Lemma 1, and vector αis de-
fined in (29). Therefore, rewriting (36) as
ui=−1
ri
Fisp(tf)
sv(tf)+1
ri
˜
BT
pαi(40)
where Fiis defined in (27) and substituting (39) into (40)
yields (25).Sincesp(0),sv(0)are in fact functions of the initial
state z(0)through (17),strategiesu∗
1,...,u∗
Nin (25) form an open-
loop Nash equilibrium. 2
Note that since M−1is generally a full matrix, implementing
open-loop Nash strategies in (25) requires individual UAVs to have
the knowledge of sp(0)and sv(0).However,sinceUAViis only
able to receive information from UAV jfor eij∈E,vectorssp(0)
and sv(0)containing global initial state information are generally
not available to individual UAVs. Hence, the open-loop Nash strat-
egy u∗
iin (25) is generally not implementable unless the underly-
ing information graph of the NUAVs is fully connected. Therefore,
to overcome the difficulty of implementing these Nash strategies,
an appropriate Nash strategy design approach must be proposed
for individual UAVs in a distributed manner, that is, each UAV can
carry out the approach with the information available to it only.
This design objective leads to the following section.
4. Nash strategies design in a distributed manner
Realizing that the Nash strategy design approach in the pre-
vious section is unable to meet the information transmission re-
quirement on the graph, we propose a novel Nash strategy de-
sign approach in a distributed manner in this section. Firstly, note
that although the Nash strategy u∗
iin (25) is not directly im-
plementable, its equivalent expression in (36) is indeed in the
distributed manner if UAV iis able to figure out its own termi-
nal position spi(tf)=pi(tf)and terminal velocity svi(tf)=vi(tf)
as well as UAV j’s terminal position spj(tf)=pj(tf)and termi-
nal velocity svj(tf)=vj(tf)for eij∈E. Because of this distributed
structure in (36), the problem really reduces down to proposing
a distributed estimation approach such that each UAV is able to
estimate its own terminal position and terminal velocity while at
the same time exchanging these estimations with other UAVs ac-
cording to the information graph. Toward that end, we have the
following result.
Theorem 2. If UAV i updates its terminal position estimate hpi and ter-
minal velocity estimate hvi in continuously in time from any initial guess
hpi(0)and hvi(0)according to
˙
hpi
˙
hvi =kispi (0)
svi(0)+1
riwpp
wvp αi−hpi
hvi
−1
ri
(W⊗I3)
eij∈E
lijhpi
hvi −hpj
hvj (41)
where kiis a positive scalar, matrix W is defined in (21),andvectorαi
is defined in (30),then
lim
τ→∞ hp(τ)
hv(τ)=sp(tf)
sv(tf),(42)
JID:AESCTE AID:3022 /FLA [m5G; v 1.131; Prn:1/04/2014; 11:27] P.5 (1-9)
W. Lin / Aerospace Science and Technology ••• (••••)•••–••• 5
where hp=[hT
p1··· hT
pN]T,h
v=[hT
v1··· hT
vN]T,andvectorss
p(tf)
and sv(tf)are UAVs’ terminal positions and terminal velocities as de-
fined in (39).
Proof. Stacking Eq. (41) from i=1toi=Nyields
˙
hp
˙
hv=Ksp(0)
sv(0)−Mhp
hv+Wαα,(43)
where K=I2⊗diag{k1,...,kN}⊗I3,matrixMis defined in (20),
matrix Wαis defined in (28), and vector αis defined in (29).Since
all the eigenvalues of matrix Mhave positive real parts as shown
in Lemma 1,matrix(−M)is Hurwitz. Therefore, linear system
with respect to [hp;hv]in (43) is asymptotically stable starting
from any initial condition h(0)and will converge to the equilib-
rium, i.e.,
lim
τ→∞ hp(τ)
hv(τ)=M−1sp(0)
sv(0)+Wαα,
where the right hand side of the above equation is equal to the
vector [sp(tf);sv(tf)]defined in (39). Therefore, Eq. (42) holds. 2
Using the estimation law in (41), individual UAVs are able to
asymptotically estimate their terminal positions and terminal ve-
locities by exchanging their estimates according to the graph. Note
that the proposed estimation law (41) is fully distributed in the
sense that in order to implement it, UAV ionly needs to
•retain its private information, ki,spi(0),svi(0),lij ,αi,
•receive the terminal state estimate(s) hjfrom UAV jfor all
eij∈E,j=1,...,N, and
•send its terminal state estimate hito UAV jif eji∈E,j=
1,...,N.
Given the distributed state estimation law (41),one possible
way to implement the open-loop Nash strategy is to let all the
agents in the system communicate for a while until satisfactory
convergent values, say ˆ
hpi and ˆ
hvi, are reached before the game
starts and the UAVs will then implement the open-loop Nash strat-
egy expressed in (36) with spi(tf)and svi(tf)with ˆ
hpi and ˆ
hvi.
Such a design approach can be regarded as an offline computa-
tion approach. Although the offline approach provides an accu-
rate enough strategy design, it may not be suitable for the sit-
uation where the real-time implementation is required. This can
be achieved by using the following real-time implementation algo-
rithm:
˙
hpi
˙
hvi =kispi (0)
svi(0)+1
riwpp
wvp ⊗I3αi
−hpi
hvi −1
ri
(W⊗I3)
×
eij∈E
lijhpi
hvi −hpj
hvj ,(44)
u∗
i=−1
ri
˜
BT
p
eij∈E
lij(hpi −hpj −αij)−1
ri
˜
BT
v
eij∈E
lij(hvi −hvj),
(45)
where Eq. (44) is the terminal state estimation law and Eq. (45) is
the open-loop Nash control using the terminal estimates directly.
Therefore, for either offline or real-time implementation, individual
UAVs are able to implement the open-loop Nash strategies in a dis-
tributed manner by measuring and calculating the initial states
s1(0),...,sN(0)and exchanging the terminal position estimates
and terminal velocity estimates among themselves according to the
graph. Since it is better to have the estimation law converge fast,
one can choose a large positive scalar gito achieve satisfactory
convergence speed.
Remark 1. One important feature of the proposed design is that
to implement it, each UAV does not need to have the knowledge
of the other UAVs’ performance indices. This feature could be sig-
nificant in the real life controller design because as we mentioned
earlier, individual UAVs choose their coefficients of their perfor-
mance indices in (15) independently based on their weights on
a variety of factors, such as local formation errors, local velocity er-
rors, and the amount of fuel left in their tanks. Moreover, each UAV
is usually not able to know the overall information graph among
all the UAVs. Therefore, the distributed strategy design in (43) con-
forms with the aforementioned real application constraints.
Remark 2. Although the proposed design approach is derived
based on the virtual double-integrator dynamics (11), the approach
can be in fact easily extended to the case where the UAVs have
heterogeneous linear time-varying systems. For more details on the
general case, please refer to [12].
Remark 3. As we know, the feedback control structure rather than
the open-loop control structure is preferred in many applications
because it can react instantaneously to the change of the system.
To make the proposed Nash strategy design approach more adap-
tive to the unexpected changes in the system including the case
where the information topology is changing over time, we can
utilize the sampled-Nash approach [19]. Toward that end, instead
of (15), we can consider the following performance indices
Ji(tk)=
eij∈E
lij
2
pi(tf)−pj(tf)−αij
2+
vi(tf)−vj(tf)
2
+ri
2
tf
tk
ui2dt ∀k=1,...,Ns(46)
where 0 =t1t2···tNs =tf. Hence, based on (43), the sam-
pled distributed Nash strategy design algorithm is proposed as
follows.
Algorithm 1. At t=tk,
1. UAV imeasures and calculates spi (tk)and svi(tk)for i=
1,...,N.
2. UAV iimplements(43) with [spi (0);svi(0)]replaced by
[spi(tk);svi(tk)]and
wpp =(tf−tk)3
3,wpv =wvp =(tf−tk)2
2,
wvv =tf−tk
for i=1,...,N.
3. Once t=tk+1arrives, the UAVs repeat the steps 1 and 2 by
letting tk→tk+1.
As we can see from the above algorithm, the UAVs measure
the state variables multiple times during the process and will
hence be more aware of the change in the system. Based on this
sampled-Nash approach, we are able to consider the Nash strate-
gies design under time-varying information topology which might
occur due to communication failure or obstacles etc. We denote
G(t)={V,E(t)}as the time-varying graph. If we assume that the
time interval between two consecutive sample instants tkand tk+1
JID:AESCTE AID:3022 /FLA [m5G; v 1.131; Prn:1/04/2014; 11:27] P.6 (1-9)
6W. Lin / Aerospace Science and Technology ••• (••••)•••–•••
Fig. 2. Triangle formation shape and information graph.
is small enough such that the information topology remains the
same within the interval, then the UAVs are subject performance
indices in (46) with Ereplaced by E(tk). Essentially, every UAV
only needs to adjust its performance index according to the infor-
mation graph every time its local communication pattern changes.
In this case, the terminal state estimation algorithm should still be
applicable with minor changes. Once there is a change in the infor-
mation graph, the equation will have a different equilibrium. Since
the algorithm is always fully distributed and asymptotically stable,
all the players will quickly reach a stable terminal state estimate
after communicate for a while according to the information graph
and hence the new Nash strategies are obtained. More property on
the switching topology is still under investigation.
5. Illustrative example and simulation results
In this section, we consider a formation problem of a five-UAV
system to illustrate the proposed distributed Nash formation con-
trol design approach. The parameters in [22] are utilized for this
simulation: The weight of UAV iis mi=20 kg for all i=1,...,N.
The gravity constant is g=9.81 kg/m2. The drag Diis calculated
as follows [24]:
Di=0.5ρ(Vi−Vwi)2SCD0+2kdk2
nL2/g2
ρ(Vi−Vwi)2S
where ρis the atmospheric density and equal to 1.225 kg/m3,Vwi
is the gust, Sis the wing area and equal to 1.37 m2,CD0is the
zero-lift drag coefficient and equal to 0.02, kdis the induced drag
coefficient and equal to 0.1, and knis the load-factor effectiveness
and equal to 1. The gust Vwi is modeled as follows [1]:
Vwi =¯
Vwi +δVwi
¯
Vwi =0.215Vmlog10 (hi)+0.285Vm
where ¯
Vwi is the normal wind shear, Vmis the mean wind speed
and equal to 4 m/s at the altitude of 80 m, and δVwi is the wind
gust turbulence on UAV iand assumed to be a Gaussian random
variable with zero mean and a standard deviation equal to 0.09Vm.
The real control inputs of the UAV have the following constraints:
Ti<125N,−294.3N<Li<392.4N, and −80 ◦Φi80 ◦for all
i=1,...,N.
The UAVs are trying to form a desired V-shape in the same
altitude shown in Fig. 2 and the underlying directed information
graph is also shown in this figure where we can see that UAV 1 is
the globally reachable node and is regarded as the leader who only
sends out its information but not receives any information. Hence,
UAV 1 will act as a reference for the other UAVs. The corresponding
graph Laplacian matrix is
L=⎡
⎢
⎢
⎢
⎣
00000
−a21 a21 000
−a31 0a31 00
0−a42 0a42 0
00
−a53 0a53
⎤
⎥
⎥
⎥
⎦
.
The desired offset vectors of the formation among the UAVs are
α21 =−100
−100
0m,α31 =100
−100
0m,
α42 =−100
−100
0m,α53 =100
−100
0m.
The initial positions of the UAVs are
p1(0)=0
0
90 m,p2(0)=−80
0
80 m,
p3(0)=90
0
70 m,p4(0)=−120
0
60 m,
p5(0)=150
0
65 m.
The initial velocities of the UAVs are
v1(0)=0
50
0m/s,v2(0)=0
60
0m/s,
v3(0)=0
40
0m/s,v4(0)=0
65
0m/s,
v5(0)=0
45
0m/s.
The five UAVs’ performance indices are given by (15) with tf=30.
We solve the formation control problem by utilizing the proposed
distributed Nash strategy design approach and present the follow-
ing simulation results under different scenarios.
Case 1. If lij =10 and ri=1 in performance index (15) for all
i,j=1,...,N, the UAVs’ trajectories in 3-dimensional space and
x–yplane are shown in Fig. 3, where the solid circles indicate the
UAVs’ initial positions and the solid triangles indicate the UAVs’
terminal positions.
In Fig. 3,the black dotted lines that link the UAVs’ terminal
position show that the desired V-formation among the UAVs is
achieved at the terminal time. For illustrative purpose, the UAVs’
trajectories on xaxis, yaxis, and haxis are shown independently
in Fig. 4.
In this case, since UAV 1 acts as a leader’, it keeps the constant
velocity during the entire process and all the other UAVs control
their terminal positions and terminal velocities with the reference
to UAV 1. Moreover, the three real control inputs are also shown in
Fig. 4 and all of them are within the specified constraints.
Case 2. If lij =10 and ri=1 in performance index (15) for all i,j=
1,...,Nwhile the information graph now becomes the one shown
in Fig. 5 where UAV 1 now receives information from UAV 2 and 3.
Therefore, the graph becomes leaderless.
JID:AESCTE AID:3022 /FLA [m5G; v 1.131; Prn:1/04/2014; 11:27] P.7 (1-9)
W. Lin / Aerospace Science and Technology ••• (••••)•••–••• 7
Fig. 3. UAVs’ trajectories i n 3D and x–yplane for Case 1.
Fig. 4. UAVs’ positions, velocities, and real control inputs for Case 1.
Fig. 5. Triangle formation shape and information graph for Case 2.
JID:AESCTE AID:3022 /FLA [m5G; v 1.131; Prn:1/04/2014; 11:27] P.8 (1-9)
8W. Lin / Aerospace Science and Technology ••• (••••)•••–•••
Fig. 6. UAVs’ trajecto ries i n 3D and x–yplane for Case 2.
Fig. 7. UAVs’ positions, velocities, and real control inputs for Case 2.
The UAVs’ trajectories in 3-dimensional space and x–yplane are
shown in Fig. 6 and the V-formation is achieved at the terminal
time.
As we can see more clearly in Fig. 7, UAV 1’s velocity changes
in the process and the terminal positions and terminal velocities
of the UAVs are different from the previous cases.
6. Conclusion
In this paper, a distributed formation control design approach
using differential game theory is proposed for multiple-UAV sys-
tems where UAVs are only able to exchange their information
according to a directed graph. In the game, each UAV tries to
minimize its terminal formation errors and terminal velocity dif-
ferences to other UAVs according to the graph while at the same
time minimizing its control efforts. We show that the classical de-
sign approach is unable to provide Nash strategies conforming to
the information graph constraint. A novel design approach utilizing
the terminal state estimation is then proposed to construct fully
distributed Nash strategy design approach. Utilizing this approach,
we solve an illustrative example under different cases and present
the simulation results. As another possible solution to the UAV for-
mation control problem, the UAVs might seek for the “noninferior
solution” [20] as a solution for the cooperative game assuming all
the UAVs are in a team and collaborate with each other. Some ex-
ploration in this direction has been made in [12].
JID:AESCTE AID:3022 /FLA [m5G; v 1.131; Prn:1/04/2014; 11:27] P.9 (1-9)
W. Lin / Aerospace Science and Technology ••• (••••)•••–••• 9
References
[1] Advisory circular, Tech. rep. AC 120-28D, U.S. Department of Transportation and
Federal Aviation Administration, 1999.
[2] M. Athans, The matrix minimum principle, Inf. Control 11 (1967) 592–606.
[3] T. Basar, G.J. Olsder, Dynamic Noncooperative Game Theory, 2nd edition, SIAM,
Philadelphia, PA, 1998.
[4] A. Bemporad, M. Morari, Robust model predictive control: A survey, in: Robust-
ness in Identification and Control, Springer, 1999, pp. 207–226.
[5] E. Bryson, Y.C. Ho, Applied Optimal Control, Hemisphere Publishing Corp., Bris-
tol, PA, USA, 1975.
[6] Y. Cao, W. Yu, W. Ren, G. Chen, An overview of recent progress in the study
of distributed multi-agent coordination, IEEE Trans. Ind. Inform. 9 (1) (2013)
427–438.
[7] F. Giulietti, M. Innocenti, M. Napolitano, L. Pollini, Dynamic and control issues
of formation flight, Aerosp. Sci. Technol. 9 (1) (2005) 65–71.
[8] D. Gu, A differential game approach to formation control, IEEE Trans. Control
Syst. Technol. 16 (1) (2008) 85–93.
[9] R. Isaacs, Differential Games, John Wiley and Sons, 1965.
[10] T.Keviczky,F.Borrelli,K.Fregene,D.Godbole,G.Balas,Decentralizedreced-
ing horizon control and coordination of autonomous vehicle formations, IEEE
Trans. Control Syst. Technol. 16 (1) (2008) 19–33.
[11] C. Li, Z. Qu, Cooperative attitude synchronization for rigid-body spacecraft via
varying communication topology, Int. J. Robot. Autom. 26 (1) (2011) 110.
[12] W. Lin, Differential games for multi-agent systems under distributed informa-
tion, Ph.D. thesis, University of Central Florida, 2013.
[13] P. Menon, G. Sweriduk, B. Sridhar, Optimal strategies for free-flight air traffic
conflict resolution, J. Guid. Control Dyn. 22 (2) (1999) 202–211.
[14] L.S. Pontryagin, The Mathematical Theory of Optimal Processes, vol. 4, CRC
Press, 1962.
[15] S.J. Qin, T.A. Badgwell, An overview of industrial model predictive control tech-
nology, in: AIChE Symposium Series, vol. 93, American Institute of Chemical
Engineers, 1971-c2002, New York, NY, 1997, pp. 232–256.
[16] Z. Qu, Cooperative Control of Dynamical Systems: Applications to Autonomous
Vehicles, Springer Verlag, London, 2009.
[17] W. Ren, R.W. Beard, E.M. Atkins, Information consensus in multivehicle cooper-
ative control: Collective group behavior through local interaction, IEEE Control
Syst. Mag. 27 (2007) 71–82.
[18] E. Semsar-Kazerooni, K. Khorasani, Multi-agent team cooperation: A game the-
ory approach, Automatica 45 (10) (2009) 2205–2213.
[19] M. Simaan, J. Cruz Jr., Sampled-data Nash controls in non-zero-sum differential
games, Int. J. Control 17 (6) (1973) 1201–1209.
[20] A. Starr, Y. Ho, Nonzero-sum differential games, J. Optim. Theory Appl. 3 (1969)
184–206.
[21] D.M. Stipanovic, G. Inalhan, R. Teo, C.J. Tomlin, Decentralized overlapping con-
trol of a formation of unmanned aerial vehicles, Automatica 40 (8) (2004)
1285–1296.
[22] J. Wang, M. Xin, Integrated optimal formation control of multiple unmanned
aerial vehicles, IEEE Control Syst. Technol. 99 (2012) 1.
[23] Y.-H. Wu, X.-B. Cao, Y.-J. Xing, P.-F. Zheng, S.-J. Zhang, Relative motion cou-
pled control for formation flying spacecraft via convex optimization, Aerosp.
Sci. Technol. 14 (6) (2010) 415–428.
[24] Y. Xu, Nonlinear robust stochastic control for unmanned aerial vehicles, J. Guid.
Control Dyn. 32 (4) (2009) 1308–1319.
