ArticlePDF Available

Distributed UAV formation control using differential game approach

Authors:
  • Openpath Security
JID:AESCTE AID:3022 /FLA [m5G; v 1.131; Prn:1/04/2014; 11:27] P.1 (1-9)
Aerospace Science and Technology ••• (••••)••••••
Contents lists available at ScienceDirect
Aerospace Science and Technology
www.elsevier.com/locate/aescte
Distributed UAV formation control using differential game approach
Wei Lin
Department of EECS, University of Central Florida, Orlando, FL 32816, USA
article info abstract
Article history:
Received 12 July 2013
Received in revised form 18 January 2014
Accepted 27 February 2014
Available online xxxx
Keywords:
Formation control
Differential games
Distributed control
This paper considers a formation control problem for a multiple-UAV (unmanned aerial vehicle) system
where each UAV is able to exchange information with other UAVs according to a fixed information graph.
In this paper, each UAV tries to minimize its own performance index which is chosen independently
based on its local information. Because of the UAVs’ different objectives, the formation control problem
is formulated and solved as a differential game problem. Realizing the incapability of the classical Nash
strategy approach in dealing with the distributed information, we propose a novel open-loop Nash
strategy design approach for each UAV to implement in a fully distributed manner through estimating its
terminal state. An illustrative example of a five-UAV formation control problem is solved under different
scenarios.
©2014 Elsevier Masson SAS. All rights reserved.
1. Introduction
A multiple-UAV (unmanned aerial vehicle) system is often char-
acterized by an environment with physical constraints such that
each UAV can only exchange information with neighboring ones.
Because of this constraint, the control design for each UAV uti-
lizing only the information available to it becomes a challenge.
An important application of this system is the multiple-aircraft
(including multi-UAV) formation control problem which is to de-
sign control inputs such that a prescribed formation is formed
among the aircrafts. In recent years, a variety of results on air-
craft formation control have emerged. Some of them are reviewed
as follows. In [21], the decentralized overlapping control was de-
sign to control a group of interconnected UAVs, where a feedback
controller was designed in the expanded space for each UAV and
then converted back to the original space. In [7], the aerodynam-
ics coupling effects of the formation flying system was studied
and the trajectory tracking control and formation keeping con-
trol were combined and designed using linear quadratic regulator
approach. In [10], the high-level formation control problem of or-
ganic air vehicles (OAVs) was considered using receding horizon
control approach. In [22], a unified optimal control approach in-
cluding formation control, trajectory tracking, and obstacle avoid-
ance was proposed for multiple-UAV coordination. In [23],thefuel
optimization of formation initialization problem for spacecraft was
considered and the optimization was convexified and solved as a
semidefinite program. In [11], the attitude synchronization prob-
lem of the spacecraft was considered and the decentralized control
E-mail address: weilin0929@gmail.com.
algorithm was developed based on nonlinear cooperative control
theory. A comprehensive review in larger scope on multi-agent
control systems including recent progress on aircraft formation
control can be found in [6]. Most of the recent results on for-
mation control problem utilize tools such as cooperative control
theory [17,16], optimal control theory [5,2], receding horizon con-
trol [15,4], etc., and all the aircrafts are usually assumed to pur-
sue a common goal of minimizing the total formation errors and
velocity differences among them. However, it is of practical inter-
est to have a more general setting where individual aircrafts can
have their own objectives. For example, one aircraft’s objectives
might be chosen based on its locally measured formation errors
and velocity differences. Therefore, given the aircrafts’ different
objectives, the formation control problem indeed becomes a dif-
ferential game problem [9]. However, only a few research works
have been done in this area. In [8], the formation control prob-
lem was formulated as a noncooperative differential game and the
receding horizon Nash equilibrium was solved. In [18], the con-
sensus problem as a special case of formation control problem
was formulated as a cooperative differential game and the Nash
bargain solution among the Pareto-efficient solutions was found
using linear matrix inequality (LMI) approach. In this paper, based
on the previous results, we consider the distributed Nash strategy
design approach for multiple-UAV formation control problem. We
will propose a novel approach that enables each UAV to implement
its Nash strategy only based on the information available to it only.
The rest of the paper is organized as follows: The problem
is formulated in Section 2. We derive the classical open-loop
Nash equilibrium in Section 3. The distributed Nash strategy de-
sign approach is provided in Section 4. An illustrative example of
http://dx.doi.org/10.1016/j.ast.2014.02.004
1270-9638/©2014 Elsevier Masson SAS. All rights reserved.
JID:AESCTE AID:3022 /FLA [m5G; v 1.131; Prn:1/04/2014; 11:27] P.2 (1-9)
2W. Lin / Aerospace Science and Technology ••• (••••)••••••
Fig. 1. UAV mo del.
a five-UAV formation control problem is solved in Section 5.The
paper is concluded in Section 6.
2. Problem formulation
2.1. UAV model
Since there exist various UAVs that are designed to complete
different real life tasks, it is impossible to have one universal
mathematical dynamic model to describe all the UAVs. This pa-
per only focuses on the high-level formation control design among
a group of UAVs and hence will adopt a representative UAV model
which has been commonly used in many literatures [13,16,22].
We consider a system of NUAVs with the following point-mass
model [13] as shown in Fig. 1:
˙
xi=Vicos γicos χi,(1)
˙
yi=Vicos γisin χi(2)
˙
hi=Vsin γi(3)
˙
Vi=TiDi
mi
gsin γi(4)
˙
γi=Lcos Φimigcos γi
miVi
(5)
˙
χi=Lisin Φi
miVicos γi
(6)
for i=1,...,N, where xiis the down-range displacement, yiis
the cross-range displacement, hiis the altitude, Viis the ground
speed which is assumed to be equal to the airspeed in this paper,
γiis the flight path angle, χiis the heading angle, Tiis the engine
thrust, Diis the drag, miis the UAV mass, gis the acceleration
due to gravity, Liis the lift, and Φiis the banking angle. The three
control inputs of UAV iis the banking angle Φi,lift Li, and engine
thrust Ti.
It is shown in [13] that the highly nonlinear UAV model in (2.1)
can be pre-linearized using feedback linearization to be
¨
xi=uxi,¨
yi=uyi,¨
hi=uhi (7)
where uxi ,uyi, and uhi are the virtual acceleration control inputs.
These virtual control inputs and the real control inputs are related
through the following equations
Φi=tan1uyi cos χiuxi sin χi
(uhi +g)cos γi(uxi cos χi+uyi sin χi)sin γi(8)
Li=mi
(uhi +g)cos γi(uxi cos χi+uyi sin χi)sin γi
cos Φi
(9)
Ti=mi(uhi +g)sin γi+(uxi cos χi+uyi sin χi)cos γi+Di
(10)
where tan χi=˙
yi/˙
xiand sin γi=˙
hi/Vi. Therefore, after the vir-
tual control inputs are designed based on the linear model (7),the
real control inputs can then be obtained by substituting the virtual
ones into (7). Expressing (7) in terms of state-space representation
yields
˙
zi=Azi+Bui,(11)
pi=Cpzi(12)
vi=Cvzi(13)
where zi=[pT
ivi]Tis the state vector, piis the position vector,
viis the velocity vector, ui=[uT
xi uT
yi uT
hi]Tis the virtual acceler-
ation control vector,
Ai=01
00
I3,Bi=0
1I3,
Cp=[10
]I3,Cv=[01
]I3,
I3R3×3is the identity matrix, and is the Kronecker product.
2.2. Information graph
Suppose that individual UAVs are able to communicate with
each other in a certain pattern to achieve the desired formation.
We define a time-invariant directed information graph G=(V,E)
to describe the information exchange pattern among them. Specif-
ically, node viVrepresents UAV iand edge eijErepresents the
directional information transmission from UAV jto UAV i.Several
terminologies from graph theory are introduced as follows.
Definition 1. Nodeiis globally reachable in graph Gif there exists
a sequence of edges directed from vito vjfor all j=1,..., N,
j= i.
A globally reachable node is also known as a root node of
a spanning tree on the graph. Based on the definition of globally
reachable node, the connectivity of a graph is defined as follows.
Definition 2. Graph Gis connected if there exists at least one glob-
ally reachable node.
In this paper, to achieve the formation requirement, the con-
nectivity of the UAVs on the information graph must be assured.
Hence, we make the following assumption:
Assumption 1. The underlying information graph among the N
UAVs is connected.
A widely used mathematical tool in graph theory is the Lapla-
cian matrix L=[Lij]∈RN×Nwhich is defined as follows:
Lij =
lij if eijEfor j= i
0ifeij /Efor j= i
N
q=1,q=iliq if j=i,
(14)
JID:AESCTE AID:3022 /FLA [m5G; v 1.131; Prn:1/04/2014; 11:27] P.3 (1-9)
W. Lin / Aerospace Science and Technology ••• (••••)•••••• 3
where lij is a positive scalar represents the weight made by UAV i
on the information transmitted through eijE. In this paper, scalar
lij can be regarded as a weighting factor of UAV i’s willingness
to keep the desired displacement between itself and UAV jfor
eijE. A larger weight indicates a stronger willingness. Further-
more, a well known property of the Laplacian matrix is that all its
eigenvalues always have nonnegative real parts.
2.3. Performance indices and Nash equilibrium
The objective of this paper is to design control inputs for in-
dividual UAVs to form a prescribed formation. Hence, assuming
that the desired displacement vector pointing from UAV jto UAV
iis αij, the formation requirement can be expressed mathemat-
ically in terms of the following performance index for UAV ito
minimize:
Ji=
eijE
lij
2
pi(tf)pj(tf)αij
2+
vi(tf)vj(tf)
2
+ri
2
tf
0
ui2dt i=1,...,N,(15)
where · is the Euclidean norm, lij is the entry of the Laplacian
matrix defined in (14), and riis a positive scalar, tf>0istheter-
minal time. Performance index (15) means that UAV iwill try to
minimize a weighted sum of the terminal formation errors and ve-
locity errors according to the information graph while at the same
time minimizing its control effort made during the entire forma-
tion control process. The coefficients in the performance index (15)
represent the penalties on the terminal formation errors, velocity
errors, and control effort. The larger the coefficients are, the more
emphases are placed on the corresponding terms. Note that coef-
ficients lij and riin performance index (15) for different UAVs are
not necessarily the same because these coefficients reflect the real
situation. For instance, if UAV ihas sufficient fuel in its tank, it will
naturally choose a large value of lij and a small value of riin order
to keep the desired formation with others actively. On the contrary,
if UAV idoes not have much fuel left in its tank, it will naturally
choose a small value of lij and a large value of rito preserve its
energy or fuel cost. Therefore, since there exist different perfor-
mance indices for the NUAVs, the formation control problem in
fact becomes a differential game problem. In this paper, we con-
sider the Nash equilibrium solution for this game problem and the
definition of Nash equilibrium is as follows: For the N-UAV differ-
ential game defined by system (11)–(13) and performance indices
described in (15),thestrategiesu
1,...,u
Nform a Nash equilib-
rium if
Jiu
1,...,u
NJiu
1,...,u
i1,ui,u
i+1,...,u
N
uiUi(16)
hold for all i=1,...,N, where Uiis UAV i’s admissible strategy
set.
In other words, once the UAVs’ strategy form a Nash equilib-
rium, no one tends to deviate from the Nash equilibrium unilater-
ally. To derive the Nash equilibrium, the admissible strategy set Ui
must be clearly specified and it is closely related with the informa-
tion structure [3]. In this paper, we will seek the open-loop Nash
equilibrium for the UAVs where UAV i’s admissible strategy set Ui
contains strategy as a function of the initial states z1(0),...,zN(0)
and tonly.
3. Classical open-loop Nash equilibrium
In this section, the open-loop Nash equilibrium for the formu-
lated differential game is derived. To begin with, we define the
following new state vectors to simplify the problem:
spi(t)=[1tft]zi(t)and svi(t)=[01
]zi(t). (17)
Differentiating both sides of (17) with respect to tand recalling
system dynamics (11)–(13) yield
˙
spi =˜
Bpuiand ˙
svi =˜
Bvui,(18)
where ˜
Bp=(tft)I3and ˜
Bv=I3.Basedontheproperties
spi(tf)=pi(tf)and svi(tf)=vi(tf)for spi and svi defined in (17),
the performance indices in (15) can be rewritten as
Ji=
eijE
lij
2
spi(tf)spj(tf)αij
2+
svi(tf)svj(tf)
2
+ri
2
tf
0
ui2dt i=1,...,N.(19)
Before we present the result of open-loop Nash equilibrium, the
following lemma is introduced first.
Lemma 1. All the eigenvalues of matrix M defined by
M=I2N+WR1LI3,(20)
have positive real parts, where Lis the graph Laplacian matrix defined
in (14),
W=wpp wpv
wvp wvv ,(21)
wpp =
tf
0
˜
Bp˜
BT
pdt =t3
f
3,wpv =
tf
0
˜
Bp˜
BT
vdt =t2
f
2(22)
wvp =
tf
0
˜
Bv˜
BT
pdt =t2
f
2,wvv =
tf
0
˜
Bv˜
BT
vdt =tf,(23)
R=diag{r1,...,rN},(24)
and diag{·} stands for “diagonal matrix”.
Proof. Firstly, since matrix Wis obviously positive define, all its
eigenvalues are positive. Secondly, since matrix Rin (24) is a
positive diagonal matrix, the product of (R1L)becomes a new
weighted Laplacian matrix whose eigenvalues still have nonneg-
ative real parts. Thirdly, since the eigenvalues of matrices’ Kro-
necker product are the product of these matrices’ eigenvalues, all
the eigenvalues of matrix [W(R1L)]have nonnegative real
parts. Therefore, all the eigenvalues of Min (20) have positive real
parts. 2
The open-loop Nash equilibrium solution is now presented as
follows.
Theorem 1. Given the differential game among N UAVs with system dy-
namics (18) and performance indices (19),thestrategies
u
i=−1
ri
FiM1sp(0)
sv(0)+Wαα+1
ri
˜
BT
pαi
i=1,...,N(25)
JID:AESCTE AID:3022 /FLA [m5G; v 1.131; Prn:1/04/2014; 11:27] P.4 (1-9)
4W. Lin / Aerospace Science and Technology ••• (••••)••••••
form an open-loop Nash equilibrium, where matrix M is defined in (20),
sp=sT
p1··· sT
pNT,sv=sT
v1··· sT
vNT,(26)
Fi=˜
BT
p˜
BT
vI2dT
iLI3,(27)
Wα=wpp
wvp R1I3,(28)
α=αT
1··· αT
NT,(29)
αi=
eijE
lijαij i=1,...,N,(30)
Lis the Laplacian matrix defined in (14),d
iRNis a vector with the ith
entry equal to 1and the other entries equal to 0, and scalars w pp and
wvp are defined in (22) and (23), respectively.
Proof. We define the Hamiltonian for UAV ias
Hi=ri
2ui2+λT
pi ˜
Bpui+λT
vi ˜
Bvui
where vectors λpi and λvi are the Lagrangian multipliers. Accord-
ing to the well known Pontryagin’s minimum principle [14],the
necessary conditions for optimality are
˙
spi =Hi
∂λpi
=˜
Bpui,˙
svi =Hi
∂λvi
=˜
Bvui(31)
˙
λpi =−Hi
spi
=0,˙
λvi =−Hi
svi
=0,(32)
λpi(tf)=
eijE
lijspi(tf)spj(tf)αij,(33)
λvi(tf)=
eijE
lijsvi(tf)svj(tf),(34)
Hi
ui
=riui+˜
BT
pλpi +˜
BT
vλvi =0,2Hi
u2
i
=ri>0.(35)
Conditions (32)–(34) indicate that λpi and λvi are constant vectors.
Substituting them into (35) yields
ui=−1
ri
˜
BT
pλpi 1
ri
˜
BT
vλvi
=−1
ri
˜
BT
p
eijE
lijspi(tf)spj(tf)αij
1
ri
˜
BT
v
eijE
lijsvi(tf)svj(tf).(36)
Substituting (36) into (31) and integrating both sides from 0 to tf
yield
spi(tf)+wpp
ri
eijE
lijspi(tf)spj(tf)
+wpv
ri
eijE
lijsvi(tf)svj(tf)=spi(0)+wpp
ri
αi(37)
And
svi(tf)+wvp
ri
eijE
lijspi(tf)spj(tf)
+wvv
ri
eijE
lijsvi(tf)svj(tf)=svi(0)+wvp
ri
αi(38)
where scalars wpp ,wpv,wvp ,wvv are defined in (22)–(23) and
αiis defined in (30). Combining (37) and (38) and stacking them
from i=1toi=Nyield
sp(tf)
sv(tf)=M1sp(0)
sv(0)+wpp
wvp R1I3α(39)
where vectors spand svare defined in (26),matrixMis defined
in (20) and invertible according to Lemma 1, and vector αis de-
fined in (29). Therefore, rewriting (36) as
ui=−1
ri
Fisp(tf)
sv(tf)+1
ri
˜
BT
pαi(40)
where Fiis defined in (27) and substituting (39) into (40)
yields (25).Sincesp(0),sv(0)are in fact functions of the initial
state z(0)through (17),strategiesu
1,...,u
Nin (25) form an open-
loop Nash equilibrium. 2
Note that since M1is generally a full matrix, implementing
open-loop Nash strategies in (25) requires individual UAVs to have
the knowledge of sp(0)and sv(0).However,sinceUAViis only
able to receive information from UAV jfor eijE,vectorssp(0)
and sv(0)containing global initial state information are generally
not available to individual UAVs. Hence, the open-loop Nash strat-
egy u
iin (25) is generally not implementable unless the underly-
ing information graph of the NUAVs is fully connected. Therefore,
to overcome the difficulty of implementing these Nash strategies,
an appropriate Nash strategy design approach must be proposed
for individual UAVs in a distributed manner, that is, each UAV can
carry out the approach with the information available to it only.
This design objective leads to the following section.
4. Nash strategies design in a distributed manner
Realizing that the Nash strategy design approach in the pre-
vious section is unable to meet the information transmission re-
quirement on the graph, we propose a novel Nash strategy de-
sign approach in a distributed manner in this section. Firstly, note
that although the Nash strategy u
iin (25) is not directly im-
plementable, its equivalent expression in (36) is indeed in the
distributed manner if UAV iis able to figure out its own termi-
nal position spi(tf)=pi(tf)and terminal velocity svi(tf)=vi(tf)
as well as UAV j’s terminal position spj(tf)=pj(tf)and termi-
nal velocity svj(tf)=vj(tf)for eijE. Because of this distributed
structure in (36), the problem really reduces down to proposing
a distributed estimation approach such that each UAV is able to
estimate its own terminal position and terminal velocity while at
the same time exchanging these estimations with other UAVs ac-
cording to the information graph. Toward that end, we have the
following result.
Theorem 2. If UAV i updates its terminal position estimate hpi and ter-
minal velocity estimate hvi in continuously in time from any initial guess
hpi(0)and hvi(0)according to
˙
hpi
˙
hvi =kispi (0)
svi(0)+1
riwpp
wvp αihpi
hvi
1
ri
(WI3)
eijE
lijhpi
hvi hpj
hvj  (41)
where kiis a positive scalar, matrix W is defined in (21),andvectorαi
is defined in (30),then
lim
τ→∞ hp(τ)
hv(τ)=sp(tf)
sv(tf),(42)
JID:AESCTE AID:3022 /FLA [m5G; v 1.131; Prn:1/04/2014; 11:27] P.5 (1-9)
W. Lin / Aerospace Science and Technology ••• (••••)•••••• 5
where hp=[hT
p1··· hT
pN]T,h
v=[hT
v1··· hT
vN]T,andvectorss
p(tf)
and sv(tf)are UAVs’ terminal positions and terminal velocities as de-
fined in (39).
Proof. Stacking Eq. (41) from i=1toi=Nyields
˙
hp
˙
hv=Ksp(0)
sv(0)Mhp
hv+Wαα,(43)
where K=I2diag{k1,...,kN}⊗I3,matrixMis defined in (20),
matrix Wαis defined in (28), and vector αis defined in (29).Since
all the eigenvalues of matrix Mhave positive real parts as shown
in Lemma 1,matrix(M)is Hurwitz. Therefore, linear system
with respect to [hp;hv]in (43) is asymptotically stable starting
from any initial condition h(0)and will converge to the equilib-
rium, i.e.,
lim
τ→∞ hp(τ)
hv(τ)=M1sp(0)
sv(0)+Wαα,
where the right hand side of the above equation is equal to the
vector [sp(tf);sv(tf)]defined in (39). Therefore, Eq. (42) holds. 2
Using the estimation law in (41), individual UAVs are able to
asymptotically estimate their terminal positions and terminal ve-
locities by exchanging their estimates according to the graph. Note
that the proposed estimation law (41) is fully distributed in the
sense that in order to implement it, UAV ionly needs to
retain its private information, ki,spi(0),svi(0),lij ,αi,
receive the terminal state estimate(s) hjfrom UAV jfor all
eijE,j=1,...,N, and
send its terminal state estimate hito UAV jif ejiE,j=
1,...,N.
Given the distributed state estimation law (41),one possible
way to implement the open-loop Nash strategy is to let all the
agents in the system communicate for a while until satisfactory
convergent values, say ˆ
hpi and ˆ
hvi, are reached before the game
starts and the UAVs will then implement the open-loop Nash strat-
egy expressed in (36) with spi(tf)and svi(tf)with ˆ
hpi and ˆ
hvi.
Such a design approach can be regarded as an offline computa-
tion approach. Although the offline approach provides an accu-
rate enough strategy design, it may not be suitable for the sit-
uation where the real-time implementation is required. This can
be achieved by using the following real-time implementation algo-
rithm:
˙
hpi
˙
hvi =kispi (0)
svi(0)+1
riwpp
wvp I3αi
hpi
hvi 1
ri
(WI3)
×
eijE
lijhpi
hvi hpj
hvj ,(44)
u
i=−1
ri
˜
BT
p
eijE
lij(hpi hpj αij)1
ri
˜
BT
v
eijE
lij(hvi hvj),
(45)
where Eq. (44) is the terminal state estimation law and Eq. (45) is
the open-loop Nash control using the terminal estimates directly.
Therefore, for either offline or real-time implementation, individual
UAVs are able to implement the open-loop Nash strategies in a dis-
tributed manner by measuring and calculating the initial states
s1(0),...,sN(0)and exchanging the terminal position estimates
and terminal velocity estimates among themselves according to the
graph. Since it is better to have the estimation law converge fast,
one can choose a large positive scalar gito achieve satisfactory
convergence speed.
Remark 1. One important feature of the proposed design is that
to implement it, each UAV does not need to have the knowledge
of the other UAVs’ performance indices. This feature could be sig-
nificant in the real life controller design because as we mentioned
earlier, individual UAVs choose their coefficients of their perfor-
mance indices in (15) independently based on their weights on
a variety of factors, such as local formation errors, local velocity er-
rors, and the amount of fuel left in their tanks. Moreover, each UAV
is usually not able to know the overall information graph among
all the UAVs. Therefore, the distributed strategy design in (43) con-
forms with the aforementioned real application constraints.
Remark 2. Although the proposed design approach is derived
based on the virtual double-integrator dynamics (11), the approach
can be in fact easily extended to the case where the UAVs have
heterogeneous linear time-varying systems. For more details on the
general case, please refer to [12].
Remark 3. As we know, the feedback control structure rather than
the open-loop control structure is preferred in many applications
because it can react instantaneously to the change of the system.
To make the proposed Nash strategy design approach more adap-
tive to the unexpected changes in the system including the case
where the information topology is changing over time, we can
utilize the sampled-Nash approach [19]. Toward that end, instead
of (15), we can consider the following performance indices
Ji(tk)=
eijE
lij
2
pi(tf)pj(tf)αij
2+
vi(tf)vj(tf)
2
+ri
2
tf
tk
ui2dt k=1,...,Ns(46)
where 0 =t1t2···tNs =tf. Hence, based on (43), the sam-
pled distributed Nash strategy design algorithm is proposed as
follows.
Algorithm 1. At t=tk,
1. UAV imeasures and calculates spi (tk)and svi(tk)for i=
1,...,N.
2. UAV iimplements(43) with [spi (0);svi(0)]replaced by
[spi(tk);svi(tk)]and
wpp =(tftk)3
3,wpv =wvp =(tftk)2
2,
wvv =tftk
for i=1,...,N.
3. Once t=tk+1arrives, the UAVs repeat the steps 1 and 2 by
letting tktk+1.
As we can see from the above algorithm, the UAVs measure
the state variables multiple times during the process and will
hence be more aware of the change in the system. Based on this
sampled-Nash approach, we are able to consider the Nash strate-
gies design under time-varying information topology which might
occur due to communication failure or obstacles etc. We denote
G(t)={V,E(t)}as the time-varying graph. If we assume that the
time interval between two consecutive sample instants tkand tk+1
JID:AESCTE AID:3022 /FLA [m5G; v 1.131; Prn:1/04/2014; 11:27] P.6 (1-9)
6W. Lin / Aerospace Science and Technology ••• (••••)••••••
Fig. 2. Triangle formation shape and information graph.
is small enough such that the information topology remains the
same within the interval, then the UAVs are subject performance
indices in (46) with Ereplaced by E(tk). Essentially, every UAV
only needs to adjust its performance index according to the infor-
mation graph every time its local communication pattern changes.
In this case, the terminal state estimation algorithm should still be
applicable with minor changes. Once there is a change in the infor-
mation graph, the equation will have a different equilibrium. Since
the algorithm is always fully distributed and asymptotically stable,
all the players will quickly reach a stable terminal state estimate
after communicate for a while according to the information graph
and hence the new Nash strategies are obtained. More property on
the switching topology is still under investigation.
5. Illustrative example and simulation results
In this section, we consider a formation problem of a five-UAV
system to illustrate the proposed distributed Nash formation con-
trol design approach. The parameters in [22] are utilized for this
simulation: The weight of UAV iis mi=20 kg for all i=1,...,N.
The gravity constant is g=9.81 kg/m2. The drag Diis calculated
as follows [24]:
Di=0.5ρ(ViVwi)2SCD0+2kdk2
nL2/g2
ρ(ViVwi)2S
where ρis the atmospheric density and equal to 1.225 kg/m3,Vwi
is the gust, Sis the wing area and equal to 1.37 m2,CD0is the
zero-lift drag coefficient and equal to 0.02, kdis the induced drag
coefficient and equal to 0.1, and knis the load-factor effectiveness
and equal to 1. The gust Vwi is modeled as follows [1]:
Vwi =¯
Vwi +δVwi
¯
Vwi =0.215Vmlog10 (hi)+0.285Vm
where ¯
Vwi is the normal wind shear, Vmis the mean wind speed
and equal to 4 m/s at the altitude of 80 m, and δVwi is the wind
gust turbulence on UAV iand assumed to be a Gaussian random
variable with zero mean and a standard deviation equal to 0.09Vm.
The real control inputs of the UAV have the following constraints:
Ti<125N,294.3N<Li<392.4N, and 80 Φi80 for all
i=1,...,N.
The UAVs are trying to form a desired V-shape in the same
altitude shown in Fig. 2 and the underlying directed information
graph is also shown in this figure where we can see that UAV 1 is
the globally reachable node and is regarded as the leader who only
sends out its information but not receives any information. Hence,
UAV 1 will act as a reference for the other UAVs. The corresponding
graph Laplacian matrix is
L=
00000
a21 a21 000
a31 0a31 00
0a42 0a42 0
00
a53 0a53
.
The desired offset vectors of the formation among the UAVs are
α21 =100
100
0m,α31 =100
100
0m,
α42 =100
100
0m,α53 =100
100
0m.
The initial positions of the UAVs are
p1(0)=0
0
90 m,p2(0)=80
0
80 m,
p3(0)=90
0
70 m,p4(0)=120
0
60 m,
p5(0)=150
0
65 m.
The initial velocities of the UAVs are
v1(0)=0
50
0m/s,v2(0)=0
60
0m/s,
v3(0)=0
40
0m/s,v4(0)=0
65
0m/s,
v5(0)=0
45
0m/s.
The five UAVs’ performance indices are given by (15) with tf=30.
We solve the formation control problem by utilizing the proposed
distributed Nash strategy design approach and present the follow-
ing simulation results under different scenarios.
Case 1. If lij =10 and ri=1 in performance index (15) for all
i,j=1,...,N, the UAVs’ trajectories in 3-dimensional space and
xyplane are shown in Fig. 3, where the solid circles indicate the
UAVs’ initial positions and the solid triangles indicate the UAVs’
terminal positions.
In Fig. 3,the black dotted lines that link the UAVs’ terminal
position show that the desired V-formation among the UAVs is
achieved at the terminal time. For illustrative purpose, the UAVs’
trajectories on xaxis, yaxis, and haxis are shown independently
in Fig. 4.
In this case, since UAV 1 acts as a leader’, it keeps the constant
velocity during the entire process and all the other UAVs control
their terminal positions and terminal velocities with the reference
to UAV 1. Moreover, the three real control inputs are also shown in
Fig. 4 and all of them are within the specified constraints.
Case 2. If lij =10 and ri=1 in performance index (15) for all i,j=
1,...,Nwhile the information graph now becomes the one shown
in Fig. 5 where UAV 1 now receives information from UAV 2 and 3.
Therefore, the graph becomes leaderless.
JID:AESCTE AID:3022 /FLA [m5G; v 1.131; Prn:1/04/2014; 11:27] P.7 (1-9)
W. Lin / Aerospace Science and Technology ••• (••••)•••••• 7
Fig. 3. UAVs’ trajectories i n 3D and xyplane for Case 1.
Fig. 4. UAVs’ positions, velocities, and real control inputs for Case 1.
Fig. 5. Triangle formation shape and information graph for Case 2.
JID:AESCTE AID:3022 /FLA [m5G; v 1.131; Prn:1/04/2014; 11:27] P.8 (1-9)
8W. Lin / Aerospace Science and Technology ••• (••••)••••••
Fig. 6. UAVs’ trajecto ries i n 3D and xyplane for Case 2.
Fig. 7. UAVs’ positions, velocities, and real control inputs for Case 2.
The UAVs’ trajectories in 3-dimensional space and xyplane are
shown in Fig. 6 and the V-formation is achieved at the terminal
time.
As we can see more clearly in Fig. 7, UAV 1’s velocity changes
in the process and the terminal positions and terminal velocities
of the UAVs are different from the previous cases.
6. Conclusion
In this paper, a distributed formation control design approach
using differential game theory is proposed for multiple-UAV sys-
tems where UAVs are only able to exchange their information
according to a directed graph. In the game, each UAV tries to
minimize its terminal formation errors and terminal velocity dif-
ferences to other UAVs according to the graph while at the same
time minimizing its control efforts. We show that the classical de-
sign approach is unable to provide Nash strategies conforming to
the information graph constraint. A novel design approach utilizing
the terminal state estimation is then proposed to construct fully
distributed Nash strategy design approach. Utilizing this approach,
we solve an illustrative example under different cases and present
the simulation results. As another possible solution to the UAV for-
mation control problem, the UAVs might seek for the “noninferior
solution” [20] as a solution for the cooperative game assuming all
the UAVs are in a team and collaborate with each other. Some ex-
ploration in this direction has been made in [12].
JID:AESCTE AID:3022 /FLA [m5G; v 1.131; Prn:1/04/2014; 11:27] P.9 (1-9)
W. Lin / Aerospace Science and Technology ••• (••••)•••••• 9
References
[1] Advisory circular, Tech. rep. AC 120-28D, U.S. Department of Transportation and
Federal Aviation Administration, 1999.
[2] M. Athans, The matrix minimum principle, Inf. Control 11 (1967) 592–606.
[3] T. Basar, G.J. Olsder, Dynamic Noncooperative Game Theory, 2nd edition, SIAM,
Philadelphia, PA, 1998.
[4] A. Bemporad, M. Morari, Robust model predictive control: A survey, in: Robust-
ness in Identification and Control, Springer, 1999, pp. 207–226.
[5] E. Bryson, Y.C. Ho, Applied Optimal Control, Hemisphere Publishing Corp., Bris-
tol, PA, USA, 1975.
[6] Y. Cao, W. Yu, W. Ren, G. Chen, An overview of recent progress in the study
of distributed multi-agent coordination, IEEE Trans. Ind. Inform. 9 (1) (2013)
427–438.
[7] F. Giulietti, M. Innocenti, M. Napolitano, L. Pollini, Dynamic and control issues
of formation flight, Aerosp. Sci. Technol. 9 (1) (2005) 65–71.
[8] D. Gu, A differential game approach to formation control, IEEE Trans. Control
Syst. Technol. 16 (1) (2008) 85–93.
[9] R. Isaacs, Differential Games, John Wiley and Sons, 1965.
[10] T.Keviczky,F.Borrelli,K.Fregene,D.Godbole,G.Balas,Decentralizedreced-
ing horizon control and coordination of autonomous vehicle formations, IEEE
Trans. Control Syst. Technol. 16 (1) (2008) 19–33.
[11] C. Li, Z. Qu, Cooperative attitude synchronization for rigid-body spacecraft via
varying communication topology, Int. J. Robot. Autom. 26 (1) (2011) 110.
[12] W. Lin, Differential games for multi-agent systems under distributed informa-
tion, Ph.D. thesis, University of Central Florida, 2013.
[13] P. Menon, G. Sweriduk, B. Sridhar, Optimal strategies for free-flight air traffic
conflict resolution, J. Guid. Control Dyn. 22 (2) (1999) 202–211.
[14] L.S. Pontryagin, The Mathematical Theory of Optimal Processes, vol. 4, CRC
Press, 1962.
[15] S.J. Qin, T.A. Badgwell, An overview of industrial model predictive control tech-
nology, in: AIChE Symposium Series, vol. 93, American Institute of Chemical
Engineers, 1971-c2002, New York, NY, 1997, pp. 232–256.
[16] Z. Qu, Cooperative Control of Dynamical Systems: Applications to Autonomous
Vehicles, Springer Verlag, London, 2009.
[17] W. Ren, R.W. Beard, E.M. Atkins, Information consensus in multivehicle cooper-
ative control: Collective group behavior through local interaction, IEEE Control
Syst. Mag. 27 (2007) 71–82.
[18] E. Semsar-Kazerooni, K. Khorasani, Multi-agent team cooperation: A game the-
ory approach, Automatica 45 (10) (2009) 2205–2213.
[19] M. Simaan, J. Cruz Jr., Sampled-data Nash controls in non-zero-sum differential
games, Int. J. Control 17 (6) (1973) 1201–1209.
[20] A. Starr, Y. Ho, Nonzero-sum differential games, J. Optim. Theory Appl. 3 (1969)
184–206.
[21] D.M. Stipanovic, G. Inalhan, R. Teo, C.J. Tomlin, Decentralized overlapping con-
trol of a formation of unmanned aerial vehicles, Automatica 40 (8) (2004)
1285–1296.
[22] J. Wang, M. Xin, Integrated optimal formation control of multiple unmanned
aerial vehicles, IEEE Control Syst. Technol. 99 (2012) 1.
[23] Y.-H. Wu, X.-B. Cao, Y.-J. Xing, P.-F. Zheng, S.-J. Zhang, Relative motion cou-
pled control for formation flying spacecraft via convex optimization, Aerosp.
Sci. Technol. 14 (6) (2010) 415–428.
[24] Y. Xu, Nonlinear robust stochastic control for unmanned aerial vehicles, J. Guid.
Control Dyn. 32 (4) (2009) 1308–1319.
... Motivated by these and other advances, the vector field method has gained popularity and studies have been made to extend it towards formation tasks: examples include circular formations with constant speed [15] or, when the velocity can be controlled, a non-uniform vector field whose vectors have different directions and magnitudes [16]. In this sense, the vector field method offers an alternative to formation-keeping methods based on PID control [17], inverse optimal control [18], Nash equilibrium [19], [20], model predictive control [21], [22], or consensus-based formation control [23]- [26]. It is worth mentioning that, while the vector field method has been originally developed for fixed-wing UAVs [27], not all the alternative methods are directly applicable to fixedwing UAVs: consensus-and Nash equilibrium-based methods have been studied mostly for quadrotors [28] or for vertical takeoff and landing UAVs [29]; methods based on inverse optimal control or model predictive control require a different architecture than the established open-source architectures of many autopilot suites (e.g. ...
... where the derivatives in (18), (19) can be calculated aṡ ...
... Theorem 1. The closed-loop system given by the leader (1), the follower (8), the commands (18), (19) with vector field (16), (17) is asymptotically stable if ...
... There are many research results in the field of distributed control of multiple UAVs, because the distributed control can effectively reduce communication costs while improving the system robustness. By describing the formation control problem as a differential game problem, an open-loop Nash strategy design method for each UAV was proposed to implement in a fully distributed manner [4]. Through the state observer and virtual structure, the output feedback formation control problem of the networked UAVs without linear and angular velocities measurement was studied in [5]. ...
Article
Full-text available
This paper proposes a fixed-time backstepping distributed cooperative control scheme based on fixed-time extended state observer (FTESO) for multiple unmanned aerial vehicles (UAVs). A fixed-time ESO, which is convergent independently of initial conditions, is designed to estimate and compensate the external disturbances in tracking process. Moreover, to eliminate the "explosion of complexity" in the traditional backstepping architecture, a nonlinear first-order filter is adopted to construct the distributed fixed-time control scheme. Based on the local information of neighboring UAVs, a fixed-time backstepping cooperative controller is designed. The proposed formation algorithm can be shown practicable for the UAV control system by using of Lyapunov stability theory and graph theory. Simulation results are given to demonstrate the effectiveness of the proposed control scheme.
... For the coordinated formation of fixed-wing UAVs, the under-actuation and control limitations of the fixed-wing make it impossible to form trajectories from all initial positions to the desired formation; the only possible trajectories involve movement that maintains normal airspeed while approaching the desired location. Additionally, the intermediate transition process of cooperative formation change is very important [23][24][25][26], and it is necessary to consider possible collisions in the process of short-range formation change. According to the control strategy of formation keeping, we design a method for changing formation while also preventing collisions. ...
Article
Full-text available
This paper reports on the formation and transformation of multiple fixed-wing unmanned aerial vehicles (UAVs) in three-dimensional space. A cooperative guidance law based on the classic missile-type parallel-approach method is designed for the multi-UAV formation control problem. Additionally, formation transformation strategies for multi-UAV autonomous assembly, disbandment, and special circumstances are formed, effective for managing and controlling the formation. When formulating the management strategy for formation establishment, its process is divided into three steps: (i) selecting and allocating target points, (ii) forming loose formations, and (iii) forming short�range formations. The management of disbanding the formation is formulated through reverse thinking: the assembly process is split and recombined in reverse, and a formation disbanding strategy that can achieve a smooth transition from close to lose formation is proposed. Additionally, a strategy is given for adjusting the formation transformation in special cases, and the formation adjustment is completed using the adjacency matrix. Finally, a hardware-in-the-loop simulation and measured flight verification using a simulator show the practicality of the guidance law in meeting the control requirements of UAV formation flight for specific flight tas
... The distributed control strategy involves the decentralized model of sharing local information (see Fig.3). The research in [11,12,13,14] was conducted in a decentralized manner, and in [15,16,17] were conducted in a distributed manner. Several general studies have been conducted to compare distribution formation; however, there has been no dedicated analysis. ...
Article
In this paper, the finite-time formation problem of UAVs is investigated with consideration of semi-Markov-type switching topologies. More precisely, finite-time passivity performance is adopted to overcome the dynamical effect of disturbances. Furthermore, an asynchronous event-triggered communication scheme is proposed for more efficient information exchanges. The mode-dependent formation controllers are designed in terms of the Lyapunov–Krasovskii method, such that the configuration formation can be accomplished. Finally, simulation results are given to demonstrate the validity of the proposed formation approach.
Article
This paper addresses the collision-free adaptive formation control problem for multiple unmanned aerial vehicles subject to input saturation and communication delays. First, a novel collision-free adaptive saturated control scheme is proposed using potential function and anti-windup compensator approaches. It shows that the velocity consensus and collision avoidance are realized when the directed communication topology is strongly connected. Second, the collision-free adaptive saturated control algorithm is proposed considering communication delays and collision avoidance, and it enables the asymptotic stability of the closed-loop system when the time-varying communication delays are decreasing to zero eventually. Finally, numerical simulations demonstrate that the proposed control approaches are effective, while the objectives of collision avoidance and velocity consensus are fulfilled.
Article
Two types of formation control problems for a group of unmanned surface vessels (USVs) are investigated by using a game‐theoretic approach. In the two formation frameworks, the USVs have the group formation objective of minimizing the formation errors with their local neighbors. Different from the existing formation control schemes for USVs with only the group objective, the formation control framework in this paper takes into account the individual objectives of the USVs in addition to the collaborative formation requirement for the USVs. Specifically, each USV in the first formation framework has the individual objective of aiming to be located at a given static reference target according to its own interests. And the USVs in the second formation framework have the individual objectives of following a given time‐varying reference target. Considering that the individual objectives of the USVs may conflict with the group formation objective, the formation problem of USVs is transformed to a noncooperative game among multiple USVs. Two controllers based on the Nash equilibrium seeking strategy are designed to solve the formation problem of USVs with conflicting objectives. It is proved that the USVs' states form an equilibrium formation pattern driven by the designed controllers. Simulations are performed to verify the theoretical results.
Chapter
The multicopter formation is widely used in many different complex circumstances. And the semi-autonomous multicopter formation controlled with a single pilot on the ground draws people’s attention due to the great adaptability to various kinds of environments. In this paper, we focus on the problem of passing through narrow channels with semi-autonomous multicopter formation. Similar to the definition for a single multicopter, the altitude hold mode for the multicopter formation is defined first. Then, the formation decision-making and formation low-level control of the altitude hold mode are proposed one by one. Finally, a hardware-in-the-loop simulation and a real experiment are performed with four multicopters to pass through a narrow channel.
Article
The finite-horizon two-person zero-sum differential game is a significant technology to solve the finite-horizon spacecraft pursuit-evasion game (SPE game). Considering that the saddle point solution of the differential game usually results in solving a high-dimensional (24 dimensional in this paper) two-point boundary value problem (TPBVP) that is challengeable, a dimension-reduction method is proposed in this paper to simplify the solution of the 24-dimensional TPBVP related to the finite-horizon SPE game and to improve the efficiency of the saddle point solution. In this method, firstly, the 24-dimensional TPBVP can be simplified to a 12-dimensional TPBVP by using the linearization of the spacecraft dynamics; then the adjoint variables associated with the relative state variables between the pursuer and evader can be expressed in the form of state transition; after that, based on the necessary conditions for the saddle point solution and the adjoint variables in the form of state transition, the 12-dimensional TPBVP can be transformed into the solving of 6-dimensional nonlinear equations; finally, a hybrid numerical algorithm is developed to solve the nonlinear equations so as to obtain the saddle point solution. The simulation results show that the proposed method can effectively obtain the saddle point solution and is robust to the interception time, the orbital altitude and the initial relative states between the pursuer and evader.
Article
Video games, as a fast-growing industry in the experience economy, have gathered numerous game prosumers (playbors), who have created value for game enterprises. However, few studies have been conducted on the mechanism of game loyalty of playbors. This study investigates the effects of game experience and fanwork creation on game loyalty through the mediation effect of three types of perceived value: functional, hedonic, and intrinsic value. Based on the Chinese video game industry, the analysis shows that game experience and fanwork creation positively influence playbors’ game loyalty. Further, while intrinsic value plays a partial mediating role between game experience and game loyalty, it has a complete mediation effect between fanwork creation and game loyalty. Our findings have important implications for engagement and loyalty research in the context of the Chinese video game industry.
Article
Full-text available
In this paper, we investigate the formation control of multiple unmanned aerial vehicles (UAVs), specifically unmanned aircraft, in an obstacle-laden environment. The main contribution of this paper is to integrate the formation control, trajectory tracking, and obstacle/collision avoidance into one unified optimal control framework. A nonquadratic avoidance cost is innovatively constructed via an inverse optimal control approach, which leads to an analytical, distributed, and optimal formation control law. The stability and optimality of the closed-loop system are proven. In addition, the proposed optimal control law is dependent only on the information from the local neighbors, rather than all UAVs' information. Simulation of multiple UAVs' formation flying demonstrates the effectiveness of the integrated optimal control design with desired behaviors including formation flying, trajectory tracking, and obstacle/collision avoidance.
Article
Full-text available
This is the revised second edition of our 1982 book with the same title, which presents a rather comprehensive treatment of static and dynamic noncooperative game theory, with emphasis placed (as in the first and second editions) on the interplay between dynamic information patterns and the structural properties of several different types of equilibria. Whereas the second edition (1995) was a major revision with respect to the original edition, this Classics edition only contains some moderate changes with respect to the second one. There has been a number of reasons for the preparation of this edition: • The second edition was sold out surprisingly fast. • After some fifty years from its creation, the field of game theory is still very alive and active, as also reinforced by the selection of three game theorists (John Harsanyi, John Nash and Reinhard Selten) to share the 1994 Nobel prize in economics. Quite a few books on game theory have been published during the last ten years or so (though most of them essentially deal with static games only and are at the undergraduate level). • The recent interest in such fields as biological games, mathematical finance and robust control gives a new impetus to noncooperative game theory. • The topic of dynamic games has found its way into the curricula of many universities, sometimes as a natural supplement to a graduate level course on optimal control theory, which is actively taught in many engineering, applied mathematics and economics graduate programs. • At the level of coverage of this book, dynamic game theory is well established by now and has reached a level of maturity, which makes the book a timely addition to SIAM's prestigious Classics in Applied Mathematics series.
Article
Non-Zero-sum differential games where measurements of the state vector are possible only at discrete instants of time during the course of play are considered, and necessary conditions for the existence of a pair of sampled-data Nash controls are obtained. These conditions are different from those corresponding to the open-loop or closed-loop solutions. Linear quadratic games are then treated and a simple illustrative example which reduces to a pursuit-evasion game is presented.
Article
Whether providing automated passenger transport systems, exploring the hostile depths of the ocean or assisting soldiers in battle, autonomous vehicle systems are becoming an important fact of modern life. Distributed sensing and communication networks allow neighboring vehicles to share information autonomously, to interact with an operator, and to coordinate their motion to exhibit certain cooperative behaviors. The less structured the operating environment and the more changes the vehicle network experiences, the more difficult to grapple with problems of control become. Cooperative Control of Dynamical Systems begins with a concise overview of cooperative behaviors and the modeling of constrained non-linear dynamical systems like ground, aerial, and underwater vehicles. A review of useful concepts from system theory is included. New results on cooperative control of linear and non-linear systems and on control of individual non-holonomic systems are presented. Control design in autonomous-vehicle applications moves evenly from open-loop steering control and feedback stabilization of an individual vehicle to cooperative control of multiple vehicles. This progression culminates in a decentralized control hierarchy requiring only local feedback information. A number of novel methods are presented: parameterisation for collision avoidance and real-time optimisation in path planning; near optimal tracking and regulation control of non-holonomic chained systems; the matrix-theoretical approach to cooperative stability analysis of linear networked systems; the comparative argument of Lyapunov function components for analysing non-linear cooperative systems; and cooperative control designs. These methods are used to generate solutions of guaranteed performance for the fundamental problems of: optimised collision-free path planning; near-optimal stabilization of non-holonomic systems; and cooperative control of heterogenous dynamical systems, including non-holonomic systems. Examples, simulations and comparative studies bring immediacy to the fundamental issues while illustrating the theoretical foundations and the technical approaches and verifying the performance of the final control designs. Researchers studying non-linear systems, control of networked systems, or mobile robot systems will find the wealth of new methods and solutions laid out in this book to be of great interest to their work. Engineers designing and building autonomous vehicles will also benefit from these ideas, and students will find this a valuable reference.
Article
Formation initialization control is of paramount importance for building a drift-free relative orbit, and is a challenging problem due to the coupled translational and rotational dynamics, e.g. the orientation of the thrust vector is constrained by the attitude and its angular velocity. We establish a nonlinear coupled dynamic model for formation flying spacecraft, and develop a relative orbit and attitude controller. The attitude angular velocity induced thrust vector constraint is then converted into thrust vector maneuverability constraint, which is nonconvex and cannot be implemented in the optimization framework directly. Thus the orbit and attitude controller can be designed separately. When designing the relative orbit controller, we use a relaxation method to convexify the nonconvex constraint and tailor the optimization problem to a semidefinite program, because of its low complexity and the existence of deterministic convergence properties. Then a variable structure attitude controller is used to track the optimized thrust direction. The validity of the proposed approach is demonstrated in a typical application of a dual-spacecraft formation initialization.