No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Distributed Data-Driven UAV Formation Control Via Evolutionary Games: Experimental Results
Abstract
This work proposes a novel data-driven distributed formation-control approach based on multi-population evolutionary games, which is structured in a leader-follower scheme. The methodology considers a time-varying communication graph that describes how the multiple agents share information to each other. We present stability guarantees for configurations given by time-varying interaction networks, making the proposed method suitable for real-world problems where communication constraints change along the time. Additionally, the proposed formation controller allows for an agent to leave or enter the group without the need to modify the behaviors of other agents in the group. This game-theoretical approach is evaluated through numerical simulations and real outdoors experimental results using a fleet of aerial autonomous vehicles, showing the control performance.
... El objetivo principal es destacar la utilidad y la idoneidad de estas técnicas para modelar dinámicas de sistemas complejos de ingeniería, así como para diseñar estrategias de gestión y control siguiendo políticas particulares y contemplando restricciones físicas y operativas, tanto locales como globales. Las estrategias desarrolladas por medio de este paradigma son de fácil implementación física, como se puede ver en trabajos como Martinez-Piazuelo et al. (2022a); Barreiro-Gomez et al. (2021), en los que se muestran criterios para seleccionar parámetros y su implementación. Por otra parte, este nuevo paradigma, también es capaz de abarcar problemáticas como los retrasos (Obando et al., 2016;Park and Leonard, 2021), dando pie a una nueva alternativa respecto a otras técnicas desarrolladas, cuya implementación tiene sus dificultades. ...
... Se puede encontrar un resumen de los trabajos que se presentaron hasta el año 2017 en Quijano et al. (2017). Se han hecho contribuciones desde entonces en términos de dinámicas distribuidas en tiempo continuo (Barreiro-Gomez et al., 2017a) y en tiempo discreto (Martinez-Piazuelo et al., 2022a), así como en la combinación de técnicas en sistemas híbridos (Ochoa et al., 2021).Últimamente, se han desarrollado temas como aplicaciones para vehículos autónomos no tripulados (Barreiro-Gomez et al., 2021) o la combinación de técnicas de control para aplicaciones en redes de agua (Barreiro-Gomez et al., 2017b;Obando et al., 2022). Además, se han reportado recientemente resultados significativos sobre la relación entre estas poblaciones dinámicas y los equilibrios generalizados de Nash (Martinez-Piazuelo et al., 2022c,d,b;Sánchez-Amores et al., 2023;Martinez-Piazuelo et al., 2023). ...
En la última década, se han venido desarrollando técnicas inspiradas por la naturaleza y la economía con el fin de resolver problemas de control y toma de decisiones. En este artículo, se presenta este nuevo paradigma que combina los juegos poblacionales y los modelos dinámicos de pago. Se introducen conceptos fundamentales en torno a estas áreas, incluyendo un desarrollo matemático formal (basado en teoría de pasividad para sistemas dinámicos, estabilidad de Lyapunov e invarianza de conjuntos) que valida su uso tanto para abordar problemas de optimización como para diseñar sistemas de control en lazo cerrado con restricciones (físicas y operacionales). Específicamente, nos enfocamos en problemas cuyos objetivos se alinean con la distribución dinámica de recursos y el alcance de equilibrios generalizados de Nash. La pertinencia del paradigma formulado se ilustra a través de diferentes problemas de ingeniería con aplicaciones en múltiples campos.
... Trajectory tracking control of the vehicle aims to combine the actual deviation between the actual vehicle location and ideal trajectory, and minimize the lateral deviation and heading deviation through steering control, to achieve dynamic tracking of the referenced trajectory [9,10]. In order to guarantee the driving safety and comfort in trajectory tracking control, while taking into account the high precision and stability requirements of trajectory tracking control, Kim E et al. [11] applied the model predictive control method to propose a trajectory tracking method, to achieve accurate and smooth tracking. ...
... When the road friction coefficient is 0.3-1, the slip rate calculated in this paper is basically consistent with the slip rate output by Carsim. When the road friction coefficient is 0.1 and 0.2, there is a big difference between the slip ratio calculated according to Equations (8) and (9) and the slip ratio output by Carsim. The reason is that the calculation equations of the two are different. ...
This paper proposes an autonomous vehicle trajectory tracking system that fully considers road friction. When an intelligent vehicle drives at high speed on roads with different friction coefficients, the difficulty of its trajectory tracking control lies in the fast and accurate identification of road friction coefficients. Therefore, an improved strategy is designed based on traditional recursive least squares (RLS), which is utilized for accurate identification of the friction coefficient. First, the tire force and slip rate required for the estimation of the road friction coefficient by constructing the vehicle dynamics model and tire effective model are calculated. In this paper, a variable forgetting factor recursive least squares (VFF-RLS) method is proposed for the construction of the friction coefficient estimator. Second, the identified results are output to the model predictive controller (MPC) constructed in this paper as a way to improve tire slip angle constraints, to realize the trajectory tracking of the intelligent vehicle. Finally, the joint simulation test results of Carsim and Matlab/Simulink show that the trajectory tracking system based on the VFF-RLS friction coefficient estimator has outstanding tracking performance.
... The document [8] considers the perception degradation of UAVs, and a radar tracking model is proposed to improve the perception technology to assist decision-making. document [9] uses the data-driven distributed formation control method of multi-population evolutionary games to improve control performance. With the development of UAV technology, scholars have begun to study UAV dynamic task allocation and cooperative control in complex environments [10,11], then the intelligent optimization method is used to optimize [12]. ...
With the rapid development of digital intelligence, drones can provide many conveniences for people’s lives, especially in executing rescue missions in special areas. When executing rescue missions in remote areas, communication cannot be fully covered. Therefore, to improve the online adaptability of the task chain link in task planning with a complex system structure as the background, a distributed source-task-capability allocation (DSTCA) problem was constructed. The first task chain coordination mechanism scheme was proposed, and a DSTCA architecture based on the task chain coordination mechanism was constructed to achieve the online adaptability of the swarm. At the same time, the existing algorithms cannot achieve this idea, and the DSTCA-CBBA algorithm based on CNP is proposed. The efficiency change, agent score, and time three indicators are evaluated through specific cases. In response to sudden changes in nodes in the task chain link, the maximum spanning tree algorithm is used to reconstruct the task chain link in a short time, thereby completing the mission task assigned to the drone entity. Meanwhile, the experimental results also prove the effectiveness of the proposed algorithm.
... For example, an effective event-driven cooperative formation control protocol is proposed for multiple UAVs to reduce the communication frequency in [8,9]. Moreover, a novel data-driven distributed control approach based on evolutionary games is proposed to solve the multi-UAV formation tracking problem subject to communication constraints change over time in [10]. Although the aforementioned control schemes can realize the formation keeping and tracking of multiple UAVs, two-fold system performance indicators should be considered in the relevant formation controller design, i.e., the finitetime convergence performance and anti-saturated performance. ...
A highly effective control method is very important to guarantee the safety of the formation of flying missions for multiple unmanned aerial vehicles (UAVs), especially in the presence of complex flying environments and actuator constraints. In this regard, this paper investigates the formation tracking control problem of multiple UAVs in the presence of actuator saturation. Firstly, a brand-novel finite-time anti-saturated control scheme is proposed for multiple UAVs to track the desired position commands, wherein the tracking performance is tuned by introducing a logarithmic function-based state-mapping policy. Then, an adaptive scheme based on projection rules is devised to compensate for the negative effects brought by the actuator saturation. Based on the proposed formation tracking controller, the finite-time formation tracking performance tuning and control saturation problems can be addressed simultaneously with a comparatively allowable system robustness. Finally, three groups of illustrative examples are organized to verify the effectiveness of the proposed formation tracking control scheme.
... Numerous scholars have utilized evolutionary game theory to study cooperation problems, yielding significant results that have been widely applied to practical issues [24][25][26]. Given that the concept of stable cooperation and target assignment in a multi-UAV system, under limited information and resources, aligns with the principles of evolutionary game theory, a comprehensive analysis of the stable long-term trends of both parties engaged in the game becomes possible by leveraging these principles. ...
Target assignment has been a hot topic of research in the academic and industrial communities for swarms of multiple unmanned aerial vehicle (multi-UAVs). Traditional methods mainly focus on cooperative target assignment in planes, and they ignore three-dimensional scenarios for the multi-UAV network target assignment problem. This paper proposes a method for target assignment in three-dimensional scenarios based on evolutionary game theory to achieve cooperative targeting for multi-UAVs, significantly improving operational efficiency and achieving maximum utility. Firstly, we construct an evolutionary game model including game participants, a tactical strategy space, a payoff matrix, and a strategy selection probability space. Then, a multi-level information fusion algorithm is designed to evaluate the overall attack effectiveness of multi-UAVs against multiple targets. The replicator equation is leveraged to obtain the evolutionarily stable strategy (ESS) and dynamically update the optimal strategy. Finally, a typical scenario analysis and an effectiveness experiment are carried out on the RflySim platform to analyze the calculation process and verify the effectiveness of the proposed method. The results show that the proposed method can effectively provide a target assignment solution for multi-UAVs.
... It can also be said that evolutionary game theory uses cooperation to replace malignant competition to allocate the limited resources and further to maximize the overall combat effectiveness. Many researchers use evolutionary game theory to study cooperation problems, and have made many important achievements, which are widely used in solving practical problems [19][20][21]. The evolutionary task allocation has been studies in uncertain environments [22], Internet of Things [23], cloud manufacturing [24], bandwidth allocation [25], and resource-constrained heterogeneous environments [26]. ...
Autonomous air combat of unmanned aerial vehicles (UAVs) is a typical and difficult problem that must be studied in the field of military intelligence. The UAVs are required to work together to accomplish different tasks in the air combat. In this note, both task allocation and cooperative control of UAVs are studied for air combat applications. A two-layer framework of the air combat is established containing a task allocation upper layer and a cooperative control lower layer. In the task allocation upper layer, an evolutionary task allocation strategy is proposed for UAVs to allocate tasks by considering their cooperative behavior. After the task allocation, the UAVs, which are allocated for the same task, form the same group. In the cooperative control lower layer, a cooperative control approach is proposed for groups of UAVs to maintain a desired formation to accomplish the corresponding task. The evolutionary stable state and stabilization of the UAVs are achieved. Finally, both Software-in-the-Loop and Model-in-the-Loop simulations are conducted based on the RflySim platform to demonstrate the effectiveness of the proposed approach and further display the multi-UAV air combat vision realistically.
... Desde entonces, se han realizado aportes en términos de dinámicas distribuidas en tiempo continuo [3] y en tiempo discreto [13], así como la combinación con técnicas en sistemas híbridos [20]. Algunas aplicaciones en vehículos autónomos no tripulados [2] o combinando técnicas de control para aplicaciones en redes de agua [4,19] han sido parte de los temas desarrolladoś ultimamente. Recientemente, se han venido obteniendo resultados importantes en la relación de estas dinámicas poblaciones y los equilibrios generalizados de Nash [15,16]. ...
Este artículo se centra en resaltar la vasta utilidad de la teoría de juegos, en particular aquella basada en la vertiente dinámica evolutiva, dentro del modelado y control de sistemas dinámicos complejos y de gran escala. Concretamente, y a través de ejemplos claros y de aplicación real, se motiva el uso de las dinámicas poblacionales para el modelado de sistemas ciberfísicos, así como los modelos dinámicos de pago para complementar un sistema de control en lazo cerrado con restricciones (físicas y operacionales), cuyos objetivos se alinean con la distribución dinámica de recursos y el alcance de equilibrios generalizados de Nash.
Despite its groundbreaking success, multi-agent reinforcement learning (MARL) still suffers from instability and nonstationarity. Replicator dynamics, the most well-known model from evolutionary game theory (EGT), provide a theoretical framework for the convergence of the trajectories to Nash equilibria and, as a result, have been used to ensure formal guarantees for MARL algorithms in stable game settings. However, they exhibit the opposite behavior in other settings, which poses the problem of finding alternatives to ensure convergence. In contrast, innovative dynamics, such as the Brown-von Neumann-Nash (BNN) or Smith, result in periodic trajectories with the potential to approximate Nash equilibria. Yet, no MARL algorithms based on these dynamics have been proposed. In response to this challenge, we develop a novel experience replay-based MARL algorithm that incorporates revision protocols as tunable hyperparameters. We demonstrate, by appropriately adjusting the revision protocols, that the behavior of our algorithm mirrors the trajectories resulting from these dynamics. Importantly, our contribution provides a framework capable of extending the theoretical guarantees of MARL algorithms beyond replicator dynamics. Finally, we corroborate our theoretical findings with empirical results.
Path following accuracy and lateral stability of autonomous vehicle are the direct embodiment of control quality in intelligent driving. To achieve the coordination of path following and lateral stability, a novel nonlinear concurrent control method for autonomous vehicle considering high-frequency interference and actuator saturation is proposed to solve the coupling problem of multi-objective control with single actuator input. The vehicle dynamics model and path following model are expressed as state equations with the same control input by model integration transformation. At the same time, the actuator saturation and high-frequency interference of AGV model is considered, which is incorporated into the integrated state equation. The Hamilton function for path following and lateral stability control are designed respectively, the existence condition of controller design based on vehicle characteristics is discussed and proved, and a nonlinear concurrent control method of AGV is presented with the control system stability being analyzed. The results show that the designed controller can ensure the path following effect and maintain the lateral stability of AGV simultaneously.
This article addresses the position control issue of multi‐quadrotor unmanned aerial vehicle (QUAV) formation. Concerning the translational dynamic of a multi‐QUAV system, on the one hand, it is an under‐actuation dynamic; on the other hand, it does not satisfy the matching condition. These features will cause inevitable thorny in the formation position control design. Furthermore, because of the state coupling problem, the formation control of multi‐QUAV system is more challenging and knotty than the control of single QUAV system. To achieve this control, both backstepping technique and neural network (NN) approximation strategy are combined by introducing an intermediary control, where NN is employed to compensate the system uncertainty. However, since the traditional adaptive NN control methods need to train a large number of adaptive parameters for the high approximation accuracy, it will cause the heavy computing burden if traditional adaptive method is used for the QUAV formation control. The proposed adaptive NN strategy in this paper only requires training a scalar adaptive parameter, which is generated from the norm of NN weight vector or matrix, thereby significantly reducing computational burden. Finally, according to Lyapunov stability proof and computer simulation, it is demonstrated that the control tasks can be successfully accomplished.
In this article, we introduce first-order and zeroth-order Nash equilibrium seeking dynamics with fixed-time and practical fixed-time convergence certificates for noncooperative games having finitely many players. The first-order algorithms achieve exact convergence to the Nash equilibrium of the game in a finite time that can be additionally upper bounded by a constant that is independent of the initial conditions of the actions of the players. Moreover, these fixed-time bounds can be prescribed
a priori
by the system designer under an appropriate tuning of the parameters of the algorithms. When players have access only to measurements of their cost functions, we consider a class of distributed multitime scale zeroth-order model-free adaptive dynamics that achieve semiglobal practical fixed-time stability, qualitatively preserving the fixed-time bounds of the first-order dynamics as the time scale separation increases. Moreover, by leveraging the property of fixed-time input-to-state stability, further results are obtained for mixed games where some of the players implement different seeking dynamics. Fast and slow switching communication graphs are also incorporated using tools from hybrid systems. We consider potential games as well as general nonpotential strongly monotone games. Numerical examples illustrate our results.
Distributed population dynamics offer a game-theoretical framework for distributed decision making. As such, the application of these methods to the control of networked systems has been widely studied in the literature. The theoretical analyses available in the literature have only considered continuous-time formulations of distributed population dynamics. However, given that modern computers can only process information in a discrete-time fashion, the practical implementation of distributed population dynamics-based methods is inevitably discrete. In consequence, it is paramount to extend the available theory to a more implementable discrete-time approach. For that reason, in this article, we provide a discrete-time analysis of a general class of distributed population dynamics, and we derive sufficient conditions on the discretization time to ensure the asymptotic stability of the discretized dynamics. To illustrate the relevance and performance of the proposed methods, we apply the developed theory to distributed optimization and control problems, including a real multirobot platform, which consider noncomplete communication networks and coupled constraints.
Division of labor is a widely studied subject of collective behavior in natural systems. It is concerned with the question of how the regulation of the division of labor may contribute to the performance of the multi-agent system. This brief investigates the evolution of the division of labor with three strategies by employing the evolutionary game theory. Thus, these available strategies are provided for the unstructured multi-agent system: strategy A (performing task A), strategy B (performing task B), and strategy C (not performing any task but only interfere other's actions). Depending on the parameter settings in the payoff matrix, four specific scenarios occur and the corresponding stability of the equilibrium points there are provided by calculation. Numerical calculation results are also provided for an intuitive description of the strategy evolution state. Results show that larger synergistic benefit contributes to the effective division of labor in the system, providing high benefits for the multi-agent system. These results help design a effective mechanism where self-organized division of labor occurs in the form of maximizing the benefit of the multi-agent system.
This paper tackles the problem of selecting stable relays for Optimized Link State Routing protocol (OLSR) in urban Internet of Vehicles (IoV) in the presence of Unmanned Aerial Vehicles (UAVs). With the evolution of Internet of Things (IoT), IoV emerged from the conventional vehicular ad-hoc network to enable Vehicle to Everything (V2X) communication through different routing protocols. In OLSR protocol, Multi-Point Relays (MPRs) are selected based on their reachability and uniqueness to route information. However, urban environments are characterized by the rapid changes in topology due to the presence of intersections and traffic lights. Although clusters were proposed as a solution, selecting stable heads and MPRs are of significance, considering environment metrics to provide higher connectivity. As a solution to improve routing in IoV, a distributed Gale-Shapley matching game is proposed for stable clustering and MPR selection, utilizing nodes' quality of service (QoS). Nodes' QoS is calculated using Bayesian Belief function of stable connections utilizing environment metrics. For further enhancement in the network performance, UAVs are integrated in the network of vehicles. Conducted simulations show high percentage of stable heads and MPRs for the proposed model compared to benchmark protocols. In addition, the proposed model shows high performance in terms of packet delivery ratio, throughput and End-to-End delay, which are further improved by the presence of UAVs.
In this paper, a new cooperative occupancy decision-making problem for multiple unmanned aerial vehicles (UAVs) in beyond-visual-range (BVR) air combat is proposed; that is, UAV formation on each side first decides the occupancy positions of their own UAVs, then decides the target that each UAV will confront, so that each side can occupy the greatest predominance and be subject to the smallest threat possible. By analyzing the influence of the occupancy positions of the UAVs on the predominance gained by both sides and the threat against them, the problem is described as a zero-sum matrix game. Because the strategy combinations of this game are very numerous, as the best algorithm for the zero-sum matrix game, the Double Oracle (DO) algorithm still has difficulty solving the game model effectively. Therefore, we propose a DO combined with neighborhood search (DO-NS) algorithm to deal with the large-scale zero-sum matrix game. The results show that compared with the DO algorithm, the DO-NS algorithm can ensure the effectiveness of the computational time and solution quality.
To combat the variety uncertainties in topologies, dynamical models and disturbances, this paper presents a distributed sliding mode control strategy for formation control of multiple AVs. In this scheme, all collected information of each AV is used for its control and different perturbations are dealt with separately to reduce the contractions among them. Furthermore, a distributed adaptive algorithm is designed to replace the witching part for smoothness of control. The convergence of sliding surfaces of both two controllers is analysed theoretically. The sliding dynamics is affected by both the feedback and interaction topology. The existing decoupling method to handle the variety topologies can be used to synthesize the sliding dynamics. Finally, this approach has been applied to vehicular platooning and validated by several comparative simulations. The results show that the proposed method can control multiple AVs better than the state feedback strategy.
This paper is to address a cooperative moving path following (CMPF) problem, in which a fleet of fixed-wing unmanned aerial vehicles (UAVs) are required to converge to and follow a desired geometric moving path while satisfying prespecified speed and spatial constraints. A representative application of the CMPF problem is the challenging mission scenario where a group of UAVs are tasked to track a moving ground target. The proposed methodology is based on the insight that a vehicle can follow a given path only through attitude control, thus leaving its speed as an extra input to be used at the coordination level. To deal with moving path following (MPF) of a single UAV, a non-singular control law is derived to steer the vehicle along the desired moving path which avoids the singularity problem in the previous MPF strategy. For multi-UAV coordination, a pursuit strategy is employed with the introduction of a virtual leader. To account for speed constraints and collision avoidance, conditions are derived under which the combined MPF and multi-UAV coordination closed-loop system is asymptotically stable while speed and spatial constraints are satisfied. Further simulation has been performed to demonstrate the effectiveness of the proposed method.
This paper presents a real-time game theoretic planning algorithm for a robotic vehicle (e.g. a drone or a car) to race competitively against multiple opponents on a racecourse. Our algorithm plans receding horizon trajectories to maximally advance the robot along the racecourse, while taking into account the opponents' intentions and responses. We build on our previous work [5], which only considered racing with two robots. Our algorithm uses an iterative best response scheme with a new sensitivity term to find approximate Nash equilibria in the space of the multiple robots' trajectories. The sensitivity term seeks Nash equilibria that are advantageous to the ego robot. We demonstrate our approach through extensive multi-player racing simulations, where our planner exhibits rich behaviors such as blocking, overtaking, nudging or threatening, similar to what we observe from racing with human participants. Statistics also reveal that our game theoretic planner largely outperforms a baseline model predictive controller that does not consider the opponents' responses. Experiments are conducted with four quadrotor aerial robots to validate our approach in real time and with physical robot hardware.
This paper introduces a novel data-driven risk metric and assessment method for UAVs operating in environments typically encountered in civilian applications. A truly “data-driven risk measure” is derived through a probabilistic formulation that not only accounts for the intrinsically stochastic nature of the considered environmental factors (such as weather and signal strength), but also incorporates extrinsic prediction uncertainties originating from the geographical sparsity of data collection sources. We present a data-driven modeling of the stochastic environmental factors using Gaussian process-based function approximations. Notably, the proposed mathematical definition of the risk metric is based on the probabilistic predictions of such a Gaussian process model and introduced through a path-integral formulation. The problem of minimizing operational risk for multiple UAVs in partially unknown environments is then defined in a multicriteria optimization framework to address the trade-off between the path-integral risk measure and classical path-efficiency (distance). We show that such approach can be embedded into current standard risk assessment methods which could be easily integrated into UAVs traffic management initiatives. We analyze the results through a number of simulations, including realistic scenarios.
In the task allocation of multi-robot system, the communication is an important condition to ensure global consistency. Unfortunately, with the popularity of WLAN, the congestion and interference among the bands are particularly severe, making traditional task allocation methods which rely on communication can not work. In this paper, the task allocation without communication is considered to be an incomplete information game, where game theory is employed to realize the task allocation and cooperation of soccer robots. Firstly, the joint probability distribution of the robot type is established according to the distance information, which can be employed to solve the incomplete information static game. Then the ball velocity information is added to modify the joint probability distribution and the incomplete information dynamic game can be solved in the same way. The experimental results show that the success rate of task allocation without communication can be improved effectively using the proposed method.
The decentralized consensus control of a formation of rigid body spacecraft is studied in the framework of geometric mechanics while accounting for a constant communication time delay between spacecraft. The relative position and attitude (relative pose) dynamics are modeled in the framework of the Lie group SE(3) while the communication topology is modeled as a digraph. The consensus problem is converted into a local stabilization problem of the error dynamics associated with the Lie algebra se(3) in the form of linear time-invariant delay differential equations (DDEs) with a single discrete delay in the case of a circular orbit while it is in the form of linear time-periodic DDEs in the case of an elliptic orbit, in which the stability may be assessed using infinite-dimensional Floquet theory. The proposed technique is applied to the consensus control of four spacecraft in the vicinity of a Molniya orbit.
In this paper, distributed optimization control for a group of autonomous Lagrangian systems is studied to achieve an optimization task with local cost functions. To solve the problem, two continuous-time distributed optimization algorithms are designed for multiple heterogeneous Lagrangian agents with uncertain parameters. The proposed algorithms are proved to be effective for those heterogeneous nonlinear agents to achieve the optimization solution in the semi-global sense, even with the exponential convergence rate. Moreover, simulation adequately illustrates the effectiveness of our optimization algorithms.
In this paper, a game-theoretical autonomous decision-making approach for efficient deployment of unmanned aerial vehicles (UAVs) in a multi-level and multi-dimensional assisted network is analyzed. The UAVs have directional antennas that work as wireless stations, which provide the best coverage for multiple ground mobile/fixed users. In general, UAVs work in a cooperative manner for achieving the suitable deployment with the optimal coverage values for the candidate region. In this paper, a game theory concept is used and the payoff function for each UAV is defined based on the coverage probability value, which depends on the altitude and the characteristic of antennas in the UAVs. We introduce a mathematical formulation for evaluating the payoff values based on a set of actions for each UAV, and the Nash equilibrium for this kind of game. This approach works in an intelligent way based on the interactions between the UAVs and their neighbors in a connected network and it might work even in harsh environments. In order to minimize interference, the UAVs’ altitudes are adjusted based on the antennas and other deployment requirements (i.e. search and surveillance purposes) by using the minimum number of UAVs to cover the candidate geographical region. The simulation results show that the proposed approach achieves the maximum coverage value, converges fast with the environmental changes based on the power levels, and robust for failure scenarios. Finally, we compare our approach against one of the traditional approaches called Collaborative Visual Area Coverage Approach (CVACA) based on uniform coverage quality. The simulation results show that the game approach outperforms the traditional approach in term of the coverage value and the computational time.
In this brief, a distributed optimal control method via reinforcement learning is proposed to address the heterogeneous unmanned aerial vehicle (UAV) formation trajectory tracking problem. The UAV formation is composed of a virtual leader with limited nonzero input and several follower vehicles with different unknown dynamics. The proposed control law contains a distributed observer and a model-free off-policy reinforcement learning (RL) protocol. The distributed optimal trajectory tracking problem is formulated for the heterogeneous formation system. A RL algorithm is designed to obtain the optimal control input online without any knowledge of the followers’ dynamics. Simulation example illustrates the effectiveness of the proposed method.
To understand the emergence and sustainment of cooperative behavior in interacting collectives, we perform global convergence analysis for replicator dynamics of a large, well-mixed population of individuals playing a repeated snowdrift game with four typical strategies, which are always cooperate (ALLC), tit-for-tat (TFT), suspicious tit-for-tat (STFT) and always defect (ALLD). The dynamical model is a three-dimensional ODE system that is parameterized by the payoffs of the base game. Instead of routine searches for evolutionarily stable strategies and sets, we expand our analysis to determining the asymptotic behavior of solution trajectories starting from any initial state, and in particular show that for the full range of payoffs, every trajectory of the system converges to an equilibrium point. What enables us to achieve such comprehensive results is studying the dynamics of two ratios of the state variables, each of which either monotonically increases or decreases in the half-spaces separated by their corresponding planes. The convergence results highlight two findings. First, the inclusion of TFT- and STFT-players, the two types of conditional strategy players in the g
To ensure the information security of UAV groups, maximize the detection range of the UAV group and minimize the communication range, this paper improves Olfati–Saber’s flocking algorithm using virtual leader and integrates the multiple UAVs with real environment. The improved swarm control algorithm can help to make all agents accurately track the speed of virtual leaders. Then, we introduce the conception of virtual communication circle to control the communication power of each UAV, and the moving function of the target position is improved to meet the limited conditions of given communication distance, to ensure the non-collision and stable communication of the UAV during the moving process. All UAVs in the group system will keep converging, avoiding collision and the speed is the same as that of the virtual pilot. Therefore, multiple UAVs can track the virtual pilot for cluster flight and a distributed cooperative control strategy can be acquired. The simulation results show that the strategy proposed in this paper makes a better communication performance among UAVs in real time, and the UAV groups achieve the information more timely, which is proved to be an effective way to ensure the security communication among multiple UAVs.
The movement of cooperative robots in a densely cluttered environment may be impossible if the formation type is invariant. Hence, we investigate a novel method of switching time-invariant formation control for a group of heterogeneous autonomous vehicles, which may include Unmanned Ground Vehicles (UGV) and Unmanned Aerial Vehicles (UAV). We have extended a Negative Imaginary (NI) consensus control approach to switch the formation shape of the robots whilst only using the relative distances between agents, in combination with those between agents and obstacles. All agents can automatically create a new safe formation to overcome obstacles, then restore the prototype formation once the obstacles are cleared. We address the oscillation and local minima problems, generated by existing algorithms, using a newly developed circle fitting approach. We also highlight the advantages of our proposed algorithms over the kinematic controller and the conventional Artificial Potential Field (APF) controller from both theoretical and experimental standpoints. Simulation and experimental results are then analyzed to validate the feasibility of our proposed approach.
We present a novel game-theoretic (GT) and reinforce-ment learning (RL) framework for computational offloading in the mobile edge computing (MEC) network operated by multiple service providers (SPs). The network is formed by MEC servers installed at stationary base stations (BSs) and unmanned aerial vehicles (UAVs) deployed as quasi-stationary BSs. Since computing powers of MEC servers are limited, the BSs in proximity can form coalitions with shared data processing resources to serve their users more efficiently. However, as BSs can be privately-owned or controlled by different SPs, in any coalition, the BSs: i) take only the actions that maximize their long-term payoffs; ii) do not coordinate their actions with other BSs in the coalition. That is, inside each coalition, BSs act in an independent and self-interested manner. Therefore, the interactions among BSs cannot be described by conventional coalitional games. Instead, the network operation is modeled by a two-level hierarchical model. The upper level is a cooperative game that defines the process of coalition formation. The lower level comprises the set of non-cooperative sub-games to represent a self-interested and independent behavior of BSs in coalitions. To enable each BS to select a coalition and decide on its action maximizing its long-term payoff, we propose two algorithms that combine coalition formation with RL and prove that these algorithms converge to the states where the coalitional structure is strongly stable and the strategies of BSs are in the mixed-strategy Nash equilibrium (NE).
Unmanned aerial vehicles (UAVs) can be deployed as wireless relays or aerial base stations to improve network connectivity and coverage in cellular networks. UAVs can also be used to significantly enhance the performance of mobile ad-hoc networks and wireless sensor networks. In the future, UAVs are expected to become an integral part of the fifth generation wireless networks as well as key enablers of the coming massive Internet of Things. However, there are still many challenging issues in designing architectures and deployment of UAV-based networks. To address the issues, game theory has recently been adopted as an effective tool for modelling and analyzing problems in UAV-aided networks. In this paper, we survey the applications of game theory in solving various UAV-assisted networks challenges. We first provide a brief introduction to wireless communications with UAVs and then introduce basic game theory concepts and their relation to wireless networks. We further present the classification and brief introduction to the games applied to solve problems in UAV-aided networks. We then provide a comprehensive literature review on game-theoretic techniques utilized in dealing with challenges in the UAV-based wireless networks. Finally, we introduce advanced distributed schemes for interference management in large UAV-assisted communication networks. This paper aims to provide readers with an understanding of UAV-aided networks in terms of their architecture, benefits, challenges, and various game theoretical solutions applied to these communications networks.
There is the recent boom in investigating the control of evolutionary games in multi-agent systems, where personal interests and collective interests often conflict. Using evolutionary game theory to study the behaviors of multi-agent systems yields an interdisciplinary topic which has received an increasing amount of attention. Findings in real-world multi-agent systems show that individuals have multiple choices, and this diversity shapes the emergence and transmission of strategy, disease, innovation and opinion in various social populations. In this sense, the simplified theoretical models in previous studies need to be enriched, though the difficulty of theoretical analysis may increase correspondingly. Here, our objective is to theoretically establish a scenario of four strategies, including competition among the cooperatives, defection with probabilistic punishment, speculation insured by some policy, and loner. And, the possible results of strategy evolution are analyzed in detail. Depending on the initial condition, the state converges either to a domination of cooperators, or to a rock-scissors-paper type heteroclinic cycle of three strategies.
In this paper, a novel method is suggested for the position and attitude tracking control of a quadrotor UAV in the presence of parametric uncertainties and external disturbance. The proposed method combines neural network adaptive scheme with sliding mode control, which preserves the advantages of the two methods. Firstly, dynamic model of quadrotor is divided into two fully actuated and under actuated subsystems. Secondly, sliding mode controllers are corresponding designed for each subsystem, and their coefficients in sliding manifolds are adaptively tuned by the neural network method. In each section, using Lyapunov theory, stability of closed loop system is proven.
Finally, the method is examined for a square path tracking and a maximum overshoot of 7.5133% and a settling time 5.6648 s are obtained. By comparing the results obtained through different methods, it is concluded that the proposed controller provides the following main advantages: (1) good transient and steady state behaviors, (2) insensitivity to parameter variations, (3) disturbance rejection capability, and (4) remarkable stability and performance robustness. Hence, for operational purposes in which the fast and accurate response are of crucial importance, using the neural network-based adaptive sliding mode control approach is recommended.
This work presents a framework for planning and perception for multi-robot exploration in large and unstructured 3D environments. We employ Gaussian mixtures to model complex environment geometries while maintaining a small memory footprint for mapping and thus enable distributed operation with a low volume of communication. A finite-horizon, information-based planner optimizes sequences of observations locally while accounting for the global distribution of information in the robot state space. A receding-horizon planner reasons about redundancies in observations to maximize information gain locally while a library of informative views models the global distribution of information in the environment. Simulation results demonstrate that the proposed system is able to maintain efficiency and completeness in exploration while only requiring a low rate of communication.
This paper presents a two-stage cascade control framework to solve hierarchically the trajectory tracking problem of a Tilt-rotor Unmanned Aerial Vehicle (UAV) carrying a suspended load. Initially, a nonlinear dynamic model is presented, which is after decoupled into two subsystems. The outer control system is designed by means of a robust tube-based Model Predictive Control (MPC) strategy, which is used to control the UAV's planar motion and stabilize the suspended load. For the inner control system, the input-output feedback linearization (IOFL) technique combined with the dynamic extension approach and a discrete mixed H2/H∞ controller is considered to control the UAV's altitude and attitude. Simulations results are carried out to corroborate the proposed control strategy.
Cooperative games-based robot cooperation is analysed for reoccurring scenarios. It is shown that potential games can be used for robot coordination when the robots have a shared objective. By observing each others' behaviour in similar scenarios, they estimate each other's expected actions, which they use for their own choice of action. The resulting learning scheme can enable "tuning" of smooth cooperation by task allocation in teams of robots for various goals and in reoccurring scenarios of their environment. The theoretical results and methods are illustrated in simulation.
In this paper a geometric approach to the trajectory tracking control of Unmanned Aerial Vehicles (UAVs) with thrust vectoring capabilities is proposed. The control problem is developed within the framework of geometric control theory, yielding a control law that is independent of any parametrization of the configuration space. The proposed design works seamlessly when the thrust vectoring capability is limited, by prioritizing position over attitude tracking. The control law guarantees almost-global asymptotic tracking of a desired full-pose (attitude and position) trajectory that is compatible with the platform underactuation according to a specific trackability condition. Finally, a numerical example is presented to test the proposed control law on a tilt-rotor quadcopter UAV. The generality of the control strategy can be exploited for a broad class of UAVs with thrust vectoring capabilities.
In this paper, we study the evolutionary dynamics of two different types of communities in an evolving environment. We model the dynamics using an evolutionary differential game consisting of two sub-games: 1) a game between two different communities and 2) a game between communities and the environment. Our interest is to clarify when the two communities and environment can coexist dynamically under the feedback from the changing environment. Mathematically speaking, we show that for specific game payoffs, the corresponding three-dimensional replicator dynamics induced by the evolutionary game have an infinite number of periodic orbits.
To gain a better understanding of environmental processes we are interested in the problem of deploying multi-robot systems for efficient collection of environmental data. For long-term autonomy, enabling persistent monitoring, it is important to consider the spatio-temporal variations of environmental phenomena. We develop a multi-robot persistent path planning method that reduces uncertainty in the environmental model. Our framework contains two components: the first component computes potential observation points that minimize model prediction uncertainty, and the second component uses this for online planning of multi-robot paths, while also taking into account the efficiency of information collection. We validated our method via simulations, and the results show that it produces multi-robot routing paths that are conflict-free, informative, and adaptive to the environmental dynamics.
Game dynamics have been widely used as learning and computational tool to find evolutionarily stable strategies. Nevertheless, most of the existing evolutionary game dynamics, i.e., the replicator, Smith, projection, Brown-Von Neumann-Nash, Logit and best response dynamics have been analyzed only in the unconstrained case. In this work, we introduce novel evolutionary game dynamics inspired from a combination of imitation dynamics. The proposed approach is able to satisfy both upper-and lower-bound constraints. Moreover, dynamics have asymptotic convergence guarantees to a generalized evolutionarily stable strategy. We show important features of the proposed game dynamics such as the positive correlation and invariance of the feasible region. Several illustrative examples handling population state constraints are provided.
In this paper, we focus on the control of multiagent formations with hybrid communication topology through a distance-based approach. By saying hybrid topology, we mean that the communication topology contains both undirected and directed links, or the underlying graph of the formation contains both undirected and directed edges. A new type of graph, ie, hybrid graph, is introduced. We discuss the persistence of hybrid graphs and present the persistence verification strategy for hybrid graphs. It is proved that all the minimally persistent hybrid graphs can be obtained from persistent directed graphs by the operation of edge transformation. As the main result, it is shown that multiagent formations modeled by acyclic persistent hybrid graphs can be stabilized locally under distance-based controllers.
The attitude control of quadrotor Unmanned Aerial Vehicle (UAV) is investigated. The aim of the paper is to develop a continuous multivariable attitude control law, which drives the attitude tracking errors of quadrotor UAV to zero in finite time. Firstly, a multivariable super-twisting-like algorithm (STLA) is proposed for arbitrary order integrator systems subject to matched disturbances. A discontinuous integral term is incorporated in the control law in order to compensate the disturbances. A rigorous proof of the finite time stability of the close-loop system is derived by utilizing the Lyapunov method and the homogeneous technique. Then, the implementation of the developed method in an indoor quadrotor UAV is performed. The remarkable features of the developed algorithm includes the finite time convergence, the chattering suppression and the nominal performance recovery. Finally, the efficiency of the proposed method is illustrated by numerical simulations and experimental verification.
Recently, there has been an increasing interest in the control community in studying large-scale distributed systems. Several techniques have been developed to address the main challenges for these systems, such as the amount of information needed to guarantee the proper operation of the system, the economic costs associated with the required communication structure, and the high computational burden of solving for the control inputs for largescale systems.
This paper presents a control technique based on distributed population dynamics under time-varying communication graphs for a multi-agent system structured in a leader-follower fashion. Here, the leader agent follows a particular trajectory and the follower agents should track it in a certain organized formation manner. The tracking of the leader can be performed in the position coordinates x, y, and z, and in the yaw angle φ. Additional features are performed with this method: each agent has only partial knowledge of the position of other agents and not necessarily all agents should communicate to the leader. Moreover, it is possible to integrate a new agent into the formation (or for an agent to leave the formation task) in a dynamical manner. In addition, the formation configuration can be changed along the time, and the distributed population-games-based controller achieves the new organization goal accommodating conveniently the information-sharing graph in function of the communication range capabilities of each UAV. Finally, several simulations are presented to illustrate different scenarios, e.g., formation with time-varying communication network, and time-varying formation.
Population dynamics have been widely used in the design of learning and control systems for networked engineering applications, where the information dependency among elements of the network has become a relevant issue. Classic population dynamics (e.g., replicator, logit choice, Smith, and projection) require full information to evolve to the solution (Nash equilibrium). The main reason is that classic population dynamics are deduced by assuming well-mixed populations, which limits the applications where this theory can be implemented. In this paper, we extend the concept of population dynamics for nonwell-mixed populations in order to deal with distributed information structures that are characterized by noncomplete graphs. Although the distributed population dynamics proposed in this paper use partial information, they preserve similar characteristics and properties of their classic counterpart. Specifically, we prove mass conservation and convergence to Nash equilibrium. To illustrate the performance of the proposed dynamics, we show some applications in the solution of optimization problems, classic games, and the design of distributed controllers.
Time-varying formation control problems for unmanned aerial vehicle (UAV) swarm systems with switching interaction topologies are studied. Necessary and sufficient conditions for UAV swarm systems with switching interaction topologies to achieve predefined time-varying formations are proposed. Based on the common Lyapunov functional approach and algebraic Riccati equation technique, an approach to design the formation protocol is presented. An explicit expression of the formation reference function is derived to describe the macroscopic movement of the whole UAV formation. A quadrotor formation platform consisting of four quadrotors is introduced. Outdoor experiments are performed to demonstrate the effectiveness of the theoretical results.
Game theory has been employed traditionally as a modeling tool for describing and influencing behavior in societal systems. Recently, game theory has emerged as a valuable tool for controlling or prescribing behavior in distributed engineered systems. The rationale for this new perspective stems from the parallels between the underlying decision-making architectures in both societal systems and distributed engineered systems. In particular, both settings involve an interconnection of decision-making elements whose collective behavior depends on a compilation of local decisions that are based on partial information about each other and the state of the world. Accordingly, there is extensive work in game theory that is relevant to the engineering agenda. Similarities notwithstanding, there remain important differences between the constraints and objectives in societal and engineered systems that require looking at game-theoretic methods from a new perspective. This chapter provides an overview of selected recent developments of game-theoretic methods in this role as a framework for distributed control in engineered systems.
This paper presents a novel asymptotic tracking controller for an underactuated quadrotor unmanned aerial vehicle using the robust integral of the signum of the error (RISE) method and an immersion and invariance (I&I)-based adaptive control methodology. The control system is decoupled into two parts: the inner loop for attitude control and the outer loop for position control. The RISE approach is applied in the inner loop for disturbance rejection, whereas the I&I approach is chosen for the outer loop to compensate for the parametric uncertainties. The asymptotic tracking of the time-varying 3-D position and the yaw motion reference trajectories is proven via the Lyapunov-based stability analysis and LaSalle's invariance theorem. Real-time experiment results, which are performed on a hardware-in-the-loop simulation testbed, are presented to illustrate the performance of the proposed control scheme.
The problem of intercepting a maneuvering target at a prespecified impact angle is posed in nonlinear zero-sum differential games framework. A feedback form solution is proposed by extending state-dependent Riccati equation method to nonlinear zero-sum differential games. An analytic solution is obtained for the state-dependent Riccati equation corresponding to the impact-angle-constrained guidance problem. The impact-angle-constrained guidance law is derived using the states line-of-sight rate and projected terminal impact angle error. Local asymptotic stability conditions for the closed-loop system corresponding to these states are studied. Time-to-go estimation is not explicitly required to derive and implement the proposed guidance law. Performance of the proposed guidance law is validated using two-dimensional simulation of the relative nonlinear kinematics as well as a thrust-driven realistic interceptor model. Copyright © 2014 by the American Institute of Aeronautics and Astronautics, Inc. All rights reserved.
Under an extended proportional-integral (PI) control scheme, a new class of distributed average tracking (DAT) algorithms has been developed for three different kinds of references: references with steady states, references with bounded derivatives, and references with a common derivative. The class of DAT algorithms are further applied to solve the DAT problem for Euler-Lagrange (EL) systems.
In distributed optimization problems, each agent can get information only from a neighborhood defined by a network topology to minimize a global objective function in a networked system. To solve this problem, we present a local strategy based on population dynamics, in order to perform a dynamic resource allocation in a system described by connected graph. The local replicator equation (LRE) is applied to a lighting control system to show that the achieved equilibrium point is an optimal solution to certain distributed optimization problems. We present some simulation results with different conditions, and a stability analysis to guarantee the convergence of the proposed algorithm.
This paper investigates the cooperative tracking control problem for a group of Lagrangian vehicle systems with directed communication graph topology. All the vehicles can have different dynamics. A design method for a distributed adaptive protocol is given which guarantees that all the networked systems synchronize to the motion of a target system. The dynamics of the networked systems, as well as the target system, are all assumed unknown. A neural network (NN) is used at each node to approximate the distributed dynamics. The resulting protocol consists of a simple decentralized proportional-plus-derivative term and a nonlinear term with distributed adaptive tuning laws at each node. The case with nonconstant NN approximation error is considered. There, a robust term is added to suppress the external disturbances and the approximation errors of the NNs. Simulation examples are included to demonstrate the effectiveness of the proposed algorithms.
Operating sensor-carrying uavs in a decentralized data-driven control of cooperating sensor-carrying uavs in a cooperating sensor-carrying UAVs in a multi-objective monitoring scenario
- J Euler
- O Stryk
J. Euler, O. von Stryk, Operating sensor-carrying uavs in a decentralized data-driven control of cooperating
sensor-carrying uavs in a cooperating sensor-carrying UAVs in a multi-objective monitoring scenario, IFAC
PapersOnLine 2017 (2017) 15828-15834.
Evolutionary game dynamics for two interacting populations in a co-evolving environment
- Gond