No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Multi-UAV target search using decentralized gradient-based negotiation with expected observation
Supplementary resources (2)
Data
March 2015
Data
March 2015
... This approach leverages probabilistic models for the targets location and dynamics, and for the sensors behavior, as core inputs for PS algorithms. The concept of a belief or target probability map, central to the PS approach, represents an estimated distribution of a target's location based on prior information available before the search mission starts, which may consider factors like the last known location of the S. Pérez-Carabaza et al. others maintain separate beliefs for different targets [8,22,24,26]. The separate belief approach is particularly beneficial for modeling heterogeneous targets and maintaining independent information for each, allowing for tailored optimization strategies based on individual target characteristics. ...
... The followed approach assigns individual beliefs and motion models to each target, allowing for the customization of search strategies, making operations more targeted and efficient by accommodating each target's unique characteristics. This implies that separate Bayesian filters can be instantiated to maintain an independent belief for each target [22,24,26]. An example application in a search and rescue application would be the search of two independent hikers lost in the bush [26]. ...
... Among the multiple-target works, two distinct philosophies to model target uncertainty can be distinguished. First, the approaches [8,22,24,26], labeled as multiple-beliefs, consider, as described in Section 2, a known number of independent targets and use different probability density functions to model the available location information for each of them. This implies that separate Bayesian filters can be instantiated to maintain an independent belief for each target [22,24,26]. ...
and military surveillance. These operations, characterized by complex and uncertain environments, demand efficient UAV trajectory optimization. The multi-target version of PS introduces additional challenges, due to their higher complexity and the need to wisely distribute the UAV's efforts among multiple targets. In order to tackle the under-explored multi-target aspect of MTS, we optimize the time to find all targets with new Ant Colony Optimization (ACO)-based planner. This novel optimization criterion is formulated using Bayes' theory, considering probability models of the targets (initial belief and motion model) and the sensor likelihood. Our work contributes significantly by (i) developing an objective function tailored for multi-target MTS, (ii) proposing an ACO-based planner designed to effectively handle the complexities of multiple moving targets, and (iii) introducing a novel constructive heuristic that is used by the ACO-based planner, specifically designed for the multi-target MTS problem. The efficacy of our approach is demonstrated through comprehensive analysis and validation across various scenarios, showing superior performance over existing methods in complex multi-target MTS problems.
... Moreover, the UAVs and sensor trajectories can be obtained by optimising probabilistic utility functions that simultaneously handle the uncertainty in the target (location and movements) and sensor (observations). Examples of these functions and of works that present strategies to optimise them, are the entropy (Yang et al., 2002), the information gain (Carpin et al., 2013;Grocholsky et al., 2006;Hu et al., 2014), the probability of detection (Delle Fave et al., 2010;Fedorov, 2019;Kratzke et al., 2010;Lanillos et al., 2014;Li et al., 2021;Saadaoui et al., 2018;Tisdale et al., 2009;Wang et al., 2017;Wong et al., 2005;Yao et al., 2017Yao et al., , 2019 and the expected time of detection (Lanillos et al., 2013;Perez-Carabaza et al., 2017, 2016Pérez-Carabaza, Scherer et al., 2019;Riehl et al., 2011). The last two utility functions are especially interesting in missions where: a) it is possible to know and exploit a probability distribution of the initial target location and b) critical to detect the target as soon as possible. ...
... In this regard, some works take into account the existence of non-flying zones, UAV collisions, communication requirements, the UAV fuel consumption, the length or smoothness of the UAV trajectories, and/ or the area coverage (Carpin et al., 2013;Li et al., 2021;Perez-Carabaza et al., 2017, 2016Pérez-Carabaza, Scherer et al., 2019;Yang et al., 2002). Alternatively, and in order to shorten the gap between the solutions (UAVs and sensor trajectories) proposed by the optimiser and its application to real-world search and rescue missions, a few optimisers include the UAV and sensor dynamical motion models (Delle Fave et al., 2010;Lanillos et al., 2014;Perez-Carabaza et al., 2017, 2016Tisdale et al., 2009;Wong et al., 2005;Yao et al., 2019) or non-ideal/non-constant likelihood functions to model the observations uncertainty within the sensor footprint (Delle Fave et al., 2010;Kratzke et al., 2010;Lanillos et al., 2014;Perez-Carabaza et al., 2017, 2016Riehl et al., 2011;Wong et al., 2005). These additional functions and models bring realism to the optimisers, usually at the expenses of hardening the evaluation and optimisation of the UAVs and sensor trajectories. ...
... In this regard, some works take into account the existence of non-flying zones, UAV collisions, communication requirements, the UAV fuel consumption, the length or smoothness of the UAV trajectories, and/ or the area coverage (Carpin et al., 2013;Li et al., 2021;Perez-Carabaza et al., 2017, 2016Pérez-Carabaza, Scherer et al., 2019;Yang et al., 2002). Alternatively, and in order to shorten the gap between the solutions (UAVs and sensor trajectories) proposed by the optimiser and its application to real-world search and rescue missions, a few optimisers include the UAV and sensor dynamical motion models (Delle Fave et al., 2010;Lanillos et al., 2014;Perez-Carabaza et al., 2017, 2016Tisdale et al., 2009;Wong et al., 2005;Yao et al., 2019) or non-ideal/non-constant likelihood functions to model the observations uncertainty within the sensor footprint (Delle Fave et al., 2010;Kratzke et al., 2010;Lanillos et al., 2014;Perez-Carabaza et al., 2017, 2016Riehl et al., 2011;Wong et al., 2005). These additional functions and models bring realism to the optimisers, usually at the expenses of hardening the evaluation and optimisation of the UAVs and sensor trajectories. ...
This paper presents a new Cloud-deployable DEVS-based framework for optimising UAV trajectories and sensor strategies in target-search missions. DEVS provides it with a well-established, flexible, and verifiable modelling strategy to include different models for the UAV, sensor, and target dynamics; the target and sensor uncertainty; and the optimising process. Its Cloud deployability speeds up the evaluations/simulations required to optimise this NP-hard problem, which involves computationally heavy models when solving real-world missions. The framework, designed to handle different types of target-search missions, currently optimises, using a multi-objective Genetic Algorithm, free-shape trajectories of multiple UAVs,eqquiped with several static/movable sensors to detect a target within a search area. It is implemented in xDEVS and deployable over a set of containers in the Google Cloud Platform. The results show that our deployment policy speeds up the computation up to 3.35 times, letting the operator simultaneously optimise several search strategies for agiven scenario.
... With the rapid development of affordable robots with embedded sensing and computation capabilities, we are quickly approaching a point at which reallife applications will involve the deployment of hundreds, if not thousands, of robots [1,2]. Among these applications, significant research effort has been devoted to multi-agent search [3,4,5,6,7], where deploying numerous agents can greatly improve the time-efficiency and robustness of search. In fact, deploying robots with various motion or sensing modalities can further improve the search performance, by leveraging the natural synergies between these capabilities (see Fig. 1). ...
... The approach in this paper is based on ergodic search processes, which, similar to other information-theoretic coverage methods [3,4,8,11,12], rely on an a priori information distribution, representing the likelihood of finding a target at any point over the search domain, to guide the search. In practice, this information distribution can be obtained from scouting missions or from expert knowledge, and is updated during search if inaccurate. ...
... For example, in gradient-based, or "information surfing", methods [4,11,12], agents guide their movement in the direction of the derivative of the information map around their positions to greedily maximize the short-term information gain. That is, agents are always driven in the direction of the greatest information gain, which naturally leads them to areas where the likelihood of finding a target is maximized. ...
This paper develops a multi-agent heterogeneous search approach that leverages the sensing and motion capabilities of different agents to improve search performance (i.e., decrease search time and increase coverage efficiency). To do so, we build upon recent results in ergodic coverage methods for homogeneous teams, where the search paths of the agents are optimized so they spend time in regions proportionate to the expected likelihood of finding targets, while still covering the whole domain, thus balancing exploration and exploitation. This paper introduces a new method to extend ergodic coverage to teams of heterogeneous agents with varied sensing and motion capabilities. Specifically, we investigate methods of leveraging the spectral decomposition of a target information distribution to efficiently assign available agents to different regions of the domain and best match the agents’ capabilities to the scale at which information needs to be searched for in these regions. Our numerical results show that distributing and assigning coverage responsibilities to agents based on their dynamic sensing capabilities leads to approximately improvement with regard to a standard coverage metric (ergodicity) and a improvement in time to search over a baseline approach that jointly plans search paths for all agents, averaged over 500 randomized experiments.
... An additional constraint in search and rescue scenarios can be considered of locating the target as fast as possible. This type of problem is well studied in literature and is termed as a minimum time search problem (MTS) [2][3][4][5][6][7][8][9][10][11][12][13][14]. The most prominent objective in these approaches is to optimize the expected time of target detection [3][4][5][6]; however, other alternative approaches involve optimizing the probability of target detection [7][8][9]15], minimizing its counterpart, i.e., probability of nondetection [10,11] or maximizing the information gain [12,13,16]. ...
... This type of problem is well studied in literature and is termed as a minimum time search problem (MTS) [2][3][4][5][6][7][8][9][10][11][12][13][14]. The most prominent objective in these approaches is to optimize the expected time of target detection [3][4][5][6]; however, other alternative approaches involve optimizing the probability of target detection [7][8][9]15], minimizing its counterpart, i.e., probability of nondetection [10,11] or maximizing the information gain [12,13,16]. Various sub-optimal and heuristics-based algorithms such as gradient-based approaches [7,[10][11][12]15], cross-entropy optimization [2,5], Bayesian optimization algorithms [4], ant colony optimization [6], or genetic algorithms [3] have been proposed to address the NP-hard complex problem [13]. ...
... The most prominent objective in these approaches is to optimize the expected time of target detection [3][4][5][6]; however, other alternative approaches involve optimizing the probability of target detection [7][8][9]15], minimizing its counterpart, i.e., probability of nondetection [10,11] or maximizing the information gain [12,13,16]. Various sub-optimal and heuristics-based algorithms such as gradient-based approaches [7,[10][11][12]15], cross-entropy optimization [2,5], Bayesian optimization algorithms [4], ant colony optimization [6], or genetic algorithms [3] have been proposed to address the NP-hard complex problem [13]. These approaches can also be differentiated based on the considered UAV dynamics models, where they either do not consider velocity at all [2,[4][5][6][7][8][9]15], or only consider simple linear velocity models [3,10,11] but not acceleration or deceleration. ...
In this article we demonstrate that acceleration and deceleration of direction-turning drones at waypoints have a significant influence to path planning which is important to be considered for time-critical applications, such as drone-supported search and rescue. We present a new path planning approach that takes acceleration and deceleration into account. It follows a local gradient ascend strategy which locally minimizes turns while maximizing search probability accumulation. Our approach outperforms classic coverage-based path planning algorithms, such as spiral- and grid-search, as well as potential field methods that consider search probability distributions. We apply this method in the context of autonomous search and rescue drones and in combination with a novel synthetic aperture imaging technique, called Airborne Optical Sectioning (AOS), which removes occlusion of vegetation and forest in real-time.
... Moreover, the UAVs and sensor trajectories can be obtained by optimising probabilistic utility functions that simultaneously handle the uncertainty in the target (location and movements) and sensor (observations). Examples of these functions and of works that present strategies to optimise them, are the entropy (Yang et al., 2002), the information gain (Carpin et al., 2013;Grocholsky et al., 2006;Hu et al., 2014), the probability of detection (Delle Fave et al., 2010;Fedorov, 2019;Kratzke et al., 2010;Lanillos et al., 2014;Li et al., 2021;Saadaoui et al., 2018;Tisdale et al., 2009;Wang et al., 2017;Wong et al., 2005;Yao et al., 2017Yao et al., , 2019 and the expected time of detection (Lanillos et al., 2013;Perez-Carabaza et al., 2017, 2016Pérez-Carabaza, Scherer et al., 2019;Riehl et al., 2011). The last two utility functions are especially interesting in missions where: a) it is possible to know and exploit a probability distribution of the initial target location and b) critical to detect the target as soon as possible. ...
... In this regard, some works take into account the existence of non-flying zones, UAV collisions, communication requirements, the UAV fuel consumption, the length or smoothness of the UAV trajectories, and/ or the area coverage (Carpin et al., 2013;Li et al., 2021;Perez-Carabaza et al., 2017, 2016Pérez-Carabaza, Scherer et al., 2019;Yang et al., 2002). Alternatively, and in order to shorten the gap between the solutions (UAVs and sensor trajectories) proposed by the optimiser and its application to real-world search and rescue missions, a few optimisers include the UAV and sensor dynamical motion models (Delle Fave et al., 2010;Lanillos et al., 2014;Perez-Carabaza et al., 2017, 2016Tisdale et al., 2009;Wong et al., 2005;Yao et al., 2019) or non-ideal/non-constant likelihood functions to model the observations uncertainty within the sensor footprint (Delle Fave et al., 2010;Kratzke et al., 2010;Lanillos et al., 2014;Perez-Carabaza et al., 2017, 2016Riehl et al., 2011;Wong et al., 2005). These additional functions and models bring realism to the optimisers, usually at the expenses of hardening the evaluation and optimisation of the UAVs and sensor trajectories. ...
... In this regard, some works take into account the existence of non-flying zones, UAV collisions, communication requirements, the UAV fuel consumption, the length or smoothness of the UAV trajectories, and/ or the area coverage (Carpin et al., 2013;Li et al., 2021;Perez-Carabaza et al., 2017, 2016Pérez-Carabaza, Scherer et al., 2019;Yang et al., 2002). Alternatively, and in order to shorten the gap between the solutions (UAVs and sensor trajectories) proposed by the optimiser and its application to real-world search and rescue missions, a few optimisers include the UAV and sensor dynamical motion models (Delle Fave et al., 2010;Lanillos et al., 2014;Perez-Carabaza et al., 2017, 2016Tisdale et al., 2009;Wong et al., 2005;Yao et al., 2019) or non-ideal/non-constant likelihood functions to model the observations uncertainty within the sensor footprint (Delle Fave et al., 2010;Kratzke et al., 2010;Lanillos et al., 2014;Perez-Carabaza et al., 2017, 2016Riehl et al., 2011;Wong et al., 2005). These additional functions and models bring realism to the optimisers, usually at the expenses of hardening the evaluation and optimisation of the UAVs and sensor trajectories. ...
... This behavior is modeled through likelihood functions, which are used by the RBFs and PFs to update the probability distributions. Their expressions depend on the type of sensors and on the realism of the planner, ranging from ideal/constant sensor models that only observe a few cells of the probability map under the UAV location [8,25,26,33,41,45,47], to range-based sensor models with probability curves that decrease with the distance between the sensor and target location [23,24,44], or to specific models of certain types of radars [30,31] and cameras [7,32,35]. ...
... heading, speed and height) and exploits their dynamical models to obtain free-shape trajectories [7, 24, 30-32, 41, 44, 47]. Within this group, note that [7,24,41,44,47] use a streamlined differential model, while [30][31][32] use a more complex one that includes the height, speed and lateral dynamics of the UAV. ...
... These are other variables to consider, as they usually increment the computation requirements and complexity of the evaluation process. In particular, the evaluation process of the works under analysis varies from single-target [1, 7, 8, 25, 30-33, 35, 41, 45, 47] to multi-target [24,26,44], and from single-UAV [1,8,45] to multi-UAV [7, 24-26, 31, 32, 35, 41, 44, 47]. Additionally, all of them only consider a single target-detection sensor within each UAV. ...
... Another metric of effectiveness is to record the maximum value of the refresh time of any cell at each time step, with specific subsections of interest having their own maximum refresh time, which can be plotted to understand the oscillatory nature of persistent UAV search [27]. When the search is probabilistic, the metric of information gained or the probability of detection [28] can be considered, as well as a measure called awareness that is related to information entropy [29]. There are also many domain-specific related measures of effectiveness such as the size of burnt land for a forest fighting mission [4], the number of targets tracked over time for a search and track task [26], and the average delay when a stochastically appearing target appears and when it is observed in a mobile sensing task [19]. ...
... Li et al. utilized a summary heat map to show which grid cells were visited more frequently over the course of a scenario dependent on the search pattern [18]. Lanillos et al. generated 3D terrain charts representing detection probability to compare how different search strategies affected the detection probability [28]. These methods work well for comparing effectiveness spatially when only varying search methods. ...
This research introduces novel analytical methods for evaluating multi-UAV persistent search and retrieval with stochastic target appearance (PSR-STA) scenarios. Traditional approaches that rely on single aggregate effectiveness measures for a scenario fail to capture the complex spatiotemporal dynamics of multi-UAV operations and provide limited insights into improving search performance. To address these limitations, we present a comprehensive analysis framework combining temporal and spatial analysis techniques. For temporal analysis, we employ a graphical comparison of line charts and discrete Fourier transform analysis to identify shared temporal patterns across scenarios. Spatial patterns are analyzed through principal components analysis and random forest surrogate modeling with profiling to understand non-linear parameter influences. Additionally, we introduce trellis charts for integrated visualization and analysis of combined spatiotemporal patterns. This research builds on a case study developed in a previous case study of multi-UAV PSR-STA. While the previous work established foundational algorithms and metrics for multi-UAV PSR-STA, this study introduces sophisticated spatiotemporal analysis techniques that reveal deep insights into system behavior and enable a nuanced understanding of UAV search performance across varied scenarios.
... [12] noted in August 2017, right after Hurricane Harvey, that subsequent to power restoration, "delivering medical care to individuals in compromised living quarterswhether in a flooded home or in a shelterpresents unique challenges to first responders, as well as doctors and nurses who serve on the front lines". Right after Hurricane Irma, in September 2017, Aircraft Owners and Pilots Association (AOPA) volunteered to deliver medicine with piloted aircrafts to flooded areas in Florida and Virgin Islands due to Hurricane Irma [13]. Such observations motivate the development of a drone-based emergency medicine delivery systems, which this paper proposes. ...
... Note that constraint sets (13) and (14), as well as (15) and (16), (17) and (18), and (19) and (20), refer to the two days separately. Constraints (21)-(23) account for the relocations, where parameter Q < p specifies the maximum number of platform relocations that are allowed in this scenario. ...
Motivated by issues dealing with delivery of emergency medical products during humanitarian disasters, this paper addresses the general problem of delivering perishable items to remote demands accessible only by helicopters or drones. Each drone operates out of platforms that may be move when not in use and each drone has a limited delivery range to service a demand points. Associated with each demand point is a disutility function, or a cost function, with respect to time that reflects preferred delivery clock time for the demanded item, as well as the item's perishability characteristic that models nonincreasing quality with time. The paper first addresses the problem of locating the platforms as well concurrently determining which platform serves which demand points and in what order to minimize total disutility for product delivery. The second scenario addresses the two-period problem where the platforms can be relocated, using useable road networks, after the first period. It can be easily proven that continuous time versions of these problems are NP-Hard. However, a practical "time-slot" version of the problem, where time is discretized into slots, can be solved by standard optimization software. Extensive computational experiments, using different drone delivery ranges as well as different drone fleet sizes, provide valuable insights on the performance of such drone delivery systems.
... It tackles situations where the target does not react to the searchers' efforts and follows a predictable behavior. While there are many studies in the non-evading target literature that focus on online computation of the searchers' positions (e.g., Alfeo et al. (2019), Lanillos et al. (2014), Senanayake et al. (2016), Zheng et al. (2019a) to name just a few), such works are not within the scope of this related work section. This is because they differ significantly from our current work, since they do not involve search resource allocation or evading targets. ...
... But the search path for UAVs need to be redesigned once environment changes. The second type is rooted in the optimization theory [8,9]. The goal of these methods is to optimize target searching and tracking, aiming for a maximum coverage rate or maximum observation rate. ...
In this paper, we propose a distributed multi-agent reinforcement learning (MARL) method to learn cooperative searching and tracking policies for multiple unmanned aerial vehicles (UAVs) with limited sensing range and communication ability. Firstly, we describe the system model for multi-UAV cooperative searching and tracking for moving targets and consider average observation rate and average exploration rate as the metrics. Moreover, we propose the information update and fusion mechanisms to enhance environment perception ability of the multi-UAV system. Then, the details of our method are demonstrated, including observation and action space representation, reward function design and training framework based on multi-agent proximal policy optimization (MAPPO). The simulation results have shown that our method has well convergence performance and outperforms other baseline algorithms in terms of average observation rate and average exploration rate.
... This can be formulated as an optimization problem that aims to minimize or maximize the team objective function while taking into account a set of equality and/or inequality constraints. To date, various methods have been explored to improve agent path planning, including Dynamic Programming (DP) [2,10], Neural Networks (NNs) [11], and Reinforcement Learning (RL) [12][13][14], gradientbased optimization [15,16], Artificial Potential Field (APF) [17,18], Meta-heuristic optimization algorithms [19][20][21], and Model Predictive Control (MPC) [22,23] which is also known as Receding Horizon Control (RHC). For instance, a distributed gradient-based optimization approach that considers overloading constraints and collision avoidance has been proposed for path planning in multi-agent cooperative search scenarios [24]. ...
... Multi-agent path planning optimization is the key to achieving the team objective function while satisfying various constraints. Several methods have been explored to improve agent path planning, including Dynamic Programming (DP) [2,10], Neural Networks (NNs) [11], Reinforcement Learning (RL) [12][13][14], gradient-based optimization [15][16][17], Artificial Potential Field (APF) [18,19], Meta-heuristic optimization algorithms [20][21][22], and Model Predictive Control (MPC) [23,24]. Each of these methods has its advantages and disadvantages, depending on the specific application scenario. ...
... The most common objectives of such operations include the maximization of the cumulative finding probability under a time constraint or the minimization of the time to find a target under a given probability. While the humanitarian drone search operations share similarities with traditional search operations [43], additional limitations have to be taken into consideration, such as the limited battery capacity, which sheds light on the need for recharging and leads to more frequent searching close to the charging station [91]; the limitations of drone sensors, which differ from the ones used by UGVs in traditional search operations [92]; and, last, the communication limitations that could arise in UAV-to-UAV, UAV-to-base station (BS), or UAV-to-UGV communication [93,94]. Search operations, by default, utilize the drone's surveying and monitoring capabilities. ...
The adoption of drones and other emerging digital technologies (DTs) has proven essential in revolutionizing humanitarian logistics as standalone solutions. However, the interoperability of humanitarian drones with other DTs has not yet been explored. In this study, we performed a systematic literature review to attempt to fill this gap by evaluating 101 mathematical models collected from Scopus. After conducting a descriptive analysis to identify the trends of publications in terms of year, type, source, and country of origin, a content analysis ensued to investigate the complementarity, interoperability, and level of integration of humanitarian drones with eight DTs. Next, we researched how these DTs can help drones exploit their capabilities to their full potential and facilitate the various drone operations deployed across different disaster scenarios, types, and stages. Last, the solving approaches employed by the models were examined. Overall, we shifted our research focus toward several overlooked aspects in the literature and identified multiple challenges needing to be addressed. Our work resulted in the formulation of a holistic framework aiming to standardize the cooperative utilization of DTs during the execution of humanitarian drone operations, so as to enhance their real-life application and scalability.
... The overall desired velocity is expressed in the x, y axes as v i,x and v i,y for each agent i, and it is computed from Equations (20) and (21). ...
A decentralized swarm of quadcopters designed for monitoring an open area and detecting intruders is proposed. The system is designed to be scalable and robust. The most important aspect of the system is the swarm intelligent decision-making process that was developed. The rest of the algorithms essential for the system to be completed are also described. The designed algorithms were developed using ROS and tested with SITL simulations in the GAZEBO environment. The proposed approach was tested against two other similar surveilling swarms and one approach using static cameras. The addition of the real-time decision-making capability offers the swarm a clear advantage over similar systems, as depicted in the simulation results.
... Likewise, the swarm robotics has drawn the attention of the operations research community as an attractive opportunity to improve the efficiency of swarm-powered missions (Otto et al. 2018;Coutinho et al. 2018). For instance, decentralized optimization methods have been leveraging search problems (Sujit and Ghose 2004;Oh et al. 2014Oh et al. , 2015Lanillos et al. 2014;Gan et al. 2011;Ji et al. 2015), target assignment problems (Tang and Ozguner 2005;Karaman et al. 2008;Niccolini et al. 2010;Viguria et al. 2010;Choi et al. 2011;Barrientos et al. 2011;Moon et al. 2013Moon et al. , 2015Turpin et al. 2014;Enright et al. 2015;Sadeghi and Smith 2017), node covering problems (Zorbas et al. 2016;Ladosz et al. 2018), scheduling problems (Caraballo et al. 2017), etc. ...
Drones have been getting more and more popular in many economy sectors. Both scientific and industrial communities aim at making the impact of drones even more disruptive by empowering collaborative autonomous behaviors—also known as swarming behaviors—within fleets of multiple drones. In swarming-powered 3D mapping missions, unmanned aerial vehicles typically collect the aerial pictures of the target area whereas the 3D reconstruction process is performed in a centralized manner. However, such approaches do not leverage computational and storage resources from the swarm members. We address the optimization of a swarm-powered distributed 3D mapping mission for a real-life humanitarian emergency response application through the exploitation of a swarm-powered ad hoc cloud. Producing the relevant 3D maps in a timely manner, even when the cloud connectivity is not available, is crucial to increase the chances of success of the operation. In this work, we present a mathematical programming heuristic based on decomposition and a variable neighborhood search heuristic to minimize the completion time of the 3D reconstruction process necessary in such missions. Our computational results reveal that the proposed heuristics either quickly reach optimality or improve the best known solutions for almost all tested realistic instances comprising up to 1000 images and fifteen drones.
... According to previous studies on various types of sensors, the ability of sensors to collect feature information decays with increasing distance [44,45]. Therefore, the HAUV's ability to collect information about the surrounding workspace at point ρ j can be expressed in the following form: ...
This paper presents a novel Rapidly-exploring Adaptive Sampling Tree (RAST) algorithm for the adaptive sampling mission of a hybrid aerial underwater vehicle (HAUV) in an air-sea 3D environment. This algorithm innovatively combines the tournament-based point selection sampling strategy, the information heuristic search process and the framework of Rapidly-exploring Random Tree (RRT) algorithm. Hence can guide the vehicle to the region of interest to scientists for sampling and generate a collision-free path for maximizing information collection by the HAUV under the constraints of environmental effects of currents or wind and limited budget. The simulation results show that the fast search adaptive sampling tree algorithm has higher optimization performance, faster solution speed and better stability than the Rapidly-exploring Information Gathering Tree (RIGT) algorithm and the particle swarm optimization (PSO) algorithm.
... Searching for targets when there is incomplete information about their location is a generalization of the MTS problem, where one or more targets are in unknown locations and need to be found as soon as possible. There are many approaches for handling the MTS problem under the assumption of a static target [17,8,12,24]; however, we are interested in moving targets. ...
This paper considers the complex problem of a team of UAVs searching targets under uncertainty. The goal of the UAV team is to find all of the moving targets as quickly as possible before they arrive at their selected goal. The uncertainty considered is threefold: First, the UAVs do not know the targets' locations and destinations. Second, the sensing capabilities of the UAVs are not perfect. Third, the targets' movement model is unknown. We suggest a real-time algorithmic framework for the UAVs, combining entropy and stochastic-temporal belief, that aims at optimizing the probability of a quick and successful detection of all of the targets. We have empirically evaluated the algorithmic framework, and have shown its efficiency and significant performance improvement compared to other solutions. Furthermore, we have evaluated our framework using Peer Designed Agents (PDAs), which are computer agents that simulate targets and show that our algorithmic framework outperforms other solutions in this scenario.
... In turn, the functionality of an information fusion strategy greatly depends on robots networking capabilities. Data aggregation can be achieved through different communication protocols, namely, centralized [37], [40], [68], decentralized [24], [71], [194], or distributed [36], [195]- [197]. ...
Active Position Estimation (APE) is the task of localizing one or more targets using one or more sensing platforms. APE is a key task for search and rescue missions, wildlife monitoring, source term estimation, and collaborative mobile robotics. Success in APE depends on the level of cooperation of the sensing platforms, their number, their degrees of freedom and the quality of the information gathered. APE control laws enable active sensing by satisfying either pure-exploitative or pure-explorative criteria. The former minimizes the uncertainty on position estimation; whereas the latter drives the platform closer to its task completion. In this paper, we define the main elements of APE to systematically classify and critically discuss the state of the art in this domain. We also propose a reference framework as a formalism to classify APE-related solutions. Overall, this survey explores the principal challenges and envisages the main research directions in the field of autonomous perception systems for localization tasks. It is also beneficial to promote the development of robust active sensing methods for search and tracking applications.
... When the UAV visits the cell, its time is reset to zero [47]. If the detection model is probabilistic, the cell value can be a measure related to the probability of a target existing in the cell instead of the time last visited [37]. ...
Groups of battery powered unmanned aerial vehicles (UAVs) are effective in a variety of scenarios that require autonomous cooperation to achieve a goal. However, the complexity of modeling and analyzing UAV cooperation in situations with stochastic elements leads to unique challenges. This paper introduces a framework for one such problem domain, the multi-UAV persistent search and retrieval task with stochastic target appearances (PSR-STA), in which UAVs continuously search an area for stochastically appearing targets to retrieve and deliver them to collector locations. Design decisions are introduced for understanding how to successfully simulate multi-UAV PSR-STA. Common tools for analyzing search algorithm effectiveness through statistical and graphical methods are presented. A case study of multi-UAV park cleanup is implemented to demonstrate the framework, where algorithms for choosing the locations of collectors and charging stations based on stochastic target appearance models are proposed, methods for continuous multi-UAV operation over a long period time are demonstrated, and the differences in effectiveness between four coverage search patterns are analyzed.
... Although target detection algorithms based on deep learning can improve performance by increasing the types and numbers of small target samples in the training set, the adaptability and accuracy of many algorithms are difficult to guarantee when dealing with small target detection. In vision tasks such as target searching [3] and target positioning [4], traditional vision systems have significant limitations in the efficiency and accuracy of image processing . However, the thorny issues are simple for the creatures [5]. ...
Small target detection is known to be a challenging problem. Inspired by the structural characteristics and physiological mechanism of eagle-eye, a miniature vision system is designed for small target detection in this paper. First, a hardware platform is established, which consists of a pan-tilt, a short-focus camera and a long-focus camera. Then, based on the visual attention mechanism of eagle-eye, the cameras with different focal lengths are controlled cooperatively to achieve small target detection. Experimental results show that the designed biological eagle-eye vision system can accurately detect small targets, which has a strong adaptive ability.
... Sin embargo, al igual que otros métodos basados en optimización, el coste computacional crece rápidamente con el número de UAVs, amplitud de los pasos temporales (longitud de la trayectoria planificada) y tamaño delárea. Un trabajo similar puede encontrarse en Lanillos et al. (2014), también basado en ventanas temporales deslizantes y optimización descentralizada. ...
span class="fontstyle0">A día de hoy, existen en el mercado una gran cantidad de aeronaves sin piloto que pueden ser comandadas con ordenes de alto nivel para realizar tareas complejas de forma casi automatica, como por ejemplo el mapeo de explotaciones agrícolas. De forma natural, nos podemos preguntar si sería posible coordinar a un grupo de estos robots para realizar esas mismas tareas de forma más rápida, flexible y robusta. En este trabajo se repasan las tareas que se han planteado resolver con sistemas compuestos por grupos de aeronaves no tripuladas y los algoritmos empleados, así como los metodos y estrategias en los que están basados. Aunque el futuro de estos sistemas es prometedor, existen ciertos obstaculos legislativos y técnicos que frenan su implantación de forma generalizada.</span
... Many targeted heuristic algorithms have been applied to solve multi-target optimization problems in recent years [19][20][21], but there is currently no general cooperative task assignment model that can simultaneously meet complex battlefield environments with coupled task relationships and strict requirements for the task timing sequence. Although existing algorithms cannot guarantee optimal results, they do provide satisfactory solutions within an acceptable time period [22,23]. ...
The multi-unmanned aerial vehicle (UAV) must autonomously perform reconnaissance-attack-evaluation tasks under multiple constraints in the battlefield environment. This paper proposes a nearest neighbor method designed with the shortest neighboring distance as an indicator which quickly solves the optimal sequence of multiple tasks for cooperative execution. Each target to be destroyed requires a different quantity of ammunition; a cooperative task assignment model for heterogeneous UAVs is established accordingly. Based on the nearest neighbor method, and with reference to fruit-picking techniques currently in use, a novel “orchard picking algorithm (OPA)” is investigated as well. This algorithm proposed in this paper is a heuristic algorithm, which has a broad application prospect in complex task assignment. A cooperative attack task assignment is simulated to test the performance of the algorithm. In essence, it balances the assignment of tasks, works within a brief execution time, and exhibits high flexibility, strong robustness, and scalability.
... Likewise, the swarm robotics has drawn the attention of the operations research community as an attractive opportunity to improve the efficiency of swarm-powered missions [23,24]. For instance, decentralized optimization methods have been leveraging search problems [25,26,27,28,29,30], target assignment problems [31,32,33,34,35,36,37,38,39,40,41], node covering problems [42,43], scheduling problems [44], etc. ...
Drones have been getting more and more popular in many economy sectors. Both scientific and industrial communities aim at making the impact of drones even more disruptive by empowering collaborative autonomous behaviors -- also known as swarming behaviors -- within fleets of multiple drones. In swarming-powered 3D mapping missions, unmanned aerial vehicles typically collect the aerial pictures of the target area whereas the 3D reconstruction process is performed in a centralized manner. However, such approaches do not leverage computational and storage resources from the swarm members.We address the optimization of a swarm-powered distributed 3D mapping mission for a real-life humanitarian emergency response application through the exploitation of a swarm-powered ad hoc cloud. Producing the relevant 3D maps in a timely manner, even when the cloud connectivity is not available, is crucial to increase the chances of success of the operation. In this work, we present a mathematical programming heuristic based on decomposition and a variable neighborhood search heuristic to minimize the completion time of the 3D reconstruction process necessary in such missions. Our computational results reveal that the proposed heuristics either quickly reach optimality or improve the best known solutions for almost all tested realistic instances comprising up to 1000 images and fifteen drones.
This paper presents a multiagent cooperative search algorithm for identifying an unknown number of targets. The objective is to determine a collection of observation points and corresponding safe paths for agents, which involves balancing the detection time and the number of targets searched. A Bayesian framework is used to update the local probability density function of the targets when the agents obtain information. We utilize model predictive control and establish utility functions based on the detection probability and decrease in information entropy. A target detection algorithm is implemented to verify the target based on minimum‐risk Bayesian decision‐making. Then, we improve the search algorithm with the target detection algorithm. Several simulations demonstrate that compared with other existing approaches, the proposed approach can reduce the time needed to detect targets and the number of targets searched. We establish an experimental platform with three unmanned aerial vehicles. The simulation and experimental results verify the satisfactory performance of our algorithm.
This paper presents a novel, sparse sensing motion planning algorithm for autonomous mobile robots in resource limited coverage problems. Optimizing usage of limited resources while effectively exploring an area is vital in scenarios where sensing is expensive, has adverse effects, or is exhaustive. We approach this problem using ergodic search techniques, which optimize how long a robot spends in a region based on the likelihood of obtaining informative measurements which guarantee coverage of a space. We recast the ergodic search problem to take into account when to take sensing measurements. This amounts to a mixed-integer program that optimizes when and where a sensor measurement should be taken while optimizing the agent’s paths for coverage. Using a continuous relaxation, we show that our formulation performs comparably to dense sampling methods, collecting information-rich measurements while adhering to limited sensing measurements. Multi-agent examples demonstrate the capability of our approach to automatically distribute sensor resources across the team. Further comparisons show comparable performance with the continuous relaxation of the mixed-integer program while reducing computational resources.
Aiming at the problem of using fixed-wing unmanned aerial vehicles (UAVs) (clusters) for rapid coverage search of submarines in large-scale sea areas, the strategy of using a single UAV for coverage searching in form of Dubins curve (SCSDC) and the strategy of using fixed-wing UAVs cluster for cooperative coverage searching in form of parallel intervals (SCSPI) were proposed after selective removal of concave points in complex irregular concave polygonal areas. First, the cost mathematical model of cooperative search for multiple UAVs was established. At the same time, considering the limitation of turning radius of UAVs, a Dubins curve coverage search strategy was proposed to avoid search blind areas. Second, the judgment conditions of “edge length threshold” and “angle threshold” were introduced, and the method of selective removal of concave points was proposed to transform the irregular concave polygon area into a regular convex polygonal area. Then, to reduce the searching cost of UAV’s “
” turning type and improve the timeliness of its submarine search, a strategy of UAV cluster cooperative coverage search in the form of parallel interval was proposed. Finally, a comparative verification experiment was designed for the strategy of SCSDC and SCSPI in a complex irregular polygon search area. The simulation results show that selection of the two different area coverage search strategies needs to be determined according to actual combat requirements. The experimental results well verify the effectiveness and application scope of the two area coverage search strategies, and provide information support for subsequent antisubmarine action.
This article presents a novel rapidly-exploring adaptive sampling tree algorithm for adaptive sampling missions using a hybrid aerial underwater vehicle (HAUV) in an air–sea 3-D environment. This algorithm innovatively combines the tournament-based point selection sampling strategy, the information heuristic search process, and the framework of the rapidly-exploring random tree algorithm. Hence, the vehicle can be guided to a region of interest to scientists for sampling and generate a collision-free path for maximizing information collection by the HAUV under the constraints of environmental effects of currents or wind and a limited budget. The simulation results show that the fast search adaptive sampling tree algorithm has higher optimization performance, faster solution speed, and better stability than the rapidly-exploring information gathering tree algorithm and the particle swarm optimization algorithm.
This paper considers the cooperative search for stationary targets by multiple unmanned aerial vehicles (UAVs) with limited sensing range and communication ability in a dynamic threatening environment. The main purpose is to use multiple UAVs to find more unknown targets as soon as possible, increase the coverage rate of the mission area, and more importantly, guide UAVs away from threats. However, traditional search methods are mostly unscalable and perform poorly in dynamic environments. A new multi-agent deep reinforcement learning (MADRL) method, DNQMIX, is proposed in this study to solve the multi-UAV cooperative target search (MCTS) problem. The reward function is also newly designed for the MCTS problem to guide UAVs to explore and exploit the environment information more efficiently. Moreover, this paper proposes a digital twin (DT) driven training framework “centralized training, decentralized execution, and continuous evolution” (CTDECE). It can facilitate the continuous evolution of MADRL models and solve the tradeoff between training speed and environment fidelity when MADRL is applied to real-world multi-UAV systems. Simulation results show that DNQMIX outperforms state-of-art methods in terms of search rate and coverage rate.
Computational models of visual attention in artificial intelligence and robotics have been inspired by the concept of a saliency map. These models account for the mutual information between the (current) visual information and its estimated causes. However, they fail to consider the circular causality between perception and action. In other words, they do not consider where to sample next, given current beliefs. Here, we reclaim salience as an active inference process that relies on two basic principles: uncertainty minimization and rhythmic scheduling. For this, we make a distinction between attention and salience. Briefly, we associate attention with precision control, i.e., the confidence with which beliefs can be updated given sampled sensory data, and salience with uncertainty minimization that underwrites the selection of future sensory data. Using this, we propose a new account of attention based on rhythmic precision-modulation and discuss its potential in robotics, providing numerical experiments that showcase its advantages for state and noise estimation, system identification and action selection for informative path planning.
Bacterial chemotaxis in unicellular Escherichia coli, the simplest biological creature, enables it to perform effective searching behaviour even with a single sensor, achieved via a sequence of “tumbling” and “swimming” behaviours guided by gradient information. Recent studies show that suitable random walk strategies may guide the behaviour in the absence of gradient information. This article presents a novel and minimalistic biologically inspired search strategy inspired by bacterial chemotaxis and embodied intelligence concept: a concept stating that intelligent behaviour is a result of the interaction among the “brain,” body morphology including the sensory sensitivity tuned by the morphology, and the environment. Specifically, we present bacterial chemotaxis inspired searching behaviour with and without gradient information based on biological fluctuation framework: a mathematical framework that explains how biological creatures utilize noises in their behaviour. Via extensive simulation of a single sensor mobile robot that searches for a moving target, we will demonstrate how the effectiveness of the search depends on the sensory sensitivity and the inherent random walk strategies produced by the brain of the robot, comprising Ballistic, Levy, Brownian, and Stationary search. The result demonstrates the importance of embodied intelligence even in a behaviour inspired by the simplest creature.
In this research, a cooperative search for multiple dynamic targets in an unknown marine environment by multiple unmanned aerial vehicles is studied based on a novel multi‐bee‐colony (MBC) elite learning algorithm. First, a specialized searching model is established which includes the UAV dynamics, the sensor model, the target probability and environmental certainty at different flight altitudes. Then, a new search strategy, which consists of rough search and accurate search, is proposed by maximizing the multiobjective utility function with the dynamic changing of the flight altitude. In order to solve the optimization problem, an improved MBC algorithm based on elite learning is designed, which can improve the adaptability and computation speed of the standard artificial bee colony (ABC) algorithm under different search missions. Finally, extensive simulations are conducted to show the effectiveness and superiority of the proposed search strategy.
Small target detection is known to be a challenging problem. Inspired by the structural characteristics and physiological mechanism of eagle-eye, a miniature vision system is designed for small target detection in this paper. First, a hardware platform is established, which consists of a pan-tilt, a short-focus camera and a long-focus camera. Then, based on the visual attention mechanism of eagle-eye, the cameras with different focal lengths are controlled cooperatively to achieve small target detection. Experimental results show that the designed biological eagle-eye vision system can accurately detect small targets, which has a strong adaptive ability.
With the rapid development of artificial intelligence technology, the multi-UAV cooperative search has wide applications in the field of Internet of Things, such as resource exploration, emergency rescue, intelligent transportation, etc. However, the communication network in an unknown environment may be inaccessible, and the real-time information sharing among UAVs cannot be guaranteed, resulting in the failure of cooperative search. Aiming at this issue, this article is devoted to the design of the multi-UAV flight strategy to improve the cooperative search capability in an uncertain communication environment. Specifically, a new cooperative architecture oriented to a local communication network is devised to control the observation locations of multiple UAVs in the search process, and some local communication networks are established based on the distance among UAVs to meet the requirements of the search task. On this foundation, we develop a multi-UAV cooperative search model (MCSM) with communication cost and formation benefit as an optimization function to ensure the effectiveness of multi-UAV search. Moreover, in the process of model solving, an improved sparrow search algorithm (ISSA) is presented with some different search strategies to enhance the optimization capability. To verify the superiority of the proposed method, we designed several groups of simulation experiments to analyze the performance of MCSM. Experimental results illustrate that our method can not only maintain high cooperative search accuracy but also has high stability and convergence speed.
Motivated by issues dealing with delivery of emergency medical products during humanitarian disasters, this paper addresses the general problem of delivering perishable items to remote demands accessible only by helicopters or drones. Each drone operates out of platforms that may be moved when not in use and each drone has a limited delivery range to service a demand point. Associated with each demand point is a disutility function, or a cost function, with respect to time that reflects preferred delivery clock time for the demanded item, as well as the item’s perishability characteristic that models nonincreasing quality with time. The paper first addresses the problem of locating the platforms as well concurrently determining which platform serves which demand points and in what order – to minimize total disutility for product delivery. The second scenario addresses the two-period problem where the platforms can be relocated, using useable road network, after the first period. It can be easily proven that continuous time versions of these problems are NP-Hard. However, a practical “time-slot” version of the problem, where time is discretized into slots, can be solved by standard optimization software. Extensive computational experiments, using different drone delivery ranges as well as different drone fleet sizes, provide valuable insights on the performance of such drone delivery systems.
As we all know, detection and recognition of distant or relatively small targets is a challenging problem. However, the eagle-eye is known for its sharp eyesight, which is called clairvoyance. Inspired by the structural characteristics and physiological mechanism of eagle-eye, a lightweight vision system is designed for target detection and recognition in this paper. First, a hardware platform is established, which consists of a pan-tilt, a short-focus camera and a long-focus camera. Then, combined with the image processing mechanism of the eagle-eye and deep learning, the cameras with different focal lengths are controlled cooperatively to achieve target searching, edge extraction, target detection and recognition. Finally, by separately detecting the targets with different focal length cameras, it can be found that the confidence of the target detection with the long-focus camera is greatly improved. Experimental results show that the designed lightweight eagle-eye vision system can accurately detect and recognize targets, which has a strong adaptive ability.
This chapter presents the proposed multi-criteria MTS algorithm with a continuous UAV dynamical model and a realistic sensor model. Contrary to the MTS algorithms for discrete UAV dynamic models presented in Chap. 4, the algorithms presented in this chapter consider continuous UAV dynamical models that fulfill fixed-wing dynamic restrictions. Besides, the algorithm considers a realistic sensor model and optimizes multiple criteria and constraints. Due to the successful performance of ACO techniques in the discrete approach, in this chapter we select a continuous ant colony based algorithm and propose a continuous MTS heuristic to test if it allows to reduce the computational time (now higher due to the complexity added by the UAV models and by the evaluation criteria).
This chapter discusses the state of the art of Minimum Time Search (MTS) problem, analysing with greater detail several works that have motivated this thesis. The chapter is divided into two sections. The first one discusses, from a general point of view, related probabilistic search problems such as coverage or the Travelling Salesman Problem (TSP), stressing their common characteristics and differences with MTS. The second section analyzes in more detail the state of the art of Probabilistic Search (PS), which aims to find the best Unmanned Vehicles (UV) search trajectories in uncertain environments and which encompasses the MTS problem.
In this chapter we introduce the mathematical formulation of Minimum Time Search (MTS). First, we state the MTS optimization problem and explain how the uncertain information of the elements involved is probabilistically modelled. Then, we introduce how the information about the target location is updated and how the search trajectories are evaluated. Next, we describe, from a general point of view, how PS algorithms solve the search problem. And finally, we introduce the ant colony optimization techniques that are chosen in this thesis to solve the MTS problem.
This paper focuses on solving the multi-agent cooperative target search problem with the demand for obtaining the maximal cumulative detection reward, given the prior target probability map and the sensor detection ability under various constraints. First, a topologically organized model of Glasius bio-inspired neural network (GBNN) is constructed individually for each agent in order to represent the searching environment. The neural activities are determined not only by the activity propagation among neurons, but also by the external input containing the single detection reward and various constraints synthetically. Then, the agent’s searching motion can be selected greedily based on the dynamic activity landscape of GBNN. With the disadvantages of propagation time delay and activity attenuation, however, the relatively global mechanism in GBNN may lead to unsatisfactory performance or even fail to avoid the local optimal problem. Hence the Gaussian mixture model (GMM) is utilized to extract the high-value subregions and compute the future detection reward quantitatively, which can be introduced into the neuron’s external excitatory input of GBNN directly. The simulation results verify the high efficiency and strong robustness of GBNN-GMM in the searching scenarios.
To realize efficient coalition formation of swarm aerial vehicles to attack static and dynamic targets on future battlefields, this study proposes a real-time dynamic network model that takes into consideration constraints in communication range and communication delay. Moreover, by using the improved particle swarm optimization algorithm for decision-making of coalition formation, the SAV resources are made best use of and the size of the coalition is minimized. The simulation result shows that in a dynamic network on the battlefield, the proposed model can realize communication and form coalitions, and the optimized algorithm shows higher efficiency than other algorithms in finding solutions. Monte-Carlo simulation result shows that when the communication delay is small, expanding the communication network to find potential coalition members can effectively reduce the time for task completion and improve the system’s overall performance, but when the communication delay is large, it is better to select coalition members from neighboring SAVs.
This paper proposes a Bayesian approach for minimizing the time of finding an object of uncertain location and dynamics using several moving sensing agents with constrained dynamics. The approach exploits twice the Bayesian theory: on one hand, it uses a Bayesian formulation of the objective functions that compare the constrained paths of the agents and on the other hand, a Bayesian optimization algorithm to solve the problem. By combining both elements, our approach handles successfully this complex problem, as illustrated by the results over different scenarios presented and statistically analyzed in the paper. Finally, the paper also discusses other formulations of the problem and compares the properties of our approach with others closely related.
This paper formulates and proposes a discrete solution for the problem of finding a lost target under uncertainty in minimum time (Minimum Time Search). Given a searching region where some information about the target is known but uncertain (i.e. location and dynamics), and a searching agent with constrained dynamics, we provide two decision making algorithms that optimizes the agent actions to find the target in a minimum time. The problem is faced as a discrete optimization: the actions and the sensor are discrete, and the target probabilistic model is described over a graph, where each vertex contains the target's location probability information and each edge defines the agent possible actions. We revisit the mathematical model of the optimal search problem and we propose a novel approach to include the time into the decision making by reinterpreting it as a maximum discounted time reward problem. The optimal decision plan for the agent is obtained by solving this non convex discrete problem using the cross entropy method. By performing statistical simulations we show how the target is found in minimum time.
This article is based on a talk given by the author at the 1988 TIMS/ORSA meeting in Washington, D.C. about developments in search theory since the Lanchester Prize for 1975 was awarded to him for Theory of Optimal Search. The answer to the question in the title is that a lot has happened, particularly in the area of the search for moving targets and in the use of computers in search theory and applications. The development of search theory began during World War II and has evolved through four eras: classical (1942-1965); Mathematical (1965-1975); algorithmic (1975-1985); dynamic (1985-Present). Each of these eras is discussed.
This paper tackles high-level decision-making techniques for robotic missions, which involve both active sensing and symbolic goal reaching, under uncertain probabilistic environments and strong time constraints. Our case study is a POMDP model of an online multi-target detection and recognition mission by an autonomous UAV. The POMDP model of the multitarget detection and recognition problem is generated online from a list of areas of interest, which are automatically extracted at the beginning of the flight from a coarse-grained high altitude observation of the scene. The POMDP observation model relies on a statistical abstraction of an image processing algorithm's output used to detect targets. As the POMDP problem cannot be known and thus optimized before the beginning of the flight, our main contribution is an "optimize-while- execute" algorithmic framework: it drives a POMDP sub-planner to optimize and execute the POMDP policy in parallel under action duration constraints. We present new results from real outdoor flights and SAIL simulations, which highlight both the benefits of using POMDPs in multi-target detection and recognition missions, and of our "optimize-while-execute" paradigm. Copyright © 2013, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.
We present a decentralized optimization method for solving the coordination problem of interconnected nonlinear discrete-time dynamic systems with multiple decision makers. The optimization framework embeds the inherent structure in which each decision maker has a mathematical model that captures only the local dynamics and the associated interconnecting global constraints. A globally convergent algorithm based on sequential local optimizations is presented. Under assumptions of differentiability and linear independence constraint qualification, we show that the method results in global convergence to ε-feasible Nash solutions that satisfy the Karush-Kuhn-Tucker necessary conditions for Pareto-optimality. We apply this methodology to a multiple unmanned air vehicle system, with kinematic aircraft models, coordinating in a common airspace with separation requirements between the aircraft.
This paper extends a recently developed statistical framework for UAV search with uncertain probability maps to the case of dynamic targets. The probabilities used to encode the information about the environment are typically assumed to be exactly known in the search theory literature, but they are often the result of prior information that is both erroneous and delayed, and will likely be poorly known to mission designers. Our previous work developed a new framework that accounted for the uncertainty in the probability maps for stationary targets, and this paper extends the approach to more realistic dynamic environments. The dynamic case considers probabilistic target motion, creating uncertain probability maps (UPMs) that take into account both poor knowledge of the probabilities and the propagation of their uncertainty through the environment. A key result of this paper is a new algorithm for implementing UPM's in real-time, and it is shown in various simulations that this algorithm leads to more cautious information updates that are less susceptible to false alarms. The paper also provides insights on the impact of the design parameters on the responsiveness of the new algorithm. Several numerical examples are presented to demonstrate the effectiveness of the new framework
This paper develops a set of methods enabling an information-theoretic distributed control architecture to facilitate search by a mobile sensor network. Given a particular configuration of sensors, this technique exploits the structure of the probability distributions of the target state and of the sensor measurements to control the mobile sensors such that future observations minimize the expected future uncertainty of the target state. The mutual information between the sensors and the target state is computed using a particle filter representation of the posterior probability distribution, making it possible to directly use nonlinear and non-Gaussian target state and sensor models. To make the approach scalable to increasing network sizes, single-node and pairwise-node approximations to the mutual information are derived for general probability density models, with analytically bounded error. The pairwise-node approximation is proven to be a more accurate objective function than the single-node approximation. The mobile sensors are cooperatively controlled using a distributed optimization, yielding coordinated motion of the network. These methods are explored for various sensing modalities, including bearings-only sensing, range-only sensing, and magnetic field sensing, all with potential for search and rescue applications. For each sensing modality, the behavior of this non-parametric method is compared and contrasted with the results of linearized methods, and simulations are performed of a target search using the dynamics of actual vehicles. Monte Carlo results demonstrate that as network size increases, the sensors more quickly localize the target, and the pairwise-node approximation provides superior performance to the single-node approximation. The proposed methods are shown to produce similar results to linearized methods in particular scenarios, yet they capture effects in more general scenarios that are not possible with linearized methods.
In this paper we address the problem of finding time-optimal search paths in known environments. In particular, we address the problem of searching a known environment for an object whose unknown location is characterized by a known probability density function (PDF). With this formulation, the time required to find the object is a random variable induced by the choice of search path together with the PDF for the object's location. The optimization problem we consider is that of finding the path that minimizes the expected value of the time required to find the object. As the complexity of the problem precludes finding an exact optimal solution, we propose a two-level, heuristic approach to finding the optimal search path. At the top level, we use a decomposition of the workspace based on critical curves to impose a qualitative structure on the solution trajectory. At the lower level, individual segments of this trajectory are refined using local numerical optimization methods. We have implemented the algorithm and present simulation results for the particular case when the object's location is specified by the uniform PDF.
POMDPsandtheirdecentralizedmultiagentcounterparts,DEC-POMDPs,offer arichframeworkforsequentialdecisionmakingunderuncertainty.Theirhighcomputational complexity,however,presentsanimportantresearchchallenge.Onewaytoaddresstheintrac- table memory requirements of current algorithms is based on representing agent policies as finite-state controllers. Using this representation, we propose a new approach that formulates the problem as a nonlinear program, which defines an optimal policy of a desired size for each agent. This new formulation allows a wide range of powerful nonlinear programming
Unmanned aerial vehicles (UAV) can be used to cover large areas searching for targets. However, sensors on UAVs are typically limited in their accuracy of localization of targets on the ground. On the other hand, unmanned ground vehicles (UGV) can be deployed to accurately locate ground targets, but they have the disadvantage of not being able to move rapidly or see through such obstacles as buildings or fences. In this paper, we describe how we can exploit this synergy by creating a seamless network of UAVs and UGVs. The keys to this are our framework and algorithms for search and localization, which are easily scalable to large numbers of UAVs and UGVs and are transparent to the specificity of individual platforms. We describe our experimental testbed, the framework and algorithms, and some results
This work addresses the problem of tracking multiple moving targets by recursively estimating the joint multitarget probability density (JMPD). Estimation of the JMPD is done in a Bayesian framework and provides a method for tracking multiple targets which allows nonlinear target motion and measurement to state coupling as well as nonGaussian target state densities. The JMPD technique simultaneously estimates both the target states and the number of targets in the surveillance region based on the set of measurements made. We give an implementation of the JMPD method based on particle filtering techniques and provide an adaptive sampling scheme which explicitly models the multitarget nature of the problem. We show that this implementation of the JMPD technique provides a natural way to track a collection of targets, is computationally tractable, and performs well under difficult conditions such as target crossing, convoy movement, and low measurement signal-to-noise ratio (SNR).
We consider decentralized control of Markov decision processes and give complexity bounds on the worst-case running time for algorithms that find optimal solutions. Generalizations of both the fullyobservable case and the partially-observable case that allow for decentralized control are described. For even two agents, the finite-horizon problems corresponding to both of these models are hard for nondeterministic exponential time. These complexity results illustrate a fundamental difference between centralized and decentralized control of Markov decision processes. In contrast to the problems involving centralized control, the problems we consider provably do not admit polynomial-time algorithms. Furthermore, assuming EXP NEXP, the problems require super-exponential time to solve in the worst case.
Since the term robot (from the Czech or Polish words robota, meaning “labour”, and robotnik, meaning “workman”) was introduced in 1923 and the first steps towards real robotic systems were taken by the early-to-mid-1940s, expectations regarding Robotics have shifted from the development of automatic tools to aid or even replace humans in highly repetitive, simple, but physically demanding tasks, to the emergence of autonomous robots and vehicles, and finally to the development of service and social robots.
Approximate dynamic programming has evolved, initially independently, within operations research, computer science and the engineering controls community, all searching for practical tools for solving sequential stochastic optimization problems. More so than other communities, operations research continued to develop the theory behind the basic model introduced by Bellman with discrete states and actions, even while authors as early as Bellman himself recognized its limits due to the "curse of dimensionality" inherent in discrete state spaces. In response to these limitations, subcommunities in computer science, control theory and operations research developed practical methods for solving stochastic, dynamic optimization problems which has emerged as a seemingly disparate family of algorithmic strategies. In this article, we show that there is actually a common theme to these strategies, and underpinning the entire field remains the fundamental algorithmic strategies of value and policy iteration that were first introduced in the 1950's and 60's.
A Dynamic Programming Example: A Shortest Path Problem The Three Curses of Dimensionality Some Real Applications Problem Classes The Many Dialects of Dynamic Programming What is New in this Book? Bibliographic Notes
This paper addresses the problem of coordinating a team of mobile autonomous sensor agents performing a cooperative mission while explicitly avoiding inter-agent collisions in a team negotiation process. Many multi-agent cooperative approaches disregard the potential hazards between agents, which are an important aspect to many systems and especially for airborne systems. In this work, team negotiation is performed using a decentralized gradient-based optimization approach whereas safety distance constraints are specifically designed and handled using Lagrangian multiplier methods. The novelty of our work is the demonstration of a decentralized form of inter-agent collision avoidance in the loop of the agents' real-time group mission optimization process, where the algorithm inherits the properties of performing its original mission while minimizing the probability of inter-agent collisions. Explicit constraint gradient formulation is derived and used to enhance computational advantage and solution accuracy. The effectiveness and robustness of our algorithm has been verified in a simulated environment by coordinating a team of UAVs searching for targets in a large-scale environment.
The paper formulates the optimal control problem for a class of mathematical models in which the system to be controlled is characterized by a finite-state discrete-time Markov process. The formulation is illustrated by a simple machine-maintenance example, and other specific application areas are also discussed. The paper demonstrates that, if there are only a finite number of control intervals remaining, then the optimal payoff function is a piecewise-linear, convex function of the current state probabilities of the internal Markov process.
While several time-optimal trajectory planning techniques have been developed for continuous non-linear systems, there has been little discussion on the subject for discrete non-linear systems. This paper, therefore, presents a technique to search for the time sub-optimal trajectory for general discrete non-linear systems. In this technique, the control inputs with respect to time are partitioned into piecewise constant functions. The piecewise constant functions and the time step interval, which are used in the discretisation of the system, are then searched by a general-purpose non-linear programming optimization method. The example of a time sub-optimal trajectory planning of a SCARA-type manipulator presented in this paper indicates that the proposed technique has the same ability as the existing time sub-optimal trajectory planning techniques for continuous systems. The second numerical example of a non-differentiable car backing-up system shows that the proposed technique also works well for general discrete systems.
This note discusses the complexity of optimally searching for a stationary target in discrete time and space. In this problem, a target's position is fixed within a gridwork of cells. A searcher moves through this gridwork in an attempt to locate the target. At each time step, the searcher must either remain at its current location or move to a neighboring cell. For the discussion of complexity, it is convenient to view the search problem in terms of a finite, connected graph. In this (equivalent) formulation, cells correspond to vertices of a graph and edges of the graph connect neighboring cells. We consider two versions of the optimal searcher path problem. In the first version (denoted by PD), the time horizon is finite and the measure of search effectiveness is probability of detection. In the second version (denoted by ET), the time horizon is infinite and the measure of search effectiveness is expected time to detection. Our main results are that PD is NP-complete and that ET is NP-hard. Refs.
Ben Grocholsky Doctor of Philosophy The University of Sydney March 2002 Information-Theoretic Control of This thesis is concerned with the development of a consistent, information-theoretic basis for understanding of coordination and cooperation decentralised multi-sensor multi-platform systems. Autonomous systems composed of multiple sensors and multiple platforms potentially have significant importance in applications such as defence, search and rescue, mining or intelligent manufacturing. However, the e#ective use of multiple autonomous systems requires that an understanding be developed of the mechanisms of coordination and cooperation between component systems in pursuit of a common goal. A fundamental, quantitative, understanding of coordination and cooperation between decentralised autonomous systems is the main goal of this thesis.
A search is conducted for a target moving in discrete time among a finite number of cells according to a known Markov process. The searcher must choose one cell in which to search in each time period. The set of cells available for search depends upon the cell chosen in the last time period. The problem is to find a searcher path, i.e., a sequence of search cells, that maximizes the probability of detecting the target in a fixed number of time periods. We formulate the problem as a partially observable Markov decision process and present a finite time horizon POMDP solution technique which is simpler than the standard linear programming methods.
From the Publisher:This timely book presents such a consistent framework for addressing data fusion and sensor management. While the framework and the methods presented are applicable to a wide variety of multi-sensor systems, the book focuses on decentralized systems. The book also describes an actual to robot navigation and presents real data and results. The vehicle makes use of sonar sensors with focus of attention capability.
This paper presents an efficient planning and execution algorithm for the navigation of an autonomous rotary wing UAV (RUAV)
manoeuvering in an unknown and cluttered environment. A Rapidly-exploring Random Tree (RRT) variant is used for the generation
of a collision free path and linear Model Predictive Control(MPC) is applied to follow this path. The guidance errors are
mapped to the states of the linear MPC structure by using the nonlinear kinematic equations. The proposed path planning algorithm
considers the run time of the planning stage explicitly and generates a continuous curvature path whenever replanning occurs.
Simulation results show that the RUAV with the proposed methodology successfully achieves autonomous navigation regardless
of its lack of prior information about the environment.
This paper presents a coordinated control technique that allows heterogeneous vehicles to autonomously search for and track multiple targets using recursive Bayesian filtering. A unified sensor model and a unified objective function are proposed to enable search-and-tracking (SAT) within the recursive Bayesian filter framework. The strength of the proposed technique is that a vehicle can switch its task mode between search and tracking while maintaining and using information collected during the operation. Numerical results first show the effectiveness of the proposed technique when a found target becomes lost and must be searched for again. The proposed technique was then applied to a practical marine search-and-rescue (SAR) scenario where heterogeneous vehicles coordinated to search for and track multiple targets. The result demonstrates the applicability of the technique to real search world scenarios
Unmanned aerial vehicles (UAVs) have shown promise in recent years for autonomous sensing. UAVs systems have been proposed for a wide range of applications such as mapping, surveillance, search, and tracking operations. The recent availability of low-cost UAVs suggests the use of teams of vehicles to perform sensing tasks. To leverage the capabilities of a team of vehicles, efficient methods of decentralized sensing and cooperative path planning are necessary. The goal of this work is to examine practical control strategies for a team of fixed-wing vehicles performing cooperative sensing. We seek to develop decentralized, autonomous control strategies that can account for a wide variety of sensing missions. Sensing goals are posed from an information theoretic standpoint to design strategies that explicitly minimize uncertainty. This work proposes a tightly coupled approach, in which sensor models and estimation objectives are used online for path planning.
This paper describes a decentralised asynchronous algorithm for negotiation in team decision and control problems, allowing multiple decision makers to propose and refine future decisions to optimise a common non-linear objective or cost function. A convergence requirement provides an intuitive relationship between the communication frequency, transmission delays and the degree of inter-agent coupling inherent in the system. The coupling is defined by the cross derivative of the objective function. The algorithm is applied to the control of multiple vehicles performing a search task with simulation results given.
This paper presents a novel contribution to the problem of coordinating a team of autonomous sensor agents searching for targets in a large scale environment. Team negotiation is performed using a decentralized gradient-based optimization algorithm. Conventional approaches use finite differencing to approximate the gradient information that is computationally less efficient and exposes the gradient-based optimizer to potential numerical errors and instability. The novelty of our work is the explicit formulation of the gradient for the target search problem that significantly enhances the efficiency in gradient evaluation and robustness for the gradient-based optimization algorithm. We present results by firstly showing the computational advantage and robustness of this explicit gradient model against the finite differencing approach and further demonstrate its application in simulation by coordinating multiple UAVs searching a large scale environment in a decentralized network.
The brain is first and foremost a control system that is capable of building an internal representation of the external world, and using this representation to make decisions, set goals and priorities, formulate plans, and control behavior with intent to achieve its goals. The internal representation is distributed throughout the brain in two forms: (1) firmware embedded in synaptic connections and axon-dendrite circuitry, and (2) dynamic state-variables encoded in the firing rates of neurons in computational loops in the spinal cord, midbrain, subcortical nuclei, and arrays of cortical columns. It assumes that clusters and arrays of neurons are capable of computing logical predicates, smooth arithmetic functions, and matrix transformations over a space defined by large input vectors and arrays. Feedback from output to input of these neural computational units enable them to function as finite-state-automata (fsa), Markov decision processes (MDP), or delay lines in processing signals and generating strings and grammars. Thus, clusters of neurons are capable of parsing and generating language, decomposing tasks, generating plans, and executing scripts. In the cortex, neurons are arranged in arrays of cortical columns that interact in tight loops with their underlying subcortical nuclei. It is hypothesized that these circuits compute sophisticated mathematical and logical functions that maintain and use complex abstract data structures. It is proposed that cortical hypercolumns together with their underlying thalamic nuclei can be modeled as a cortical computational unit (CCU) consisting of a frame-like data structure (containing attributes and pointers) plus the computational processes and mechanisms required to maintain it and use it for perception cognition, and sensory-motor behavior. In sensory processing areas of the brain, CCU processes enable focus of attention, segmentation, grouping, and classification. Pointers stored in CCU frames define relationships that link pixels and signals to objects and events in situations and episodes. CCU frame pointers also link objects and events to class prototypes and overlay them with meaning and emotional values. In behavior generating areas of the brain, CCU processes make decisions, set goals and priorities, generate plans, and control behavior. In general, CCU pointers are used to define rules, grammars, procedures, plans, and behaviors. CCU pointers also define abstract data structures analogous to lists, frames, objects, classes, rules, plans, and semantic nets. It is suggested that it may be possible to reverse engineer the human brain at the CCU level of fidelity using next-generation massively parallel computer hardware and software.
This paper addresses planning of continuous paths for mobile sensors to reduce uncertainty in some quantities of interest in the future.
The mutual information between the continuous measurement path and the future verification variables defines the information reward.
Two expressions for computing this mutual information are presented: the filter form extended from the state-of-the-art and the smoother
form inspired by the conditional independence structure. The key properties of the approach using the filter and smoother strategies are
presented and compared. The smoother form is shown to be preferable because it provides better computational efficiency, facilitates easy
integration with existing path synthesis tools, and most importantly, enables correct quantification of the rate of information accumulation.
A spatial interpolation technique is used to relate the motion of the sensor to the evolution of the measurement matrix, which leads to
the formulation of the optimal path planning problem. A gradient-ascent steering law based on the concept of information potential field
is also presented as a computationally efficient suboptimal strategy. A simplified weather forecasting example is used to compare several
planning methodologies and to illustrate the potential performance benefits of using the proposed planning approach.
This paper presents a Bayesian approach to the problem of searching for a single lost target by a single autonomous sensor
platform. The target may be static or mobile but not evading. Two candidate utility functions for the control solution are
highlighted, namely the Mean Time to Detection, and the Cumulative Probability of Detection. The framework is implemented
for an airborne vehicle looking for both a stationary and a drifting target at sea. Simulation results for different control
solutions are investigated and compared to demonstrate the effectiveness of the method.
This paper presents a technique for dynamically reconfiguring search spaces in order to enable Bayesian au- tonomous search and tracking missions with moving targets. In particular, marine search and rescue scenarios are con- sidered, highlighting the need for space reconfiguration in situations where moving targets are involved. The proposed technique improves the search space configuration by main- taining the validity of the recursive Bayesian estimation. The advantage of the technique is that autonomous search and tracking can be performed indefinitely, without loss of in- formation. Numerical results first show the effectiveness of the technique with a single search vehicle and a single mov- ing target. The efficacy of the approach for coordinated au- tonomous search and tracking is shown through simulation, incorporating multiple search vehicles and multiple targets. The examples also highlight the added benefit to human mis- sion planners resulting from the technique's simplification of the search space allocation task.
Decision field theory (DFT), widely known in the field of mathematical psychology, provides a mathematical model for the evolution of the preferences among options of a human decision-maker. The evolution is based on the subjective evaluation for the options and his/her attention on an attribute (interest). In this paper, we extend DFT to cope with the dynamically changing environment. The proposed extended DFT (EDFT) updates the subjective evaluation for the options and the attention on the attribute, where Bayesian belief network (BBN) is employed to infer these updates under the dynamic environment. Four important theorems are derived regarding the extension, which enhance the usability of EDFT by providing the minimum time steps required to obtain the stabilized results before running the simulation (under certain assumptions). A human-in-the-loop experiment is conducted for the virtual stock market to illustrate and validate the proposed EDFT. The preliminary result is quite promising.
This paper addresses the problem of coordinating a team of multiple heterogeneous sensing platforms searching for a single lost target. In this approach, the utility of a control sequence is a function of the probability density function (PDF) of the target state. Each decision maker builds an equivalent estimate of this PDF by communicating and fusing the information from the other sensor nodes. Coupled utilities incite the agents to collaborate and to agree on the next best set of actions. Decentralized cooperative planning is achieved via anonymous negotiation based on communication of expected observed information. Simulation results demonstrate the efficiency of the cooperative trajectories for a team of autonomous airborne search vehicles.
This paper addresses the problem of cooperative search in a given environment by a team of unmanned aerial vehicles (UAVs). We present a decentralized control model for cooperative search and develop a real-time approach for online cooperation among vehicles, which is based on treating the possible paths of other vehicles as "soft obstacles" to be avoided. Using the approach of "rivaling force" between vehicles to enhance cooperation, each UAV takes into account the possible actions of other UAVs such that the overall information about the environment is increased. The simulation results illustrate the effectiveness of the proposed strategy.
We introduce the concept of a Rapidly-exploring Random Tree (RRT) as a randomized data structure that is designed for a broad class of path planning problems. While they share many of the beneficial properties of existing randomized planning techniques, RRTs are specifically designed to handle nonholonomic constraints (including dynamics) and high degrees of freedom. An RRT is iteratively expanded by applying control inputs that drive the system slightly toward randomly-selected points, as opposed to requiring point-to-point convergence, as in the probabilistic roadmap approach. Several desirable properties and a basic implementation of RRTs are discussed. To date, we have successfully applied RRTs to holonomic, nonholonomic, and kinodynamic planning problems of up to twelve degrees of freedom.
Asynchronous Decision Making for Decentralised Autonomous Systems
- G Mathews
G. Mathews, Asynchronous Decision Making for Decentralised Autonomous Systems, Ph.D. Thesis, The University of Sydney, 2008.
A bayesian approach for constrained multi-agent minimum time search in uncertain dynamic domains
- P Lanillos
- J Zuluaga
- J J Ruz
- E Besada-Portas
P. Lanillos, J. Yañez Zuluaga, J.J. Ruz, E. Besada-Portas, A bayesian approach for constrained multi-agent minimum time search in uncertain dynamic
domains, in: Proc. of the Fifteenth Annual Conference on Genetic and Evolutionary Computation Conference, GECCO 2013, ACM, New York, NY, USA,
2013, pp. 391-398.
Minimum time search for lost targets using cross entropy optimization
- P Lanillos
- E Besada-Portas
- G Pajares
- J J Ruz
P. Lanillos, E. Besada-Portas, G. Pajares, J.J. Ruz, Minimum time search for lost targets using cross entropy optimization, in: Proc. IEEE/RSJ International
Conference on Intelligent Robots and Systems, IROS 2012, pp. 602 -609.