Chapter

Chapter

# Autonomous Swarm of Heterogeneous Robots for Surveillance Operations

If you want to read the PDF, try requesting it from the authors.

## Abstract

The introduction of Unmanned vehicles (UxVs) in the recent years has created a new security field that can use them as both a potential threat as well as new technological weapons against those threats. Dealing with these issues from the counter-threat perspective, the proposed architecture project focuses on designing and developing a complete system which utilizes the capabilities of multiple UxVs for surveillance objectives in different operational environments. Utilizing a combination of diverse UxVs equipped with various sensors, the developed architecture involves the detection and the characterization of threats based on both visual and thermal data. The identification of objects is enriched with additional information extracted from other sensors such as radars and RF sensors to secure the efficiency of the overall system. The current prototype displays diverse interoperability concerning the multiple visual sources that feed the system with the required optical data. Novel detection models identify the necessary threats while this information is enriched with higher-level semantic representations. Finally, the operator is informed properly according to the visual identification modules and the outcomes of the UxVs operations. The system can provide optimal surveillance capacities to the relevant authorities towards an increased situational awareness.

## No full-text available

Article
Full-text available
This paper presents a distributed algorithm applicable to a wide range of practical multi-robot applications. In such multi-robot applications, the user-defined objectives of the mission can be cast as a general optimization problem, without explicit guidelines of the subtasks per different robot. Owing to the unknown environment, unknown robot dynamics, sensor nonlinearities, etc., the analytic form of the optimization cost function is not available a priori. Therefore, standard gradient-descent-like algorithms are not applicable to these problems. To tackle this, we introduce a new algorithm that carefully designs each robot’s subcost function, the optimization of which can accomplish the overall team objective. Upon this transformation, we propose a distributed methodology based on the cognitive-based adaptive optimization (CAO) algorithm, that is able to approximate the evolution of each robot’s cost function and to adequately optimize its decision variables (robot actions). The latter can be achieved by online learning only the problem-specific characteristics that affect the accomplishment of mission objectives. The overall, low-complexity algorithm can straightforwardly incorporate any kind of operational constraint, is fault tolerant, and can appropriately tackle time-varying cost functions. A cornerstone of this approach is that it shares the same convergence characteristics as those of block coordinate descent algorithms. The proposed algorithm is evaluated in three heterogeneous simulation set-ups under multiple scenarios, against both general-purpose and problem-specific algorithms.
Article
Full-text available
We present a novel and practical deep fully convolutional neural network architecture for semantic pixel-wise segmentation termed SegNet. This core trainable segmentation engine consists of an encoder network, a corresponding decoder network followed by a pixel-wise classification layer. The architecture of the encoder network is topologically identical to the 13 convolutional layers in the VGG16 network [1]. The role of the decoder network is to map the low resolution encoder feature maps to full input resolution feature maps for pixel-wise classification. The novelty of SegNet lies is in the manner in which the decoder upsamples its lower resolution input feature map(s). Specifically, the decoder uses pooling indices computed in the max-pooling step of the corresponding encoder to perform non-linear upsampling. This eliminates the need for learning to upsample. The upsampled maps are sparse and are then convolved with trainable filters to produce dense feature maps. We compare our proposed architecture with the widely adopted FCN [2] and also with the well known DeepLab-LargeFOV [3] , DeconvNet [4] architectures. This comparison reveals the memory versus accuracy trade-off involved in achieving good segmentation performance. SegNet was primarily motivated by scene understanding applications. Hence, it is designed to be efficient both in terms of memory and computational time during inference. It is also significantly smaller in the number of trainable parameters than other competing architectures and can be trained end-to-end using stochastic gradient descent. We also performed a controlled benchmark of SegNet and other architectures on both road scenes and SUN RGB-D indoor scene segmentation tasks. These quantitative assessments show that SegNet provides good performance with competitive inference time and most efficient inference memory-wise as compared to other architectures. We also provide a Caffe implementation of SegNet and a web demo at http://mi.eng.cam.ac.uk/projects/segnet/.
Article
Full-text available
This paper deals with the path planning problem of a team of mobile robots, in order to cover an area of interest, with prior-defined obstacles. For the single robot case, also known as single robot coverage path planning (CPP), an 𝓞(n) optimal methodology has already been proposed and evaluated in the literature, where n is the grid size. The majority of existing algorithms for the multi robot case (mCPP), utilize the aforementioned algorithm. Due to the complexity, however, of the mCPP, the best the existing mCPP algorithms can perform is at most 16 times the optimal solution, in terms of time needed for the robot team to accomplish the coverage task, while the time required for calculating the solution is polynomial. In the present paper, we propose a new algorithm which converges to the optimal solution, at least in cases where one exists. The proposed technique transforms the original integer programming problem (mCPP) into several single-robot problems (CPP), the solutions of which constitute the optimal mCPP solution, alleviating the original mCPP explosive combinatorial complexity. Although it is not possible to analytically derive bounds regarding the complexity of the proposed algorithm, extensive numerical analysis indicates that the complexity is bounded by polynomial curves for practical sized inputs. In the heart of the proposed approach lies the DARP algorithm, which divides the terrain into a number of equal areas each corresponding to a specific robot, so as to guarantee complete coverage, non-backtracking solution, minimum coverage path, while at the same time does not need any preparatory stage (video demonstration and standalone application are available on-line http://tinyurl.com/DARP-app).
Conference Paper
Full-text available
We present a method for detecting objects in images using a single deep neural network. Our approach, named SSD, discretizes the output space of bounding boxes into a set of default boxes over different aspect ratios and scales per feature map location. At prediction time, the network generates scores for the presence of each object category in each default box and produces adjustments to the box to better match the object shape. Additionally, the network combines predictions from multiple feature maps with different resolutions to naturally handle objects of various sizes. SSD is simple relative to methods that require object proposals because it completely eliminates proposal generation and subsequent pixel or feature resampling stages and encapsulates all computation in a single network. This makes SSD easy to train and straightforward to integrate into systems that require a detection component. Experimental results on the PASCAL VOC, COCO, and ILSVRC datasets confirm that SSD has competitive accuracy to methods that utilize an additional object proposal step and is much faster, while providing a unified framework for both training and inference. For $$300 \times 300$$ input, SSD achieves 74.3 % mAP on VOC2007 test at 59 FPS on a Nvidia Titan X and for $$512 \times 512$$ input, SSD achieves 76.9 % mAP, outperforming a comparable state of the art Faster R-CNN model. Compared to other single stage methods, SSD has much better accuracy even with a smaller input image size. Code is available at https:// github. com/ weiliu89/ caffe/ tree/ ssd.
Article
Full-text available
We present a novel and practical deep fully convolutional neural network architecture for semantic pixel-wise segmentation termed SegNet. This core trainable segmentation engine consists of an encoder network, a corresponding decoder network followed by a pixel-wise classification layer. The architecture of the encoder network is topologically identical to the 13 convolutional layers in the VGG16 network . The role of the decoder network is to map the low resolution encoder feature maps to full input resolution feature maps for pixel-wise classification. The novelty of SegNet lies is in the manner in which the decoder upsamples its lower resolution input feature map(s). Specifically, the decoder uses pooling indices computed in the max-pooling step of the corresponding encoder to perform non-linear upsampling. This eliminates the need for learning to upsample. The upsampled maps are sparse and are then convolved with trainable filters to produce dense feature maps. We compare our proposed architecture with the fully convolutional network (FCN) architecture and its variants. This comparison reveals the memory versus accuracy trade-off involved in achieving good segmentation performance. The design of SegNet was primarily motivated by road scene understanding applications. Hence, it is efficient both in terms of memory and computational time during inference. It is also significantly smaller in the number of trainable parameters than competing architectures and can be trained end-to-end using stochastic gradient descent. We also benchmark the performance of SegNet on Pascal VOC12 salient object segmentation and the recent SUN RGB-D indoor scene understanding challenge. We show that SegNet provides competitive performance although it is significantly smaller than other architectures. We also provide a Caffe implementation of SegNet and a webdemo at http://mi.eng.cam.ac.uk/projects/segnet/
Article
Full-text available
Congress has expressed a great deal of interest in using Unmanned Aerial Vehicles (UAVs) to surveil the United States' international land border. U.S. Customs and Border Protection (CBP) utilizes advanced technology to augment its USBP agents' ability to patrol the border, including a fleet of six UAVs. This report examines the strengths and limitations of deploying UAVs along the borders and related issues for Congress. UAVs come with several costs and benefits. One potential benefit of UAVs is that they could fill a gap in current border surveillance by improving coverage along remote sections of the U.S. borders. Moreover, the range of UAVs is a significant asset when compared to border agents on patrol or stationary surveillance equipment. Yet, despite potential benefits of using UAVs for homeland security, various problems encountered in the past may hinder UAV implementation on the border. There are concerns regarding the high accident rates of UAVs, which have historically been multiple times higher than that of manned aircraft. Inclement weather conditions can also impinge on a UAV?s surveillance capability. Also, according to the CBP Inspector General, the costs of operating a UAV are more than double the costs of operating a manned aircraft. Recent attention has focused on the expanding area of operations for CBP-operated UAVs. On June 23, 2010, the Federal Aviation Administration (FAA) granted a certificate of authorization requested by CBP, clearing the UAV flights along the Texas border and Gulf region. Other requests have reportedly been delayed due to safety concerns, some of which stem from previous incidents. Despite safety concerns, some policymakers continue to call for the increased domestic use of UAVs. The Supplemental Appropriations Bill of FY2010 (H.R. 4899) would include \$32 million for the acquisition of two additional UAVs by CBP. This report will be updated as events warrant.
Article
Full-text available
Airborne and space-borne sensors are reviewed and evaluated in terms of their usefulness in responding to oil spills. Recent developments and trends in sensor technology are illustrated with specific examples. The discussion of the sensors is divided into two main categories, namely active and passive. Active sensors are those that provide their own source of illumination or excitation, whereas passive sensors rely on illumination from a secondary source. A common passive sensor is an infrared camera or an IR/UV (infrared/ultraviolet) system. The inherent weaknesses include the inability to discriminate oil on beaches, among seaweeds or debris. Among active sensors, the laser fluorosensor is a most useful instrument because of its unique capability to identify oil on backgrounds that include water, soil, ice and snow. It is the only sensor that can positively discriminate oil on most backgrounds. Disadvantages include the large size, weight and high cost. Radar offers the only potential for large area searches and foul weather remote sensing. Radar is costly, requires a dedicated aircraft, and is prone to many interferences. Equipment that measures relative slick thickness is still under development. Passive microwave has been studied for several years, but many commercial instruments lack sufficient spatial resolution to be practical, operational instruments. A laser-acoustic instrument, which provides the only technology to measure absolute oil thickness, is under development. Equipment operating in the visible region of the spectrum, such as cameras and scanners, is useful for documentation or providing a basis for the overlay of other data. It is not useful beyond this because oil shows no spectral characteristics in the visible region which can be used to discriminate oil.
Article
Full-text available
The Pascal Visual Object Classes (VOC) challenge is a benchmark in visual object category recognition and detection, providing the vision and machine learning communities with a standard dataset of images and annotation, and standard evaluation procedures. Organised annually from 2005 to present, the challenge and its associated dataset has become accepted as the benchmark for object detection. This paper describes the dataset and evaluation procedure. We review the state-of-the-art in evaluated methods for both classification and detection, analyse whether the methods are statistically different, what they are learning from the images (e.g. the object or its context), and what the methods find easy or confuse. The paper concludes with lessons learnt in the three year history of the challenge, and proposes directions for future improvement and extension.
Article
Full-text available
On a grand scale, visual analytics solutions provide technology that combines the strengths of human and electronic data processing. Visualization becomes the medium of a semi-automated analytical process, where humans and machines cooperate using their respective distinct capabilities for the most effective results. The diversity of these tasks can not be tackled with a single theory. Visual analytics research is highly interdisciplinary and combines various related research areas such as visualization, data mining, data management, data fusion, statistics and cognition science (among others).
Article
Full-text available
A novel approach for bidimensional empirical mode decomposition (BEMD) is proposed in this paper. BEMD decomposes an image into multiple hierarchical components known as bidimensional intrinsic mode functions (BIMFs). In each iteration of the process, two-dimensional (2D) interpolation is applied to a set of local maxima (minima) points to form the upper (lower) envelope. But, 2D scattered data interpolation methods cause huge computation time and other artifacts in the decomposition. This paper suggests a simple, but effective, method of envelope estimation that replaces the surface interpolation. In this method, order statistics filters are used to get the upper and lower envelopes, where filter size is derived from the data. Based on the properties of the proposed approach, it is considered as fast and adaptive BEMD (FABEMD). Simulation results demonstrate that FABEMD is not only faster and adaptive, but also outperforms the original BEMD in terms of the quality of the BIMFs.
Conference Paper
Full-text available
This paper discusses the design considerations related to the transceiver hardware elements within a software defined radio (SDR). Receiver architectures are reviewed and the viability of manufacturing these in the short to medium term is considered. The filtering functions required for a conventional receiver are examined, and the problems associated with implementation of these filters within a SDR receiver discussed, including that of image filtering. Receiver linearity requirements are evaluated in terms of typical user applications and it is shown that these will be onerous in the case of a SDR receiver. A novel technique for mixer linearisation is presented as a potentially enabling element within a re-configurable transceiver and some initial performance results are reported. The possibility of constructing an electronically tunable preselection filter using micro-electromechanical systems (MEMS) technology is examined. Some simulation and prototype measurement results are presented.
Article
Spatial pyramid pooling module or encode-decoder structure are used in deep neural networks for semantic segmentation task. The former networks are able to encode multi-scale contextual information by probing the incoming features with filters or pooling operations at multiple rates and multiple effective fields-of-view, while the latter networks can capture sharper object boundaries by gradually recovering the spatial information. In this work, we propose to combine the advantages from both methods. Specifically, our proposed model, DeepLabv3+, extends DeepLabv3 by adding a simple yet effective decoder module to refine the segmentation results especially along object boundaries. We further explore the Xception model and apply the depthwise separable convolution to both Atrous Spatial Pyramid Pooling and decoder modules, resulting in a faster and stronger encoder-decoder network. We demonstrate the effectiveness of the proposed model on the PASCAL VOC 2012 semantic image segmentation dataset and achieve a performance of 89% on the test set without any post-processing. Our paper is accompanied with a publicly available reference implementation of the proposed models in Tensorflow.
Conference Paper
State-of-the-art object detection networks depend on region proposal algorithms to hypothesize object locations. Advances like SPPnet [7] and Fast R-CNN [5] have reduced the running time of these detection networks, exposing region pro-posal computation as a bottleneck. In this work, we introduce a Region Proposal Network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals. An RPN is a fully-convolutional network that simultaneously predicts object bounds and objectness scores at each position. RPNs are trained end-to-end to generate high-quality region proposals, which are used by Fast R-CNN for detection. With a simple alternating optimization, RPN and Fast R-CNN can be trained to share convolu-tional features. For the very deep VGG-16 model [18], our detection system has a frame rate of 5fps (including all steps) on a GPU, while achieving state-of-the-art object detection accuracy on PASCAL VOC 2007 (73.2% mAP) and 2012 (70.4% mAP) using 300 proposals per image. The code will be released.
Article
This survey summarizes almost 50 years of research and development in the field of Augmented Reality (AR). From early research in the 1960's until widespread availability by the 2010's there has been steady progress towards the goal of being able to seamlessly combine real and virtual worlds. We provide an overview of the common definitions of AR, and show how AR fits into taxonomies of other related technologies. A history of important milestones in Augmented Reality is followed by sections on the key enabling technologies of tracking, display and input devices. We also review design guidelines and provide some examples of successful AR applications. Finally, we conclude with a summary of directions for future work and a review of some of the areas that are currently being researched.
Article
Cyber-physical systems are ubiquitous in power systems, transportation networks, industrial control processes, and critical infrastructures. These systems need to operate reliably in the face of unforeseen failures and external malicious attacks. In this paper: (i) we propose a mathematical framework for cyber-physical systems, attacks, and monitors; (ii) we characterize fundamental monitoring limitations from system-theoretic and graph-theoretic perspectives; and (ii) we design centralized and distributed attack detection and identification monitors. Finally, we validate our findings through compelling examples.
Conference Paper
Dynamic Data Driven Application Systems (DDDAS) entails the ability to incorporate additional data into an executing application - these data can be archival or collected on-line; and in reverse, the ability of applications to dynamically steer the measurement process. The paradigm offers the promise of improving modeling methods, and augmenting the analysis and prediction capabilities of application simulations and the effectiveness of measurement systems. This presents the potential to transform the way science and engineering are done, and induce a major impact in the way many functions in our society are conducted, such as manufacturing, commerce, hazard management, medicine. Enabling this synergistic feedback and control-loop between application simulations and measurements requires novel application modeling approaches and frameworks, algorithms tolerant to perturbations from dynamic data injection and steering, and systems software to support the dynamic environments of concern here. Recent advances in complex applications, the advent of grid computing and of sensor systems, are some of the technologies that make it timely to embark in developing DDDAS capabilities. Research and development of such technologies requires synergistic multidisciplinary collaboration in the applications, algorithms, software systems, and measurements systems areas, and involving researchers in basic sciences, engineering, and computer sciences. The rest of the papers in the proceedings of this workshop provide examples of ongoing research developing DDDAS technologies within the context of specific and important application areas.
Increasing Cooperation between the European Maritime Domain Authorities
• I Tikanmäki
• H Ruoslahti
Tikanmki, I., Ruoslahti, H. : "Increasing Cooperation between the European Maritime Domain Authorities", 2017.
• SK Card