Conference Paper

# UAV-based 3D modelling of disaster scenes for Urban Search and Rescue

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

## Abstract

Natural or man-made disasters often result in trapped victims under rubble piles. In such emergency response situations, Urban Search and Rescue (USaR) teams have to make quick decisions to determine the location of possible trapped humans. The fast 3D modelling of collapsed buildings using images from Unmanned Aerial Vehicles (UAVs) can significantly help the USaR operations and improve disaster response. The a-priori establishment of a proper workflow for fast and reliable image-based 3D modelling and the careful parameterization in every step of the photogrammetric process are crucial aspects that ensure the readiness in an emergency situation. This paper evaluates powerful commercial and open-source software for the creation of 3D models of disaster scenes using UAV imagery for rapid response situations and conducts a thorough analysis on the parameters of the various modelling steps that may lead to the desired results for USaR operations. The main result of our analysis is the establishment of optimized photogrammetric procedures with the scope of fast 3D modelling of disaster scenes, to assist USaR teams and increase survival rates.

## Supplementary resource (1)

... They are expected to communicate with many different smart objects, such as sensors and embedded devices [64]. Their mobility [61], the ability to provide real-time data [65] and the potential to carry hardware for on-board decision making [66] make UAVs the ideal candidate. Indeed, as projected by in [63], UAVs are expected to play a critical role in smart cities and city-wide IoT (Internet-of-Things) infrastructure. ...
... As such, they can operate in three spacial dimensions (i.e., get close to objects even if these are at a distance from the ground, assume elevated positions above locations or objects of interest and chose their position on the basis of it being sustainable and not affecting other participants of smart city operations), are not restricted by congested networks on the ground and can autonomously adapt their exact location and the settings of their sensors to deliver the best possible service. Platform mounted sensors can contribute to public safety in the areas of imagery [65,74,75], monitoring [34,40,53,69,74,[76][77][78][79][80][81][82][83][84][85][86][87][88] and surveillance [34,40,46,48,53,78,82,87,[89][90][91]]. ...
... UAVs have been used to support patrolling [70,88] (e.g., border patrol [34,53], coast patrol [95] and even indoor patrol [99]), to inform situational awareness [33,65,81,88,96,97], for costal management [68], in the context of terrain survey and mapping [90] and, generally, for data collection [89]. ...
Article
Full-text available
The use of UAVs in areas ranging from agriculture over urban services to entertainment or simply as a hobby has rapidly grown over the last years. Regarding serious/commercial applications, UAVs have been considered in the literature, especially as mobile sensing/actuation platforms (i.e., as a delivery platform for an increasingly wide range of sensors and actuators). With regard to timely, cost-effective and very rich data acquisition, both, NEC Research as well as TNO are pursuing investigations into the use of UAVs and swarms of UAVs for scenarios where high-resolution requirements, prohibiting environments or tight time constraints render traditional approaches ineffective. In this review article, we provide a brief overview of safety and security-focused application areas that we identified as main targets for industrial and commercial projects, especially in the context of intelligent autonomous systems and autonomous/semi-autonomously operating swarms. We discuss a number of challenges related to the deployment of UAVs in general and to their deployment within the identified application areas in particular. As such, this article is meant to serve as a review and overview of the literature and the state-of-the-art, but also to offer an outlook over our possible (near-term) future work and the challenges that we will face there.
... A.2 Rescue in areas with difficult access [63][64][65][66]. ...
... As an example, none of the UAS used in the disasters and emergency category (A) provides processing features, although these features are required for the category (see Table 3). These features are replaced, in most cases, by storage capacity so that the images recorded are stored in the device and processed by the server later on, once the UAS has landed (e.g., in [60,64] or [65]). ...
Article
Full-text available
Sustainability is at the heart of many application fields where the use of Unmanned Aerial Systems (UAS) is becoming more and more important (e.g., agriculture, fire detection and prediction, environmental surveillance, mapping, etc.). However, their usage and evolution are highly conditioned by the specific application field they are designed for, and thus, they cannot be easily reused among different application fields. From this point of view, being that they are not multipurpose, we can say that they are not fully sustainable. Bearing this in mind, the objective of this paper is two-fold: on the one hand, to identify the whole set of features that must be provided by a UAS to be considered sustainable and to show that there is no UAS satisfying all these features; on the other hand, to present an open and sustainable UAS architecture that may be used to build UAS on demand to provide the features needed in each application field. Since this architecture is mainly based on software and hardware adaptability, it contributes to the technical sustainability of cities.
... Second, these airborne machine's mere operation, which relies intensely on their potential cyber pose a great menace to individuals and property. 48 In disaster, UAVs have been widely used for collection of data, 49 missions including search and rescue (SAR), [50][51][52][53][54][55][56][57] triage, 58 surveillance (urban, 59 base station, 38 border 60 ) and monitoring, 53,[61][62][63][64][65] imagery. 51,61,66 This study covers the vast applications of UAVs from military to disaster situations, thereby finding its way in numerous applications of the real world. ...
... 48 In disaster, UAVs have been widely used for collection of data, 49 missions including search and rescue (SAR), [50][51][52][53][54][55][56][57] triage, 58 surveillance (urban, 59 base station, 38 border 60 ) and monitoring, 53,[61][62][63][64][65] imagery. 51,61,66 This study covers the vast applications of UAVs from military to disaster situations, thereby finding its way in numerous applications of the real world. To the best of the author's knowledge, this is the first comprehensive study to use the three powerful techniques [blockchain, machine learning (ML), watermarking] for securing UAVs applications. ...
Article
Full-text available
Unmanned aerial vehicles (UAVs) or Drones technology has a huge potential for supporting different efficient solutions for the smart applications in our world. The applications include smart things, smart transportation, smart cities, smart healthcare, smart personal care, smart house, smart industries, and so on. Due to the sensitive applications of UAVs, the security has become a major concern, and therefore, efficient techniques are required to protect captured data from hackers and the fictitious activities from illegitimate users. Machine learning (ML) techniques play a vital role in improving UAVs' security intelligently, while blockchain is recent technology for decentralized UAVs and security. Furthermore, watermarking guarantees digital media to be authenticated, protected, and copyright. Therefore, we provide a comprehensive survey of optimal techniques, which are used for securing UAVs applications in terms of blockchain, ML, and watermarking. Furthermore, we introduce each technique with the advantages and suitably used for securing UAVs collaboration applications. This survey contributes to a better understanding of the blockchain, ML, and watermarking techniques for securing UAVs and sheds new light on challenges and opportunities on subject applications.
... For this reason, increasing attention is being paid to alternative airborne remote sensing systems, such as UAVs. Photos taken by UAVs can be used to construct a 3D model of a particular point of interest (POI) using the structure-from-motion (SfM) and multiview stereo (MVS) methods [7][8][9] . ...
Article
Full-text available
Emergency responders require accurate and comprehensive data to make informed decisions. Moreover, the data should be acquired and analyzed swiftly to ensure an efficient response. One of the tasks at hand post-disaster is damage assessment within the impacted areas. In particular, building damage should be assessed to account for possible casualties, and displaced populations, to estimate long-term shelter capacities, and to assess the damage to services that depend on essential infrastructure (e.g. hospitals, schools, etc.). Remote sensing techniques, including satellite imagery, can be used to gathering such information so that the overall damage can be assessed. However, specific points of interest among the damaged buildings need higher resolution images and detailed information to assess the damage situation. These areas can be further assessed through unmanned aerial vehicles and 3D model reconstruction. This paper presents a multi-UAV coverage path planning method for the 3D reconstruction of postdisaster damaged buildings. The methodology has been implemented in NetLogo3D, a multi-agent model environment, and tested in a virtual built environment in Unity3D. The proposed method generates camera location points surrounding targeted damaged buildings. These camera location points are filtered to avoid collision and then sorted using the K-means or the Fuzzy C-means methods. After clustering camera location points and allocating these to each UAV unit, a route optimization process is conducted as a multiple traveling salesman problem. Final corrections are made to paths to avoid obstacles and give a resulting path for each UAV that balances the flight distance and time. The paper presents the details of the model and methodologies, and an examination of the texture resolution obtained from the proposed method and the conventional overhead flight with the nadir-looking method used in 3D mappings. The algorithm outperforms the conventional method in terms of the quality of the generated 3D model.
... Recently, drones have been proposed as a tool to support several aspects of disaster response [7,8] such as gathering information on the ground state, 3D mapping [9], situational awareness, logistics planning, damage assessment, communications relay points, and direct SAR missions. They have also been used to support SAR missions [10] through delivery of live-saving equipment such as life vests [11,12], or allowing for audible communication through speakers with people once they are located. ...
Article
Full-text available
Search and rescue (SAR) is a vital line of defense against unnecessary loss of life. However, in a potentially hazardous environment, it is important to balance the risks associated with SAR action. Drones have the potential to help with the efficiency, success rate and safety of SAR operations as they can cover large or hard to access areas quickly. The addition of thermal cameras to the drones provides the potential for automated and reliable detection of people in need of rescue. We performed a pilot study with a thermal-equipped drone for SAR applications in Morecambe Bay. In a variety of realistic SAR scenarios, we found that we could detect humans who would be in need of rescue, both by the naked eye and by a simple automated method. We explore the current advantages and limitations of thermal drone systems, and outline the future path to a useful system for deployment in real-life SAR.
... Use of UAV-borne photogrammetry in emergency has long been the object of several researches [15][16][17][18][19]. ...
Conference Paper
Fire Departments feature specialized teams for Urban Search And Rescue (USAR) activities, operating in case of disasters and in collapse contexts, where the actual situation no longer coincides with any previous survey. In this context, current UAV-borne photogrammetry may offer very effective methods, enabling achievement of to-date knowledge of the status quo. Their effectiveness in these contexts is due to the ability of drones to operate in triple-D areas (Dull, Dusty, Dangerous) and to the remote sensing features inherent in photogrammetry.
... Mainly, the fact that oblique aerial images depict not only horizontal structures (e.g., roofs) but also vertical ones (e.g., building façades) [50], is very fundamental for 3D modelling, as the need for additional terrestrial images may be eliminated thanks to the use of multi-perspective oblique aerial imagery. In this way, less time is required for image acquisition, which constitutes a major advantage for emergency response situations, where the time parameter is very crucial [53]. What is more, image acquisition during an emergency situation is performed under stress and may result either in large unordered datasets or in incomplete datasets. ...
Article
Full-text available
Natural and man-made disasters that may take place due to a catastrophic incident (e.g., earthquake, explosion, terrorist attack) often result in trapped humans under rubble piles. In such emergency response situations, Urban Search and Rescue (USaR) teams have to make quick decisions under stress in order to determine the location of possible trapped victims. Fast 3D modelling of fully or partially collapsed buildings using images from Unmanned Aerial Vehicles (UAVs) can considerably help USaR efforts, thus improving disaster response and increasing survival rates. The a-priori establishment of a proper workflow for fast and reliable image-based 3D modelling and the a priori determination of the parameters that have to be set in each step of the photogrammetric pipeline are critical aspects that ensure the readiness in an emergency response situation. This paper evaluates powerful commercial and open-source software solutions for the 3D reconstruction of disaster scenes for rapid response situations. The software packages are tested using UAV datasets of a real earthquake scene. A thorough analysis on the parameters of the various modelling steps that may lead to desired results for USaR tasks is made and indicative processing chains are proposed, taking into account the restriction of time. Furthermore, some weaknesses of the data acquisition process that have been detected by performing the experiments are outlined and some improvements and additions are proposed, including an initial preprocessing of the images using a graph-based approach.
... Some indicative use cases are the following: (i) applications for which a small number of coplanar GCPs (e.g., lying on the roof of a building) that are distributed in a small area compared with the area of the total image dataset is available; (ii) applications for which the measurement of GCPs in the area of interest is not possible; and (iii) applications for which the person in charge of the orientation process is a non-photogrammetrist. For instance, one application scenario for which the advantages of the proposed algorithm are particularly significant is the orientation of a block of images taken be an unmanned aerial vehicle (UAV) in the case of an emergency scenario for 2D mapping (e.g., Boccardo et al., 2015) or 3D modelling (e.g., Ferworn et al., 2011;Verykokou et al., 2016aVerykokou et al., , 2016bVerykokou et al., 2018) of a disaster area. In such emergency response situations (e.g., earthquake, explosion), the measurement of GCPs is usually not possible in the area of the disaster (e.g., debris of a building). ...
Article
Full-text available
The purpose of this paper is the presentation of a novel algorithm for automatic estimation of the exterior orientation parameters of image datasets, which can be applied in the case that the scene depicted in the images has a planar surface (e.g., roof of a building). The algorithm requires the measurement of four coplanar ground control points (GCPs) in only one image. It uses a template matching method combined with a homography-based technique for transfer of the GCPs in another image, along with an incremental photogrammetry-based Structure from Motion (SfM) workflow, coupled with robust iterative bundle adjustment methods that reject any remaining outliers, which have passed through the checks and geometric constraints imposed during the image matching procedure. Its main steps consist of (i) determination of overlapping images without the need for GPS/INS data; (ii) image matching and feature tracking; (iii) estimation of the exterior orientation parameters of a starting image pair; and (iv) photogrammetry-based SfM combined with iterative bundle adjustment methods. A developed software solution implementing the proposed algorithm was tested using a set of UAV oblique images. Several tests were performed for the assessment of the errors and comparisons with well-established commercial software were made, in terms of automation and correctness of the computed exterior orientation parameters. The results show that the estimated orientation parameters via the proposed solution have comparable accuracy with those ones computed through the commercial software using the highest possible accuracy settings; in addition, double manual work was required by the commercial software compared to the proposed solution.
... Using depth sensors to construct 3D models of the disaster scenes is common in many works; such as [31,32], which provides accurate distances information of the detected objects. In [33], a rescue assistance tool is presented, where a system based on the 3D modeling of the rubble piles is implemented in order to perform photometric analysis of 3D modeling of the disaster scenes and increase the ability to detect possible victims. With the aim of improving the 3D mapping result, a heterogeneous robot collaboration of UGV-UAV is presented in [34]. ...
Article
Full-text available
The automation of the Wilderness Search and Rescue (WiSAR) task aims for high levels of understanding of various scenery. In addition, working in unfriendly and complex environments may cause a time delay in the operation and consequently put human lives at stake. In order to address this problem, Unmanned Aerial Vehicles (UAVs), which provide potential support to the conventional methods, are used. These vehicles are provided with reliable human detection and tracking algorithms; in order to be able to find and track the bodies of the victims in complex environments, and a robust control system to maintain safe distances from the detected bodies. In this paper, a human detection based on the color and depth data captured from onboard sensors is proposed. Moreover, the proposal of computing data association from the skeleton pose and a visual appearance measurement allows the tracking of multiple people with invariance to the scale, translation and rotation of the point of view with respect to the target objects. The system has been validated with real and simulation experiments, and the obtained results show the ability to track multiple individuals even after long-term disappearances. Furthermore, the simulations present the robustness of the implemented reactive control system as a promising tool for assisting the pilot to perform approaching maneuvers in a safe and smooth manner.
... Medium-quality photogrammetric processing refers to resolution downscaling of one sixteenth of the input resolution (e.g. Verykokou et al., 2016), and it took between 1 h and 14 h to complete the construction of the virtual outcrops analyzed here ( Table 1). The high-quality setting refers to resolution downscaling of a quarter of the original input (e.g. ...
Article
Although virtual outcrops or digital outcrop model applications in geological studies are becoming increasingly common, virtually no attention has been given to acquisition of palaeocurrent data from cross‐strata. Palaeocurrent indicators are abundant in the sedimentary record, mainly as cross‐strata, and present compelling implications for basin analysis, plate tectonics and palaeoenvironmental reconstructions, as well as for sediment transport systems. Palaeocurrents from cross‐strata also comprise a quantifiable parameter in sedimentary successions that can be used in quantitative modelling of depositional architecture with implications for reservoir characterization through outcrop analogue studies. A common obstacle in obtaining large cross‐strata orientation datasets is the limited access to steep and high outcrop faces, as well as financial and time restrictions on field work. To overcome these issues, the simplest workflow for drone‐based acquisition, photogrammetric processing and analysis of virtual outcrops aiming at the attainment of palaeocurrent data from cross‐strata is appraised here. The results show that orientations of cross‐beds measured on virtual outcrops, with and without ground control points, and using two different levels of resolution downsampling, are comparable to field measurements using analogue and electronic devices. Time and financial resources can thus be saved by using the straightforward method presented here to supplement palaeocurrent mapping across wide areas and distinct stratigraphic intervals.
... Photogrammetric approaches are therefore mostly applied to large scale scenarios, e.g. earthquakes [4]. In the TRADR (Long-Term Human-Robot Teaming for Disaster Response) project 3D point clouds are generated from camera images and combined with 3D point clouds from laser scanners carried by ground robots [5]. ...
... To address these two problems, the exploitation of the UAVs' processing of captured images and their knowledge of the covered area. The work in [7] provides a 3D modeling system based on UAVs to help rescue teams to detect the victims. However, this system can be easily affected by the weather or other factors distorting the captured images. ...
Article
Full-text available
The response time to emergency situations in urban areas is considered as a crucial key in limiting material damage or even saving human lives. Thanks to their "bird's eye view" and their flexible mobility, Unmanned Aerial Vehicles (UAVs) can be a promising candidate for several vital applications. Under these perspectives, we investigate the use of communicating UAVs to detect any incident on the road, provide rescue teams with their exact locations, and plot the fastest path to intervene, while considering the constraints of the roads. To efficiently inform the rescue services, a robust routing scheme is introduced to ensure a high level of communication stability based on an efficient backbone, while considering both the high mobility and the restricted energy capacity of UAVs. This allows both predicting any routing path breakage prior to its occurrence, and carrying out a balanced energy consumption among UAVs. To ensure a rapid intervention by rescue teams, UAVs communicate in an ad hoc fashion with existing vehicles on the ground to estimate the fluidity of the roads. Our system is implemented and evaluated through a series of experiments. The reported results show that each part of the system reliably succeeds in achieving its planned objective.
... In Urban Search and Rescue (USaR) operations have limited time to rescue the trapped victims. For this kind of rapid response situations, the creation of 3D models of disaster scenes using Drone imagery can assist USaR team and increase survival rates [3]. In [4], they use augmented reality system with robot rescuer for Urban Search and Rescue operations. ...
Conference Paper
Natural disasters are increasing day by day as the climate crisis becomes more serious. In a disastrous situation, search and rescue missions are very risky, time-consuming, and resource constraints. A hybrid drone-assisted mixed reality-based system (termed as "MR-Drone") is proposed for conducting search and rescue operations very quickly and efficiently in a disastrous situation like an earthquake, floods, fire breakout, etc. In our proposed system, we use drone’s real-time steaming for generating a 3D map of the target area and this map can be shown in mixed reality device like HoloLens for monitoring the rescue mission. Multiple users can monitor the emergency together in a mixed reality environment using MR-Drone thus reduce the rescue time.
... Via comparison to the achievements of similar technologies in other domains, the hypothesis formulated for this research assumes that three-dimensional and orthophoto mapping will improve aerial assessments of flood-related needs and damages [37,39,40]. The hypothesis is substantiated by the fact that a three-dimensional model enables the user to see a building's elevations from different angles, which is not possible from a two-dimensional perspective. ...
Article
Full-text available
The aim of this research is to provide disaster managers with the results of testing three-dimensional modeling and orthophoto mapping, so as to add value to aerial assessments of flood-related needs and damages. The relevant testing of solutions concerning the real needs of disaster managers is an essential part of the pre-disaster phase. As such, providing evidence-based results of the solutions’ performance is critical with regard to purchasing them and their successful implementation for disaster management purposes. Since disaster response is mostly realized in complex and dynamic, rather than repetitive, environments, it requires pertinent testing methods. A quasi-experimental approach, applied in a form of a full-scale trial meets disaster manager’s requirements as well as addressing limitations resulting from the disaster environment’s characteristics. Three-dimensional modeling and orthophoto mapping have already proven their potential in many professional fields; however, they have not yet been broadly tested for disaster response purposes. Therefore, the objective here is to verify the technologies regarding their applicability in aerial reconnaissance in sudden-onset disasters. The hypothesis assumes that they will improve the efficiency (e.g., time) and effectiveness (e.g., accuracy of revealed data) of this process. The research verifies that the technologies have a potential to facilitate disaster managers with more precise damage assessment; however, their effectivity was less than expected in terms of needs reconnaissance. Secondly, the overall assessment process is heavily burdened by data processing time, however, the technologies allow a reduction of analytical work.
... Significant progress has been made in the estimation of damage sustained from a disaster using satellite remote sensing 7,19 ; however, the details of damage to buildings typically remain unclear because of the scarcity of information about the condition of building walls. For this reason, increasing attention is being paid to structure-from-motion (SfM) and multiview stereo (MVS) photographic surveys conducted using UAVs 11,21,22 . ...
Preprint
Full-text available
For effective disaster relief decision-making, responders require extensive and rapid information on the damage situation in affected areas. Areas with unknown conditions pose a high risk of injury, and working on the ground limits the coverage and speed of information acquisition. An alternative is to exploit aerial observations and, in particular, unmanned aerial vehicles (UAVs). UAVs can be rapidly deployed to access remote areas without risking survey teams. Moreover, large-scale disasters impact wide areas, and multiple UAVs are needed to increase coverage without compromising resolution or speed. Of particular importance for evaluation are assets such as hospitals, shelters and essential infrastructures. UAVs can survey such structures to construct three-dimensional (3D) models for inspection.A structure-from-motion (SfM) survey generates 3D models from multiple images. However, most path planning algorithms for SfM focus on points of interest taken from an individual UAV and consider a single structure. Here, we propose a path design method for multi-UAV SfM surveys. By designing flight paths with sufficient overlap and sidelap ratios for all faces of the surveyed objects, more precise 3D models can be constructed than with conventional methods. The fuzzy C-means method is adopted to reduce the UAV flight loads to a uniform minimum to ensure full battery utilization.
... ορθοεικόνες ή αληθείς ορθοεικόνες) ή την περαιτέρω αξιοποίησή τους σε άλλες εφαρμογές όπως ταξινόμησης, εντοπισμού μεταβολών κ.ά. Στην πράξη οι διαδικασίες DIM έχουν εφαρμοστεί με επιτυχία σε πολλές εφαρμογές, όπως γεωμετρική τεκμηρίωση μνημείων [120][121][122][123][124], αποτύπωση και 3Δ ανακατασκευή κρίσιμων υποδομών [125], αντιμετώπιση καταστάσεων έκτακτης ανάγκης [126][127], 3Δ ανακατασκευή πόλεων [128][129][130][131] κ.ά. Τα τελευταία χρόνια η τεχνική DIM συνδυάζεται με τη μέθοδο υπολογισμού δομής από κίνηση (Structure from Motion-SfM) για την αυτοβαθμονόμηση της χρησιμοποιούμενης μηχανής και τον αυτόματο υπολογισμό των εξωτερικών προσανατολισμών των εικόνων [132][133][134]. ...
... 40 A quantitative comparative study regarding reconstruction accuracy and computational cost among these commercial, free, and/or open-source software tools is shown in one of our recent works. 41 The results show that the minimum computational time required for the acquisition of a coarse 3-D model is lower in PhotoScan (commercial software), but the generated model has significantly lower density than the one created by the other toolkits due to the smoothing effect. On the contrary, PhotoScan models may contain some holes, which are not present in other software tools such as VisualSFM and MicMac. ...
Article
Full-text available
A system designed and developed for the three-dimensional (3-D) reconstruction of cultural heritage (CH) assets is presented. Two basic approaches are presented. The first one, resulting in an "approximate" 3-D model, uses images retrieved in online multimedia collections; it employs a clustering-based technique to perform content-based filtering and eliminate outliers that significantly reduce the performance of 3-D reconstruction frameworks. The second one is based on input image data acquired through terrestrial laser scanning, as well as close range and airborne photogrammetry; it follows a sophisticated multistep strategy, which leads to a "precise" 3-D model. Furthermore, the concept of change history maps is proposed to address the computational limitations involved in four-dimensional (4-D) modeling, i.e., capturing 3-D models of a CH landmark or site at different time instances. The system also comprises a presentation viewer, which manages the display of the multifaceted CH content collected and created. The described methods have been successfully applied and evaluated in challenging real-world scenarios, including the 4-D reconstruction of the historic Market Square of the German city of Calw in the context of the 4-D-CH-World EU project.
... Quadcopter UAVs are being used in many practical applications such as data collection [1], [2], crop spraying pesticide in agriculture [3], [4], search and rescue tasks [5], [6], payload carrying [7], [8], disaster management [9], [10], and military operations [11], [12]. As the technological development of computers, electronics, mechanics, control theories and communication technologies, the UAVs will become more and more maneuverable and smarter. ...
Article
Full-text available
An effective robust adaptive sliding mode controller to reject time-varying disturbances and uncertainties for improving the tracking performance of attitude and altitude control of quadcopter Unmanned Aerial Vehicles (UAVs) is presented in this paper. The flight controller provides a strict robustness, fast response, and rapid adaptation for the vehicle under the undesirable effect of perturbations. The proposed controller design is based on the super-twisting algorithm for excellently eliminating the chattering effect. In addition, an efficient adaptive law is achieved from the Lyapunov stability to ensure that the controller gains can be automatically adjusted to compensate for any effect of perturbations. Thus, the proposed algorithm greatly guarantees the robust control for the vehicles even without knowing the upper bound of a time-varying disturbance. A numerical simulation was executed and compared with recently alternative methods. A superior stability and excellent tracking performance of the attitude and altitude control of a quadcopter UAV in the simulation results demonstrated the effectiveness of the proposed method.
Article
In this paper, we investigate an unmanned aerial vehicle (UAV)-enabled full-duplex relaying system. By assuming that the UAV follows a circular trajectory and applies decode-and-forward relaying strategy, we study the joint design of beamforming and power allocation to maximize the instantaneous data rate, under both the individual and the sum power constraints over the source and relay nodes. As the problem is non-convex, we propose an efficient sub-optimal solution based on block-coordinate descent method by decomposing the problem into two sub-problems: a beamforming optimization sub-problem with given power allocation and a power allocation sub-problem with fixed beamforming. For the beamforming design sub-problem, the optimal solution is obtained based on the semi-definite relaxation technique. For low-complexity implementation, we also propose two sub-optimal designs based on transmit zero-forcing/ maximum ratio combing and maximum ratio transmission/ receive zero-forcing. For the power allocation sub-problem, the optimal solution is obtained in closed form. Then, closed form cumulative distribution function and outage probability expressions for sub-optimal beamforming with both uniform power allocation and optimal power allocation are derived. In addition, simple and informative high signal-to-noise ratio (SNR) approximations for outage probability expressions are presented to gain insights, in terms of diversity order, average SNR and relations between outage probability and UAV position. We show that the proposed power allocation does not affect the diversity order but it increases the average SNR, and hence reduces the outage probability in the high SNR regime. The optimal flying altitude that minimizes the average outage probability can be obtained via a one-dimensional search. The fixed wing UAV based mobile relay following a circular trajectory is much more energy efficient than the rotary wing UAV based static relay hovering at a fixed location.
Chapter
Although high quality 3D representations of important cultural landmarks can be obtained via sophisticated photogrammetric techniques, their demands in terms of resources and expertise pose limitations on the scale at which such approaches are used. In parallel, the proliferation of multimedia content posted online creates new possibilities in terms of the ways that such rich content can be leveraged, but only after addressing the significant challenges associated with this content, including its massive volume, unstructured nature, and noise. In this chapter, two strategies are proposed for using multimedia content for 3D reconstruction: an image-based approach that employs clustering techniques to eliminate outliers and a video-based approach that extracts key frames via a summarization technique. In both cases, the reduced and outlier-free image data set is used as input to a structure from motion framework for 3D reconstruction. The presented techniques are evaluated on the basis of the reconstruction of two world-class cultural heritage monuments.
Article
Rapid assessment of building damages due to natural disasters is a critical element in disaster management. Although airborne-based remote sensing techniques have been successfully applied in many postdisaster scenarios, automated building component-level damage assessment with terrestrial/mobile LiDAR data is still challenging to achieve due to lack of reliable segmentation methods for damaged buildings. In this research, a novel building segmentation and damage detection approach is proposed to realize automated component-level damage assessment for major building envelop elements including wall, roof, balcony, column, and handrail. The proposed approach first conducts semantic segmentation of building point cloud data using a rule-based approach. The detected building components are then evaluated to determine if the components are damaged. The authors applied this method on a mobile LiDAR data set collected after Hurricane Sandy. The results demonstrate that the approach is capable of achieving 96% and 86% parsing accuracy for wall façades and roof facets, and obtain 82% and 78% of detection accuracy for damaged walls and roof facets.
Article
Damage due to age or accumulated effects from hazards on existing structures poses a worldwide problem. In order to evaluate the current status of aging, deteriorating and damaged structures, it is vital to accurately assess the present conditions. It is possible to capture the in situ condition of structures by using laser scanners that create dense three-dimensional point clouds. This paper investigates the use of high resolution three-dimensional terrestrial laser scans coupled with images to capture geometric range data of complex scenes for surface damage detection and quantification. Although using images with varying resolution to detect cracks is an extensively researched topic, damage detection using laser scanners with and without color images is a new research area that holds many opportunities for enhancing the current practice of visual inspections. Thus, this paper mainly focuses on combining the best features of laser scans and images to create an automatic and effective surface damage detection method, which will reduce the need for skilled labor during visual inspections and allow automatic documentation of related information. A novel surface normal-based damage detection and quantification method that uses the local surface properties extracted from laser scanner data along with color information is presented. Color data provides information in the fourth dimension that enables detecting damage types such as cracks, corrosion, and related surface defects that are generally difficult to identify using only laser scanner.
Conference Paper
Recent development in micro-electronics technologies as well as the computer vision techniques increased demand to use Unmanned Aerial Vehicles (UAVs) in several industrial and civil applications. This paper proposed a vision based system, that is used in UAVs for search, rescue and transportation purposes. The proposed system is divided into two main parts: Vision-based object detection and classification, in which, a Kinect V2 sensor is used; to extract the objects from the ground plane, and estimate the distance to the UAV. In addition, Support Vector Machine (SVM) human detector based on Histograms of Oriented Gradients (HOG) features is applied to classify the human bodies from the all detected objects. Secondly, a semi-autonomous reactive control for visual servoing system is implemented; to control the position and the velocity of the UAV for performing safe approaching maneuvers to the detected objects. The proposed system has been validated by performing several real flights, and the obtained results show the high robustness and accuracy of the system.
Article
Oblique photogrammetry with multiple cameras onboard unmanned aerial vehicle (UAV) has been widely applied in the construction of photorealistic three-dimensional (3-D) urban models, but how to obtain the optimal building facade texture images (BFTIs) from the abundant oblique images has been a challenging problem. This article presents an optimization method for selection of BFTIs from the image flows acquired by five oblique cameras onboard UAV. The proposed method uses multiobjective functions, which consists of the smallest occlusion of the BFTI and the largest façade texture area, to select the optimal BFTIs. Geometric correction, color equalization, and texture repairment are also considered for correction of BFTI’s distortions, uneven color, and occlusion by other objects such as trees. Visual C++ and OpenGL under the Windows Operating System are used to implement the proposed methods and algorithms. The proposed method is verified using 49 800 oblique images collected by five cameras onboard the Matrice 600 Pro (M600 Pro) UAV system over Dongguan Street, in the City of Ji’nan, Shandong, China. To restore the partially occluded textures, different thresholds and different sizes of windows are experimented, and a template window of $200\times200$ pixels <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> is recommended. With the proposed method, 2740 BFTIs are extracted from 49 800 oblique images. As compared with the Pix4Dmapper and Smart 3-D method, it can be concluded that the optimal texture can be selected from the image flow acquired by multiple cameras onboard UAV and the approximately 95% memory occupied by the original BFTIs is reduced.
Conference Paper
The creation of 3D models of Cultural Heritage (CH) sites that have undergone a severe disaster due to a catastrophic incident (e.g., earthquake, explosion, terrorist attack) is of great importance for several use cases. Different actors, like Urban Search and Rescue crews, structural, civil and surveying engineers, people in charge of restoration plans, archaeologists, architects, reporters, television presenters and computer engineers, may exploit the 3D information in a different way. Hence, each of them needs models of different scales/levels of detail and under different time constraints. In this paper the need for multi-scale 3D models of severely damaged or collapsed CH sites is addressed and various use cases are discussed. Also, image-based workflows are established for creating multi-scale 3D products via UAV images of a damaged church due to an earthquake. The models of different scales require very different amounts of time for their generation and may be used for search and rescue, damage assessment, geometric documentation, planning of repair works and simple visualization.
Article
We present a new active imaging architecture that enables rapid spectropolarimetric imaging in a compact system. The transmitter laser produces a synchronous cascade of closely spaced wavelengths, each of which is modulated with a unique amplitude pattern. Temporal multiplexing of the optical return signals is used to reduce system cost, size, weight, and power. This architecture enables a single laser and a single detector for all wavelengths and polarization states. This in turn enables pixel-by-pixel scene characterization, which could be used for target identification in future work. The basic hardware and software architecture will be presented in addition to technologies that are being investigated to develop this system architecture. We will introduce analytical expressions for the temporally multiplexed transmitted and detected signals based on the proposed hardware configuration. We then derive optimal and computationally efficient algorithms for the estimation of the overall range per pixel and the reflectivity per wavelength per pixel, as well as the Cramer-Rao lower bound on estimator variance. The bound and simulated performance yield guidelines for the system parameters required to achieve a desired level of fidelity of the spectral and polarimetric reflectivity. The proposed system is validated with laboratory data and explored via simulations.
Conference Paper
Full-text available
Unmanned aerial vehicles (UAVs) are becoming very popular recently due to their accessibility to targets, promptness of dispatch, and high-spatial resolution of acquired images. In this study, the authors tried to obtain digital images from air and ground for a turned-down and washed-away building in Onagawa Town, Miyagi Prefecture, Japan due to the 2011 Tohoku earthquake and tsunami. A small multirotor-type UAV was used in this aerial survey. Then 3D models were constructed in computer using the acquired images based on the Structure-from-Motion (SfM) technique. The most accurate model was made when both the aerial and ground images were used for 3D modelling. As the second example, an aerial video taken from UAVs was used to model affected districts in Kathmandu, Nepal, due the Mw7.8 Gorkha earthquake on April 25, 2015. The video was taken by the Canadian humanitarian organization and released on Internet. The 3D models were developed from the video footage, and they show the details of damage situation in the urban districts.
Article
Full-text available
Exploiting the decrease of costs related to UAV technology, the humanitarian community started piloting the use of similar systems in humanitarian crises several years ago in different application fields, i.e., disaster mapping and information gathering, community capacity building, logistics and even transportation of goods. Part of the author's group, composed of researchers in the field of applied geomatics, has been piloting the use of UAVs since 2006, with a specific focus on disaster management application. In the framework of such activities, a UAV deployment exercise was jointly organized with the Regional Civil Protection authority, mainly aimed at assessing the operational procedures to deploy UAVs for mapping purposes and the usability of the acquired data in an emergency response context. In the paper the technical features of the UAV platforms will be described, comparing the main advantages/disadvantages of fixed-wing versus rotor platforms. The main phases of the adopted operational procedure will be discussed and assessed especially in terms of time required to carry out each step, highlighting potential bottlenecks and in view of the national regulation framework, which is rapidly evolving. Different methodologies for the processing of the acquired data will be described and discussed, evaluating the fitness for emergency response applications.
Article
Full-text available
Structural damage assessment is critical after disasters but remains a challenge. Many studies have explored the potential of remote sensing data, but limitations of vertical data persist. Oblique imagery has been identified as more useful, though the multi-angle imagery also adds a new dimension of complexity. This paper addresses damage assessment based on multi-perspective, overlapping, very high resolution oblique images obtained with unmanned aerial vehicles (UAVs). 3-D point-cloud assessment for the entire building is combined with detailed object-based image analysis (OBIA) of façades and roofs. This research focuses not on automatic damage assessment, but on creating a methodology that supports the often ambiguous classification of intermediate damage levels, aiming at producing comprehensive per-building damage scores. We identify completely damaged structures in the 3-D point cloud, and for all other cases provide the OBIA-based damage indicators to be used as auxiliary information by damage analysts. The results demonstrate the usability of the 3-D point-cloud data to identify major damage features. Also the UAV-derived and OBIA-processed oblique images are shown to be a suitable basis for the identification of detailed damage features on façades and roofs. Finally, we also demonstrate the possibility of aggregating the multi-perspective damage information at building level.
Article
Full-text available
The concept of employing unmanned aerial vehicles (UAVs) to acquire imagery for disaster research and management has progressed into actual implementation in recent years. UAV usage in disaster assessment, response and management is an active area of research. UAVs have been utilized following ecological, meteorological, geological, hydrological and human-induced disasters. The flexibility, safety, ease of operation, and relatively low-cost of ownership and operation facilitate UAV implementation in disaster situations. This paper provides a review of recent utilization of UAVs for imagery collection for disaster monitoring and management.
Conference Paper
Full-text available
Google Maps and Google Earth were milestones in the evolution of Geographic Information Systems (GIS). A new era of 3D virtual geo-referenced worlds began. The need of exploration and creation personalized virtual tours is imperative. This work implements a best viewing algorithm for presenting a 3D model by considering both semantic and geometric features of the 3D model. The camera trajectory is estimated by respecting the user's preferences using a personalized entropy-based measurement. Moreover the camera speed is adjusted properly based on the complexity of the projected view of the model.
Article
Full-text available
Unmanned aerial vehicle (UAV) platforms are nowadays a valuable source of data for inspection, surveillance, mapping, and 3D modeling issues. As UAVs can be considered as a low-cost alternative to the classical manned aerial photogrammetry, new applications in the short- and close-range domain are introduced. Rotary or fixed-wing UAVs, capable of performing the photogrammetric data acquisition with amateur or SLR digital cameras, can fly in manual, semiautomated, and autonomous modes. Following a typical photogrammetric workflow, 3D results like digital surface or terrain models, contours, textured 3D models, vector information, etc. can be produced, even on large areas. The paper reports the state of the art of UAV for geomatics applications, giving an overview of different UAV platforms, applications, and case studies, showing also the latest developments of UAV image processing. New perspectives are also addressed.
Article
Full-text available
Conference Paper
Full-text available
Small-scale unmanned aerial vehicles (UAVs) have recently gained a lot of interest for various applications such as surveillance, environmental monitoring and emergency response operations. These battery-powered and easy-to-steer aerial robots are equipped with cameras and can promptly acquire aerial images. In this paper we describe our system of multiple UAVs that are able to fly autonomously over an area of interest and generate an overview image of that area. Intuitive and easy user interaction is a key property of our system: The user specifies the area of interest on an electronic map. The flight routes for the UAVs are automatically computed from this specification and the generated overview is presented in a Google-Earth like user interface. We have tested and demonstrated our multi-UAV system on a large fire service drill. Our system provided a high-resolution overview image of the 5.5 ha large test site with regular updates, proved that it is easy to handle, fast to deploy, and useful for the firefighters.
Conference Paper
Full-text available
In this paper, we present a novel scale- and rotation-invariant interest point detector and descriptor, coined SURF (Speeded Up Robust Features). It approximates or even outperforms previously proposed schemes with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared much faster. This is achieved by relying on integral images for image convolutions; by building on the strengths of the leading existing detectors and descriptors (in casu, using a Hessian matrix-based measure for the detector, and a distribution-based descriptor); and by simplifying these methods to the essential. This leads to a combination of novel detection, description, and matching steps. The paper presents experimental results on a standard evaluation set, as well as on imagery obtained in the context of a real-life object recognition application. Both show SURF’s strong performance.
Article
This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.
Article
Unmanned aerial vehicle systems are currently in limited use for public service missions worldwide. Development of civil unmanned technology in the United States currently lags behind military unmanned technology development in part because of unresolved regulatory and technological issues. Civil unmanned aerial vehicle systems have potential to augment disaster relief and emergency response efforts. Optimal design of aerial systems for such applications will lead to unmanned vehicles which provide maximum potentiality for relief and emergency response while accounting for public safety concerns and regulatory requirements. A case study is presented that demonstrates application of a civil unmanned system to a disaster relief mission with the intent on saving lives. The concept utilizes unmanned aircraft to obtain advanced warning and damage assessments for tornados and severe thunderstorms. Overview of a tornado watch mission architecture as well as commentary on risk, cost, need for, and design tradeoffs for unmanned aerial systems are provided.
Conference Paper
Various natural and human-made events can occur in urban settings resulting in buildings collapsing and trapping victims. The task of a structural engineer is to survey the resulting rubble to assess its safety and arrange for structural stabilization, where necessary. Urban Search and Rescue (USAR) operations can then begin to locate and rescue people. Our previous work reported the use of an Unmanned Aerial Vehicle (UAV) equipped with a RGB-Depth sensor to build 3D point cloud models of disaster scenes. In this paper we extend this work by converting the point clouds into 3D models and importing them into a state-of-the-art game engine. We present a method to use these models to allow first responders to interact with the simulated rubble environment in real-time, without risk to human life. Experiments are conducted measuring traversal time both in the real world environment and using the simulation. We argue that this work will improve the safety of workers and allow a better understanding of extremely dangerous environments without unnecessary exposure during disaster response planning.
Chapter
This research project evaluates the utilization of a low-cost Unmanned Aerial Vehicle (UAV) digital imaging platform developed in Manitoba, Canada for emergency response situations. Such a platform allows for the timely acquisition of high resolution imagery during emergency situations by personnel with relatively limited UAV flight training. Although military use of UAVs has been around since the First World War, their relatively high expense and operational requirements have limited their civilian use. The UAV platform, as tested in this project, is a very valuable and useful tool for disaster response situations, allowing for the easy and timely acquisition of high resolution imagery, by meeting the following criteria: fly fully autonomously with little pilot experience assembled and ready for launching in less that one half hour acquire high resolution imagery (greater than 25 centimetre resolution) transported in a midsized car relatively low cost (\$8,000 US) A 5.0 mega pixel digital camera enables the UAV platform to acquire colour imagery with a 22 cm spatial resolution from a height of 2100 feet. With a stable and reliable UAV platform already developed, future research will focus on the development and integration of other imaging and sampling techniques such as thermal technologies and air sampling abilities for smoke plume analysis.
Conference Paper
The use of unmanned aerial vehicles (UAVs) has the potential to significantly improve the situation awareness of emergency first responders working at urban disaster sites. Having the characteristics of being small, light-weight and quickly deployable, UAVs offer the ability to fly over an urban disaster and provide intelligence to Urban Search and Rescue (USAR) task force efforts before precious operational resources are committed on the ground. In this paper we discuss our experience with using a small UAV to perform the task of creating a 3D model of a rubble pile's surface using commercial off the shelf (COTS) components in the form of an available UAV equipped with a modified video game sensing package.
Article
Efficient management of natural disasters impose great research challenges to the current environmental crisis management systems in terms of both architecture and services. This is mainly due to the fact that a large amount of geospatial content is usually distributed, non-compliant to standards, and needs to be transmitted under a QoS guaranteed framework to support effective decision making either in case of an emergency or in advance planning. Incorporating real time capabilities in Web services, both in terms of dynamic configuration and service selection, is an open research agenda. The things get worst in geospatial context due to the huge amount of data transmitted from distributed sensors under heterogeneous platforms, making the need of synchronization an important issue. In this paper, we propose a flexible service oriented architecture for planning and decision support in environmental crisis management. The suggested architecture uses real time geospatial data sets and 3D presentation tools, integrated with added-value services, such as simulation models for assisting decision making in case of emergency. The proposed architectural framework goes beyond integration and presentation of static spatial data, to include real time middleware that is responsible for selecting the most appropriate method of the available geospatial content and service in order to satisfy the QoS requirements of users and/or application. A case study of a complete, real world implementation of the suggested framework dealing with forest fire crisis management system is also presented.
Article
This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.
INACHUS: Integrated Wide Area Situation Awareness and Survivor Localisation in Search and Rescue Operations
• G Athanasiou
G. Athanasiou et al., "INACHUS: Integrated Wide Area Situation Awareness and Survivor Localisation in Search and Rescue Operations," GiT4NDM 2015, Al Ain, UAE, 2015.