No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
A Vision System for Landing an Unmanned Aerial Vehicle
Abstract
We present the design and implementation of a real-time computer vision system for a rotorcraft unmanned aerial vehicle to land onto a known landing target. This vision system consists of customized software and off-the-shelf hardware which perform image processing, segmentation, feature point extraction, camera pan/tilt control, and motion estimation. We introduce the design of a landing target which significantly simplifies the computer vision tasks such as corner detection and correspondence matching. Customized algorithms are developed to allow for realtime computation at a frame rate of 30 Hz. Such algorithms include certain linear and nonlinear optimization schemes for model-based camera pose estimation. We present results from an actual flight test which show the vision-based state estimates are accurate to within 5 cm in each axis of translation, and 5 degrees in each axis of rotation, making vision a viable sensor to be placed in the control loop of a hierarchical flight management system.
... En los vuelos a baja altura, además, presenta serias dificultades ya que muy comúnmente solo están disponibles vistas parciales o sesgadas de la figura. Sharp et al. [43] también se decantaron por usar cuadrados en blanco y negro utilizando descriptores de puntos de interés, tales como el operador de Harris [44] para la detección de esquinas, figura 2-2. No obstante, en este diseño aún persisten las dificultades y limitaciones descritas previamente en lo que respecta a la ausencia de partes relevantes de la imagen como consecuencia de los brillos y las distorsiones. ...
... Sin embargo, el problema relativo a la ausencia de determinadas partes de la figura todavía persiste. [43] (c) Saripalli et al. [48,49] usaron también una figura en forma de H impresa sobre la superficie de la plataforma de aterrizaje, figura 2-3. El método está basado en las siguientes técnicas de procesamiento de imágenes: filtrado, umbralización, segmentación y etiquetado de componentes conexas. ...
... En otros, como en [54], suponen que el UAV se sitúa en una posición cenital con respecto a la plataforma durante la fase de aterrizaje, ignorando así el ángulo de inclinación. Los bordes y los puntos de interés también han sido considerados en otras propuestas, si bien es necesario tener en consideración, la alta sensibilidad al ruido que este tipo de estructuras conllevan [41,43,45,53]. ...
The full-text is available at: http://e-spacio.uned.es/fez/view/tesisuned:ED-Pg-IngSisCon-Jagarcia
ABSTRACT:
The use of Unmanned Aerial Vehicles (UAVs) is growing significantly for many and varied purposes. Whilst advanced UAVs use auto-guided landing supported by GPS devices, this is a precise critical maneuver requiring additional support. Many UAVs are equipped with vision-based cameras operating in the visible spectral range, and this offers an excellent tool to aid in automatic landing with the required precision.
Careful consideration is needed for enhanced security and prevention of possible issues derived from GPS problems, such as signal loss, support for indoor environments, accuracy issues, etc. This contributes greatly to the overall success of autonomous flight. In this sense, to address this problem, this thesis proposes as its main contribution, two different methods that are based on the recognition of a landing platform in color images by using Computer Vision techniques.
On the one hand, it has been developed a novelty automatic Expert System [1] that recognizes the platform in an image. The platform contains a black geometric figure drawn on a white background [2], used in the SALACOM project [3] funded by the Ministry of Economy and Competitiveness. The designed system incorporates the human expert knowledge using a sequence of high-level vision and image processing techniques. A smart application for the recognition of the landing platform has been developed as a result. It is reliable enough for its incorporation in a UAV equipped with a color camera, being especially robust for the rotary wing UAVs.
During the experimental phase the inherent problematic for this type of designs has been addressed, derived mainly from two adverse circumstances: 1) the reflections originated by the natural lighting condition that causes the disappearance of certain parts of the figure, as a consequence of the saturation of the intensity levels, causing black parts to appear white as a result of the reflection; 2) the necessity of interrelating regions, by extracting relative measures, causes the distortions of the figures, for example, circles perceived as ellipses, or the same disappearance of regions due to the above-mentioned saturation, origin relative measurements to be insufficient for recognition.
In this sense and to overcome this drawback, a new strategy based on the artificial vision field has been proposed. Both the color image processing and pattern recognition techniques has been combined together with a Euclidean Distance Geometric Analysis (EDGA). This leads to the design of a new specific platform for identification, which allows solving the problems above. The design of this new platform is based on: 1) color information in the visible spectral range, with unique colors for each figure, with little or no presence in natural and urban environments and 2) geometric figures, which allow partial individualized perception. This method calculates the recognition degree of the platform from the overall detected figures in conjunction with the geometric relationships between these figures, in such a way, that the 2-groups of regions allow a decision to be performed about recognition.
The experimental evidence carried out allows the validation of the two developed recognition procedures, taking into account that the problem identified during the first approach has been fixed in the second one.
... These include stored video processing [1,2], real-time video [1,3], automatic inspection [4], computer-based diagnostic [5,6], human surveillance [7][8][9], etc. Some frequently used applications that require object tracking are feature-based image registration [10], augmented reality conferencing [11], unmanned aerial vehicle [12,13], etc. Object tracking is defined as a problem of estimating the trajectory of an object in the image plane as it moves around a scene. Object tracking methods are primarily dependent upon object attributes such as, representation, features, motion, appearance, shape, and environment in which tracking is performed. ...
... Where C 1 and C 2 represent accumulated values of (4) and (6), respectively. Mean shift weights (12) are utilized in mean shift vector computation. ...
... Here, BRAM-1 and BRAM-2 are used to compute the kernel-smoothed local histogram (KSLH) as per (11,12). The architecture shown in Fig. 7, works in 5 different stages as described below: Stage 0: BRAM-1, BRAM-2, and register (REG) are initialized to zeros. ...
Moving object tracking is a computation-intensive operation that requires accelerating hardware solution. In this work, a high-performance design for mean shift based moving object tracking algorithm and its FPGA implementation is done. Here, associated circuits are utilized as intellectual-property cores to implement an embedded system-on-a-chip (SoC) framework for real-time moving object tracking application. Real-time video with resolution at 60 frames per second is captured and buffered in SDRAM, and processing is performed on the temporal frames. In the implemented FPGA-SoC framework, PowerPC processor embedded inside the FPGA device is used for the platform configuration, IPs control, and running the application program. The design require 30.08% slices, 35.14% BRAMs, and 43.75% DSP48E slices of Xilinx Virtex-5 xc5vfx70t FPGA device on the ML-507 platform. The computed power is 48.3 mW.
... The FPS can reach 53.7, which meets the requirements of realtime detection. Second, in comparison with previous methods [19][20][21], our method possesses much higher actual landing accuracy. Third, we introduce deep-learning methods into the UAV landing process, which are characterized by powerful feature extraction and characterization capabilities. ...
... UAV visual positioning landing mainly relies on image sensors and uses image processing technology to achieve an accurate landing, and this is a research hotspot for scholars in China and abroad. Sharp et al. [19] proposed a precision landing system for the autonomous landing of multi-rotor UAVs. This system uses a large square and five small squares as landmark patterns. ...
The application of UAVs is becoming increasingly extensive. However, high-precision autonomous landing is still a major industry difficulty. The current algorithm is not well-adapted to light changes, scale transformations, complex backgrounds, etc. To address the above difficulties, a deep learning method was here introduced into target detection and an attention mechanism was incorporated into YOLOX; thus, a UAV positioning algorithm called attention-based YOLOX (A-YOLOX) is proposed. Firstly, a novel visual positioning pattern was designed to facilitate the algorithm’s use for detection and localization; then, a UAV visual positioning database (UAV-VPD) was built through actual data collection and data augmentation and the A-YOLOX model detector developed; finally, corresponding high- and low-altitude visual positioning algorithms were designed for high- and low-altitude positioning logics. The experimental results in the actual environment showed that the AP50 of the proposed algorithm could reach 95.5%, the detection speed was 53.7 frames per second, and the actual landing error was within 5 cm, which meets the practical application requirements for automatic UAV landing.
... The recognition algorithm used the well-known faster regions with convolution (Faster R-CNN) technique to recognize the figure based on its feature map. Sharp et al. [42] also used black-white squares with the aim of applying corner detection as a descriptor, encountering identical difficulties and limitations. The use of complex figures, based on curvatures, causes distortions because of image perspective projection. ...
... Other methods [1,30,35,40,[52][53][54][55]58] used (intertwined or interspersed) figures or complex figures based on circles [59] and/or ellipses to address the missing parts problem, some of which [1] allow a certain plane distortion, but others, like [35], assume the UAV is parallel to the ground during the landing phase, ignoring the angle of inclination. Edges and interest points are also considered with high noise sensitivity [42,43,45,49]. Table 1 classifies and summarizes the previously described methods. ...
Unmanned aerial vehicles (UAVs) are excellent tools with extensive demand. During the last phase of landing, they require additional support to that of GPS. This can be achieved through the UAV’s perception system based on its on-board camera and intelligence, and with which decisions can be made as to how to land on a platform (target). A cognitive computation approach is proposed to recognize this target that has been specifically designed to translate human reasoning into computational procedures by computing two probabilities of detection which are combined considering the fuzzy set theory for proper decision-making. The platform design is based on: (1) spectral information in the visible range which are uncommon colors in the UAV’s operating environments (indoors and outdoors) and (2) specific figures in the foreground, which allow partial perception of each figure. We exploit color image properties from specific-colored figures embedded on the platform and which are identified by applying image processing and pattern recognition techniques, including Euclidean Distance Smart Geometric Analysis, to identify the platform in a very efficient and reliable manner. The test strategy uses 800 images captured with a smartphone onboard a quad-rotor UAV. The results verify the proposed method outperforms existing strategies, especially those that do not use color information. Platform recognition is also possible even with only a partial view of the target, due to image capture under adverse conditions. This demonstrates the effectiveness and robustness of the proposed cognitive computing-based perception system.
... Object tracking is a critical component for a broad variety of applications in the area of image and video processing, including recorded real-time video [1], human monitoring [2], automated inspection [3], and computer-based diagnostics [4][5][6]. It is commonly utilized in technologies such as feature-based image registration [7], unmanned aerial vehicles [8], and augmented reality conferencing [9]. Determining the route that an item takes in the image plane as it moves through a scene is an essential aspect of the object-tracking process. ...
Computer vision has become a crucial area of research due to the increasing use of electronics, information technology, and network communication. Adding more algorithms to the process of locating text, faces, vehicles, and other moving objects can improve the effectiveness of the search when compared to a "ground truth" reference. This work aims to track a moving object as it passes through the fields of view of multiple outdoor cameras. Both the temporal difference algorithm and the fixed background algorithm have been used in the search process for objects in a video with a frame dimension of 120 by 160 pixels. The FPGA panel used in the system was the Xilinx Spartan-6 LX45T. This method can speed up the overall process because it does not require the objects to be registered first. The proposed approach is highly resistant to object orientation and robust, with an error rate of less than (0.05), resulting in the best possible results in terms of global recognition. Consequently, it attains exceptional global recognition results while maintaining its reliable performance.
... Several researchers have been focusing the development of various technologies that would ultimately enable the systematic inclusion of unmanned systems into the airspace (McAree & Chen, 2013). These technologies include sensing and data logging (Berni, Zarco-Tejada, Suárez, & Fereres, 2009;Corke et al., 2004;Everaerts et al., 2008;Sharp, Shakernia, & Sastry, 2001;Shakernia, Vidal, Sharp, Ma, & Sastry, 2002), fault tolerant flight control (Coombes, McAree, Chen, & Render, 2012;Ducard, 2009;B. S. Kim & Calise, 1997; H. J. Kim, Shim, & Sastry, 2002;, simultaneous localization and mapping (SLAM) (Hening, Ippolito, Krishnakumar, Stepanyan, & Teodorescu, 2017;Caballero, Merino, Ferruz, & Ollero, 2009;Wang et al., 2008;Gupte, Mohandas, & Conrad, 2012;J.-H. ...
This paper investigates propsective architectures for decisionmaking in unmanned aerial systems. When these unmanned vehicles operate in urban environments, there are several sources of uncertainty that affect their behavior, and decisionmaking algorithms need to be robust to account for these different sources of uncertainty. It is important to account for several risk-factors that affect the flight of these unmanned systems, and facilitate decision-making by taking into consideration these various risk-factors. In addition, there are several technical challenges related to autonomous flight of unmanned aerial systems; these challenges include sensing, obstacle detection, path planning and navigation, trajectory generation and selection, etc. Many of these activities require significant computational power and in many situations, all of these activities need to be performed in real-time. In order to efficiently integrate these activities, it is important to develop a systematic architecture that can facilitate realtime decision-making. Four prospective architectures are discussed in this paper; on one end of the spectrum, the first architecture considers all activities/computations being performed onboard the vehicle whereas on the other end of the spectrum, the fourth and final architecture considers all activities/computations being performed in the cloud, using a new service known as “Prognostics as a Service” that is being developed at NASA Ames Research Center. The four different architectures are compared, their advantages and disadvantages are explained and conclusions are presented.
... But it is not effective in the case of automatically approaching the cart stations because the laser beam is partially blocked by the legs of the cart that causes the loss of input information, thereby reducing the reliability of the positioning measurement. Using a combination system between cameras and fiducial tags is also a widely applied solution in the docking problem of mobile robots [7][8][9]. This option has the advantages of low cost, high stability and less affected by changes in lighting conditions. ...
This article presents a positioning solution for a mobile robot system, the Vibot-2 robot, in the docking problem where the robot must move backwards precisely to the charging position or the location to pick up the delivery cart (or return the cart) during the execution tasks. The localization system needs to calculate the robot's posture (including position coordinates and heading angle) in a fixed coordinate system attached to a fixed station with an accuracy within an allowable limit. The solution to build a positioning module for the Vibot-2 robot uses a stereo camera in combination with AprilTags mounted on a locator plate fixed at the station. Compared with the previous positioning solution applied to the Vibot-2 system, the proposed solution in this article changes the camera type and fiducial tag layout. Experimental results show that the new solution helps to increase the field of view of the positioning module by 30%, increase the measurement accuracy by more than 4 times in term of the average error in the horizontal axis, and at the same time, reduce the rate of measurements with errors outside the allowable limit in term of position by more than 1.5 times and in term of heading angle by more than 2 times.
... O incrível sucesso desses aparelhos em atividades como reconhecimento, inteligência, apoio aéreo, combate e controle de artilharia fez com que, posteriormente, fossem sendo descobertas importantes funções civis para os mesmos nas áreas de policiamento, patrulhamento costeiro, proteção ambiental e atividades de busca, dentre outras. Devido à grande utilidade e importância desses equipamentos, o projeto e construção dos VANT's têm constituído uma promissora área de pesquisa e desenvolvimento [3,4,5,6]. ...
The use of Unmanned Aerial Vehicles (UAV's) is becoming increasingly more intense in civilian and military areas, as they can replace humans in dangerous or routine missions. Although well developed, many of these devices still require constant monitoring and human command to perform its functions. This research aims to decrease such dependence by enabling an UAV to orient itself autonomously based on the recognition of predetermined routes. By means of incorporation of a market IP video camera and the development of a computer vision system in Python using the OpenCV library, the captured images are sent to a PC on land, which is in charge of processing. Resumo-O uso de Veículo Aéreo Não Tripulado (VANT) vem se tornando cada vez mais intenso nas áreas civis e militares, uma vez que VANTs podem substituir seres humanos em missões perigosas ou rotineiras. Embora sejam bem desenvolvidos, muitos destes aparelhos ainda requerem constante monitoramento e comando humano para o desempenho de suas funções. Essa pesquisa busca diminuir tal dependência, capacitando um VANT a orientar-se de forma autônoma com base no reconhecimento de rotas pré-determinadas. Através da incorporação de uma câmera de vídeo IP de mercado e do desenvolvimento de um sistema de visão computacional implementado em linguagem Python, usando a biblioteca OpenCV, as imagens capturadas são enviadas para um PC terrestre, que é encarregado de processá-las. I. INTRODUÇÃO Criados inicialmente para uso militar, os Veículos Aéreos Não Tripulados (VANT's), foram projetados e construídos para substituir e auxiliar o homem em missões de grande risco [1, 2]. O incrível sucesso desses aparelhos em atividades como reconhecimento, inteligência, apoio aéreo, combate e controle de artilharia fez com que, posteriormente, fossem sendo descobertas importantes funções civis para os mesmos nas áreas de policiamento, patrulhamento costeiro, proteção ambiental e atividades de busca, dentre outras. Devido à grande utilidade e importância desses equipamentos, o projeto e construção dos VANT's têm constituído uma promissora área de pesquisa e desenvolvimento [3, 4, 5, 6]. Podendo se diversificar em pequenas aeronaves de asas fixas ou rotativas, esses veículos são operados de duas formas principais: por controle remoto, isto é, pilotados por seres humanos em uma base fixa de comando; ou por autonomia própria, ou seja, voam de forma automática, através de computadores e softwares programáveis [7]. Além dos sistemas básicos de voo, o projeto dos VANT's completamente autônomos inclui a necessidade de controladores lógicos programáveis que comandem o aparelho. Tais comandos dependem da função e das missões a serem desempenhadas pelo veículo em questão. Um VANT de patrulhamento costeiro, por exemplo, tem de ser capaz de identificar veículos estranhos e de agir imediatamente, seja se movimentando em direção ao mesmo ou informando uma central operacional sobre a presença desconhecida. Por essas razões, aparelhos completamente autônomos precisam de sistemas integrados de reconhecimento, ação e controle, para que possam voar e desempenhar suas funções de forma conjunta, sem a necessidade de supervisão humana imediata. Devido ao fato de a pesquisa e o desenvolvimento desses sistemas se encontrarem levemente defasados e ainda necessitarem de esforços e aperfeiçoamento, observa-se um maior uso de VANT's controlados pelo homem. Entretanto, devido à redução de custos de operação e pessoal que podem trazer, os veículos autônomos representariam uma vantagem nessa área, sendo poucas as missões que estes não podem executar. O propósito desse trabalho é, portanto, a capacitação de um quadricóptero (um VANT de asas rotativas [8]) quanto à capacidade de se locomover de forma independente. Através de metodologias computacionais, do desenvolvimento e integração dos sistemas de visão e comando, esse projeto busca habilitar o quadricóptero a identificar trajetórias e possíveis caminhos, além de tomar ações em relação a tais detecções. II. EQUIPAMENTOS Para a execução geral deste trabalho foi utilizado um quadricóptero já desenvolvido em outra pesquisa na UFABC [8]. Trata-se de um VANT de asas rotativas, dotado de quatro motores, que funcionam sob placas Arduino (Kit Arduino Pilot Mega), responsável pelo controle do quadricóptero. Para a captura de imagens para processamento foi usada uma câmera IP modelo FOSCAM FI8909W, que fica acoplada ao quadricóptero durante seus voos. A resolução desta câmera Proceedings of XI Workshop de Visão Computacional-October 05th-07th, 2015 67
... Therefore, we can mount numerous sensors on a drone to execute different remote sensing tasks including aerial videography, photography, and surveillance. The system uses Google's MediaPipe application programming interface (API) (8)(9)(10) to capture the user's head pose and gesture and perform feature point annotation and signal conversion. These movements are used as controller signals transmitted to the drone. ...
... In order to realize the accurate recycle of UAV, it is of necessity to carry out accurate navigation and positioning of UAV. Aiming at these scenarios, Sharp et al [1] designed a special target for visual positioning, but the positioning accuracy of the target is of ordinary state. Lange et al [2] designed a rhomboid and circular target to estimate UAV pose through visual detection, thus improving the positioning accuracy. ...
A UAV (Unmanned Aerial Vehicle) landing robot system based on a six-degree-of-freedom hexapod and computer vision with ArUco marker and sparse optical flow is implemented in this paper. By using binocular camera measuring the real-time position and orientation of UAV, a dynamic target tracking method based on ArUco marker and sparse optical flow is proposed. In this method, the ArUco marker is combined with sparse optical flow to improve the accuracy of target detection and the matching error of binocular stereo matching. The binocular vision system with ArUco marker is used to measure and calculate the relative coordinates of markers in the camera coordinate system. Then the UAV contour fitting is conducted in order to obtain the real-time position and orientation of this mobile target. The effectiveness of the novel UAV landing hexapod robot system based on ArUco marker combined with sparse optical flow is verified through the UAV capture experiment.
... To avoid contact between the plane and the airbag, some works use a symbol on the ground [8,9]. For all such methods above, the objects must large enough and have a different shape from the nearby environment. ...
This research proposes a method to track a known runway image to land an unmanned aerial vehicle (UAV) automatically by finding a perspective transform between the known image and an input image in real-time. Apparently, it improves the efficiency of feature detectors in real-time, so they can better respond to perspective transformation and reduce the processing time. A UAV is an aircraft that is controlled without a human pilot on board. The flight of a UAV operates with various degrees of autonomy, either autonomously using computational-limited on-board computers or under remote control by a human operator. UAVs were originally applied for missions where human access was not readily available or where it was dangerous for humans to go. Nowadays, the most important problem in monitoring by an autopilot is that the conventional system using only the GPS sensors provides inaccurate geographical positioning. Therefore, controlling the UAV to take off from or land on a runway needs professional input which is a scarce resource. The characteristics of the newly developed method proposed in this paper are: (1) using a lightweight feature detector, such as SIFT or SURF, and (2) using the perspective transformation to reduce the effect of affine transformation that results in the feature detector becoming more tolerant to perspective transformation. In addition, the method is also capable of roughly localizing the same template in consecutive frames. Thus, it limits the calculation area that feature matching needs to work on.
... Cameras have been crucial IoT elements concerning UAVs, in which they mean application-specific or user demand preferably. Amongst diverse applications, their application uses is for independent vision procedures used for a safe docking Sharp et al. (2001) assessed the UAVs for traffic surveillance and traffic control and concluded that they are valuable and efficient for traffic monitoring. Sensors remain the essential IoT utilities of any information-gathering apparatus. ...
Evidence of the IoT is expanding the number of connected devices, including UAVs. UAVs overcome the flaws in the physical IoT infrastructure already in place. Low-altitude views are expected to be dominant swiftly in urban areas. In a short period of time, they are able to cover a large area and distribute goods and information around the globe. Additionally, how to provide a safe and secure UAV operation in high-level traffic circumstances is also a topic of investigation. When operating an UAV in a limited area, the IoLoUA system is used to maintain order. Additionally, it aids with node exploration. Basic principles that can be used to create new structural designs are analysed for both networks (IoLoUA). There has been an explanation of the IoLoUA strategy’s approach to implementation so far. Among the issues covered in this article are UAV-generated IoT data collection and delivery, security threats, and typical workflow approaches. This work presents a theoretical model of future design evolution.
... Instead of detecting an object as a whole using machine learning, computer vision techniques are applied to extract particular points of interest. To name a few, edge and line detection [19,20], corner detection [21,22], and contour detection [23] are previously applied to aircraft landing applications. ...
This study discusses a fully autonomous vertical flight aircraft ship landing procedure in presence of wind disturbances. The proposed study closely follows the established Navy helicopter ship landing procedure wherein the pilot utilizes the ship as the visual reference for long-range tracking; however, upon coming closer, the pilot follows a gyro-stabilized horizon bar installed on most Navy ships to approach and land vertically independent of deck motions. This was accomplished by developing a unique vision system and a hybrid control system validating its performance in simulations and flight tests. The vision system serves the purpose of a pilot's eye by obtaining the visual information required for a safe approach and landing. The vision system can be engaged from 250 meters away from the landing pad, initially utilizing machine learning strategies to detect the ship for long-range tracking and switches to a unique combination of classical computer vision techniques to detect the horizon bar to precisely estimate the aircraft position and orientation relative to the bar during the final approach and landing. The distance and attitude estimations were validated using the measurements from an accurate 3D motion capture system (VICON), which demonstrated sub-centimeter and sub-degree accuracy. Finally, a hybrid control system is developed to control the aircraft using the perceived visual information. The hybrid control system is a combination of a non-linear controller and a Deep Reinforcement Learning(RL) controller. The non-linear controller demonstrated robust tracking capability even in presence of estimation noise and varying time delays between successive control actions. The RL controller is developed exclusively for disturbance rejection. When conducted flight testing in presence of 5 m/s wind, the RL controller shows a 50% reduction in drift and a 3 times faster rate of correction compared to a conventional control system. The vision and hybrid control system was implemented on a quadrotor UAV and extensive flight tests were conducted to demonstrate accurate tracking in challenging conditions and safe vertical landing on a sub-scale ship platform undergoing 6 degrees of freedom deck motions.
... The quadrotor vehicle is a nonlinear system with under-actuation and strong coupling. Fig. 1 depicts an X-configuration quadrotor, where the four rotors are divided into two pairs of (1,3) and (2,4) which rotate in opposite directions in order to compensate for the interaction of the reaction torques generated. The right cooperation between the rotors speed ensures the quadrotor's basic movements in aerial space as follows: vertical motion is achieved by increasing or decreasing the speed of all rotors; the differential speeds of rotors (1,3) and (2,4) contributes into the roll and pitch motions coupled with forward motions respectively, while yaw motion is performed through the difference of counter-torques generated by each propeller. ...
This paper presents a novel method for tuning a quadrotor trajectory-tracking model predictive control (MPC) framework, based on cooperative particle swarm optimization (PSO). The overall control strategy is decomposed into two different schemes; the first scheme, which is responsible for position control, is based on MPC, whereas the second one, which regulates the attitude of the quadrotor, makes use of three standard PID controllers. To optimize the trajectory tracking ability of the quadrotor, a cooperative PSO framework is introduced for tuning the large number of parameters of diverse nature, which are employed by the different controllers. The method makes use of two distinct swarms, with the first one containing the position controller parameters and the second one the attitude controller parameters. By exchanging information, the two swarms work together in a cooperative way in order to explore more efficiently the search space and discover tuning parameters that improve the trajectory tracking performance. Simulation results on a detailed quadcopter model over trajectories with different geometric characteristics, and comparisons to other tuning approaches verified by statistical testing, illustrate the efficiency of the proposed scheme in optimizing the control parameters. The scheme's robustness is also verified by testing the performance of the resulting controller on trajectories different than the ones used for tuning it.
... Research studies have suggested several methods of detecting the landing target: an algorithm that identifies an "H-shaped" landing target using invariant moments [6], a black and white pattern consisting of six squares of different sizes [7], or a target design with four white rings [8]. Recently, researchers have began to use ArUco markers as targets [9][10][11]. ...
This paper presents a framework for performing autonomous precise landing of unmanned aerial vehicles (UAVs) on dynamic targets. The main goal of this work is to design the methodology and the controlling algorithms that will allow multi−rotor drones to perform a robust and efficient landing in dynamic conditions of changing wind, dynamic obstacles, and moving targets. Unlike existing GNSS−based vertical landing solutions, the suggested framework does not rely on global positioning and uses adaptive diagonal approaching angle visual landing. The framework was designed to work on existing camera−drone platforms, without any need for additional sensors, and it was implemented using DJI’s API on Android devices. The presented concept of visual sliding landing (VSL) was tested on a wide range of commercial drones, performing hundreds of precise and robust autonomous landings on dynamic targets, including boats, cars, RC−boats, and RC−rovers.
... Guatam specifically describes landing autonomy as being broken up into 4 tasks: pose estimation, guidance, recovery and control [1]. Pose estimation traditionally is performed using monocular vision and specially designed landing targets the UAV must land on [3][4][5][6][7][8][9][10][11][12][13], stereo vision with known targets or features [14][15][16][17], IR vision with known targets [18][19][20][21] or ground-based vision with known points on the UAV [22,23]. Guidance defines the path the UAV will take and desired controls. ...
Unmanned Aerial Vehicles (UAVs) can be further optimized as tools on-board ships with the development of lacking infrastructure, like their recovery at sea. Current technologies focus on vision-based systems with little consideration for ship motion. A novel autonomous landing technique is tested experimentally, featuring acoustic positioning to allow for landings in a wider breadth of conditions and to reduce the reliance on specially designed landing targets. A potential fields path planner is used to adapt for ship motion and provide obstacle avoidance and natural biasing away from the heaving ship deck. A sea state predictor is used to compensate for harsher sea conditions and ship motion, allowing the UAV to look for appropriate landing windows in higher sea states. Autonomous landings are demonstrated in a lab setting for sea conditions up to, and including, sea state 5. The ship motions are defined using real sea trials data from the decommissioned HMCS Nipigon.
... All the previous studies have a common process that is to estimate or track ship deck motions first, then control the aircraft attitude to match the ship motions for landing. In order to extract the ship deck motion information, specific objects or patterns are targets for detection such as H landing marking [54], T landing marking [55], points dispersed on the deck [33], lights [30], and infrared cooperated targets on a ship [56,57] as shown in Fig To detect such fiducial markings on a deck or features of a ship, several computer vision algorithms are used such as edge and line detection [58,59], corner detection [60,61], and contour detection [62]. ...
The current study focuses on developing an autonomous vertical flight aircraft ship landing system by directly automating the established Navy helicopter ship landing procedure. The cen- tral idea involves visually tracking a gyro-stabilized horizon bar installed on most Navy ships to approach and land vertically independent of deck motions. This was accomplished through the de- velopment of a rotorcraft flight dynamics modeling framework and vision-based control systems as well as conducting simulations and flight tests.
The framework, named Texas A&M Rotorcraft Analysis Code (TRAC), was developed as a modular tool that could model any rotorcraft configuration at a low computational cost. A UH-60 helicopter was modeled as a baseline aircraft and validated using the US Army flight test data. A linear quadratic regulator (LQR) controller was utilized to stabilize and control the helicopter during autonomous ship landing simulations.
The vision system was developed to obtain the visual information that a pilot perceives during ship approach and landing. It detects the ship at long-distance by utilizing machine/deep learning- based detection and at close range, it utilizes uniquely developed vision algorithms to detect the horizon bar to precisely estimate the aircraft position and orientation relative to the bar. It demon- strated 250 meters of detection range for a 6 x 6 ft sub-scale ship platform, which translates to a range of 17.3 kilometers for a full-scale 50 x 50 ft typical small ship. The distance and attitude estimations were validated using the measurements from an accurate 3D motion capture system (VICON), which demonstrated sub-centimeter and sub-degree accuracy.
To control the aircraft based on the perceived visual information, both nonlinear control and deep reinforcement learning control strategies were developed. The nonlinear controller demon- strated robust tracking capability even with 0.5 seconds of time delay and estimation noise. When flight-tested in 5 m/s wind gust, the deep reinforcement learning control demonstrated superior disturbance rejection capability, with 50% reduced drift at a 3 times faster rate compared to con- ventional control systems. Both vision and control systems were implemented on a quadrotor unmanned aircraft and extensive flight tests were conducted to demonstrate accurate tracking in challenging conditions and safe vertical landing on a translating ship platform with 6 degrees of freedom motions.
... 19,20), corner detection (Refs. 21,22), and contour detection (Ref. 23) are previously applied to aircraft landing applications. ...
The paper discusses a machine learning vision and nonlinear control approach for autonomous ship landing of vertical flight aircraft without utilizing GPS signal. The central idea involves automating the Navy helicopter ship landing procedure where the pilot utilizes the ship as the visual reference for long-range tracking, but refers to a standardized visual cue installed on most Navy ships called the "horizon bar" for the final approach and landing phases. This idea is implemented using a uniquely designed nonlinear controller integrated with machine vision. The vision system utilizes machine learning-based object detection for long-range ship tracking, and classical computer vision for object detection and the estimation of aircraft relative position and orientation during the final approach and landing phases. The nonlinear controller operates based on the information estimated by the vision system and has demonstrated robust tracking performance even in the presence of uncertainties. The developed autonomous ship landing system is implemented on a quad-rotor vertical takeoff and landing (VTOL) capable unmanned aerial vehicle (UAV) equipped with an onboard camera and was demonstrated on a moving deck, which imitates realistic ship deck motions using a Stewart platform and a visual cue equivalent to the horizon bar. Extensive simulations and flight tests are conducted to demonstrate vertical landing safety, tracking capability, and landing accuracy while the deck is in motion.
... The vast majority of the vision-based landing systems [Kong et al. , 2014] use the onboard camera and external markers [Lange et al. , 2008;Merz et al. , 2006;Saripalli, 2009;Saripalli et al. , 2002Saripalli et al. , , 2003Sharp et al. , 2001;Wenzel et al. , 2011;Xiang et al. , 2012;Zhao & Pei, 2013;Yang & Tsai, 1998;Huh & Shim, 2010;Sereewattana et al. , 2015] to waive the use of GPS. The autonomous landing of a small fixed-wing UAV into a net using a ground-based vision system without GPS was successfully tested by Kim et al. [2013]. ...
This thesis aims to develop a monocular vision system to track an Unmanned Aerial Vehicle (UAV) pose (3D position and orientation) relative to the camera reference frame during its landing on a ship. The vast majority of accidents and incidents occur during take-off or landing since, in the vast majority of systems, an external pilot takes control. Having less human intervention increases system reliability and alleviates the use of certified pilots. Due to UAV size and weight, take-off is easily performed by hand, so the main focus will be in the landing maneuver. The vision system is located on the ship’s deck, which reduces demands on the UAV’s processing power, size, and weight. The proposed architecture is based on an Unscented Particle Filter (UPF) scheme with two stages: (i) pose boosting, and (ii) tracking. In the pose boosting stage, we detect the UAV on the captured frame using Deep Neural Networks (DNNs) and initialize a set of pose hypotheses that are likely to describe the true pose of the target using a pre-trained database indexed by bounding boxes. In the tracking stage, we use a UPF based approach to obtain an online estimate of the true pose of the target. On contrary to many vision-based particle filters that sample particles from a distribution that is based solely on predictions from the previous frames, in this work, we also use information from the current frame to improve the convergence of the filter. We fuse information from current and previous time steps with Unscented Transform (UT) filters, and use, for the first time in this type of problem, the Bingham and Bingham-Gauss distributions to model the dynamics and noise of the orientation in its natural manifold. These filters depend on the computation of importance weights that use sub-optimal approximations to the likelihood function. We evaluate different similarity metrics that compute a distance measure between an artificial rendered image with the hypothetic state of the system and the captured frame. Since we are approximating the likelihood function, we enrich the filter with additional refinement steps to abridge its sub-optimality. We have developed a “realistic” simulator for a quantitative analysis of the results. The entire description and experimental analysis of the system is based on the tracking error and processing time. When analyzing a landing sequence with a real sky gradient filled with clouds, we have obtained approximately 81% less rotation error using the Unscented Bingham Filter (UBiF) and the Unscented Bingham-Gauss Filter (UBiGaF) when compared to the simple Unscented Kalman Filter (UKF) without considering the use of pose optimization. When we use pose optimization, we can decrease the obtained rotation error in more than 50%.
The detection, localization, and tracking of unmanned aerial vehicles (UAVs) are pivotal for ensuring reliable decision-making and intelligent control in scenarios involving heterogeneous agent cooperation. A dynamic localization framework with asynchronous LiDAR-Camera fusion is considered in this paper, which is used to provide absolute attitude and position observations of unmanned aerial vehicle and to achieve robust localization in outdoor environments. Firstly, a fast search architecture based on the depth cluster is presented to transform point clouds into distance images and establish distance image target extraction based on any two neighboring points. Besides, a neural network framework is introduced for the recognition of UAVs, where the feature maps are fed into a region suggestion network obtain optimal suggestions for object classification and bounding box regression. Furthermore, we designed a dual servo turntable integrated with multi-sensors to dynamically track the coordinates of the UAVs, ensuring that the vehicle remains centered within the detection area at all times. Finally, the heterogeneous agent is employed to evaluate the localization performance of UAVs in real-world situations. This indicates that asynchronous LiDAR-Camera fusion can run fully on embedded devices and productively in heterogeneous agent systems.
The automated operation of Unmanned Aerial Vehicles (UAVs) has become increasingly important, and one key step in the procedure is the accurate landing of such vehicles. A major problem is that when trying to design a control algorithm for the landing procedure, it is difficult to accurately measure the 3D pose of the vehicle with respect to the local environment. Three dimensional pose estimation and subsequent corrections in velocity to design the control system was omitted by using an image-based visual servoing controller. Another important development is choosing the appropriate visual features which ensures convergence to desired position. The proposed controller with image moments as features was able to converge and land the vehicle accurately using only the image moments as the visual features from two fiducial markers without any local position or global position information. The proposed method has been verified using simulations considering an existing model of UAV.
Algorithms for determining the attitude position of an aircraft or helicopter-type unmanned aerial vehicle relative to the landing platform with special optical marks are considered. An assessment is made of the possibility of calculating the angular position, height and distance to the landing platform in real time based on image processing by a separate on-board processor combined with a digital optical camera into a single measuring unit. The results of calculating the aircraft attitude relative to the landing platform moving along a program trajectory using computer vision algorithms are presented. Simulation of the process of recognizing optical marks on a moving platform from a moving aircraft confirmed that using a processor with a program for recognizing and identifying optical marks by using computer vision and algorithms for calculating the position of the aircraft relative to landing platforms can assuredly provide reliable information about the positioning of an unmanned aerial vehicle relative to the landing platform in real time and can be used in conjunction with other navigation aids (or independently) to ensure accurate landing of unmanned aircraft.
The vision-based system is a crucial component of unmanned aerial vehicle (UAV) autonomous flying and is frequently regarded as a difficult phase to accomplish. Most UAV accidents happen while landing or due to obstacles in the path. Hence, it is considered as one of the most important to think about the auto landing of UAVs to reduce accidents. Some technologies, like GPS, frequently don’t function indoors or in places where GPS transmissions aren’t allowed. They can land up to a few meters but lack accuracy. A system that operates in such circumstances is required to overcome this and be far more suitable. Cameras are used to offer much information about their surroundings and may be helpful in certain circumstances. A vision-based system’s accuracy can be as low as a few centimeters and better than GPS-based location estimation. This work involves designing a vision-based landing system that can recognize a marker by providing the bounding box around it. Typically, the H mark is employed in helicopter landing pads for vision-based landing systems. Here, the position is identified using the YOLOv5 algorithm. An image of A4-sized sheet and 2ft 2ft printed with H mark is taken using quadcopter and is used for data set. The algorithm is tested to locate the marker at any orientation and scale. Thus, YOLOv5 identifies the marker at any distance, orientation, or size in the given data set and performs better than SVM-based approach. This could be further used to find the distance on the ground from the UAV center that aids in auto landing.
Although advanced UAVs use self-guided landing with the support of GPS devices, issues such as signal loss, support for indoor environments, accuracy issues, etc., must be considered, which contribute greatly to the overall success of autonomous flight. Therefore, this chapter proposes two different methods that allow for the recognition of a landing platform using Computer Vision techniques in order to assist autonomous landing. The first method is based on an Expert System that allows for the recognition of a patented black and white platform by performing a geometric analysis of the regions based on thresholds that allow for a degree of plane distortion. The second method, based on Cognitive Computing, can be used to solve the limitations to plane distortion inherent to the first approach, and further uses a specific landing platform with six different colors in order to combine color segmentation techniques with pattern recognition algorithms, together with an intelligent geometric analysis based on a decision-making technique. As a result, recognition can be achieved at different ranges and inclination angles between the vision system and the platform. It is not affected by distortions to the image figures due to perspective projection, even making it possible to perform the recognition with only a partial view of the platform, something that has received scant attention in the literature to date. The novelty is therefore the robustness and precision in the recognition from a wide variety of perspectives, different lighting conditions, and even problems that result in only a partial view of the platform, such as those resulting from partial focus or blind spots due to overexposure.
The visual landing navigation of UAV is studied under two modes of visible and infrared imaging. The positive and negative sample libraries were established in visible and infrared thermal imaging modes respectively, and the cascade classifier was constructed by Adaboost training algorithm. The edge detection operator combined with mathematical morphology is used to preprocess the images taken in visible light mode to improve the detection ability of cooperative target corners. A method combining image gradient and mathematical morphology is proposed to preprocess images taken in infrared thermal imaging mode. Finally, a mathematical model of pose estimation algorithm is established based on radial consistency principle.
Autonomous landing in non-cooperative environments is a key step toward full autonomy of unmanned aerial vehicles (UAVs). Existing studies have addressed this problem by finding a flat area of the ground as the landing place. However, all these methods have poor generalization ability and ignore the semantic feature of the terrain. In this paper, we propose a well-designed binocular-LiDAR sensor system and a robust terrain understanding model to overcome these deficiencies. Our model infers both morphologic and semantic features of the ground by simultaneously performing depth completion and semantic segmentation. Moreover, during the inference phase, it self-evaluates the accuracy of the predicted depth map and dynamically selects the LiDAR accumulation time to ensure accurate depth prediction, which greatly improves the robustness of the UAV in completely unknown environments. Through extensive experiments on our collected low-altitude aerial image dataset and real UAVs, we verified that our model effectively learned two tasks simultaneously and achieved better performance than existing depth estimation-based landing methods. Furthermore, the UAV can robustly select a safe landing site in several complex environments with about 98% accuracy.
With the rapid development of the UAV, it is widely used in rescue and disaster relief, where autonomous landing is the key technology. Vision-based autonomous landing has the advantages of strong autonomy, low cost, and strong anti-interference ability. Moreover, vision navigation has higher guidance and positioning accuracy combined with other navigation methods, such as GPS/INS. This paper summarizes the research results in the field of vision-based autonomous landing for the UAV, and divides it into static, dynamic, and complex scenarios according to the type of landing destination. Among them, the static scenario includes two categories: cooperative targets and natural landmarks; the dynamic scenario is divided into two categories: vehicle-based autonomous landing and ship-based autonomous landing. The key technologies are summarized, compared, and analyzed and the future development trends are pointed out, which can provide a reference for the research on vision-based autonomous landing of UAVs.
An aircraft landing system is most essential to ensure quality air traffic and safe landing process. In most airports an advanced aircraft landing system is necessary. This project proposes a system wherein all departments functioning separately are integrated. This system will overcome disadvantages of the existing system such as slow landing process and possibilities of manual error. The project deals with developing an economical and affordable solution for perfect landing in airports based on physical ambient conditions in the airport. The proposed system continuously monitors all the physical parameters and sends an audio signal output in case of an abnormal condition. Various parameters which are required for safe landing such as air temperature, humidity, wind speed, wind direction, fog and visibility are measured using suitable sensors and interfaced to a controller unit. A Light Detection And Ranging (LIDAR) model is used to determine the ground level clearance. The distance of other aircrafts from the ground station is determined using Received Signal Strength Indicator (RSSI). A simulation output screen is used to continuously display the measured parameter values.
Today, unmanned vehicles get involved in challenging missions like search and rescue, surveillance, recognition, border patrolling, and other information-gathering roles. These vehicles prevent humans from being in dangerous situations, and their cost of production is lower than manned vehicles. Many researchers in past decades have studied the problem of tracking maneuvering targets based on noisy sensor measurements. The key to successfully tracking a target is to extract useful information from observations about the target state. Indeed, a proper model of the target dynamic and sensor observation will facilitate the extraction of this information, significantly. The filters used for estimation are the base model because there is knowledge of the target motion model. The purpose of this paper is to investigate and compare the capability of different dynamic models in tracking a high-maneuverability target using a 3D space by using a visual sensor. The goal is to test 10 different dynamic models with several different random processes and filters to find the most suitable model for tracking an aerial target. Sensor failure and model processing error have been selected as the two main criteria in measuring the performance of these models. We have introduced the best dynamic model based on the behavior of these models against these defects.
This work presents a novel landing assistance system (LAS) capable of locating a drone for a safe landing after its inspection mission. The location of the drone is achieved by a fusion of ultra-wideband (UWB), inertial measurement unit (IMU) and magnetometer data. Unlike other typical landing assistance systems, the UWB fixed sensors are placed around a 2 × 2 m landing platform and two tags are attached to the drone. Since this type of set-up is suboptimal for UWB location systems, a new positioning algorithm is proposed for a correct performance. First, an extended Kalman filter (EKF) algorithm is used to calculate the position of each tag, and then both positions are combined for a more accurate and robust localisation. As a result, the obtained positioning errors can be reduced by 50% compared to a typical UWB-based landing assistance system. Moreover, due to the small demand of space, the proposed landing assistance system can be used almost anywhere and is deployed easily.
The cooperation of Unmanned Aerial Vehicles (UAVs) and Mobile Robot Manipulators (MRMs) offers enormous possibilities to modern industry. It paves the way for logistics, cooperative assembling or manipulation and will provide even more flexibility and autonomy to today’s manufacturing processes. Currently, some systematic literature reviews exist that provide an overview on research fields and gaps in the field of UAVs and MRMs. However, an investigation of the research landscape for combined use of UAVs and MRMs does not exist to the best of the authors’ knowledge. Therefore, in this paper, a systematic review of the current research landscape for the combined use of UAV and MRM is conducted to finally identify fields of action that need to be addressed in the future to harness the full potential.
In general, modern operating systems can be divided into two essential parts, real-time operating systems (RTOS) and general-purpose operating systems (GPOS). The main difference between GPOS and RTOS is the system is time-critical or not. It means that; in GPOS, a high-priority thread cannot preempt a kernel call. But, in RTOS, a low-priority task is preempted by a high-priority task if necessary, even if it’s executing a kernel call. Most Linux distributions can be used as both GPOS and RTOS with kernel modifications. In this study, two Linux distributions, Ubuntu and Pardus, were analyzed and their performances were compared both as GPOS and RTOS for path planning of the multi-robot systems. Robot groups with different numbers of members were used to perform the path tracking tasks using both Ubuntu and Pardus as GPOS and RTOS. In this way, both the performance of two different Linux distributions in robotic applications were observed and compared in two forms, GPOS, and RTOS.
For more efficient aerial surveillance, charging pads are set up at corresponding distances so that an unmanned aerial vehicle (UAV) can sustain its operations without landing. Usually manual intervention is required to land a UAV for charging and so extend its mission. To enable a UAV to operate autonomously, wireless power charging using inductive coupling is proposed. Using this method, the UAV's battery is charged until it reaches the next charging station. This paper focuses on two significant aspects of the process: vision-based navigation for charging pad detection, and wireless power charging. The coils were designed, and other parameters like mutual inductance, coupling coefficient and the distance between the coils for effective power transmission were analysed, using Ansys and Maxwell software. A quadcopter was built, with battery and Lidar sensor connected to the Arduino controller for low battery voltage detection and height measurement, respectively. Whenever the battery voltage is low, the UAV is steered towards the nearest charging pad using the global position navigation system. To test the process, the quadcopter was flown over the charging pad using a vision-based algorithm pre-defined in the image processor (Raspberry Pi B+).
This chapter provides a review of various use cases for unmanned aerial vehicles (UAVs) to provide some context, and gives a brief literature review on existing efforts for modeling propagation characteristics of aerial links. It explains some of the unique propagation channel characteristics for UAV air‐to‐ground (AG) channels such as operating frequencies, scattering geometries, antenna effects, and Doppler, all in comparison with terrestrial channels. A review of some key considerations for AG channel measurements is also provided, including actual measurement frequencies, platform configurations, measurement environments, unique challenges for AG measurements, sounding waveform types for AG measurements, and the effects of elevation angle on measurement results. The chapter reviews the path loss and shadowing, delay dispersion, narrowband fading, Doppler spread, throughput and bit error rate characteristics, and the effects of different measurement environment types. It discusses AG propagation channel models, including models based on deterministic and stochastic models, their combination, and ray tracing simulations.
Selecting safe landing sites in non-cooperative environments is a key step towards the full autonomy of UAVs. However, the existing methods have the common problems of poor generalization ability and robustness. Their performance in unknown environments is significantly degraded and the error cannot be self-detected and corrected. In this paper, we construct a UAV system equipped with low-cost LiDAR and binocular cameras to realize autonomous landing in non-cooperative environments by detecting the flat and safe ground area. Taking advantage of the non-repetitive scanning and high FOV coverage characteristics of LiDAR, we come up with a dynamic time depth completion algorithm. In conjunction with the proposed self-evaluation method of the depth map, our model can dynamically select the LiDAR accumulation time at the inference phase to ensure an accurate prediction result. Based on the depth map, the high-level terrain information such as slope, roughness, and the size of the safe area are derived. We have conducted extensive autonomous landing experiments in a variety of familiar or completely unknown environments, verifying that our model can adaptively balance the accuracy and speed, and the UAV can robustly select a safe landing site.
This research focuses on developing a helicopter autonomous ship landing algorithm based on the real helicopter ship landing procedure which is already proven and currently used by Navy pilots. It encompasses the entire ship landing procedure from approach to landing using a pilot-inspired vision-based navigation system. The present thesis focuses on the first step towards achieving this overarching objective, which involves modeling the flight dynamics and control of a helicopter and some preliminary simulations of a UH-60 (Blackhawk) helicopter landing on a ship.
We focus on the use of an on-board camera as an additional
navigation sensor for an autonomous helicopter. We focus on the critical
landing phase, where we exploit the structure of the problem to obtain
computationally tractable algorithms. We use the other available
navigational elements present on the helicopter to simplify the vision
processing and resolve typical image interpretation ambiguities. Thus,
the processing of the vision system is done not in isolation, but in the
larger context of the overall navigation problem. By using a tight
integration of vision with inertial estimates the algorithm can be
implemented at modest frame rates (~5 Hz) with off-the-shelf hardware
This paper presents a new method for the computation of the position and orientation of a camera with respect to a known object, using four or morecoplanarfeature points. Starting with the scaled orthographic projection approximation, this method iteratively refines up to two different pose estimates, and provides an associated quality measure for each pose. When the camera distance is large compared with the object depth, or when the accuracy of feature point extraction is low because of image noise, the quality measures for the two poses are similar, and the two pose estimates are plausible interpretations of the available information. In contrast, known methods using a closed form pose solution for four coplanar points are not robust for distant objects in the presence of image noise because they provide only one of the two possible poses and may choose the wrong pose.
Addresses the problem of navigation system design for autonomous
aircraft landing. New nonlinear filter structures are introduced to
estimate the position of an aircraft with respect to a possibly moving
landing site, such as a naval vessel, based on measurements provided by
airborne vision and inertial sensors. By exploring the geometry of the
navigation problem, the navigation filter dynamics are cast in the
framework of linear parametrically varying systems (LPVs). Using this
set-up, filter performance and stability are studied in an H<sub>∞
</sub> setting by resorting to the theory of linear matrix inequalities
(LMIs). The design of nonlinear, globally stable filters to meet
adequate H∞ performance measures is thus converted into
that of determining the feasibility of a related set of LMIs and finding
a solution to them, if it exists. This is done by resorting to widely
available numerical tools that borrow from convex optimization
techniques. The paper develops the mathematical framework that is
required for integrated vision/inertial navigation system design and
details a design example for an air vehicle landing on an aircraft
carrier
This paper introduces the development of multiple number of Unmanned Arial Vehicle (UAV) system as a part of BErkeley AeRobot (BEAR) project, highlighting the recent achievements in the design and implementation of rotorcraft-based UAV (RUAV) control system. Based on the experimental flight data, linear system model valid near hover condition is found by applying time-domain numerical methods to experimental flight data. The acquired linear model is used to design feedback controller consisting of inner-loop attitude feedback control, mid-loop velocity feedback control and the outer-loop position control. The proposed vehicle-level controller is implemented and tested in Berkeley UAV, Ursa Magna 2, and shows superior hovering performance. The vehicle level controller is integrated with higher-level control using a script language framework to command UAV.
Estimating motion and structure of the scene from image sequences is a very important and active research area in computer vision. The results of research have applications in vision-guided navigation, robot vision, 3-D object recognition and manipulation etc. Many theoretical results and new techniques developed may also apply to the related problems of other fields.
Computing the image displacement field, or matching two images is one of the difficult problems in motion analysis. A computational approach to image matching has been developed that uses multiple attributes associated with images to yield a generally overdetermined system of matching constraints, taking into account possible structural discontinuities and occlusions.
From the computed image displacement field, the next step is to compute the motion parameters and the structure of the scene. A two-step approach is introduced to solve the nonlinear optimization problem reliably and efficiently. The uniqueness of solution, robustness of the solution in the presence of noise, estimation of errors, dependency of the reliability of solution on motion, scene, and the parameters of image sensors have been investigated. It is analyzed that a batch processing technique (Levenberg-Marquardt nonlinear least-squares method) generally performs better than a sequential processing technique (iterated extended Kalman filtering) for nonlinear problems. For those problems where estimates are needed before all the data are acquired, a recursive batch processing technique has been developed to improve performance and computational efficiency. The performance of the motion estimation algorithm has essentially reached the Cramer-Rao bound.
The algorithm has been applied to real world scenes with depth discontinuities and occlusions to compute motion parameters, dense depth maps and occlusion maps, from two images taken at different unknown positions and orientations relative to the scene. The standard discrepancy between the projection of the inferred 3-D scene and the actually observed projection is as small as one half of a pixel.
Other problems investigated include: (1) motion and structure from point correspondences for planar scenes. (2) motion and structure from line correspondences. (3) dynamic motion estimation and prediction from long image sequences.
This paper proposes a hierarchical hybrid architecture of a reactive system that governs the operation of a helicopter based UAV. The mission of the helicopter is to search for, investigate and locate objects in an unknown environment. To perform visual navigation, the helicopter is equipped with a Flight Vehicle Management System(FVMS) and a vision system that scans the ground for objects and landmarks. Hybrid system combines continuous dynamics with discrete automata, and therefore provides a congenial framework to integrate planning, vision and control. Computer Aided Design tools for real-time embedded systems enhance the conception and verification of the FVMS.
The 4D approach to dynamic machine vision has been validated for the application area of on-board autonomous landing approaches in the visual flight regime with computing technology available today; sensors are a video-camera, inertial gyros and an air velocity meter. The key feature of the method is the reconstruction and servo-maintained adjustment by prediction error feedback of an internal spatiotemporal model about the process to be controlled. This encompasses both the egomotion state of the aircraft carrying the sensors and the relevant geometric properties of the runway and its spatial environment. The efficiency of the approach is proved both in a hardware-in-the-loop simulation and in real test flights with a twin turbo-prop aircraft. For accuracy evaluation of the data gathered, the results of differential GPS and radiometric altitude measurements have been recorded simultaneously.
In this paper, we study the problem of using computer vision as a
sensor to control the landing of an unmanned air vehicle (UAV). The
vision problem we address is a special case of the general ego-motion
estimation problem due to the fact that all feature points lie on a
plane. We propose a new geometric estimation scheme for solving the
differential version of the planar ego-motion estimation problem. The
algorithm is computationally inexpensive and amenable for real-time
implementation. We present a performance evaluation of the algorithm
under different levels of image measurement noise and camera motions
relative to the landing pad. We also present a full dynamic model of a
UAV, discuss a nonlinear controller based on differential flatness, and
show through simulation that the vision guided UAV performs stable
landing maneuvers even under large levels of image measurement noise
An approach to landmark location estimation by computer vision techniques is proposed. The objective is to derive the position and the orientation of the landmark with respect to the vehicle by a single image. Such information is necessary for automatic vehicle navigation. This approach requires lower hardware cost and simple computation. The vanishing points of the parallel lines on the landmark are used to detect the landmark orientation. The detected vanishing points are used to derive the relative orientation between the landmark and the camera, which is then utilized to compute the landmark orientation with respect to the vehicle. The size of the landmark is used to determine the landmark position. Sets of collinear points are extracted from the landmark and their inter-point distances are computed. The positions of the collinear point sets are evaluated and used to determine the landmark position. Landing site location estimation by using the identification marking H on the helicopter landing site for automatic helicopter landing is presented as an application of the proposed approach. Simulations and experiments have been conducted to prove the feasibility of the proposed approach.
In this paper, we use computer vision as a feedback sensor in a control loop for landing an unmanned air vehicle (UAV) on a landing pad. The vision problem we address here is then a special case of the classic ego-motion estimation problem since all feature points lie on a planar surface (the landing pad). We study together the discrete and differential versions of the ego-motion estimation, in order to obtain both position and velocity of the UAV relative to the landing pad. After briefly reviewing existing algorithm for the discrete case, we present, in a unified geometric framework, a new estimation scheme for solving the differential case. We further show how the obtained algorithms enable the vision sensor to be placed in the feedback loop as a state observer for landing control. These algorithms are linear, numerically robust, and computationally inexpensive hence suitable for real-time implementation. We present a thorough performance evaluation of the motion estimation algorith...
obot (BEAR) Project homepage
- Berkeley Aeh
BErkeley AeH.obot (BEAR) Project homepage.
http://robotics.eecs.berkeley.edu/bear.