Article

Towards High Speed Aerial Tracking of Agile Targets

Authors:
To read the full-text of this research, you can request a copy directly from the author.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the author.

Article
Full-text available
This paper describes novel implementations of the KLT feature track- ing and SIFT feature extraction algorithms that run on the graphics processing unit (GPU) and is suitable for video analysis in real-time vision systems. While significant acceleration over standard CPU implementations is obtained by ex- ploiting parallelism provided by modern programmable graphics hardware, the CPU is freed up to run other computations in parallel. Our GPU-based KLT im- plementation tracks about a thousand features in real-time at 30 Hz on 1024 £ 768 resolution video which is a 20 times improvement over the CPU. It works on both ATI and NVIDIA graphics cards. The GPU-based SIFT implementation works on NVIDIA cards and extracts about 800 features from 640 £ 480 video at 10Hz which is approximately 10 times faster than an optimized CPU implementation.
Article
Full-text available
Necessary structural criteria are obtained for linear multivariable regulators which retain loop stability and output regulation in the presence of small perturbations, of specified types, in system parameters. It is shown that structural stability thus defined requires feedback of the regulated variable, together with a suitably reduplicated model, internal to the feedback loop, of the dynamic structure of the exogenous reference and disturbance signals which the regulator is required to process. Necessity of these structural features constitutes the internal model principle.
Conference Paper
Full-text available
In this paper, we discuss our implementation of a graphics hardware acceleration of the Vector Coherence Mapping vision processing algorithm. Using this algorithm as our test case, we discuss our optimization strategy for various vision processing operations using NVIDIA’s new CUDA programming framework. We also demonstrate how flexibly and readily vision processing algorithms can be mapped onto massively parallelized GPU architecture. Our results and analysis show the GPU implementation exhibits a performance gain of more than 40-fold of speedup over state-of-art CPU implementation of VCM algorithm.
Conference Paper
Full-text available
This paper presents a nonlinear airspeed controller for an unmanned aerial vehicle (UAV). It is designed using the technique of nonlinear dynamic inversion. The thrust force of the UAV is generated by a propeller. The nonlinear dynamics of the airspeed are appropriately expressed as a function of the thrust force, which is a nonlinear function of the engine speed. The latter variable is the only physical control input of the system. The airspeed control system is constituted of a linear controller responsible for the generation of the desired airspeed dynamics, which are successively converted into a desired thrust command and an engine speed command through two consecutive nonlinear transformations. The proposed architecture is modular, easy to implement, and computationally efficient. Nonlinear simulation results demonstrate the effectiveness of the method.
Conference Paper
Full-text available
Aerodynamic parameter estimation provides an effective way for aerospace system modelling using measured data from flight test, especially for the purpose of developing elaborate simulation environments and control systems design of Unmanned Aerial Vehicle (UAV) with short design cycles and reduced cost. However, parameter identification of airplane dynamics is complicated because of its nonlinear identification models and the combination of noisy and biased sensor measurements. The combined difficulties mentioned above make the problem of state and parameter estimation a nonlinear filtering problem. Extended Kalman Filter (EKF) is an excellent tool for this matter with the property of recursive parameter identification and excellent filtering. The standard EKF algorithm is based on a first order approximation of system dynamics. More refined linearization techniques such as iterated EKF can be used to reduce the linearization error in the EKF for highly nonlinear systems, which leads to a theoretically better result. In this paper we concentrate on the application and comparison of EKF and iterated EKF for aerodynamic parameter estimation of a fixed wing UAV. The result shows that the two methods have been able to provide accurate estimations.
Conference Paper
Full-text available
Feature tracking is one of fundamental steps in many computer vision algorithms and the KLT (Kanade-Lucas-Tomasi) method has been successfully used for optical flow estimation. There has been also much effort to implement KLT on GPUs to increase the speed with more features. Many implementations have chosen the translation model to describe a template motion because of its simplicity. However, a more complex model is demanded for appearance change especially in outdoor scenes or when camera undergoes roll motions. We implement the KLT tracker using an affine-photometric model on GPUs which has not been in a popular use due to its computational complexity. With careful attention to the parallel computing architecture of GPUs, up to 1024 feature points can be tracked simultaneously at a video rate under various 3D camera motions. Practical implementation issues will be discussed in the NVIDIA CUDA framework. We design different thread types and memory access patterns according to different computation requirements at each step of the KLT. We also suggest a CPU-GPU hybrid structure to overcome GPU limitations.
Article
Full-text available
Aircraft in extreme flight conditions can encounter severe non-linear effects generated from high angles of attack and high angular rates. Flight control systems based upon non-linear inverse dynamics offer the potential for providing improved levels of safety and performance in these flight conditions over the competing designs developed using linearizing assumptions. Inverse dynamics are generated for specific command variable sets of a 12-state non-linear aircraft model to develop a control system which is valid over the entire flight envelope. Detailed descriptions of the inertial dynamic and aerodynamic models are given, and it is shown how the command variable sets are altered as a function of the system state to add stall prevention features to the system. Simulation results are presented for various mission objectives over a range of flight conditions to confirm the effectiveness of the design.
Conference Paper
Full-text available
We propose a simple and elegant algorithm to track nonrigid objects using a covariance based object description and a Lie algebra based update mechanism. We represent an object window as the covariance matrix of features, therefore we manage to capture the spatial and statistical properties as well as their correlation within the same representation. The covariance matrix enables efficient fusion of different types of features and modalities, and its dimensionality is small. We incorporated a model update algorithm using the Lie group structure of the positive definite matrices. The update mechanism effectively adapts to the undergoing object deformations and appearance changes. The covariance tracking method does not make any assumption on the measurement noise and the motion of the tracked objects, and provides the global optimal solution. We show that it is capable of accurately detecting the nonrigid, moving objects in non-stationary camera sequences while achieving a promising detection rate of 97.4 percent.
Conference Paper
Full-text available
We describe a new region descriptor and apply it to two problems, object detection and texture classification. The covariance of d-features, e.g., the three-dimensional color vector, the norm of first and second derivatives of intensity with respect to x and y, etc., characterizes a region of interest. We describe a fast method for computation of covari- ances based on integral images. The idea presented here is more general than the image sums or histograms, which were already published before, and with a series of integral images the covariances are obtained by a few arithmetic operations. Covariance matrices do not lie on Euclidean space, therefore we use a distance metric involving generalized eigenval- ues which also follows from the Lie group structure of positive definite matrices. Feature matching is a simple nearest neighbor search under the distance metric and performed extremely rapidly using the integral images. The performance of the covariance features is superior to other methods, as it is shown, and large rotations and illumination changes are also absorbed by the covariance matrix.
Conference Paper
Full-text available
We present a new parallel approach for the computation of an optical flow field from a video image sequence. This approach incorporates the various local smoothness, spatial and temporal coherence constraints transparently by the application of fuzzy image processing techniques. Our Vector Coherence Mapping VCM approach accomplishes this by a weighted voting process in local vector space, where the weights provide high level guidance to the local voting process. Our results show that VCM is capable of extracting flow fields for video streams with global dominant fields (e.g. owing to camera pan or translation, moving camera and moving object(s), and multiple moving objects. Our results also show that VCM is able to operate under strong image noise and motion blur, and is not susceptible to boundary oversmoothing.
Article
Full-text available
In this survey, we present a brief analysis of single camera object detection and tracking methods. We also give a comparison of their computational complexities. These methods are designed to perform accurately under difficult conditions such as erratic motion, drastic illumination change, and noise contamination. I. I NTRODUCTION Object tracking is one the most important tasks in computer vision. In video surveillance, it assists understanding the movement patterns of people to uncover suspicious events. It is a key technology in traffic management to estimate flux and congestion statistics. Advanced vehicle control systems depend on the tracking information to keep the vehicle in lane and prevent from collisions. In physical therapy, analyzing the mobility of patients improves the accuracy of their diagnosis. Learning the shopping behavior of customers by tracking assists the architecture design in retail space instrumentation. In robotics, tracking bridges the gap between the raw visual information and environmental awareness. In video summarization, it is applied to generate object-based representations and automatic content annotations. Tracking is also a fundamental technology to extract regions of interest and video object layers as defined in JPEG-2000 and MPEG-4 standards. Even though it is essential to many applications, robust object tracking under uncontrolled conditions still poses a challenge. Real-life systems are required to track objects not only when the background scene is static but also when lighting changes suddenly, camera-object motion becomes large, color contrast becomes low, image noise soars to an unacceptable level, etc. In addition, the computational complexity is required to be kept minimum for real-time performance. In the following sections, we describe object detection and tracking methods that are designed to resolve the above issues.
Conference Paper
Full-text available
Image registration finds a variety of applications in computer vision. Unfortunately, traditional image registration techniques tend to be costly. We present a new image registration technique that makes use of the spatial intensity gradient of the images to find a good match using a type of Newton-Raphson iteration. Our technique is taster because it examines far fewer potential matches between the images than existing techniques Furthermore, this registration technique can be generalized to handle rotation, scaling and shearing. We show how our technique can be adapted tor use in a stereo vision system.
Article
Full-text available
In this paper, we report fast calculation of a computer-generated-hologram using a new architecture of the HD5000 series GPU (RV870) made by AMD and its new software development environment, OpenCL. Using a RV870 GPU and OpenCL, we can calculate 1,920×1,024 resolution of a CGH from a 3D object consisting of 1,024 points in 30 milli-seconds. The calculation speed realizes a speed approximately two times faster than that of a GPU made by NVIDIA.
Conference Paper
Full-text available
This paper reports on the implementation of a GPU-based, real-time eye blink detector on very low contrast images acquired under near-infrared illumination. This detector is part of a multi-sensor data acquisition and analysis system for driver performance assessment and training. Eye blinks are detected inside regions of interest that are aligned with the subject's eyes at initialization. Alignment is maintained through time by tracking SIFT feature points that are used to estimate the affine transformation between the initial face pose and the pose in subsequent frames. The GPU implementation of the SIFT feature point extraction algorithm ensures real-time processing. An eye blink detection rate of 97% is obtained on a video dataset of 33,000 frames showing 237 blinks from 22 subjects.
Article
Full-text available
An inversion procedure is introduced for nonlinear systems which constructs a bounded input trajectory in the preimage of a desired output trajectory. In the case of minimum phase systems, the trajectory produced agrees with that generated by Hirschorn's inverse dynamic system; however, the preimage trajectory is noncausal (rather than unstable) in the nonminimum phase case. In addition, the analysis leads to a simple geometric connection between the unstable manifold of the system zero dynamics and noncausality in the nonminimum phase case. With the addition of stabilizing feedback to the preimage trajectory, asymptotically exact output tracking is achieved. Tracking is demonstrated with a numerical example and compared to the well-known Byrnes-Isidori regulator. Rather than solving a partial differential equation to construct a regulator, the inverse is calculated using a Picard-like interaction. When preactuation is not possible, noncausal inverse trajectories can be truncated resulting in the tracking-error transients found in other control schemes
Article
This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.
Conference Paper
A new MEMS-based small, light, and cost-effective avionics is proposed in order to improve the current situation of system identification of small unmanned aerial vehicles (UAVs), which mostly depends on difficult wind tunnel tests. The avionics is capable of gathering navigation, wind, and control input data accurately. Flight tests with a small UAV are conducted, and its longitudinal stability derivatives are obtained accurately using flight data obtained by the avionics and system identification techniques.
Article
A comparison is made of four algorithms for parameter estimation in linear and nonlinear systems accounting for both process and measurement noise using two different approaches: direct approach and filtering approach. In the direct approach, the iterative Gauss-Newton method incorporating a suitable state estimator is used to estimate the unknown parameters by optimization of the likelihood function. For the state estimation both time-varying and steady-state filters are used. In the filtering approach, the unknown parameters are estimated as augmented states using the extended Kalman filter. The various algorithms are used to estimate from simulated as well as flight-test data the aircraft dimensional and nondimensional derivatives. Three model postulates, one linear and two nonlinear, are employed for this purpose. The parameter-estimation results indicate that the Gauss-Newton method with a steady-state filter, found to be adequate for the typical aircraft-estimation examples is generally preferable.
Article
The book treats the dynamics of atmospheric flight, with special reference to the stability and control of airplanes. An extensive set of numerical examples covers STOL airplane, subsonic jet transport, hypersonic airplane, stability augmentation, and wind and density gradients. The book completely covers equations of motion, including effects of round rotating Earth and distortional motion. There are complete chapters on human pilots and handling qualities and flight in turbulence, with numerical examples for a jet transport. Small-perturbation equations for longitudinal and lateral motion are presented in convenient matrix forms, both in time domain and Laplace transforms, dimensional and non-dimensional. (Author)
Article
In this report we summarize our recent research on the development of a systematic methodology for the design of feedback laws achieving stabilization and regulation of nonlinear control systems. We consdier the stabilization and control of both lumped nonlinear systems and nonlinear distributed parameter systems. The principal control objective is output regulation, the ability of the output of a system to track a desired signal while rejecting signals produced by a known exogenous system. This objective must also be achieved in a manner that is robust with respect to variation in unknown plant parameters; i.e., we develop a methodology for output regulation that is robust against real parametric uncertainty.
Article
The use of modern, high-performance graphical processing units (GPUs) for acceleration of scientific com-putation has been widely reported. The majority of this work has used the CUDA programming model sup-ported exclusively by GPUs manufactured by NVIDIA. An industry standardisation effort has recently pro-duced the OpenCL specification for GPU programming. This offers the benefits of hardware-independence and reduced dependence on proprietary tool-chains. Here we describe a source-to-source translation tool, "Swan" for facilitating the conversion of an existing CUDA code to use the OpenCL model, as a means to aid programmers experienced with CUDA in evaluating OpenCL and alternative hardware. While the performance of equivalent OpenCL and CUDA code on fixed hardware should be comparable, we find that a real-world CUDA application ported to OpenCL exhibits an overall slow-down of ≈ 50%, a reduction attributable to the immaturity of contemporary compilers. The ported application is shown have plat-form independence, running on both NVIDIA and AMD GPUs without modification. We conclude that OpenCL is a viable platform for developing portable GPU applications but that the more mature CUDA tools continue to provide best performance.
Book
This valuable volume offers a systematic approach to flight vehicle system identification and covers exhaustively the time-domain methodology. It addresses in detail the theoretical and practical aspects of various parameter estimation methods, including those in the stochastic framework and focusing on nonlinear models, cost functions, optimization methods, and residual analysis. A pragmatic and balanced account of pros and cons in each case are provided. The book also presents data gathering and model validation and covers both large-scale systems and high-fidelity modeling. Real world problems dealing with a variety of flight vehicle applications are addressed and solutions are provided. Examples encompass such problems as estimation of aerodynamics, stability, and control derivatives from flight data, flight path reconstruction, nonlinearities in control surface effectiveness, stall hysteresis, unstable aircraft, and other critical considerations. Beginners, as well as practicing researchers, engineers, and working professionals who wish to refresh or broaden their knowledge of flight vehicle system identification, will find this book highly beneficial. Based on years of experience, the book also provides recommendations for overcoming problems likely to be faced in developing complex nonlinear and high-fidelity models and can help the novice negotiate the challenges of developing highly accurate mathematical models and aerodynamic databases from experimental flight data. Software that runs under MATLAB® and sample flight data are provided to assist the reader in reworking the examples presented in the text. The software can also be adapted to the reader’s own interests.
Conference Paper
Flight test results using a SeaScan UAV with a gimballing camera to track both stationary and moving ground targets are presented. We experimentally studied the effect of UAV altitude above the target, camera field of view, and orbit center offsets within the geolocation tracking performance for both stationary and moving targets. In addition, all of the tests were performed using two different aircraft navigation systems, showing important sensitivities within the system. Sensor biases are shown to directly cause slowly varying errors in the geolocation estimates which can dominate tracking performance. These errors, which typically oscillate with the UAV orbit, are adequately bounded with a geolocation estimator which captures both the target tracking uncertainty, as well as unobservable sensor biases.
Conference Paper
This paper presents a state of the art report on using graphics hardware for image processing and computer vision. Then we describe GPUCV, an open library for easily developing GPU accelerated image processing and analysis operators and applications
Conference Paper
We present a simple control strategy to visually track a ground target with a fixed wing UAV. The approach uses a lateral guidance law that aims at reaching a given target view angle, which is determined on the basis of simple geometric considerations. Simulations compare our approach with previous contributions in terms of target visibility ratio, and results obtained during flight tests are presented.
Article
Aerodynamic parameter estimation is an integral part of aerospace system design and life cycle process. Recent advances in computational power have allowed the use of online parameter estimation techniques in varied applications such as reconfigurable or adaptive control, system health monitoring, and fault tolerant control. The combined problem of state and parameter identification leads to a nonlinear filtering problem; furthermore, many aerospace systems are characterized by nonlinear models as well as noisy and biased sensor measurements. Extended Kalman filter (EKF) is a commonly used algorithm for recursive parameter identification due to its excellent filtering properties and is based on a first order approximation of the system dynamics. Recently, the unscented Kalman filter (UKF) has been proposed as a theoretically better alternative to the EKF in the field of nonlinear filtering and has received great attention in navigation, parameter estimation, and dual estimation problems. However, the use of UKF as a recursive parameter estimation tool for aerodynamic modeling is relatively unexplored. In this paper we compare the performance of three recursive parameter estimation algorithms for aerodynamic parameter estimation of two aircraft from real flight data. We consider the EKF, the simplified version of the UKF and the augmented version of the UKF. The aircraft under consideration are a fixed wing aircraft (HFB-320) and a rotary wing UAV (ARTIS). The results indicate that although the UKF shows a slight improvement in some cases, the performance of the three algorithms remains comparable.
Article
This paper sketches the main research developments in the area of computational methods for eigenvalue problems during the 20th century. The earliest of such methods dates back to work of Jacobi in the middle of the 19th century. Since computing eigenvalues and vectors is essentially more complicated than solving linear systems, it is not surprising that highly significant developments in this area started with the introduction of electronic computers around 1950. In the early decades of this century, however, important theoretical developments had been made from which computational techniques could grow. Research in this area of numerical linear algebra is very active, since there is a heavy demand for solving complicated problems associated with stability and perturbation analysis for practical applications. For standard problems, powerful tools are available, but there still remain many open problems. It is the intention of this contribution to sketch the main developments of this century, especially as they relate to one another, and to give an impression of the state of the art at the turn of our century.
Article
We consider a single-input-single-output nonlinear system which has a uniform relative degree equal to the dimension of the state vector. The system can be transformed into a normal form with no zero dynamics. We allow the system's equation to depend on bounded uncertain parameters which do not change the relative degree. Disturbances are assumed to satisfy a strict-feedback condition which allows us to use a time-varying, disturbance-dependent transformation to transform the system into an error space where disturbances and uncertainties satisfy the matching condition. The nonlinear functions in the error space are used to choose an internal model which is augmented with the system, and a robust state feedback control is designed to drive the error to a positively invariant set that contains the origin. We then show the existence of a zero-error manifold inside this set which attracts all trajectories inside the set. To implement this control using output feedback, we saturate the state feedback control outside a compact set of interest and estimate the state using a high-gain observer. The output feedback controller recovers the robustness and asymptotic tracking properties of the state feedback controller.
Conference Paper
Graphics and vision are approximate inverses of each other: ordinarily Graphics Processing Units (GPUs) are used to convert "numbers into pictures" (i.e. computer graphics). In this paper, we propose using GPUs in approximately the reverse way: to assist in "converting pictures into numbers" (i.e. computer vision). The OpenVIDIA project uses single or multiple graphics cards to accelerate image analysis and computer vision. It is a library and API aimed at providing a graphics hardware accelerated processing framework for image processing and computer vision. OpenVIDIA explores the creation of a parallel computer architecture consisting of multiple Graphics Processing Units (GPUs) built entirely from commodity hardware. OpenVIDIA uses multiple Graphic.Processing Units in parallel to operate as a general-purpose parallel computer architecture. It provides a simple API which implements some common computer vision algorithms. Many components can be used immediately and because the project is Open Source, the code is intended to serve as templates and examples for how similar algorithms are mapped onto graphics hardware. Implemented are image processing techniques (Canny edge detection, filtering), image feature handling (identifying and matching features) and image registration, to name a few.
Conference Paper
We propose a parallel Mean Shift (MS) tracking algorithm on Graphics Processing Unit (GPU) using Compute Unified Device Architecture (CUDA). Traditional MS algorithm uses a large number of color histogram, say typically 16x16x16, which makes parallel implementation infeasible. We thus employ K-Means clustering to partition the object color space that enables us to represent color distribution with a quite small number of bins. Based on this compact histogram, all key components of the MS algorithm are mapped onto the GPU. The resultant parallel algorithm consist of six kernel functions, which involves primarily the parallel computation of the candidate histogram and calculation of the Mean Shift vector. Experiments on public available CAVIAR videos show that the proposed parallel tracking algorithm achieves large speedup and has comparable tracking performance, compared with the traditional serial MS tracking algorithm.
Conference Paper
This paper proposes a vision-based algorithm to autonomously track and chase a moving target with a small-size flying UAV. The challenging constraints associated with the UAV flight led us to consider a density-based representation of the object to track. The proposed approach to estimate the target's position, orientation and scale, is built on a robust color-based tracker using a multi-part representation. This object tracker can handle large displacements, occlusions and account for some image noise due to partial loss of wireless video link, thanks to the use of a particle filter. The information obtained from the visual tracker is then used to control the position and yaw angle of the UAV in order to chase the target. A hierarchical control scheme is designed to achieve the tracking task. Experiments on a quad-rotor UAV following a small moving car are provided to validate the proposed approach.
Article
In the field of computer vision, it is becoming increasingly popular to implement algorithms, in sections or in their entirety, on a graphics processing unit (GPU). This is due to the superior speed GPUs offer compared to CPUs. In this paper, we present a GPU library, MinGPU, which contains all of the necessary functions to convert an existing CPU code to GPU. We have created GPU implementations of several well known computer vision algorithms, including the homography transformation between two 3D views. We provide timing charts and show that our MinGPU implementation of homography transformations performs approximately 600 times faster than its C++ CPU implementation.
Article
This paper presents a method for extracting distinctive invariant features from images that can be used to perform reliable matching between different views of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through least-squares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.
Article
A maximum likelihood estimation method was applied to flight data and procedures to facilitate the routine analysis of a large amount of flight data were described. Techniques that can be used to obtain stability and control derivatives from aircraft maneuvers that are less than ideal for this purpose are described. The techniques involve detecting and correcting the effects of dependent or nearly dependent variables, structural vibration, data drift, inadequate instrumentation, and difficulties with the data acquisition system and the mathematical model. The use of uncertainty levels and multiple maneuver analysis also proved to be useful in improving the quality of the estimated coefficients. The procedures used for editing the data and for overall analysis are also discussed.
Article
The aircraft parameter estimation problem is used to illustrate the utility of parameter estimation, which applies to many engineering and scientific fields. Maximum likelihood estimation has been used to extract stability and control derivatives from flight data for many years. This paper presents some of the basic concepts of aircraft parameter estimation and briefly surveys the literature in the field. The maximum likelihood estimator is discussed, and the basic concepts of minimization and estimation are examined for a simple simulated aircraft example. The cost functions that are to be minimized during estimation are defined and discussed. Graphic representations of the cost functions are given to illustrate the minimization process. Finally, the basic concepts are generalized, and estimation from flight data is discussed. Some of the major conclusions for the simulated examples are also developed for the analysis of flight data from the F-14, highly maneuverable aircraft technology (HiMAT), and Space Shuttle vehicles.
Conference Paper
Graphics and vision are approximate inverses of each other: ordinarily graphics processing units (GPUs) are used to convert ldquonumbers into picturesrdquo (i.e. computer graphics). In this paper, we discuss the use of GPUs in approximately the reverse way: to assist in ldquoconverting pictures into numbersrdquo (i.e. computer vision). For graphical operations, GPUs currently provide many hundreds of gigaflops of processing power. This paper discusses how this processing power is being harnessed for image processing and computer vision, thereby providing dramatic speedups on commodity, readily available graphics hardware. A brief review of algorithms mapped to the GPU by using the graphics API for vision is presented. The NVIDIA CUDA programming model is then introduced as a way of expressing program parallelism without the need for graphics expertise.
Conference Paper
The Canny edge detector is a very popular and effective edge feature detector that is used as a pre-processing step in many computer vision algorithms. It is a multi-step detector which performs smoothing and filtering, non-maxima suppression, followed by a connected-component analysis stage to detect ldquotruerdquo edges, while suppressing ldquofalserdquo non edge filter responses. While there have been previous (partial) implementations of the Canny and other edge detectors on GPUs, they have been focussed on the old style GPGPU computing with programming using graphical application layers. Using the more programmer friendly CUDA framework, we are able to implement the entire Canny algorithm. Details are presented along with a comparison with CPU implementations. We also integrate our detector in to MATLAB, a popular interactive simulation package often used by researchers. The source code will be made available as open source.
Conference Paper
A field programmable gate array (FPGA) system implementation capable of being mounted onboard a micro aerial vehicle (MAV) (less than 5 pounds) that can perform the processing tasks necessary to identify and track a marked target landing site in real-time is presented. This implementation was designed to be an image processing subsystem that is mounted on a MAV to assist an autopilot system with vision-related tasks. This paper describes the FPGA vision system architecture and algorithms implemented to segment and locate a colored cloth target that specifies the exact landing location. Once the target landing site is identified, the exact location of the landing site is transmitted to the autopilot, which then implements the trajectory adjustments required to autonomously land the MAV on the target. Results of two flight test situations are presented. In the first situation, the MAV lands on a static target. The second situation includes a moving target, which in our tests was the back of a moving vehicle. This FPGA system is an application-specific configuration of the helios robotic vision platform developed at Brigham Young University.
Conference Paper
We present a strategy for obtaining aggressive flight maneuvers similar to what might be produced by a pilot. We use a highly simplified aircraft model that can account for the lift/drag and thrust characteristics of an aircraft under consideration. When thrust is considered an input, this model is differentially flat (and dynamically feedback linearizable) so that the system can be made to follow any reasonable trajectory. Most trajectories will require significant movement of the throttle. Except in close formation, a pilot does not tend to make rapid throttle adjustments. Indeed, many maneuvers are accomplished using a fixed throttle position. The objective is to fly a desired path rather than a (time indexed) trajectory. Although the aircraft model with throttle fixed (thrust predetermined) is no longer differentially flat, we show that the system can still be made to follow any reasonable path. Roughly speaking, an appropriate derivative of the path parametrization variable acts as the missing control allowing flatness-like results to be obtained
Article
A cooperative tracking approach for uninhabited aerial vehicles (UAVs) with camera-based sensors is developed and verified with flight data. The approach utilizes a square root sigma point information filter, which takes important properties for numerical accuracy (square root), tracking accuracy (sigma points), and fusion ability (information). Important augmentations to the filter are also developed for delayed data, by estimating the correlated processes, and moving targets, by using multiple models in a square root interacting multiple model formulation. The final form of the algorithm is general and scales well to any tracking problem with multiple, moving sensors. Flight data using the SeaScan UAV is used to verify the algorithms for stationary and moving targets. Cooperative tracking results are evaluated using multiple test flights, showing excellent results.
Article
We study the problem of tracking control and maneuver regulation control for a nonlinear nonminimum phase control system. First, a tracking controller, consisting of feedforward and static-state feedback, is designed to guarantee uniform asymptotic trajectory tracking. The feedforward is determined by solving a stable noncausal inversion problem. Constant feedback gains are determined based on linear quadratic regulator (LQR) optimization and assumed satisfaction of a robustness inequality. A maneuver regulation controller is obtained from the tracking controller by introducing a suitable state projection that is related to the LQR feedback gains. Properties of the closed loop, including local asymptotic convergence of the transverse errors are described. A multivariable flight control problem is used to demonstrate the approach.