Ravi K. Sharma’s research while affiliated with Oregon Institute of Technology and other places

We describe an approach for registration of images from different sensors that may generate local polarity reversals and inconsistent features. The developed technique is based on modifying the objective function by transforming the images into a representation that is less dependent on the local image intensity, and using an iterative process that reduces the effect of inconsistent features.

We present a probabilistic method for fusion of images produced by multiple sensors. The approach is based on an image formation model in which the sensor images are noisy, locally linear functions of an underlying true scene (latent variable). A Bayesian framework then provides for maximum likelihood or maximum a posteriori estimates of the true scene from the sensor images. Least squares estimates of the parameters of the image formation model involve (local) second order image statistics, and are related to local principal component analysis. We demonstrate the efficacy of the method on images from visible-band and infrared sensors.

In this paper, we describe an approach for registration of video sequences from a suite of multiple sensors including television, infrared and radar. Video sequences generated by these sensors may contain abrupt changes in local contrast and inconsistent image features, which pose additional difficulties for registration. Our approach to registration addresses the difficulties caused by using multiple sensors. We use a representation for registration that is invariant to local contrast changes, followed by smoothing of the resulting error measure used for registration, for robust estimation of registration parameters. We use an iterative procedure to reduce the effect of inconsistent features. Finally, we describe a method that uses samesensor registration to aide in performing registration of sequences of video frames across multiple sensors. 1 Introduction Image registration is a critical component of multi-sensor systems that typically use different types of sensors. An example of...

We present a probabilistic method for fusion of images produced by multiple sensors. The approach is based on an image formation model in which the sensor images are noisy, locally linear functions of an underlying, true scene. A Bayesian framework then provides for maximum likelihood or maximum a posteriori estimates of the true scene from the sensor images. Maximum likelihood estimates of the parameters of the image formation model involve (local) second order image statistics, and thus are related to local principal component analysis. We demonstrate the efficacy of the method on images from visible-band and infrared sensors. 1 Introduction Advances in sensing devices have fueled the deployment of multiple sensors in several computational vision systems [1, for example]. Using multiple sensors can increase reliability with respect to single sensor systems. This work was motivated by a need for an aircraft autonomous landing guidance (ALG) system [2, 3] that uses visible-b...

A multisensor suite that consists of a forward looking infrared camera, a radar, and a low light television camera is likely to be a useful component of enhanced and synthetic vision systems. We examine several aspects of signal processing needed to combine effectively the individual sensor outputs. First we discuss transformations of individual sensor image data to a common representation. Our focus is on rectification of radar data without relying on a flat-earth assumption. We then describe a novel approach to image representation that minimizes loss of information in the transformation process. Finally, we discuss an optimal algorithm for fusion of radar and infrared images. Keywords: sensor fusion, multiresolution representation, conformal representation, imaging radar rectification. 1 INTRODUCTION Future enhanced and synthetic vision systems will most likely rely on multiple sources of information that will include millimeter wave radar (MMWR), forward looking infrared (FLIR),...

We describe a sensor fusion algorithm based on a set of simple assumptions about the relationship among the sensors. Under these assumptions we estimate the common signal in each sensor, and the optimal fusion is then approximated by a weighted sum of the common component in each sensor output at each pixel. We then examine a variety of techniques to map the sensor signals onto perceptual dimensions (e.g., color), such that the human operator can benefit from the enhanced fused image, and simultaneously, be able to identify the source of the information. We examine several color mapping schemes. Keywords: sensor fusion, image fusion, color fusion, color vision, color mapping, multisensor fusion, enhanced vision 1. INTRODUCTION The efficiency, robustness, and safety of many visually guided systems, such as flight control, can be improved by providing the operator with the necessary visual information at all times, even in low visibility conditions. One way to achieve this objective, i...

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : xii 1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1 1.1 Single Sensor Computational Vision System . . . . . . . . . . . . . . . . . . 3 1.2 Multisensor Image Fusion System . . . . . . . . . . . . . . . . . . . . . . . . 4 1.2.1 Application of fusion for navigation guidance in aviation . . . . . . . 7 1.3 Issues in Fusing Imagery from Multiple Sensors . . . . . . . . . . . . . . . . 10 1.3.1 Mismatch in features . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.3.2 Combination of multisensor images . . . . . . . . . . . . . . . . . . . 11 1.3.3 Geometric representations of imagery from different sensors . . . . . 12 1.3.4 Registration of misaligned image features . . . . . . . . . . . . . . . 12 1.3.5 Spatial resolution of different sensors . . . . . . . . . . . . . . . . . . 13 1.3.6 Differences in frame rate . . . . . . . . . . . . . . . . . . . . . ....

We present an approach for fusion of video streams produced by multiple imaging sensors such as visible-band and infrared sensors. Our approach is based on a model in which the sensor images are noisy, locally affine functions of the true scene. This model explicitly incorporates reversals in local contrast, sensor-specific features and noise in the sensing process. Given the parameters of the local affine transformations and the sensor images, a Bayesian framework provides a maximum a posterior estimate of the true scene. This estimate constitutes the rule for fusing the sensor images. We also give a maximum likelihood estimate for the parameters of the local affine transformations. Under Gaussian assumptions on the underlying distributions, estimation of the affine parameters is achieved by local principal component analysis. The sensor noise is estimated by analyzing the sequence of images in each video stream. The analysis of the video streams and the synthesis of the fused stream is performed in a multiresolution pyramid domain.

We describe a sensor fusion algorithm based on a set of simple assumptions about the relationship among the sensors. Under these assumptions we estimate the common signal in each sensor, and the optimal fusion is then approximated by a weighted sum of the common component in each sensor output at each pixel. We then examine a variety of techniques to map the sensor signals onto perceptual dimensions, such that the human operator can benefit from the enhanced fused image, and simultaneously, be able to identify the source of the information. We examine several color mapping schemes.© (1997) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.