Harvey Mitchell

• MitchellDataFusion

In Chapt. 1 we introduced a formal framework in which we represented a multi-sensor data fusion system as a distributed system of autonomous modules. In this chapter we shall consider the architecture of a multi-sensor fusion system and, in particular, the architecture of the “data fusion block” (Sect. 1.4). The modules in the data fusion block are...

The subject of this chapter is sequential Bayesian inference in which we consider the Bayesian estimation of a dynamic system which is changing in time. Let θ k denote the state of the system, i. e. a vector which contains all relevant information required to describe the system, at some (discrete) time k. Then the goal of sequential Bayesian infer...

In this chapter we consider parameter estimation, which is our first application of Bayesian statistics to the problem of multi-sensor data fusion. Let \(\mathbf{y}=\bigl(\mathbf{y}_{i}^{T},\mathbf{y}_{2}^{T},\ldots,\) \(\mathbf{y}_{N}^{T}\bigr)^{T}\) denote a set of N multi-dimensional sensor observations y i ,i ∈ {1,2,…,N}, where \(\mathbf{y}_{i}...

In this chapter we consider the sensors. These are special devices which interact directly with the environment and which are ultimately the source of all the input data in a multi-sensor data fusion system [12]. The physical element which interacts with the environment is known as the sensor element and may be any device which is capable of percei...

In this chapter we give an overview of Bayesian statistics, and in particular, the methods of Bayesian inference as used in multi-sensor data fusion. The basic premise of Bayesian statistics is that all unknowns are treated as random variables and that the knowledge of these quantities is summarized via a probability distribution. The main advantag...

In this chapter we complete our study of multi-sensor data fusion by analyzing the control application/resource management block shown in Fig. 1.1. To make our discussion more concrete, we shall consider the case when decisions made by the control application are fed back to the sensors. In this case, the control block is more commonly known as a s...

The subject of this chapter, and the one that follows it, is Bayesian decision theory and its use in multi-sensor data fusion. To make our discussion concrete we shall concentrate on the pattern recognition problem [19] in which an unknown pattern, or object, O is to be assigned to one of K possible classes {c 1, c 2, ... , c K }. In this chapter w...

In this chapter we shall consider the subject of robust statistics and, in particular, robust parameter estimation. Robust statistics is defined as the study of statistical methods which are relatively insensitive to the presence of outliers, i. e. input data which is “strange” or “incompatible” with the remaining input data. It might be thought th...

The subject of this chapter is radiometric calibration, or normalization, which we define as the conversion of all sensor values to a common scale. This is the fourth and last function listed in Sect. 4.1 which is required for the formation of a common representational format. Although conceptually the radiometric normalization and semantic alignme...

The subject of this chapter is semantic alignment. This is the conversion of multiple input data or measurements which do not refer to the same object, or phenomena, to a common object or phenomena. The reason for performing semantic alignment is that different inputs can only be fused together if the inputs refer to the same object or phenomena. I...

The subject of this chapter is temporal alignment, or registration, which we define as the transformation T(t) which maps local sensor observation times t to a common time axis t ′. Temporal alignment is one of the basic processes required for creating a common representational format. It often plays a critical role in applications involving in man...

Introduction.- Sensors.- Architecture.- Common Representational Format.- Spatial Alignment.- Temporal Alignment.- Semantic Alignment.- Radiometric Normalization.- Bayesian Inference.- Parameter Estimation.- Robust Statistics.- Sequential Bayesian Inference.- Bayesian Decision Theory.- Ensemble Learning.- Sensor Management.

The subject of this chapter is spatial alignment. In image fusion this is defined as the process of geometrically aligning two or more images of the same scene acquired at different times (multi-temporal fusion), or with different sensors (multi-modal fusion), or from different viewpoints (multi-view fusion). It is a crucial pre-processing operatio...

The subject of this chapter is image fusion using the methods of ensemble learning. Ensemble learning is a method for constructing accurate predictors or classifiers from an ensemble of weak predictors or classifiers. In the context of image fusion, we use the term ensemble learning to denote the fusion of K input images I k ,k ∈ {1,2, . . .,K}, wh...

The subject of this chapter is the common representational format. Conversion of all sensor observations to a common format is a basic requirement for image fusion. The reason for this is that only after conversion to a common format are the input images compatible, i. e. the input images “speak a common language” and image fusion may be performed....

The subject of this chapter is image sub-space techniques. These techniques are a special class of image transformations whose effect is project the input image into a lower dimensional space or sub-space. We shall concentrate on statistical sub-space methods which rely on a covariance matrix which is constructed from the input images. The techniqu...

The subject of this chapter is a collection of miscellaneous effects which affect the brightness and color perception in a input image or in a input video. For the sake of concreteness, we shall concentrate on three important effects: Vignetting, automatic gain control and white balance. Vignetting we define as a positiondependent loss of brightnes...

This textbook provides a comprehensive introduction to the theories, techniques and applications of image fusion. It is aimed at advanced undergraduate and first-year graduate students in electrical engineering and computer science. It should also be useful to practicing engineers who wish to learn the concepts of image fusion and use them in real-...

The subject of this chapter is color image spaces. In the chapter we provide a brief summary of the different color spaces.

The subject of this chapter is pan-sharpening. Present-day remote sensors produce multi-spectral images with low spatial resolution and panchromatic images with high spatial resolution. Pan-sharpening is an image fusion application in which we we generate a multi-spectral image with high spatial resolution by fusing together the multi-spectral and...

The subject of this chapter is image fusion techniques which rely on simple pixel-by-pixel operations. The techniques include the basic arithmetic operations, logic operations and probabilistic operations as well as slightly more complicated mathematical operations. The image values include pixel gray-levels, feature map values and decision map lab...

The subject of this chapter is the STAPLE (Simultaneous Truth and Performance Level Estimation) algorithm. This is a method for fusing together several segmented images and is based on the expectation-maximization (EM) algorithm.

The subject of this chapter is radiometric calibration. This is the conversion of the input image values to a common radiometric scale. The transformation to such a scale is of critical importance in image fusion. Without a common radiometric base it is not possible to fuse images which were acquired at different illuminations, or under different a...

The subject of this chapter is image thresholding in which we transform an input image, A, into a binary image B, where the pixel gray-levels in B are restricted to {0,1}. If a m is the gray level of the mth pixel in A, then the corresponding value in B is $$ b_{m}=\left\{ \begin{array}{ll} 1 & \mbox{if $a_{m} \geq t_{m}$} \;, \\ 0 & \mbox{otherwis...

In this chapter we provide a brief overview of biometric technology and in particular, multi-modal biometric technology.

The subject of this chapter is multi-resolution analysis for images. We shall concentrate on the discrete wavelet transform (DWT) which provide a framework for the multi-resolution analysis of an input image by decomposing an input image into a sequence of wavelet planes and a residual image. We start by giving a brief review of multi-resolution a...

The subject of this chapter is image similarity measures. These measure provide a quantitative measure of the degree of match between two images, or image patches, A and B. Image similarity measures play an important role in many image fusion algorithms and applications including retrieval, classification, change detection, quality evaluation and r...

The subject of this chapter is image key points which we define as a distinctive point in an input image which is invariant to rotation, scale and distortion. In practice, the key points are not perfectly invariant but they are a good approximation. To make our discussion more concrete we shall concentrate on two key point algorithms: SIFT and SURF...

The subject of this chapter is ensemble color image segmentation. This is an image fusion application in which combine several simple image segmentation algorithms to obtain a state-of-the-art image segmentation algorithm. The goal of image segmentation is to decompose the input image into a set of meaningful or spatially coherent regions sharing...

The subject of the present chapter is the objective assessment of the image quality of the output image in image fusion. A brief review of the different quality measures is given.

The subject of this chapter is the image sensor or camera. This is a special device which interacts directly with the environment and is ultimately the source of all the input data in an image fusion system [3]. The image sensor may be any device which is capable of perceiving a physical property, or environmental attribute, such as heat, light, so...

The subject of this chapter is the Markov Random Field (MRF) and its use in image fusion. A Markov random field is a probabilistic model defined by local conditional probabilities. Markov random field (MRF) theory thus provides a convenient and consistent way for modeling context dependent entities such as image pixels and correlated features. Cont...

"This textbook provides an introduction to the theories and techniques of multi-sensor data fusion. No previous knowledge of multi-sensor data fusion is assumed, although some familiarity with the basic tools of linear algebra, calculus and simple probability theory is recommended. This book is illustrated with many applications and contains a list...

Type-2 fuzzy sets are a generalization of the ordinary fuzzy sets in which each type-2 fuzzy set is characterized by a fuzzy membership function. In this paper, we consider the problem of ranking a set of type-2 fuzzy numbers. We adopt a statistical viewpoint and interpret each type-2 fuzzy number as an ensemble of ordinary fuzzy numbers. This enab...

Type-2 fuzzy sets are a generalization of the ordinary fuzzy sets in which each fuzzy set is characterized by a fuzzy membership function. In this article we consider how to define the correlation coefficient between two type-2 fuzzy sets. We have adopted the embedded function model and interpret each type-2 fuzzy set as a weighted ensemble of ordi...

The OWA (Ordered Weighted Average) operator is a powerful non-linear operator for aggregating a set of inputs ai,i∈{1,2,…,M}. In the original OWA operator the inputs are crisp variables ai. This restriction was subsequently removed by Mitchell and Schaefer who by application of the extension principle defined a fuzzy OWA operator which aggregates a...

Intuitionistic fuzzy sets are a generalization of ordinary fuzzy sets which are characterized by a membership function and a non-membership function. We consider the problem of ranking a set of intuitionistic fuzzy numbers. We adopt a statistical viewpoint and interpret each intuitionistic fuzzy number as an ensemble of ordinary fuzzy numbers. This...

Intuitionistic fuzzy sets are a generalization of the ordinary fuzzy sets in which we have both a membership function μ and a nonmembership function ν. In this article we consider the problem of defining a correlation coefficient between two intuitionistic fuzzy sets. We show that by interpreting an intuitionistic fuzzy set as an ensemble of ordina...

Recently Dengfeng and Chuntian defined the first operational definition of a similarity measure for intuitionistic fuzzy sets and showed how it may be used in pattern recognition problems. Unfortunately the Dengfeng and Chuntian operator may give counter-intuitive results. We show how a simple modification of the Dengfeng–Chuntian operator may corr...

Recently, a crisp robust median-of-intercept (MI) straight-line fitting algorithm was devised for use in image-processing applications. The algorithm is specifically designed for use in noisy images when the input data is corrupted with both noise and outliers. In this article we describe a fuzzy MI algorithm whose performance is significantly bett...

The Weighted Ordered Weighted Average (WOWA) operator is a powerful operator used for aggregating a set of M input arguments which may derive from different sources. The WOWA operator allows the user to take into account both the importance or reliability of the different information sources and the relative position of the argument values. We desc...

The median adaptive predictor (MAP) is widely used in many modern-day lossless picture compression algorithms, including the new JPEG-LS international standard. The basis of MAP is the crisp median operator. In this article we describe how we may fuzzify the MAP and obtain two new “soft” predictors. We compare the performance of the new predictors...

Most of the current lossless compression algorithms, including the new international baseline JPEG-LS algorithm, do not exploit the interspectral correlations that exist between the color planes in an input color picture. To improve the compression performance (i.e., lower the bit rate) it is necessary to exploit these correlations. A major concern...

Solving the assignment problem, i.e. correctly associating target measurements to tracks, is critical to the correct working of any multiple-target tracking system. Early assignment algorithms did not account for missing tracks and observations and thus had poor performance. In order to raise the performance level, the modern approach is to include...

The K-Nearest Neighbor (K-NN) voting scheme is widely used in problems requiring pattern recognition or classification. In this voting scheme an unknown pattern is classified according to the classifications of its K nearest neighbors. If a majority of the K nearest neighbors have a given classification C*, then the unknown pattern is also given th...

At the heart of many statistical processing algorithms lies the concept of ordering a set of crisp numbers, either according to their own values (“direct” sorting), or according to the values of a second set of numbers (“indirect” sorting). In this paper we show how the concept of direct and indirect sorting may be generalized to fuzzy numbers. We...

Point-matching involves the matching of pairs of points from two sets of partially correlated points. It is an important task which is used in many different areas of signal processing. Although it is possible to perform point-matching using a brute-force algorithm, the high computational complexity makes it unfeasible even for a moderate number of...

The ordered weighted averaging (OWA) operator of Yager was introduced to provide a method for nonlinearly aggregating a set of input arguments ai. A fundamental aspect of the OWA operator is a reordering step in which the input arguments are rearranged according to their values. Recently, an induced OWA operator was described in which each input ar...

The removal of additive noise in digital images is currently a subject of intense interest. Recent research has shown that to be effective the filter should adapt itself to the local structure existing in the image: In regions which have no dominant structure the adaptive filter should act as a linear statistics filter while in regions with a stron...

Picture compression algorithms, using a parallel structure of neural networks, have recently been described. Although these algorithms are intrinsically robust, and may therefore be used in high noise environments, they suffer from several drawbacks: high computational complexity, moderate reconstructed picture qualities, and a variable bit-rate. I...

The ordered weighting averaging (OWA) Operator of Yager was introduced to provide a method for nonlinearly aggregating a set of input arguments ai. A fundamental aspect of the OWA operator is a reordering step in which the input arguments are rearranged according to their values. Recently, a generalized OWA operator was described in which each inpu...

Robust signal processing algorithms rely on integer ordered statistics. This means that such algorithms cannot be used in a fuzzy environment where variables have fuzzy ranks. In this paper, we show how to calculate the kth order statistic for a set of arguments with fuzzy rank and thereby use classical robust signal processing algorithms in a fuzz...

Robust signal processing algorithms rely on integer ordered statistics. This means that such algorithms cannot be used in a fuzzy environment where variables have fuzzy ranks. In this paper, we show how to calculate the kth order statistic for a set of arguments with fuzzy rank and thereby use classical robust signal processing algorithms in a fuzz...

The ordered weighted averaging (OWA) operator of Yager was introduced to provide a method for aggregating several inputs which lies between the max and min operators. The fundamental aspect of the OWA operator is a reordering step in which the input arguments are rearranged according to their integer ranks. In this paper, we generalize the OWA oper...

Differential pulse code modulation (DPCM) algorithms are widely used to compress gray-scale pictures. A critical element in a DPCM algorithm is the accurate prediction of the pixel gray-levels in the input picture. The switched predictors used in modern DPCM algorithms are generally accurate when the input picture is free of noise. However, even sm...

Many multi-target tracking systems work by continually updating the target tracks on the basis of received target measurements. This involves solving the assignment problem, i.e. finding the maximum set of track-to-measurement associations with the largest overall likelihood. To minimize the number of association errors, it is traditional to includ...

Recently multilayer neural networks have been used for still picture compression. In these networks it is necessary to normalize the gray levels in the input picture before they are fed into the neural network. In this paper we investigate six different normalization functions, of which four are new and appear for the first time in this paper. We s...

The ordered weighted averaging (OWA) operator of Yager was introduced to provide a method for aggregating several inputs which lies between the Max and Min operators. The fundamental aspect of the OWA operator is a reordering step in which the input arguments are re-arranged according to their actual relative value. In this paper we describe a modi...

A new single-structure image compression neural network is described. We find that the new network, based on a new normalization scheme, consistently outperforms other presently available single-structure networks and has a performance similar to that obtained with much more complicated multiple-structure neural networks and hierarchical neural net...

Recently Oshri et. al. described a three-level interpolative Block Truncation Coding image compression algorithm. The interpolative function used by Oshri was a simple Laplacian filter. In this paper we describe the design of a new interpolation function specifically designed for this purpose using a functional-link neural network. We show that by...

For a high-quality real-time image compression at moderate bit-rates the equi-spaced 3-level block truncation coding algorithms is an attractive coding method. Unfortunately, the present-day equi-spaced 3-level algorithms are not optimum (in the mean square error sense). In this paper we describe a new equi-spaced 3-level algorithm which is fast an...

Many different versions of the block truncation coding image compression algorithm exist. In this paper we compare the different versions from the viewpoint of reconstructed image quality and computational complexity.

Block Truncation Coding-Vector Quantization (BTC-VQ) is a simple, fast, non-adaptive block-based image compression technique with a moderate bit-rate of 1.0 bit/pixel. By making the algorithm adaptive it is possible to further lower the bit-rate. In this paper we show how the algorithm may optimally adapt itself to the local nature of the picture w...

Zeng (see ibid., vol.27, p.1126-8, 1991) has described an interpolative two-level block truncation coding (BTC) image compression algorithm. The authors extend the work of Zeng to a three-level interpolative BTC algorithm. Compared to Zeng, the new algorithm gives reconstructed images with substantially higher image quality with no increase in bit...

The authors extend the analysis of the block truncation coding (BTC) algorithm using a Hopfield neural network (HNN). They show that its performance is suboptimum (in the mean square error sense) and that alternative (non-neural network) BTC algorithms are available with virtually the same performance.

Commonly used gradient edge operators such as the Sobel, Prewitt and Roberts operators all required a square root operation; this is, however, computationally intensive and, consequently, simple but very inaccurate approximations are often used instead. The author describes a new square root algorithm specifically designed for use with these edge o...

Noise smoothing is a basic operation in image processing. Numerous filters, each to be used for different noise conditions and picture types, have been proposed in the literature. A comparison study showed that the K-Nearest Neighbour filter performs extremely well for both additive noise and multiplicative noise; especially when applied in an iter...

Block truncation coding-vector quantization (BTC-VQ) is an extremely simple non-adaptive block-based image compression technique. It has a relatively low compression ratio; however, the simplicity of the algorithm makes it an attractive option. Its main drawback is the fact that the reconstructed pictures suffer from ragged edges. In this paper we...