An Area based Technique for Image-to-Image Registration of Multi-Modal Remote Sensing Data
ABSTRACT This paper outlines further tests of the automatic image-to-image registration technique of Bunting et al with the inclusion of joint histogram image similarity metrics. These metrics were expected to increase the accuracy of the algorithms results given that previous literature has highlighted these metrics as optimal for multi-modal registration. But, this study has demonstrated the opposite with the correlation coefficient producing better matching results than the mutual information, kolmogorov distance and distance to independence metrics.
- [show abstract] [hide abstract]
ABSTRACT: A new feature-based approach to automated multitemporal and multisensor image registration is presented. The characteristics of this technique is that it combines moment invariant shape descriptors with modified chain code correlation to establish the correspondences between potential matched regions in two images. It also overcomes the difficulties in control point correspondence in image matching caused by the problem of feature inconsistency. In image segmentation, the authors use the improved Laplacian of Gaussian (LoG) zero-crossing edge detector. Feature matching is done in both feature space and image space based on moment invariant distance and improved chain code correlation. The centers of gravity are then extracted from matched regions and used as control points. The final transformation parameters are estimated based on the final matched control points. Experimental results using multitemporal Landsat TM imagery are presentedGeoscience and Remote Sensing, 1997. IGARSS '97. Remote Sensing - A Scientific Vision for Sustainable Development., 1997 IEEE International; 09/1997
- [show abstract] [hide abstract]
ABSTRACT: A method for automatic image registration which is characterized by its insensitivity to scaling, rotation, and intensity changes is described. The method is based on similarity assessment of the structures in the images and on a check of their spatial arrangement. Pairs of structures that correspond to each other provide sets of control points to geometric mapping functions. An application of the method to remote-sensing image alignment with a reference map is presentedIEEE Transactions on Geoscience and Remote Sensing 06/1990; · 3.47 Impact Factor
- [show abstract] [hide abstract]
ABSTRACT: Multisensor image registration is needed in a large number of applications of remote sensing imagery. The accuracy achieved with usual methods (manual control points extraction, estimation of an analytical deformation model) is not satisfactory for many applications where a subpixel accuracy for each pixel of the image is needed (change detection or image fusion, for instance). Unfortunately, there are few works in the literature about the fine registration of multisensor images and even less about the extension of approaches similar to those based on fine correlation for the case of monomodal imagery. In this paper, we analyze the problem of the automatic multisensor image registration and we introduce similarity measures which can replace the correlation coefficient in a deformation map estimation scheme. We show an example where the deformation map between a radar image and an optical one is fully automatically estimated.IEEE Transactions on Geoscience and Remote Sensing 11/2004; · 3.47 Impact Factor
AN AREA BASED TECHNIQUE FOR IMAGE-TO-IMAGE REGISTRATION OF
MULTI-MODAL REMOTE SENSING DATA
Peter Bunting, Richard Lucas
Institute of Geography and Earth Science
Ceredigion, SY23 2DB, UK
Fr´ ed´ eric Labrosse
Department of Computer Science
Ceredigion, SY23 2DB, UK
To allow remotely sensed datasets to be used for data fusion, either to gain additional insight into the scene or for change
detection, reliable spatial referencing is required . With modern remote sensing systems, reliable registration can be gained
by applying an orbital model or through the use of a global positioning system (GPS) and inertial navigation system (INS) in the
case of airborne and spaceborne datasets respectively. Whilst individually, these datasets appear well registered when compared
to a second dataset from another source (e.g., Optical to LiDAR or Optical to SAR), the resulting images may still be several
pixels out of alignment. Manual registration techniques are often slow and labour intensive and although an improvement in
registration is gained, there can still be some misalignment of the datasets.
A large number of techniques for automating image-to-image registration have therefore been developed , with these
generally implementing one of two approaches. The first (feature matching) aims to extract and match features across the
image as two independent steps [3, 4] whilst the second (area matching) uses a metric to match regions of the image without
explicitly extracting features [5, 6].
This study extended the area matching technique outlined in  by introducing the image similarity metrics based on a joint
histogram. The original study used the correlation coefficient (cc) for identifying matching image regions but previous studies
[2, 8] have demonstrated that joint histogram based techniques are potentially more reliable for matching multi-modal datasets.
The algorithm developed by Bunting et al.,  used a series of steps to produce the final registration, the first of which was to
produce a pair of image pyramids from the original image data of matching image resolutions. Image matching started at the top
of the image pyramid, the lowest resolution, and refined the registration as it descended the pyramid. A regular grid of tie points
was defined in the overlapping region and at each level of the pyramid, where the overlapping region of the images was defined
by the geographic metadata associated with each file. To control the movement of the tie points and maintain the topological
relationships between the neighbouring tie points, a network structure was constructed between the tie points at each level of
the pyramid and between the pyramid levels. Tie points were matched individually where a optimum was identified within a
user-restricted search space formed by the similarity metric and through an exhaustive search. Following the identification of a
new tie point position, the network was updated by propagating the identified transformation to neighbouring tie points using
an inverted weighted distance.
The quantitative assessment of the accuracy of image registration has often proved to be a difficult task  because of the
lack of ground truth data. This study adopted for the same approach and datasets as Bunting et al.,  where two images
pairs for each pair of selected datasets were manually registered and used for testing. The datasets used were 1 m LiDAR to
1 m CASI, 1 m LiDAR to 2.6 m HyMap, 1 m LiDAR to 3 m AIRSAR, 2.6 m HyMap to 1 m CASI, 2.6 m HyMap to 3 m
AIRSAR and 25 m Landsat to 18 m ALOS-PALSAR. Four tests where carried out where the first used single modality image
pairs without any introduced transformation. The other three used the multi-modal image pairs, outlined above, and the second
test introduced no transformation. The third introduced a series of translations in the X and Y axis’, from 2 pixels in the X
axis and 3 pixels in the Y to 16 pixels in the X axis to 18 pixels in the Y axis results. The final test introduced non-linear sine
wavelength independently into the X and Y axis’ with amplitudes of 5, 10 and 15. The results of these tests are given in Table 1,
where the first row provides the results using cc from the previous study while the remaining rows provides the results for the
joint histogram based measures distance to independence (d2i), Kolmgorov distance (kolm) and mutual information (mi). As
the transformation between each image pair was known, the accuracy of the registration could be calculated by generating a
set of correct tie points for each experiment to which the tie points generated by the registration algorithm could be compared.
The comparison was made at the highest pixel resolution of the two images where the linear distance between the produced tie
point to its correct location was calculated and the mean and standard deviation values provided across all tests.
Table 1. The results for automatic image-to-image registration using correlation coefficient, distance to independence, Kolm-
gorov distance and mutual information, where the mean and standard deviation (in pixels) are provided.
These results indicated that, under these condition, the correlation coefficient was the more reliable similarity measure.
Obtaining poorer results for the joint histogram measures was unexpected given that previous studies have suggested that the
correlation coefficient should not work for multi-modal registration [2, 8]. In fact, Inglada and Giros  even suggested the
joint histogram methods, and in particular mutual information, should be more robust than the correlation coefficient under all
circumstances. In this study, it is believed that this discrepancy occurs because of the size of the error when an incorrect result
is returned and the effect of the network used to control the tie points. If a large error is inputted to the network at the top level,
this error will be propagated down through the network and between levels multiplied by the scale factor (between the levels
within the pyramid). Therefore, any large errors entered at the top of the network can significantly move lower levels of the
network out of alignment, generating large errors in the final registration. It is, therefore, considered that the joint histogram
based measures (e.g., mi) create a small number of high magnitude errors when compared to the correlation coefficient and
that these errors, particularly if they occur higher up in the network, are magnified by the network structure. Furthermore, as
a regular grid of tie points is required rather than identifying areas of high similarity, in which tie points are identified, large
errors are less avoidable than in other studies .
 X. Dia and S. Khorram, “The effects of image misregistration on the accuracy of remotely sensed change detection,” IEEE
Transactions on Geoscience and Remote Sensing, vol. 36, no. 5, pp. 1556–1577, 1998.
 B. Zitov´ a and J. Flusser, “Image registration methods: A survey,” Image and Vision Computing, vol. 21, pp. 977 – 1000,
 A. D. Ventura, A. Rampini, and R. Schettini, “Image registration by recognition of corresponding structures,” IEEE
Transactions on Geoscience and Remote Sensing, vol. 28, pp. 305–314, 1990.
 X. Dia and S. Khorram, “Development of feature-based approach to automated image registration for multitemporal and
mutlisensor remotely sensed imagery,” in International Geoscience and Remote Sensing Symposium IGARSS’97, 1997, pp.
 D. I. Barnea and H. F. Silverman, “A class of algorithms for fast digital image registration,” IEEE Transactions on
Computing, vol. 21, no. 179-186, 1972.
 M. Svedlow, C. D. McGillem, and P. E. Anuta, “Experimental examination of similarity measures and preprocessing
methods used for image registration,” in The Symposium on Machine Processing of Remotely Sensed Data, 1976, pp.
 P. Bunting, F. Labrosse, and R. M. Lucas, “An area based technique for image-to-image registration of multi-modal remote
sensing data,” Image and Vision Computing, vol. In Preparation, 2008.
 J. Inglada and A. Giros, “On the possibility of automatic multisensor image registration,” IEEE Transactions on Geoscience
and Remote Sensing, vol. 42, no. 10, pp. 2104 – 2120, 2004.