Conference Paper

Fusion of Visual and Thermal Signatures with Eyeglass Removal for Robust Face Recognition

The University of Tennessee, Knoxville
DOI: 10.1109/CVPR.2004.77 Conference: Computer Vision and Pattern Recognition Workshop, 2004 Conference on
Source: IEEE Xplore

ABSTRACT This paper describes a fusion of visual and thermal infrared (IR) images for robust face recognition. Two types of fusion methods are discussed: data fusion and decision fusion. Data fusion produces an illumination-invariant face image by adaptively integrating registered visual and thermal face images. Decision fusion combines matching scores of individual face recognition modules. In the data fusion process, eyeglasses, which block thermal energy, are detected from thermal images and replaced with an eye template. Three fusion-based face recognition techniques are implemented and tested: Data fusion of visual and thermal images (Df), Decision fusion with highest matching score (Fh), and Decision fusion with average matching score (Fa). A commercial face recognition software FaceIt® is used as an individual recognition module. Comparison results show that fusion-based face recognition techniques outperformed individual visual and thermal face recognizers under illumination variations and facial expressions.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: This paper is dedicated to multispectral facial image recognition, using decision fusion of neural classifiers. The novelty of this paper is that any classifier is based on the model of Concurrent Self-Organizing Maps (CSOM), previously proposed by first author of this paper. Our main achievement is the implementation of a real time CSOM face recognition system using the decision fusion that combines the recognition scores generated from visual channels {(R, G, and B) or Y} with a thermal infrared classifier. As a source of color and infrared images, we used our VICFACE database of 38 subjects. Any picture has 160 times 120 pixels; for each subject there are pictures corresponding to various face expressions and illuminations, in the visual and infrared spectrum. The spectral sensitivity of infrared images corresponds to the long wave range of 7.5 - 13 mum. The very good experimental results are given regarding recognition score.
    Advanced Video and Signal Based Surveillance, 2007. AVSS 2007. IEEE Conference on; 10/2007
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Recently, a number of studies have demonstrated that thermal infrared (IR) imagery offers a promising alternative to visible imagery in face recognition problems due to its invariance to visible illumination changes. However, thermal IR has other limitations including that it is opaque to glass. As a result, thermal IR imagery is very sensitive to facial occlusion caused by eyeglasses. Fusion of the visible and thermal IR images is an effective way to solve this problem. In this paper, using the face reconstruction information of the visible images, we propose a nonlinear eyeglasses removing algorithm which can successfully reconstruct the thermal images. Experiments on publicly available data set show the excellent performance of our algorithm.
    Information Fusion, 2007 10th International Conference on; 08/2007
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: The objective of this work is to recognize faces using sets of images in visual and thermal spectra. This is challenging because the former is greatly affected by illumination changes, while the latter frequently contains occlusions due to eye-wear and is inherently less discriminative. Our method is based on a fusion of the two modalities. Specifically: we examine (i) the effects of preprocessing of data in each domain, (ii) the fusion of holistic and local facial appearance, and (iii) propose an algorithm for combining the similarity scores in visual and thermal spectra in the presence of prescription glasses and significant pose variations, using a small number of training images (5-7). Our system achieved a high correct identification rate of 97% on a freely available test set of 29 individuals and extreme illumination changes.
    Computer Vision and Pattern Recognition Workshop, 2006. CVPRW '06. Conference on; 07/2006


1 Download
Available from

Seong G. Kong