Conference Paper

Fusion of Visual and Thermal Signatures with Eyeglass Removal for Robust Face Recognition

The University of Tennessee, Knoxville
DOI: 10.1109/CVPR.2004.77 Conference: Computer Vision and Pattern Recognition Workshop, 2004 Conference on
Source: IEEE Xplore


This paper describes a fusion of visual and thermal infrared (IR) images for robust face recognition. Two types of fusion methods are discussed: data fusion and decision fusion. Data fusion produces an illumination-invariant face image by adaptively integrating registered visual and thermal face images. Decision fusion combines matching scores of individual face recognition modules. In the data fusion process, eyeglasses, which block thermal energy, are detected from thermal images and replaced with an eye template. Three fusion-based face recognition techniques are implemented and tested: Data fusion of visual and thermal images (Df), Decision fusion with highest matching score (Fh), and Decision fusion with average matching score (Fa). A commercial face recognition software FaceIt® is used as an individual recognition module. Comparison results show that fusion-based face recognition techniques outperformed individual visual and thermal face recognizers under illumination variations and facial expressions.

15 Reads
  • Source
    • "An optimized wavelet domain fusion is proposed, which has the best performance with an accuracy of 95.8%. In [110], data fusion and decision fusion schemes are implemented based on commercial face recognition software, FaceIt, and a method is introduced to detect glasses regions in a thermal image and to replace them with template eye patterns. Experiments conclude: 1) decision fusion with average matching score has a superior recognition accuracy over data fusion, and 2) glasses removal gives great improvements on thermal and data fusion face recognition. "
    [Show abstract] [Hide abstract]
    ABSTRACT: High performance for face recognition systems occurs in controlled environments and degrades with variations in illumination, facial expression, and pose. Efforts have been made to explore alternate face modalities such as infrared (IR) and 3-D for face recognition. Studies also demonstrate that fusion of multiple face modalities improve performance as compared with singlemodal face recognition. This paper categorizes these algorithms into singlemodal and multimodal face recognition and evaluates methods within each category via detailed descriptions of representative work and summarizations in tables. Advantages and disadvantages of each modality for face recognition are analyzed. In addition, face databases and system evaluations are also covered.
    IEEE Transactions on Human-Machine Systems 12/2014; 44(6):701-716. DOI:10.1109/THMS.2014.2340578 · 1.98 Impact Factor
  • Source
    • "The fusion in biometric system can take place at various levels. Sensor level fusion[11] [12], modalities obtained from different sensors are fused together. Feature level fusion [13] takes place after extracting the features from biometric templates and then concatenating the features into a fused feature vector of higher dimension. "
    [Show abstract] [Hide abstract]
    ABSTRACT: In real life, images obtained from video cameras or scanners are usually exposed to different levels of noises and blurring effects. In this paper we propose a new robust score level fusion technique to recognize faces in the presence of noise and blurring effects. The Proposed Score Level Fusion Technique (PSLFT) is obtained by using combinatory approach and Z-Score normalization using the scores obtained from appearance based techniques: Principal Component Analysis (PCA), Fisher faces (FF), Independent Component Analysis (ICA), Fourier Spectra (FS), Singular Value Decomposition (SVD) and Sparse Representation (SR). The system is trained in the absence of noise, blurring effect but tested by imposing different levels of noises and blurring effects thus we have tried to imitate the real world scenarios. To investigate the performance of PSLFT, we simulate the real world scenario by adding noises: Median noise, Salt and pepper noise and also adding blurring effects: Motion blur and Gaussian blur. To evaluate performance of the PSLFT, we have considered six standard public face databases: IITK, ATT, JAFEE, CALTECH, GRIMANCE, and SHEFFIELD.
    Proceedings of the 2013 2nd International Conference on Advanced Computing, Networking and Security; 12/2013
  • Source
    • "Decision Level Fusion for 3D Face Recognition [23] Discriminant Analysis (LDA) based and a Support Vector Machine (SVM) based algorithm. J. Heo et al. [21], use data and decision fusion for robust face recognition of visual and thermal images. Data Fusion produces an illuminationinvariant face image by detecting the eyeglass, which blocks the thermal energy, and replaced them with an eye template from visual and thermal face images, and Decision Fusion combines matching scores of individual face recognition modules. "
    [Show abstract] [Hide abstract]
    ABSTRACT: This paper demonstrates two different fusion techniques at two different levels of a human face recognition process. The first one is called data fusion at lower level and the second one is the decision fusion towards the end of the recognition process. At first a data fusion is applied on visual and corresponding thermal images to generate fused image. Data fusion is implemented in the wavelet domain after decomposing the images through Daubechies wavelet coefficients (db2). During the data fusion maximum of approximate and other three details coefficients are merged together. After that Principle Component Analysis (PCA) is applied over the fused coefficients and finally two different artificial neural networks namely Multilayer Perceptron(MLP) and Radial Basis Function(RBF) networks have been used separately to classify the images. After that, for decision fusion based decisions from both the classifiers are combined together using Bayesian formulation. For experiments, IRIS thermal/visible Face Database has been used. Experimental results show that the performance of multiple classifier system along with decision fusion works well over the single classifier system.
Show more

Preview (2 Sources)

15 Reads
Available from

Seong G. Kong