Conference PaperPDF Available

Meta Fusion of Trained Heterogeneous Image Classifier using Classifier Selection

Authors:

Abstract

Several fusion methods are used to integrate different trained classification models to generalize classification over the distributed dataset. This paper proposes a Meta fusion approach for integrating trained heterogeneous image classifier using classifier selection. In this approach, classifier selector selects a trained classifier from the set of trained classifier on distributed heterogeneous data. The selected model classifies an input image into its predicted class. Integrated model, based on Meta fusion approach, of these trained classifier works as a generalized classification model for combined distributed heterogeneous image data. Meta fusion approach is verified on MNIST image dataset in an environment of distributed heterogeneous data and achieves competitive results.
Meta Fusion of Trained Heterogeneous Image
Classifier using Classifier Selection
Suresh Prasad Kannojia
ICT Research Lab, Department of Computer Science,
University of Lucknow, Lucknow, India
Email: spkannojia@gmail.com
Gaurav Jaiswal
ICT Research Lab, Department of Computer Science,
University of Lucknow, Lucknow, India
Email: gauravjais88 @gmail.com
AbstractSeveral fusion methods are used to integrate
different trained classification models to generalize classification
over the distributed dataset. This paper proposes a Meta fusion
approach for integrating trained heterogeneous image classifier
using classifier selection. In this approach, classifier selector
selects a trained classifier from the set of trained classifier on
distributed heterogeneous data. The selected model classifies an
input image into its predicted class. Integrated model, based on
Meta fusion approach, of these trained classifier works as a
generalized classification model for combined distributed
heterogeneous image data. Meta fusion approach is verified on
MNIST image dataset in an environment of distributed
heterogeneous data and achieves competitive results.
Keywordsmeta fusion, heterogeneous image classifier,
classifier selection, image classification, CNN
I. INTRODUCTION
In the present scenario, image data are generated and
stored in the distributed location. These distributed data can
be homogeneous or heterogeneous. Learning of image
classifier on this distributed small dataset consumes less
computing power and time. Different fusion and integration
approaches are available for combining these trained models
to generalize the combine classification of whole data.
Fusion approaches are divided into three categories
according to when and where fusion techniques are applied
in the image classification process [1]. Pre classification
inputs are combined with early fusion approaches while post
classification outputs are combined in late fusion approaches.
Mid level fusion approaches combine the features of the
different classifier to exploit the benefit of multiple features.
In literature, for image level fusion, Rokni et al. [2]
explore the different pixel level image fusion techniques in
application of surface water change detection and Xiao et al.
[3] propose an integration of object level and part level
attention of images. Gunes et al. [4] investigate the effect of
recognition using early fusion and late fusion approaches.
For mid level feature fusion, Amarsaikhan et al. [5] explore
the different data and feature level fusion for image
classification and Fernando et al. [6] present a logistic based
fusion method which adaptively selects set of diverse and
complementary visual features. For integration of
homogeneous image classifier, Kannojia et al. [7] propose
the ensemble of CNN-ELM classifier for image
classification. This ensemble model combines the trained
classifier using majority voting of prediction.
Distributed set of homogeneous data contains samples of
each class in each partition of distributed data while in
heterogeneous distributed dataset, each partition contains
samples of the different set of classes. No class is overlapped
in each partitioned dataset. Therefore, combining of trained
classifier on these dataset is difficult as these heterogeneous
dataset does not have samples of similar classes.
Figure 1 (A) Different trained Image classifier and (B) their integration by Meta fusion approach
To overcome the problem of combining trained classifier
on heterogeneous dataset, a Meta fusion approach using
classifier selection is proposed. In this approach, classifier
selector selects the appropriate classifier for corresponding
input data. Trained classifier and Meta fusion approach is
shown in Figure 1. This Meta fusion approach has been
verified on the integration of trained convolutional neural
network models, which are trained on heterogeneous sub
dataset of MNIST dataset [8].
Remaining paper is organized as follow: Section II
describes the proposed Meta fusion approach of trained
image classifier using classifier selection. In section III, the
proposed Meta fusion approach is implemented and verified
on the integration of trained image classifier of
heterogeneous sub dataset of MNIST dataset. Experimental
results have been evaluated on performance metrics and
compared with basic and homogeneous fusion methods.
Section IV concludes the paper. Notations used in this paper
are given Table I.
Table I Symbols and their description
Sr.
No.
Symbols
Description
1
D1, D2, ... , Dn
Heterogeneous Sub Dataset
2
C1, C2, ... , Cn
Trained Image Classifier
3
P1, P2, ... , Pn
Predicted Class Set
4
Cselect
Classifier Selector
5
MNIST1, MNIST2
Heterogeneous Sub Dataset of
MNIST dataset
II. PROPOSED APPROACH
For given image dataset D1, D2, ..., Dn and their
predicted predefined class set P1, P2, ..., Pn, the
heterogeneous dataset follows condition that each class
belongs to P1, P2, ..., Pn does not overlap. For example, if a
dataset contains images of different type flowers and another
dataset contains images of different breed dogs, these
combined datasets are heterogeneous which contains the
different class images, and no common class belongs to both
dataset. This can be understood mathematically in (1) as:
      (1)
where is class value and  is predicted class set.
Suppose C1, C2, ..., Cn are trained image classifier which
is trained on distributed heterogeneous dataset D1, D2, ...,
Dn respectively. The proposed Meta fusion approach for
integrating trained image classifier selects the trained
classifier which is appropriate and trained on the
corresponding input image. Selection of classifier is made by
Classifier selection model. For training of this model, super
set of D1, D2, ..., Dn is generated with a select vector which
is mapped to classifier number. The classifier selection
model is trained on this super set data.
Working flow of this Meta fusion approach is the same as
traditional image classification process. Here, combined
heterogeneous data works as input image data, the pool of
trained image classifier with classifier selector works as a
classification model. When an input image is given to this
integrated classification model, classifier selector selects the
trained image classifier from the pool of trained classifier
according to the input image. This classifier classifies an
input image into its output class. The algorithm of Meta
fusion of trained image classifier using classifier selection is
shown in Algorithm 1. A detailed block diagram of this Meta
fusion approach is depicted in Figure 2.
Algorithm 1 Algorithm for Meta fusion of trained image classifier using classifier selection
Algorithm for Meta Fusion of Trained Image classifier using Classifier Selection:
INPUT: Image Dataset [D1, D2, ..., Dn] , Trained image classifier [C1, C2, ..., Cn] and
corresponding predefined class set [P1, P2, ..., Pn]
OUTPUT: Class value
PROCEDURE:
Combine all training image dataset and map it to a selection vector (1 to n).
[D1, D2, ..., Dn] [1, 2, ..., n]
Train Cselect with the combined training dataset
For each image in the combined dataset
Select classifier Ci from [C1, C2, ..., Cn] by Cselect
Classify input image by selected classifier Ci
Predict class value which belongs to Pi
End For
RETURN Predicted Output class value
Figure 2 Detailed block diagram of the proposed approach
III. EXPERIMENT AND RESULT
A. Dataset:
For verification of Meta fusion approach, MNIST image
dataset is selected. This dataset contains 8-bit images of
handwritten digits (0-9). To create heterogeneous dataset, it
is divided into two sub dataset, i.e. MNIST1 and MNIST2.
MNIST1 contains all sample of digit 0 to 4 and MNIST2
contains all sample of digit 5 to 9. No class overlapping is
presented in both datasets.
B. Experimental Setup
The purpose of this experiment is to verify proposed
Meta fusion approach for integration of trained image
classifier. For this, two separate convolutional neural
networks i.e.CNN1, CNN2 are trained on MNIST1 and
MNIST2 respectively which work as trained image classifier.
CNN1 and CNN2 achieve the classification accuracy of
0.9980 and 0.9954. Proposed Meta fusion approach is
implemented in python library sklearn, Keras, TensorFlow to
integrate these trained image classifiers. Binary
convolutional neural network is exploited as a classifier
selection model which selects CNN1, CNN2 when the input
image belongs to MNIST1, MNIST2 respectively.
C. Result
The integrated model based on Meta fusion approach is
evaluated on combined MNIST dataset (MNIST1, MNIST2)
over standard performance metrics viz. accuracy, precision,
recall and F1 score. Table II shows the experimental result of
classification for combined MNIST dataset. These
performance metrics are calculated on the predicted class
value and actual class value of a test image of MNIST
dataset. Precision score is the probability of exactness while
recall score is the probability of completeness. F1 score is the
harmonic mean of precision and recall. Average score of
performance metrics shows that integrated model performs
accurate as well as exact and completeness.
Table II Evaluated Performance score of integrated model
based on Meta fusion approach
Class
Recall
F1 Score
Accuracy
0
0.9980
0.9959
0.9982
1
1.0000
0.9969
0.9999
2
0.9961
0.9947
0.9962
3
0.9980
0.9946
0.9983
4
0.9908
0.9934
0.9909
5
0.9888
0.9905
0.9886
6
0.9896
0.9927
0.9898
7
0.9912
0.9941
0.9913
8
0.9928
0.9943
0.9934
9
0.9921
0.9911
0.9924
Average
0.9939
0.9939
0.9939
D. Comparison
To measure the accuracy, integrated model has compared
with ELM, CNN, CNN-ELM and ensemble of CNN-ELM
(trained on distributed homogeneous dataset). This integrated
model achieves the improved accuracy up to 99.39%. Table
III shows a comparison of accuracy score of ELM, CNN,
CNN-ELM and ensemble of CNN-ELM (trained on
distributed homogeneous dataset) with proposed Meta fusion
approach. Comparison graph is shown in Figure 3 that
clearly shows integrated model based on Meta fusion
approach perform better than state of art methods.
Table III Comparison of Proposed Meta fusion approach to the
state of art methods
Methods
Training Data Type
Accuracy
ELM [7]
Whole dataset
97.54%
CNN [7]
Whole dataset
99.20%
CNN-ELM [7]
Whole dataset
99.24%
Ensemble of CNN-ELM
[7]
Distributed homogeneous
Dataset
99.33%
Meta Fusion approach
for CNN
(Proposed)
Distributed heterogeneous
Dataset
99.39%
Figure 3 Comparison graph of proposed approach with state of
art methods
97.54%
99.20%
99.24%
99.33%
99.39%
96.50%
97.00%
97.50%
98.00%
98.50%
99.00%
99.50%
100.00%
ELM
CNN
CNN-ELM
Ensemble
of CNN-
ELM
Meta
Fusion
approach
(Proposed)
Accuracy
Classification Methods
Accuracy
IV. CONCLUSION
Proposed Meta fusion approach successfully integrates
the trained image classifiers, which are trained on distributed
heterogeneous data. The effectiveness of this integrated
model is evaluated on distributed heterogeneous MNIST
dataset over standard performance metrics. The experimental
result shows that the integrated model based on Meta fusion
approach outperformed in comparison of core CNN, core
ELM, Hybrid CNN-ELM and ensemble of CNN-ELM
trained on distributed homogeneous data.
ACKNOWLEDGMENT
This work is supported by UGC SRF Fellowship
(3894/Net June 2013). We are thankful to University Grant
Commission.
REFERENCES
[1] Du, P., Xia, J., Zhang, W., Tan, K., Liu, Y., & Liu, S. (2012).
Multiple classifier system for remote sensing image classification: A
review. Sensors, 12(4), 4764-4792.
[2] Rokni, K., Ahmad, A., Solaimani, K., & Hazini, S. (2015). A new
approach for surface water change detection: Integration of pixel level
image fusion and image classification techniques. International
Journal of Applied Earth Observation and Geoinformation, 34, 226-
234.
[3] Xiao, T., Xu, Y., Yang, K., Zhang, J., Peng, Y., & Zhang, Z. (2015).
The application of two-level attention models in deep convolutional
neural network for fine-grained image classification. In Proceedings
of the IEEE Conference on Computer Vision and Pattern Recognition
(pp. 842-850).
[4] Gunes, H., & Piccardi, M. (2005, October). Affect recognition from
face and body: early fusion vs. late fusion. In Systems, Man and
Cybernetics, 2005 IEEE International Conference on (Vol. 4, pp.
3437-3443). IEEE.
[5] Amarsaikhan, D., & Douglas*, T. (2004). Data fusion and
multisource image classification. International Journal of Remote
Sensing, 25(17), 3529-3539.
[6] Fernando, B., Fromont, E., Muselet, D., & Sebban, M. (2012, June).
Discriminative feature fusion for image classification. In Computer
Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on
(pp. 3434-3441). IEEE.
[7] Kannojia, S. P., Jaiswal, G. (2018) Ensemble of Hybrid CNN-ELM
Model for Image Classification. In Signal Processing and Integrated
Networks (SPIN), 2018 5th International Conference on (pp. 538-
541). IEEE
[8] LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-
based learning applied to document recognition. Proceedings of the
IEEE, 86(11), 2278-2324.
... For incremental learning environment, these image datasets are rearranged in such a way that it provides the continuous increasing dataset to model with the help of incremental learning strategies. For this, these datasets are divided and rearranged in two distribution manner such as sample-wise distribution [17] and class-wise distribution [18]. These distributed set of image dataset can be used in incremental order to input the image classification model using incremental learning strategies. ...
Conference Paper
Full-text available
Deep learning-based image classification model learns from the fixed and specific training dataset. For the generalization and adaptation of human learning behaviour, some of these models adapted incremental learning to enhance the learning and knowledge from updated and incremented dataset. An incremented dataset can be in form of increment of examples or new class dataset images. This incremented dataset is learned by deep learning models by two incremental learning strategies i.e. sample-wise and class-wise. This paper proposes a performance analysis methodology and experimentally analyze the performance of these incremental learning strategies in CNN based image classification model on prepared incremented dataset. The evaluation of performance of these deep image classification model on classification performance metrics such as accuracy, precision, recall and f1 score shows that these model’s learning and classification capabilities are increased with incremental learning on the incremented dataset. Two different incremental learning shows the relation between performances of model to the increment of dataset.
Article
This paper presents a novel prediction model based on the convolutional recurrent fusion for performance optimization by additional temporal feature learning. The different temporal patterns of input data are generated as additional high-level features learning. Different generated patterns compute with two modalities and generate the intermediate results simultaneously. This learning strategy increases the performance of the model and improves the prediction results. In addition, the fusion of the intermediary results is especially contributing to prediction outcome optimization with less computational complexity. We performed the experiments on different open-sourced, standard, temporal datasets for performance evaluation of our proposed model by comparing with Baselines. Our proposed model achieves more accuracy than the CNN-LSTM and CNN-GRU with approximately 1.16% and 1.23% improvement respectively.
Conference Paper
Full-text available
To leverage feature representation of CNN and fast classification learning of ELM, Ensemble of Hybrid CNN-ELM model is proposed for image classification. In this model, image representation features are learned by Convolutional Neural Network (CNN) and fed to Extreme Learning Machine (ELM) for classification. Three hybrid CNN-ELMs are ensemble in parallel and final output is computed by majority voting ensemble of these classifier’s outputs. The experiments show this ensemble model improves the classifier’s classification confidence and accuracy. This model has been benchmarked on MNIST benchmark dataset and effectively improved the accuracy in comparison of single hybrid CNN-ELM classifier with accuracy upto 99.33%. Proposed ensemble model has been also compared with Core CNN, Core ELM, Hybrid CNN-ELM and achieves competitive accuracy.
Conference Paper
Full-text available
Bag-of-words-based image classification approaches mostly rely on low level local shape features. However, it has been shown that combining multiple cues such as color, texture, or shape is a challenging and promising task which can improve the classification accuracy. Most of the state-of-the-art feature fusion methods usually aim to weight the cues without considering their statistical dependence in the application at hand. In this paper, we present a new logistic regression-based fusion method, called LRFF, which takes advantage of the different cues without being tied to any of them. We also design a new marginalized kernel by making use of the output of the regression model. We show that such kernels, surprisingly ignored so far by the computer vision community, are particularly well suited to achieve image classification tasks. We compare our approach with existing methods that combine color and shape on three datasets. The proposed learning-based feature fusion process clearly outperforms the state-of-the art fusion methods for image classification.
Article
Full-text available
Over the last two decades, multiple classifier system (MCS) or classifier ensemble has shown great potential to improve the accuracy and reliability of remote sensing image classification. Although there are lots of literatures covering the MCS approaches, there is a lack of a comprehensive literature review which presents an overall architecture of the basic principles and trends behind the design of remote sensing classifier ensemble. Therefore, in order to give a reference point for MCS approaches, this paper attempts to explicitly review the remote sensing implementations of MCS and proposes some modified approaches. The effectiveness of existing and improved algorithms are analyzed and evaluated by multi-source remotely sensed images, including high spatial resolution image (QuickBird), hyperspectral image (OMISII) and multi-spectral image (Landsat ETM+). Experimental results demonstrate that MCS can effectively improve the accuracy and stability of remote sensing image classification, and diversity measures play an active role for the combination of multiple classifiers. Furthermore, this survey provides a roadmap to guide future research, algorithm enhancement and facilitate knowledge accumulation of MCS in remote sensing community.
Conference Paper
This paper presents an approach to automatic visual emotion recognition from two modalities: face and body. Firstly, individual classifiers are trained from individual modalities. Secondly, we fuse facial expression and affective body gesture information first at a feature-level, in which the data from both modalities are combined before classification, and later at a decision-level, in which we integrate the outputs of the monomodal systems by the use of suitable criteria. We then evaluate these two fusion approaches, in terms of performance over monomodal emotion recognition based on facial expression modality only. In the experiments performed the emotion classification using the two modalities achieved a better recognition accuracy outperforming the classification using the individual facial modality. Moreover, fusion at the feature-level proved better recognition than fusion at the decision-level.