Feature selection using Haar wavelet power spectrum.

Abstract

Feature selection is an approach to overcome the 'curse of dimensionality' in complex researches like disease classification using microarrays. Statistical methods are utilized more in this domain. Most of them do not fit for a wide range of datasets. The transform oriented signal processing domains are not probed much when other fields like image and video processing utilize them well. Wavelets, one of such techniques, have the potential to be utilized in feature selection method. The aim of this paper is to assess the capability of Haar wavelet power spectrum in the problem of clustering and gene selection based on expression data in the context of disease classification and to propose a method based on Haar wavelet power spectrum.
Haar wavelet power spectra of genes were analysed and it was observed to be different in different diagnostic categories. This difference in trend and magnitude of the spectrum may be utilized in gene selection. Most of the genes selected by earlier complex methods were selected by the very simple present method. Each earlier works proved only few genes are quite enough to approach the classification problem 1. Hence the present method may be tried in conjunction with other classification methods. The technique was applied without removing the noise in data to validate the robustness of the method against the noise or outliers in the data. No special software or complex implementation is needed. The qualities of the genes selected by the present method were analysed through their gene expression data. Most of them were observed to be related to solve the classification issue since they were dominant in the diagnostic category of the dataset for which they were selected as features.
In the present paper, the problem of feature selection of microarray gene expression data was considered. We analyzed the wavelet power spectrum of genes and proposed a clustering and feature selection method useful for classification based on Haar wavelet power spectrum. Application of this technique in this area is novel, simple, and faster than other methods, fit for a wide range of data types. The results are encouraging and throw light into the possibility of using this technique for problem domains like disease classification, gene network identification and personalized drug design.

Full text

Page 1

BioMed Central
Page 1 of 11
(page number not for citation purposes)
BMC Bioinformatics
Open Access
Research article
Feature selection using Haar wavelet power spectrum
Prabakaran Subramani*, Rajendra Sahu and Shekhar Verma
Address: ABV-Indian Institute of Information Technology and Management, Gwalior, India
Email: Prabakaran Subramani* - pra_spin2@yahoo.com; Rajendra Sahu - rsahu@iiitm.ac.in; Shekhar Verma - sverma@iiitm.ac.in
* Corresponding author
Abstract
Background: Feature selection is an approach to overcome the 'curse of dimensionality' in
complex researches like disease classification using microarrays. Statistical methods are utilized
more in this domain. Most of them do not fit for a wide range of datasets. The transform oriented
signal processing domains are not probed much when other fields like image and video processing
utilize them well. Wavelets, one of such techniques, have the potential to be utilized in feature
selection method. The aim of this paper is to assess the capability of Haar wavelet power spectrum
in the problem of clustering and gene selection based on expression data in the context of disease
classification and to propose a method based on Haar wavelet power spectrum.
Results: Haar wavelet power spectra of genes were analysed and it was observed to be different
in different diagnostic categories. This difference in trend and magnitude of the spectrum may be
utilized in gene selection. Most of the genes selected by earlier complex methods were selected by
the very simple present method. Each earlier works proved only few genes are quite enough to
approach the classification problem [1]. Hence the present method may be tried in conjunction
with other classification methods. The technique was applied without removing the noise in data
to validate the robustness of the method against the noise or outliers in the data. No special
softwares or complex implementation is needed. The qualities of the genes selected by the present
method were analysed through their gene expression data. Most of them were observed to be
related to solve the classification issue since they were dominant in the diagnostic category of the
dataset for which they were selected as features.
Conclusion: In the present paper, the problem of feature selection of microarray gene expression
data was considered. We analyzed the wavelet power spectrum of genes and proposed a clustering
and feature selection method useful for classification based on Haar wavelet power spectrum.
Application of this technique in this area is novel, simple, and faster than other methods, fit for a
wide range of data types. The results are encouraging and throw light into the possibility of using
this technique for problem domains like disease classification, gene network identification and
personalized drug design.
Background
Modern technologies like microarrays facilitate the study
of expression levels of thousands of genes simultaneously.
This study is useful to determine whether these genes are
active, hyperactive or inactive in various tissues. The vast
amount of microarray data is so important for the appli-
Published: 05 October 2006
BMC Bioinformatics 2006, 7:432 doi:10.1186/1471-2105-7-432
Received: 09 February 2006
Accepted: 05 October 2006
This article is available from: http://www.biomedcentral.com/1471-2105/7/432
© 2006 Subramani et al; licensee BioMed Central Ltd.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0),
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Page 2

BMC Bioinformatics 2006, 7:432 http://www.biomedcentral.com/1471-2105/7/432
Page 2 of 11
(page number not for citation purposes)
cations like disease classification and identifying the
genetic networks. Solutions for complex problems like
identification of cancer types and their subtypes need
more accuracy for utilizing them in treating this disease
and in preparing more effective therapeutic solution like
individual drug design. So, it is important and necessary
to select only genes containing the expression data con-
tributing to the problem domain and to filter irrelevant
data to increase the performance of the methods used.
Feature selection is the problem of identifying such genes
or features [2]. That is, this can be used to identify the
important genes with significant information content
when the problem is poorly structured. This improves the
generalization performance and inference of classification
models [3] by overcoming the 'curse of dimensionality'.
One important problem with feature selection methods is
that both problem relevance and biological relevance of
the features selected may not be achieved completely.
Also, most of the feature selection methods do not fit for
the wide range of datasets. They are coupled with a partic-
ular classification method and time consuming. Statistical
methods are in use in this domain for a long time. But,
extensive preprocessing and lesser consensus among them
are major problems with them. Transform oriented signal
processing methods are simpler and may provide an alter-
native platform to the statistical methods. They have been
successfully utilized in many domains like image process-
ing. But, they have not been much utilized in the field of
bioinformatics. The key advantage of these transform ori-
ented methods is their power of capturing some inherent
properties of the data. The aim of this paper is to analyse
the capabilities of Haar wavelet power spectrum in select-
ing informative features in microarray data on the basis of
the inherent properties captured by them. The present
work utilizes some earlier works in feature selection for
illustration and analyses the comparability of wavelet
strategy with those of earlier works.
Feature selection
Feature selection can be approached in three ways. First,
we may handle feature selection method independently
irrespective of the further applications utilizing these fea-
tures. That is, the features selected may be used for any
classifier algorithms. This approach of feature selection is
called a filter method. Second, features may be selected for
a specific classifier algorithm. In this approach called a
'wrapper method' [4], the qualities or accuracies of all
possible subsets are analyzed to select the optimal one to
the specific classification algorithm. Finally, feature selec-
tion and classifier design may be accomplished together.
This strategy is found in embedded methods. Embedded
methods are incorporated into the learning procedure,
and hence are dependent on the classification model.
Systematizations and surveys on feature selection algo-
rithms have been presented in a variety of review articles
like Blum and Langley [5], Kohavi and John [4] and
Guyon [6]. So far, a number of variable (or gene) selection
methods like the support vector machine method [5], the
genetic algorithm [7], the perceptron method [8], Baye-
sian variable selection [9-12], and the voting technique
[13], mutual information-based gene and feature selec-
tion method [1], entropy based feature selection [14] and
many artificial intelligent techniques like hill climbing,
best first search [15], simulated annealing [16], backward
elimination [17], forward selection and their combina-
tions have been proposed. Specific to filter approach, Kira
and Rendell's Relief algorithm [18] which selects features
based on a threshold of weights assigned to each feature is
a good example but it was tested on small set of features.
In case of high dimensional datasets containing thou-
sands of genes, filters are preferred to wrappers due to
their independency over the models [19,20]. Xiang et al
[14] devised a hybrid of filter and wrapper approaches
and tested it on high dimensional gene expression data
with 72 samples and 7129 features. Another such work on
high dimensional gene expression data was done by
Golub et al [13] on the same dataset using correlation
measures. Califano et al [21] also worked on a high
dimensional dataset of 6817 genes using a supervised
learning algorithm. All these works revealed the fact that
the result was better while using selected features instead
of the whole data set.
Most commonly used filters are based on information-
theoretic or statistical principles. Score based feature selec-
tion methods are popular among filters using statistical
principles. These methods calculate statistical scores on
the gene expression data. They sort genes according the
scores assigned and filter them by applying some thresh-
old. χ 2-score, t-test metrics[22], Wilcoxon rank sum
test[22], correlation co efficient [23] and B-scatter score
are some prominent examples. Some other strategies used
in feature selection through ranking are SNR based rank-
ing used in Shipp's approach [24] and sensitivity analysis
based ranking used in Mean square classifier [25]. The
strategy of selecting features using sensitivity analysis is to
rank a feature according to the change in the value of an
objective function caused by the removal of that feature
from the dataset. SNR method is more capable of detect-
ing and ranking a smaller number of significant variables.
Apart from ranking methods, several other approaches
like Relief [18,26], gini-index [27], relevance, average
absolute weight of evidence [28] and bi-normal separa-
tion [29] are also in use.
Most of the methods of feature selection are complex and
consume more time to converge. Many of them do not fit

Page 3

BMC Bioinformatics 2006, 7:432 http://www.biomedcentral.com/1471-2105/7/432
Page 3 of 11
(page number not for citation purposes)
for all data types in addition that they require more sam-
ples. No consensus among various statistical methods is
achieved to use them. The selection of a statistical method
for a dataset is a hit and run approach. So, a more generic
method which can cope up with a variety of data is in dire
need. Further, a very few model independent approaches
for feature selection are available since most of the meth-
ods of feature selection are coupled with classification. In
this paper, we analyse the capability of wavelet power
spectrum in feature selection and we propose a method of
feature selection based on Haar wavelet power spectrum.
This method is found fit for a wide range of data sets and
also works with smaller number of samples. It can be used
in conjunction with other classification methods. The
algorithm is very simple and requires comparatively less
time to be executed. The method is a model independent
approach, a filter feature selection method, based on the
Haar wavelet power spectrum of the microarray data.
Unlike most of the other methods, it is relatively a very
simple algorithm. We observed that the features selected
by our method can be used in conjunction with more clas-
sification algorithms.
Wavelet and its power spectrum
Wavelets are a family of basis functions that can be used
to approximate other functions by expansion in orthonor-
mal series. They combine such powerful properties as
orthonormality, compact support, varying degrees of
smoothness, localization both in time or space and scale
(frequency), and fast implementation. One of the key
advantages of wavelets is their ability to spatially adapt to
features of a function such as discontinuities and varying
frequency behaviour. A wavelet transform is a lossless lin-
ear transformation of a signal or data into coefficients on
a basis of wavelet functions [30]. Performing the discrete
wavelet transform (DWT) of a signal x is done by passing
it through low pass filters (scaling functions) and high
pass filters simultaneously. Down-sampling or decima-
tion by a factor 2 is performed after each pass through fil-
ters. Decimation by 2 means removing every alternative
coefficient in the function is performed after each pass
through filters. Figure 1 depicts a two level wavelet trans-
form.
Mathematically, the wavelet transform of a function x [k]
can be represented by the following formula:
where ylow (n) and yhigh (n) are responses from low and
high pass filters respectively. In matrix form, wt = [WXT]T
where W = [L;H] where L and H are impulse responses of
low pass and high pass filters and wt is wavelet transform
of the one dimensional input signal X. The two filters used
at each stage of decomposition must be related to each
other by g [l-1-n] = (-1)n·h[n] where g and h are the
impulse responses of the two filters, l is the filter length in
number of points, n is the order of the data points and l is
such that 0 ≤ n <l. For example, there are two data points
for each filter of Haar wavelet with n = 0, 1. These filters
are known as quadrature mirror filters. A wavelet trans-
form of a data after i level of decompositions contains the
approximation coefficients at ith level and all detailed
coefficients up to ith level. The detailed coefficients at dif-
ferent levels incorporate the variations in information at
those levels. Level of decomposition is also termed as
band.
A number of wavelet families like symlet, coiflet, daub-
echies and biorthogonal wavelets are already in use. They
vary in various basic properties of wavelets like compact-
ness. Among them, Haar wavelets belonging to daub-
yn x k g
[ ]
nk
yn x k h
[ ]
nk
high
k
low
k
( )[]
( )[]
=⋅⋅ −
=⋅ ⋅ −
=−∞
∞
∑
∞
∑
=−∞
2
2
A two level DWT for N data
Figure 1
A two level DWT for N data. The number of data is halved
after every filtering and down sampling operation. A wavelet
transform is applied on output of low pass filter (h [n])
(approximation coefficients) recursively keeping the output
coefficients of each high pass filtering operation (g [n])
(detailed coefficients) at each stage. The wavelet transform of
a data at any level i of decomposition consists of approxima-
tion coefficients only at ith level and all detailed coefficients
up to ith level.

Page 4

BMC Bioinformatics 2006, 7:432 http://www.biomedcentral.com/1471-2105/7/432
Page 4 of 11
(page number not for citation purposes)
echies wavelet family are most commonly used wavelets
in database literature because they are easy to compre-
hend and fast to compute [3]. Haar transform can be
viewed as a series of averaging and differentiating opera-
tions on a discrete function. The impulse response for
high pass filter is given by [1/, -1/ ] and for low
2
pass filter, the impulse response is [1/
the minimum number of elements in input data should
be 2. The input data should always contain the number of
elements 2n where n is an integer. In matrix form, the Haar
wavelet filter can be expressed as
,1/ ]. That is,
It can be easily examined that both the low pass and high
pass filters of Haar wavelet are quadratic in nature using
the discussion presented in the previous paragraph. For a
data having more than two elements, the Haar wavelet
matrix of can be constructed by diagonally repeating these
basic filters to form a matrix of the size of input data.
Upper part of the matrix is created by repeating impulse
responses of low pass filter diagonally and lower part of
the matrix is created by repeating impulse responses of
high pass filter diagonally. From Figure 1, it is evident that
the size of the data points to be used for wavelet transform
in a level is equal to half of the data points used in the pre-
vious level. Accordingly, the size of the Haar wavelet
matrix also reduced. For example, if we use a signal of four
data points, the size of the Haar wavelet matrix will be 4 ×
4 in the first step of wavelet transform. From Figure 1, it is
evident that the number of data points to be used for the
second step of wavelet transform is 2. These are the output
of low pass filtering operation as shown in Figure 1. So,
the Haar wavelet matrix to be used is of the size 2 × 2.
More details of wavelets may be referred at [31-33].
The minimum number of data points in an input signal
should be 2 in the case of Haar wavelet and the number of
data points needed for n times decomposition is 2n. If the
number of input data points is less than this required
number, 2n, zeros may be padded (appended) at the right
end of the input data to compensate the required number.
In the present work, the number of data points refers to
the number of samples which is equal to the number of
columns present in the microarray data matrix. That is, the
expression of a gene in a sample is considered as a data
point of a one dimensional signal X. Accordingly, the col-
umns of the microarray data matrix were prepared so as to
be amenable for satisfying the required number criterion.
In some experiments, a reduced number of the columns
of the microarray data matrix equal to the nearest power
of 2 were randomly selected and used. Since we use the
strategy of finding the average value of wavelet power
spectrum for each gene per sample, in the present work,
the choice of columns selected for replication or reduction
is immaterial. We used a random selection of the columns
for the purpose of reduction and replication for data input
preparation. It was observed that such a random selection
of columns did not affect much the robustness and the
accuracy of the present method used. In the present work,
expression data of each gene across various tissue samples
or various experiments is modeled to a one dimensional
signal. Therefore, the entire microarray data is modeled to
a group of M number of one-dimensional signals where M
is the total number of genes present in the gene microar-
ray data. More mathematical details of wavelets may be
referred at [31-33].
Local wavelet power spectrum at a particular decomposi-
tion level is calculated by summing up the squares of
wavelet coefficients at that level [11]. For a set of wavelet
coefficients Cj,k, where j is level of decomposition and k is
the order of the coefficient, the wavelet power spectrum is
given below.
If there are N elements in an array, there will be log2(N)
coefficient bands or levels of decomposition for Haar
wavelet. That is, the power spectrum can be referred as a
graphical representation of cumulative information varia-
tion at each scale of decomposition. Global wavelet power
spectrum [34] is the average of such local power spectra.
Results and discussion
Our proposed algorithm for feature selection has been
applied on various datasets and top genes are reported
here. In all these experiments, we have used Haar wavelets
since the number of minimum features for wavelet trans-
formation at lowest level is smaller than that required by
the other wavelets. We applied our method on three data-
sets namely Golub dataset, Hedenfalk breast cancer data-
set and Khan SRBCT dataset. All experiments were carried
out without filtering any data to validate the robustness of
the method against the noise or outliers in the data.
SRBCT dataset
First, we focus on feature selection for the small, round
blue cell tumors (SRBCT) of childhood. The dataset of
SRBCT used for experimentation here is available at [35].
This dataset is composed of 2308 genes and 63 samples
from four cancers which includes Neuroblastoma (NB)
(12 samples), Rhabdomyosacoma (RMS) (23 samples),
Burkitt Lymphomas (BL) (8 samples) and Ewing's family
22
2
1
2
1
2
11
22
−
⎡
⎢
⎢
⎢
⎢
⎣
⎤
⎥
⎥
⎥
⎥
⎦
spectrum jCj k
k
j
[]
,
=
=
−
∑
2
0
2 1

Page 5

BMC Bioinformatics 2006, 7:432 http://www.biomedcentral.com/1471-2105/7/432
Page 5 of 11
(page number not for citation purposes)
of tumors (EWS) (20 samples). Originally, Khan et al [35]
classified this dataset using artificial neural networks on
gene expression profiles. The feature selection and classi-
fication using this dataset has also been performed by
Zhou et al using Gibb's sampler and SMC [36]. Khan et al
[35] selected a list of 96 discriminating genes pertaining to
classification. This list included some genes being identi-
fied important to two classes out of four classes and some
genes which were not categorized for any class. Our
method has identified some of them important for one of
the four classes. It has selected almost all these features
with comparatively simpler calculations. Also, we used
only 32 samples out of 63 sample set. First four samples
from each diagnostic category have been selected to form
this group of 32 samples. It exhibits the possibility of
using our methods for datasets with a lesser number of
samples. Most of the top ranked genes listed in the present
work have been used in classification of the dataset in ear-
lier works [36,35].
A list of top genes selected by our method has been listed
in Table 1. Genes with index IDs 1319, 1645, 1954, 1831,
2303, 1980, 373 and 1626 were also reported to be differ-
entially expressed in EWS in Khan et al's work [35]. Gene
851 was not allocated to any class in [35]. Most of the
other genes like gene 951 reported to be discriminating
EWS have been selected as important genes for EWS but
with a lower ranking. The rank of Gene 951 was 59 in our
work and its RPV was 59.36. Gene 1200 was not selected
in Khan's [35] work but it was ranked fourth in our work.
Of all genes selected for EWS, neural-specific genes
[37,38] like TUBB5 (Gene 373), ANXA1 (Gene 1831),
and NOE1 (Gene 1645) lend more credence to the pro-
posed neural histogenesis of EWS [39]. Most of the top
ranked genes are dominant in EWS category in compari-
son with their expression in other classes. This implies
that most of the top ranked genes in Table 1 are highly
related to the classification of EWS from other categories.
When tested with Golub's algorithm [13], first 23 samples
of SRBCT dataset except sample number 12 were catego-
rized as EWS and remaining 43 samples were categorized
as non EWS samples.
A list of strongest genes selected for classifying BL versus
others using our method has been reported in Table 2. In
the SRBCT dataset, the samples of BL category spans from
sample number 24 to sample number 31. Among them,
Genes 836 and 1158 were reported to be differentially
expressed in BL in Khan's original work [35]. Genes with
indices 1916, 783, 846, 1735,335,1884,2230,1915, 85
and 276 were also selected in [35] but not assigned to a
particular class. Of six differentially expressed genes in BL
and some other classes, remaining four genes except genes
836 and 1158 were selected for other classes. So, our
method selects proper genes effectively with simple algo-
rithm and calculations. Most of the top ranked genes are
dominant in BL category in comparison with their expres-
sion in other classes (See additional file1). This implies
that most of the top ranked genes in Table 2 are highly
related to the classification of BL from other categories.
Expression levels of these genes show that they can be
classified using Golub's classification algorithm [13] since
they are highly correlated to the "idealized expression pat-
tern" [13].
In the case of NB (Neuroblastoma) class, there were no
exclusively discriminating genes reported in [35]. They
were reported to be differentially expressed either with
EWS or RMS. Among them, Genes 951, 1980, 2303 and
1626 have been identified as stronger genes pertaining to
EWS by our method. Genes with index numbers 1764,
742, 236, 255, 417, 1601, 2199, 153, 1066, 2144, 2050
and 1662 were listed among 96 discriminating genes not
Table 2: A list of top ranked genes selected by using relative
percentage variation of gene expression profiles between BL
versus others of SRBCT dataset
Rank Index no.Clone ID RPV (%)
1
2
3
4
5
6
7
8
9
10
14
17
19
25
1916
836
783
846
1735
1387
335
1884
1725
1295
2230
1915
1158
85
80109
241412
767183
183337
200814
740604
1469292
609663
813630
344134
417226
840942
814526
700792
98.61
98.24
98.04
98.02
97.81
97.40
96.35
96.16
95.69
95.48
94.45
94.22
93.24
91.70
Table 1: Differentially expressed genes selected for classifying
EWS diagnostic category of SRBCT data (RPV – Relative
percentage variation).
RankIndex no. Clone ID RPV (%)
1
2
3
4
5
6
7
8
9
10
16
19
20
1319
1645
1954
1200
696
1140
1070
851
404
1831
1980
373
1626
866702
52076
814260
838856
753587
824922
1475730
563673
1422723
208718
841641
291756
811000
99.52
97.92
97.91
96.45
95.63
92.71
91.06
89.27
88.28
87.64
83.46
81.31
81.22