Research on Improvement in Self-Organization Capability Using Two SOMs

Full text

Research on Improvement in Self-Organization Capability Using Two SOMs
Haruna MATSUSHITA
y
, YokoUWATE
y
and Yoshifumi NISHIO
y
y
Department of Electrical and Electronics Engineering
Tokushima University
2-1 Minami-Josanjima, Tokushima, Japan
Phone:+81-88-656-7470, Fax:+81-88-656-7471
Email:
f
haruna, uwate, nishio
g
@ee.tokushima-u.ac.jp
1. Introduction
The Self-Organizing Map (SOM) is unsupervised neu-
ral networks introduced by Kohonen in 1982 [1]. Self-
organization is to change an internal structure to adjust
to the signal from the outside though teacher signals are
not given from the outside. SOM is a model simplifying
self-organization process of the brain. SOM obtains a
statistical feature of input data and is applied to a wide
eld of data classications. And, the eectiveness is said
to b e very high [2]-[7].
SOM has the position relation b etween neurons with
d
-dimensions, and the neuron which won a comp etition,
and some neurons around the winner learn. For exam-
ple, if input data are 2-dimensional random data which
are uniformly-distributed between 0 and 1, SOM is dis-
tributed like a square. Thus, a distribution of input data
can be expressed with a weight vector by learning. How-
ever, when input data is nonuniform, some experience is
needed to extract the dense part.
In this research, in order to solve this problem, we
propose a method of using simultaneously two kinds of
SOMs whose features are dierent. One is distributed
in the area for which input data gathers and the other
over the whole. As a result, the feature of data can be
extracted more eectively by using two kinds of SOMs.
2. Self-Organizing Map
We concretely explain the learning method of SOM.
m
neurons are arranged on a 2-dimensional grid as shown
in Fig. 1(a). The range of the elements of
l
-dimensional
input vectors
x
j
=(
x
j
1
;x
j
2
;
111
;x
jl
)(
j
=1
;
2
;
111
;N
)
are assumed to be from 0 to 1.
(SOM1)
The initial values of all the weight vectors
w
are given between 0 and 1 at random.
(SOM2)
An input vector
x
j
is inputted to all the neu-
rons at the same time in parallel.
(SOM3)
The internal activity degree
net
j
i
is calculated
from the distance between the input vector
x
j
and the
weight vector
w
i
=(
w
i
1
;w
i
2
;
111
;w
il
)(
i
=1
;
2
;
111
;m
)
of the neuron
i
, according to:
net
j
i
=
k
w
i
0
x
j
k
0
1
:
(1)
In this study, Euclidean distance is used for (1).
(SOM4)
The neuron with the maximum internal activ-
ity degree is winner neuron
c
. In other words, the winner
neuron
c
is the neuron with the weight vector nearest to
the input vector.
(SOM5)
The weight vector of the winner neuron
c
and
the neighborhood neuron
N
c
(as Fig. 1(b)) are up dated
as:
w
i
(
t
+1)=
w
i
(
t
)+

(
t
)(
x
j
0
w
i
(
t
))
;i
2
N
c
(
t
)
;
(2)
where

(
t
) is the learning coecient.

(
t
)and
N
c
(
t
)are
decreased with time according to the following equations:

(
t
)=

0
(1
0
t=T
)
;
(3)
N
c
(
t
)=[
N
c
(0) (1
0
t=T
)]
;
(4)
where [ ] denotes the Gauss' notation,
T
is the maximum
number of learning,

0
is the initial value of the learning
coecient,
N
c
(0) is the initial value of the neighborhood
coecient.
(SOM6)
The steps from (SOM2) to (SOM5) are re-
peated for all the input vectors.
3. Two SOMs
In this study, we propose a method of extracting the
feature of the input data more eectively using twokinds
of SOMs whose features are dierent. Namely, one self-
organizes the area on which input data are concentrated,
and the other self-organizes the whole of the input space.
We call the former SOM
L
and the latter SOM
G
.
We explain the learning method of the proposed
SOMs.
(2-SOM1)
The initial values of all the weightvectors
w
L
of SOM
L
are given between 0 and 1 at random. The
2005 RISP International Workshop on Nonlinear
Circuit and Signal Processing (NCSP'05)
Hawaii, USA, Mar. 4-6, 2005.
- 307 -

(a) Structure of SOM. (b) Example of neighborhood area.
Figure 1: Self-organiging map.
0 0.2 0.4 0.6 0.8
1
0
0
.2
0
.4
0
.6
0
.8
1
Figure 2: Example of initial states of two SOMs
initial values of all the weight vectors
w
G
of SOM
G
are
given between 0.4 and 0.6 at random, as Fig. 2.
(2-SOM2)
An input vector
x
j
is inputted to all the
neurons of SOM
G
and SOM
L
at the same time in paral-
lel.
(2-SOM3)
The distance between
x
j
and the weight
vector
w
G
=(
w
G
i
1
;w
G
i
2
;
111
;w
G
il
) of the neuron
i
of
SOM
G
, and the distance between
x
j
and the weight vec-
tor
w
L
=(
w
L
i
1
;w
L
i
2
;
111
;w
L
il
) of the neuron
i
of SOM
L
are calculated. The internal activity degree
net
G
j
i
and
net
L
j
i
are obtained as:
net
G
j
i
=
k
w
G
i
0
x
j
k
0
1
;
(5)
net
L
j
i
=
k
w
L
i
0
x
j
k
0
1
:
(6)
(2-SOM4)
The winner neuron
c
is the neuron with the
maximum internal activity degree in all
net
G
and
net
L
.
(2-SOM5)
If the winner neuron
c
is the neuron in
SOM
G
, the weight vector of the winner neuron and the
neighborhood neuron
N
G
c
are updated as:
w
G
i
(
t
+1) =
w
G
i
(
t
)+

G
(
t
)(
x
j
0
w
G
i
(
t
))
;i
2
N
G
c
(
t
)
;
(7)
where

G
(
t
) is the learning co ecient.
If the winner neuron
c
is in SOM
L
, the weight vector
of the winner neuron and the neighborhood neuron
N
L
c
are updated as:
w
L
i
(
t
+1)=
w
L
i
(
t
)+

L
(
t
)(
x
j
0
w
L
i
(
t
))
;i
2
N
L
c
(
t
)
;
(8)
where

L
(
t
) is the learning coecient.

G
(
t
),

L
(
t
),
N
G
c
, and
N
L
c
are decided as:

G
(
t
)=

G
0
(1
0
t=T
)
;
(9)

L
(
t
)=

L
0
n
1
0
(
t=T
)
1
2
o
;
(10)
N
G
c
(
t
)=
h
N
G
c
(0)

N
G
c
(0)
0
1
N
G
c
(0)
0
n

(1
0
t=T
)
i
;
N
G
c
(
t
)
<N
G
c
(0)
;
(11)
N
L
c
(
t
)=[
N
L
c
(0) (1
0
t=T
)]
;
(12)
where

G
0
and

L
0
are the initial values of learning
coecients,
N
G
c
(0) and
N
L
c
(0) are the initial values
of neighborho o d coecients and
n
is the number of all
SOMs.
(2-SOM6)
The steps from (2-SOM2) to (2-SOM5) are
repeated for all the input vectors.
(2-SOM7)
Furthermore, only SOM
G
learns. Time is
reset as
t
=0. We compute
net
G
j
i
and
net
L
j
i
according
to the steps (2-SOM2) and (2-SOM3).
(2-SOM8)
If the distance between
x
j
and the neuron
of SOM
L
with the maximum
net
L
is larger than a small
distance
"
, the weight vector of the neuron of SOM
G
with the maximum
net
G
and the neighborhoo d neuron
N
Gs
c
are updated as:
w
G
i
(
t
+1)=
w
G
i
(
t
)+

Gs
(
t
)(
x
j
0
w
G
i
(
t
))
;
i
2
N
Gs
c
(
t
)
;
max(
net
L
j
i
)

1
"
;
(13)
where

Gs
(
t
) is the learning coecient.
(2-SOM9)
The steps from (2-SOM7) to (2-SOM8) are
repeated for all the input vectors.

Gs
(
t
)and
N
Gs
c
(
t
)
are decreased with time according to the following equa-
tions:

Gs
(
t
)=

Gs
0
(1
0
t=T
s
)
;
(14)
N
Gs
c
(
t
)=[
N
Gs
c
(0) (1
0
t=T
s
)]
;
(15)
where
T
s
is the maximum number of learning since step
(2-SOM7),
- 308 -

4. Simulation Results
4.1. Input Data
Input data is 2-dimensional random data of 450 points
whose distribution is non-uniform as Fig. 3. 225 points
are distributed within a small range from 0.2 to 0.45 hor-
izontally and from 0.7 to 0.95 vertically. The remaining
225 points are uniformly distributed between 0 and 1.
0 0.2 0.4 0.6 0.8
1
0
0
.2
0
.4
0
.6
0
.8
1
Figure 3: 2-dimensional input data.
4.2. Setting
4.2.1. One SOM
SOM has 441 neurons (21
2
21). The initial values of
the learning co ecient and the neighborhoo d co ecient
are chosen as follows:

0
=0
:
9
; N
c
(0) = 7
:
In the learning process, all the input data are given 8
times rep eatedly.
4.2.2. Two SOMs
SOM
G
and SOM
L
have 225 neurons (15
2
15) each,
namely, 2 SOMs have totally 445 neurons. The initial
values of the learning coecients, the neighborhood co-
ecients and the small distance are chosen as follows:

G
0
=

L
0
=0
:
9
; N
G
c
(0) =
N
L
c
(0) = 6
;

Gs
0
=0
:
7
; N
Gs
c
(0) = 4
; "
=0
:
02
:
After we give all the input data to SOM
G
and SOM
L
4 times, furthermore we give them for learning only of
SOM
G
4 times repeatedly.
4.3. Results
The simulation result is shown in Fig. 4. From Fig.
4(a), we can say that One SOM is self-organized. How-
ever, Two SOMs can be self-organized more eectively as
shown in Fig. 4(b). In addition, SOM
L
is self-organized
in the area for which input data concentrate, and SOM
G
is self-organized over the whole input space.
0 0.2 0.4 0.6 0.8
1
0
0
.2
0
.4
0
.6
0
.8
1
0 0.2 0.4 0.6 0.8
1
0
0
.2
0
.4
0
.6
0
.8
1
(a) One SOM. (b) Two SOMs.
Figure 4:
Simulation Results.
5.
n
SOMs
The number of SOM
L
can be increased depending on
distribution of input data. We carry out simulation for
various input data. For instance, when the data of Fig.
5(a) are input, one SOM
G
and twoSOM
L
are eective.
The learning method for 2 or more SOM
L
is similar to
the case of Two SOMs.
Figure 6 shows the respective results. In all the results,
SOM
L
is self-organized in the area for which input data
concentrate, and SOM
G
is self-organized over the whole
input space.
We consider that the concept using two kinds of SOMs
can be used to extract the data only in a dense part of the
input data, because the SOM
L
can nd such an area by
themselves. Figures 7(a) and (b) show one of the SOM
L
and the data whose nearest neuron is in the SOM
L
,re-
spectively, after the self-organization in the previous sim-
ulation of Fig. 6(c). The data in Fig. 7(b) is almost same
as one of the dense parts of the input data in Fig. 5(c).
We can also extract other dense parts of the input data
in Fig. 5(c) using the other SOM
L
as shown in Fig. 8.
6. Conclusions
In this study, we proposed the method of extracting
the feature of the input data more eectively using the
two kinds of SOMs whose features are dierent. The
number of SOM
L
is increased depending distribution of
input data. We conrmed by computer simulations that
n
SOMs could extract the dence parts of the input data.
In the future, we hope to develop this method further
and to apply it to various data.
References
[1] T. Kohonen, \Self-Organising Maps," 2nd ed, Springer
Series in Information Sciences, Vol. 30, Springer, Berlin,
1995.
[2] B. Angeniol, G. de La C. Vaubois and J. Y. Le Texier,
\Self-Organizing Feature Maps and the Travelling Sales-
- 309 -

0 0.2 0.4 0.6 0.8
1
0
0
.2
0
.4
0
.6
0
.8
1
0 0.2 0.4 0.6 0.8
1
0
0
.2
0
.4
0
.6
0
.8
1
0 0.2 0.4 0.6 0.8
1
0
0
.2
0
.4
0
.6
0
.8
1
(a) With 2 dense parts. (b) With 3 dense parts. (c) With 4 dense parts.
Figure 5:
Input data with various distributions.
0 0.2 0.4 0.6 0.8
1
0
0
.2
0
.4
0
.6
0
.8
1
0 0.2 0.4 0.6 0.8
1
0
0
.2
0
.4
0
.6
0
.8
1
0 0.2 0.4 0.6 0.8
1
0
0
.2
0
.4
0
.6
0
.8
1
(a) 3 SOMs. (b) 4 SOMs. (c) 5 SOMs.
Figure 6:
Self-organization by
n
SOMs.
0 0.2 0.4 0.6 0.8
1
0
0
.2
0
.4
0
.6
0
.8
1
0 0.2 0.4 0.6 0.8
1
0
0
.2
0
.4
0
.6
0
.8
1
(a) SOM
L
1
. (b) Extracted data.
Figure 7:
Extraction of dense parts of input data by SOM
L
.
man Problem," Neural Networks, Vol. 1, pp. 289-293,
1988
[3] H. Tokutaka, \Condensed Review of SOM and LVQRe-
search in Japan," Proc. of WSOM'97, pp. 322-329, 1997.
[4] M. Cottrell, E. de Bodt, M. Verleysen, \Kohonen Maps
Versus Vector Quantization for Data Analysis," Proc. of
ESANN'97, pp. 187-193, 1997.
[5] G. Simon, A. Lendasse, M. Cottrell, J.-C. Fort, M. Ver-
leysen, \Double SOM for Long-term Time Series Predic-
tion," Proc. of WSOM'03, pp. 35-40, 2003.
[6] M. Strickert and B. Hammer, \Neural Gas for Se-
quences," Proc. of WSOM'03, pp. 53-58, 2003.
[7] T. Ehara, H. Sasamura, T. Saito, \An Approach to the
TSP based on Growing Self-Organizing Maps," Proc. of
NOLTA'04, pp. 709-712, 2004.
0 0.2 0.4 0.6 0.8
1
0
0
.2
0
.4
0
.6
0
.8
1
0 0.2 0.4 0.6 0.8
1
0
0
.2
0
.4
0
.6
0
.8
1
(a) Extraction by SOM
L
2
. (b) Extraction by SOM
L
3
.
0 0.2 0.4 0.6 0.8
1
0
0
.2
0
.4
0
.6
0
.8
1
(c) Extracted by SOM
L
4
.
Figure 8:
Dense parts extraction.
- 310 -