ATMOSPHERE - Automatic Track Mining and Objective Satellite Pattern Hunting System Using Enhanced RBF and EGDLM.

Full text

ATMOSPHERE - Automatic Track Mining and Objective Satellite
Pattern Hunting system using Enhanced RBF and EGDLM
Raymond S. T. Lee and James N. K. Liu
Hong Kong Polytechnic University
Hung Hom, Kowloon, Hong Kong
csstlee@comp.polyu.edu.hk, csnkliu@comp.polyu.edu.hk,
Abstract
Severe weather prediction, such as tropical cyclone (TC)
forecast is a typical data mining and forecasting problem
that involves high level data manipulation and interpretation
of meteorological information such as satellite pictures and
other meteorological observation data. In this paper, we
present a fully automatic and integrated system known as
"ATOMOSPHER" - Automatic Track Mining and Object
Satellite Pattern Hunting system using Enhanced RBF and
EGDLM - to provide a neural network based TC
identification and tracking system. The proposed system
consists of two main modules: 1) Object Dvorak technique
for TC satellite pattern identification based on an Elastic
Graph Dynamic Link Model (EGDLM) and 2) TC tracking
system based on an Enhanced Radial Basis Function (RBF)
network model.
For system evaluation, 120 TC cases appeared in the
period from 1985 to 1998 (provided by National Oceanic
and Atmospheric Administration (NOAA)) are adopted.
Promising results of over 87% of TC pattern segmentation
and 97% of correct classification rate are attained
respectively. For TC tracking, an overall of over 86%
correct prediction result is achieved.
Keywords: EGDLM, Enhanced RBF, Track mining, TC
identification, time series prediction.
1. Introduction
Time series prediction so far is one of the most vital
research topics, not only because of its significant practical
values, ranging from stock prediction (Liu and Lee 1997;
Liu and Tang 1996) to general weather forecast such as
rainfall prediction (Li et. al. 1998; Lee and Liu 1999c; Liu
and Lee 1999) in meteorology, but also of its academic
values.
In a typical weather prediction scenario, factors that
affect the coming weather depend not only on local weather
elements such as temperatures, relative humidity, air
pressure, wind speed and directions, but also regional
weather elements such as the effect from global weather
pattern, for example El Nino effect (Bao and Xiang, 1991).
This constitutes to the handling of a huge amount of
information including "extraction", "filtering",
"interpretation", "discrimination" and "processing", and
also a sophisticate data mining and knowledge discovery
Copyright © 2000, American Association for Artificial Intelligence
(www.aaai.org). All rights reserved.
process as well. Especially in the case of severe weather
prediction such as topical cyclone (TC) forecast, an
additional level of complexity with the usage and
interpretation of satellite and radar images is being
imposed. Unlike those weather elements such as
temperature and pressure which can be predicted by
classical numerical modeling (Liu 1988) or contemporary
neural network models (Liu and Lee 1999), these imagery
data are highly variant in the sense that so far only
subjective human interpretations by the Dvorak technique
is adopted (Dvorak 1973, 1975).
This paper presents an integrated model that provides an
effective and fully automatic system for the tropical
cyclone (TC) identification and track prediction called
ATMOSPHERE (Automatic Track Mining and Satellite
Pattern Hunting system using Enhanced RBF and
EGDLM). The proposed model consists of two main
modules: 1) TC pattern recognition system from satellite
pictures known as Elastic Graph Dynamic Link Model
(EGDLM), a neural network based model that involves the
automatic TC pattern segmentation and elastic pattern
matching from the pre-defined TC templates, a process that
simulates human TC identification technique called Dvorak
analysis (Dvorak 1973, 1975); 2) A time series TC
intensity and track mining system using Enhanced Radial
Basis Function network, a time series recurrent neural
network prediction model that integrates the conventional
RBF network with Time Difference and Structural
Learning (TDSL) technique.
The paper is organized as follows. Section 2 will present
the EGDLM for automatic TC pattern identification.
Section 3 will discuss the Enhanced RBF model for TC
intensity and track mining. In section 4, an overview of
system implementation using 120 TC cases appeared
between 1985 to 1998 (information provided by National
Oceanic and Atmospheric Administration (NOAA)) will be
presented. Comparisons of the proposed system with other
contemporary TC tracking system will be conducted as
well, together with the conclusion discussed in Section 5.
2. Objective Satellite Pattern Identification of
Tropical Cyclone using EGDLM
2.1 Dvorak Technique for TC Identification
In view of the various meteorological phenomena
interpreted from satellite images, one of the most valuable

contributions is the identification of tropical cyclones -
including storms, extra-tropical cyclones, typhoon and
hurricanes that threaten numerous human lives and
properties.
One of the worldwide accepted methods for TC
identification and interpretation was developed by Dvorak
(1973; 1975) known as Dvorak technique. From his theory,
each TC may go through a life cycle that can be classified
into eight categories ranging from TC1 to TC8 (Figure 1).
In addition to classifying the tropical cyclones, Dvorak
analysis also provides an effective scheme to determine the
current strength of the cyclones from the satellite images
which are based on the "T-number" (T1-T8) determined
from Dvorak technique.
Figure 1 - Dvorak TC templates from T1 to T8 (Each T-
number with eight sub-categories corresponds to eight
different possible appearance of the TC patterns)
Nowadays, Dvorak technique is still the worldwide
agreed official tool for the determination of TC intensity.
But due to the highly variation of TC patterns that can
appear in satellite pictures, the Dvorak technique is highly
subjective which depends on the human justification done
by the weather forecasters and meteorologists.
2.2 A Perspective of EGDLM for Invariant TC
Pattern Recognition
In this paper, an elastic attribute graph recognition model
known as Elastic Graph Dynamic Link Model (EGDLM) is
proposed to provide a fully automatic and objective
solution for Dvorak technique on TC pattern identification.
In short, object recognition using EGDLM is based on
the framework of Dynamic Link Architecture (DLA)
(Malsburg 1981) which describes the recognition problem
as an elastic graph matching mechanism between the
attribute graphs of the image vectors (input layer) with the
set of model graphs in the "memory layer". The neural
interactions are governed by the onset/offset of the dynamic
links between the layers which simulate the functionality of
the dynamic memory association in the Short-term
memory. Active researches in this area have been done
including handwritten character recognition (Liu and Lee
1997, 1998; Lee and Liu 1999e), human face recognition
(Lee et. al. 1999, Lee and Liu 1999d) and TC pattern
recognition with the integration of Active Snake model for
object segmentation based on elastic templates (Lee and
Liu 1999a, 2000).
Different from the traditional DLA model, EGDLM
makes use of the Composite Neural Oscillatory model to
facilitate a fully automatic object segmentation scheme.
Recent research of such also involves scene analysis (Lee
and Liu 1999b). A schematic diagram of EGDLM for TC
pattern matching is shown in Figure 2.
Figure 2 - EGDLM for TC Pattern Identification
2.3 Invariant Properties
One of the most striking features of the EGDLM is the
“invariant” property. In the network model, only the
topological relations between the composite neural
oscillators (ie. dynamic links) are encoded into the
network. The pattern matching process is resembled to the
“Elastic Graph Matching” which is invariant under various
transformations such as translation, rotation, reflection,
dilation and occlusion. This occurs commonly in natural
scenes (Lee and Liu 1999b).
Composite Neural
Oscillatory model
Phase I
Feature
Extraction
Module
Phase II
Figure-ground
Segmentation
Module
Phase III
EGDLM TC Pattern
Matching Module
Visible satellite
image
Extract Each
Pattern
EGDLM Encoding Scheme
(Dynamic Link Encoding)
EGDLM Pattern Matching
with Dvorak Template Patterns
Return matching results
(TC number, “eye” position)
Gabor filter
Extraction
Module

3. Enhanced Radial Basis Function (RBF)
Network for TC Track Mining
3.1 Enhanced RBF Network - System Overview
The proposed Enhanced RBF Network (ERBFN)
incorporates with two main technologies into the
conventional RBF network for temporal time series
prediction problem: 1) Structural learning technique that
integrates the “forgetting” factor into the RBF BP
algorithm; 2) A Time Difference with Decay (TDD)
method is corporated into the network to strengthen the
temporal time series relation of the input data sequence for
network training. A schematic diagram of the proposed
network is shown in Figure 3.
Figure 3 - Schematic diagram of the Enhanced RBF
Network
The ERBFN shown in Figure 3 consists of three layers.
The first layer is the input layer which consists of two
portions: 1) Past network outputs that feedback into the
network; 2) Major co-relative variables are concerned with
the prediction problem. Past network outputs enter into the
network by time-delay unit as the first inputs. These
outputs are also affected by a decay factor γ that is
governed by the following equation.
(1)
where
λ
is the decay constant,
α
is the normalization constant
and k is the forecast horizon.
In general, the time series prediction of the proposed
network is to predict the outcome of the sequence x1t+k at
the time of t+k that is based on the past observation
sequence of size n, i.e. x1t, x1t-1, x1t-2, x1t-3, …, x1t-n+1 and the
major variables that influence the outcome of the time
series at time t. For convenience, the following notations
are used throughout the following network description: The
numbers of input nodes in the first and second portions are
set to n and m respectively. The number of hidden nodes is
set to p. The predictive steps are set to k, so the number of
output nodes is k. At time t, the inputs will be [x1t, x1t-1, x1t-
2, x1t-3,…,x1t-n+1] and [x21,x22,…,x2m] respectively. The
output is given by xt+k, denoted by pkt for simplicity, wijt
denotes the connection weight between the i-th node and
the j-th node at time t.
3.2 Enhanced RBF Network - Structural Learning
Algorithms
The main idea of RBF learning algorithm with a
“forgetting” factor is to introduce a constant decay to
connected weights that make the redundancy weight(s) fade
out quickly. The cost function of the structural learning
algorithm is given by equation (2).
(2)
where E1t denotes the error square in traditional RBF learning,
the second term is the penalty criteria.
If delta rule is used, the learning rule of the weights is
given by:
(3)
where the first term in the first line is the weight change obtained
by the traditional RBF network,
η
is the learning rate and
α
is
the momentum. Besides,
ε
=
ηε
’ is the “forgetting” factor.
By using this structural learning method, the main
“skeleton” of the network can be constructed with weight
adapting over a time series of training.
3.3 Enhanced RBF Network – TDD Method
The structural learning algorithm discussed above does
provide a “dynamic” structure building of the neural
network, but it cannot adapt the temporal time series
relations of the input and output feedback data sequences
into the model. In order to code with this problem, a
temporal difference method with decay feedback is
hybridized into the learning algorithm of the proposed
model. The basic concepts are presented as follows.
In a typical time series prediction problem, given a series
of past observations of time-step n at time t, i.e. [x1t, x1t-1,
x1t-2, x1t-3,…,x1t-n+1], with the predictive time-step of k, we
not only obtain the predicted output at time t+k, i.e. pkt , but
more importantly is the sequence of future events starting
from time t, i.e. [ p1t, p2t, p3t …, pkt ]. In other words, the
network can provide an overlapping and inter-related event
sequence as an additional “hint” for network learning,
which can be implemented by using Temporal Difference
technique (Sutton 1983). Besides, by considering a
sequence of temporal difference operations from time t+1
to t+k, the prediction from a “nearer” future normally has a
higher level of confidence than a “far” future, so a decay
operator is integrated into the learning algorithm in order to
reflect the situation.
k
e
λ
αγ
−
=
X1tX1t-1 X1t-n+1 X21X22X2m
. . . . . . . . . . . .
. . . . . .
. . . . . .
p1tp2tpkt
output layer
hidden layer
input layer
γ(k)
past observations
feedback with major input
variables
∑
+= ji
t
ij
tt wEE ,
'
1
ε
)sgn(1
1t
ij
t
ij
t
ij
t
ij
t
ij
t
t
ij wwww
w
E
w
εααη
−∆+∆=∆+
∂
∂
−=∆ +

With the integration of TDD methodology, the learning
algorithm discussed in equation (3) will be modified into:
(4)
where
ε
” is defined as:
(5)
From equation (4), the learning algorithm consists of
three basic components, the first two terms refer to the
error adjustment based on the temporal differences between
the output event sequences between the two consecutive
time steps, which is “weighted” by the decay functions γ(l),
γ(h) given by equation (1). The third and fourth terms refer
to the structural learning operations, which is explained in
equation (4). Also, λ denotes the exponential scale for a
connection weight that determines the effective length of
the history windows.
4. ATMOSHPERE - System Implementation
4.1 Introduction
In sampling data for simulation, 120 tropical cyclones
cases appeared in the period between 1985 and 1998 were
identified. All the time series (3-hourly) satellite images
and grid point meteorological data are provided by
National Oceanic and Atmospheric Administration
(NOAA). For the grid point meteorological data being used
in the time series prediction, meteorological data including:
mean sea level air pressure (MSLP), surface and upper-air
(700mb, 500mb, 300mb) wind speed and direction, dry
bulb and wet bulb temperature, that is, all the data that will
affect the “steering” motion and development of TC are
considered.
System implementation tests of ATMOSHPEREE are
mainly divided into two phases: 1) TC identification tests
based on the TC pattern recognition using EGDLM; 2) TC
intensity and track mining tests using Enhanced RBFN
Network (ERBFN). Comparisons on system performance
with other neural network models and bureau TC tracking
system such as OCTM and TKS will be conducted as well.
The whole system implementation and performance
evaluations were carried out on Sun Sparc 20 workstation.
4.2 TC Pattern Recognition Tests
In the tests, for each of the 120 TC cases, 5 satellite images
were randomly chosen for the test, so totally 600 satellite
pictures were being used. Two set of tests were conducted
for TC pattern recognition using EGDLM:
1) TC patterns segmentation tests
TC patterns segmentation done by EGDLM model using
Composite Neural Oscillating technique verse that using
Active Contour Model (Lee and Liu 1999a) is shown in
Table I. A snapshot of the segmented TC patterns from a
satellite image that contains four TCs in 1997 is shown in
Figure 4. TABLE I
TC PATTERNS SEGMENTATION COMPARED WITH ACM MODEL
Segmentation Rate
Segmentation
Models TC 1-3 TC 4-6 TC 7-8
Av.
Speed*
(sec)
EGDLM 92% 96% 99.5% 50 sec
Hybrid ACM 80% 88.2% 97% 195 sec
2) TC patterns recognition tests
In the test, 600 satellite images were undergone TC
segmentation and they were matched with the Dvorak
templates (Figure 1) using the elastic graph matching
technique of EGDLM. Two sets of tests were conducted:
They were "TC Pattern Classification Test of the 600
satellite images" and "TC “Eye” Position Identification
Test". Results are presented in Tables II and III.
Figure 4 – Segmented TC patterns from satellite image
TABLE II
TC PATTERN CLASSIFICATION RESULTS
No. of matches for each category
TC cat. 1-3
Total no.:
245
TC cat. 4-6
Total no. :
276
TC cat. 7-8
Total no.
231
Overall
Total no.:752
EGDLM 229
(93%) 271
(98%) 231
(100%) 731
(97%)
Hybrid ACM 178
(73%) 223
(81%) 211
(91%) 612
(81%)
TABLE III
TC “EYE” POSITION IDENTIFICATION RESULTS
No. of
TC No. of
matches
Recognit-
ion rate
(%)
“eye”
position
deviation*
EGDLM 278 99% 2.3 km
Hybrid ACM 281 250 89% 3.0 km
Remark :TC “Eye” position deviation amounts are calculated by the deviation from
the TC “eye” location reported by reconnaissance aircraft
()
)sgn(")()(
)((
1
1
1
01
1
1
2
1
1
1
t
ij
t
ij
t
lt
ij
l
ttt
t
lt
ij
l
h
t
k
h
t
h
t
h
t
ij
ww
w
p
lpp
w
p
pphw
εαλγ
λγη
−∆+







∂
∂
⋅⋅−
+
∂
∂
−⋅=∆
∑
∑∑
=
−
=
−
=
+
−
+


=<
=otherwise
wt
ij
0
"
θε
ε

4.3 TC Intensity and Track Mining Tests
Using the Enhanced RBF network (ERBFN) for time series
TC tracking mining, the 120 TC cases were randomly
divided into two sets. The first 60 TC cases were used for
network training while the rest used for system testing.
For system evaluation, two types of tests were
conducted: 1) Evaluation for the system performance of the
Enhanced RBF Network (ERBFN) against two different
time series neural network prediction models: conventional
RBF network and BBPT (Williams and Zipser, 1995)
recurrent network; 2) Comparisons of the TC track mining
performance of the proposed ERBFN with the bureau TC
tracking systems: OCTM and TKS systems.
In order to provide a systematic performance measure of
the proposed model against other neural network prediction
tools and the enhancement achieved as compared with the
conventional RBF model, the comparison with two other
time series prediction models was taken. This included: 1)
Conventional RBF network; 2) Backpropagation Time
Series Recurrent Network (BBTT). Network training and
testing results are shown in Table IV. Storm tracks being
successfully mined by all the models for TC Bonnie (1998)
against the “Actual” TC track are shown in Figure 5.
TABLE IV
NETWORKS TRAINING AND TESTING RESULTS
TC Intensity
Prediction TC Track
Prediction
Network Models MSE
training MSE
testing MSE
training MSE
testing
ERBFN 0.014 0.061 0.013 0.079
RBF model 0.412 0.513 0.493 0.581
BPTT model 0.082 0.103 0.093 0.112
Compared with the conventional RBF model, the
proposed Enhanced RBF model has achieved a significant
enhancement by over 20 and 10 times in mean square error
(MSE) in TC intensity and track mining. Even when
compared with the BPTT recurrent network, an overall
40% and 30% improvement in these dimensions have been
attained respectively.
A quantitative evaluation of the proposed model was
performed and tested for its applicability in real time TC
track mining as compared with that of the bureau TC Track
mining systems: 1) One-way interactive Tropical Cyclone
Model (OTCM) used by the JTWC in Guam; 2) Track
Forecast System (TKS) – Enhanced numerical prediction
forecast used in Central Weather Bureau in Taiwan.
In the experiment, 55 TCs appearing during the period
between 1989 to 1992 were used. Track mining accuracy
is determined by the great circle distance between the
forecast position and the best track position provided by
the Central Weather Bureau of Taiwan. Table V shows the
comparison results of the Enhanced RBF model with
OTCM and TKS models based on the 48-hour TC position
forecast during the year 1989-90, detailed TC track mining
accuracy figures (in km) for each TC were given for
illustration.
Figure 5 – TC tracks predicted by different models Vs. actual TC
track for TC Bonnie (1998)
TABLE V
COMPARISON OF THE HRBF MODEL WITH BUREAU TC TRAC4KING
SYSTEMS AT 48-HOUR POSITION FORECAST (1989-1990)
TC/ year No. of
cases Enhanced
RBF OTCM TFS
1989 52 296 486 425
1990 41 309 305 322
Overall 93 301 406 379
As shown in Table V, for the 18 TCs appeared in the
period between 1989 and 1990, the average 48-hour
forecast error reported by Enhanced RBF, OTCM and TFS
are 301 km, 406 km and 379 km respectively. The overall
improvement comparing with the two models are
respectively OTCM (25.8%) and TFS (25.9%),
corresponding to less than 3° error. This represents a
significant improvement to predict the landfall of TC for
warning and precaution measures.
Comparing with the TC track mining results between
1989 and 1990, it can be seen that the errors are somewhat
correlated. The track forecasts also depend on the
capability in predicting the large scale flow, which is
believed to be related to the type of flow regimes. When
interpreting Table V, it is worthwhile to realize that 1989 is
one of the worst years for OTCM performance. However,
the HRBF model still maintains promising track mining
capability, giving a good prediction of cyclone movement.
5. Conclusion
This paper proposed a fully integrated and automatic
system called ATMOSPHERE (Automatic Track Mining
and Objective Satellite Pattern Hunting system using
Enhanced RBF and EGDLM) for TC identification and
time series intensity and track prediction. Such a task is

historically highly dependent on human subjective
justification on vast supply of information.
The main contributions of the ATMOSPHERE is two
folds: 1) Automate the Dvorak technique for TC
identification from satellite images, a process that highly
depends on human intervention. With the adoption of
EGDLM, the system provides a fully automatic, accurate
(over 97% accuracy) solution in a reasonable speed; 2) The
adaptability and real time learning track learning technique,
which is particularly important for TC Track mining
problem due to the rapid development of TC in severe
weather conditions. This is also the major weakness of
numerical prediction models in forecasting these severe and
locally developed weather phenomena.
Acknowledgement
We are grateful to NOAA for the provision of
meteorological data and satellite pictures for TC between
1985 to 1998. The authors are also grateful to the partial
supports of the Central Research Grants G-S484 of Hong
Kong Polytechnic University.
References
Bao C. L. and Xiang Y. Z., 1991. Impact of El Nino Event on the
Subtropical High and Meteorological: Oceanic Disaster. In
Proceedings of CIES: 25-30, Japan.
Dvorak, V. F., 1973. A Technique for the Analysis and
Forecasting of Tropical Cyclone Intensities from Satellite
Pictures. NOAA Tech. Memo., NESS 45, U. S. Department of
Commerce, Washington, D. C.
Dvorak, V. F., 1975. Tropical Cyclone Intensity Analysis and
Forecasting from Satellite Imagery. Mon. Wea. Rev. 103: 420-
430.
Lee, R.S.T. and Liu, J.N.K., 1999a. An Automatic Satellite
Interpretation of Tropical Cyclone Patterns Using Elastic Graph
Dynamic Link Model. To appear in International Journal of
Pattern Recognition and Artificial Intelligence.
Lee, R.S.T. and Liu, J.N.K., 1999b. An Oscillatory Elastic Graph
Matching Model for Scene Analysis. In Proceedings of
International Conference on Imaging Science, Systems, and
Technology (CISST'99): 42-45, Las Vegas, Nevada, USA.
Lee, R. and Liu, J., 1999c. Teaching and Learning the A. I.
Modeling”, Innovative Teaching Tools for Neural Nets, Fuzzy
Systems and Genetic Algorithms, eds. L. Jain, CRC Press: 31-86.
Lee, R.S.T. and Liu, J.N.K. 1999d. An Integrated Elastic Contour
Fitting and Attribute Graph Matching Model for Automatic Face
Coding and Recognition”, In Proceedings of the Third
International Conference on Knowledge-Based Intelligent
Information Engineering Systems (KES’99): 292-295, Australia.
Lee, Raymond S. T. and Liu, James N. K., 1999e. An Oscillatory
Elastic Graph Matching Model for Recognition of Offline
Handwritten Chinese Characters. In Proceedings of the Third
International Conference on Knowledge-Based Intelligent
Information Engineering Systems (KES’99): 284-287, Adelaide,
Australia.
Lee, Raymond S. T. and Liu, James N. K., 2000. An approach
Using Elastic Graph Dynamic Link Model for Automating the
Satellite Interpretation of Tropical Cyclone Patterns”. To appear
in Invariants for Pattern Recognition and Classification: Series in
Machine Perception and Artificial Intelligence, World Scientific
Publishing.
Lee, R., Liu, J and You, Y., 1999. Face Recognition: Elastic
Relation Encoding and Structural Matching. In Proceedings of
IEEE International Conference on Systems, Man, and
Cybernetics (SMC'99): 172-177, Tokyo, Japan.
Li, B., Liu, J. and Dai, H., 1998. Forecasting from Low Quality
Data with Applications in Weather Forecasting. Int. Journal of
Computing and Informatics, 22(3): 351-358.
Liu, N. K., 1988. Computational Aspects of a Fine-mesh Sea
Breeze Model. M. Phil. Dissertation, Department of Mathematics,
Murdoch University, Western Australia.
Liu, N. K. and Lee, K. K., 1997. An Intelligence Business
Advisor for Stock Investment. Expert Systems, 14(3): 129-139,
Blackwell Publishers, U. K.
Liu, J.N.K. and Lee, R.S.T., 1997. Invariant Character
Recognition in Dynamic Link Architecture. In Proceedings of
IEEE KDEX97: 188-195, Newport Beach USA.
Liu, J. N. K. and Lee, R. S. T., 1998. Invariant Handwritten
Chinese Character Recognition by Dynamic Link Architecture",
In Proceedings ICONIP/JNNS'98 (1): 275-278, Kitakyushu,
Japan.
Liu, J. N. K. and Lee, R. S. T., 1999. Rainfall Forecasting from
Multiple Point Source Using Neural Networks. In Proceedings of
IEEE International Conference on Systems, Man, and
Cybernetics (SMC'99): 429-434, Tokyo, Japan.
Liu, J. N. K. and Tang, T. I., 1996. An Intelligence System for
Financial Market Prediction. In Proceedings of the 2nd South
China International Business Symposium: 199-209.
Malsburg, 1981. Nervous Structures with Dynamic Links.
Berichte Bunsegesellschaft for Physical Chemistry, 89: 703-710.
Sutton, R. S., 1983. Learning to Predict by the Methods of
Temporal Difference. Machine Learning, 3:8-44.
Williams R.J. and Zipser D., 1995. Gradient-based Learning
Algorithm for Recurrent Network. Backpropagation: Theory,
Architecture and Application, Hillsdale, NJ, Erlbanum.