Article

Forecasting nonlinear time series with a hybrid methodology

Applied Mathematics Letters (Impact Factor: 1.34). 09/2009; 22(9):1467-1470. DOI: 10.1016/j.aml.2009.02.006
Source: DBLP
ABSTRACT
a b s t r a c t In recent years, artificial neural networks (ANNs) have been used for forecasting in time series in the literature. Although it is possible to model both linear and nonlinear structures in time series by using ANNs, they are not able to handle both structures equally well. Therefore, the hybrid methodology combining ARIMA and ANN models have been used in the literature. In this study, a new hybrid approach combining Elman's Recurrent Neural Networks (ERNN) and ARIMA models is proposed. The proposed hybrid approach is applied to Canadian Lynx data and it is found that the proposed approach has the best forecasting accuracy.

Full-text

Available from: Cem Kadilar
Applied Mathematics Letters 22 (2009) 1467–1470
Contents lists available at ScienceDirect
Applied Mathematics Letters
journal homepage: www.elsevier.com/locate/aml
Forecasting nonlinear time series with a hybrid methodology
Cagdas Hakan Aladag
a,
, Erol Egrioglu
b
, Cem Kadilar
a
a
Department of Statistics, Hacettepe University, Ankara, Turkey
b
Department of Statistics, Ondokuz Mayis University, Samsun, Turkey
a r t i c l e i n f o
Article history:
Received 4 February 2009
Accepted 4 February 2009
Keywords:
ARIMA
Canadian lynx data
Hybrid method
Recurrent neural networks
Time series forecasting
a b s t r a c t
In recent years, artificial neural networks (ANNs) have been used for forecasting in time
series in the literature. Although it is possible to model both linear and nonlinear structures
in time series by using ANNs, they are not able to handle both structures equally well.
Therefore, the hybrid methodology combining ARIMA and ANN models have been used
in the literature. In this study, a new hybrid approach combining Elman’s Recurrent Neural
Networks (ERNN) and ARIMA models is proposed. The proposed hybrid approach is applied
to Canadian Lynx data and it is found that the proposed approach has the best forecasting
accuracy.
© 2009 Elsevier Ltd. All rights reserved.
1. Introduction
In recent years, the artificial neural networks (ANN) have been applied to many areas of statistics. One of these areas is
time series forecasting [1]. Since ANN can model both nonlinear and linear structures of time series, using neural networks
in forecasting can give better results than the other methods. Zhang et al. [2] review the literature of forecasting time series
using ANN.
Both theoretical and empirical findings in the literature show that combining different methods can be an affective and
efficient way to improve forecasts. Therefore, hybrid ARIMA and ANNs methods have been used for modeling both linear and
nonlinear patterns equally well. Pai and Lin [3] proposed hybrid ARIMA and support vector machines model. Tseng et al. [4]
combined seasonal time series ARIMA model and feedforward neural network (FNN). Zhang [5] proposed a hybrid ARIMA
and FNN model, composed of linear and nonlinear components as follows:
y
t
= L
t
+ N
t
, (1)
where y
t
denotes original time series, L
t
denotes the linear component and N
t
denotes the nonlinear component. Linear
component is estimated by ARIMA model and residuals obtained from the ARIMA model
e
t
= y
t
ˆ
L
t
, (2)
are estimated by FNN. Here
ˆ
L
t
is the forecasting value for time t of the time series y
t
by ARIMA. Zhang [5] claims that any
ARIMA model can be selected for the data as this does not affect the final forecast accuracy.
With n input nodes, the ANN model for the residuals can be written as
e
t
= f (e
t1
, e
t2
, . . . , e
tn
) + ε
t
, (3)
Corresponding author. Tel.: +90 312 2992016.
E-mail addresses: aladag@hacettepe.edu.tr (C.H. Aladag), erole@omu.edu.tr (E. Egrioglu), kadilar@hacettepe.edu.tr (C. Kadilar).
0893-9659/$ see front matter © 2009 Elsevier Ltd. All rights reserved.
doi:10.1016/j.aml.2009.02.006
Page 1
1468 C.H. Aladag et al. / Applied Mathematics Letters 22 (2009) 1467–1470
context units
1
1
x[k]
y[k]
output neurons
hidden neurons
external input neurons
u[k]
Fig. 1. Structure of an ERNN model [9].
where f is a nonlinear function determined by the FNN and ε
t
is the random error. The estimation of e
t
by (3) will yield
the forecasting of nonlinear component of time series, N
t
. By this way, forecasting values of the time series are obtained as
follows:
ˆ
y
t
=
ˆ
L
t
+
ˆ
N
t
. (4)
In the next section, we modify Zhang’s hybrid approach mentioned above. To obtain
ˆ
N
t
, we propose to use ERNN instead of
FNN. In Section 3, the proposed hybrid method is applied to Canadian lynx data which is also used in Zhang [5] and Kajitani
et al. [1]. By this way, we can compare the forecasting accuracy of the proposed method with the alternative methods. In
the last section, we discuss the results of the application.
2. The proposed hybrid method
ARIMA and seasonal ARIMA (SARIMA) models were introduced by Box and Jenkins [6] and these models have recently
been used successfully in forecasting linear time series. However, it is well known that the approximation of ARIMA models
to complex nonlinear problems is not adequate [5]. Therefore, nonlinear time series have been forecasted by using nonlinear
methods like ANNs. Although FNN has been used in many applications of ANNs, it is also possible to use recurrent neural
networks. One type of recurrent neural networks is ERNN which was introduced by Elman [7]. According to the general
principle of the recurrent networks, there is a feedback from the outputs of some neurons in the hidden layer to neurons
in the context layer which seems to be an additional input layer. In the case of comparison with other type of multilayered
network, the most important advantage of ERNN is a robust feature extraction ability, which provides feedback connections
from the hidden layer to a context layer [8]. The structure of an ERNN is illustrated in Fig. 1.
Zhang [5]’s hybrid approach uses FNN to estimate N
t
in (1). Since ERNN contains the context layer, it is certain that using
ERNN, instead of FNN, can improve forecasting accuracy. Therefore, we propose a new hybrid approach as follows:
Step 1. Box–Jenkins models are used to analyze the linear part of the problem. That is,
ˆ
L
t
is obtained by using Box–Jenkins
method.
Step 2. ERNN model is developed to fit the residuals from the Box–Jenkins models. That is,
ˆ
N
t
is obtained by using ERNN.
Step 3. Using (4), forecasts of the hybrid method are obtained by adding the estimates of linear and nonlinear components
of the time series, found in Step 1 and Step 2, respectively.
3. Application
The proposed hybrid method is applied to Canadian lynx data consisting of the set of annual numbers of lynx trappings
in the Mackenzie River District of North–West Canada for the period from 1821 to 1934. Canada lynx data, which is plotted
in Fig. 2, was also examined by Zhang [5] and Kajitani et al. [1], beyond the other various studies in the time series literature.
We would like to note that we use the logarithms (to the base 10) of the data in the analysis.
The proposed hybrid method is applied to the data as follows:
Firstly, Box–Jenkins method is used for estimating linear part of the problem. The Canadian lynx data shows a periodicity
of approximately 10 years. Because of this, the data is fitted by SARIMA (2, 0, 0) × (0, 1, 1)
10
model. We check that this
model satisfies all statistical assumptions such as no autocorrelation, homoskedasticity, etc. using Box–Pierce and White
Tests. Secondly, residuals obtained from SARIMA (2, 0, 0) × (0, 1, 1)
10
model are estimated by the ERNN model. Note that
the residuals are divided into training set (100 data points) and test set (last 14 data points). Number of input nodes is varied
from 1 to 12, number of hidden layer nodes is also varied from 1 to 12 and by this way 114 architectures are examined totally.
We find that the most appropriate ERNN architecture is 4 × 4 × 1. Thirdly, forecasts of last 14 years were obtained using
Page 2
C.H. Aladag et al. / Applied Mathematics Letters 22 (2009) 1467–1470 1469
1 112131415161718191101111
0
1000
2000
3000
4000
5000
6000
7000
8000
Fig. 2. Canadian lynx data series (1821–1934).
Fig. 3. Hybrid prediction of Canadian lynx data.
Table 1
Canadian lynx data forecasting results.
Method MSE
FNN 0.020
Zhang [5] Hybrid 0.017
Kajitani [1] SETAR 0.014
Proposed Hybrid 0.009
the proposed hybrid method. Finally, these forecasting values for last 14 years are shown in Fig. 3. Solid line represents the
original time series data and dot line represents the forecasts.
The mean square error (MSE) values for the last 14 observations of the proposed approach, Zhang [5] and Kajitani et al. [1]
are summarized in Table 1.
It is observed from Table 1 that the MSE of the proposed method is the smallest. Thus, it is concluded that the proposed
approach has the best forecasting values for this widely used data.
4. Conclusions
Since artificial neural networks (ANN) can model both nonlinear and linear structures of time series, using ANN can
give better results than other methods in forecasting. Therefore, in the literature, there have been many studies in which
time series are solved by using ANN in recent years [10,2,11]. One type of ANN is recurrent neural network and one of the
recurrent nets is ERNN.
Statisticians have studied to obtain better forecasts for long years and by these studies hybrid methods have been
improved in the literature. In this paper, we consider that using ERNN instead of FNN in Zhang’s hybrid method should
improve the forecasting accuracy. Therefore, we propose a hybrid ARIMA and recurrent neural network model. It is observed
that the proposed method yields better result than other methods for Canadian lynx data. It is well known that forecasting
accuracy of ERNN is better than FNN, because of containing a context layer. Since ERNN is used in the proposed hybrid
approach, as expected this approach is found better than Zhang [5]’s hybrid approach. In the future work we hope to increase
the forecasting accuracy by changing the type of ANN used in hybrid methods such as Jordan recurrent neural networks [12].
References
[1] Y. Katijani, W.K. Hipel, A.I. Mcleod, Forecasting nonlinear time series with feedforward neural networks: A case study of Canadian lynx data, Journal
of Forecasting 24 (2005) 105–117.
[2] G. Zhang, B.E. Patuwo, Y.M. Hu, Forecasting with artificial neural networks: The state of the art, International Journal of Forecasting 14 (1998) 35–62.
Page 3
1470 C.H. Aladag et al. / Applied Mathematics Letters 22 (2009) 1467–1470
[3] P.F. Pai, C.S. Lin, A hybrid ARIMA and support vector machines model in stock price forecasting, The International journal of Management Science 33
(2005) 497–505.
[4] F.M. Tseng, H.C. Yu, G.H. Tzeng, Combining neural network model with seasonal time series ARIMA model, Technological Forecasting & Social Change
69 (2002) 71–87.
[5] G. Zhang, Time series forecasting using a hybrid ARIMA and neural network model, Neurocomputing 50 (2003) 159–175.
[6] G.E.P. Box, G.M. Jenkins, Time Series Analysis: Forecasting and Control, Holdan-Day, San Francisco, CA, 1976.
[7] J.L. Elman, Finding structure in time, Cognitive Science 14 (1990) 179–211.
[8] S. Seker, E. Ayaz, E. Turkcan, Elman’s recurrent neural network applications to condition monitoring in nuclear power plant and rotating machinery,
Engineering Applications of Artificial Intelligence 16 (2003) 647–656.
[9] D.T. Pham, D. Karaboga, Training Elman and Jordan networks for system identification using genetic algorithms, Artificial Intelligence in Engineering
13 (1999) 107–117.
[10] T.Y. Kim, K.J. Oh, C. Kim, J.D. Do, Artificial neural networks for non-stationary time series, Neurocomputing 61 (2004) 439–447.
[11] G. Zhang, B.E. Patuwo, Y.M. Hu, A simulation study of artificial neural network for nonlinear time series forecasting, Computers & Operations Research
28 (2001) 381–396.
[12] M.I. Jordan, Attractor dynamics and parallelism in a connectionist sequential machine, in: Conference of the Cognitive Science Society (1986).
Page 4
    • "In general, this fusion of models increases model accuracy and reduces overfitting problems. For example, Aladag et al. [24] proposed a hybrid model linking ARIMA and ANN models; this hybrid model was applied to yearly demand forecast providing accurate results. A similar approach is used in [14] modelling water demand intervened (e.g. by open/close valve manoeuvres). "
    [Show abstract] [Hide abstract] ABSTRACT: The most important factor in planning and operating water distribution systems is satisfying consumer demand. This means continuously providing users with quality water in adequate volumes at reasonable pressure, thus ensuring reliable water distribution. In recent years, the application of statistical, machine learning, and artificial intelligence methodologies has been fostered for water demand forecasting. However, there is still room for improvement; and new challenges regarding on-line predictive models for water demand have appeared. This work proposes applying support vector regression, as one of the currently better machine learning options for short-term water demand forecasting, to build a base prediction. On this model, a Fourier time series process is built to improve the base prediction. This addition produces a tool able to eliminate many of the errors and much of the bias inherent in a fixed regression structure when responding to new incoming time series data. The final hybrid process is validated using demand data from a water utility in Franca, Brazil. Our model, being a near-real time model for water demand, may be directly exploited in water management decision-making processes.
    No preview · Article · Feb 2016 · Journal of Computational and Applied Mathematics
  • Source
    • "At the solutions smallest among th We concl where m= In additio set obtain in the lite Unauthenticated Download Date | 1/16/16 3:10 PM RB-ANN, Zhang [19], Aladag et al. [20] and Yolcu et al. [21]. – The model orders in other words the number of inputs of MNM-ANN-DEA (m) was changed from 1 to 4. "
    [Show abstract] [Hide abstract] ABSTRACT: In recent years, artificial neural networks have been commonly used for time series forecasting by researchers from various fields. There are some types of artificial neural networks and feed forward artificial neural networks model is one of them. Although feed forward artificial neural networks gives successful forecasting results they have a basic problem. This problem is architecture selection problem. In order to eliminate this problem, Yadav et al. (2007) proposed multiplicative neuron model artificial neural network. In this study, differential evolution algorithm is proposed for the training of multiplicative neuron model for forecasting. The proposed method is applied to two well-known different real world time series data.
    Full-text · Article · Jan 2016 · Journal of Artificial Intelligence and Soft Computing Research
  • Source
    • "Tseng et al. [38] proposed a hybrid method that employs a seasonal autoregressive integrated moving-average model and multilayer perceptron ANN (MLP-ANN). Zhang et al. [39], BuHamra et al. [40], Pai and Lin [41], Wang et al. [42], Jain and Kumar [43], Chen and Wang [44], Aladag et al. [45] and Lee and Tong [46] proposed hybrid algorithms. These hybrid methods have two stages. "
    [Show abstract] [Hide abstract] ABSTRACT: Non-probabilistic forecasting methods are commonly used in various scientific fields. Fuzzy-time-series methods are well-known non-probabilistic and nonlinear forecasting methods. Although these methods can produce accurate forecasts, linear autoregressive models can produce forecasts that are more accurate than those produced by fuzzy-time-series methods for some real-world time series. It is well known that hybrid forecasting methods are useful techniques for forecasting time series and that they have the capabilities of their components. In this study, a new hybrid forecasting method is proposed. The components of the new hybrid method are a high-order fuzzy-time-series forecasting model and autoregressive model. The new hybrid forecasting method has a network structure and is called a fuzzy-time-series network (FTS-N). The fuzzy c-means method is used for the fuzzification of time series in FTS-N, which is trained by particle swarm optimization. Istanbul Stock Exchange daily data sets from 2009 to 2013 and the Taiwan Stock Exchange Capitalization Weighted Stock Index data sets from 1999 to 2004 were used to evaluate the performance of FTS-N. The applications reveal that FTS-N produces more accurate forecasts for the 11 real-world time-series data sets.
    Full-text · Article · Sep 2015 · Applied Intelligence
Show more