Content uploaded by Filipe R. Ramos
Author content
All content in this area was uploaded by Filipe R. Ramos on Oct 17, 2021
Content may be subject to copyright.
Didier Lopes
UNL and CEDOC-UNL
dro.lopes@campus.fct.unl.pt
Contacts
1. Aue, A., & Horváth, L. (2013). Structural breaks in time series. Journal of Time Series Analysis,34(1), 1-16.
2. Costa, A., Ramos, F., Mendes, D., & Mendes, V. (2019). Forecasting financial time series using deep learning techniques. In IO 2019 -XX Congresso da APDIO 2019. Instituto Politécnico de Tomar - Tomar.
3. Greff, K., Srivastava, R. K., Koutník, J., Steunebrink, B. R., & Schmidhuber, J. (2015). LSTM: A Search Space Odyssey. IEEE Transactions on Neural Networks and Learning Systems,28(10), 2222–2232.
4. Hang, N. T. (2019). Research on a number of applicable forecasting techniques in economic analysis, supporting enterprises to decide management. World Scientific News,119,52–67.
5. Jozefowicz, R., Zaremba, W., & Sutskever, I. (2015). An Empirical Exploration of Recurrent Network Architectures. In ICML - International Conference on Machine Learning.
6. Kirchgässner, G., & Wolters, J. (2007). Introduction to Modern Time Series Analysis (Springer). Berlin.
7. Lopes, D., Ramos, F. (2020) Univariate Time Series Forecast. Retrieved from https://github.com/DidierRLopes/UnivariateTimeSeriesForecast.
8. Ramos, F., Costa, A., Mendes, D., & Mendes, V. (2018). Forecasting financial time series: a comparative study. In JOCLAD 2018, XXIV Jornadas de Classificação e Análise de Dados. Escola Naval –Alfeite.
9. Ravichandiran, S. (2019). Hands-On Deep Learning Algorithms with Python: Master deep learning algorithms with extensive math by implementing them using TensorFlow. Packt Publishing Ltd.
10. Tkác, M., Verner, R. (2016) Articial neural networks in business: Two decades of research. Applied Soft Computing,38,788-804. doi:10.1016/J.ASOC.2015.09.040
11. Wilson, J. H., Spralls III, S. A. (2018) What do business professionals say about forecasting in the marketing curriculum?. International Journal of Business, Marketing, Decision Science.11(1), 1-20.
12. Zhang, Y., Guo, Q., & Wang, J. (2017). Big data analysis using neural networks, 49, 9–18.
References
In a year where the word “forecast” has been
extensively used, it’s more important than ever to
have accurate forecasting models. In particular, in
economic-finance and business areas; these types of
forecasting techniques are used to support
enterprises to decide a company’s direction, which
helps determine the success of these same
enterprises. However, in order for the forecasting
techniques to be efficient, these must be truly
understood and tested in real context, not only to
improve already existing models but also test new
approaches. From the methodologies most cited in
the scientific literature, the classical models, such as
autoregressive moving average (ARMA) and
exponential smoothing (ETS), are the most utilised by
professionals. Nonetheless, due to promising results,
the literature has been keen on Deep Learning
methodologies, in particular Deep Neural Networks
(DNN). In fact, investigating what type of models
should be used for each time-series based on their
characteristics is the goal of this work. Three distinct
models –ARMA, ETS and DNN –are assessed in the
forecast of time-series with distinct patterns. The
discussion of the results will take into account not
only the forecasting ability, but also its interpretability
and computational cost.
Keywords: ARMA, DNN, ETS, forecasting, time-series
Introduction
Models
▪Forecast and Prediction error
Considering the time series and the past
observations from period , and being an
unknown value in the future and its
forecast, the prediction error corresponds to,
- Mean Absolute Error (MAE):
- Mean Absolute Percentage Error (MAPE):
Where corresponds to the forecasting window, i.e.
number of observation in the forecasting samples
▪Computational implementation
For classical models, we relied on Akaike Information
Criteria (AIC), Bayesian Information Criterion (BIC)
and Hannan–Quinn Information Criterion (HQIC) to
assess the relative quality of the statistical model for
several different parameters for a given set of data.
For neural networks, we relied on the following cross-
validation and tuning methodology (Figure 2).
Methodology
Conclusion
▪Autoregressive Moving Average (ARMA)
- models: combination of an
Autoregressive model of order , , with a
Moving Average model of order ,.
- models: ARIMA with an integration
component ,until the time-series achieves
stationarity.
- models: seasonal
ARIMA used when time-series shows seasonality.
▪Exponential Smoothing (ETS)
Model that explicitly uses an exponentially decreasing
weight for past observations. These may contain a
level, trend and season component. In addition, the
model may be additive or multiplicative.
▪Deep Neural Networks (DNN)
Within DNN there are three main architectures,
where their difference is in the learning process, and
occurs at the neuron level (Figure 1).
.
- Multilayer Perceptron (MLP): Static model, where
each neuron receives an input vector
and a bias.
- Recurrent Neural Network (RNN): Dynamic model,
where in each training epoch, t,the neuron is fed by
an input vector
and a learning input that is a
result from the previous training epoch, t-1.
- Long-Short Term Memory (LSTM): Based on RNN,
LSTMs utilize gates at the neuron level to allow for
long-term memory, for past data deemed important.
Figure 1. Comparison of a hidden cells of: MLP, RNN and LSTM
▪CPIAUSCL:Consumer Price Index for All Urban
Consumers –All Items in U.S. City Average (873
monthly observations –Figure 3 (A)).
▪VMT:Vehicle-Miles Travelled (597 monthly
observations –Figure 3 (B))
▪PSI20:Portuguese Stock Index 20 (4534 daily
observations –Figure 3 (C))
▪SPY:Standard & Poor's 500 Exchange-Traded Fund
(6715 daily observations –Figure 3 (D))
The time-series represent a time period up of
September of 2019 (CPIAUSCL/VMT up to the 1st and
PSI20/SPY up to 27th). These are distinguished by
presenting different behaviors in terms of trend
and/or seasonality. However, all time-series are non-
stationary (ADF and KPSS Test), not i.i.d (BDS Test)
and a non-normal distribution (Jarque-Bera Test).
Results
▪In time series with strong perturbations, the
advantages of DNN models are evident.
▪In time series with a clear trend and/or
seasonality, the ARMA and ETS models are
adequate (and have low computational cost).
▪The additional computational power required in
more complex models is not always justified (e.g.
time-series CPIAUCSL and VMT).
Forecasting models for time-series:
a comparative study between classical
methodologies and Deep Learning
D.R. Lopes1; F.R. Ramos2; D.A. Mendes2; A.R. Costa3
1UNL and CEDOC-UNL, 2ISCTE-IUL and BRU-IUL, 3ISCTE-IUL and CMAF-CIO
Data
Figure 3. Time-series
Filipe Ramos
ISCTE-IUL and BRU-IUL
frjrs@iscte-iul.pt
RNNMLP
LSTM
250
200
150
50
300
Closing Price ($)
Business Days/Year
100
1992 1996 2000 2004 2008 20202012 2016
(A) CPIAUCSL (B) VMT
(C) PSI 20 (D) SPY
Past and future observations
290000
280000
270000
260000
230000
250000
240000
Vehicle Miles Travelled (millions)
-13 -5-22 5 8 12
-12 -11 -10 -9-8-7 -6 -4 -3 -1 0 1 3 4 6 7 9 10 11
SARIMA(1,1,1)n(1,1,1,12)
SARIMA(1,1,1)n(1,1,2,12)
SARIMA(1,1,4)n(1,1,2,12)
SARIMA(1,1,4)n(2,1,2,12)
SARIMA(4,1,1)n(1,1,2,12)
SARIMA(4,1,1)n(1,1,3,12)
-14
Real data
Past and future business days
-14 -12 0 6 14 20
-10 -8-6-4 -2 2 4 8 10 12 16 18
Closing Price ($)
300.0
297.5
295.0
292.5
285.0
290.0
287.5
ARIMA(1,1,1)
ARIMA(1,1,2)
ARIMA(4,1,2)
ARIMA(4,1,3)
ARIMA(4,1,4)
ARIMA(2,1,1)
Real data
4700
-14
5050
5000
4950
4900
4750
4850
4800
Past and future business days
-12 0 6 14 20
-10 -8-6-4 -2 2 4 8 10 12 16 18
Closing Price (€)
ARIMA(1,1,1)
ARIMA(2,1,1)
ARIMA(1,1,2)
ARIMA(3,1,3)
ARIMA(4,1,2)
ARIMA(4,1,4)
Real data
Past and future observations
258
256
254
252
246
244
250
248
Consumer Price Index
-14 -13 -5-22 5 8 12
-12 -11 -10 -9-8-7 -6 -4 -3 -1 0 1 3 4 6 7 9 10 11
ARIMA(1,1,2)
ARIMA(1,1,3)
ARIMA(3,1,1)
ARIMA(3,1,4)
ARIMA(4,1,3)
ARIMA(4,1,4)
Real data
(B)
(A)
(D)
(C)
-14 Past and future business days
-12 0 6 14 20
-10 -8-6-4 -2 2 4 8 10 12 16 18
4700
5050
5000
4950
4900
4750
4850
4800
Consumer Price Index
Past and future observations
256
254
252
246
244
250
248
-14 -13 -5-22 5 8 12
-12 -11 -10 -9-8-7 -6 -4 -3 -1 0 1 3 4 6 7 9 10 11
Real data
ETS (A, N)
ETS (Ad, N)
ETS (A, M)
290000
280000
270000
260000
230000
250000
240000
-5-22 5 8 12
-11 -10 -9-8-7 -6 -4 -3 -1 0 1 3 4 6 7 9 10 11
Vehicle Miles Travelled (millions)
Past and future observations
Real data
ETS (A, A)
ETS (Ad, A)
ETS (A, M)
ETS (Ad, M)
Closing Price (€)
ETS (N, N)
ETS (A, N)
ETS (Ad, N)
Real data
Past and future business days
-14 -12 0 6 14 20
-10 -8-6-4 -2 2 4 8 10 12 16 18
Closing Price ($)
300.0
297.5
295.0
292.5
285.0
290.0
287.5
Real data
ETS (A, N)
ETS (Ad, N)
(B)
(A)
(D)
(C)
Consumer Price Index
255
250
240
235
245
Past and future observations 0
Real data
In-Sample Forecast
Out-of-Sample Forecast
(A)
300000
280000
260000
220000
240000
Vehicle Miles Travelled (millions)
Past and future observations 0
(B)
Real data
In-Sample Forecast
Out-of-Sample Forecast
4700
5400
5300
5200
5100
4800
5000
4900
Closing Price (€)
Past and future observations 0
(C)
Real data
In-Sample Forecast
Out-of-Sample Forecast
305
300
295
290
275
285
280
Closing Price ($)
Past and future observations 0
(D)
Real data
In-Sample Forecast
Out-of-Sample Forecast
300000
280000
260000
220000
240000
Vehicle Miles Travelled (millions)
Past and future observations 0
Real data
In-Sample Forecast
Out-of-Sample Forecast
4700
5400
5300
5200
5100
4800
5000
4900
Closing Price (€)
Past and future observations 0
Real data
In-Sample Forecast
Out-of-Sample Forecast
305
300
295
290
275
285
280
Closing Price ($)
Past and future observations 0
Real data
In-Sample Forecast
Out-of-Sample Forecast
Consumer Price Index
255
250
240
235
245
Past and future observations 0
Real data
In-Sample Forecast
Out-of-Sample Forecast
(A)
(B)
(C)
(D)
ARMA Models ETS Models
MLP Models LSTM Models
Figure 2. Methodology for computational implementation (DNN)
Figure 4. Models fitting and forecasting of time-series:
(A) CPIAUCSL; (B) VMT; (C) PSI 20; (D) SPY
(Time-
series)
CPIAUCSL
VMT
SARIMA(4,1,1) × (1,1,3)
(: / =12)
PSI 20
SPY
(Model)
ARIMA(3,1,4)
ARIMA(4,1,4)
ARIMA(4,1,2)
Month
Quarter
Year
Month
Quarter
Year
Day
Week
Month
Day
Week
Month
MAPE
0.12%
0.15%
0.37%
1.3%
0.9%
0.98%
1.45%
2.16%
3.29%
1.18%
0.91%
2.63%
(Time-
serie)
CPIAUCSL
VMT
PSI 20
SPY
(Model)
,
,
,
,
= 1 , = 0.12
= 0.37, = 0, = 1
= 1, = 0.29,= 0.24
= 0.95 , = 0
= 0.33, =12
Month
Quarter
Year
Month
Quarter
Year
Day
Week
Month
Day
Week
Month
MAPE
0.14%
0.14%
0.19%
0.74%
0.75%
0.92%
1.49%
2.12%
3.15%
1.28%
1.57%
2.83%
Table 1. Predictions errors of the ARMA models
Table 2. Predictions errors of the ETS models
* Minimum values - Maximum values (trimmed by 5%) obtained in a total of 60 runs
(Time-
Serie)
CPIAUCSL
VMT
PSI 20
SPY
Model
Month
Quarter
Year
Month
Quarter
Year
Day
Week
Month
Day
Week
Month
MAPE(*)
0.13%
–
0.26%
0.13%
–
0.27%
0.18%
–
0.29%
0.21%
–
0.38%
0.27%
–
0.51%
0.48%
–
0.82%
0.83%
–
1.39%
1.31%
–
1.84%
1.57%
–
2.01%
0.55%
–
0.94%
0.60%
–
1.03%
1.02%
–
1.82%
Table 3. Predictions errors of the MLP models
Table 4. Predictions errors of LSTM models
* Minimum values - Maximum values (trimmed by 5%) obtained in a total of 20 runs
(Time-
Serie)
CPIAUCSL
VMT
PSI 20
SPY
Model
Month
Quarter
Year
Month
Quarter
Year
Day
Week
Month
Day
Week
Month
MAPE(*)
0.11%
–
0.21%
0.12%
–
0.19%
0.16%
–
0.24%
0.04%
–
0.10%
0.16%
–
0.25%
0.35%
–
0.71%
0.87%
–
1.41%
1.19%
–
1.63%
0.97%
–
1.52%
0.48%
–
0.83%
0.50%
–
0.96%
0.79%
–
1.12%