ArticlePDF Available

Feedforward Neural Nets as Models for Time Series Forecasting

DOI:10.1287/ijoc.5.4.374
Authors:
Zaiyong Tang at Salem State University
Zaiyong Tang
Paul A. Fishwick at University of Texas at Dallas
Paul A. Fishwick
Download full-text PDF
Read full-text
Download citation

Abstract and Figures

We have studied neural networks as models for time series forecasting, and our research compares the Box-Jenkins method against the neural network method for long and short term memory series. Our work was inspired by previously published works that yielded inconsistent results about comparative performance. We have since experimented with 16 time series of differing complexity using neural networks. The performance of the neural networks is compared with that of the Box-Jenkins method. Our experiments indicate that for time series with long memory, both methods produced comparable results. However, for series with short memory, neural networks outperformed the Box-Jenkins model. Because neural networks can be easily built for multiple-step-ahead forecasting, they may present a better long term forecast model than the Box-Jenkins method. We discussed the representation ability, the model building process and the applicability of the neural net approach. Neural networks appear to provide a promising alternative for time series forecasting. INFORMS Journal on Computing, ISSN 1091-9856, was published as ORSA Journal on Computing from 1989 to 1995 under ISSN 0899-1499.
Figures - uploaded by Zaiyong Tang
Author content
All figure content in this area was uploaded by Zaiyong Tang
Content may be subject to copyright.
Content uploaded by Zaiyong Tang
Author content
All content in this area was uploaded by Zaiyong Tang on Jul 09, 2014
Content may be subject to copyright.
x1 x2
O1 O2
Output
Hidden
Input
O3
x3
W
W
ij
jk
0
500
1000
1500
05 10 15 20 25 30
ser211
80
100
120
140
160
05 10 15 20 25 30 35 40
ser292
40
60
80
100
120
010 20 30 40 50 60
ser301
2000
3000
4000
5000
6000
7000
010 20 30 40 50 60
ser310
0
100
200
300
400
500
010 20 30 40 50 60
ser355
0
1000
2000
3000
010 20 30 40 50 60 70 80
ser409
0
0.5
1
1.5
2x104
020 40 60 80 100 120
ser454
1000
1500
2000
2500
3000
010 20 30 40 50 60 70 80
ser526
1
2
3
4
5
6x104
010 20 30 40 50 60 70
ser571
40
60
80
100
120
140
020 40 60 80 100 120
ser787
150
200
250
300
350
02 4 6 8 10 12 14 16 18 20
ser193
2
4
6
8
10
12
010 20 30 40 50 60
ser535
2
4
6
8
10
12
010 20 30 40 50 60 70 80
ser562
0
5
10
15
20
25
010 20 30 40 50 60 70
ser715
0
200
400
600
800
020 40 60 80 100 120 140 160
Airline data
0
200
400
600
800
1000
010 20 30 40 50 60 70 80
Sales data
Xt-1
Xt-2
Xt-3Xt-4
Xt-5
Xt-6Xt-7
Xt Xt+1
Xt-8
Xt-1Xt-2Xt-3 Xt
Xt+1
... All these methods aim to provide precise short-term predictions while capturing the long-term trajectories of the systems. Successful deep learning-based forecasting methods (such as long shortterm memory networks (LSTM) [17], convolutional neural networks (CNN) [18], feed-forward neural networks (FFNN) [19], reservoir computing (RC) [20]) can capture the evolution of time series in chaotic systems but usually do not use physical laws in their architecture. ...
... For comparative analysis, we considered the following competitive forecasting models: Long-Short Term Memory (LSTM) [17]; Feed-Forward Neural Networks [19]; Convolutional Neural Networks (1D CNN) [18], and Reservoir Computing (ESN) [20]. In the experiments, we fit an LSTM with 3 hidden layers and 50 neurons, an RC with a reservoir size of 500, an FFNN with one hidden layer and 50 neurons, a 1D CNN with 64 filters and a kernel size of 2, and a timestep of 3 followed by a MaxPooling and Flatten layers to a fully connected layer with 50 neurons. ...
View
... In this paper, two models commonly used by researchers in stock forecasting, Artificial Neural Network and the autoregressive integrated moving average, are employed to forecast Nvidia's stock price in the next few years from an artificial intelligence perspectives and a statistical perspective respectively, and the results of the two models are also compared. Artificial Neural Network, proposed by Tang and Fishwick [9]; Wang and Le in 1993, was the first application of ANN to predict stock prices on the Tokyo Stock Exchange Kimoto et al. [10]. In the past 20 years, ANN stock price prediction has been more maturely applied in stock exchange market index. ...
A Comparison of Stock Price Prediction with ANN and ARIMA
Article
Full-text available
  • Mar 2023
Nvidia is one of the most competitive companies in the semiconductor industry, whose stock has risen a remarkable 5,427% in the past decade. Forecasting its stock price has always been one of the most important topics for investors, as its stock price fluctuates dramatically with the release of new products and the rising price of cryptocurrencies worldwide. This paper compares the degree of accuracy of ANN and ARIMA, which are considered to be the most commonly used and accurate models in stock price forecasting for the past 20 years, in predicting Nvidia's stock price. The time period chosen for the prediction is from June 2020 to June 2021, when Nvidia's stock price rises sharply, and the data is obtained from Kaggle and Yahoo Finance. According to the analysis, the ANN model is clearly capable of predicting the sharp fluctuations in Nvidia's stock price between 2020 and 2021, which also indicates the potential of artificial neural network-based models for stock price predictions. These results shed light on guiding further exploration of stock price prediction.
View
... The optimal values of the hyper-parameters of the examined deep learning methods were determined by searching grid on a training set. For the number of input node, it is based on the number of AR terms used in the Box-Jenkins model (p = 1) [29]. Deep learning models are trained on 100 epochs with 4 sample in batch sizes based on the GPU memory. ...
View
... The network will be trained using the Levenberg-Marquardt backpropagation algorithm, with scaled conjugate gradient backpropagation utilized if memory is insufficient. The lag(s) can be calculated using the autoregressive (AR) order [17][18][19]. ...
Hybrid Holts-Winter’s model and artificial neural network for short term load data
Conference Paper
  • Feb 2023
Since seasonal data incorporates a seasonal cycle, forecasting seasonal data differs from forecasting ordinary time series data. Because of its utility in forecasting a linear relationship with other factors, Holt-Winter's model has been frequently employed in load forecasting. However, Holt-has Winter's the drawback of having difficulty modeling a nonlinear connection between the variables and influencing factors. On the other hand, the neural network model is an excellent model for representing nonlinear data. As a result, a combination of Holt-Winter's and NN models is proposed in this work to anticipate future load demand. This hybrid model is then compared to the Holt-Winter and NN models to assess how well it performs. As a performance metric, the RMSE and MAE are utilized, and a fractional residual plot is presented to visualize the error graphically. This model, based on the findings, provides a better prognosis than the other two models.
View
... Artificial neural networks (ANNs) are frequently considered the most accurate forecasting tool when compared with traditional forecasting techniques, especially for nonstationary, nonlinear, discontinuous, and complex problems [13]. Such algorithms can easily handle nonlinear, noisy, or incomplete data due to their learning characteristics, making them a natural tool for electrical markets, considering the characteristics of electricity prices [14]. For these reasons, AI algorithms, more precisely ANNs, were used in this study. ...
Influence of Increasing Renewable Power Penetration on the Long-Term Iberian Electricity Market Prices
Article
Full-text available
  • Jan 2023
In recent years, there has been a significant increase in investment in renewable energy sources, leading to the decarbonization of the electricity sector. Accordingly, a key concern is the influence of this process on future electricity market prices, which are expected to decrease with the increasing generation of renewable power. This is important for both current and future investors, as it can affect profitability. To address these concerns, a long-term analysis is proposed here to examine the influence of the future electricity mix on Iberian electricity prices in 2030. In this study, we employed artificial intelligence forecasting models that incorporated the main electricity price-driven components of MIBEL, providing accurate predictions for the real operation of the market. These can be extrapolated into the future to predict electricity prices in a scenario with high renewable power penetration. The results, obtained considering a framework featuring an increase in the penetration of renewables into MIBEL of up to 80% in 2030, showed that electricity prices are expected to decrease by around 50% in 2030 when compared to 2019, and there will be a new pattern of electricity prices throughout the year due to the uneven distribution of renewable electricity. The study’s findings are relevant for ongoing research on the unique challenges of energy markets with high levels of renewable generation.
View
... there has been a growing demand to test and implement intelligent techniques for monitoring programs of current conditions, and neural networks have emerged as one of the most promising techniques in this field. Neural networks have the potential to represent any complex and non-linear mapping that may govern any changes in a time series [13]. ...
Improving Distribution Systems Maintenance Management, by Predicting Reliability Indices Using Fully Connected Neural Networks
Conference Paper
Full-text available
  • Dec 2022
Due to the increasing amount of industrial data worldwide, deep learning solutions have become extensively popular for preventive maintenance programs. Preventive maintenance reduces the costs of equipment failures, thus crafting detailed preventive maintenance programs are very effective. Neural networks are intelligent computing techniques inspired by biological neurons. Neural networks are one of the most common and practical machine learning algorithms that are utilized in many industrial applications. In this research, a fully connected neural network is developed to predict the systems reliability indices, i.e. the number of failures occurring in feeders and the amount of energy not served (ENS) in electricity distribution feeders. As a result, a data-driven improvement is provided for the preventive maintenance program.
View
View
Proceedings JANO14
Conference Paper
Full-text available
  • Feb 2023
View
Development of a mathematical model to forecast black carbon concentration using ARIMA and soft computing
Article
  • Mar 2023
Fresh air is imminent for life and to thrive on this planet. However, this vital component of life is ill-effected by fast-paced industrialization, urbanization, automobiles, factories, and coal-based thermal power generation as over the years these have jeopardized the air quality index. Due to hazardous impact of black carbon on the environment as well as human health, researchers have turned their attention towards its study. The artificial neural network and econometric ARIMA model are used to predict black carbon emissions from three major coal mines located at Bokaro, Jharia, and Raniganj in India. A comparative analysis of these two techniques over three different forecasting horizons is carried out for investigating short-term or long-term efficiencies of both the models. The three coal mines have large emissions of particulate matter (\(P{M}_{2.5}\)) that contribute significantly to pollution levels. A multilayer perceptron feedforward artificial neural network with Bayesian regularization-backpropagation neural network (BR-BPNN) algorithm is employed. The efficiency of neural network models is evaluated by mean absolute deviation (MAD), root mean square error (RMSE), and coefficient of determination (R2) values. For all the forecasting horizons and in all the accuracy measures, the BR-BPNN outperformed the traditional econometric ARIMA model in predicting the black carbon concentration values with a considerable reduction in errors ranging from 60 to 70% for all the sites. In addition, the performance of an ARIMA model is found to be dependent on the length of the forecasting horizon. However, no such evidence is found for ANN model in forecasting black carbon concentration data.
View
Neural Network Modeling and What-If Scenarios: Applications for Market Development Forecasting
Chapter
  • Feb 2023
In this chapter, we demonstrate the use of neural networks for forecasting automotive market sales. Using data from an entire country, with a large set of micro- and macroeconomic variables and other factors, we show how to supplement “black box” neural network calculations with a solid theoretical foundation from social sciences. Doing so allows to not only create exceptionally accurate forecasts, but understand the “black box” weights on different variables that are used in generating predictions. We also demonstrate how individual variables fit into the overall market dynamics, and how political changes play a role in economic outcomes.
View
View
Threshold Autoregression, Limit Cycles and Cyclical Data
Article
  • Jul 1980
The notion of a limit cycle, which can only exist in a non‐linear system, plays the key role in the modelling of cyclical data. We have shown that the class of threshold autoregressive models is general enough to capture this notion, a definition of which in discrete time is proposed. The threshold value has an interesting interpretation. Simulation results are presented which demonstrate that this new class of models exhibits some well‐known features of non‐linear vibrations. Detailed analyses of several real data sets are discussed.
View
View
View
View
View
View
Some Asymptotic Results for Learning in Single Hidden-Layer Feedforward Network Models
Article
  • Dec 1989
We investigate the properties of a recursive estimation procedure (the method of “back-propagation”) for a class of nonlinear regression models (single hidden-layer feedforward network models) recently developed by cognitive scientists. The results follow from more general results for a class of recursive m estimators, obtained using theorems of Ljung (1977) and Walk (1977) for the method of stochastic approximation. Conditions are given ensuring that the back-propagation estimator converges almost surely to a parameter value that locally minimizes expected squared error loss (provided the estimator does not diverge) and that the back-propagation estimator is asymptotically normal when centered at this minimizer. This estimator is shown to be statistically inefficient, and a two-step procedure that has efficiency equivalent to that of nonlinear least squares is proposed. Practical issues are illustrated by a numerical example involving approximation of the Hénon map.
View
View
View