Content uploaded by Salih Tutun
Author content
All content in this area was uploaded by Salih Tutun on Oct 17, 2016
Content may be subject to copyright.
Proceedings of the 5th Annual World Conference
of the Society for Industrial and Systems Engineering,
San Francisco, CA, USA
October 13-14, 2016
ISBN: 97819384960-8-0 193
The Optimized Elastic Net Regression Model for Electricity Consumption
Forecasting
S. Tutun, M. Bataineh, M. Aladeemy, and M. Khasawneh
Department of Systems Science and Industrial Engineering
State University of New York at Binghamton
Binghamton, New York 13902, USA
Corresponding author's Email: stutun1@binghamton.edu
Abstract: Electricity is a significant power resource that is being hard to store physically. Energy policy aims to maintain
continuous power supply to customers without shortage or waste of energy. To address this issue, this paper aims to provide a
framework to forecast the future net electricity consumption of Turkey based on four factors (independent variables), namely,
imports, exports, transmitted energy and gross generation, which have an impact on the net electricity consumption. The
framework involves forecasting the independent variables using a nonlinear autoregressive-based neural network
(NARANN) model. Afterward, an elastic net regression model is proposed to forecast the net electricity consumption of
Turkey. Simulated annealing (SA) and evolutionary strategy (ES) were used to optimize the coefficients of the elastic net
regression model. The results show that the proposed approach provides high accuracy for net electricity consumption
forecasting.
Keywords: Energy Management, Time Series Forecasting, Regularization
1. Introduction
Electricity planning is a vital part of any energy policy in order to provide a continuous and stable power supply to
customers in a cost-effective manner. In some countries, such as Turkey, the energy policy considers purchasing energy
supply from other countries due to limited energy resources. Moreover, electricity is a difficult energy source for investment,
cannot be stored and not possible to measure its physical flow of consumption, which has a volatile structure. Therefore,
future electricity consumption needs to be accurately forecasted in order to set the power generation and/or energy purchase
plans accordingly. Poor power generation and/or energy purchase plans can lead to either power shortages (i.e., discontinue
power supply to customers) or excessive capacity (i.e., extra unnecessary costs) which are undesirable.
Several studies addressed the problem of electricity consumption using linear and nonlinear models. Tunç et al.,
(2006) used regression analysis to forecast the electricity consumption of Turkey for 2010 until 2020. Erdogdu (2007)
developed an autoregressive integrated moving average (ARIMA) model to forecast the electricity consumption of Turkey
for the years 2005-2014. Similarly, ARIMA and seasonal ARIMA (SARIMA) models were used to forecast the electric
energy demand of Turkey for the years 2005-2020 (Ediger & Akar, 2007). Akay and Atak (2007) proposed the grey
prediction with rolling mechanism (GPRM) method for electricity demand forecasting.
Nonlinear models were also used to forecast the electricity consumption. Azadeh et al. (2009) used artificial neural
network (ANN), genetic algorithm (GA), adaptive neural fuzzy inference system (ANFIS), Monte Carlo simulation (MCS),
particle swarm optimization (PSO), and artificial immune system (AIS). A simulation model for creating random variables
was developed by Azadeh et al. (2014). A seasonal and monthly electricity consumption were forecasted using ANN,
principle component analysis (PCA) and data envelopment analysis (DEA) (Kheirkhah et al., 2013). Ediger and Tatlıdil,
(2002) used a nonlinear technique that involves the analysis of cyclic patterns of annual additional amounts relevant to
energy consumption on time series. Also, Hamzaçebi (2007) proposed ANN as the sectoral electric data of Turkey have a
nonlinear structure. Hamzaçebi and Kutay (2004) used ANN and Box-Jenkins models. Kavaklioglu et al. (2009) built a
multilayer backpropagation ANN (MLP-ANN) model to forecast the electricity consumption as a function of economic
factors. ANN and support vector regression (SVR) models were used for electricity consumption forecasting (Oğcu et al.,
2012). Support vector regression (SVR) was also used to forecast the electricity consumption of Turkey until 2026 by
analyzing data that spans from 1975 to 2006 and using the following independent variables: population, gross national
product (GNP), imports and exports (Kavaklioglu, 2011). Kavaklioglu (2014) developed a multivariate regression model for
net electricity consumption forecasting in which singular value decomposition (SVD) was used to downsize the problem.
Proceedings of the 5th Annual World Conference
of the Society for Industrial and Systems Engineering,
San Francisco, CA, USA
October 13-14, 2016
194
Furthermore, some studies employed metaheuristics to optimize the parameters of energy models. Ozturk (2005)
developed two different nonlinear models with quadratic and exponential behaviors using a genetic algorithm (GA) to
forecast the industrial energy demand of Turkey. Toksarı (2007) used ant colony optimization (ACO) and Ünler (2008)
particle swarm optimization (PSO) used to forecast the electricity consumption of Turkey. Many of these studies assumed
that the electricity consumption increases with independent variables at a constant growth rate. However, this assumption is
not practically feasible (Toksarı, 2009). Tutun et al. (2015) used least absolute shrinkage and selection operator (LASSO) and
ridge regression models to forecast the net electricity consumption of Turkey. The nonlinear autoregressive-based artificial
neural network (NARANN) model was used to forecast the independent variables and both LASSO and ridge regression
models (optimized by simulated annealing (SA) and evolutionary strategy (ES)) were used to forecast the net electricity
consumption.
The objective of this paper is to provide a forecasting approach for net electricity consumption of Turkey.
Specifically, the NARANN model is used to forecast the future values of the independent variables and then an elastic net
regression model (instead of using separate LASSO and ridge regression models as in Tutun et al. (2015)) optimized by SA
and ES is employed to select the best independent variables in order to be used to forecast the net electricity consumption of
Turkey. This paper is organized as follows. In Section 2, methods used in the proposed approach are briefly presented in
addition to the proposed forecasting approach. Section 3 presents the experimental results using a case study and section 4
concludes this paper and presents the future work.
2. Methodology
In this paper, an approach to forecast the net electricity consumption of Turkey is proposed. First, a nonlinear
autoregressive-based artificial neural network (NARANN) model was used to forecast future values of inputs (independent
variables) which are, imports, exports, transmitted energy and gross generation. Then, an optimized elastic net (OPEN)
regression model was used to forecast the net electricity consumption of Turkey. The forecasted net electricity consumption
was compared with the out-of-sample data (testing set). The methods incorporated in the proposed approach will be briefly
presented in the following sub-sections. The mathematical notation used in this paper is as follows:
Sl(j): Input function of node j in layer l
wl(i,j): Weight between node i and j in layers l and l+1, respectively
bl: Bias in layer l
Yl(j)=al(j): Output of node j in layer l
εj: Error for node j
Cl(j): Cost for weights for node j in layer l
Old wl(i,j): Old weight between nodes i and j in layers l and l+1, respectively
New wl(i,j): New weight between nodes i and j in layers l and l+1, respectively
Y(t): Actual net electricity consumption at time t
d: Number of delays
xi: Independent variable i, (i=1,2,3,...,n)
β0 and βi: Intercept and coefficient of independent variable xi, respectively
λ: Non-negative regularization parameter
RI: Forecasted imports
RE: Forecasted exports
RT: Forecasted transmitted energy
RG: Forecasted gross production
2.1 Nonlinear Autoregressive Artificial Neural Network (NARANN)
This method uses the artificial neural network (ANN) to explore the nonlinear relationship between the time lags of
inputs (independent variables). It finds patterns by using each output as input for next time lag. The process of tuning ANN
weights is known as fitting (learning) which is conducted on the training set. These weights are randomly initiated and
updated in an iterative manner based on performance measures such as minimum squared error (MSE). The input function of
node j in layer l from nodes, i = 1,2,3,...,I and the bias in layer l-1 can be expressed by:
Proceedings of the 5th Annual World Conference
of the Society for Industrial and Systems Engineering,
San Francisco, CA, USA
October 13-14, 2016
195
Sl(j)= ∑
i=1
I wl1(i,j)*al1(i)+b11 (1)
Activation functions map the input of the node to the output and there are several functions can be used as activation
functions. The sigmoid function is commonly used as an activation function which can be expressed for node j in the layer l
as follows:
Yl(j)=al(j)= 1
1+eSl(j) (2)
The final estimated value is the output of the output layer, which is compared with the target value in order to
calculate the difference (i.e., error). If the difference between the estimated value and actual value is higher than a very small
value, the weights will be rearranged in order to reduce this difference. The effect of each output node on the error can be
determined by calculating the cost of weights for each node as follows:
Cl(j)=(j)*al(j)*(1al(j)) (3)
where l is the output layer. In multilayer perceptron NN (MLP-ANN), these procedures can be achieved by taking
the values of Cl(j) equal to zero at the beginning. At the same time, learning factor (ρ), which is set by the user, is used to
calculate the new weights as follows:
New wl(j,i) = old wl(j,i) – ρ*Cl(j) (4)
The procedure is repeated by deducting the error signals of every layer, if the network has multiple hidden layers,
from the corrected procedures of the previous layer until it reaches the input region in a reverse manner by starting with the
output layer. Finally, the procedure is repeated until the performance of the network is satisfactory. This method is called
backpropagation algorithm of the error. For more details about this algorithm, the reader is referred to Donaho and Palmer
(1989). The neural network is proposed to model the nonlinear part of the data that cannot be captured by the autoregressive-
based models by using the observations at previous time lags as inputs to forecast the observation at the next time lag
(output), as follows:
Y(t)=f(y(t1),y(t2),...,y(td)),t d+1 (5)
2.2 The Optimized Elastic Net (OPEN) Regression Model
In this model, two metaheuristic algorithms, namely; simulated annealing (SA) and evolutionary strategy (ES)) were
used to optimize the elastic net regularization. The elastic net combined both LASSO regression (ℓ1-penalty) and ridge
regression (ℓ2-penalty) to prevents over-training (over-fitting) by selecting the best independent variables.
SA is a search technique that mimics the annealing process in materials in which the metal is heated up and then the
temperature is lowered slowly into the crystalline state with a minimum energy level to increase the crystal size and decrease
its defects. The efficiency of the algorithm for optimization depends on the control of temperature and cooling schedule.
Moreover, in order to move to search areas, the algorithm uses a random walk to avoid being trapped into local minima. The
reader is referred to Kirkpatrick (1984) for more details about SA algorithm.
ES is a population-based algorithm in which the new solutions (i.e., offspring) are compared with old solutions (i.e.,
parents) and the best solutions that optimize the objective function are used. In this paper, ES is used to find a good initial
solution for SA algorithm. The LASSO penalty included in the elastic net regularization prevents over-fitting by selecting
best independent variables.
2.2.1 Formulation of the New Model
In the model formulation, the forecasting model of each independent variable xi was determined using the
NARANN model. Then, elastic net regularization was used to select the best independent variables by optimizing the
Proceedings of the 5th Annual World Conference
of the Society for Industrial and Systems Engineering,
San Francisco, CA, USA
October 13-14, 2016
196
coefficients of the independent variables (decision variables). That is, the OPEN regression model aims to minimize the
objective function, shown in Equation (6), by minimizing the errors (i.e., differences between actual values and forecasted
values).
min0,1,2,3,...,I( 1
2N ∑
n=1
N (Y(n)F(n))2+(P())) as
jj
d
j
P
β
αβ
α
β
α
+
−
=
∑
=
2
1
)
2
1
(
)(
(6)
where Y(n) and F(n) are the actual and estimated values of net electricity consumption, respectively, N is the number
of observations and is a nonnegative regularization parameter. Pα(β) interpolates between LASSO and ridge regression
(LASSO = 1 and ridge = 0).
2.3 Performance Indicator
The accuracy measure that was used to assess the model is the mean absolute percentage of error (MAPE), which
uses the absolute value of the difference between actual value and estimated value, as follows:
MAPE=
( ∑
i=1
n | YiFi
Fi|)
n (7)
3. Experimental Results
The monthly data includes each independent variable (input) and spans over 35 years were obtained from the
Turkish electric transmission company (TEIAS). The independent variables, namely; the transmitted energy, gross
generation, imports, and exports, were used to forecast the net electricity consumption (dependent variable) of Turkey. The
accuracy measure, MAPE, presented in subsection 2.3 was employed to assess the model. As mentioned earlier in this
context, previous studies used constant growing values for future inputs which are not practically feasible (Toksarı, 2009).
Therefore, the NARANN model was used to forecast each input (independent variable). The linear relationship between the
forecasted and actual values of all inputs are represented in Equation (8), which all of them have R2 score higher than 0.9, as
shown in Table 1.
RI=0.88*target+6.8, RE=0.78*target+11, RG=0.99*target+52, RT=0.98*target+110 (8)
In the NARANN model, there are a number of hidden layers, weights and input time lags, which were determined
using Levenberg-Marquardt training algorithm in MATLAB software. The number of hidden layers, number of input time
lags and the percentages of training, validation and testing sets in addition to have R2 scores are summarized in Table 1. The
scatter plot of the elastic net regression model is shown in Figure 1. The forecasted values and actual values are shown in
Figure 2. The elastic net regression model is reliable for net electricity consumption forecasting as the score achieved by
accuracy measure used in this study, MAPE, is 1.96%.
Table 1. Parameters and R2 (on testing data) values of the NARANN models for inputs (Note: datasets were split
into training, validation and testing).
Parameters
Gross Production (x1)
Energy Imports (x2)
Transmitted Energy (x3)
Energy Exports (x4)
Dataset
%70-15-15
%70-15-15
%70-15-15
%70-15-15
Hidden Layers
10
5
10
10
Lags
R
2
7
0.997
3
0.958
4
0.993
4
0.939
Proceedings of the 5th Annual World Conference
of the Society for Industrial and Systems Engineering,
San Francisco, CA, USA
October 13-14, 2016
197
Figure 1. Scatter plot of the Optimized Elastic Net Regression Model.
Figure 2. Comparative Results for Actual and Forecasted Net Electricity Consumption (with testing data).
4. Conclusions and Future Work
In this paper, a forecasting framework for net electricity consumption of Turkey is proposed. Four independent
variables (inputs), namely; imports, exports, transmitted energy and gross power generation were used to forecast the net
electricity consumption of Turkey. A nonlinear autoregressive-based neural network (NARANN) model was used to forecast
the future values of these independent variables. Afterward, an optimized elastic net (OPEN) regression model was employed
to forecast the net electricity consumption of Turkey. The elastic net was optimized using simulated annealing in which the
evolutionary strategy was used to find good initial values for the simulated annealing search. The results show that the
proposed approach provides high accuracy for net electricity consumption forecasting as the achieved MAPE score was
1.96%. For the future work, forecasted independent values will be improved to be used by the elastic net regression model to
make the projections. Also, other optimization methods will be used to solve the elastic net problem.
Proceedings of the 5th Annual World Conference
of the Society for Industrial and Systems Engineering,
San Francisco, CA, USA
October 13-14, 2016
198
5. Acknowledgment
The authors would like to thank the Turkish Electricity Transmission Company and the Turkish Ministry of Energy
and Natural Resources for their help in providing data.
6. References
Akay, D., & Atak, M. (2007). Grey prediction with rolling mechanism for electricity demand forecasting of
Turkey. Energy, 32(9), 1670-1675.
Azadeh, A., Saberi, M., Gitiforouz, A., & Saberi, Z. (2009). A hybrid simulation-adaptive network based fuzzy inference
system for improvement of electricity consumption estimation. Expert Systems with Applications,36(8), 11108-
11117.
Azadeh, A., Taghipour, M., Asadzadeh, S. M., & Abdollahi, M. (2014). Artificial immune simulation for improved
forecasting of electricity consumption with random variations. International Journal of Electrical Power & Energy
Systems, 55, 205-224.
Donahoe, J. W., & Palmer, D. C. (1989). The interpretation of complex human behavior: some reactions to parallel
distributed processing, edited by J.L. McClelland, De Rumelhart, and the PDP research group. Journal of the
Experimental Analysis of Behavior, 51(3), 399-416.
Ediger, V. Ş., & Akar, S. (2007). ARIMA forecasting of primary energy demand by fuel in Turkey. Energy Policy, 35(3),
1701-1708.
Ediger, V. Ş., & Tatlıdil, H. (2002). Forecasting the primary energy demand in Turkey and analysis of cyclic patterns. Energy
Conversion and Management, 43(4), 473-487.
Erdogdu, E. (2007). Electricity demand analysis using cointegration and ARIMA modelling: A case study of Turkey. Energy
policy, 35(2), 1129-1146.
Hamzaçebi, C. (2007). Forecasting of Turkey's net electricity energy consumption on sectoral bases. Energy Policy, 35(3),
2009-2016.
Hamzaçebi, C., & Kutay, F. (2004). Yapay sinir ağlari ile Türkiye elektrik enerjisi tüketiminin 2010 yilina kadar
tahmini. Gazi Üniversitesi Mühendislik-Mimarlık Fakültesi Dergisi, 19(3).
Kavaklioglu, K., Ceylan, H., Ozturk H., & Canyurt O. (2009). Modeling and prediction of Turkey’s electricity consumption
using artificial neural networks. Energy Conversion and Management, 50(11):2719–2727, 2009.
Kavaklioglu K. (2011). Modeling and prediction of Turkey’s electricity consumption using support vector regression.
Applied Energy, 88(1):368–375.
Kavaklioglu, K. (2014). Robust electricity consumption modeling of Turkey using singular value
decomposition. International Journal of Electrical Power & Energy Systems, 54, 268-276.
Kheirkhah, A., Azadeh, A., Saberi, M., Azaron, A., & Shakouri, H. (2013). Improved estimation of electricity demand
function by using of artificial neural network, principal component analysis and data envelopment
analysis. Computers & Industrial Engineering, 64(1), 425-441.
Kirkpatrick, S. (1984). Optimization by simulated annealing: Quantitative studies. Journal of statistical physics, 34(5-6),
975-986.
Oğcu, G., Demirel, O. F., & Zaim, S. (2012). Forecasting electricity consumption with neural networks and support vector
regression. Procedia-Social and Behavioral Sciences, 58, 1576-1585.
Toksarı, M. D. (2009). Estimating the net electricity energy generation and demand using the ant colony optimization
approach: case of Turkey. Energy Policy, 37(3), 1181-1187.
Tunç, M., Çamdali, Ü., & Parmaksizoğlu, C. (2006). Comparison of Turkey's electrical energy consumption and production
with some European countries and optimization of future electrical power supply investments in Turkey. Energy
Policy, 34(1), 50-59.
Tutun, S., Chou, C. A., & Canıyılmaz, E. (2015). A new forecasting framework for volatile behavior in net electricity
consumption: A case study in Turkey. Energy, 93, 2406-2422.
Ünler, A. (2008). Improvement of energy demand forecasts using swarm intelligence: The case of Turkey with projections to
2025. Energy Policy, 36(6), 1937-1944.
