Content uploaded by Salih Tutun

Author content

All content in this area was uploaded by Salih Tutun on Oct 17, 2016

Content may be subject to copyright.

Proceedings of the 5th Annual World Conference

of the Society for Industrial and Systems Engineering,

San Francisco, CA, USA

October 13-14, 2016

ISBN: 97819384960-8-0 193

The Optimized Elastic Net Regression Model for Electricity Consumption

Forecasting

S. Tutun, M. Bataineh, M. Aladeemy, and M. Khasawneh

Department of Systems Science and Industrial Engineering

State University of New York at Binghamton

Binghamton, New York 13902, USA

Corresponding author's Email: stutun1@binghamton.edu

Abstract: Electricity is a significant power resource that is being hard to store physically. Energy policy aims to maintain

continuous power supply to customers without shortage or waste of energy. To address this issue, this paper aims to provide a

framework to forecast the future net electricity consumption of Turkey based on four factors (independent variables), namely,

imports, exports, transmitted energy and gross generation, which have an impact on the net electricity consumption. The

framework involves forecasting the independent variables using a nonlinear autoregressive-based neural network

(NARANN) model. Afterward, an elastic net regression model is proposed to forecast the net electricity consumption of

Turkey. Simulated annealing (SA) and evolutionary strategy (ES) were used to optimize the coefficients of the elastic net

regression model. The results show that the proposed approach provides high accuracy for net electricity consumption

forecasting.

Keywords: Energy Management, Time Series Forecasting, Regularization

1. Introduction

Electricity planning is a vital part of any energy policy in order to provide a continuous and stable power supply to

customers in a cost-effective manner. In some countries, such as Turkey, the energy policy considers purchasing energy

supply from other countries due to limited energy resources. Moreover, electricity is a difficult energy source for investment,

cannot be stored and not possible to measure its physical flow of consumption, which has a volatile structure. Therefore,

future electricity consumption needs to be accurately forecasted in order to set the power generation and/or energy purchase

plans accordingly. Poor power generation and/or energy purchase plans can lead to either power shortages (i.e., discontinue

power supply to customers) or excessive capacity (i.e., extra unnecessary costs) which are undesirable.

Several studies addressed the problem of electricity consumption using linear and nonlinear models. Tunç et al.,

(2006) used regression analysis to forecast the electricity consumption of Turkey for 2010 until 2020. Erdogdu (2007)

developed an autoregressive integrated moving average (ARIMA) model to forecast the electricity consumption of Turkey

for the years 2005-2014. Similarly, ARIMA and seasonal ARIMA (SARIMA) models were used to forecast the electric

energy demand of Turkey for the years 2005-2020 (Ediger & Akar, 2007). Akay and Atak (2007) proposed the grey

prediction with rolling mechanism (GPRM) method for electricity demand forecasting.

Nonlinear models were also used to forecast the electricity consumption. Azadeh et al. (2009) used artificial neural

network (ANN), genetic algorithm (GA), adaptive neural fuzzy inference system (ANFIS), Monte Carlo simulation (MCS),

particle swarm optimization (PSO), and artificial immune system (AIS). A simulation model for creating random variables

was developed by Azadeh et al. (2014). A seasonal and monthly electricity consumption were forecasted using ANN,

principle component analysis (PCA) and data envelopment analysis (DEA) (Kheirkhah et al., 2013). Ediger and Tatlıdil,

(2002) used a nonlinear technique that involves the analysis of cyclic patterns of annual additional amounts relevant to

energy consumption on time series. Also, Hamzaçebi (2007) proposed ANN as the sectoral electric data of Turkey have a

nonlinear structure. Hamzaçebi and Kutay (2004) used ANN and Box-Jenkins models. Kavaklioglu et al. (2009) built a

multilayer backpropagation ANN (MLP-ANN) model to forecast the electricity consumption as a function of economic

factors. ANN and support vector regression (SVR) models were used for electricity consumption forecasting (Oğcu et al.,

2012). Support vector regression (SVR) was also used to forecast the electricity consumption of Turkey until 2026 by

analyzing data that spans from 1975 to 2006 and using the following independent variables: population, gross national

product (GNP), imports and exports (Kavaklioglu, 2011). Kavaklioglu (2014) developed a multivariate regression model for

net electricity consumption forecasting in which singular value decomposition (SVD) was used to downsize the problem.

Proceedings of the 5th Annual World Conference

of the Society for Industrial and Systems Engineering,

San Francisco, CA, USA

October 13-14, 2016

194

Furthermore, some studies employed metaheuristics to optimize the parameters of energy models. Ozturk (2005)

developed two different nonlinear models with quadratic and exponential behaviors using a genetic algorithm (GA) to

forecast the industrial energy demand of Turkey. Toksarı (2007) used ant colony optimization (ACO) and Ünler (2008)

particle swarm optimization (PSO) used to forecast the electricity consumption of Turkey. Many of these studies assumed

that the electricity consumption increases with independent variables at a constant growth rate. However, this assumption is

not practically feasible (Toksarı, 2009). Tutun et al. (2015) used least absolute shrinkage and selection operator (LASSO) and

ridge regression models to forecast the net electricity consumption of Turkey. The nonlinear autoregressive-based artificial

neural network (NARANN) model was used to forecast the independent variables and both LASSO and ridge regression

models (optimized by simulated annealing (SA) and evolutionary strategy (ES)) were used to forecast the net electricity

consumption.

The objective of this paper is to provide a forecasting approach for net electricity consumption of Turkey.

Specifically, the NARANN model is used to forecast the future values of the independent variables and then an elastic net

regression model (instead of using separate LASSO and ridge regression models as in Tutun et al. (2015)) optimized by SA

and ES is employed to select the best independent variables in order to be used to forecast the net electricity consumption of

Turkey. This paper is organized as follows. In Section 2, methods used in the proposed approach are briefly presented in

addition to the proposed forecasting approach. Section 3 presents the experimental results using a case study and section 4

concludes this paper and presents the future work.

2. Methodology

In this paper, an approach to forecast the net electricity consumption of Turkey is proposed. First, a nonlinear

autoregressive-based artificial neural network (NARANN) model was used to forecast future values of inputs (independent

variables) which are, imports, exports, transmitted energy and gross generation. Then, an optimized elastic net (OPEN)

regression model was used to forecast the net electricity consumption of Turkey. The forecasted net electricity consumption

was compared with the out-of-sample data (testing set). The methods incorporated in the proposed approach will be briefly

presented in the following sub-sections. The mathematical notation used in this paper is as follows:

Sl(j): Input function of node j in layer l

wl(i,j): Weight between node i and j in layers l and l+1, respectively

bl: Bias in layer l

Yl(j)=al(j): Output of node j in layer l

εj: Error for node j

Cl(j): Cost for weights for node j in layer l

Old wl(i,j): Old weight between nodes i and j in layers l and l+1, respectively

New wl(i,j): New weight between nodes i and j in layers l and l+1, respectively

Y(t): Actual net electricity consumption at time t

d: Number of delays

xi: Independent variable i, (i=1,2,3,...,n)

β0 and βi: Intercept and coefficient of independent variable xi, respectively

λ: Non-negative regularization parameter

RI: Forecasted imports

RE: Forecasted exports

RT: Forecasted transmitted energy

RG: Forecasted gross production

2.1 Nonlinear Autoregressive Artificial Neural Network (NARANN)

This method uses the artificial neural network (ANN) to explore the nonlinear relationship between the time lags of

inputs (independent variables). It finds patterns by using each output as input for next time lag. The process of tuning ANN

weights is known as fitting (learning) which is conducted on the training set. These weights are randomly initiated and

updated in an iterative manner based on performance measures such as minimum squared error (MSE). The input function of

node j in layer l from nodes, i = 1,2,3,...,I and the bias in layer l-1 can be expressed by:

Proceedings of the 5th Annual World Conference

of the Society for Industrial and Systems Engineering,

San Francisco, CA, USA

October 13-14, 2016

195

Sl(j)= ∑

i=1

I wl1(i,j)*al1(i)+b11 (1)

Activation functions map the input of the node to the output and there are several functions can be used as activation

functions. The sigmoid function is commonly used as an activation function which can be expressed for node j in the layer l

as follows:

Yl(j)=al(j)= 1

1+eSl(j) (2)

The final estimated value is the output of the output layer, which is compared with the target value in order to

calculate the difference (i.e., error). If the difference between the estimated value and actual value is higher than a very small

value, the weights will be rearranged in order to reduce this difference. The effect of each output node on the error can be

determined by calculating the cost of weights for each node as follows:

Cl(j)=(j)*al(j)*(1al(j)) (3)

where l is the output layer. In multilayer perceptron NN (MLP-ANN), these procedures can be achieved by taking

the values of Cl(j) equal to zero at the beginning. At the same time, learning factor (ρ), which is set by the user, is used to

calculate the new weights as follows:

New wl(j,i) = old wl(j,i) – ρ*Cl(j) (4)

The procedure is repeated by deducting the error signals of every layer, if the network has multiple hidden layers,

from the corrected procedures of the previous layer until it reaches the input region in a reverse manner by starting with the

output layer. Finally, the procedure is repeated until the performance of the network is satisfactory. This method is called

backpropagation algorithm of the error. For more details about this algorithm, the reader is referred to Donaho and Palmer

(1989). The neural network is proposed to model the nonlinear part of the data that cannot be captured by the autoregressive-

based models by using the observations at previous time lags as inputs to forecast the observation at the next time lag

(output), as follows:

Y(t)=f(y(t1),y(t2),...,y(td)),t d+1 (5)

2.2 The Optimized Elastic Net (OPEN) Regression Model

In this model, two metaheuristic algorithms, namely; simulated annealing (SA) and evolutionary strategy (ES)) were

used to optimize the elastic net regularization. The elastic net combined both LASSO regression (ℓ1-penalty) and ridge

regression (ℓ2-penalty) to prevents over-training (over-fitting) by selecting the best independent variables.

SA is a search technique that mimics the annealing process in materials in which the metal is heated up and then the

temperature is lowered slowly into the crystalline state with a minimum energy level to increase the crystal size and decrease

its defects. The efficiency of the algorithm for optimization depends on the control of temperature and cooling schedule.

Moreover, in order to move to search areas, the algorithm uses a random walk to avoid being trapped into local minima. The

reader is referred to Kirkpatrick (1984) for more details about SA algorithm.

ES is a population-based algorithm in which the new solutions (i.e., offspring) are compared with old solutions (i.e.,

parents) and the best solutions that optimize the objective function are used. In this paper, ES is used to find a good initial

solution for SA algorithm. The LASSO penalty included in the elastic net regularization prevents over-fitting by selecting

best independent variables.

2.2.1 Formulation of the New Model

In the model formulation, the forecasting model of each independent variable xi was determined using the

NARANN model. Then, elastic net regularization was used to select the best independent variables by optimizing the

Proceedings of the 5th Annual World Conference

of the Society for Industrial and Systems Engineering,

San Francisco, CA, USA

October 13-14, 2016

196

coefficients of the independent variables (decision variables). That is, the OPEN regression model aims to minimize the

objective function, shown in Equation (6), by minimizing the errors (i.e., differences between actual values and forecasted

values).

min0,1,2,3,...,I( 1

2N ∑

n=1

N (Y(n)F(n))2+(P())) as

jj

d

j

P

β

αβ

α

β

α

+

−

=

∑

=

2

1

)

2

1

(

)(

(6)

where Y(n) and F(n) are the actual and estimated values of net electricity consumption, respectively, N is the number

of observations and is a nonnegative regularization parameter. Pα(β) interpolates between LASSO and ridge regression

(LASSO = 1 and ridge = 0).

2.3 Performance Indicator

The accuracy measure that was used to assess the model is the mean absolute percentage of error (MAPE), which

uses the absolute value of the difference between actual value and estimated value, as follows:

MAPE=

( ∑

i=1

n | YiFi

Fi|)

n (7)

3. Experimental Results

The monthly data includes each independent variable (input) and spans over 35 years were obtained from the

Turkish electric transmission company (TEIAS). The independent variables, namely; the transmitted energy, gross

generation, imports, and exports, were used to forecast the net electricity consumption (dependent variable) of Turkey. The

accuracy measure, MAPE, presented in subsection 2.3 was employed to assess the model. As mentioned earlier in this

context, previous studies used constant growing values for future inputs which are not practically feasible (Toksarı, 2009).

Therefore, the NARANN model was used to forecast each input (independent variable). The linear relationship between the

forecasted and actual values of all inputs are represented in Equation (8), which all of them have R2 score higher than 0.9, as

shown in Table 1.

RI=0.88*target+6.8, RE=0.78*target+11, RG=0.99*target+52, RT=0.98*target+110 (8)

In the NARANN model, there are a number of hidden layers, weights and input time lags, which were determined

using Levenberg-Marquardt training algorithm in MATLAB software. The number of hidden layers, number of input time

lags and the percentages of training, validation and testing sets in addition to have R2 scores are summarized in Table 1. The

scatter plot of the elastic net regression model is shown in Figure 1. The forecasted values and actual values are shown in

Figure 2. The elastic net regression model is reliable for net electricity consumption forecasting as the score achieved by

accuracy measure used in this study, MAPE, is 1.96%.

Table 1. Parameters and R2 (on testing data) values of the NARANN models for inputs (Note: datasets were split

into training, validation and testing).

Parameters

Gross Production (x1)

Energy Imports (x2)

Transmitted Energy (x3)

Energy Exports (x4)

Dataset

%70-15-15

%70-15-15

%70-15-15

%70-15-15

Hidden Layers

10

5

10

10

Lags

R

2

7

0.997

3

0.958

4

0.993

4

0.939

Proceedings of the 5th Annual World Conference

of the Society for Industrial and Systems Engineering,

San Francisco, CA, USA

October 13-14, 2016

197

Figure 1. Scatter plot of the Optimized Elastic Net Regression Model.

Figure 2. Comparative Results for Actual and Forecasted Net Electricity Consumption (with testing data).

4. Conclusions and Future Work

In this paper, a forecasting framework for net electricity consumption of Turkey is proposed. Four independent

variables (inputs), namely; imports, exports, transmitted energy and gross power generation were used to forecast the net

electricity consumption of Turkey. A nonlinear autoregressive-based neural network (NARANN) model was used to forecast

the future values of these independent variables. Afterward, an optimized elastic net (OPEN) regression model was employed

to forecast the net electricity consumption of Turkey. The elastic net was optimized using simulated annealing in which the

evolutionary strategy was used to find good initial values for the simulated annealing search. The results show that the

proposed approach provides high accuracy for net electricity consumption forecasting as the achieved MAPE score was

1.96%. For the future work, forecasted independent values will be improved to be used by the elastic net regression model to

make the projections. Also, other optimization methods will be used to solve the elastic net problem.

Proceedings of the 5th Annual World Conference

of the Society for Industrial and Systems Engineering,

San Francisco, CA, USA

October 13-14, 2016

198

5. Acknowledgment

The authors would like to thank the Turkish Electricity Transmission Company and the Turkish Ministry of Energy

and Natural Resources for their help in providing data.

6. References

Akay, D., & Atak, M. (2007). Grey prediction with rolling mechanism for electricity demand forecasting of

Turkey. Energy, 32(9), 1670-1675.

Azadeh, A., Saberi, M., Gitiforouz, A., & Saberi, Z. (2009). A hybrid simulation-adaptive network based fuzzy inference

system for improvement of electricity consumption estimation. Expert Systems with Applications,36(8), 11108-

11117.

Azadeh, A., Taghipour, M., Asadzadeh, S. M., & Abdollahi, M. (2014). Artificial immune simulation for improved

forecasting of electricity consumption with random variations. International Journal of Electrical Power & Energy

Systems, 55, 205-224.

Donahoe, J. W., & Palmer, D. C. (1989). The interpretation of complex human behavior: some reactions to parallel

distributed processing, edited by J.L. McClelland, De Rumelhart, and the PDP research group. Journal of the

Experimental Analysis of Behavior, 51(3), 399-416.

Ediger, V. Ş., & Akar, S. (2007). ARIMA forecasting of primary energy demand by fuel in Turkey. Energy Policy, 35(3),

1701-1708.

Ediger, V. Ş., & Tatlıdil, H. (2002). Forecasting the primary energy demand in Turkey and analysis of cyclic patterns. Energy

Conversion and Management, 43(4), 473-487.

Erdogdu, E. (2007). Electricity demand analysis using cointegration and ARIMA modelling: A case study of Turkey. Energy

policy, 35(2), 1129-1146.

Hamzaçebi, C. (2007). Forecasting of Turkey's net electricity energy consumption on sectoral bases. Energy Policy, 35(3),

2009-2016.

Hamzaçebi, C., & Kutay, F. (2004). Yapay sinir ağlari ile Türkiye elektrik enerjisi tüketiminin 2010 yilina kadar

tahmini. Gazi Üniversitesi Mühendislik-Mimarlık Fakültesi Dergisi, 19(3).

Kavaklioglu, K., Ceylan, H., Ozturk H., & Canyurt O. (2009). Modeling and prediction of Turkey’s electricity consumption

using artificial neural networks. Energy Conversion and Management, 50(11):2719–2727, 2009.

Kavaklioglu K. (2011). Modeling and prediction of Turkey’s electricity consumption using support vector regression.

Applied Energy, 88(1):368–375.

Kavaklioglu, K. (2014). Robust electricity consumption modeling of Turkey using singular value

decomposition. International Journal of Electrical Power & Energy Systems, 54, 268-276.

Kheirkhah, A., Azadeh, A., Saberi, M., Azaron, A., & Shakouri, H. (2013). Improved estimation of electricity demand

function by using of artificial neural network, principal component analysis and data envelopment

analysis. Computers & Industrial Engineering, 64(1), 425-441.

Kirkpatrick, S. (1984). Optimization by simulated annealing: Quantitative studies. Journal of statistical physics, 34(5-6),

975-986.

Oğcu, G., Demirel, O. F., & Zaim, S. (2012). Forecasting electricity consumption with neural networks and support vector

regression. Procedia-Social and Behavioral Sciences, 58, 1576-1585.

Toksarı, M. D. (2009). Estimating the net electricity energy generation and demand using the ant colony optimization

approach: case of Turkey. Energy Policy, 37(3), 1181-1187.

Tunç, M., Çamdali, Ü., & Parmaksizoğlu, C. (2006). Comparison of Turkey's electrical energy consumption and production

with some European countries and optimization of future electrical power supply investments in Turkey. Energy

Policy, 34(1), 50-59.

Tutun, S., Chou, C. A., & Canıyılmaz, E. (2015). A new forecasting framework for volatile behavior in net electricity

consumption: A case study in Turkey. Energy, 93, 2406-2422.

Ünler, A. (2008). Improvement of energy demand forecasts using swarm intelligence: The case of Turkey with projections to

2025. Energy Policy, 36(6), 1937-1944.