Big Data Based Electricity Price Forecasting Using Enhanced Convolutional Neural Network in the Smart Grid

Abstract

With the development of renewable energies resources and uncertainties in load time series, it is necessary to predict an accurate price for efficient scheduling and operation of generation that is reliable and reduce the power losses in smart grid. The machine learning algorithms are mostly used for power and price forecasting. However, on large data set, it causes over fitting, computational overhead and complexity. To cope with these challenges, in this paper, an electricity price forecasting model is developed using deep learning technique. The proposed method is composed of three modules. First, to performs better feature selection, a hybrid model composed of Mutual Information (MI) and ReliefF is used. Second, Kernel Principal Component Analysis (KPCA) is performed to avoid feature redundancy. Finally, Enhanced Convolution Neural Network (ECNN) performs the regression. To investigate the performance of the proposed model, the results of the proposed is compared with Multilayer Perceptron (MLP) and Support Vector Machine (SVM) as benchmark schemes. The accuracy results show that the performance of our model is better than benchmark schemes. Our technique is robust and helps in better operation and planning of generation in smart grid.

Full text

Big Data Based Electricity Price Forecasting
Using Enhanced Convolutional Neural Network
in the Smart Grid
Muhammad Adil, Nadeem Javaid, Nazia Daood, Muhammad Asim, Irfan ullah,
and Muhammad Bilal
Abstract With the development of renewable energies resources and uncertainties
in load time series, it is necessary to predict an accurate price for efficient schedul-
ing and operation of generation that is reliable and reduce the power losses in smart
grid. The machine learning algorithms are mostly used for power and price forecast-
ing. However, on large data set, it causes over fitting, computational overhead and
complexity. To cope with these challenges, in this paper, an electricity price fore-
casting model is developed using deep learning technique. The proposed method
is composed of three modules. First, to performs better feature selection, a hybrid
model composed of Mutual Information (MI) and ReliefF is used. Second, Kernel
Principal Component Analysis (KPCA) is performed to avoid feature redundancy.
Finally, Enhanced Convolution Neural Network (ECNN) performs the regression.
To investigate the performance of the proposed model, the results of the proposed is
compared with Multilayer Perceptron (MLP) and Support Vector Machine (SVM) as
benchmark schemes. The accuracy results show that the performance of our model
is better than benchmark schemes. Our technique is robust and helps in better oper-
ation and planning of generation in smart grid.
1 Introduction
The reliability and losses in power system due to inappropriate generation of elec-
tricity are the key issues in power system. The recent efforts made in development
of data science in smart grid for electricity load and price forecasting has played a
vital role in balancing the supply and demand and planning an efficient operation of
generation. Smart grid provides the two-way communication between supplier and
Muhammad Adil, Nadeem Javiad (Coressponding Author), Nazia Daood, Muhammad Asim, Irfan
ullah, and Muhammad Bilal
COMSATS University Islamabad, Islamabad 44000, Pakistan e-mail: : nadeem-
javaidqau@gmail.com
1

2 Authors Suppressed Due to Excessive Length
user to prevent the electricity outages and blackouts. To reduce the losses in power
system, it is necessary to predict an accurate price prediction of electricity prices
to make an efficient generation and planning of electricity consumption. With the
introduction of Renewable Energies (RE) and Distributed Energy Resources (RES)
in the recent years, the prediction of electricity prices becomes difficult as they are
intermittent in nature. The generation of solar and wind power depend on solar ra-
diance and wind speed respectively. Moreover,there are various factors affecting the
electricity price forecasting such as renewable energies like solar and wind, fuel
prices, temperature, humidity and seasons that majorly effect the consumption be-
haviour of electricity prices.
Machine learning techniques have been introduced for prediction of electricity
prices for short and long-term forecasting to solve the aforementioned problems.
Machine learning uses the artificial intelligence algorithms that first take the training
data and predict an output function based on learned experience. For minimizing the
loss function, the optimization techniques are used that tune the parameters of the
machine learning algorithm to get an optimal result. Most of the price forecasting
work has been done in literature. The authors addressed the problem to take big
data into consideration for prediction of electricity prices to improve the prediction
accuracy [1]. The Voltage fluctuation in power systems cause the major wastage
of electricity. However, in accurate electricity price forecasting there are still some
issues like cost of computation, reliability and execution time [1]. The MLP is used
for classification of electricity prices that causes the problem of over fitting. In [2],
the authors addressed the problem of long-term forecasting of electricity prices.
In [3], the Stacked De noising Autoencoders (SDA) is used for feature selection.
However, to train an autoencoder, it requires a lot of data processing time, hyper
tuning parameter and over fitting. The authors use Stacked Sparse Autoencoders
(SSAE) which use back propagation to reduce the error. However, back propagation
has problems in weight adjustment [4]. In [7], the authors addressed the issues on
the residential load forecasting to support future grid application.
The aforementioned literature has some limitations which need some attention
to overcome and enhance the performance of electricity price prediction. To tackle
these limitations, an enhanced deep neural network is introduced. First, for good
performance of classfier, a hybrid feature selection is performed using ReliefF and
MI. To avoid feature redundancy, KPCA is performed. The refned features are given
to ECNN for classifcation of electricity prices. The framework performs better when
compared with existing benchmark schemes in terms of performance matrix. The
benchmark candidates are NB, DT, SVR and MLP. The major contributions of this
paper are:
•The complexity and execution time of feature selection is reduced which overall
reduce the computational time and enhance the performance of the framework.
•The problem of overfitting is overcome which increasing the accuracy perfor-
mance of forecasting model.
•The framework performs better on complex data set by predicting the prices
through ECNN which work better on big kernel function time series data.

Title Suppressed Due to Excessive Length 3
The rest of paper is organized as follow. Section II represents the literature re-
view. In Section III and IV, the problem statement and proposed solution is demon-
strated respectively. Section V discusses the methodology for the proposed method.
In VI, the simulations results are discussed. Finally, conclusion and future work are
talk about in the last section.
2 Related Work
The literature review related to our work is discussed in this section.
2.1 Load Forecasting Work
Table 1: Summary of price forecasting work from existing literature
Problem Addressed Proposed Solution Techniques Simulation Validation Limitations
Big data Prediction[1] HSEC RF,ReliefF,
KPCA, SVM
Accuracy and error Complexity and execu-
tion time
Long term forecasting[2] ANN MI,KNN, ANN MAD,RMSE Robustness, overfitting
Overfitting[3] DPP KNN,SVR,
ELM,ENN
SDE,RMSE MAPE Complexity
STLF[10] SVR, BiLSTM EWT,SVR BiL-
STM,BO
MAE,MAPE, RMSE
Complex data[11] EPNet CNN,LSTM MAE,RMSE Execution Time
Limited data selec-
tion[13]
Re-routed method SVR,NN DNN MSE Limited RE integration
In [3], the authors utilize SDA for feature extraction and SVR for regression
for accurate load forecasting. However, to train an autoencoder it requires a lot of
data processing time, hyper tuning parameter and overfitting. In [5], the authors ad-
dressed the problem of extracting features and proposed a framework for load fore-
casting.In [7], the authors addressed the issues on the residential load forecasting to
support future grid application. In [9], the authors addressed the issue of accurately
forecasting the power, the authors proposed SVR technique to predict short-term
load forecasting model. An optimal function is used to predict the forecasting func-
tion. The authors used the case study of four cities to evaluate proposed model and
the result was good. In [14], the authors addressed to predict accurate load forecast-
ing using large time series data, the authors proposed a framework for electric load
forecasting using SVR. In [20-22], authors focused on feature selection techniques
for prediction of load forecasting using large time series data.

4 Authors Suppressed Due to Excessive Length
Table 2: Summary of load forecasting work from existing literature
Problem Addressed Proposed
Solution
Techniques Simulation Validation Limitations
Separate data Prepro-
cessing[3]
SDA SVR,SDA MAPE Over-fitting
Feature extraction[5] Deep NN MLP MAPE,MAE, MRPE Computationally
expensive
STLF[7] LSTM RNN MAPE Gradient vanishing
Fair base line in DR[9] SVR SVR RMSE,MAPE Over-fitting
Automatic lag selec-
tion[14]
KP-SVR ANN,ARIMA
KP-SVR
MAPE Computational time
Computational
Cost[15]
ENN ENN IEEE 14 bus system Limited bus system
2.2 Price Forecasting Work
In [6], the authors addressed the fluctuations in electricity price data that usually
cause the overfitting. In [10], the authors addressed the short-term load and power
forecasting problems. The authors in [11], addressed the problems of traditional
machine learning techniques on complex data set. In [13], the authors addressed
problem of accurate electricity price forecasting. In [23-26], the authors focused on
short term power forecasting time series data and achieve significant results.
3 Problem Statement
In [1], the Support Vector Machine (SVM) is used for regression. However, the al-
gorithm complexity and memory requirements are high. In case of high dimensional
kernel, it generates too many support vectors which reduce the training speed and
decrease the accuracy. For tuning the parameters, Differential Evolution (DE) opti-
mization technique is used. However, DE is prone to converge to local optima. In
[19], the Multilayer Perceptron (MLP) is used for classification of electricity prices.
However, it converges slow and easily trap to local optima. The performance of MLP
on large time series data cause the over fitting and degrade the accuracy of classi-
fier. The authors do not perform the selection and extraction techniques for feature
refinement, which makes the model more complex and reduces the accuracy.

Title Suppressed Due to Excessive Length 5
4 Proposed Model
In this paper, our proposed system model is shown in figure 1. The proposed model
has basically three modules. In the first module, a hybrid model is used for fea-
ture selection using Mutual Information (MI) and ReliefF technique. In the second
module, the Kernel Principle Component analysis (KPCA) avoid duplicate informa-
tion and third module is used for classification of electricity prices using Enhanced
Convolution Neural Network (ECNN).
Fig. 1: Proposed System Model
5 Overview of Proposed Methodology Approach
5.1 Data Collection and Pre-processing
In the data pre-processing stage, two steps are performed are data cleaning and data
normalization. In data cleaning, the null or undefined and irrelevant features from
the data set are removed. In data normalization, the data is scaled in to the range 0
and 1. The normalization formula given in [17] is used to normalize the data given
in equation 1. The purpose of normalization is to get data on a common scale.
A‘=(A−Min(A))
(Max(A)−Min(A)∗((D−C) + C)(1)

6 Authors Suppressed Due to Excessive Length
Where A’ is normalized component of data sequence. C and D are the pre-defined
boundaries which is set between 0 and 1 respectively
5.2 Feature Selection
The accuracy of regression depend on the input features. Therefore, it necessary to
provide the relevant information to the classifier. A hybrid model is used for feature
selection that consist of MI and ReliefF.
5.2.1 Mutual Information Algorithm
MI calculate the relationship of two random variables x and y and measures how
much information is communicated. If information between two random variables
is zero, then it means that they are independent on each other and information gain
is zero. The information gain in equation 2,3 and 4 given in [18] is calculated as
IG(H(fi)−H(fi|C)) (2)
from the above equation, H(Fi)represents the entropy for the feature of index i,
while H(fi|C)shows measure the entropy of feature of index i belonging to class C.
H(fi) = −∑
j
p(xj)log2p(xj)(3)
H(fi|C) = −∑p(xj|ck)∑
j
p(xj|ck)log2(p(xj|ck)) (4)
In this paper, the threshold value is set 0.1, any feature value having mutual
information gain greater than 0.1, the feature is selected, else it is rejected.
5.2.2 ReliefF Algorithm
ReliefF is widely used for feature selection in data science and it is enhanced version
of Relief. In ReliefF, the features importance value is calculated on the basis of
score and rank of each feature which is based on feature value difference observed
between the nearest neighbour instant pair find by Euclidean distance. The formula
to calculate the feature importance in [1] is given as
WF[Tk] = WF[Tk]∑k
j=1di f f (A,a∗
,Hj)
m∗k+∑C6=class(λ)di f f (A,a∗
,M j)
m∗k(5)

Title Suppressed Due to Excessive Length 7
If feature value difference is observed in nearest neighbour within the class called
miss, the feature value is decreased and get less feature importance value. If feature
value difference is observed with nearest neighbour pair within different class called
hit, then feature value increases and get high importance value.
The combined importance of ReliefF and MI is calculted for final feature selec-
tion. The thresold is selected as 0.5. Any feature value having importance less than
0.5 are rejected and feature importance having value greater than 0.5 are selected.
5.3 Extraction of Features
Extraction of feature is performed to extract redundant information from already
selected features. PCA is mostly used to avoid feature redundancy and assumes lin-
ear mapping. However, electricity price forecasting requires non-linear mapping.
In this paper, KPCA is used for feature redundancy and better exploit the compli-
cated structure from high to low dimension. The KPCA out performs better feature
extraction than the existing techniques.
5.4 Price Forecasting Model
The ECNN is deep neural network is used for electricity price forecasting in this
framework. The working of ECNN is shown in the flow diagram as shown in fig-
ure 5. The four layer performs the prediction of electricity prices. Firstly, the pre-
processed data after selection and extraction is fed at the input layer. Second is
the convolution layer, that contain filters and perform features mapping function. It
takes the input image and learn by calculating the weights and bias. The activation
function is Relu as given in [17].
Relu(x) = max(0,x)(6)
In equation 6, x shows the input features. If x is positive, the function returns the
actual value and return zero for negative value of x. The output from convolution
layer is given as input to Pooling layer. Pooling layer as shown in figure lies between
drop out layer and convolution layer. It combines the output of neurons and reduces
the computational overhead due to a smaller number of parameters. The Adam is
used as an optimizer that adjusts the learning weight and momentum.

8 Authors Suppressed Due to Excessive Length
Fig. 2: Flow daigram of ECNN
6 Simulation Results
6.1 Feature Selection
A hybrid feature selection technique using MI and ReliefF is used for feature selec-
tion to select the important features.

Title Suppressed Due to Excessive Length 9
Fig. 3: Feature Selection
6.2 Computational Time Comparison of MI with RF
In order to reduce the execution time for feature selection as discussed in section
three, in the proposed model, for feature selection MI is applied which is simple
technique to overcome this limitation. The Random Forest (RF) has hig computa-
tional time and complexity. To construct the decision tree it takes enough time . RF
computational time is twice greater than MI.
6.3 Comparison of Radial PCA with PCA and Linear KPCA
Feature extraction is performed to avoid the irrelevant information from the selected
features. KPCA performs the feature extraction that remove the duplicate informa-
tion to reduce overfitting. The comparison of Redial KPCA, linear KPCA and PCA
is shown in figure 5. Redial KPCA efficently extract the important principle compo-
nents than the other two methods.

10 Authors Suppressed Due to Excessive Length
Fig. 4: Performance of Redial and linear KPCA
Fig. 5: Comparision of proposed ECNN and benchmark schemes

Title Suppressed Due to Excessive Length 11
6.4 Performance of ECNN on Electricity Price Forecasting
The ECNN performs better for large time series data. In order to verify the effec-
tiveness of proposed model, the prediction and actual results as shown in figure 5
are very near to each other. The dotted yellow line shows price forecasting of ECNN
and red line represent actual price forecasting. To show effectiveness of ECNN, the
proposed model is compared with benchmark schemes and the results are shown in
the mention figures. The accuracy performance in figure 6 shows that the proposed
model performance is better results than existing benchmark schemes.
Fig. 6: Comparision of ECNN and benchmarks
7 Conclusion
This paper proposed a framework for forecasting the electricity price using deep
learning model on large time series data. The prime goal is to overcome the prob-
lems of over fitting, accuracy and complexity. To cope with these problems, we
proposed a model consists three modules. First, a hybrid module that comprised of
MI and ReliefF performs feature selection. The refined features are given to KPCA

12 Authors Suppressed Due to Excessive Length
to avoid further feature redundancy. Finally, an efficient prediction of power prices
is performed using ECNN. The proposed model is compared with existing mod-
els and the result of proposed method is promising. The system model is flexible
enough to achieve better result. For future work, the proposed model will be inte-
grated with more external features and increase system memory capacity to enhance
the performance of the model .
References
1. Wang, Kun, et al. ”Robust big data analytics for electricity price forecasting in the smart grid.”
IEEE Transactions on Big Data 5.1 (2017): 34-45.
2. Keles, Dogan, et al. ”Extended forecast methods for day-ahead electricity spot prices applying
artificial neural networks.” Applied energy 162 (2016): 218-230.
3. Tong, Chao, et al. ”An efficient deep model for day-ahead electricity load forecasting with
stacked denoising auto-encoders.” Journal of Parallel and Distributed Computing 117 (2018):
267-273.
4. Chen, Kunjin, Jun Hu, and Jinliang He. ”A framework for automatically extracting overvoltage
features based on sparse autoencoder.” IEEE Transactions on Smart Grid 9.2 (2016): 594-604
5. Guo, Zhifeng, et al. ”A deep learning model for short-term power load and probability density
forecasting.” Energy 160 (2018): 1186-1200.
6. Shayeghi, Hossein, A. Ghasemi, Mohammad Moradzadeh, and M. Nooshyar. “Day-ahead elec-
tricity price forecasting using WPT, GMI and modified LSSVM-based S-OLABC algorithm.”
Soft Computing 21, no. 2 (2017): 525-541.
7. Kong, Weicong, et al. ”Short-term residential load forecasting based on LSTM recurrent neural
network.” IEEE Transactions on Smart Grid 10.1 (2017): 841-851.
8. Gu, Quanquan, Zhenhui Li, and Jiawei Han. “Generalized fisher score for feature selection.”
arXiv preprint arXiv:1202.3725 (2012).
9. Chen, Yongbao, et al. ”Short-term electrical load forecasting using the Support Vector Regres-
sion (SVR) model to calculate the demand response baseline for office buildings.” Applied
Energy 195 (2017): 659-670.
10. Cheng, Hangyang, et al. ”A hybrid electricity price forecasting model with Bayesian opti-
mization for German energy exchange.” International Journal of Electrical Power and Energy
Systems 110 (2019): 653-666.
11. Kuo, Ping-Huan, and Chiou-Jye Huang. ”An electricity price forecasting model by hybrid
structured deep neural networks.” Sustainability 10.4 (2018): 1280.
12. Lee, Woonghee, et al. ”Forecasting Solar Power Using Long-Short Term Memory and Con-
volutional Neural Networks.” IEEE Access 6 (2018): 73068-73080.
13. Luo, Shuman, and Yang Weng. ”A two-stage supervised learning approach for electricity price
forecasting by leveraging different data sources.” Applied energy 242 (2019): 1497-1512.
14. Maldonado, Sebastian, Agustin Gonzalez, and Sven Crone. ”Automatic time series analysis
for electric load forecasting via support vector regression.” Applied Soft Computing 83 (2019):
105616.
15. Devarajan, Sandhiya, and S. Chitra. ”LOAD FORECASTING MODEL FOR ENERGY
MANAGEMENT SYSTEM USING ELMAN NEURAL NETWORK.” International Research
Journal of Multidisciplinary Technovation 1.5 (2019): 48-56.
16. Devarajan, Sandhiya, and S. Chitra. ”LOAD FORECASTING MODEL FOR ENERGY
MANAGEMENT SYSTEM USING ELMAN NEURAL NETWORK.” International Research
Journal of Multidisciplinary Technovation 1.5 (2019): 48-56.
17. Zahid, Maheen, et al. ”Electricity price and load forecasting using enhanced convolutional
neural network and enhanced support vector regression in smart grids.” Electronics 8.2 (2019):
122.

Title Suppressed Due to Excessive Length 13
18. Yu, Lei, and Huan Liu. “Feature selection for high-dimensional data: A fast correlation-
based filter solution.” In Proceedings of the 20th international conference on machine learning
(ICML-03), pp. 856-863. 2003.
19. Giovanelli, Christian, et al. ”Exploiting Artificial Neural Networks for the Prediction of An-
cillary Energy Market Prices.” Energies 11.7 (2018): 1906.
20. Omaji Samuel, Fahad A. Alzahrani, Raja Jalees Ul Hussen Khan, Hassan Farooq, Muham-
mad Shafiq, Muhammad Khalil Afzal and Nadeem Javaid, ”Towards Modified Entropy Mutual
Information Feature Selection to Forecast Medium-Term Load Using a Deep Learning Model
in Smart Homes”, Entropy, Volume: 22, Issue: 1, Article Number: 68, Pages: 1-31, Published:
January 2020, ISSN: 1874-4907. DOI: https://doi.org/10.3390/e22010068.
21. Rabiya Khalid, Nadeem Javaid, Fahad A. Al-zahrani, Khursheed Aurangzeb, Emad-ul-
Haq Qazi and Tehreem Ashfaq, ”Electricity Load and Price Forecasting Using Jaya-
Long Short Term Memory (JLSTM) in Smart Grids”, Entropy, Volume: 22, Issue: 1,
Article Number: 10, Pages: 1-21, Published: January 2020, ISSN: 1099-4300. DOI:
https://doi.org/10.3390/e22010010.
22. Sana Mujeeb and Nadeem Javaid, ”ESAENARX and DE-RELM: Novel Schemes for Big Data
Predictive Analytics of Electricity Load and Price”, Sustainable Cities and Society, Volume:
51, Article Number: 101642, Pages: 1-16, Published: November 2019, ISSN: 2210-6707. DOI:
https://doi.org/10.1016/j.scs.2019.101642.
23. Sana Mujeeb, Turki Ali Alghamdi, Sameeh Ullah, Ayesha Fatima, Nadeem Javaid and Tanzila
Saba, ”Exploiting Deep Learning for Wind Power Forecasting based on Big Data Analytics”,
Applied Sciences, Volume: 9, Issue: 20, Article Number: 4417, Pages: 1-19, Published: October
2019, ISSN: 2076-3417, DOI: https://doi.org/10.3390/app9204417.
24. Aqdas Naz, Nadeem Javaid, Muhammad Babar Rasheed, Abdul Haseeb, Musaed Alhussein
and Khursheed Aurangzeb, ”Game Theoretical Energy Management with Storage Capacity Op-
timization and Photo-Voltaic Cell Generated Power Forecasting in Micro Grid”, Sustainability,
2019, Volume: 11, Issue: 10, Article Number: 2763, Pages: 1-22, Published: May 2019, ISSN:
2071-1050. DOI:https://doi.org/10.3390/su11102763.
25. Aqdas Naz, Muhammad Umar Javed, Nadeem Javaid, Tanzila Saba, Musaed Alhussein
and Khursheed Aurangzeb, ”Short-Term Electric Load and Price Forecasting Using En-
hanced Extreme Learning Machine Optimization in Smart Grids”, Energies, Volume: 12,
Issue: 5, Article Number: 866, Pages: 1-30, Published: March 2019, EISSN 1996-1073.
DOI:https://doi.org/10.3390/en12050866.
26. Sana Mujeeb, Nadeem Javaid, Manzoor Ilahi, Zahid Wadud, Farruh Ishmanov and Muham-
mad Khalil Afzal, ”Deep Long Short-Term Memory: A New Price and Load Fore-
casting Scheme for Big Data in Smart Cities”, Sustainability, Volume: 11, Issue: 4,
Article Number: 987, Pages: 1-29, Published: February 2019, ISSN: 2071-1050. DOI:
https://doi.org/10.3390/su11040987.