Conference PaperPDF Available

Load and Price Forecasting based on Enhanced Logistic Regression in Smart Grid

Authors:

Abstract and Figures

Smart Grid (SG) is a modern electricity grid that enhance the efficiency and reliability of electricity generation, distribution and consumption. It plays an important role in modern energy infrastructure. Energy consumption and generation have fluctuating behaviour in SG. Load and price forecasting can decrease the variation between energy generation and consumption. In this paper, we proposed a model for forecasting , which consists of feature selection, extraction and classification. With the combination of Fast Correlation Based Filter (FCBF) and Re-cursive Feature Elimination (RFE) is used to perform feature selection to minimize the redundancy. Furthermore, Mutual Information technique is used for feature extraction. To forecast electricity load and price, we applied Naive Bayes (NB), Logistic Regression (LR) and Enhanced Logistic Regression (ELR) techniques. Our proposed technique ELR beats other techniques in term of forecasting accuracy of load and price. The load forecasting accuracy of ELR, LR and NB techniques are 80%, 82% and 85%, while price forecasting accuracy are 78%, 81% and 84%. Locational Marginal Price of Pennsylvania, Jersey, Maryland (LBM-PJM) market data used in our proposed model. Forecasting performance is assessed by using RMSE, MAPE, MAE and MSE. Simulation results show that our proposed technique ELR is perform better than other techniques.
Content may be subject to copyright.
Load and Price Forecasting Based
on Enhanced Logistic Regression
in Smart Grid
Aroosa Tahir1, Zahoor Ali Khan2, Nadeem Javaid3(B
), Zeeshan Hussain2,
Aimen Rasool2, and Syeda Aimal2
1Sardar Bhadur Khan Women University Quetta, Quetta 87300, Pakistan
2Computer Information Science, Higher Colleges of Technology, Fujairah 4114, UAE
3COMSATS University Islamabad, Islamabad 44000, Pakistan
nadeemjavaidqau@gmail.com
http://www.njavaid.com/
Abstract. Smart Grid (SG) is a modern electricity grid that enhance
the efficiency and reliability of electricity generation, distribution and
consumption. It plays an important role in modern energy infrastruc-
ture. Energy consumption and generation have fluctuating behaviour
in SG. Load and price forecasting can decrease the variation between
energy generation and consumption. In this paper, we proposed a model
for forecasting, which consists of feature selection, extraction and classi-
fication. With the combination of Fast Correlation Based Filter (FCBF)
and Recursive Feature Elimination (RFE) is used to perform feature
selection to minimize the redundancy. Furthermore, Mutual Informa-
tion technique is used for feature extraction. To forecast electricity load
and price, we applied Naive Bayes (NB), Logistic Regression (LR) and
Enhanced Logistic Regression (ELR) techniques. Our proposed technique
ELR beats other techniques in term of forecasting accuracy of load and
price. The load forecasting accuracy of ELR, LR and NB techniques
are 80%, 82% and 85%, while price forecasting accuracy are 78%, 81%
and 84%. Locational Marginal Price of Pennsylvania, Jersey, Maryland
(LBM-PJM) market data used in our proposed model. Forecasting per-
formance is assessed by using RMSE, MAPE, MAE and MSE. Simu-
lation results show that our proposed technique ELR is perform better
than other techniques.
Keywords: Smart Grid ·Naive Bayes ·Logistic Regression ·
Fast Correlation Based Filter ·Recursive Feature Elimination ·
Mutual Information
1 Introduction
Traditional Grid (TG) is an interconnection of many elements such as trans-
mission lines, distribution lines and different types of load. They are located far
from power consumption areas, so power consumption transmitted from long
c
Springer Nature Switzerland AG 2019
L. Barolli et al. (Eds.): EIDWT 2019, LNDECT 29, pp. 221–233, 2019.
https://doi.org/10.1007/978-3-030-12839-5_21
222 A. Tahir et al.
transmission lines. Traditional power grid is old version of Smart Grid (SG).
SG precisely manage distribution, consumption and generation of energy which
introduce communication and control of technology in power grid [1]. Smart grid
helps the consumers to use electricity in secure manner. The new generation is
attached against the SG due to extensive shortage of energy during summer. It
suggest electricity consumers for load transferring and storage that can overcome
their power consumption cost [2]. SG creates connection between consumer and
utility. Customer consumes operation in SG for decreasing price by load shifting
and energy storage. Consumer can manage their demand of energy on Demand
Side Management (DSM) according to price variation [3]. Customer has choice
to minimize their electricity cost through energy control and shifting the load.
Consumer can switch the load and off depending on the electricity pricing. Com-
petitive market takes advantage from load and price forecasting. Many valuable
operational decisions are depends on load predication such as power reliability
analysis, maintenance planning and generation scheduling. Consumer takes part
in operation of SG that shifts the load from on-peak hours to off-eak hours.
In this paper, we highlight the electricity load and price problems, to solve
these problems we used many techniques. Machine learning methods works excel-
lent on data analytics, they categorize the data into training and testing step
by step. Feature selection is an important operation for classification and selects
important features. However, Feature selection have many common problems
in the process is that redundancy of selected feature is not minimizing. Fast
Correlation Based Filter (FCBF) and Recursive Feature Elimination (RFE) are
used as a solution of feature selection problem. Mutual Information (MI) feature
extraction method is used for feature reduction. We use two techniques Naive
Bayes (NB) and Logistic Regression (LR) classifiers that have better capability
of non-linear estimation and self learning. We proposed Enhance (ELR) classi-
fier that suitable for load and price forecasting. These methods are capable for
extract the complex data representation with better prediction. ELR has better
computation power when comparing it with LR. The objective of our paper is
to predict Load and price accurately using data analytics in the smart grid. For
this problem we use NB, LR and ELR to predict the best results of electricity
load and price forecasting.
1.1 Motivation
After reviewing papers [4,5], existing forecasting method have following motiva-
tions:
Data analytics is not taken in the discussion of load and price forecasting
method. Performance Matrix is taken on the electricity price data which is
not enough large that minimize prediction.
Intelligent methods such as ANN, SVM, DE and DWT have limited general-
ization capacity, they have fitting problem.
Non-linear and other patterns of energy price is critical to predict with tra-
ditional data. The usage of data analytics make it feasible that generates the
pattern of data and predicts accuracy.
Data Analytics Load and Price Forecasting in Smart Grid 223
Automatically feature selection and extraction can efficiently extract and
select useful hidden patterns in the data (Table 1).
Table 1. List of abbreviations
Abbreviation Full f o r m
ARMA Auto Regressive Moving Average
AI Artificial Intelligence
ANN ArticialNeuralNetwork
DSM Demand Side Management
DE Differential Evaluation
DWT Discrete Wavelet Transformation
ELR Enhanced Logistic Regression
FCBF Fast Correlation based Filter
LR Logistic Regression
MI Mutual Information
MAPE Mean Average Percentage Error
MSE Mean Square Error
MAE Mean Absolute Error
MLR Multi Linear Regression
NB Naive Bayes
RFE Recursive Feature Elimination
RMSE Root Mean Square Error
SG Smart Grid
SVM Support Vector Machine
SNN Shallow Neural Network
SVR Support Vector Regression
TG Traditional Grid
1.2 Problem Statement
There is high variation between energy generation and consumption [6]. There is
no proper strategy of energy generation, which leads to extra energy generation.
To avoid spare energy generation, we perform forecasting. With the help of load
and price forecasting, generation of energy can be controlled with in the limits.
1.3 Contributions
This paper have major contributions such as:
We implement a framework for achieving accurate load and price forecasting
by using the data analytics in SG.
224 A. Tahir et al.
Feature selection, extraction and classification is proposed in our model to
solve the addressed problem.
In feature selection, we combine FCBF and RFE which gives us importance
of feature selection. MI is used for feature extraction.
Naive Bayes and LR classifiers are used for forecasting.
However, we enhance LR to beat these two techniques.
We have use the real world energy load and price data which performs better
simulation that have better results.
2 Related Work
In the paper [1] Data Analytics Demand response technique propose for res-
idential load. Data analytics scheme tested on PJM and Open Energy Infor-
mation and show that it minimizing the load. In this paper author forecast
day-ahead price forecasting, which contain six year long dataset and four auto
regressive expert model transformed price [2]. ANN present for price forecast-
ing, the focus is on the selection and preparation of fundamental data that has
effect on electricity price [4]. In this paper [5], author admit the effect of data
integrity attacks on the accuracy of four representative load forecasting model
such as Multiple Linear regression (MLR), Support Vector Regression (SVR),
ANN and Fuzzy Integration Regression (FIR). These models used to generate
one-year-ahead forecasting to provide the comparison of their forecast error. The
concepts of interaction for feature selection is introduce as Mutual Information
(MI) and Interaction Gain (IG). These techniques measure the feature relevancy
and redundancy, measures the load and price by merging them [6]. In [7] author
reduce the energy purchase from wholesale market at the high price with operat-
ing Behind The Meter Storage System (BESS). MI use for feature selection and
Intra Rolling Horizon (IRH) and Pre-Dispatch Price use for prediction. Forecast-
ing the electricity price, three models are used to merge the Random Forest (RF)
and Relief-F algorithm based on Gray Correlation Analysis (GCA) for feature
selection to eliminate the feature redundancy [8].
Kernel Function and Principle Component Analysis (KPCA) used for feature
extraction. Author use Differential Evaluation (DE) for price forecasting, which
based on Support Vector Machine (SVM) classifier. Author hybrid the Discrete
Wavelet Transformation (DWT), Empirical Mode Decomposition (EMD) and
Random Vector Functional Link Network (RVFL) Techniques to forecast the
short term load [9]. In paper [10], filter the feature by Stacked De-noising Auto-
Encoder (SDAs) from energy load to train the SVR for prediction. The day-
ahead load forecasting compare SVR and ANN, validates the performance for
improvement. Author combine the Kernel Extreme Learning Machine (KELM)
based on self-adapting (SAPSO) and (ARMA) [11]. Their experimental result
show that developed method has more accurate prediction. Multi variation MI
and SVR used for feature selection and price prediction [12].
These two methods also measure relation between price and load. Index
bad sample matrix (IBSM), Optimize Algorithm (OA) and Dynamic Choos-
ing (DCANN) are proposed for day-ahead price forecasting [13]. Proposed new
Data Analytics Load and Price Forecasting in Smart Grid 225
feature strategy for short term load forecasting [17], uses for feature selection are
MI and Neural Network (NN) which remove unnecessary and redundant candi-
dates input. Data analysis is the way of predicting future value, if future stock
price will decreases or increases [18]. To predict the future price used prediction
Table 2. Summary of related work
Techniques used Objectives Dataset Achievement Limitation
DA [1]Peak load
Reduction by
analyzing the
consumption
data of SH
PJM and open
energy
information
Load forcasting Tra d in g o f u se r
satisfaction in
response to the
price for further
boost the saving
MAP, SVT [2]Price prediction EPEX, PJM,
Nord Pool
Electricity price
forecasting
Redundancy in
feature are not
describe
ANN [4]Develop ANN
model for
electricity
forecasting from
existing ANN
model
European Power
Exchange
(EPEX)
Price forecasting Measured on
historical data
and represented
by connection
weight
MLR, SVR, ANN [5]Retrive load
forecasting
Global electricity
forcasting
competition 2012
Load fecasting Data intergrated
attack in other
similer
forecasting feild
NN, SCA, MI [6]Short term Price
prediction
PJM Price forecasting Redundant
feature not
removed
IRH, MI [7]Efficient for price
forecasting
Ontario’s
electricity market
Price Redundant
feature not
removed
DWT, EM I, RVFL [8]Load prediction
of time series
AEMO Load forecasted No good results
for long term
data
RF, Relief-f, GCA,
KCPA, DE-S VM [9]
Price prediction HSEC Price forecasted KPCA cannot
model non-linear
data
SDAs, SVR, ANN [10]Compare SVR
and ANN and
forecast load
City or State of
U.S
Load forecasting Feature reduction
do not remove
SAPSO, ARMA,
KELM [11]
Price forecasting PJM Price forecasted Big data is not
used
IG, MI [12]Best Feature
selection
PJM Price forecasted More reduction
of features,
decreases the
accuracy score
Multi variate MI,
SVR [13]
Feature selection
and price
prediction
AEMO Measure relation
b/w price and
load, Price
forecasted
Redundancy in
features not
eliminated
IBSM, OA, DCANN
[14]
Find the best
parameter
PJM, AEM Price forecasting Limited for both
dataset
NB, ANN [17]Predict future
Price using
prediction
concept
Logs from yahoo
finance and store
in stock market
Price forecasting This method can
not be use for big
data
BR, NN, TB, GPR,
MLR [18]
Predict future
cooling load
WSHP Load forecasting Error arise
because of
weather change
226 A. Tahir et al.
concept. NB classifier use for prediction concept. Six performance indicators were
made for use to assess the prediction performance of 6 models [19]. In simulation
results display that precision of TB, GPR, NN, MLR and MAPE are compare
for load forecasting (Table 2).
3SystemModel
Our proposed system consist of many steps such as: preprocessing of data and
splitting by training and testing, feature selection by RFE and FCBF, Feature
extraction by MI, and forecasting of load and price oerformed by three techniques
NB, LR and ELR shown in Fig. 1.
Fig. 1. System model
3.1 Preprocessing
For electricity load and price forecasting, it is necessary to collect load and
price data that reflect the power usage of real world. In propose system model,
Locational Marginal Price Pennsylvania, Jersey, Maryland (LBMP PJM) 2015
to 2017 market dataset is used for load and price forecasting [20]. Three years
data is used and divided month wise, i.e. January 2015, January 2016, January
2017 etc. this dataset is a regional transmission organization, which is managed
by system operator, also responsible for whole sale energy market operations.
Data is categorized into three steps as training, validation, and testing. First
three weeks of month are used as training and last week for testing.
3.2 Methodology
We present two techniques RFE and FCBF method for feature selection. RFE
removes the weakest feature until the specific number of feature is reached. FCBF
works sequentially, where one feature is selected and then unusable features are
removed by the selected one. FCBF is not use as single algorithm, however
we combine other classifier with RFE for feature selection. MI that measure
the dependency between two variable train for each data, extract the selected
feature and remove the redundancy. After that Feature selection and extraction
have normalized data. The output of all features are combined and classified by
Data Analytics Load and Price Forecasting in Smart Grid 227
NB and LR, when all weakest features are removed until the specified feature
is received and then formulate final prediction. We enhance LR to get better
result. Proposed model procedure can be concluded as:
RFL and FCBF use for select the feature and MI to extract each feature into
several inputs.
NB and LG used as classifier for each obtained sub-series.
ELR used to comparison with these classifiers.
The load data is split and normalized into categories as train, validation and
test.
Training data is used for train the network and tested on validation data.
Errors of prediction are calculated on data for validation.
When network is tuned then update on new data.
The network is tested on data, and predict the load and price. Prediction
performance is performed by RMSE, MAPE, MAE, and MSE.
4 Description of Forecasting Techniques
Forecasting the load and price many techniques use such as RFE, MI, FCBF,
NB and LR. Description of these techniques are as follows:
4.1 RFE
RFE is feature selection method that fits the model and remove the weakest fea-
ture until specific features are receive. RFE used to find best number of feature,
cross validation and calculate the different feature subset and select the best col-
lection of feature. Feature is selected in RFE by resourcefully considering smaller
set of feature. Firstly RFE model fits on data, after that we have many features
and its importance. We drop the feature with least importance, and then our
model fits on the remaining feature. Process is repeated many times until we get
best feature.
4.2 MI
MI used to detect the most applicable feature with less inessential information
and measure the quality between two random variables [14]. Two random vari-
able are a and b, which can be explained the information of b that we get by
studying a. a and b denoted by (a; b) as continuous variable. Defined as joint
probability distribution P (a; b) and individual probability distribution P(a) and
P(b).
SC =a1,a
2,a
3...an(1)
Where SC is set of conditional value and b is variable for forecasting. Appli-
cability of each input variable aiwith target variable b, which is denoted by
Q(ai).
Q(ai)=|(ai;b)|(2)
228 A. Tahir et al.
For example, if a and b are not dependent to each other, then b does not give
any information about a. so their mutual information can be zero.
4.3 FCBF
FCBF method is used for feature selection. Feature selection involves discrete
expression before calculating the feature. FCBF have two categories filter and
wrapper method. Filter method depend on the training data that select the fea-
tures without including the training data. The wrapper methods depend upon
the proposed algorithm in the feature selection and used to calculate and deter-
mine that which feature is selected. In filter method, use learning algorithm
from feature selection is RFE which work with SVM and use efficiently because
it show the dependency of them. In wrapped method, we use MI to guide the
search process to weight the feature.
4.4 NB
Naive Bayes is classification technique depend on bayes theorem. NB classifier
assume that the existence of specific feature in a class is separated to the exis-
tence of other feature. Bayes algorithm is used to calculate the posterior property.
NB easy to construct and useful for very large dataset. NB also perform well in
multi class forecasting. When assumption independent holds, NB perform better
than other model such as logistic regression.
4.5 LR
Logistic Regression used for evaluate the dataset in which there are many vari-
able that regulate an outcome. The outcome is measure with binary variable
in which there are only two dependent variables on outcome (true or false).
Algorithm 1. Algorithm of Naive Bayes.
Require: Input: [Training data P]
1: (F=(F1,F
2,F
3......Fn)) value of predict variation in the test data
2: Output: [Class of testing dataset]
3: Read the Training data P
4: Calculate the mean and Standard derivation
5: Predict the variable in each class
6: for i=1toPopulation do
7: Calculate the probability of Fi using distribution of each class;
8: Until the probability of all predictor variable;
9: (F1,F
2, ...Fn) has been calculated;
10: end for
11: end for
12: Calculated the probability of class
13: achieved the greatest result
Data Analytics Load and Price Forecasting in Smart Grid 229
LR generates combination of formulas to predict the logit transformation of the
probability that being the characteristic of interest. LR algorithm use as perfor-
mance baseline because easy to implement many task.
Algorithm 2. Algorithm of Logistic Regression.
Require: Input: [Initialize the prediction as t=0 tow=0]
1:
2: for t=1,2,3, ...n do
3: Compute the prediction gradient for (Xi);
4: For each example (Xi,Y
i);
5: For each non Zero feature of Xi with index (Xj);
6: if j is not w, set w[j]=0 then
7: set w[j]=w[j]+((YiPi)Xj);
8: end if
9: end if
10: Update F
11: end for
12: end for
13: iterate to the next step until it is time to stop
14: Output: [of the feature is W.]
5 Results
Simulation results performed by python, daily electricity load data of LBM PJM
are taken of three year as input data for simulator. To achieve the accuracy of
load and price forecasting we are using three years electricity load and price data.
FCBF and RFE are used for feature selection. We apply these two techniques
which give us feature importance, control and grade of our feature. Every feature
in feature selection have sequence like vector. After feature selection we apply MI
for feature extraction. Mutual information measure the dependency between two
variables that used for training the data. To remove the redundancy of selected
feature we use MI. Feature that has effect on exact point are removed. After that
load and price data is normalized, Then we apply NB, LR and ELR classifier on
Fig. 2. Normalized load and price
230 A. Tahir et al.
load and price data. Data is divided into month wise and split into categories i.e.
train, validate and test. After that network is tuned and validated, then network
is tested the data. When we test the data then load and price is forecast.
5.1 Load Forecast
Firstly load data is used for training the forecast model. Data is splitting into
training and testing in which training is 225% and testing is 75%. Figure 2shows
normalized load and price forecasting. Figure 3(a) Show one week prediction,
Fig. 3(b) Show one month prediction and Fig.3(c) show all nine months (January
2015 to March 2015, January 2016 to March 2016 and January 2017 to March
2017) prediction. All the similar months data are trained in the same pattern. we
are comparing NB, LR with ELR for better prediction. Accuracy of NB is 80%,
LG 82% and ELR 85% shows that our proposed technique ELR beats other two
techniques.
5.2 Price Forecast
Price data is taken to forecast the price. Figure 4(a) Show one week prediction,
Fig. 4(b) Show one month prediction and Fig.4(c) show all nine months (January
2015 to March 2015 etc.) prediction. All the similar months of the data are
trained in the same pattern. These graphs show that ELR performing better
than NB and LR. Accuracy of these techniques show that NB has 78%, LG 81%
and ELR 84%.
Fig. 3. Fig (a) shows one week prediction, fig (b) shows one month prediction, fig (c)
shows nine months prediction
Fig. 4. Fig (a) shows one week prediction, fig (b) shows one month prediction, fig (c)
shows nine month prediction
Data Analytics Load and Price Forecasting in Smart Grid 231
6 Performance Evaluation
To measure the performance of load and price four indicators are used as: Root
Mean Square Error (RMSE), Mean Average Percentage Error (MAPE), Mean
Square Error (MSE) and Mean Absolute Error (MAE). Figure 5(a) shows the
comparison of load prediction and Fig. 5(b) shows the comparison of price pre-
diction. MSE has the lowest error value in ELR i.e., 0.784 in price forecasting
and 0.723 in load forecasting. Formulas of MAPE, RMSE MSE and MAE are
given in Eqs. 3,4,5and 6.
MAPE =1
T
T
n=1
|(sYs)|(3)
RMSE =
1
T
TM
tm=1
(Av Fv)2(4)
MSE =1
T
TM
tm=1
(Av Fv)2(5)
MAE =N
n=1 |(FvAv)|
N(6)
Where AVis test value at time t and Fvis predicted value at time t. MSE has
less error than MSE, MAE and RMSE. Experimental results show that ELR
have better because it shows less error than other performance matrics.
Fig. 5. Error value comparison of load and price
7 Conclusion
In this paper, three year LBM-PJM 2015 to 2017 market data is taken for forecat-
ing the load and price. The proposed model comprises form data preprocessing,
selection, extraction and classification. We combine FCBF and RFE for feature
232 A. Tahir et al.
selection and MI for Extraction. After selection of our selected feature, we fore-
cast load and price with NB, LR and ELR techniques. We compare our proposed
technique ELR with NB and LR to get better result. Experimental results prove
the effectiveness of proposed ELG technique in forecasting. ELR beats other tech-
niques in term of forecasting accuracy. ELR, LR and NB techniques accuracy is
84%, 82% and 80%. Numerical results show that ELG based forecasting model
has lesser in MSE and MAPE than NB and LG. The feasibility of proposed ELR
model is confined by its performance that is well known in our data.
References
1. Jindal, A., Singh, M., Kumar, N.: Consumption aware data analytical demand
response scheme for peak load reduction in smart grid. IEEE Trans. Ind. Electron.
65, 8993–9004 (2018)
2. Amjady, N., Keynia, F.: Day-ahead price forecasting of electricity markets by a
new feature selection algorithm and cascaded neural network technique. Energy
Convers. Manag. 50(12), 2976–2982 (2009)
3. Huang, D., Zareipour, H., Rosehart, W.D., Amjady, N.: Data mining for electricity
price classification and the application to demand-side management. IEEE Trans.
Smart Grid 3(2), 808–817 (2012)
4. Keles, D., Scelle, J., Paraschiv, F., Fichtner, W.: Extended forecast methods for
day-ahead electricity spot prices applying artificial neural networks. Appl. Energy
162, 218–230 (2016)
5. Luo, J., Hong, T., Fang, S.C.: Benchmarking robustness of load forecasting models
under data integrity attacks. Int. J. Forecast. 34(1), 89–104 (2018)
6. Wang, K., Xu, C., Zhang, Y., Guo, S., Zomaya, A.: Robust big data analytic for
electricity price forecasting in the smart grid. IEEE Trans. Big Data (2017)
7. Chitsaz, H., Zamani-Dehkordi, P., Zareipour, H., Parikh, P.: Electricity price
forecasting for operational scheduling of behind-the-meter storage systems. IEEE
Trans. Smart Grid 9, 6612–6622 (2017)
8. Qiu, X., Suganthan, P.N., Amaratunga, G.A.: Ensemble incremental learning ran-
dom vector functional link network for short-term electric load forecasting. Knowl.-
Based Syst. 145, 182–196 (2018)
9. Ahmad, A., Javaid, N., Guizani, M., Alrajeh, N., Khan, Z.A.: An accurate and
fast converging short-term load forecasting model for industrial applications in a
smart grid. IEEE Trans. Industr. Inf. 13(5), 2587–2596 (2017)
10. Tong, C., Li, J., Lang, C., Kong, F., Niu, J., Rodrigues, J.J.: An efficient deep model
for day-ahead electricity load forecasting with stacked denoising auto-encoders. J.
Parallel Distrib. Comput. 117, 267–273 (2018)
11. Yang, Z., Ce, L., Lian, L.: Electricity price forecasting by a hybrid model, combin-
ing wavelet transform, ARMA and kernel-based extreme learning machine meth-
ods. Appl. Energy 190, 291–305 (2017)
12. Abedinia, O., Amjady, N., Zareipour, H.: A new feature selection technique for load
and price forecast of electrical power systems. IEEE Trans. Power Syst. 32(1), 62–
74 (2017)
13. Shi, H., Xu, M., Li, R.: Deep learning for household load forecasting—a novel
pooling deep RNN. IEEE Trans. Smart Grid 9(5), 5271–5280 (2018)
Data Analytics Load and Price Forecasting in Smart Grid 233
14. Wang, J., Liu, F., Song, Y., Zhao, J.: A novel model: dynamic choice artificial
neural network (DCANN) for an electricity price forecasting system. Appl. Soft
Comput. 48, 281–297 (2016)
15. Wang, L., Zhang, Z., Chen, J.: Short-term electricity price forecasting with stacked
denoising autoencoders. IEEE Trans. Power Syst. 32(4), 2673–2681 (2017)
16. Ryu, S., Noh, J., Kim, H.: Deep neural network based demand side short term load
forecasting. Energies 10(1), 3 (2016)
17. Mahajan Shubhrata, D., Deshmukh Kaveri, V., Thite Pranit, R., Samel Bhavana,
Y., Chate, P.J.: Stock market prediction and analysis using Na¨ıvebayes.Int.J.
Recent Innov. Trends Comput. Commun. 4(11), 121–124 (2016)
18. Ahmad, T., Chen, H.: Short and medium-term forecasting of cooling and heating
load demand in building environment with data-mining based approaches. Energy
Build. 166, 460–476 (2018)
19. Hawarah, L., Ploix, S., Jacomino, M.: User behavior prediction in energy consump-
tion in housing using Bayesian networks, pp. 372–379. Springer, Heidelberg (2010).
http://link.springer.com/10.1007/978-3-642-13208-747
20. Dataset. https://www.nyiso.com/public/marketsoperations/marketdata/custom-
report/index.jsp
... For example, the Support vector machine (SVM) technique is one of the best classification models proposed [65]. The decision tree (DT) learning technique and logistic regression approach, are another simple to develop and understand algorithms and have also been extensively modified to find application power grid systems [66][67]. The k-nearest neighbours (KNN) system is one of the fastest algorithms in terms of training data set. ...
Article
Full-text available
Smartgrid is a paradigm that was introduced into the conventional electricity network to enhance the way generation, transmission, and distribution networks interrelate. It involves the use of Information and Communication Technology (ICT) and other solution in fault and intrusion detection, mere monitoring of energy generation, transmission, and distribution. However, on one hand, the actual and earlier smartgrid, do not integrate more advanced features such as automatic decision making, security, scalability, self-healing and awareness, real-time monitoring, cross-layer compatibility, etc. On the other hand, the emergence of the digitalization of the communication infrastructure to support the economic sector which among them are energy generation and distribution grid with Artificial Intelligence (AI) and large-scale Machine to Machine (M2M) communication. With the future Massive Internet of Things (MIoT) as one of the pillars of 5G/6G network factory, it is the enabler to support the next generation smart grid by providing the needed platform that integrates, in addition to the communication infrastructure, the AI and IoT support, providing a multitenant system. This paper aim at presenting a comprehensive review of next smart grid research trends and technological background, discuss a futuristic next-generation smart grid driven by artificial intelligence (AI) and leverage by IoT and 5G. In addition, it discusses the challenges of next-generation smart-grids as it relate to the integration of AI, IoT and 5G for better smart grid architecture. Also, proffers possible solutions to some of the challenges and standards to support this novel trend. A corresponding future work will dwell on the implementation of the discussed integration of AI, IoT and 5G for next-generation smart grid, using Matlab, NS2/NS3, Open-daylight and Mininet as soft tools and compare with related literature.
... For example, the Support vector machine (SVM) technique is one of the best classification models proposed [65]. The decision tree (DT) learning technique and logistic regression approach, are another simple to develop and understand algorithms and have also been extensively modified to find application power grid systems [66][67]. The k-nearest neighbours (KNN) system is one of the fastest algorithms in terms of training data set. ...
Article
Full-text available
Smart grid is a paradigm that was introduced into the conventional electricity network to enhance the way generation, transmission, and distribution networks interrelate. It involves the use of Information and Communication Technology (ICT) and other solution in fault and intrusion detection, mere monitoring of energy generation, transmission, and distribution. However, on one hand, the actual and earlier smart-grid, do not integrate more advanced features such as automatic decision making, security, scalability, self-healing and awareness, real-time monitoring, cross-layer compatibility, etc. On the other hand, the emergence of the digitalization of the communication infrastructure to support the economic sector which among them are energy generation and distribution grid with Artificial Intelligence (AI) and large-scale Machine to Machine (M2M) communication. With the future Massive Internet of Things (MIoT) as one of the pillars of 5G/6G network factory, it is the enabler to support the next generation smart grid by providing the needed platform that integrates, in addition to the communication infrastructure, the AI and IoT support, providing a multitenant system. This paper aim at presenting a comprehensive review of next smart grid research trends and technological background, discuss a futuristic next-generation smart grid driven by artificial intelligence (AI) and leverage by IoT and 5G. In addition, it discusses the challenges of next-generation smart-grids as it relate to the integration of AI, IoT and 5G for better smart grid architecture. Also, proffers possible solutions to some of the challenges and standards to support this novel trend. A corresponding future work will dwell on the implementation of the discussed integration of AI, IoT and 5G for next-generation smart grid, using Matlab, NS2/NS3, Open-daylight and Mininet as soft tools and compare with related literature.
... For example, the Support vector machine (SVM) technique is one of the best classification models proposed [65]. The decision tree (DT) learning technique and logistic regression approach, are another simple to develop and understand algorithms and have also been extensively modified to find application power grid systems [66][67]. The k-nearest neighbours (KNN) system is one of the fastest algorithms in terms of training data set. ...
Article
Full-text available
Smartgrid is a paradigm that was introduced into the conventional electricity network to enhance the way generation, transmission, and distribution networks interrelate. It involves the use of Information and Communication Technology (ICT) and other solution in fault and intrusion detection, mere monitoring of energy generation, transmission, and distribution. However, on one hand, the actual and earlier smartgrid, do not integrate more advanced features such as automatic decision making, security, scalability, self-healing and awareness, real-time monitoring, cross-layer compatibility, etc. On the other hand, the emergence of the digitalization of the communication infrastructure to support the economic sector which among them are energy generation and distribution grid with Artificial Intelligence (AI) and large-scale Machine to Machine (M2M) communication. With the future Massive Internet of Things (MIoT) as one of the pillars of 5G/6G network factory, it is the enabler to support the next generation smart grid by providing the needed platform that integrates, in addition to the communication infrastructure, the AI and IoT support, providing a multitenant system. This paper aim at presenting a comprehensive review of next smart grid research trends and technological background, discuss a futuristic next-generation smart grid driven by artificial intelligence (AI) and leverage by IoT and 5G. In addition, it discusses the challenges of next-generation smart-grids as it relate to the integration of AI, IoT and 5G for better smart grid architecture. Also, proffers possible solutions to some of the challenges and standards to support this novel trend. A corresponding future work will dwell on the implementation of the discussed integration of AI, IoT and 5G for next-generation smart grid, using Matlab, NS2/NS3, Open-daylight and Mininet as soft tools and compare with related literature.
... The k-nearest neighbors (KNN) algorithm, which is very fast for training, is also used for classification and regression in smart grid systems [29][30][31]. The decision tree learning model and logistic regression, which are very easy to interpret and implement, have also been widely adapted in smart gird systems [32,33]. Regression methods-such as linear regression (LR) [34], Gaussian process regression (GPR) [35], support vector regression (SVR) [36], and multivariate adaptive regression spline (MARS) [37,38]-provide solutions for problems with smart gird forecasting, fault detection, demand response, and so on. ...
Article
Full-text available
The smart grid is enabling the collection of massive amounts of high-dimensional and multi-type data about the electric power grid operations, by integrating advanced metering infrastructure, control technologies, and communication technologies. However, the traditional modeling, optimization, and control technologies have many limitations in processing the data; thus, the applications of artificial intelligence (AI) techniques in the smart grid are becoming more apparent. This survey presents a structured review of the existing research into some common AI techniques applied to load forecasting, power grid stability assessment, faults detection, and security problems in the smart grid and power systems. It also provides further research challenges for applying AI technologies to realize truly smart grid systems. Finally, this survey presents opportunities of applying AI to smart grid problems. The paper concludes that the applications of AI techniques can enhance and improve the reliability and resilience of smart grid systems.
Thesis
Full-text available
Smart Grid (SG) is a modernized grid that provides efficient, reliable and economic energy to the consumers. Energy is the most important resource in the world and almost everything relies on it. As smart devices are increasing dramatically with the rapid increase in population, there is a need for an efficient energy distribution mechanism. Furthermore, the forecasting of electricity consumption is supposed to be a major constituent to enhance the performance of the SG. Various learning algorithms have been proposed in the literature for efficient load and price forecasting. However, there exist some issues in the proposed work like increased computational complexity. The sole purpose of the work done in this thesis is to efficiently predict electricity load and price using different techniques with minimum computational complexity. Chapter 1 provides an introduction of various concepts present in the power grids. Afterwards, the unified system model, different sub-problems and the contributions made in the thesis are also presented. Chapter 2 discusses the existing work done by different researchers for performing electricity load and price forecasting. In Chapter 3, Enhanced Logistic Regression (ELR) and Enhanced Recurrent Extreme Learning Machine (ERELM) are proposed for performing short-term load and price forecasting. The former is an enhanced form of Logistic Regression (LR); whereas, the weights and biases of the latter are optimized using Grey Wolf Optimizer (GWO). Classification And Regression Tree (CART), Relief-F and Recursive Feature Elimination (RFE) are used for feature selection and extraction. On the basis of selected features, classification is performed using ELR. Moreover, cross validation is done using Monte Carlo and K-Fold methods. In order to ensure optimal and secure functionality of Micro Grid (MG), Chapter 4 focuses on coordinated energy management of traditional and Renewable Energy Sources (RES). Users and MG with storage capacity are taken into account to perform efficient energy management. A two stage Stackelberg game is formulated. Every player in the game tries to increase its payoff, and ensure user comfort and system reliability. Furthermore, two forecasting techniques are proposed in order to forecast Photo-Voltaic Cell (PVC) generation for announcing optimal prices. Both the existence and uniqueness of Nash Equilibrium (NE) for the energy management algorithm are also considered. In Chapter 5, a novel forecasting model, termed as ELS-net, is proposed. It is a combination of an Ensemble Empirical Mode Decomposition (EEMD) method, multi-model Ensemble Bi Long Short Term Memory (EBiLSTM) forecasting technique and Support Vector Machine (SVM). In the proposed model, EEMD is used to distinguish between linear and non-linear Intrinsic Mode Functions (IMFs). EBiLSTM is used to forecast the non-linear IMFs and SVM is employed to forecast the linear IMFs. The usage of separate forecasting techniques for linear and non-linear IMFs decreases the computational complexity of the model. In Chapter 6, a novel deep learning model, termed as Gated-FCN, is introduced for short-term load forecasting. The key idea is to introduce automated feature selection and a deep learning model for forecasting, which includes an eight layered FCN (FCN-8). It ensures that hand crafted feature selection is avoided as it requires expert domain knowledge. Furthermore, Gated-FCN also helps in reducing noise as it learns internal dependencies as well as the correlation of the time-series. Enhanced Bidirectional Gated Recurrent Unit (EBiGRU) model is dovetailed with FCN-8 in order to learn temporal long-term dependencies of the time-series. Furthermore, weight averaging mechanism of multiple snapshot models is adapted in order to take optimized weights of BiGRU. At the end of FCN-8 and BiGRU, a fully connected dense layer is used that gives final prediction results. The simulations are performed and the results are provided at the end of each chapter. In Chapter 3, the simulations are performed using UMass electric and UCI datasets. ELR shows better performance with the former dataset; whereas, ERELM has better accuracy with the latter. The proposed techniques are then compared with different benchmark schemes. The comparison is done to verify the adaptivity of the proposed techniques. The simulation results show that the proposed techniques outperform the benchmark schemes and increase the prediction accuracy of electricity load and price. Similarly, in Chapter 4, simulations are performed using Elia, Belgium dataset. The results clearly show that the proposed game theoretic approach along with storage capacity optimization and forecasting techniques give benefits to both users and MG. In Chapter 5, simulations are performed to examine the effectiveness of the proposed model using two different datasets: New South Wales (NSW) and Victoria (VIC). From the simulation results, it is obvious that the proposed ELS-net model outperforms the benchmark techniques: EMD-BILSTM-SVM, EMD-PSO-GA-SVR, BiLSTM, MLP and SVM in terms of forecasting accuracy and minimum execution time. Similarly, the simulation results of Chapter 6 depict that Gated-FCN gives maximum forecasting accuracy as compared to the benchmark techniques. For performance evaluation of the proposed work, different performance metrics are used: Mean Absolute Percentage Error (MAPE), Mean Absolute Error (MAE), Mean Squared Error (MSE) and Root Mean Square Error (RMSE). The overall results prove that the work done in this thesis outperforms the existing work in terms of electricity load and price forecasting, and computational complexity.
Article
Full-text available
Electricity price forecast plays a key role in strategic behavior of participants in competitive electricity markets. With the growth of behind-the-meter energy storage, price forecasting becomes important in energy management and control of such small-scale storage systems. In this paper, a forecasting strategy is proposed for real-time electricity markets using publicly available market data. The proposed strategy uses high-resolution data along with hourly data as inputs of two separate forecasting models with different forecast horizons. Moreover, an intra-hour rolling horizon framework is proposed to provide accurate updates on price predictions. The proposed forecasting strategy has the capability to detect price spikes and capture severe price variations. The real data from Ontario’s electricity market is used to evaluate the performance of the proposed forecasting strategy from the statistical point of view. The generated price forecasts are also applied to an optimization platform for operation scheduling of a battery energy storage system within a grid-connected micro-grid in Ontario to show the value of the proposed strategy from an economic perspective.
Article
Full-text available
In the smart grid, one of the most important research areas is load forecasting; it spans from traditional time series analyses to recent machine learning approaches and mostly focuses on forecasting aggregated electricity consumption. However, the importance of demand side energy management, including individual load forecasting, is becoming critical. In this paper, we propose deep neural network (DNN)-based load forecasting models and apply them to a demand side empirical load database. DNNs are trained in two different ways: a pre-training restricted Boltzmann machine and using the rectified linear unit without pre-training. DNN forecasting models are trained by individual customer's electricity consumption data and regional meteorological elements. To verify the performance of DNNs, forecasting results are compared with a shallow neural network (SNN), a double seasonal Holt-Winters (DSHW) model and the autoregressive integrated moving average (ARIMA). The mean absolute percentage error (MAPE) and relative root mean square error (RRMSE) are used for verification. Our results show that DNNs exhibit accurate and robust predictions compared to other forecasting models, e.g., MAPE and RRMSE are reduced by up to 17% and 22% compared to SNN and 9% and 29% compared to DSHW.
Article
The ever-increasing load demand of the residential sector gives rise to concerns such as-decreased quality of service and increased demand-supply gap in the electricity market. To tackle these concerns, the utilities are switching to smart grids (SGs) to manage the demand response (DR) of the connected loads. However, most of the existing DR management schemes have not explored the concept of data analytics for reducing peak load while taking consumer constraints into account. To address this issue, a novel data analytical demand response (DADR) management scheme for residential load is proposed in this paper with an aim to reduce the peak load demand. The proposed scheme is primarily based on the analysis of consumers' consumption data gathered from smart homes (SHs) for which factors such as-appliance adjustment factor, appliance priority index, etc. have been considered. Based on these factors, different algorithms with respect to consumer's and utility's perspective have been proposed to take DR. In addition to it, an incentive scheme has also been presented to increase the consumers' participation in the proposed scheme. The results obtained show that it efficiently reduces the peak load at the grid by a great extent. Moreover, it also increases the savings of the consumers by reducing their overall electricity bills.
Article
This paper depicted the novel data mining based methods that consist of six models for predicting accurate future heating and cooling load demand of water source heat pump, with the objective of enhancing the prediction accuracy and the management of future load. The proposed model was developed to ease generalization to other buildings, by making use of readily available measurements of a comparatively small number of variables related to water source heat pump operation in the building environment. The six models are - Tree Bagger, Gaussian process regression, multiple linear regression, Bagged Tree, Boosted Tree and neural network. The input parameter comprised the prescribed period, external climate data and the diverse load conditions of water source heat pump. The output was electrical power consumption of water source heat pump. In this study, simulations were conducted in three sessions - 7-day, 14-day and 1-month from 8th July to 7th August 2016. The forecast precisions of data mining models were measured by diverse indices. The performance indices which were used in assessing the prediction performance were - mean absolute error, coefficient of correlation, coefficient of variation, root mean square error, mean square error and mean absolute percentage error. The Mean absolute percentage error results for 7-day future energy demand forecasting from Tree Bagger, Gaussian process regression, Bagged Tree, Boosted Tree, neural network and multiple linear regression were 3.544%, 0.405%, 1.703%, 1.928%, 2.592% and 13.053%, respectively. Moreover, when the proposed data mining model performance was compared with the existing studies, the mean absolute percentage error of 2.515% was found out for the first session, 7-day. The results also showed that the six models were efficient in foreseeing the abnormal behavior and future cooling and heating load demand in the building environment.
Article
Short-term electric load forecasting plays an important role in the management of modern power systems. Improving the accuracy and efficiency of electric load forecasting can help power utilities design reasonable operational planning which will lead to the improvement of economic and social benefits of the systems. A hybrid incremental learning approach composed of Discrete Wavelet Transform (DWT), Empirical Mode Decomposition (EMD) and Random Vector Functional Link network (RVFL) is presented in this work. RVFL network is a universal approximator with good efficiency because of the randomly generated weights between input and hidden layers and the close form solution for parameter computation. By introducing incremental learning, along with ensemble approach via DWT and EMD into RVFL network, the forecasting performance can be significantly improved with respect to both efficiency and accuracy. The electric load datasets from Australian Energy Market Operator (AEMO) were used to evaluate the effectiveness of the proposed incremental DWT-EMD based RVFL network. Moreover, the attractiveness of the proposed method can be demonstrated by the comparison with eight benchmark forecasting methods.
Article
As the internet's footprint continues to expand, cybersecurity is becoming a major concern for both governments and the private sector. One such cybersecurity issue relates to data integrity attacks. This paper focuses on the power industry, where the forecasting processes rely heavily on the quality of the data. Data integrity attacks are expected to harm the performances of forecasting systems, which will have a major impact on both the financial bottom line of power companies and the resilience of power grids. This paper reveals the effect of data integrity attacks on the accuracy of four representative load forecasting models (multiple linear regression, support vector regression, artificial neural networks, and fuzzy interaction regression). We begin by simulating some data integrity attacks through the random injection of some multipliers that follow a normal or uniform distribution into the load series. Then, the four aforementioned load forecasting models are used to generate one-year-ahead ex post point forecasts in order to provide a comparison of their forecast errors. The results show that the support vector regression model is most robust, followed closely by the multiple linear regression model, while the fuzzy interaction regression model is the least robust of the four. Nevertheless, all four models fail to provide satisfying forecasts when the scale of the data integrity attacks becomes large. This presents a serious challenge to both load forecasters and the broader forecasting community: the generation of accurate forecasts under data integrity attacks. We construct our case study using the publicly-available data from Global Energy Forecasting Competition 2012. At the end, we also offer an overview of potential research topics for future studies.
Article
Electricity price forecasting is a significant part of smart grid because it makes smart grid cost efficient. Nevertheless, existing methods for price forecasting may be difficult to handle with huge price data in the grid, since the redundancy from feature selection cannot be averted and an integrated infrastructure is also lacked for coordinating the procedures in electricity price forecasting. To solve such a problem, a novel electricity price forecasting model is developed. Specifically, three modules are integrated in the proposed model. First, by merging of Random Forest (RF) and Relief-F algorithm, we propose a hybrid feature selector based on Grey Correlation Analysis (GCA) to eliminate the feature redundancy. Second, an integration of Kernel function and Principle Component Analysis (KPCA) is used in feature extraction process to realize the dimensionality reduction. Finally, to forecast price classification, we put forward a differential evolution (DE) based Support Vector Machine (SVM) classifier. Our proposed electricity price forecasting model is realized via these three parts. Numerical results show that our proposal has superior performance than other methods.
Article
In real word it is quite meaningful to forecast the day-ahead electricity load for an area, which is beneficial to reduction of electricity waste and rational arrangement of electric generator units. The deployment of various sensors strongly pushes this forecasting research into a "big data" era for a huge amount of information has been accumulated. Meanwhile the prosperous development of deep learning (DL) theory provides powerful tools to handle massive data and often outperforms conventional machine learning methods in many traditional fields. Inspired by these, we propose a deep learning based model which firstly refines features by stacked denoising auto-encoders (SDAs) from history electricity load data and related temperature parameters, subsequently trains a support vector regression (SVR) model to forecast the day-ahead total electricity load. The most significant contribution of this heterogeneous deep model is that the abstract features extracted by SADs from original electricity load data are proven to describe and forecast the load tendency more accurately with lower errors. We evaluate this proposed model by comparing with plain SVR and artificial neural networks (ANNs) models, and the experimental results validate its performance improvements.
Article
The key challenge for household load forecasting lies in the high volatility and uncertainty of load profiles. Traditional methods tend to avoid such uncertainty by load aggregation (to offset uncertainties), customer classification (to cluster uncertainties) and spectral analysis (to filter out uncertainties). This paper, for the first time, aims to directly learn the uncertainty by applying a new breed of machine learning algorithms – deep learning. However simply adding layers in neural networks will cap the forecasting performance due to the occurrence of overfitting. A novel pooling-based deep recurrent neural network (PDRNN) is proposed in this paper which batches a group of customers’ load profiles into a pool of inputs. Essentially the model could address the over-fitting issue by increasing data diversity and volume. This work reports the first attempts to develop a bespoke deep learning application for household load forecasting and achieved preliminary success. The developed method is implemented on Tensorflow deep learning platform and tested on 920 smart metered customers from Ireland. Compared with the state-of-art techniques in household load forecasting, the proposed method outperforms ARIMA by 19.5%, SVR by 13.1% and classical deep RNN by 6.5% in terms of RMSE.
Article
Electricity prices have rather complex features such as high volatility, high frequency, nonlinearity, mean reversion and non-stationarity that make forecasting very difficult. However, accurate electricity price forecasting is essential to market traders, retailers, and generation companies. To improve prediction accuracy using each model’s unique features, this paper proposes a hybrid approach that combines the wavelet transform, the kernel extreme learning machine (KELM) based on self-adapting particle swarm optimization and an auto regressive moving average (ARMA). Self-adaptive particle swarm optimization (SAPSO) is adopted to search for the optimal kernel parameters of the KELM. After testing the wavelet decomposition components, stationary series as new input sets are predicted by the ARMA model and non-stationary series are predicted by the SAPSO-KELM model. The performance of the proposed method is evaluated by using electricity price data from the Pennsylvania-New Jersey-Maryland (PJM), Australian and Spanish markets. The experimental results show that the developed method has more accurate prediction, better generality and practicability than individual methods and other hybrid methods.