Conference PaperPDF Available

Electricity Load and Price Forecasting using Machine Learning Algorithms in Smart Grid: A Survey

Authors:

Abstract and Figures

Conventional grid moves towards Smart Grid (SG). In conventional grids, electricity is wasted in generation-transmissions-distribution, and communication is in one direction only. SG is introduced to solve prior issues. In SG, there are no restrictions, and communication is bi-directional. Electricity forecasting plays a significant role in SG to enhance operational cost and efficient management. Load and price forecasting gives future trends. In literature many data-driven methods have been discussed for price and load forecasting. The objective of this paper is to focus on literature related to price and load forecasting in last four years. The author classifies each paper in terms of its problems and solutions. Additionally, the comparison of each proposed technique regarding performance are presented in this paper. Lastly, papers limitations and future challenges are discussed.
Content may be subject to copyright.
Electricity Load and Price Forecasting using
Machine Learning Algorithms in Smart Grid: A
Survey
Arooj Arif, Nadeem Javaid, Mubbashra Anwar, Afrah Naeem, Hira Gul, and
Sahiba Fareed
Abstract Conventional grid moves towards Smart Grid (SG). In conventional grids,
electricity is wasted in generation-transmissions-distribution, and communication
is in one direction only. SG is introduced to solve prior issues. In SG, there are
no restrictions, and communication is bi-directional. Electricity forecasting plays a
significant role in SG to enhance operational cost and efficient management. Load
and price forecasting gives future trends. In literature many data-driven methods
have been discussed for price and load forecasting. The objective of this paper is
to focus on literature related to price and load forecasting in last four years. The
author classifies each paper in terms of its problems and solutions. Additionally, the
comparison of each proposed technique regarding performance are presented in this
paper. Lastly, papers limitations and future challenges are discussed.
1 Introduction
Smart Grid (SG) is a setup that allows the devices to communicate between supplier
and consumer. SG aims to calculate the best generation- transmission-distribution
patterns, save energy and reduce cost [1]. Electricity forecasting plays a significant
role in SG [2].
The most common task in SG is to forecast electricity. Accurate forecasting helps
in rationally scheduling the electric generator, which is beneficial for reducing pro-
duction cost and saving electric power [3].
Due to deregulated electricity market, the dynamics of electricity trade is com-
pletely changing. Electricity encompasses a set of characteristics that are unusual
to alternative markets such as load are closely related to environmental factor like
weather conditions, unexpected price peaks. Electricity has become a central re-
Arooj Arif, Arooj Arif, Nadeem Javaid (Corresponding Author), Mubbashra Anwar, Afrah Naeem,
Hira Gul, Sahiba Fareed
COMSATS University Islamabad, email: nadeemjavaidqau@gmail.com
1
2 Arooj et al.
search area in the energy sector due to its distinctive behaviour. Electricity forecast-
ing is one of the most significant challenges faced by different market participants
and electricity market is build to make grid stable. As a result of uncertain predic-
tion, the stability of grid compromised and increases the risk of blackout [4]. As
Figure 1, shows different steps of forecasting.
Machine Learning (ML) techniques are well-known techniques in electricity
forecasting. Different machine learning algorithms proposed for price and load fore-
casting.
ML seems like a tool to deal with a significant quantity of data generated in an
electric grid. Machine Learning Algorithms (MLAs) gives a systematic way to an-
alyze the data and make appropriate decisions to run the grid. MLA functionalities
include price and load forecasting, fault detection, power generation, future opti-
mum schedule during data breach [5].
MLAs focus on to build programs that learn from experiences. The purpose of
MLA is to provide automatic information learning from raw data that are used in
the decision-making process to implement advanced predictive models [6].
In addition, ML is divided into four categories: Supervised - (SL) algorithm,
Unsupervised-Learning (UL) algorithm, Semi-Supervised- Learning (SSL) algo-
rithm, and Reinforcement Learning (RL) algorithm. SL algorithms train on labeled
data; however, all data is unlabeled in UL. In SSL algorithms, some data is labelled,
and significant data is unlabeled, while RL algorithms learn through delayed feed-
back by interacting with environment [6].
SL classified into two categories: classification and regression. In classification,
output variables are often called labels such as ”drama” or ”movie,” etc. Contrarily,
output variables of regression are real values or floating-point values. These are
often quantities such as amount and size. In simple words, classification predicting
a discrete class and regression predicting a continuous quantity [6].
Clustering, association and anomaly detection are USL techniques. In clustering,
Deep Learning (DL) model identify similarity and group them in the same feature.
In association, USL model identifies the critical feature that is correlated with one
another. Anomaly detection identifies false data by noticing abnormal patterns [6].
SSL applies both on labelled and unlabeled data. Active learning is a type of SSL
in which model is trained with a limited number of labels [6].
RL is an iterative process to predict the best next step to perform task. It is suit-
able for games such a ”packman” or ”tic-tac-toe”, playing a game against an oppo-
nent team or train a computer to drive a car [6].
DL is a subset of MLA. It is a well-known subset of ML due to its unique ability.
In MLA, designers manually adjust, if MLA gives unprecise prediction; however,
DL algorithm automatically concludes if forecast is precise or not [6]. There are
many deep learning models reviewed in literature.
The remainder of the paper is structured as follows: Section 2, related to problems
of each paper is addressed. In Section 3, the papers contribution are summarized. In
Section 4, different performance metrics are mentioned that are used for evaluation.
Limitations in the literature and future challenges are discussed in Section 5.
Electricity Load and Price Forecasting using Machine Learning Algorithms . 3
Fig. 1: Steps of Forecasting
Define the Problem
First you need to do is, find the
problem.What is your exact
problem,that you need to solve.At
the end of this stage, you have all of
the information and context you
need to solve this. problem.
Data Mining
Once you define the problem, then you need
data to turn the problem around with a solution.
You find ways to get that data,
DataClean & Explore
Oftentimes, data has not been
well-maintained. You will see errors
that will corrupt your analysis. When
you normalize your data then you can
start playing with it.
Feature Engineering
Select meaningful feature from raw data.
Feature Selection and Feature Extraction is
done in this phase text here
Predictive Modeling &
Data Visualization
Train machine learning
algorithms.,evaluate the
performance and use them for
predictions.See the results using
plots
2 Problems Addressed
2.1 Load Forecasting
Electricity forecasting plays a vital role in SG. Different forecasting models exist
to forecast electricity load [2]. Very-Short-Term- Load-Forecasting (VSTLF) pre-
dicts the load from minutes to hours. Short-Term-Load-Forecasting (STLF) forecast
the load from hour to atleast one week ahead [7, 8]. Long-Term-Load-Forecasting
(LTLF) estimates year ahead load [7]. Accurate forecasting is very challenging task,
if load is non stationary [8]. In [7], the issue addressed in Korean university build-
ings to forecast the electric load accurately. In Korea, educational buildings have
the highest electric load. It shows diverse energy-consuming patterns depends on the
semester and their diverse holidays. Moreover, load patterns also depend on various
features such as events, weather, and academic schedule. In [3], the problem has
declaimed that some machine learning algorithms have been carried out for load
forecasting algorithms. However, features are directly used to train a model without
preprocessing separately.Day-ahead load forecasting is a common task in electri-
cal grids. Accurate and proper forecasting model is necessary to save energy and
reducing cost. In [9], authors highlight that accurate load demand forecasting is a
challenging task due to some exterior factors which cause data to be unpredictable.
Artificial Neural Network (ANN) has been successfully used for classification and
4 Arooj et al.
regression problems, however, it is often trap in local minima. In [10], authors ad-
dress that each model has its own behaviour, so we can not get the accurate results
on the basis of individual models.
An electricity forecasting method for region and city is a severe problem. As
the development of region and city relies on the betterment of electricity market.
So accurate electric load forecasting model is needed. In [11], authors highlight
that, it is very challenging to deal with nonlinear patterns in load forecasting. Al-
though in Singlehidden-Layer Feedforward-Neural network (SLFN), weights are
traditionally tuned by Back Propagation(BP). Although it is time-consuming and
their gradient boosting is also often trapped in local minima. In [12], authors ad-
dressed that traditional short term load forecasting models are widely used, there
are still some outstanding problems; e.g. it is difficult to simulate the relations be-
tween variables affecting load through mathematical model, forecasting results re-
quire improvements, forecasting results are not reflected in real-time situations. In
[13], authors adressed that electricity load forecasting is very challenging as many
factors affects such as weather conditions and researchers proposed many models
to deal with this problem. In general models are divided into TS (Time Series) mod-
els and AI (Artificial Intelligence) models. Each model has its own pros and cons
and individually these models does not give accurate results. Authors addressed that
many STLF models have been developed for electricity load forecasting as per [14].
They classified STLF models into two sections: traditional models and AI models.
Traditional models are theoratically based on linear definition while AI models have
nonlinear computing capability. However, there is need to combine these models to
achieve higher and satisfied accuracy. In [15], authors highlight that SVR is very
popular in providing satisfactory results; however, SVR suffering from local minima,
when evolutionary algorithms decide its parameters that is the biggest drawback of
SVR. Moreover, robustness does not recieve satisfied level.
2.2 Price Forecasting
In the electric power industry, accurate price forecasting is a valuable tool for mar-
ket participants. Electricity price has many complex features(non-stationary, high-
volatility, price spikes, non-linearity) that makes forecasting very difficult. High-
quality price forecasting plays an essential role in transmission expansion and de-
cisions of power investment [16]. Authors in [17], discussed that the selection of
optimized model is not only challenging to improve the accuracy of price forecast, it
is also concerned problem of electricity market participants due to heavily depend
on electricity. A lot of hybrid models have been developed through different op-
timization techniques; however, researchers aim to determine appropriate features
for accurate electricity forecasting. In [18], authors addressed that accurate price
forecasting is very challenging due to its complex features and price spikes. Many
models for price forecasting exist in literature, and authors highlight two main is-
sues. The first problem is that EMD combine with hybrid model gives the end effect
Electricity Load and Price Forecasting using Machine Learning Algorithms . 5
Table 1: Related Work of Short Term Load Forecasting.
Validation Comparison Techniques
MAPE [3] SVR, ANN
Time Series Cross Validation,
MAPE, MAE, RMSE [7]
TBATS, DT, Multiple Regression,
Gradient Boosting Machine, SVR, ANN
RMSE, MAPE [9] SVR, ANN, DBN, RF, EDBN, EMD-SVR,
EMDSLFN, EMD-ANN, EMD-RF
AE, MAE, RMSE, MAPE [10] MFES, ESPLSSVM, Combined Method
based on ANFIS and NN
MAPE [11] ANN, RF, mRVFL, EMD-ANN, EMD-RF,
EMDRVFL
RMSE, NMSE, MAE, MAPE [12] ARMA, BPNN, KNN
RMSE, MPE, MAE, MAPE [13] ENN, ELM, SVN
RMSE, MAE, MAPE [14] SVR, SVR-PSO, SVR-GA
RMSE, MAE, MAPE ,AIC, BIC [15] AFCM, SVR, SVR-PSO, SVR-GA
which affects the accuracy of EMD. Second, hybrid model can not give accurate
and effective results when combining with linear and non-linear model. In [19], au-
thors highlight that, as the integration of renewable energy increases, volatility in
price also increases and market participants behaviour become more unpredictable.
The electrical grid becomes unstable due to improper balance of generation and
consumption. While there are many price forecasting models; however, the area of
deep learning is still unexplored, and extensive benchmarks are also missing for
predicting electricity prices. Traditionally fundamental models forecast electricity
price by describing demand and supply as accurate as possible as per [20]. Highly
inconsistent electricity generation from Renewable Energy Resources (RES) is a
challenging factor. Fundamental time series are based on assumptions, e.g. auto-
correlation and volatility clustering; however, they ignore the relation of price to
fundamentals. In [21], authors addressed that existing price forecasting methods are
unable to handle a tremendous amount of price data. Feature selection did not help
to remove redundancy.
Furthermore, integrated infrastructure is also lacked for synchronize the methods
of electricity prices. The stability of electricity is discussed in [22], where authors
highlight that electricity price is highly unstable in electricity market due to various
visible, and invisible factors and it is further increased by the deployment of SG. So,
accurate short term forecasting is necessary as it is beneficial both for consumer and
seller. In [23, 24, 25, 26, 27, 28, 29], authors highlight the issues in price and load
forecasting.
6 Arooj et al.
Table 2: Related Work of Short Term Price Forecasting.
Validation Comparison Techniques
MAPE, MAE [17] BPANN, LSSVM
MAPE, RMSE,Theil U statistic1 [18],
Theil U statistic2 WARMA-BP, WARMA-LSSVM,W-ARMA-KELM
sMAPE [19] DNN,GRU,LSTM, Moving Average Models, Machine Learning vs
Statistical Methods, Hybrid Models,TBATS
MAD, RMSE [20] ARIMA, RMA, GARCH (AR-GARCH)
Different values of threshold [21] HFS+DE-SVM, KPCA+DE-SVM, DE-SVM, DT, RF
MAPE[22]
Mixed Model, ARIMA with 2 Variables,Neural Networks,
Weighted Nearest Neighbour, Wavelet-ARIMA with 4 Variables,
Fuzzy Neural Network, Adaptive Wavelet Neural Network
3 Contributions
3.1 Dataset
In [7], KEPCO and KMA dataset is used to evaluate the proposed model. USA
dataset is applied to test the model in paper [3]. In paper [9, 10, 11, 13, 14, 15],
authors used AEMO dataset to evaluate the forecasting model. NEM dataset is used
in paper [12]. In [17], authors put in AEM and PJM dataset for the evaluation of
proposed model. Spanish Market and Australian Market dataset are taken in paper
[18] for evaluation. In paper [19, 20], EPEX dataset is used. ISO-NE dataset is used
in paper [21]. Authors put in Iberian dataset in paper [22]. NYISO dataset is applied
for performance evaluation in [13, 14, 15]. As Figure 2 graphically represent the
datasets.
3.1.1 Critical Analysis
NYISO dataset and AEMO dataset are extensivley used to test the forecasting mod-
els in Section3.1.
3.2 Load Forecasting
In [7], authors come up with short term load forecasting model for the university
campus. They developed a two-stage predictive model based on Simple Moving
Average (SMA) and Random Forecast (RF). They collected two clusters of data.
Electricity Load and Price Forecasting using Machine Learning Algorithms . 7
Fig. 2: Distribution based on Dataset
short term load forecasting
Dataset
Dataset
Dataset
Your text here
Electricity Price and Load
Forecasting
Dataset + Classifier
NYISO
EPEX
ISO-NE
PJM
AEMO
Dataset
Dataset
short term load forecasting short term price forecasting short term price forecasting
short term load forecasting
Hybrid Model (IEMD, ARIMA,
WNN) ,(ARIMA + (RF, SVM,
LOWESS, GLM)),PSRBSK,
EMD-PSO-GA-SVR
Hybrid GRU-DNN, Hybrid
LSTM-DNN,CNN, DNN as an
extension to traditional MLP,ANN
DE-SVM DCANN, updated DCANN
EMD based DBF,
EMD-Mixed-ELM, Hybrid
Approach based on DWT, EMD
and RVFL
The first cluster consists of thirty-two buildings with academic purpose and the sec-
ond cluster consists of sixteen dormitory buildings. Another data also considered
relevant to academic schedule, weather, and calendar. In [3], authors discussed a
framework consists of two modules: feature extraction module and load forecasting
module. Feature extraction module has three subnetworks: the first subnetwork re-
ceives previous day’s electric load, the second subnetwork considers the latest three
days’hourly electric load, and the last subnetwork is designed for weather param-
eters. The extraction of features by using Stack Denoising Autoencoders (SDAs),
were concatenated with a season parameter and then train Support Vector Regres-
sion (SVR) model for electricity forecasting. In [9], authors presented a proce-
dure in which TS is decomposed into several Intrinsic-Mode-Functions (IMFs) and
one residue by Empirical-Mode-Decomposition (EMD) method. For each IMF and
residue, build a training matrix for one Deep Belief Network (DBN). Train DBN
to obtain results for each extracted IMF and residue. The predicted results summed
to formulate final prediction. In [10], authors proposed an STLF model based on
EMD-Mixed-ELM. EMD is used to decompose the load series and to capture the
complicated features and denoise the data, then apply denoise data to mix kernel-
based ELM. Radial-Basis-Function (RBF) and Unscented Kalman Filter (UKF) ker-
nel are used in mixed kernel method. In [11], authors adopted an ensemble approach,
DWT-EMD based incremental Randomized Version Functional Link (RVFL) net-
work. In [12], authors used a novel STLF model based on the Weighted- K-Nearest-
Neighbor (W-K-NN) algorithm to achieve accuracy. The authors proposed a hybrid
8 Arooj et al.
model in paper [13]. The main focus of this paper to propose a robust model that
can be used for different types of hours, days, and different types of markets that
will attract many market participants. In paper [14] authors proposed STLF model
by hybridizing Phase-Space-Reconstruction (PSR) and Bi-Square-Kernel (BSK) re-
gression model. A hybrid EMD-PSO-GA-SVR forecasting model is proposed in
paper [15]. The objective of this model to overcome the drawback of SVR model
that gives poor results on large dataset.
3.3 Price Forecasting
Authors in [17], proposed DCANN and its updated version. It consists of super-
vised and unsupervised components. The supervised component aims to absorb
knowledge from previous data using ANN and OA, while an unsupervised com-
ponent can delete bad samples by building an IBSM. In [18], authors proposed
improved EMD to reduce loss of information and extract better characteristics
of electricity price. Furthermore, an integrated model combines with Exponential
Generalized Autoregressive Conditional Heteroscedasticity (EGARCH), Adaptive
Network-based Fuzzy Inference System (ANFIS), Auto-Regressive Moving Av-
erage with exogenous variables (ARMAX), and Improved Empirical Mode De-
composition (IEMD) is proposed for day-ahead price forecasting. In [19], authors
described a deep learning modeling framework consists of four models: a hybrid
GRU-DNN, hybrid LSTM-DNN, CNN, DNN as an extension to traditional Multi-
Layer Perceptron (MLP). In [20], authors proposed ANN model to forecast elec-
tricity prices. In [21], authors proposed an integrated framework for electricity price
forecasting. The framework consists of Grey Correlation Analysis (GCA) based hy-
brid Feature Selector (HFS), combining RF and ReliefF. KPCA is used for feature
extraction. Differential evolution based SVM also designed to tune parameters of
SVM. In [22], authors proposed ARIMA combine with different machine learning
techniques.
3.4 Feature Selection/Extraction
In [3], authors proposed SDAs for feature extraction module. SDAs consists of au-
toencoders in each layer. The output of each layer is the input of next layer. In
[9, 10, 15], EMD is implemented to extract features. EMD breaking down signal
into IMFs without leaving time domain. EMD is useful for analysing the signal that
is non-stationary. In [11], feature extraction done by DWT-EMD. EMD has mode
mixing problem. DWT decomposes TS signals into frequency components. DWT
solves frequency mixing problem of EMD. In [12], authors calculate Euclidean Dis-
tance for feature selection. In [17], authors select best inputs using Index of Bad-
Sample-Matrix (IBSM). IBSM is used to delete bad training samples. In [18], IEMD
Electricity Load and Price Forecasting using Machine Learning Algorithms . 9
Table 3: Feature Selection and Extraction Techniques.
Dataset FeatureExtraction and Selection
USA [3] SDAs
AEMO [9] EMD
AEMO [10] EMD
AEMO [11] DWT-EMD
NEM [12] Euclidean Distance
NYISO, AEMO [13] IEMD, PC
NYISO, AEMO [14] PSR
NYISO, AEMO [15] EMD
AEM, PJM [17] IBSM
Spanish Market,
Australian Market
[18]
IEMD
EPEX [20] KNN
ISO-NE [21] KPCA, Relief-F, RF
is used to extract linear and non-linear components accurately. In [20], KNN is used
as a filter method, it is not only considered the relationship between input and output
variables. Furthermore, it can identify non-linear relationships in dataset. In [21],
KPCA is implemented for feature extraction to minimize the redundancy among
features. For feature selection, authors proposed GCA based hybrid feature selec-
tion (Relief-F and RF). It is fusion of RF and Relief-F. In [13] IEMD is implemented
to extract features and Pearson’s correlation coefficient (PC) is used to select fea-
tures. Fruit-Fly-Optimization-Algorithm (FOA) is used to optimize the parameters
of Wavelet-Neural- Network (WNN). In [14], PSR algorithm is used to extract the
valuable features to improve forecasting results.
3.4.1 Critical Analysis
EMD is widely used for feature extraction in Section 3.4.
4 Validation
4.1 Load Forecasting
In [7], authors applied Time Series Cross-Validation to evaluate the performance of
the proposed model. Several performance metrics such as Mean Absolute Percent-
age Error (MAPE), Root Mean Square Error (RMSE), and Mean Absolute Error
(MAE) are also used to predict the accuracy of forecasting model. To evaluate model
10 Arooj et al.
in [3], authors used MAPE as a performance indicator. In paper [9], the accuracy of
the model examined via RMSE and MAPE. In [10], four metrics such as Mean
Error (AE), MAE, MAPE, and RMSE are presented. MAPE is applied to evaluate
the performance of proposed model in [11]. Accuracy metrics such as RMSE, Nor-
malized Mean Square Error (NMSE), MAE, and MAPE are used for evaluation in
[12]. In [13], MAPE, RMSE, MAE, and MPE are used for model evaluation. The
results in paper [14], forecasting the errors in terms of MAPE, RMSE and MAE.
In [15], RMSE, MAE, MAPE, AIC, and BI evaluation metrics are used to measure
forecasting errors.
4.2 Price Forecasting
In [17], MAPE and MAE are used to evaluate the performance of model. In [18],
the four evaluation metrics are used to evaluate the performance, which includes
MAPE, RMSE, Theil U statistic1, and Theil U statistic2. In [19], symmetric Mean
Absolute Percentage Error is used to evaluate the performance of forecasters. In
[20], two metrics are used to evaluate the performance, that is MAD and RMSE. In
[21], different values of the threshold are used to control feature selection. In [22],
validation is done through MAPE.
4.2.1 Critical Analysis
MAPE, RMSE, and MAE are widely used as performance metrics in Section 4.
5 Limitations / Open Research Areas
5.1 Load Forecasting
In [7], if the number of decision trees increases in RF, it causes overfitting problem.
RF does not perform well if the number of trees is higher than 128. More trees in RF
require more computational resources. SMA predicts load on previous trends. Al-
though, there is more volatility (abrupt changes) in short time frames as compared to
long time frames. So, it does not show good results in short time frame. In [3], sea-
sonal parameter is directly passed to SVR for training. As we know, load increases in
summer and winter due to excessive use of air conditioning and electric heat appli-
ances. So, there are massive spikes in dataset during summer and winter that affects
the accuracy and gives unfortunate result. It is not very practical to handle extensive
scale data. In [9], Load demand data is decomposed by EMD that is transferred to
DBN. DBN composed of two Restricted Boltzmann Machines’ (RBMs) and one
Electricity Load and Price Forecasting using Machine Learning Algorithms . 11
ANN is applied for load demand forecasting. Although because of its longer com-
putational time some optimisation technique can be applied to increase efficiency of
deep learning model.
In [10], EMD is used to denoise input electrical load signal. This processed data
is then transferred to mixed-Extreme Learning Machine (ELM) model for forecast-
ing. Mixed kernel combines RBF kernel and UKF kernel. However, EMD has mode
mixing problem. Extreme Learning Machine (ELM) performs forecasting in one
go as it is single FFNN, though, it cannot tune parameters effectively. In [11], ap-
ply Discrete Wavelet Transform (DWT) to solve mode mixing problem of EMD
method. It has somewhat solved this issue; however, it takes longer compression
time. Incremental learning of RVFL model is applied to update itself whenever new
input patterns are available. Weights are updated on basis of its error rate. RVFL
has efficient approximation ability despite this; it takes longer computational time.
In [12], we may combine short term with medium-term to detect market demand
shrinking information problem. We also optime the weights to improve forecasting
accuracy by using optimizing approach. In [13], we may use advance models to se-
lect best inputs for electricity load forecasting, and some other influential factors can
also be added in hybrid model as future work. PSR-BSK model proposed in paper
[14] can also be applied in natural gas forecasting to accurate demand forecasting
results for decision-makers.
5.2 Price Forecasting
In [17], the aim of authors, is to select the best input against desired output. How-
ever, they do not prove that the model selects the best input. The computational
time of the model is very high as compared to benchmarks. In this paper, model
does not give accurate results in summer, autumn and spring due to high volatility
in price. If steps are less than 10 in multi-step ahead forecasting, then it shows good
results; otherwise, it gives poor results. In [18], when data will be available in pub-
lic sources for market participants, many other influencing factors can also use to
improve the accuracy of price forecasting. In [19], the proposed model can enhance
with further four advance deep learning models like autoencoders to check the ef-
fect of forecasting accuracy. The individual benchmark combine with expert advice
can also be studied. The effect of model can also be analysed by using a large num-
ber of experiments. In [20], model shows less accuracy on final prices. The model
fails to produce price peaks as high as in reality. It only shows the liability of price
increases.
The applied ANN is calibrated on historical data. If there is any change in mar-
ket design or introduce large amount of new capacity then model should be adjusted
and recalibrated. In [21], for tuning parameters, the authors used Differential Evo-
lution(DE)algorithm. However, DE is prone to converging to local optima and do
not cover the entire search space, the computational time and complexity of the
system model are high as the Random Forest (RF) take much time and harder to
12 Arooj et al.
construct the decision tree. The prediction accuracy is low for RF. The RF when
taking garbage input,its prediction accuracy decreases. In [22], the proposed model
could also investigate on large datasets and see the impact on MAPE values.
6 Conclusion
Load and price forecasting shows future trends. Accurate electricity forecasting is
the key for the secureness of grid. In this paper author focus on literature in last
few years. This paper concerns to provide a review on forecasting in terms of price
and load with different ML approaches. The author classifies the papers regarding
problems, solutions, constraints, and future challenges.The objective of this paper
to provide a survey on price and load and compare their evaluation metrics to check
which approach gives best solution. For future, survey of different deep learning
approaches like CNN, ResNet, LSTM and their computational time need to be con-
sidered, though computational time is critical in deep learning applications.
References
1. Hossain, E., Khan, I., Un-Noor, F., Sikander, S.S. and Sunny, M.S.H., 2019.ApplicationofBig-
DataandMachineLearninginSmartGrid,and Associated Security Concerns: A Review. IEEE
Access, 7, pp.1396013988.
2. Kuster, C., Rezgui, Y. and Mourshed, M., 2017. Electrical load forecasting models: A critical
systematic review. Sustainable cities and society, 35, pp.257-270.
3. Tong, C., Li, J., Lang, C., Kong, F., Niu, J. and Rodrigues, J.J., 2018. An efficient deep model
for day-ahead electricity load forecasting with stacked denoising auto-encoders. Journal of
Parallel and Distributed Computing, 117, pp.267-273.
4. Lago, J., De Ridder, F., Vrancx, P. and De Schutter, B., 2018. Forecasting day-ahead electric-
ity prices in Europe: the importance of considering market integration. Applied Energy, 211,
pp.890-903.
5. Mujeeb, S., Javaid, N. and Javaid, S., 2018, November. Data Analytics for Price Forecasting
in Smart Grids: A Survey. In 2018 IEEE 21st International Multi-Topic Conference (INMIC)
(pp. 1-10). IEEE. DOI: 10.1109/INMIC.2018.8595571.
6. Miraftabzadeh, S.M., Foiadelli, F., Longo, M. and Pasetti, M., 2019, June. A Survey of Ma-
chine Learning Applications for Power System Analytics. In 2019 IEEE International Confer-
ence on Environment and Electrical Engineering and 2019 IEEE Industrial and Commercial
Power Systems Europe (EEEIC/I and CPS Europe) (pp. 1-5). IEEE.
7. Moon, J., Kim, K.H., Kim, Y. and Hwang, E., 2018, January. A short-term electric load fore-
casting scheme using 2-stage predictive analytics. In 2018 IEEE International Conference on
Big Data and Smart Computing (BigComp) (pp. 219-226). IEEE.
8. Zheng, J., Xu, C., Zhang, Z. and Li, X., 2017, March. Electric load forecasting in smart
grids using long-short-term-memory based recurrent neural network. In 2017 51st Annual
Conference on Information Sciences and Systems (CISS) (pp. 1-6). IEEE.
9. Qiu, X., Ren, Y., Suganthan, P.N. and Amaratunga, G.A., 2017. Empirical mode decompo-
sition based ensemble deep learning for load demand time series forecasting. Applied Soft
Computing, 54, pp.246-255.
10. Chen, Y., Kloft, M., Yang, Y., Li, C. and Li, L., 2018. Mixed kernel based extreme learning
machine for electric load forecasting. Neurocomputing, 312, pp.90-106.
Electricity Load and Price Forecasting using Machine Learning Algorithms . 13
11. Qiu, X., Suganthan, P.N. and Amaratunga, G.A., 2018. Ensemble incremental learning ran-
dom vector functional link network for short-term electric load forecasting. Knowledge-
Based Systems, 145, pp.182-196.
12. Fan, G.F., Guo, Y.H., Zheng, J.M. and Hong, W.C., 2019. Application of the Weighted K-
Nearest Neighbor Algorithm for Short-Term Load Forecasting. Energies, 12(5), p.916.
13. Zhang, J., Wei, Y.M., Li, D., Tan, Z. and Zhou, J., 2018. Short term electricity load forecasting
using a hybrid model. Energy, 158, pp.774-781.
14. Fan, G.F., Peng, L.L. and Hong, W.C., 2018. Short term load forecasting based on phase
space reconstruction algorithm and bi-square kernel regression model. Applied energy, 224,
pp.13-33.
15. Fan, G.F., Peng, L.L., Zhao, X. and Hong, W.C., 2017. Applications of hybrid EMD with
PSO and GA for an SVR-based load forecasting model. Energies, 10(11), p.1713.
16. ZhangYang,LiC,LiLian,2017.Electricity price forecasting by a hybrid model, combining
wavelet transform, ARMA and kernel-based extreme learning machine methods.190,pp.291-
305
17. Wang, J., Liu, F., Song, Y. and Zhao, J., 2016. A novel model: Dynamic choice artificial neural
network (DCANN) for an electricity price forecasting system. Applied Soft Computing, 48,
pp.281-297.
18. Zhang, J.L., Zhang, Y.J., Li, D.Z., Tan, Z.F. and Ji, J.F., 2019. Forecasting day-ahead electric-
ity prices using a new integrated model. International Journal of Electrical Power and Energy
Systems, 105, pp.541-548.
19. Lago, J., De Ridder, F. and De Schutter, B., 2018. Forecasting spot electricity prices: Deep
learning approaches and empirical comparison of traditional algorithms. Applied Energy, 221,
pp.386-405.
20. Keles, D., Scelle, J., Paraschiv, F. and Fichtner, W., 2016. Extended forecast methods for
day-ahead electricity spot prices applying artificial neural networks. Applied energy, 162,
pp.218-230.
21. Wang, K., Xu, C., Zhang, Y., Guo, S. and Zomaya, A.Y., 2017. Robust big data analytics for
electricity price forecasting in the smart grid. IEEE Transactions on Big Data, 5(1), pp.34-45.
22. Angamuthu Chinnathambi, R., Mukherjee, A., Campion, M., Salehfar, H., Hansen, T.M., Lin,
J. and Ranganathan, P., 2019. A multi-stage price forecasting model for day-ahead electricity
markets. Forecasting, 1(1), pp.26-46.
23. Samuel, O., Alzahrani, F.A., Hussen Khan, R.J.U., Farooq, H., Shafiq, M., Afzal, M.K. and
Javaid, N., 2020. Towards Modified Entropy Mutual Information Feature Selection to Fore-
cast Medium-Term Load Using a Deep Learning Model in Smart Homes. Entropy, 22(1),
p.68.
24. Khalid, R., Javaid, N., Al-zahrani, F.A., Aurangzeb, K., Qazi, E.U.H. and Ashfaq, T., 2020.
Electricity Load and Price Forecasting Using Jaya-Long Short Term Memory (JLSTM) in
Smart Grids. Entropy, 22(1), p.10.
25. Mujeeb, S. and Javaid, N., 2019. ESAENARX and DE-RELM: Novel schemes for big data
predictive analytics of electricity load and price. Sustainable Cities and Society, 51, p.101642.
26. Mujeeb, S., Alghamdi, T.A., Ullah, S., Fatima, A., Javaid, N. and Saba, T., 2019. Exploiting
Deep Learning for Wind Power Forecasting Based on Big Data Analytics. Applied Sciences,
9(20), p.4417.
27. Naz, A., Javaid, N., Rasheed, M.B., Haseeb, A., Alhussein, M. and Aurangzeb, K., 2019.
Game Theoretical Energy Management with Storage Capacity Optimization and Photo-
Voltaic Cell Generated Power Forecasting in Micro Grid. Sustainability, 11(10), p.2763.
28. Naz, A., Javed, M.U., Javaid, N., Saba, T., Alhussein, M. and Aurangzeb, K., 2019. Short-
term electric load and price forecasting using enhanced extreme learning machine optimiza-
tion in smart grids. Energies, 12(5), p.866.
29. Mujeeb, S., Javaid, N., Ilahi, M., Wadud, Z., Ishmanov, F. and Afzal, M.K., 2019. Deep long
short-term memory: A new price and load forecasting scheme for big data in smart cities.
Sustainability, 11(4), p.987.
... Conventionally, smart meter data are transferred to a data center or another centralized system for storage and for training ML models. There are two main categories of these centralized solutions: the first category trains a single model for each smart meter and, thus, generates individual forecasts for each meter, while the second category trains a machine learning model with the aggregated data from several smart meters and, hence, provides aggregated load forecasts [6]. Although both of these centralized solutions have shown great results, they require transferring all data to a centralized location, which results in significant network traffic [7]. ...
... Various load forecasting techniques have been developed over the years [6], but here we focus on machine-learning techniques as they have been dominant approaches in sensorbased forecasting [6]. Grolinger et al. [27] introduced an approach based on Support Vector Regression (SVR) and local learning for load forecasting with big data. ...
... Various load forecasting techniques have been developed over the years [6], but here we focus on machine-learning techniques as they have been dominant approaches in sensorbased forecasting [6]. Grolinger et al. [27] introduced an approach based on Support Vector Regression (SVR) and local learning for load forecasting with big data. ...
Article
Full-text available
Load forecasting is essential for energy management, infrastructure planning, grid operation, and budgeting. Large scale smart meter deployments have resulted in ability to collect massive energy data and have created opportunities in sensor-based forecasting. Machine learning (ML) has demonstrated great successes in sensor-based load forecasting; however, when prediction is needed on a smart meter level, typically a single model is trained for each smart meter. With a large number of meters, this becomes computationally expensive or even infeasible. On the other hand, with conventional ML, training a single model for several smart meters requires participants to share their data with the central server. Consequently, this paper proposes federated learning for load forecasting with smart meter data: this strategy enables training a single model with all participating smart meters without the need to share local data. Two alternative federated learning strategies are examined: FedSGD, which performs one step of gradient descent on client before merging updates on the server, and FedAVG, which carries out several steps before the merging. Specifically, residential consumers are diverse what makes training a single model challenging as load profiles vary across consumers. The results show that the FedAVG achieves better accuracy than FedSGD while also requiring fewer communication rounds. Comparing to individual models for each meter and a single central models for all meters, FedAVG achieves comparable or better accuracy.
... In recent years, NNs have been widely applied to obtain latent information to build prediction models, e.g., dynamic choice artificial neural network model [21], generalized regression neural network [22], and nonlinear autoregressive neural network models with exogenous input (NARX) [23]. Nevertheless, NNs are notorious for overfitting and are also suffered from local optima in the backpropagation [24]. To mitigate the overfitting problem by raising a new data dimension, Shi et al. [25] proposed to use a pooling-based deep recurrent neural network for short-term household load forecasting. ...
Preprint
New micro-grid design with renewable energy sources and battery storage systems can help improve greenhouse gas emissions and reduce the operational cost. To provide an effective short-/long-term forecasting of both energy generation and load demand, time series predictive modeling has been one of the key tools to guide the optimal decision-making for planning and operation. One of the critical challenges of time series renewable energy forecasting is the lack of historical data to train an adequate predictive model. Moreover, the performance of a machine learning model is sensitive to the choice of its corresponding hyperparameters. Bearing these considerations in mind, this paper develops a BiLO-Auto-TSF/ML framework that automates the optimal design of a few-shot learning pipeline from a bi-level programming perspective. Specifically, the lower-level meta-learning helps boost the base-learner to mitigate the small data challenge while the hyperparameter optimization at the upper level proactively searches for the optimal hyperparameter configurations for both base- and meta-learners. Note that the proposed framework is so general that any off-the-shelf machine learning method can be used in a plug-in manner. Comprehensive experiments fully demonstrate the effectiveness of our proposed BiLO-Auto-TSF/ML framework to search for a high-performance few-shot learning pipeline for various energy sources.
... In recent times, DNNs have found application in obtaining the possible knowledge for a predicting paradigm. However, the ANN technique is habitually stuck in local minima [115] and this hence fails to forecast future load consistently. Hence [116] proposed a deep RNN for STLF to tackle the over-fitting problem by enhancing volume and data multiplicity in the smart grid. ...
Article
Full-text available
Smartgrid is a paradigm that was introduced into the conventional electricity network to enhance the way generation, transmission, and distribution networks interrelate. It involves the use of Information and Communication Technology (ICT) and other solution in fault and intrusion detection, mere monitoring of energy generation, transmission, and distribution. However, on one hand, the actual and earlier smartgrid, do not integrate more advanced features such as automatic decision making, security, scalability, self-healing and awareness, real-time monitoring, cross-layer compatibility, etc. On the other hand, the emergence of the digitalization of the communication infrastructure to support the economic sector which among them are energy generation and distribution grid with Artificial Intelligence (AI) and large-scale Machine to Machine (M2M) communication. With the future Massive Internet of Things (MIoT) as one of the pillars of 5G/6G network factory, it is the enabler to support the next generation smart grid by providing the needed platform that integrates, in addition to the communication infrastructure, the AI and IoT support, providing a multitenant system. This paper aim at presenting a comprehensive review of next smart grid research trends and technological background, discuss a futuristic next-generation smart grid driven by artificial intelligence (AI) and leverage by IoT and 5G. In addition, it discusses the challenges of next-generation smart-grids as it relate to the integration of AI, IoT and 5G for better smart grid architecture. Also, proffers possible solutions to some of the challenges and standards to support this novel trend. A corresponding future work will dwell on the implementation of the discussed integration of AI, IoT and 5G for next-generation smart grid, using Matlab, NS2/NS3, Open-daylight and Mininet as soft tools and compare with related literature.
... In recent times, DNNs have found application in obtaining the possible knowledge for a predicting paradigm. However, the ANN technique is habitually stuck in local minima [115] and this hence fails to forecast future load consistently. Hence [116] proposed a deep RNN for STLF to tackle the over-fitting problem by enhancing volume and data multiplicity in the smart grid. ...
Article
Full-text available
Smart grid is a paradigm that was introduced into the conventional electricity network to enhance the way generation, transmission, and distribution networks interrelate. It involves the use of Information and Communication Technology (ICT) and other solution in fault and intrusion detection, mere monitoring of energy generation, transmission, and distribution. However, on one hand, the actual and earlier smart-grid, do not integrate more advanced features such as automatic decision making, security, scalability, self-healing and awareness, real-time monitoring, cross-layer compatibility, etc. On the other hand, the emergence of the digitalization of the communication infrastructure to support the economic sector which among them are energy generation and distribution grid with Artificial Intelligence (AI) and large-scale Machine to Machine (M2M) communication. With the future Massive Internet of Things (MIoT) as one of the pillars of 5G/6G network factory, it is the enabler to support the next generation smart grid by providing the needed platform that integrates, in addition to the communication infrastructure, the AI and IoT support, providing a multitenant system. This paper aim at presenting a comprehensive review of next smart grid research trends and technological background, discuss a futuristic next-generation smart grid driven by artificial intelligence (AI) and leverage by IoT and 5G. In addition, it discusses the challenges of next-generation smart-grids as it relate to the integration of AI, IoT and 5G for better smart grid architecture. Also, proffers possible solutions to some of the challenges and standards to support this novel trend. A corresponding future work will dwell on the implementation of the discussed integration of AI, IoT and 5G for next-generation smart grid, using Matlab, NS2/NS3, Open-daylight and Mininet as soft tools and compare with related literature.
... In recent times, DNNs have found application in obtaining the possible knowledge for a predicting paradigm. However, the ANN technique is habitually stuck in local minima [115] and this hence fails to forecast future load consistently. Hence [116] proposed a deep RNN for STLF to tackle the over-fitting problem by enhancing volume and data multiplicity in the smart grid. ...
Article
Full-text available
Smartgrid is a paradigm that was introduced into the conventional electricity network to enhance the way generation, transmission, and distribution networks interrelate. It involves the use of Information and Communication Technology (ICT) and other solution in fault and intrusion detection, mere monitoring of energy generation, transmission, and distribution. However, on one hand, the actual and earlier smartgrid, do not integrate more advanced features such as automatic decision making, security, scalability, self-healing and awareness, real-time monitoring, cross-layer compatibility, etc. On the other hand, the emergence of the digitalization of the communication infrastructure to support the economic sector which among them are energy generation and distribution grid with Artificial Intelligence (AI) and large-scale Machine to Machine (M2M) communication. With the future Massive Internet of Things (MIoT) as one of the pillars of 5G/6G network factory, it is the enabler to support the next generation smart grid by providing the needed platform that integrates, in addition to the communication infrastructure, the AI and IoT support, providing a multitenant system. This paper aim at presenting a comprehensive review of next smart grid research trends and technological background, discuss a futuristic next-generation smart grid driven by artificial intelligence (AI) and leverage by IoT and 5G. In addition, it discusses the challenges of next-generation smart-grids as it relate to the integration of AI, IoT and 5G for better smart grid architecture. Also, proffers possible solutions to some of the challenges and standards to support this novel trend. A corresponding future work will dwell on the implementation of the discussed integration of AI, IoT and 5G for next-generation smart grid, using Matlab, NS2/NS3, Open-daylight and Mininet as soft tools and compare with related literature.
... In recent years, DNNs have been used to obtain the potential knowledge for a forecasting model. However, the ANN method is often trapped in local minima [77] and over-fitting problems. Shi et al. [78] proposed a pooling-based deep RNN for STLF to address the over-fitting issue by increasing data diversity and volume. ...
Article
Full-text available
The smart grid is enabling the collection of massive amounts of high-dimensional and multi-type data about the electric power grid operations, by integrating advanced metering infrastructure, control technologies, and communication technologies. However, the traditional modeling, optimization, and control technologies have many limitations in processing the data; thus, the applications of artificial intelligence (AI) techniques in the smart grid are becoming more apparent. This survey presents a structured review of the existing research into some common AI techniques applied to load forecasting, power grid stability assessment, faults detection, and security problems in the smart grid and power systems. It also provides further research challenges for applying AI technologies to realize truly smart grid systems. Finally, this survey presents opportunities of applying AI to smart grid problems. The paper concludes that the applications of AI techniques can enhance and improve the reliability and resilience of smart grid systems.
... Forecasting of electricity plays an important role in the smart grid to minimize operational costs and effective management. Load and price prediction provide future trends (Arif et al., 2020). The latest developments in information technologies and the accessibility of bulky and varied data in power grids are opening the ways for the deployment of intelligent algorithms. ...
Article
Full-text available
A smart city is an idea that is realized by the computing of a large amount of data collected through sensors, cameras, and other electronic methods to provide services, manage resources and solve daily life problems. The transformation of the conventional grid to a smart grid is one step in the direction towards smart city realization. An electric grid is composed of control stations, generation centres, transformers, communication lines, and distributors, which helps in transferring power from the power station to domestic and commercial consumers. Present electric grids are not smart enough that they can estimate the varying power requirement of the consumer. Also, these conventional grids are not enough robust and scalable. This has become the motivation for shifting from a conventional grid to a smart grid. The smart grid is a kind of power grid, which is robust and adapts itself to the varying needs of the consumer and self‐healing in nature. In this way, the transformation from a conventional grid to a smart grid will help the government to make a smart city. The emergence of machine learning has helped in the prediction of the stability of the grid under the dynamically changing requirement of the consumer. Also, the usage of a variety of sensors will help in the collection of real‐time consumption data. Through machine learning algorithms, we can gain an insight view of the collected data. This has helped the smart grid to convert into a robust smart grid, as this will help in avoiding the situation of failure. In this work, the authors have applied logistic regression, decision tree, support vector machine, linear discriminant analysis, quadratic discriminant analysis, naïve Bayes, random forest, and k‐nearest neighbour algorithms to predict the stability of the grid. The authors have used the smart grid stability dataset freely available on Kaggle to train and test the models. It has been found that a model designed using the support vector machine algorithm has given the most accurate result.
... Furthermore, Rahmat et al. [40] developed an RNN based model for a residential and commercial building to forecast the energy demand and compared it with the ANN model to show their effectiveness. Hence, researchers have focused on the sequential model and [41,42] proposed the energy forecasting systems by utilizing LSTM and GRU networks. The outcomes of these models were better than RNN and other sequential models in time-series and forecasting problems. ...
Article
Full-text available
Industrial and building sectors demand efficient smart energy strategies, techniques of optimization, and efficient management for reducing global energy consumption due to the increasing world population. Nowadays, various artificial intelligence (AI) based methods are utilized to perform optimal energy forecasting, different simulation tools, and engineering methods to predict future demand based on historical data. Nevertheless, nonlinear energy demand modeling is still unfledged for a better solution to handle short-term and long-term dependencies and avoid static nature because it is purely on historical data-driven. In this paper, we propose an ensemble deep learning-based approach to predict and forecast energy demand and consumption by using chronological dependencies. Our system initially processes the data, cleaning, normalization, and transformation to ensure the model performance. Furthermore, the preprocess data is fed to propose the ensemble model to extract hybrid discriminative features by using convolution neural network (CNN), stacked, and bi-directional long-short term memory (LSTM) architectures. We trained our proposed system on the historical data to forecast the energy demand and consumption with a different time interval. In the proposed technique, we utilized the concept of active learning based on moving windows to ensure and improve the prediction performance of the system. The proposed system could be applicable to employ energy consumption in industrial and building sectors to demonstrate their significance and effectiveness. We evaluated the proposed system by using benchmark, residential UCI, and local Korean commercial building datasets. We conducted different extensive experimentation to show the error rate and used various kinds of evaluation matrices, which indicate the lower error rate of the proposed system.
... It includes AI methods. AI methods categorized into machine learning [39,40,41,42,43,44] and DL algorithms [14,25,26,45]. Machine learning methods are impressive, though most of the methods involve manual feature extraction due to the limited ability to manage a large set of features. ...
Thesis
Electricity theft (ET) is a major problem in developing countries. It a�ects the economy that causes revenue loss. It also decreases the reliability and stability of electricity utilities. Due to these losses, the quality of supply e�ects and tari � imposed on legitimate consumers. ET is an essential part of Non-technical loss (NTL) and it is challenging for electricity utilities to �nd the responsible people. Several methodologies have developed to identify ET behaviors automatically. However, these approaches mainly assess records of consumers' electricity usage, may prove inadequate in detecting ET due to a variety of theft attacks and irregularity of consumers' behavior. Moreover, some important challenges are needed to be addressed. (i) The number of normal consumers has been wrongly identi�ed as fraudulent. This leads to high False-positive rate (FPR). After the detection of theft, on-site inspection is needed to validate the detected person, either is it fraudulent or not and it is costly. (ii) The imbalanced nature of datasets which negatively a�ect on the model's performance. (iii) The problem of over�tting and generalization error is often faced in deep learning models, predicts unseen data inaccurately. So, the motivation for this work to detect illegal consumers accurately. We have proposed four Arti�cial intelligence (AI) models in this thesis. In system model 1, we have proposed Enhanced arti�cial neural network blocks with skip connections (EANNBS). It makes training easier, reduces over�tting, FPR and generalization error, as well as execution time. Temporal convolutional network with enhanced multi-layer perceptron (TCN-EMLP) is proposed in system model 2. It analyzes the sequential data based on daily electricity-usage records, obtained from smart meters. At the same time, EMLP integrates the non-sequential auxiliary data, such as data related to electrical connection type, property area, electrical appliances usage, etc. System model 3 based on Residual network (RN) that is used to automate feature extraction while three tree-based classi�ers such as Decision tree (DT), Random forest (RF) and Adaptive boosting (AdaBoost) are trained on the obtained features for classi�cation. Hyperparameter tuning toolkit is presented in this system model, named as Hyperactive optimization toolkit. Bayesian is used as an optimizer in this toolkit that aims to simplify the tuning process of DT, RF and AdaBoost. In system model 4, input is forwarded to three di�erent and well-known Machine learning (ML) techniques, i.e., Support vector machine (SVM), as an input. At this stage, a meta-heuristic algorithm named Simulated annealing (SA) is employed to acquire optimal values for ML models' hyperparameters. Finally, ML models' outputs are used as features for meta-classi�ers to achieve �nal classi�cation with Light Gradient boosting machine (LGBM) and Multi-layer perceptron (MLP). Furthermore, Pakistan residential electricity consumption dataset (PRECON1), State grid corporation of china (SGCC2) and Commission for energy regulation (CER3) datasets is used in this thesis. SGCC dataset contains 9% fraudulent consumers, which are extremely less than non-fraudulent consumers, due to the imbalance nature of data. Furthermore, many classi�cation techniques have poor predictive class accuracy for the positive class. These techniques mainly focus on minimizing the error rate while ignoring the minority class. Many re-sampling techniques are used in literature to adjust the class ratio; however, sometimes, these techniques remove the important information that is necessary to learn the model and cause over�tting. By using six previous theft attacks, we generate theft cases to mimic the real world theft attacks in original data. We have proposed the combination of oversampling and under-sampling techniques that is Near miss borderline synthetic minority oversampling technique (NMB-SMOTE), Tomek link borderline synthetic minority oversampling technique with support vector machine (TBSSVM) and Synthetic minority oversampling technique with near miss (SMOTE-NM) to handle imbalanced classi�cation problem. We have conducted a comprehensive experiment using SGCC, CER and PRECON datasets. The performance of suggested model is validated using di�erent performance metrics that are derived from Confusion matrix (CM).
Article
Full-text available
Over the last decades, load forecasting is used by power companies to balance energy demand and supply. Among the several load forecasting methods, medium-term load forecasting is necessary for grid’s maintenance planning, settings of electricity prices, and harmonizing energy sharing arrangement. The forecasting of the month ahead electrical loads provides the information required for the interchange of energy among power companies. For accurate load forecasting, this paper proposes a model for medium-term load forecasting that uses hourly electrical load and temperature data to predict month ahead hourly electrical loads. For data preprocessing, modified entropy mutual information-based feature selection is used. It eliminates the redundancy and irrelevancy of features from the data. We employ the conditional restricted Boltzmann machine (CRBM) for the load forecasting. A meta-heuristic optimization algorithm Jaya is used to improve the CRBM’s accuracy rate and convergence. In addition, the consumers’ dynamic consumption behaviors are also investigated using a discrete-time Markov chain and an adaptive k-means is used to group their behaviors into clusters. We evaluated the proposed model using GEFCom2012 US utility dataset. Simulation results confirm that the proposed model achieves better accuracy, fast convergence, and low execution time as compared to other existing models in the literature.
Article
Full-text available
Recently, power systems are facing the challenges of growing power demand, depleting fossil fuel and aggravating environmental pollution (caused by carbon emission from fossil fuel based power generation). The incorporation of alternative low carbon energy generation, i.e., Renewable Energy Sources (RESs), become crucial for energy systems. Effective Demand Side Management (DSM) and RES incorporation enable power systems to maintain demand, supply balance and optimize energy in an environment friendly manner. The wind power is a popular energy source because of its environmental and economical benefits. However, the uncertainty of wind power, makes its incorporation in energy systems really difficult. To mitigate the risk of demand-supply imbalance, an accurate estimation of wind power is essential. Recognizing this challenging task, an efficient deep learning based prediction model is proposed for wind power forecasting. The proposed model has two stages. In the first stage, Wavelet Packet Transform (WPT) is used to decompose the past wind power signals. Other than decomposed signals and lagged wind power, multiple exogenous inputs (such as, calendar variable and Numerical Weather Prediction (NWP)) are also used as input to forecast wind power. In the second stage, a new prediction model Efficient Deep Convolution Neural Network (EDCNN) is employed to forecast wind power. A DSM scheme is formulated based on forecasted wind power, day-ahead demand and price. The proposed forecasting model's performance is evaluated on big data of Maine wind farm ISO NE, USA.
Conference Paper
Full-text available
Recent advances in computing technologies and the availability of large amounts of heterogeneous data in power grids are opening the way for the application of state-of-art machine learning techniques. Compared to traditional computational approaches, machine learning algorithms could gain an advantage from their intrinsic generalization capability, by also providing accurate short-term power flow forecasts from distributed measurement units, with greater computational efficiency and scalability. Several studies in the literature investigated the use of suitable machine learning models to address different issues in the field of power grid operation and management. Furthermore, the ongoing transition towards smart grids is generating new research opportunities for the real-time application of machine learning algorithms in power systems. In this paper, a literature survey on the application of different machine learning techniques in power systems is presented and critically reviewed, by evaluating the main advantages and criticalities of each technique for the selected applications.
Article
Full-text available
In order to ensure optimal and secure functionality of Micro Grid (MG), energy management system plays vital role in managing multiple electrical load and distributed energy technologies. With the evolution of Smart Grids (SG), energy generation system that includes renewable resources is introduced in MG. This work focuses on coordinated energy management of traditional and renewable resources. Users and MG with storage capacity is taken into account to perform energy management efficiently. First of all, two stage Stackelberg game is formulated. Every player in game theory tries to increase its payoff and also ensures user comfort and system reliability. In the next step, two forecasting techniques are proposed in order to forecast Photo Voltaic Cell (PVC) generation for announcing optimal prices. Furthermore, existence and uniqueness of Nash Equilibrium (NE) of energy management algorithm are also proved. In simulation, results clearly show that proposed game theoretic approach along with storage capacity optimization and forecasting techniques give benefit to both players, i.e., users and MG. The proposed technique Gray wolf optimized Auto Regressive Integrated Moving Average (GARIMA) gives 40% better result and Cuckoo Search Auto Regressive Integrated Moving Average (CARIMA) gives 30% better results as compared to existing techniques.
Article
Full-text available
In this paper, the historical power load data from the National Electricity Market (Australia) is used to analyze the characteristics and regulations of electricity (the average value of every eight hours). Then, considering the inverse of Euclidean distance as the weight, this paper proposes a novel short-term load forecasting model based on the weighted k-nearest neighbor algorithm to receive higher satisfied accuracy. In addition, the forecasting errors are compared with the back-propagation neural network model and the autoregressive moving average model. The comparison results demonstrate that the proposed forecasting model could reflect variation trend and has good fitting ability in short-term load forecasting.
Article
Full-text available
A Smart Grid (SG) is a modernized grid to provide efficient, reliable and economic energy to the consumers. Energy is the most important resource in the world. An efficient energy distribution is required as smart devices are increasing dramatically. The forecasting of electricity consumption is supposed to be a major constituent to enhance the performance of SG. Various learning algorithms have been proposed to solve the forecasting problem. The sole purpose of this work is to predict the price and load efficiently. The first technique is Enhanced Logistic Regression (ELR) and the second technique is Enhanced Recurrent Extreme Learning Machine (ERELM). ELR is an enhanced form of Logistic Regression (LR), whereas, ERELM optimizes weights and biases using a Grey Wolf Optimizer (GWO). Classification and Regression Tree (CART), Relief-F and Recursive Feature Elimination (RFE) are used for feature selection and extraction. On the basis of selected features, classification is performed using ELR. Cross validation is done for ERELM using Monte Carlo and K-Fold methods. The simulations are performed on two different datasets. The first dataset, i.e., UMass Electric Dataset is multi-variate while the second dataset, i.e., UCI Dataset is uni-variate. The first proposed model performed better with UMass Electric Dataset than UCI Dataset and the accuracy of second model is better with UCI than UMass. The prediction accuracy is analyzed on the basis of four different performance metrics: Mean Absolute Percentage Error (MAPE), Mean Absolute Error (MAE), Mean Square Error (MSE) and Root Mean Square Error (RMSE). The proposed techniques are then compared with four benchmark schemes. The comparison is done to verify the adaptivity of the proposed techniques. The simulation results show that the proposed techniques outperformed benchmark schemes. The proposed techniques efficiently increased the prediction accuracy of load and price. However, the computational time is increased in both scenarios. ELR achieved almost 5% better results than Convolutional Neural Network (CNN) and almost 3% than LR. While, ERELM achieved almost 6% better results than ELM and almost 5% than RELM. However, the computational time is almost 20% increased with ELR and 50% with ERELM. Scalability is also addressed for the proposed techniques using half-yearly and yearly datasets. Simulation results show that ELR gives 5% better results while, ERELM gives 6% better results when used for yearly dataset.
Article
Full-text available
This paper focuses on analytics of an extremely large dataset of smart grid electricity price and load, which is difficult to process with conventional computational models. These data are known as energy big data. The analysis of big data divulges the deeper insights that help experts in the improvement of smart grid’s (SG) operations. Processing and extracting of meaningful information from data is a challenging task. Electricity load and price are the most influential factors in the electricity market. For improving reliability, control and management of electricity market operations, an exact estimate of the day ahead load is a substantial requirement. Energy market trade is based on price. Accurate price forecast enables energy market participants to make effective and most profitable bidding strategies. This paper proposes a deep learning-based model for the forecast of price and demand for big data using Deep Long Short-Term Memory (DLSTM). Due to the adaptive and automatic feature learning mechanism of Deep Neural Network (DNN), the processing of big data is easier with LSTM as compared to the purely data-driven methods. The proposed model was evaluated using well-known real electricity markets’ data. In this study, day and week ahead forecasting experiments were conducted for all months. Forecast performance was assessed using Mean Absolute Error (MAE) and Normalized Root Mean Square Error (NRMSE). The proposed Deep LSTM (DLSTM) method was compared to traditional Artificial Neural Network (ANN) time series forecasting methods, i.e., Nonlinear Autoregressive network with Exogenous variables (NARX) and Extreme Learning Machine (ELM). DLSTM outperformed the compared forecasting methods in terms of accuracy. Experimental results prove the efficiency of the proposed method for electricity price and load forecasting.
Article
Full-text available
This paper conducts a comprehensive study on the application of big data and machine learning in the electrical power grid, introduced through the emergence of the next-generation power system – the smart grid (SG). Connectivity lies at the core of this new grid infrastructure, which is provided by the internet of things (IoT). This connectivity, and constant communication required in this system, also introduced a massive data volume that demands techniques far superior than conventional methods for proper analysis and decision-making. IoT integrated SG system can provide efficient load forecasting and data acquisition technique, along with cost effectiveness. Big data analysis and machine learning techniques are essential to reap these benefits. In the complex connected system of SG, cyber security becomes a critical issue; IoT devices and their data turning into major targets of attacks. Such security concerns and their solutions are also included in this study. Key information obtained through literature review are tabulated in corresponding sections to provide a clear synopsis; and the findings of this rigorous review are listed to give a concise picture of this area of study and promising future fields of academic and industrial research, with current limitations with viable solutions along with their effectiveness.
Conference Paper
Electricity Price Forecasting (EPF) plays a significant role in competitive electricity markets. Market participants rely on price forecast for generation, assets scheduling and effective bidding plan formulation. The uncertainty and volatility of energy market makes price forecasting a very challenging task. This paper gives a survey on methods and techniques used in literature for price forecasting in last four years. This work provides general background of price forecasting and brief discussion of the models and approaches implemented in the area of price forecasting. A comparison of price forecasting techniques with respect to different categories is presented that draws out the important aspects of the implemented methodologies.
Article
Electricity price forecasting proves useful for power producers and consumers to make proper decisions in a market-oriented environment. However, due to the complex drivers and sharp fluctuation of electricity prices, accurate electricity price forecasting turns to be very difficult. To better capture the characteristics of day-ahead electricity prices, a new integrated model based on the improved empirical mode decomposition (IEMD), autoregressive moving average with exogenous terms (ARMAX), exponential generalized autoregressive conditional heteroscedasticity (EGARCH) and adaptive network-based fuzzy inference system (ANFIS) is proposed in this paper. Then it is validated by using the data from Spanish and Australian electricity markets. The results indicate that the forecasting accuracy of the new integrated model proves higher than that of some well-recognized models in the literature.