Conference PaperPDF Available

Electricity Price and Load Forecasting using Data Analytics in Smart Grid : A Survey


Abstract and Figures

Smart grid (SG) is bringing revolutionary changes in the electric power system. SG is supposed to provide economic, social, and environmental benefits for many stakeholders. A smart meter is an essential part of the SG. Data acquisition, transmission, processing, and interpretation are factors to determine the success of smart meters due to the excess amount of data in the grid. Electricity price and load are considered the most influential factors in the energy management system. Moreover, electricity price and load forecasting performed through data analytics give future trends and patterns of consumption. The energy market trade is based on price forecasting. Accurate forecasting of electricity price and load improves the reliability and management of electricity market operations. The aim of this paper is to explore the state of the art proposed for price and load forecasting in terms of their performance for reliable and efficient smart energy management systems.
Content may be subject to copyright.
Electricity Price and Load Forecasting using
Data Analytics in Smart Grid : A Survey
Mubbashra Anwar, Afrah Naeem, Hira Gul, Arooj Arif, Sahiba Fareed, and
Nadeem Javaid
Abstract Smart grid (SG) is bringing revolutionary changes in the electric power
system. SG is supposed to provide economic, social, and environmental benefits for
many stakeholders. A smart meter is an essential part of the SG. Data acquisition,
transmission, processing, and interpretation are factors to determine the success of
smart meters due to the excess amount of data in the grid. Electricity price and
load are considered the most influential factors in the energy management system.
Moreover, electricity price and load forecasting performed through data analytics
give future trends and patterns of consumption. The energy market trade is based
on price forecasting. Accurate forecasting of electricity price and load improves the
reliability and management of electricity market operations. The aim of this paper
is to explore the state of the art proposed for price and load forecasting in terms of
their performance for reliable and efficient smart energy management systems.
1 Introduction
Smart grid (SG) is an electricity distribution network. Benefits provided by SG in-
clude: maintaining the balance between electricity supply and demand, managing
energy peak load, reliability, cost efficiency, and two-way communication [1]. SG
includes smart meters, renewable energy resources, and energy-efficient resources.
Smart meters are being installed in many countries.
Advanced metering infrastructure (AMI) is an essential component of SG. AMI
is the system that measures, collects, analyzes, transfers and stores the power related
information and communicates with network. AMI collects energy data from smart
Mubbashra Anwar
COMSATS University Islamabad, e-mail:
Corresponding author
Nadeem Javaid, COMSATS University Islamabad, e-mail:
2 Mubbashra et al.
meter and transfers it to utility. Hence, it creates a communication link between
consumers and utility.
Electricity price and load forecasting plays an important role in decision mak-
ing and operations of SG and energy industry [2],[3]. It helps utilities to plan their
capacity and different operations in order to reliable supply to all consumers. Elec-
tricity load and price forecasting categorization is based on time scale: short-term
(ST), mid-term (MT) , and long-term (LT) forecasting [3], [4]. ST covers days, MT
includes months, and LT covers years. LT forecasting is used for power system
planning, while MT and ST are used in operations. Moreover, LT forecasting plays
a significant role in the strategy-making of energy trading [3]. In particular, accurate
forecasting maximizes the profit and minimizes the cost of electricity [5]. On the
other hand, inaccuracy leads to electricity shortage, blackouts and high prices [6].
Therefore, there is a need for an effective and efficient approach to achieve accuracy.
Forecasting includes an entire set of steps, which are not often linear from top
to bottom. It starts with defining the goal, getting the data, exploring the data, pre-
processing the data, applying forecasting methods, comparing the results, evaluat-
ing the performance, and finally implementing some forecasting system. The whole
forecasting process is depicted in Fig. 1. Integration of renewable energy sources,
high volatility, uncertainty, huge data and different types of external and internal
factors are the key challenges to electricity load and price forecasting. Season and
weather parameters affect electricity load [1]. Various factors have an impact on
electricity price, such as weather, population growth, fuel price, and different eco-
nomic attributes [4], [7]. These factors are considered as input features for forecast-
ing. Furthermore, non stationary, nonlinear, and high volatility nature of electricity
data make load and price forecasting complicated and challenging.
As accurate electricity price and load forecasting is essential for the stability
of power industry. To improve the accuracy, researchers have proposed many ap-
proaches and models. Comprehensive literature exists in the context of electricity
price and load forecasting. Though, existing models are classified into different cat-
egories based on the method used for forecasting, such as data mining, time series,
artificial intelligence, machine learning and hybrid methods.
Various factors have an impact on electricity price and load such as, weather,
population growth, fuel price, and different economic attributes [1], [4], [7]. These
factors are considered as input features for training of the model. Furthermore, non
stationary, nonlinear, and high volatility nature of electricity price and load dataset
makes forecasting complicated and challenging.
Accurate electricity forecasting is essential for the stability of grid. Machine
learning is considered well-known technique that is used for electricity forecast-
ing. Deep learning is considered as the most promising technique as it has received
success in different areas: medical, computer vision and image processing. This
paper aims to provide a review on electricity price and load forecasting using differ-
ent techniques. It also compares the performance of various techniques to see which
technique gives best results. The limitations and future challenges are also discussed
for further research.
Electricity Price and Load Forecasting 3
The order of remainder paper is arranged as follows: in Section 2, the problems
of each paper are addressed. In Section 3, the main contributions of each paper are
summarized. In Section 4, performance evaluation metrics are reviewed. Limitations
and future research challenges are discussed in Section 5.
2 Problems Addressed
This section describes the problems that have been addressed by different authors.
Define Goal
Get Data
Explore Data Series
Partition Series
Forecasting Method
Fig. 1: The Forecasting Process
4 Mubbashra et al.
2.1 Electricity Price Forecasting
There exists multiple relations between input and output. According to [3], uni-
fied models used for the day-ahead electricity price forecasting (DAEPF), identify
single relation between historical data and future data. It leads to errors due to fluc-
tuations in data. Existing approaches have achieved satisfactory results for different
conditions. Now it is hard to improve the accuracy considering only optimization
techniques. For the further improvement of forecasting accuracy, main focus should
be the formulation of the prediction problem. Overall accuracy can be improved by
predicting a specific pattern.
According to [7], big data is not considered for electricity price forecasting. Fur-
thermore, existing systems having difficulties to manage the excess amount of cost
data in the grid. There is a need of an integrated system to regulate energy system.
Data preprocessing is not performed well that leads to inefficient and inaccurate
prediction. Accuracy could be raised by addressing these problems.
Different types of techniques have been used by researchers for forecasting
power load and price [8]. Feature selection plays a significant role in forecasting
models. Some researchers proposed feature selection techniques to get optimal re-
sults. The major drawback of the existing techniques is the same and fixed number
of features. These features are fed to model as input. While each output have differ-
ent suitable input and the other problem is the reliability of selected features that are
used to train the model.
In [9], different problems are addressed. The time-series models are linear and
perform well for low-frequency data whereas, electricity price signals are nonlin-
ear and are of high-frequency that causes errors in price prediction. Moreover, the
stationary process is unable to capture non-stationary features from price signals.
Tuning of irrelevant features and less amount of input-output data decreased the
performance of proposed model.
The main challenges for the electricity price forecasting are high voltage and
non-linear behavior of electricity [10]. Moreover, external factors are also difficult
to handle. These uncertainties are addressed in this work. Non-linearity in electricity
price data questions the accuracy of prediction. The accuracy of forecasting models
depends on the selected variables for input. Many techniques have been proposed
for feature selection. How to select robust feature is the main issue that is addressed
in paper [11].
All the approaches proposed to forecast the electricity price, have some draw-
backs [12]. In data-driven approaches, initialization of parameters is difficult due
to their sensitivity. In machine learning models, there is a need for new methods to
improve accuracy. Season affects the pattern of electricity demand. ST load fore-
casting is more helpful for the electricity market to make a decision rather than long
term electricity cost forecasting. Considering the non-linearity of the cost signal, it
is very difficult to select the best feature to train the data. Robust feature can enhance
the accuracy and efficiency of the forecasting models.
Electricity Price and Load Forecasting 5
Table 1: Electricity Price Forecasting
Task Dataset Data Preprocessing Proposed Model Evaluation Benchmarks
Day-ahead electricity
price forecasting
PJM ——— k-mean clustering
Short-term electricity
price forecasting
Iberian ———- ARIMA with other
forecasting models MAPE
Electricity price
New South Wales
———– DE-LSTM
Short-term electricity
price forecasting
DE-SVM ——-
Radial KPCA
Electricity price forecasting
Day-ahead electricity
price forecasting
Time lagged analysis
Forecast skill
BRNN + K-means
BRNN + EGT-Cluster
Electricity price forecasting
[10] ERCOT ————
Rerouted method
(Polynomial regression
MSE Simulations over four
Electricity price forecasting
Absolute error
2.1.1 Electricity Load Forecasting
In [2], different models have been proposed for electricity load forecasting. Explana-
tory variables: ambient temperature, solar radiation, relative humidity, wind speed,
and weekday index are used to train the regression models. These variables varies
building to building. Some of them are restricted to only one building that causes
overfitting problem. It deceases the performance of the model. To overcome this
problems deep learning is used as it has gained fame in many other fields due to
flexible nature.
According to [13], for the last decades, researchers and scholars are proposing
different models and approaches for electricity demand forecasting. Accuracy of the
forecasting is affected by some factors: holiday, season, price, day and hour, econ-
omy, policy, electricity power data, and management policy. Existing techniques
predict electric load without considering different factors and relationship between
them. That leads to inaccuracy and difficulty in forecasting electricity load. Taking
into account, pre-processing of original data enhances the accuracy of prediction
that is used in the proposed technique.
Limited predictability of weather factors raises operational challenges for power
systems in supply and demand of electricity [14]. ST forecasting (up to 12 hours-
6 Mubbashra et al.
ahead) improve operational efficiency of power systems. In literature there are many
existing models but have low efficiency than the proposed VAR model.
Feature selection is very important for the models used for prediction [15]. If data
is fed without preprocessing it increases computational time and overfitting prob-
lem. Although many researchers proposed different techniques for feature selection,
on the contrary, a small number of them merged and explored feature section tools
and prediction models. In different studies, it is stated that application of response
concept in decentralized energy system can reduced electricity cost. Whereas, tech-
niques used for prediction of electricity demand are not well explored. Most of the
methods forecast electricity demand only on large-scale and are unable to predict
specific electric power at lower-scale, that is equally important for efficient and re-
liable energy system.
Nonlinear and complicated patterns are the hurdles for the electricity load fore-
casting [16]. A semi-parametric additive model was proposed to forecast short term
electricity load that is not efficient for the nonlinear and serial correlation. Artificial
neural network is considered as a good approach for regression and classification,
which is trained mostly using back-propagation (BP). BP can easily be trapped and
also have slow convergence rate that are considered as drawbacks. The network fails
if the entire data set is fed to model, but incremental learning succeeds. Empirical
mode decomposition (EMD) has been used in different research fields due to good
performance. To improve accuracy, efficiency, and to overcome existing problems a
hybrid approach is proposed. For the short-term load forecasting many researchers
proposed different models [17]. As climate, economy, holidays and other random
variables are the challenges for electricity load forecasting. This paper combines a
set of models that perform better than others.
Table 2: Electricity Load Forecasting
Task Dataset Data Preprocessing Proposed Model Evaluation Benchmarks
Day-ahead electricity load
Los Angeles
New York
Short-term electricity load
Probability density forecasting
(deep learning model,
quantile regression, and
kernel density)
Short-term electricity load
Ireland ——–
(MS-CNN and
time-cognition models)
Short-term electricity load
New South Wales
(RBF kernel and UKF kernel)
Short-term electricity load
Australian Energy Market Operator DWT-EMD DWT-EMD based RVFL RMSE
Short-term electricity load
Italy-North Feature-fusion module CNN-LSTM
Electricity Price and Load Forecasting 7
3 Contributions
In the following section, contributions made by different authors in terms of elec-
tricity price and load forecasting are described.
3.1 Electricity Price Forecasting
The basic objective targeted in [7], is to forecast the accurate electricity cost using
big data. The proposed model is the combination of three modules. Initially, fea-
tures are selected from random data using a combination of random forest (RF) and
Relief-F algorithm. Redundancy is removed from features using grey correlation
analysis (GCA). The kernel function and principal component analysis (KPCA) is
used to reduce the dimensions. Finally, classification is performed, using differen-
tial evolution (DE) based on support vector machine (SVM) classifier to forecast the
electricity cost.
A novel model, dynamic choice of artificial neural network (DCANN) is pro-
posed in [8]. To overcome the problem of feature selection, index of bad sample
matrix (IBSM) is proposed that identifies bad features and selects appropriate input
for the corresponding output. DCANN gives input to the artificial neural network
(ANN). The unsupervised learning approach is used to select input for correspond-
ing output and a supervised learning approach is used for the training of these inputs.
To enhance the performance of clustering, enhanced game theoretic (EGT) clus-
ter is proposed [9]. Load and price both time series are considered through 2-
dimensional input selection. For the selection of robust cluster a persistence ap-
proach is introduced. Bayesian recurrent neural network (BRNN), data clustering
and time-lagged signal analysis are used in proposed hybrid method.
Soft computing techniques are used to forecast the real-time market [10]. Ini-
tially, mapping is performed between historical wind power generation and histor-
ical price. The second step is forecasting rule for wing power generation. For the
selection of the best model, different machine learning approaches are discussed.
A hybrid forecasting framework is proposed. Systematic analysis is performed to
extract the hidden features of electricity price data. To choose the input feature, the
hybrid feature selection (HFS) method is developed by integrating singular spec-
trum analysis (SSA), cuckoo search algorithm, and SVM [11].
It revealed that classification modeling is better than unified modeling for map-
ping relations [3]. As the unified model considers a single relation between historical
data and future data while in classification modeling, a model is established to iden-
tify the multiple relations for each pattern. A DPP is proposed that can be used with
different time series forecasting algorithms. The main idea of DPP is to predict the
price pattern through the conventional forecasting method and then classification is
performed to improve the accuracy of the forecasting. To improve the accuracy of
the pattern prediction a weighted voting mechanism (WVM) method is proposed.
In WVM multiple pattern predictions are merged.
8 Mubbashra et al.
In [12], a hybrid electricity cost method is proposed in this paper. A combination
of backtracking search algorithm (BSA) and adaptive neuro-fuzzy inference sys-
tem (ANFIS) approach is presented to improve the accuracy of forecasting. Multi-
objective binary-valued backtracking search algorithm (MOBBSA) and ANFIS are
merged for optimal feature selection. The main contribution of this paper is, solu-
tion of the feature selection problem, a multi-objective feature selection approach is
proposed. Feature selection approach is made of MOBBSA and ANFIS. MOBBSA
selects feature subsets from different combinations of inputs and ANFIS evaluates
feature subset based on the performance.
3.2 Electricity Load Forecasting
In this paper [13], EMD and ELM are used to overcome the problems of existing
systems. The EMD-Mixed-ELM method, initially decomposes the electricity load
data to cut the noise. EMD decomposes data into IMFs and residues, known as a
trend. This de-noised data is used by ELM to forecast the electricity load. A combi-
nation of kernels, RBF and UKF kernel are integrated with ELM because only one
kernel can not extract all the features of electricity demand data.
Main focus of paper [17] is to forecast the one day ahead load using hybrid incre-
mental learning model. DWT, EMD, and RVFL are used in this model. The prob-
lem of EMD is frequency mixing that is solved by using DWT. Defined incremental
DWT-EMD based on RVFL network improved both accuracy and efficiency.
To forecast short-term electricity, deep learning-based model is proposed that
discovers all key factors and also probability density prediction [2]. Initially, a deep
neural network is used to forecasting of electric load. DNN consist on MLP used as
a deep learning model.Feature selection is performed using feature engineering and
then to get electricity load pattern data visualization is employed. for load forest-
ing purposes. Probability density forecasting is applied by merging deep learning
model, quantile regression, and kernel density estimation method.
In [1], a deep learning-based multi-modal is proposed that uses four types of
data sets: i.e., season parameter, previous day vector, previous three days’vector, and
weather parameter. SDAs is used in deep feature extraction module. Then extracted
features are used to train the SVR to predict the day-ahead electricity load.
VAR model is employed to capture correlation of three different variables [14].
Regression technique is used to capture the autocorrelation of the variables. Further-
more, VAR model also useful to capture time-lagged correlation of different corre-
lation. Time series approach used in the proposed model, capture different charac-
teristics of weather: linear trend, seasonal component, and stochastic component.
A hybrid feature selection based on machine learning is proposed, to get the
most applicable feature for the accurate prediction of electric load in a decentralized
energy system [15]. BGA is used for the feature selection. GPR is used to measure
the importance of the features. Main contributions are as follows: investigation and
suggestion related to FS for accurate prediction of electric load.
Electricity Price and Load Forecasting 9
To overcome the above mentioned problems [16], this paper introduced a multi-
scale convolution neural network with time-cognition. Initially, a CNN network
model based on MS-CNN is used for the robust feature selection. A unique periodic
code is introduce to improve the accuracy of sequential model regarding time cogni-
tion. The combination of multi-scale and time cognitive feature builds an multi-step
deep learning model.
It describes the use of wavelet ensembles to build a forecast model [17]. Main
focus of the paper is to build a pipeline from raw data and compare its results with
existing techniques. Wavelet is used in this pipeline to extract the features and for
learning process.
4 Validation
In this section performance evaluation of the existing literature is discussed. Perfor-
mance metrics: mean absolute percentage error (MAP), absolute percentage error
(APE),root mean square error (RMSE), and mean absolute error (MAE) are widely
used in the following papers.
4.1 Electricity Price Forecasting
In [7], several simulations are performed to evaluate the performance of the pro-
posed model using real-world data. For this purpose, a simulator is developed that
uses hourly electricity cost data and power generation data of ISO new England con-
trol area (ISO NE-CA) from 2010 to 2015. Naive bayes (NB) and decision tree are
used as a benchmark to compare the efficiency of DE-SVM. KPCA performance is
compared with principal component analysis (PCA). Simulation results show that
the proposed model performs better than other benchmarks.
The performance of the proposed model [8] is validated, using original data of
electricity price from PJM. Moreover, MAE and MAPE are used as performance
metrics. Other benchmarks used for the comparison of the proposed model are
back propagation artificial neural network (BPANN), radial basis function network
(RBFN), least squares support vector machine (LSSVM), and SVM. Numerical re-
sults show that the proposed model DCANN has the best performance consider-
ing PJM dataset. Furthermore, only LSSVM and DCANN give relevant results, in
high volatility electricity price. DCANN is an efficient model compared with other
In [9] evaluation of the proposed EGT-cluster is performed using MSE, MAPE,
RMSE, and forecast-skill used to evaluate the performance. EGT-cluster is com-
pared with k-means, original SOM and NG algorithms using different price series
data. Data used to evaluate the performance of the forecasting algorithm is New
York independent system operator (NYISO) market price data from 2008 to 2014.
10 Mubbashra et al.
A comparison is conducted to the conventional method verifies the higher accu-
racy and reliable relation between input and output [10]. Simulations results verified
that the performance of the proposed model is effective and efficient. The noise was
added to check the stability of the proposed model, results prove it stable model.
4.2 Electricity Load Forecasting
In this paper [13], EMD and ELM are used to overcome the problems of existing
systems. The EMD-Mixed-ELM method, initially decomposes the electricity load
data to cut the noise. EMD decomposes data into intrinsic mode functions (IMFs)
and residues, known as a trend. This de-noised data is used by ELM to forecast the
electricity load. A combination of kernels, radial basis function kernel (RBF) and
UKF kernel are integrated with ELM because only one kernel can not extract all
the features of electricity demand data. In [3], Time Series Cross-Validation is used
to evaluate the performance. Several Metrics such as MAPE, RMSE, and MAE are
also used to represent prediction accuracy. MAPE is used as a performance indicator
in [4]. In [5], the accuracy of model examined via RMSE and MAPE. In [6], four
metrics such as Mean Error (AE), MAE, MAPE, and RMSE are presented. MAPE is
applied to evaluate the performance in [7]. In [8], accuracy metrics such as RMSE,
Normalized Mean Square Error (NMSE), MAE, and MAPE are used for evaluation.
In [9], NYISO and AEMO data set is used for testing the model and MAPE, RMSE,
MAE, and mean percentage error (MPE) are used for performance metrics.
5 Future Work
This section enlightens the limitations of existing approaches that can be addressed
in future for further improvement.
5.1 Electricity Price Forecasting
The effectiveness of the proposed model [3] is verified using data from same market,
while data from different markets have different characteristics can affect the results.
In time series forecasting model there are many factors with different ratio of impact
and there exist a relation between them. In feature selection impact of these feature
plays an important role to improve the accuracy. So correlation analysis should be
used to analyze the impact of the feature. The proposed method performs multiple
steps for pattern prediction of next day, accuracy of final pattern depends on the
accuracy of each step. So to avoid this limit a state prediction method can be used
to predict the pattern.
Electricity Price and Load Forecasting 11
Computational sources used in the proposed model [7] increases the cost of the
model, so it is costly to implement it on real-time data. DE-SVM is used in this
paper do not give optimal results because it does not cover whole search space
for the tuning of the parameters so the accuracy rate decreases. Random forest is
used for the extraction that takes random values and gives result, as in the proposed
method big data is used so random forest can randomly select garbage value that
affects output. Computational time increases with the complexity of the model and
the proposed model is complex.
Dynamic choice artificial neural network (DCANN) is an extended form of ANN,
so it has same limitations as ANN. Complications of the model increases computa-
tional time moreover, it do not cover whole search space that lead to inefficient fore-
casting. DCANN has maximum computational time compared to other benchmarks
[8]. The proposed method in [9] is complex in nature, data scalability becomes a
problem for the model by increasing the amount of data. In [10], to achieve better
results features with different types of parameters should be considered. Selected
features have causal relation towards electricity price.
In the proposed model [11], only electricity price data is considered that increases
the complexity of forecasting process. Other factors should also be integrated in
forecasting model. Proposed framework is designed using seasonal characteristics
of the electricity price.
5.2 Electricity Load Forecasting
CNN and recurrent neural networks based on deep learning and can handle large and
complex datasets, that can be used for better results. Location is also a significant
factor that can be used for the improvement of accuracy of forecasting methods [2].
As in the proposed model [13], we are predicting electric load and claiming the ac-
curacy and performance of the model, some external factors affect the consumption
of electricity and are ignored by this model. EMD used in proposed method de-
composes high-frequency signals into low frequency. Results lead to overshoot or
undershoot and that shows, EMD is incapable of separating components of closely
spaced frequencies.
Efficiency of the proposed model decreases in terms of small dataset. Due to
complexity of the model it is expensive to train the data [15]. Proposed model in
[17], works only on electricity load data, by merging proposed decomposition tech-
nique with other learning models electricity cost data can also be analyzed. Use
of classification methods with proposed model can provide result in the form of
intervals, scenarios, and density functions. The complexity of the model increases
computational time.
12 Mubbashra et al.
6 Conclusion
Smart grids are replacing traditional grids. It involves two-way communication, gen-
eration, transmission, distribution and consumption of electricity. This paper pro-
vides a limited review, that represents the different techniques used for electricity
load and price forecasting. Accurate electricity price and load forecasting is essen-
tial for the stability of grid. Machine learning is a well-known technique that is
used for electricity price and load forecasting. The paper is classified in terms of
problems addressed, proposed solutions, limitations, and open research challenges.
Results are shown in the form of tables. This paper provides enough information,
insights, and references in the area of load and price forecasting. We hope that this
paper will help scientific community, researchers, and practitioners to assist in fu-
ture development and to contribute to this new challenging and important area.
1. Tong, C., Li, J., Lang, C., Kong, F., Niu, J. and Rodrigues, J.J., 2018.“ An efficient deep model
for day-ahead electricity load forecasting with stacked denoising auto-encoders.” Journal of
Parallel and Distributed Computing, 117, pp.267-273.
2. Guo, Z., Zhou, K., Zhang, X. and Yang, S., 2018.“ A deep learning model for short-term
power load and probability density forecasting.” Energy, 160, pp.1186-1200.
3. Wang, F., Li, K., Zhou, L., Ren, H., Contreras, J., Shafie-Khah, M. and Catalao, J.P., 2019.“
Daily pattern prediction based classification modeling approach for day-ahead electricity
price forecasting.” International Journal of Electrical Power & Energy Systems, 105, pp.529-
4. Angamuthu Chinnathambi, R., Mukherjee, A., Campion, M., Salehfar, H., Hansen, T.M., Lin,
J. and Ranganathan, P., 2019.“ A multi-stage price forecasting model for day-ahead electricity
markets.” Forecasting, 1(1), pp.26-46.
5. Peng, L., Liu, S., Liu, R. and Wang, L., 2018. “Effective long short-term memory with differ-
ential evolution algorithm for electricity price prediction.” Energy, 162, pp.1301-1314.
6. Deng, Z., Wang, B., Xu, Y., Xu, T., Liu, C. and Zhu, Z., 2019.“ Multi-scale convolutional
neural network with time-cognition for multi-step short-term load forecasting.” IEEE Access,
7, pp.88058-88071.
7. Wang, K., Xu, C., Zhang, Y., Guo, S. and Zomaya, A.Y., 2017.“ Robust big data analytics for
electricity price forecasting in the smart grid.” IEEE Transactions on Big Data, 5(1), pp.34-45.
8. Wang, J., Liu, F., Song, Y. and Zhao, J., 2016.“ A novel model: Dynamic choice artificial neu-
ral network (DCANN) for an electricity price forecasting system.” Applied Soft Computing,
48, pp.281-297.
9. Ghayekhloo, M., Azimi, R., Ghofrani, M., Menhaj, M.B. and Shekari, E., 2019.“ A combina-
tion approach based on a novel data clustering method and Bayesian recurrent neural network
for day-ahead price forecasting of electricity markets.” Electric Power Systems Research, 168,
10. Luo, S. and Weng, Y., 2019.“ A two-stage supervised learning approach for electricity price
forecasting by leveraging different data sources.” Applied energy, 242, pp.1497-1512.
11. Zhang, X., Wang, J. and Gao, Y., 2019.“ A hybrid short-term electricity price forecast-
ing framework: Cuckoo search-based feature selection with singular spectrum analysis and
SVM.” Energy Economics.
Electricity Price and Load Forecasting 13
12. Pourdaryaei, A., Mokhlis, H., Illias, H.A., Kaboli, S.H.A. and Ahmad, S., 2019. “Short-
Term Electricity Price Forecasting via Hybrid Backtracking Search Algorithm and ANFIS
Approach.” IEEE Access, 7, pp.77674-77691.
13. Chen, Y., Kloft, M., Yang, Y., Li, C. and Li, L., 2018. “Mixed kernel based extreme learning
machine for electric load forecasting.” Neurocomputing, 312, pp.90-106.
14. Yixian, L.I.U., Roberts, M.C. and Sioshansi, R., 2018.“ A vector autoregression weather
model for electricity supply and demand modeling.” Journal of Modern Power Systems and
Clean Energy, 6(4), pp.763-776.
15. Eseye, A.T., Lehtonen, M., Tukia, T., Uimonen, S. and Millar, R.J., 2019.“ Machine learning
based integrated feature selection approach for improved electricity demand forecasting in
decentralized energy systems.” IEEE Access, 7, pp.91463-91475.
16. Tan, T.Y., Zhang, L. and Lim, C.P., 2019.“ Adaptive melanoma diagnosis using evolving
clustering, ensemble and deep neural networks.” Knowledge-Based Systems, 187, p.104807.
17. Qiu, X., Suganthan, P.N. and Amaratunga, G.A., 2018. “Ensemble incremental learning ran-
dom vector functional link network for short-term electric load forecasting.” Knowledge-
Based Systems, 145, pp.182-196.
... These interventions slightly brought electricity demand close to its supply, while at the same time reasonable reserve margins were being maintained. In energy sectors risk denotes a probability distribution of future returns; hence, uncertainty is considered as a broader concept that incorporates indistinctness about the parameters of this probability distribution Babatunde et al. (2020) and Anwar et al. (2020). ...
Full-text available
The use of extreme value theory (EVT) is usually aimed at quantifying the asymptotic behaviour of extreme quantiles. The generalised Pareto distribution (GPD) with peaks-over-threshold (POT) approach is applied to bootstrap uncertainty intervals for the return periods of extreme daily electricity consumption in South Africa. The leeway of extremes on daily electricity consumption studied here is the impetus behind this study. To examine the effect of a time-based and extreme non-stationary trend in a dataset, a non-stationary GPD is cast-off in computing the shape parameter and, this resulted in the establishment of a type III GPD known as a Weibull class for the South African electricity sector. Results of this study revealed a non-stationary trend with a prediction power of 89.6% for the winter season and 85.65% non-winter season. This means that EVT provides a robust basis for statistical modelling of extreme values. Furthermore, a base for future researchers for conducting studies on emerging markets, more specifically in the South African context has also been contributed.
... Wind generators, photovoltaic modules (PVM), fuel cells (FC), storage systems (batteries), and many other energy sources should be integrated into green power distributed generation plants (hybrid sources) [1][2][3][4] and modern smart grids [5][6][7][8]. If any energy sources cannot provide energy, the rest could satisfy the customers' energy demand as a multiple uninterrupted power supply. ...
Full-text available
Paralleled boost asymmetric configurations operating in discontinuous conduction mode (DCM) are suitable for integrating dissimilar green energy generating sources and control algorithms in versatile scenarios where voltage step-up, low cost, stable operation, low output ripple, uncomplicated design, and acceptable efficiency are needed. Unfortunately, research has mainly been conducted on the buck, sepic, switched-capacitor, among other asymmetric configurations operating in continuous conduction mode (CCM), to the authors’ knowledge. For asymmetric boost type topologies, achieving simultaneous CCM is not a trivial task, and other problems such as circulating currents arise. Research for interleaved converters cannot be easily extended to asymmetric boost topologies due to the dissimilarity of control algorithms and types of sources and parallel stages. This paper analytically establishes properties of stability, output ripple, output voltage, and design for asymmetrical paralleled boost converters operating in DCM with simultaneous or phase delayed (sequential) triggering. A 300 W experimental design and the respective tests allow validation of such properties, resulting in an easy-to-implement configuration with acceptable efficiency.
Electricity theft (ET) is a major problem in developing countries. It a�ects the economy that causes revenue loss. It also decreases the reliability and stability of electricity utilities. Due to these losses, the quality of supply e�ects and tari � imposed on legitimate consumers. ET is an essential part of Non-technical loss (NTL) and it is challenging for electricity utilities to �nd the responsible people. Several methodologies have developed to identify ET behaviors automatically. However, these approaches mainly assess records of consumers' electricity usage, may prove inadequate in detecting ET due to a variety of theft attacks and irregularity of consumers' behavior. Moreover, some important challenges are needed to be addressed. (i) The number of normal consumers has been wrongly identi�ed as fraudulent. This leads to high False-positive rate (FPR). After the detection of theft, on-site inspection is needed to validate the detected person, either is it fraudulent or not and it is costly. (ii) The imbalanced nature of datasets which negatively a�ect on the model's performance. (iii) The problem of over�tting and generalization error is often faced in deep learning models, predicts unseen data inaccurately. So, the motivation for this work to detect illegal consumers accurately. We have proposed four Arti�cial intelligence (AI) models in this thesis. In system model 1, we have proposed Enhanced arti�cial neural network blocks with skip connections (EANNBS). It makes training easier, reduces over�tting, FPR and generalization error, as well as execution time. Temporal convolutional network with enhanced multi-layer perceptron (TCN-EMLP) is proposed in system model 2. It analyzes the sequential data based on daily electricity-usage records, obtained from smart meters. At the same time, EMLP integrates the non-sequential auxiliary data, such as data related to electrical connection type, property area, electrical appliances usage, etc. System model 3 based on Residual network (RN) that is used to automate feature extraction while three tree-based classi�ers such as Decision tree (DT), Random forest (RF) and Adaptive boosting (AdaBoost) are trained on the obtained features for classi�cation. Hyperparameter tuning toolkit is presented in this system model, named as Hyperactive optimization toolkit. Bayesian is used as an optimizer in this toolkit that aims to simplify the tuning process of DT, RF and AdaBoost. In system model 4, input is forwarded to three di�erent and well-known Machine learning (ML) techniques, i.e., Support vector machine (SVM), as an input. At this stage, a meta-heuristic algorithm named Simulated annealing (SA) is employed to acquire optimal values for ML models' hyperparameters. Finally, ML models' outputs are used as features for meta-classi�ers to achieve �nal classi�cation with Light Gradient boosting machine (LGBM) and Multi-layer perceptron (MLP). Furthermore, Pakistan residential electricity consumption dataset (PRECON1), State grid corporation of china (SGCC2) and Commission for energy regulation (CER3) datasets is used in this thesis. SGCC dataset contains 9% fraudulent consumers, which are extremely less than non-fraudulent consumers, due to the imbalance nature of data. Furthermore, many classi�cation techniques have poor predictive class accuracy for the positive class. These techniques mainly focus on minimizing the error rate while ignoring the minority class. Many re-sampling techniques are used in literature to adjust the class ratio; however, sometimes, these techniques remove the important information that is necessary to learn the model and cause over�tting. By using six previous theft attacks, we generate theft cases to mimic the real world theft attacks in original data. We have proposed the combination of oversampling and under-sampling techniques that is Near miss borderline synthetic minority oversampling technique (NMB-SMOTE), Tomek link borderline synthetic minority oversampling technique with support vector machine (TBSSVM) and Synthetic minority oversampling technique with near miss (SMOTE-NM) to handle imbalanced classi�cation problem. We have conducted a comprehensive experiment using SGCC, CER and PRECON datasets. The performance of suggested model is validated using di�erent performance metrics that are derived from Confusion matrix (CM).
Full-text available
Electric load forecasting has always been a key component of power grids. Many countries have opened up electricity markets and facilitated the participation of multiple agents, which creates a competitive environment and reduces costs to consumers. In the electricity market, multi-step short-term load forecasting becomes increasingly significant for electricity market bidding and spot price calculation, but performances of traditional algorithms are not robust and unacceptable enough. In recent years, the rise of deep learning gives us the opportunity to improve the accuracy of multi-step forecasting further. In this paper, we propose a novel model multi-scale convolutional neural network with time-cognition (TCMS-CNN). At first, a deep convolutional neural network model based on multi-scale convolutions (MS-CNN) extracts different level features that are fused into our network. In addition, we design an innovative time coding strategy called the periodic coding strengthening the ability of sequential model for time cognition effectively. At last, we integrate MS-CNN and periodic coding into the proposed TCMS-CNN model with an end to end training and inference process. With ablation experiments, MS-CNN and periodic coding methods had better performances obviously than the most popular methods at present. Specifically, for 48 steps point load forecasting, TCMS-CNN had been improved by 34.73%, 14.22%, and 19.05% on MAPE than the state-of-the-art methods recursive multi-step LSTM (RM-LSTM), direct multi-step MS-CNN (DM-MS-CNN), and the direct Multi-step GCNN(DM-GCNN) respectively. For 48 steps probabilistic load forecasting, TCMS-CNN had been improved by 3.54% and 6.77% on average pinball score than DM-MS-CNN and DM-GCNN. These results show a great promising potential applied in practice.
Full-text available
Improved performance electricity demand forecast can provide decentralized energy system operators, aggregators, managers, and other stakeholders with essential information for energy resource scheduling, demand response management and energy market participation. Most previous methodologies have focused on predicting the aggregate amount of electricity demand at national or regional scale and disregarded the electricity demand for small-scale decentralized energy systems (buildings, energy communities, microgrids, local energy internets, etc.), which are emerging in the smart grid context. Furthermore, few research groups have performed attribute selection before training predictive models. This paper proposes a machine learning (ML) based integrated feature selection approach to obtain the most relevant and nonredundant predictors for accurate short-term electricity demand forecasting in distributed energy systems. In the proposed approach, one of the ML tools -Binary Genetic Algorithm (BGA) is applied for the feature selection process and Gaussian Process Regression (GPR) is used for measuring the fitness score of the features. In order to validate the effectiveness of the proposed approach, it is applied to various building energy systems located in the Otaniemi area of Espoo, Finland. The findings are compared with those achieved by other feature selection techniques. The proposed approach enhances the quality and efficiency of the predictor selection, with minimal chosen predictors to achieve improved prediction accuracy. It outperforms the other evaluated feature selection methods. Besides, a feedforward artificial neural network (FFANN) model is implemented to evaluate the forecast performance of the selected predictor subset. The model is trained using two-year hourly dataset and tested with another one-year hourly dataset. The obtained results verify that the FFANN forecast model based on the BGA-GPR FS selected training feature subset has achieved an annual MAPE of 1.96%, which is a very acceptable and promising value for electricity demand forecasting in small-scale decentralized energy systems.
Full-text available
In this work, a hybrid electricity price forecasting method which is composed of two-stage feature selection method and optimized adaptive neuro-fuzzy inference system (ANFIS) technique as a forecasting engine is proposed to accurately forecast electricity price. A multi-objective feature selection approach comprises of multi-objective binary-valued backtracking search algorithm (MOBBSA) as an efficient evolutionary search algorithm and ANFIS method is developed in this study to extract the most influential subsets of input variables with maximum relevancy and minimum redundancy. Through the combination of backtracking search algorithm (BSA) in learning process of ANFIS approach, a hybrid machine learning algorithm has been developed to forecast the electricity price more accurately. Real-world electricity demand and price dataset from Ontario power market; which is reported as among the most volatile market worldwide, has been used as case study to validate the performance of the proposed approach. From, the simulation results it has been seen that the proposed hybrid forecasting method were effective in accurately forecast the Ontario electricity price. In addition, to prove the superiority of the proposed hybrid forecasting method the simulation results obtained using ANN and ANFIS models optimized by other well-known optimization methods have been compared with that of proposed method.
Full-text available
Forecasting hourly spot prices for real-time electricity markets is a key activity in economic and energy trading operations. This paper proposes a novel two-stage approach that uses a combination of Auto-Regressive Integrated Moving Average (ARIMA) with other forecasting models to improve residual errors in predicting the hourly spot prices. In Stage-1, the day-ahead price is forecasted using ARIMA and then the resulting residuals are fed to another forecasting method in Stage-2. This approach was successfully tested using datasets from the Iberian electricity market with duration periods ranging from one-week to ninety days for variables such as price, load and temperature. A comprehensive set of 17 variables were included in the proposed model to predict the day-ahead electricity price. The Mean Absolute Percentage Error (MAPE) results indicate that ARIMA-GLM combination performs better for longer duration periods, while ARIMA-SVM combination performs better for shorter duration periods.
Under the liberalization and deregulation of the power industry, price forecasting has become a cornerstone for market participants' decision-making such as bidding strategies and purchase plans. However, the exclusive nonlinearity dynamics of electricity price is a challenge problem that largely affects forecasting accuracy. To address this task, this paper presents a hybrid forecasting framework for short-term electricity price forecasting by exploiting and mining the important information hidden in the electricity price signal. Moreover, a hybrid feature selection method (HFS) is introduced into the forecasting strategy. To exhibit the dynamical characteristics of electricity price, we primarily perform a singular spectrum analysis (SSA)-based systematic analysis process by using the merit of SSA and analyzing the multiple seasonal patterns of short-term electricity price series, providing a meaningful representation of the hidden patterns and time-varying volatility of electricity price series. Aiming at selecting the key features, the candidate variables are constructed considering the dynamic behavior of price series; further, to capture the optimal features from the candidates, the correlation threshold θ is defined for the adjustable parameters in HFS and optimally determined by the intelligent search algorithm. Additionally, triangulation based on the Pearson, Spearman and Kendall rank correlation coefficient is performed to strengthen the reliability of the proposed method. The proposed hybrid forecasting framework is validated in the New South Wales electricity market, which demonstrates that the developed approach is truly better than the benchmark models used and a reliable and promising tool for short-term electricity price forecasting.
In this research, we propose a variant of the Particle Swarm Optimization (PSO) algorithm, namely hybrid learning PSO (HLPSO), for skin lesion segmentation and classification. HLPSO combines diverse search mechanisms including modified Firefly Algorithm (FA) operations, a new spiral research action, probability distributions, crossover, and mutation procedures to diversify and improve the original PSO algorithm. It is used in conjunction with the K-Means clustering algorithm to enhance lesion segmentation. Its cost function takes both intra-class and inter-class variations into account to increase scalability. Two lesion classification systems are formulated based on HLPSO. In the first system, HLPSO is used to devise evolving convolutional neural networks (CNN) with optimized topologies and hyper-parameters for lesion classification. In the second system, shape and colour features, as well as texture features extracted using the Kirsch operator and Shift Local Binary Patterns are used to produce an initial discriminative lesion representation. HLPSO is then used to identify the most significant components of each feature vector for ensemble lesion classification. Evaluated using several skin lesion data sets, both systems depict superior capabilities in lesion segmentation, deep CNN architecture generation, and discriminative feature selection for ensemble lesion classification, and outperform a number of advanced PSO and FA variants, classical search methods, as well as other related models on skin lesion classification significantly. HLPSO also yields better performances over other classical and advanced search methods in solving a number of benchmark tasks related to mathematical landscapes and those in the complex CEC 2014 test suite.
Over the years, the growing penetration of renewable energy into the electricity market has resulted in a significant change in the electricity market price. This change makes the existing forecasting method prone to error, decreasing the economic benefits. Hence, more precise forecasting methods need to be developed. This paper starts with a survey and benchmark of existing machine learning approaches for forecasting the real-time market (RTM) price. While these methods provide sufficient modeling capability via supervised learning, their accuracy is still limited due to the single data source, e.g., historical price information only. In this paper, a novel two-stage supervised learning approach is proposed by diversifying the data sources such as highly correlated power data. This idea is inspired by the recent load forecasting methods that have shown extremely well performances. Specifically, the proposed two-stage method, namely the rerouted method, learns two types of mapping rules. The first one is the mapping between the historical wind power and the historical price. The second is the forecasting rule for wind generation. Based on the two rules, we forecast the price via the forecasted generation and the first learned mapping between power and price. Additionally, we observed that it is not the more training data the better, leading to our validation steps to quantify the best training intervals for different datasets. We conduct comparisons of numerical results between existing methods and the proposed methods based on datasets from the Electric Reliability Council of Texas (ERCOT). For each machine learning step, we examine different learning methods, such as polynomial regression, support vector regression, neural network, and deep neural network. The results show that the proposed method is significantly better than existing approaches when renewables are involved.
Day-ahead electricity price forecasting (DAEPF) plays a very important role in the decision-making optimization of electricity market participants, the dispatch control of independent system operators (ISOs) and the strategy formulation of energy trading. Unified modeling that only fits a single mapping relation between the historical data and future data usually produces larger errors because the different fluctuation patterns in electricity price data show different mapping relations. A daily pattern prediction (DPP) based classification modeling approach for DAEPF is proposed to solve this problem. The basic idea is that first recognize the price pattern of the next day from the “rough” day-ahead forecasting results provided by conventional forecasting methods and then perform classification modeling to further improve the forecasting accuracy through building a specific forecasting model for each pattern. The proposed approach consists of four steps. First, K-means is utilized to group all the historical daily electricity price curves into several clusters in order to assign each daily curve a pattern label for the training of the following daily pattern recognition (DPR) model and classification modeling. Second, a DPP model is proposed to recognize the price pattern of the next day from the forecasting results provided by multiple conventional forecasting methods. A weighted voting mechanism (WVM) method is proposed in this step to combine multiple day-ahead pattern predictions to obtain a more accurate DPP result. Third, the classification forecasting model of each different daily pattern can be established according to the clustering results in step 1. Fourth, the credibility of DPP result is checked to eventually determine whether the proposed classification DAEPF modeling approach can be adopted or not. A case study using the real electricity price data from the PJM market indicates that the proposed approach presents a better performance than unified modeling for a certain daily pattern whose DPP results show high reliability and accuracy.
Accurate load forecasting is critical for power system planning and operational decision making. In this study, we are the first to utilize a deep feedforward network for short-term electricity load forecasting. Our results are compared to those of popular machine learning models such as random forest and gradient boosting machine models. Then, electricity consumption patterns are explored based on monthly, weekly and temperature-based patterns in terms of feature importance. Also, a probability density forecasting method based on deep learning, quantile regression and kernel density estimation is proposed. To verify the efficiency of the proposed methods, three case studies based on daily electricity consumption data for three Chinese cities for 2014 are conducted. The empirical results demonstrate that (1) the proposed deep learning-based approach exhibits better forecasting accuracy in terms of measuring electricity consumption relative to the random forest and gradient boosting model; (2) monthly, weekly and weather-related variables are key factors that have a great influence on household electricity consumption; and (3) the proposed probability density forecasting method is capable of forecasting high-quality prediction intervals via probability density forecasting.