Electricity Load and Price Forecasting
Using Enhanced Machine Learning
Hamida Bano1, Aroosa Tahir2, Ishtiaq Ali1, Raja Jalees ul Hussen Khan1,
Abdul Haseeb1, and Nadeem Javaid1(B
1COMSATS University Islamabad, Islamabad 44000, Pakistan
2Sardar Bhadur Khan Women University Quetta,
Quetta 87300, Pakistan
Abstract. The exponential increase in electricity generation and
consumption pattern are the two main issues in the wholesale markets.
To handle these issues diﬀerent machine learning techniques are used for
load and price prediction in the research ﬁeld. The wholesale utilities
provide real-time data of load and price for the better prediction of elec-
tricity generation purposes. The New York Independent System Operator
(NY-ISO) is one of the utility which provide electricity to diﬀerent coun-
ties like United States, Canada and Israel. In this paper, hourly data of
2016–2017 is used for the forecasting process of load and price of New
York City. Feature selection and extraction are used to achieve important
features. The feature selection is done by two techniques Classiﬁcation
and Regression Tree (CART) and Recursive Feature Elimination (RFE)
and Feature extraction by using Singular Value Decomposition (SVD).
The Multiple Layer Perceptron (MLP), Support Vector Machine (SVM)
and Logistic Regression (LR) classiﬁers are separately used for forecast-
ing purposes of electricity load and price. Further enhance these three
techniques EMLP, ESVM and ELR to take more accurate results for
electricity load and price forecasting.
Electricity is the basic need of smart environment (smart city, smart industry,
smart homes, smart devices etc.). Due to the exponential increase in the demand
of electricity from diﬀerent sectors (residential, commercial and industrial), it is
important to proposed a management scheme or smart system which betterly
handle the generation and consumption rate of both utility and consumption
Smart Grid (SG) is the network of transmission lines from where electricity is
eﬃciently transferred from the power plants to residential, commercial or indus-
trial sectors. It has a two way communication between the utility and consumers
Springer Nature Switzerland AG 2020
L. Barolli et al. (Eds.): IMIS 2019, AISC 994, pp. 255–267, 2020.
256 H. Bano et al.
Table 1. List of abbreviations
ARIMA AutoRegressive Integrated Moving Average
CART Classiﬁcation and Regression Tree
DAs Data Analytics
DSM Demand Side Management
EMLP Enhanced Multilayer Perceptron
ELR Enhanced Logistic Regression
IWNN Improved Wavelet Neural Network
LR Logistic Regression
LSTM Long Short Term Memory
MLP Multilayer Perceptron
NN Neural Network
NY-ISO New York Independent System Operator
NLSSVM Normal Least Square Support Vector Machine
NSW New South Wales (Australia market)
PJM Pennsylvania, Jersey, Maryland
RNN Recurrent Neural Network
RFE Recursive Feature Elimination
STLF Short Term Load Forecasting
SG Smart Grid
SVM Support Vector Machine
SVD Singular Value Decomposition
however, the traditional grids have one way communication. The communica-
tion capability of SG eﬃciently increases its usage in the management schemes
of load and price. The SG is beneﬁcial in many aspects quick restoration after
power outage, cost management of both utilities and consumers, reduced peak
load and improve security. Due to these beneﬁcial factors utilities move to smart
grid to reduce the demand load in the peak hours.
Data Analytics (DA) is the broad ﬁeld of qualitative and quantitative exam-
ining of large data sets. The techniques and technologies of DA are used for the
better decision making in business industries which increase the business revenue
and improve the operational eﬃciency.
Predictive DA is one of the types of DA which analyze the current or historical
facts to make future prediction. The utilities need to enhance the generation
and consumption rate of electricity, so that accurate prediction of load and price
of electricity is necessary. Electricity load and price forecasting techniques are
used in demand response and load management systems. Demand response is
the balance of supply side (utility) and demand side (consumer) to eﬃciently
control the load in peak hours (to minimize load and price).
Electricity Load and Price Forecasting Using Enhanced Machine ... 257
The utilities provide incentive programs to customers  to manage the
demand response. Electricity load forecasting techniques are classiﬁed into short
term, medium term and long term methods. Short term forecasting in smart
grids shows the continuous supply of electricity by saving cost. In short term
forecasting, prediction of a few minutes, hours or days ahead while in medium
term forecasting a week or month ahead and in long term forecasting prediction
of 1–10 years.
Recurrent Neural Network (RNN) is a type of Neural Network (NN) used for
the processing of sequential data. In , authors describe the training procedure
of LSTM network useful for the short term load forecasting. LSTM sequential
models are eﬃciently used for data to memorize by its memory unit for long
sequence. In paper , the LSTM technique improves the accuracy of model by
presenting the optimum minimization of gradient vanishing problem.
Deep Neural network (DNN)  , Gated Recurrent Unit (GRU) , Hybrid
method + MLP , Convolution Neural Network (CNN), Shallow Neural Net-
work (SNN) and LSTM  models are eﬃciently used in the forecasting methods
which increase the prediction accuracy. Classiﬁer based models are mostly used
in forecasting such as Sperm Whale algorithm + LSSVM . The NN type MLP
has traditionally have one hidden layer network however the simpler models of
NN have potential to predict the accurate accuracy of price . Other energy
related applications shows excellent results obtained in time series prediction
[13,14], the electricity price prediction is possible by using DL architectures.
These prediction techniques motivate the worth of the techniques to enhance
the prediction accuracy of load and price of power markets.
1.2 Problem Statement
Traditional techniques are diﬃcult to handle big data in SGs. However, the
accurate load and price forecasting using huge amount of data from the smart
grid is the main challenges in the data analytic ﬁeld. The exponential increase of
the electricity demand and the weather conditions also aﬀect on the generation
side in terms of load and price. The high demand (on peak hours) of electricity
eﬀect on the pricing schemes. The high complex computational time during
training process directly aﬀect on accuracy. However, load forecasting is in 
ignored the computational time.
In , tune the hyperparameters (cost penalty, insensitive loss function and
kernal) of SVM are addressed for price forecasting however, the over ﬁtting
problem is not handle.
•The main contribution of this paper is the time series hourly prediction of
load and price of electricity of NY-ISO market.
258 H. Bano et al.
•First step to normalize the dataset in which selection and extraction techniques
•In feature selection method, two steps are used to ﬁnd the relevance and
validity of data from the dataset which eﬀect on the forecasting accuracy.
•Feature Selection: CART technique is used to investigate the relationship
between a dependent variables (target) and independent variables (predictor).
It shows the strength or important impact of multiple independent variables on
RFE technique is the second step of feature selection to further reduce the
dimensionality of dataset. To remove the low important features, RFE directly
improve the accuracy of prediction and reduce the model complexity.
•Feature Extraction: Further reduce the high dimensionality of dataset and
computational cost SVD technique is applied.
•Classiﬁer: Three classiﬁers MLP, SVM and LR models are separately used for
the time series hourly prediction of load and price. Furthermore, enhance the
three techniques for more accuracy in forecasting process.
•Simulation results shows that MLP, SVM and LR gives best accuracy in
electricity load and price forecasting process.
2 Related Work
Literature reviews shows that diﬀerent heuristic and meta heuristic techniques
are used in electricity markets to predict load and price. The accurate forecasting
techniques are beneﬁcial for utility companies to enhance the stability in the
market. Management of supply and demand side by reducing the cost is also
depend on the forecasting models. Long term hourly load prediction for the
period of ﬁve years by LSTM-RNN is introduced in . Abedinia et al. 
shows the hourly solar energy load prediction of PV plant generation. In 
proposed a novel hybrid algorithm (ARIMA and NLSSVM) for simultaneous
prediction of load and price. The DSM schedule the connected moments of all
the shiftable devices in a smart grid to bring the load consumption curve close
to the objective load consumption curve.
In  hybrid model LSSVM with the combined kernal function is used for
price forecasting of Australian electricity market. A hybridize model of Phase
Space Reconstruction (PSR) with Bi-Square Kernel (BSK) regression model
called PSR-BSK model  . The PSR extract the evolutionary trends of power
system and the valuable features improve the forecasting procedure. The main
goal of electricity price and load prediction is to increase and decrease the power
generation of utility companies. The precise prediction of cost and load of elec-
tricity increase the competency and stability of utilities in the power markets
. The medium term load and price forecasting by Multi-block NN (Elman
NN) has high capability to reduce error and improve the training mechanism of
forecast process. It also shows that the hybrid topology present an accurate pre-
diction as compared to the prediction models . A day ahead price prediction
using Wavelet Packet Transform, Generalized Mutual Information for normal-
ization of data, Least Square Support Vector Machine for forecast engine and
Electricity Load and Price Forecasting Using Enhanced Machine ... 259
Artiﬁcial Bee Colony for optimization. (WRT, GMI, modiﬁed LSSVM, ABC)
in  Hybrid algorithm is applied on diﬀerent datasets of NY-ISO, PJM and
NSW for simultaneous and accurate peak hour’s prediction of load and price and
optimizes DSM in . The electricity load is predicted in microgrid scenario by
hybrid evolutionary fuzzy technique in . In , STLF method is eﬃcient
in managing the scheduling pattern and reducing cost of utility companies. In
, Wavelet Least Square Support Vector Machine (WLSSVM) model is used
to optimize by fruit ﬂy algorithm. The summary of related work is mentioned
in Table 2.
Table 2. Summary of related work
References Fore cast
Dataset Techniques Objectives Limitations
LSTM-RNN Hourly load
PJM CNN, LSTM Price
on natural gas
260 H. Bano et al.
The proposed scheme shows the ﬂow of the techniques which involve in the fore-
casting procedure of load and cost of electricity market. The electricity genera-
tion and consumption rate are greatly increasing by smart grids. These increasing
rates create serious problems for utility markets to manage the load and cost of
electricity changes. In the research ﬁeld diﬀerent machine learning techniques are
used to ﬁnd the nearer predictive cost and load of electricity provider companies.
The real time electricity dataset 2016–2017 per hour are collect from NY-
ISO which is known as one of the electricity provider company and manage the
wholesale energy markets of diﬀerent areas. The New York City hourly load and
cost are predict and store in a database for pre-processing. Preprocessing is a
machine learning technique which purify the dataset from redundancy, missing
values and quality of data. Data preprocessing is done by both the selection and
extraction techniques. The selection and extraction techniques are generally used
for preprocessing in machine learning techniques. The data set are divided into
training set and test set. 70% data is used for training and 30% for testing. In
the feature selection, two steps are involve by using CART and RFE techniques
to recognize the relevant and important features for forecasting purpose. SVD
technique is used for feature extraction to select the good quality of data however,
bagging is also used in extraction technique  . The main importance of feature
extraction is to reduce the dimensionality of dataset which directly aﬀect the
computational time of the system. The hourly data of NY-ISO are rearranged
to compatible with the classiﬁers (MLP, LR and SVM) to predict the electricity
load and price. After the completion of the forecasting engine the predicted
values of load and price are evaluated by four performance metrics i.e RMSE,
MSE, MAE and MAPE. The proposed system shows the ﬂow of forecasting
procedure of electricity load and price in Fig. 1.
Fig. 1. Proposed system model
Electricity Load and Price Forecasting Using Enhanced Machine ... 261
4 Techniques Description
In this section, all the techniques are described which are used to proposed
forecasting models for electricity load and price of NY-ISO New York market.
4.1 Data Preprocessing
The selection technique is mainly used to overcome the redundancy and
dimensionality of data which have low importance in the performance of pre-
The CART is a binary tree and information based learning algorithm. The
nodes split and the terminal leaf node contains an output which is used to
make prediction. This technique also investigate the relationship between
a dependent (target) and independent variables (predictor). It shows the
strength or importance impact of multiple independent variables on depen-
RFE commonly used to reduce the redundancy and discard the weak fea-
tures of data which has no eﬀect on the electricity prediction performance.
The RFE technique also overcomes the uninformative data from the dataset
and ﬁnd the prediction errors. In the algorithm the training set is used for
three purposes for predictor selection, model ﬁtting and performance evalua-
For further extracting the interested features and to reduce the high-
dimensional data into low-dimensional data, extraction process is done by
SVD is a numerical technique based on simple linear algebra. This technique is
a factorization or decomposing a complex matrix to reduce the dimensionality
of dataset features. Classiﬁcation and selection techniques improve forecasting
accuracy and simplify the classiﬁer complexity.
MLP is a deep learning technique and is the class of Feed-Forward Artiﬁcial
Neural Network (FFANN). It consist of three layer; input, hidden and output
layers. Each neuron has its own activation function. The input layer has no
activation function just introduced the input values. Hidden layers perform
the classiﬁcation of features obtained from the input layer and the output
layer produce output through an activation function. MLP is fully connected,
each node in one layer connected with certain weights to every node in the
The enhancement in MLP is done by tuning the parameters. Three types
of solvers ‘lbfgs, sgd and adam’ are used for weight optimization in MLP
classiﬁer. In EMLP lbfgs solver gives better results than other two optimizers.
262 H. Bano et al.
SVM is a supervised machine learning algorithm which can be used for both
classiﬁcation and regression methods. The SVM linearly separable binary
sets of data. The objective to design a hyperplane in between the number
of features to classiﬁes all training vectors in two classes. Many hyperplanes
could be chosen to separate the two classes of data points. The plane has
the maximum margin distance provides future data points can be classiﬁed.
The eﬀectiveness of SVM depend on the high dimentional spaces of support
vectors (data points near to hyperplane) to the hyperplanes.
LR is a statistical method which is used for predictive analysis. When one
or more independent variables exist in the dataset the logistic function (sig-
moid function) analyze the outcomes. The sigmoid function was developed to
describe the properties of population growth. In the plots the sigmoid func-
tion is represented by S-shape curve which take any real number and map it
into a value 0 and 1. This machine learning technique can be used for both
binary or multivariate classiﬁcation tasks.
Algorithm 1 MLP
1: Set MLP Network (mlp signals, weights and activation function)
2: Summation of signals with weights
3: Load and normalized the dataset
4: Splitting the dataset (training and testing)
5: Select training data
6: FOR nepochs and batchsize
7: Train the Network
9: Run Prediction using Network
10: Calculate the loss function
Algorithm 2 SVM
1: Require: X and y loaded with training labeled data, a = 0
2: partially trained SVM
3: C = some value (for example 10)
5: FOR (xi, yi),(xj, yj )do
6: Optimize ai and aj
8: until no change in a or other resource constraint criteria met
9: Ensure: Retain only the support vectors (ai > 0)
Electricity Load and Price Forecasting Using Enhanced Machine ... 263
5 Simulation and Results
In this paper the prediction of electricity load and price by using MLP, LR and
SVM Classiﬁers. The system type x64-based processor, 4.00 GB RAM and Intel
(R) core i5 processor. The implementation environment is in anaconda (spyder)
by using python language.
The four months (ﬁrst two months of 2016 and 2017) hourly electricity data
of New York are taken as input from NY-ISO wholesale electricity provider. The
diﬀerent features for example humidity, temperature, pressure, wind speed and
wind direction impact on the target load (TWIActualLoad) and target price
Figure 2shows the importance of features of a dataset in forecasting process.
Here we can see that the zonal transmission losses has more important role than
zonal price version in load forecasting. The features play an important role in
forecasting process, after the pre-processing of data, the normalized data of four
months is shown in Fig. 3.
Fig. 2. Feature importance of load Fig. 3. Normalized load data
Prediction of load and price of electricity market by using MLP, SVM, LR
with EMLP, ESVM and ELR techniques is shown in the following plots. The
hourly dayahead load with MLP and EMLP model and price prediction with
LR and ELR are shown in Figs. 4and 5.
Fig. 4. Dayahead load prediction Fig. 5. Dayahead price prediction
264 H. Bano et al.
The electricity load and price of ﬁrst week, January 2016 and four months
of electricity load with MLP and EMLP techniques are shown in Fig. 6.
Fig. 6. a Load prediction of 1st-week, bload prediction of January, celectricity load
of four months
Electricity price prediction of week ahead and month based are shown in
Fig. 7. a Price prediction of 1st-week, bprice prediction of January, cprice prediction
of four month
5.1 Performance Evaluation
Diﬀerent performance evaluators are used in this section for evaluation of the
simulation results. In particular four metrics are used, which include, Mean
Absolute Error (MAE), Mean Squared Error (MSE), Root Mean Square Error
(RMSE), and Mean Absolute Percentage Error (MAPE). Figures 8and 9show
the performance graph of electricity load and price data.
Fig. 8. Performance matrix of load Fig. 9. Performance matrix of price
Electricity Load and Price Forecasting Using Enhanced Machine ... 265
Table 3shows the performance evaluation comparison of diﬀerent techniques
which are used for electricity load and price prediction of New York market by
using Eqs. 1–4.
Table 3. The diﬀerent evaluation criterion of price and load prediction
Model Criterion Price error values Load error values
266 H. Bano et al.
In this paper, MLP, EMLP, SVM, ESVM, LR and ELR techniques are used
for the time series hourly prediction of load and price of NY-ISO electricity
market. Feature selection and extraction techniques (CART, RFE and SVD)
are used for the normalization of the dataset. These normalization techniques
remove the redundancy and irrelevant features which have less impact on the
forecasting process. The high dimensionality reduction of the dataset improved
the computation of the system and reduces cost. The LSTM and MLP models
presents clear representation of load and price forecast. In our scenario LSTM
model performs better prediction than MLP. Due to this problem, we proposed
enhanced MLP model for load and price prediction. The three techniques MLP,
SVM and LR gives 65, 64.85 and 82.62% accuracy for price prediction while
54.58, 68.71 and 67.47% for load prediction. The EMLP, ESVM and ELR models
present 76.30, 70.85, 84.52 and 79.40% and 65.01, 70.40 and 67.78% accuracy
for price and load prediction. These accurate prediction performance models
maximize the proﬁts and formulate the long term strategies for utility companies.
1. Jindal, A., Singh, M., Kumar, N.: Consumption-aware data analytical demand
response scheme for peak load reduction in smart grid. IEEE Trans. Ind. Electron.
65, 8993–9004 (2018)
2. Liu, C., Jin, Z., Gu, J., Qiu, C.: Short-term load forecasting using a long short-
term memory network. In: Innovative Smart Grid Technologies Conference Europe
(ISGT-Europe), 2017 IEEE PES, pp. 1–6. IEEE (2017)
3. Zheng, J., Xu, C., Zhang, Z., Li, X.: Electric load forecasting in smart grids using
long-short-term-memory based recurrent neural network. In: 2017 51st Annual
Conference on Information Sciences and Systems (CISS), pp. 1–6. IEEE (2017)
4. Wang, F., Li, ., Zhou, L., Ren, H., Contreras, J., Shaﬁe-Khah, M., Catal˜ao, J.P.:
Daily pattern prediction based classiﬁcation modeling approach for day-ahead elec-
tricity price forecasting. Int. J. Electr. Power Energy Syst. 105, 529–540 (2019)
5. Raviv, E., Bouwman, K.E., van Dijk, D.: Forecasting day-ahead electricity prices:
utilizing hourly prices. Energy Econ. 50, 227–239 (2015)
6. Mosbah, H., El-Hawary, M.: Hourly electricity price forecasting for the next month
using multilayer neural network. Can. J. Electr. Comput. Eng. 39(4), 283–291
7. Wang, K., Xu, C., Zhang, Y., Guo, S., Zomaya, A.: Robust big data analytics for
electricity price forecasting in the smart grid. IEEE Trans. Big Data (2017)
8. Chahkoutahi, F., Khashei, M.: A seasonal direct optimal hybrid model of compu-
tational intelligence and soft computing techniques for electricity load forecasting.
Energy 140, 988–1004 (2017)
9. Ahmad, A., Javaid, N., Guizani, M., Alrajeh, N., Khan, Z.A.:. An accurate and
fast converging short-term load forecasting model for industrial applications in a
smart grid. IEEE Trans. Ind. Inf. 13(5), 2587–2596 (2017)
10. Liu, J.P., Li, C.L.: The short-term power load forecasting based on sperm whale
algorithm and wavelet least square support vector machine with DWT-IR for fea-
ture selection. Sustainability 9(7), 1188 (2017)
Electricity Load and Price Forecasting Using Enhanced Machine ... 267
11. Raﬁei, M., Niknam, T., Khooban, M.-H.: Probabilistic forecasting of hourly elec-
tricity price by generalization of ELM for usage in improved wavelet neural net-
work. IEEE Trans. Ind. Inform. 13(1), 71–79 (2017)
12. Keles, D., Scelle, J., Paraschiv, F., Fichtner, W.: Extended forecast methods for
day-ahead electricity spot prices applying artiﬁcial neural networks. Appl. Energy
162, 218–230 (2016)
13. Feng, C., Cui, M., Hodge, B.-M., Zhang, J.: A data-driven multi-model methodol-
ogy with deep feature selection for short-term wind forecasting. Appl. Energy 190,
14. Wang, H.Z., Wang, G.B., Li, G.Q., Peng, J.C., Liu, Y.T.: Deep belief network based
deterministic and probabilistic wind speed forecasting approach. Appl. Energy 182,
15. Wang, K., Xu, C., Zhang, Y., Guo, S., Zomaya, A.: Robust big data analytics for
electricity price forecasting in the smart grid. IEEE Trans. Big Data (2017)
16. Agrawal, R.K., Muchahary, F., Tripathi, M.M.: Long term load forecasting with
hourly predictions based on long-short-term-memory networks. In: Texas Power
and Energy Conference (TPEC), 2018 IEEE, pp. 1–6. IEEE (2018)
17. Abedinia, O., Amjady, N., Ghadimi, N.: Solar energy forecasting based on hybrid
neural network and improved metaheuristic algorithm. Comput. Intell. 34(1), 241–
18. Kuo, P.-H., Huang, C.-J.: An electricity price forecasting model by hybrid struc-
tured deep neural networks. Sustainability 10(4), 1280 (2018)
19. Chen, Y., Li, M., Yang, Y., Li, C., Li, Y., Li, L.: A hybrid model for electricity
price forecasting based on least square support vector machines with combined
kernel. J. Renew. Sustain. Energy 10(5), 055502 (2018)
20. Liu, Y., Wang, W., Ghadimi, N.: Electricity load forecasting by an improved fore-
cast engine for building level consumers. Energy 139, 18–30 (2017)
21. Gao, W., Darvishan, A., Toghani, M., Mohammadi, M., Abedinia, O., Ghadimi, N.:
Diﬀerent states of multi-block based forecast engine for price and load prediction.
Int. J. Electr. Power Energy Syst. 104, 423–435 (2019)
22. Shayeghi, H., Ghasemi, A., Moradzadeh, M., Nooshyar, M.: Day-ahead electricity
price forecasting using WPT, GMI and modiﬁed LSSVM-based S-OLABC algo-
rithm. Soft Comput. 21(2), 525–541 (2017)
23. Khwaja, A.S., Naeem, M., Anpalagan, A., Venetsanopoulos, A., Venkatesh, B.:
Improved short-term load forecasting using bagged neural networks. Electric Power
Syst. Res. 125, 109–115 (2015)
24. Ghasemi, A., Shayeghi, H., Moradzadeh, M., Nooshyar, M.: A novel hybrid algo-
rithm for electricity price and load forecasting in smart grids with demand-side
management. Appl. Energy 177, 40–59 (2016)
25. Coelho, V.N., Coelho, I.M., Coelho, B.N., Reis, A.J.R., Enayatifar, R., Souza,
M.J.F., Guimar˜aes, F.G.: A self-adaptive evolutionary fuzzy model for load fore-
casting problems on smart grid environment. Appl. Energy 169, 567–584 (2016)
26. Tarsitano, A., Amerise, I.L.: Short-term load forecasting using a two-stage sarimax
model. Energy 133, 108–114 (2017)
27. Dongxiao, N., Tiannan, M., Bingyi, L.: Power load forecasting by wavelet least
squares support vector machine with improved fruit ﬂy optimization algorithm. J.
Comb. Optim. 33(3), 1122–1143 (2017)