Short Term Load and Price Forecasting using Tuned Parameters for K-Nearest Neighbors

Abstract

Short term electricity load and price accurate fore-casting are the key areas that need to be addressed in Smart Grids (SG). In this paper, a new model is proposed for accurate short term forecasting of load and price. First, irrelevant features are removed using Recursive Feature Elimination (RFE) and Decision Tree Regressor (DTR). Then, further dimensionality reduction is done by removing redundancy using Singular Value Decomposition (SVD) for feature extraction. Finally, remaining features are given to Enhanced K-Nearest Neighbor (EKNN)for forecasting of both load and price. The proposed EKNN achieved better results than Support Vector Regressor (SVR)and normal K-Nearest Neighbors (KNN). EKNN achieved 12%more accuracy in terms of forecasting than normal KNN and63% more than SVR.

Full text

1
Short Term Load and Price Forecasting using Tuned
Parameters for K-Nearest Neighbors
Sohail Khan1, Zahoor Ali Khan2, Zainib Noshad1, Sakeena Javaid1and Nadeem Javaid 1,∗
1COMSATS University Islamabad, Islamabad 44000, Pakistan
2CIS, Higher Colleges of Technology, Fujairah 4114, United Arab Emirates
*Correspondence: nadeemjavaidqau@gmail.com, www.njavaid.com
Abstract—Short term electricity load and price accurate fore-
casting are the key areas that need to be addressed in Smart
Grids (SG). In this paper, a new model is proposed for accurate
short term forecasting of load and price. First, irrelevant features
are removed using Recursive Feature Elimination (RFE) and
Decision Tree Regressor (DTR). Then, further dimensionality
reduction is done by removing redundancy using Singular Value
Decomposition (SVD) for feature extraction. Finally, remaining
features are given to Enhanced K-Nearest Neighbor (EKNN)
for forecasting of both load and price. The proposed EKNN
achieved better results than Support Vector Regressor (SVR)
and normal K-Nearest Neighbors (KNN). EKNN achieved 12%
more accuracy in terms of forecasting than normal KNN and
63% more than SVR.
Index Terms—Smart Grid, Feature Selection, Feature Extrac-
tion, Load and Price Forecasting, K-Nearest Neighbors, Demand-
side Management.
I. INTRODUCTION
Traditional grids of today are not capable of maintaining
consumer satisfaction and providing cost effective and efficient
energy overall. They are a century old system with completely
different view of environment. Specially, now with governments
going forward for environment control by controlling
CO2
emissions and getting their focus towards renewable energy. A
new system is needed that can solve these occurring problems.
Smart Grids (SG) is a new electricity management digital
system that utilizes the use of smart meters and sensor’s data
to help utilities in making smart decisions. SG can not only
collect the data of an area, it can even get a single home’s
energy consumption in real time. That kind of data gives new
insights into how people use energy, what appliances are used
more often than others, what similar patterns occur among
different households, areas, or even cities. Data generated from
smart meters and sensors can be used in making predictions
1
of
energy consumption by performing data analyses. Utilities can
use that information for determining how much energy would
be required for some foreseeable future. That information helps
utilities by minimizing their generation costs, while giving
them more profit. SG generates data on such a large scale
that it is getting more and more difficult to perform data
analyses on it. The computational capabilities we have are
limited and cannot coupe with such a large amount of data.
Therefore, some smart and efficient techniques are needed for
utilizing that data quickly and accurately, since data loses its
1Prediction and Forecasting are used interchangeably in this paper
relevance with time. Many techniques have been proposed over
the last years, both statistical and machine learning. Statistical
methods such as different variants of Autoregressive Integrated
Moving Average (ARIMA) have been used for forecasting[
1
]
and [
2
]. In machine learning, techniques such as SVM [
3
]-
[
4
], which is a very popular technique. Neural Network (NN)
[
1
], [
5
]-[
6
], has gained a lot of popularity over the last years
in machine learning, is also used for forecasting. Techniques
such as Long-Short Term Memor (LSTM) [
7
], Feed Forward
Neural Network (FFNN) [
8
] and [
9
], Wavelet Neural Network
(WNN) [
5
], are popular techniques in neural network. Other
techniques such as Quantile Smoothing Spline Regression
(QSSR) [
10
], ENSEMBLE [
11
], Improvised Rolling Horizon
(IRH) [
12
], Inverse Optimization (IO) [
13
], Quantile Score and
Bootstrap Method (QSBM) [
14
], Kernel Function (KF) [
15
],
Ridge Regression (RR) [
16
], Quantile Regression Averaging
(QRA) [
17
], Bayesian Network (BN) [
18
], Generalize Neuron
(GN) [
19
], K-means [
20
], Maximizing Relevancy, Minimizing
Redundancy, and Maximizing Synergy (MRMRMS) [
21
] are
also popular techniques in machine learning.
Huge volume of data is generated by smart meters, which
needs to be examined in a sophisticated manner. Utilities
cannot store the vast amount of energy generated, therefore,
most of it is wasted. They need to know the approximate
load forecasting in order to make plans for the generation
of energy and maintenance of their resources. An accurate
estimate of future consumption helps utilities maintain a balance
between generation and consumption. Many techniques have
been proposed over the last years for load forecasting. KNN is
one of the most simple classification technique [
22
] that uses
’K’ nearest neighbors to predict the value of new data. However,
KNN has to be checked first for the value of ’K’ and also
the distance function that it uses. The most common distance
function used is the Euclidean distance function [
22
]. Support
Vector Machine (SVM) is another very popular technique used
for forecasting, however, the problem with SVM is that it
requires tuning of its parameters to predict accurately [
23
]-[
4
].
In this paper, we have proposed a simple framework that
utilizes KNN regressor with best value for ’K’ after many
simulations and also utilizes the best distance function. This
framework eliminates the irrelevant features first by using
Recursive Feature Elimination (RFE) with Decision Tree
Regressor (DTR). It also removes redundancy among features
by using Singular Value Decomposition (SVD) analysis for
feature extraction.

2
II. PRO PO SE D SYS TE M MOD EL
The proposed scheme comprises of 4 steps; preprocessing of
the dataset, splitting the data into training and testing, forecast-
ing using test data, and checking performance evaluation using
Mean Absolute Percentage Error (MAPE), Mean Absolute Error
(MAE), Mean Squared Error (MSE), Root Mean Squared Error
(RMES) and Friedman statistical test. Proposed system model
is shown in Fig. 1.
A. Preprocessing of Dataset
Dataset is acquired from New York Independent System
Operator (NYISO) containing hourly data of January and
February. Dataset is normalized first to be passed for feature
selection. Floating point values are not accepted by some
techniques, which is why data must be normalized to an integer
value. After normalization of data is done, target attribute is
taken out of the feature set to be passed as a parameter for
RFE with DTR. RFE recursively goes through each feature
and determines its relationship with the target attribute by
using DTR as a model. After irrelevant features are removed
from the dataset, remaining features are passed to SVD for
further dimensionality reduction. SVD treats the features set
as a matrix and divides it into 3 different matrices and finds
features that have a high correlation with each other. After the
process is done, a reduced feature set is returned.
B. Training and Forecasting
After preprocessing is done, data is split into training and
testing. Enhanced K-Nearest Neighbors (EKNN) model is
trained using the training data first. Testing data is then used for
forecasting. After acquiring the results, different performance
evaluators are used for evaluation of the proposed model.
C. EKNN
EKNN is a KNN based technique that has been derived
in this paper. KNN is a classification technique that uses ’K’
nearest neighbors to classify new data. KNN requires a value
for ’K’ which can be any integer and a distance function to
compute the distance between values to determine the neighbors.
One of the most important challenges in KNN is to select the
most accurate value of ’K’ for any given dataset. One common
approach is to select square root of the total number records.
However, it needs careful examination, otherwise, it can lead to
over fitting. KNN is considered to be a lazy learning algorithm,
i.e, it does not require any prior training, it calculates the
distance of each value every time a new test data is introduced.
Another problem with KNN is the distance function to use.
Mostly, that depends on the dataset that is being used. There are
multiple types of data, categorical, numerical, and combination
of both. If the data has many dimensions, it is often difficult for
KNN to make accurate predictions. This issue in KNN is called
the
curseof dimensionality
. However, we have eliminated
some of the issues in our new proposed technique. Best value
for ’K’ is selected after checking the error rate for different
values. Distance function has also been checked for accurate
predictions.
D. Proposed Models Algorithm
Following steps are followed in the proposed model that
utilize all above mentioned methods:
1. Data is normalized using minmax normalization.
2. Relevant features are selected using a new feature selector
that is combination of RFE and DTR.
3. Two subsets are made from data, 70% for training and 30%
for testing.
4. The EKNN model is trained on the training data and its
performance is evaluated on the test data.
III. SIMULATION RES ULTS A ND SUMMARY
This section covers the simulations and discusses the results
of proposed technique. The dataset used is from NYISO that
recorded the hourly data from January 2016 to February 2016.
The simulations are performed on a system having processor
core i3 4th generation 1.7Gh and 4GB of RAM. Data is split
into training and testing batch. Where 1296 instances used for
training and 144 for testing.
A. Feature Selection using RFE with DTR
Feature selection is performed on the dataset. A total of
8 features are passed for feature selection. DTR is used to
determine the feature importance. And then, DTR is passed as
a model to RFE for finding the features ranking and support
values. RFE with DTR is applied on the features to recursively
remove the features that have a minimal relation with the target
attribute, which in this dataset is ’TWI Zonal LBMP’ for price
and ’Load’ for load. Fig. 2 shows the importance of features
for price data. As visible from the graph only 2 features are
relevant against the target attribute. Fig. 3 shows the importance
of features for load data.
B. Feature extraction using SVD
Further dimensions are reduced by using SVD on the
remaining features we got after feature selection. SVD removes
redundancy from the features to ensure high accuracy during
prediction.
C. Predicting using EKNN
Final features set is then provided to our proposed model
for prediction with tuned parameters. Fig. 4 shows the plot
results of our model against the SVR and KNN for price. This
graphical representation shows that EKNN is close to actual
values as compared to the SVR and KNN. Whereas, SVR
is shown the mean results. Fig. 5 shows the plot results of
our proposed model against SVR and KNN for load and our
proposed technique outperforms other two techniques.
D. Performance Evaluation
Different performance evaluators are used in this section for
evaluation of results. In particular, 5 metrics are used, which
include, MAE, MSE, RMSE, MAPE and Friedman statistical
test as shown in Table I. These parameters are widely used
for assessment of prediction model’s performance. The first

3
Fig. 1: System Model
TABLE I: Performance metrics of load and price forecasting
Price Forecasting Load Forecasting
Model MAE MSE RMSE MAPE
Friedman
Score
MAE
MSE RMSE MAPE
Friedman
Score
SVR 84.24
69.15
92.97 68.81 2
50.52 27.67
74.39 68.72 3
KNN 84.02
42.32
58.78 10.50 3
58.76 25.00
55.90 16.48 2
EKNN 55.84
31.06
41.54 2.37 1
18.99
2.49 35.35 4.52 1
Fig. 2: Feature Importances for Price Fig. 3: Feature Importance for Load

4
Fig. 4: Comparison Graph of our Model for Price
Fig. 5: Comparison Graph of our Model for Load
Fig. 6: Performance metrics after price forecasting
four aforementioned metrics are error measures that show
the error rate of model. The Friedman statistical test shows
that a prediction model has statistically significant accuracy
improvement as compared to the comparable models. It assigns
a score to all compared models, where score 1 means the model
is best, 2 means second best and so on. The table shows that
minimum MSE for price and load is shown by KNN. Fig. 6
shows the performance graph for price forecasting and Fig.
7 shows the performance graph for load forecasting. These
Fig. 7: Performance metrics after load forecasting
graphs show that KNN out performs other three techniques
for price and load forecasting as minimum MES ad MPAE is
shown by KNN.
IV. CONCLUSION
In this work, both price and load forecasting are performed.
EKNN is proposed along with feature selection and extraction
techniques. The parameters of KNN are tuned to enhance their
performance. RFE with DTR is used for feature selection to
provide better accuracy rate. Further dimensions are reduced
using SVD to remove the redundant features. Results are then
compared with different techniques. The proposed technique
has outperformed SVR and k-NN. EKNN achieved 12% better
accuracy than KNN and 13% more than SVR.
REFERENCES
[1]
B. Stephen, X. Tang, P. R. Harvey, S. Galloway, and K. I. Jennett,
“Incorporating practice theory in sub-profile models for short term
aggregated residential load forecasting,” IEEE Transactions on Smart
Grid, vol. 8, no. 4, pp. 1591–1598, 2017.
[2]
S. ZAKARYA, H. ABBAS, and M. BELAL, “Long-term deep learning
load forecasting based on social and economic factors in the kuwait
region.” Journal of Theoretical & Applied Information Technology,
vol. 95, no. 7, 2017.
[3]
A. Tajer, “Load forecasting via diversified state prediction in multi-area
power networks,” IEEE Transactions on Smart Grid, vol. 8, no. 6, pp.
2675–2684, 2017.
[4]
H. Jiang, Y. Zhang, E. Muljadi, J. J. Zhang, and D. W. Gao, “A short-term
and high-resolution distribution system load forecasting approach using
support vector regression with hybrid parameters optimization,” IEEE
Transactions on Smart Grid, vol. 9, no. 4, pp. 3341–3350, 2018.
[5]
M. Rafiei, T. Niknam, J. Aghaei, M. Shafie-khah, and J. P. Catal
˜
ao,
“Probabilistic load forecasting using an improved wavelet neural network
trained by generalized extreme learning machine,” IEEE Transactions
on Smart Grid, 2018.
[6]
M. Rafiei, T. Niknam, and M.-H. Khooban, “Probabilistic forecasting of
hourly electricity price by generalization of elm for usage in improved
wavelet neural network,” IEEE Transactions on Industrial Informatics,
vol. 13, no. 1, pp. 71–79, 2017.
[7]
W. Kong, Z. Y. Dong, D. J. Hill, F. Luo, and Y. Xu, “Short-term residential
load forecasting based on resident behaviour learning,” IEEE Transactions
on Power Systems, vol. 33, no. 1, pp. 1087–1088, 2018.
[8]
J. Xie, Y. Chen, T. Hong, and T. D. Laing, “Relative humidity for load
forecasting models,” IEEE Transactions on Smart Grid, vol. 9, no. 1, pp.
191–198, 2018.
[9]
F. Y. Xu, X. Cun, M. Yan, H. Yuan, Y. Wang, and L. L. Lai, “Power
market load forecasting on neural network with beneficial correlated
regularization,” IEEE Transactions on Industrial Informatics, 2018.

5
[10]
A. Gerossier, R. Girard, G. Kariniotakis, and A. Michiorri, “Probabilistic
day-ahead forecasting of household electricity demand,” CIRED-Open
Access Proceedings Journal, vol. 2017, no. 1, pp. 2500–2504, 2017.
[11]
Y. Wang, Q. Chen, M. Sun, C. Kang, and Q. Xia, “An ensemble
forecasting method for the aggregated load with sub profiles,” IEEE
Transactions on Smart Grid, 2018.
[12]
H. Chitsaz, P. Zamani-Dehkordi, H. Zareipour, and P. P. Parikh,
“Electricity price forecasting for operational scheduling of behind-the-
meter storage systems,” IEEE Transactions on Smart Grid, vol. 9, no. 6,
pp. 6612–6622, 2018.
[13]
J. Saez-Gallego and J. M. Morales, “Short-term forecasting of price-
responsive loads using inverse optimization,” IEEE Transactions on Smart
Grid, vol. 9, no. 5, pp. 4805–4814, 2018.
[14]
J. Xie and T. Hong, “Temperature scenario generation for probabilistic
load forecasting,” IEEE Transactions on Smart Grid, vol. 9, no. 3, pp.
1680–1687, 2018.
[15]
J.-B. Fiot and F. Dinuzzo, “Electricity demand forecasting by multi-task
learning,” IEEE Transactions on Smart Grid, vol. 9, no. 2, pp. 544–551,
2018.
[16]
C.-N. Yu, P. Mirowski, and T. K. Ho, “A sparse coding approach
to household electricity demand forecasting in smart grids,” IEEE
Transactions on Smart Grid, vol. 8, no. 2, pp. 738–748, 2017.
[17]
B. Liu, J. Nowotarski, T. Hong, and R. Weron, “Probabilistic load
forecasting via quantile regression averaging on sister forecasts,” IEEE
Transactions on Smart Grid, vol. 8, no. 2, pp. 730–737, 2017.
[18]
N. Bassamzadeh and R. Ghanem, “Multiscale stochastic prediction of
electricity demand in smart grids using bayesian networks,” Applied
energy, vol. 193, pp. 369–380, 2017.
[19]
N. Singh, S. R. Mohanty, and R. D. Shukla, “Short term electricity price
forecast based on environmentally adapted generalized neuron,” Energy,
vol. 125, pp. 127–139, 2017.
[20]
X. Dong, L. Qian, and L. Huang, “Short-term load forecasting in smart
grid: A combined cnn and k-means clustering approach,” in Big Data
and Smart Computing (BigComp), 2017 IEEE International Conference
on. IEEE, 2017, pp. 119–125.
[21]
O. Abedinia, N. Amjady, and H. Zareipour, “A new feature selection
technique for load and price forecast of electrical power systems,” IEEE
Transactions on Power Systems, vol. 32, no. 1, pp. 62–74, 2017.
[22]
F. Wahid and D. Kim, “A prediction approach for demand analysis of
energy consumption using k-nearest neighbor in residential buildings,”
International Journal of Smart Home, vol. 10, no. 2, pp. 97–108, 2016.
[23]
P. Vrablecov
´
a, A. B. Ezzeddine, V. Rozinajov
´
a, S.
ˇ
S
´
arik, and A. K.
Sangaiah, “Smart grid load forecasting using online support vector
regression,” Computers & Electrical Engineering, vol. 65, pp. 102–117,
2018.