ChapterPDF Available

An Approximate Forecasting of Electricity Load and Price of a Smart Home Using Nearest Neighbor

Authors:

Abstract and Figures

In Smart Grid, electricity demand and price forecasting literature has focused on Industrial, Buildings, and Residential sector demand, but this paper focuses on short term electricity demand and price forecasting for residential customer. Here we take smart meter data of hourly based from a smart home. First standardize and selected important features by using Recursive Feature Elimination with Linear Support Vector Classifier (RFE-LSVC). Second, do forecasting through K-Nearest Neighbors (KNN), Decision Tree (DT), Random Forest (RF) and Support Vector Regression (SVR) models and perform comparative analysis among models against four scenarios and provided best solution among all for individual scenario. This work proposed best solution of smart home’s load and price forecasting for smart grid to manage demand response efficiently. We evaluated every Models with Mean Absolute Percentage Error (MAPE).
Content may be subject to copyright.
An Approximate Forecasting
of Electricity Load and Price of a Smart
Home Using Nearest Neighbor
Muhammad Nawaz1, Nadeem Javaid1(B
), Fakhar Ullah Mangla2,
Maria Munir2,FarwaIhsan
2, Atia Javaid1, and Muhammad Asif3
1COMSATS University Islamabad, Islamabad 44000, Pakistan
nawazkhan.cui2018@gmail.com, nadeemjavaidqau@gmail.com
2University of Sargodha, Sargodha 40100, Pakistan
fakhar.mangla@uos.edu.pk, mariamunir.uos2016@gmail.com,
farwaihsan.uos2016@gmail.com
3The Islamia University of Bahawalpur, Bahawalpur 63100, Pakistan
muhammadasifkhancs@gmail.com
http://www.njavaid.com/
Abstract. In Smart Grid, electricity demand and price forecasting lit-
erature has focused on Industrial, Buildings, and Residential sector
demand, but this paper focuses on short term electricity demand and
price forecasting for residential customer. Here we take smart meter
data of hourly based from a smart home. First standardize and selected
important features by using Recursive Feature Elimination with Linear
Support Vector Classifier (RFE-LSVC). Second, do forecasting through
K-Nearest Neighbors (KNN), Decision Tree (DT), Random Forest (RF)
and Support Vector Regression (SVR) models and perform comparative
analysis among models against four scenarios and provided best solution
among all for individual scenario. This work proposed best solution of
smart home’s load and price forecasting for smart grid to manage demand
response efficiently. We evaluated every Models with Mean Absolute Per-
centage Error (MAPE).
1 Introduction
Electricity demand is increasing by the ever increasing global population. Devel-
oping countries governments are shifting their Traditional Grid (TG) to Smart
Grid (SG) for efficient demand response of electricity. The smart grid is modern
electric grid that intelligently and efficiently manages the power generation, dis-
tribution and consumption of electricity by introducing such type of technologies
which enables two way communication between utility and consumer of electric-
ity and also focus on satisfaction of consumer. By 2020, the EU aims to take
the place of 80% traditional electric meters with smart meters to support the
objectives of control increasing electricity prices and for comfort of consumer [3].
The Advanced Metering Infrastructures (AMIs) consists of Smart-Meters (SMs)
c
Springer Nature Switzerland AG 2020
L. Barolli et al. (Eds.): CISIS 2019, AISC 993, pp. 521–533, 2020.
https://doi.org/10.1007/978-3-030-22354-0_46
522 M. Nawaz et al.
is main functional technological innovation in SG domain which function for
Demand-Response (DR) model in which energy demand through individual SM
is based on consumer usage demand and in parallel provides response for energy
off peak hours or on peak hours from utilities. In SG, utility manage electric-
ity demand from SM of Smart-Homes (SHs) so efficiently as remain minimum
burden on the SG [6].
To manage the DR of the SH, Electricity markets get benefits form load
forecasting on the base of consumer usage pattern data gathered from their
SM. This load forecasting becomes base for several important decisions such as
price for specific hours, power generation schedules to fulfill demand and using
source of power generations. Now price forecasting is also crucial and beneficial
for consumers and energy market participants for auction strategies formulation
and speculation planning. From price forecasting, consumer get benefit of low
electricity price as to shift their load from on peak hours to off peak hours.
Through price forecasting, electricity producers can maximize their profit and
consumers can minimize the cost of their electricity usage.
AMIs main function of dual connection between utility and consumer bring
much benefits for both consumers and utilities. Consumer is now aware of his
load used by individual appliance and contribute to shift load from on peak
hours to off peak hours for getting compensations from utility. Utility on the
other hand also get benefits from the goal of AMIs mentioned as:
Electricity buying decisions on consumer’s hand.
Customer satisfaction in selecting best service provider.
Easiness of electricity theft detection.
Minimize electricity price cost.
Minimum error in electricity bills due to accurate calculation and two way
involvement.
In SHs installed AMI, consumer is well known of every appliance’s consumed
electricity in every moment. Utility provide options to consumer for different
prices against specific hours and this is possible only through precise prediction
of load and price against it. Consumer take decisions for usage of electricity and
totally aware of expected bill for that month.
1.1 Motivation
After reviewing different forecasting techniques for different scenario in litera-
ture, the following is motivation of our work.
Day-ahead load and price forecasting for individual SH is not taken in con-
sideration. Literature take into account mostly residential or market data for
short term load forecasting. Very Short Term Load and Price Forecasting
(VSTLPF) and Short Term Load and Price Forecasting (STLPF) for indi-
vidual SH capable SG to check individual’s patterns [3]. With VSTLPF and
STLPF, SG will be able to know one-hour, twelve-hours, one-day and one-
week ahead load demand from SH and efficiently manage DR.
Forecasting of Electricity Load and Price 523
Classifiers like Artificial Neural Networks (ANN), Support Vector Machine
(SVM) and Wavelet Transform (WT) etc. have low generalization potential
therefore have an over-fitting problems when they are used.
Data preprocessing as noise removal, Feature Importance (FI), Feature Selec-
tion (FS) and Feature Extraction (FE) through Decision Tree (DT), Recursive
Feature Elimination (RFE) etc. used on being classified data, become base
for good accuracy of forecasting classifier.
1.2 Related Work
For load and price forecasting in SG, there are several techniques discussed in
literature. Electricity consumption and forecasting is divided into four main cat-
egories [3], as: (1) Very Short Term Load and Price Forecasting (VSTLPF),
(2) Short Term Load and Price Forecasting (STLPF), (3) Medium Term Load
and Price Forecasting (MTLPF) and (4) Long Term Load and Price Forecasting
(LTLPF). These categories are based on their time differences (e.g. minute-to-
minute, hour-to-hour, day-to-day, week-to-week, year-to-year, etc.) for calcula-
tion and forecasting of load. There is also a category of forecasting techniques for
these type of load data. In different scenario researcher use different techniques as
Random Forest (RF), Neural Networks (NNs) and Particle Swam Optimizations
(PSO).
In paper [17], using EMD-LSTM NN with XGboost as feature importance,
performs better in short term load forecasting than without XGboost. Author
implemented Stacked Denoising Auto encoders (SDA) model for day-ahead
hourly forecasting in [16] and get better result with extended SDA. For con-
tinuously day-ahead price forecasting is also done by consideration of market
integration in [7] by using deep neural network and novel feature selection algo-
rithm. They focus only two markets and increase 3.2% accuracy. [12] predicted
day-ahead price as hourly base for market.Author predict 24h individual price
and reduce Root Mean Squared Error 16%. Feature Selection (FS) is important
process for accurate forecasting. In [1], Author proposed new techniques of fea-
ture selection for accurate forecasting of load and price in SG. Author reduce
redundancy using information theoretic criteria and hybrid filter-wrapper app-
roach. Short term load forecasting on five year data from university campus
is done by using moving average method and random forest method in [11]to
get better accuracy than using only random forest as classifier. Author in [4]
calculate short-term load forecasting but in large data by using combination of
convolutional neural network with K-means algorithm. They perform forecasting
as clustered large data into subsets using K-means and training convolutional
neural network on that subsets.
SG consists of a large data and for forecasting many excellent classifier cannot
perform well due to redundancy in features. So it is most important to be vary
clean and preprocessed a data which is going to use for classification. Author in
[15] use Random Forest (RF) and Relief-F based on Gray Correlation Analysis
(GCA) for feature selection and Kernel Function (KF) and Principle Component
524 M. Nawaz et al.
Analysis (PCA) for feature Extraction to get better forecasting performance of
classifier. Short term load forecasting by using Integrated Intelligent Energy
Management process is proposed in [2] where 56 different scenarios are checked
with compared their results. Author in [8] forecast load of electricity of building
consumer with Sliding Window Empirical Mode Decomposition (SWEMD) fea-
ture selection and Improved Elman Neural Network (IENN) for classification. In
[10] for short term residential load forecasting, Author compared result of Regres-
sion Trees (RT), Neural Networks (NN) and Support Vector Regression (SVR)
and showed that RT is better than others. One important factor as lifestyle of
consumer from the load pattern is mentioned in [14] where Author forecasted
short term load using private aggregate data from AMI Applications to Enhance
Privacy Technologies (EPT) to protect private data. In SG prosumers are those
who buy and sale energy and [9] proposed distributed electricity trading system
to facilitate the peer-to-peer electricity sharing among prosumers. Here through
agent coalition system, prosumers became able to form coalitions and negoti-
ate electricity trading. Sometime there occurs security problems in SG, so [5]
proposed a model where consumer can monitor and send their data over SG
network without any data compromisation. We see forecasting techniques with
different scenarios in literature and notice that involvement of big data, data
normalization and classification increase DR efficiency of SG (Table 1).
1.3 Contributions
In this paper, we emphasize short term electricity load and price forecasting for
smart home. Our aim is to highlight the problem as one classifier is not suitable
for different scenario of forecasting in SG. Here this problem with some solu-
tions is discussed very precisely. To obtain our goal, we take into consideration
four forecasting models and proposed four forecasting scenarios and check per-
formance of each model at each scenario with similar parameter. We provide
following contribution in this paper.
Model’s Comparative Analysis: We implemented K-Nearest Neighbors
(K-NN), Random Forest (RF), Decision Tree (DT), Support Vector Regres-
sion (SVR) models for load and price forecasting for SH. After the compara-
tive analysis among these four models, we find the best solution model for load
and price forecasting for small data which will benefit much to SH and SG.
Actually literature shows many forecasting models for different scenarios but
we perform forecasting with four models and compare them with respect to
their accuracy, implementation complexity, time consumption for implemen-
tation. One model can never be used for all different forecasting scenarios like
one-hour, one-day, one-week, one-month and one-year ahead forecasting. This
paper showed that there is one best model for specific forecasting scenario but
that model could behave bad in other scenarios. This model’s analysis pro-
poses best forecasting model for specific scenario over small data and these
models are best solution for SG while dealing with different scenarios. This
work actually benefit SG much for accurate load and price forecasting for a
SH.
Forecasting of Electricity Load and Price 525
Table 1. Related work
Scheme Techniques used Consumer
satisfaction
Objectives
[1]Hybrid Filter Wrapper Approach ×New feature selection method is proposed for
better feature selection during forecasting in
energy market
[2]ANN, PSO, MAACPSO, ANFIS ×Short-term load forecasting with integrated
intelligent energy management process for
smart grid
[3]Deep Neural Network Short-Term Appliance level energy profiling
and forecasting for residential household
[4]CNN, K-Means ×Short-term load forecasting using 1.4 million
of load records as big data
[7]Deep Neural Network Forecasting day-ahead electricity price in
Europe with market integration and novel
feature selection algorithm
[8]Improved Elman Neural Network ×Short term load forecasting and proposed
enhanced version of empirical model
decomposition for feature selection
[10]Regression Tree, SVR, NN ×Short-term residential load forecasting with
three models and compare their results
[11] Moving Average, RF Short-term load forecasting with 2-stage
predictive analytics using data of private
university Seoul, Korea
[12]Multivariate Model ×Hourly based day-ahead price forecasting for
market with Root Mean Squared Error 16%
[13]Artificial Neural Network Day-ahead average electricity load
forecasting using smart meter data
[15]DE-SVM Get efficient result of electricity price
forecasting by using feature selection,
extraction and dimensionality reduction
techniques
[16]Stacked Denoising Autoencoders Day-ahead electricity price forecasting and
model comparison with others
[17]EMD-LSTM NN ×Short-term load forecasting with
Xgboost-based k-means framework for
Feature Importance Evaluation
VSTLPF: From a SH, we perform forecasting in SM’s minutes and hourly-
based data. We done very short term electricity load and price forecasting for
SH with the objective of enabling utilities and consumer to precisely know
the electricity load and price. In very short term load and price forecasting
we implemented and compare forecasting models for one-hour and twelve-
hours ahead forecasting at SH data. This capable SG to know about one-hour
and twelve-hours ahead load demand from SH and take proper decision to
response their upcoming demand.
STLPF: Short term load and price forecasting consist of one-day and one-
week ahead load and price forecasting for SH. Utilities became able to know
daily and week ahead demand of load from consumer and decided energy
generation on this base. Consumer remains updated about their demand and
price against energy from utilities so that he can shift their load from on peak
hours to off peak hours in order to save money. Ahead load demand and price
knowledge make consumer more satisfied and involvement for energy saving,
shifting load through on peak hours to off peak hours in SG.
526 M. Nawaz et al.
We addressed these issues in SG which are much benefited for both utility and
consumer for electricity generation and consumption. SG decide energy genera-
tion resources and show energy price detail of specific hours to consumer so that
they became able to decide energy consumption and transfer their load from on
peak hours to off peak hours.
The rest of paper is organized as follows: Sect. 2describes proposed sys-
tem model and Sect. 3is simulation results of system model. In Sect. 4, there
described Performance Evaluation. Paper’s conclusions and future work is
described in Sect. 5.
2SystemModel
Proposed system model is based on following steps (Fig. 1).
2.1 Input Data Configuration
The dataset used in this work is Appliance-level home dataset procured from
the Smart* project1in the year 2016. There are four SMs against HomeG, so we
chose common appliances as shown in Table 2. Here February month 2016 data
of HomeG is taken for model’s performance analysis. For models training and
testing, we divided dataset into 80% and 20% respectively. In order to achieve
higher accuracy, we normalize data as first convert data into watt per hour and
then remove outliers from it. Price against load is calculated by multiplication
of load (Use) data with specific value and then use this price feature to forecast
price against load according our scenarios.
Fig. 1. Proposed system model
1Smart* Dataset is taken from UMass Smart Repository. The Goal of Smart* project
is to optimize home energy consumption. http://traces.cs.umass.edu/index.php/
Smart/Smart.
Forecasting of Electricity Load and Price 527
Table 2. Dataset main features
Feat u res
Date & Time Use Price Wate r P ump
Kitchen Outlets1 Ejector Pump Range Oven PGR Outlets
Dishwasher Refrigerator Wall Oven Kitchen Lights
Basement Lights Wilo Pum HVAC Air Handler
2.2 Feature Engineering
Here Feature Engineering process is performed on dataset. This step is crucial for
better performance of models so here Important features are selected by RFE-
LSVC. RFE-LSVC is feature selection method that fits the model and remove the
features which have weak impact on prediction target. Table 3show the selected
features which have higher impact on prediction accuracy. Another important
thing is dataset splitting ratio which impact much in forecasting accuracy so we
checked it with different ratio but results are with 80% at training and 20% at
testing.
Table 3. Selected features
Feat u res
Date & Time Use Price Wate r P ump
Ejector Pump Range Oven PGR Outlets Refrigerator
Wilo Pum HVAC Air Handler
2.3 Forecasting
Accurate load and price forecasting of SH is main issue of literature and major
concern of this article, here we explained how each forecasting model work. After
feature engineering, selected features became base for forecasting through models
as discussed below.
1. Decision Tree (DT):
DT raise regression model in tree structure form. Simultaneously it breaks
down dataset into small subsets and incrementally develop decision tree. At
the end their is a tree with leaf nodes and decision nodes. Here Standard Devi-
ation (SD) is used to homogeneity of numerical sample and build branches of
tree. If numerical sample is homogeneous then its answer will be zero.
SD =Σ(x¯x)2
n(1)
528 M. Nawaz et al.
Coefficient of Variance (CV) decide when to stop branching of tree.
CV =S
¯x
100% (2)
SDR(T,X)=S(T)S(T,X) (3)
Our dataset is splitted in different attributes and SD of each branch is cal-
culated. The resulting SD is then subtraction from SD of before the split
to form Standard Deviation Reduction (SDR) as described in equation. For
decision node, attribute with high value of SDR is chosen this process run
recursively and terminate when CV for branch become smaller than threshold
or few branches e.g. 3. So when data points for all branches is equal to 3 then
algorithm stop creating branches and assign average of each branch to its leaf
node. If number of instances at leaf node are more than one then it calculate
average of them which is final value for target.
2. Random Forest (RF):
A supervised machine learning model which create a forest of trees and then
makes it random. RF creates multiple decision trees from dataset and then
merge them to get more accurate prediction results. During tree generation,
it add randomness to model and search best feature among random subsets
of features instead of most important feature while splitting a node. RF ran-
domly select features to generate decision trees and average the result.
3. Support Vector Regression (SVR):
SVR work with the principles of Support Vector Machine with few minor
differences. But the main idea is individualizing hyperplane which should
maximize margin, minimize error and part of error is tolerated. In SVR, first
input ‘x’ is mapped onto k-dimensional feature with nonlinear mapping then
linear model f(x, w) is created.
f(x, w)=
k
i=1
wigi(x)+b(4)
Here gi(x),i =1, ..., k are a set of nonlinear transformations and bis ‘bias’.
SVR use ε-insensitive loss function L(y, f(x, w)) which is used to measure
quality of estimation.
4. K-Nearest Neighbors (K-NN): K-NN model is supervised machine learn-
ing algorithm that store available cases and predict on the base of distance
function ‘d’. For prediction, it calculate ‘k’ number of neighbors with dis-
tance function and then assign a class or value of its nearest neighbor which
is output value. s
d=
k
i=1
(xiyi)2(5)
Forecasting of Electricity Load and Price 529
2.4 Performance Evaluation
After getting output from classifiers, here we check the performance accuracy
by using classifier performance evaluation technique. Here we calculate accuracy
of classifier with the calculation of Mean Absolute Error (MAE), Mean Squared
Error (MSE), Root Mean Square Error (RMSE) and Mean Absolute Percent-
age Error (MAPE). And then calculate accuracy in percentage. Also evaluated
models with respect to their training time and accuracy with each forecasting
scenario.
3 Simulations
For simulation of proposed work, we performed it with python simulator running
on Intel core i3 system, 8 GB RAM and 320 GB storage. We use four forecasting
models with four different scenarios. We considered K-NN as best forecasting
model and name it ‘Actual’ then compare it with others in all scenarios. Simu-
lation results are discussed below.
3.1 Hour-Ahead Forecasting
After training at 80% of dataset, here perform one and twelve-hours ahead fore-
casting through four models and get their results. Simulation results of models
are given bellow. Figure 2is the load forecasting for 12 h ahead load demand.
Figure 3show the model’s price forecasting comparison against 12 h ahead load.
Fig. 2. 12 h ahead load forecasting
comparison of models
Fig. 3. Price forecasting for 12 h ahead
load
3.2 Day-Ahead Forecasting
This section is simulations of day-ahead load and price forecasting. All models
are trained with same parameters but they behave differently in forecasting as
their results are shown here. Figure 4is model’s result against one-day ahead load
forecasting. And Fig. 5show the price forecasting results for one-day ahead.
530 M. Nawaz et al.
Fig. 4. Model’s results for One-day
ahead load forecasting
Fig. 5. Price forecasting for One-day
ahead load
3.3 Week-Ahead Forecasting
Here simulations of one week-ahead load and price forecasting from all models
are shown. Figure 6is graph of week-ahead load forecasting. Figure 7show the
results of one-week ahead price forecasting by models. But Fig. 8show the MAPE
of each models for day-ahead forecasting and Fig. 9is MAPE of models for week-
ahead forecasting.
Fig. 6. One-week ahead load forecast-
ing comparison
Fig. 7. One-week ahead price forecasting
Fig. 8. Performance evaluation at One-
day ahead forecasting
Fig. 9. Performance evaluation at One-
week ahead forecasting
Forecasting of Electricity Load and Price 531
4 Performance Evaluation
To evaluate performance of classifiers, we used MAE, MSE, RMSE and MAPE.
MAPE is percentage deviation therefore we use it for calculation of classifier’s
forecasting accuracy in percentage. Table 4is comparison of classifiers for fore-
casting accuracy.
Classifier’s forecasting results against four different scenarios is shown in
table where ‘Time (ms)’ column show their training and forecasting time in
millisecond. From this table we can analyze that in first scenario, K-NN perform
well and RF is better than DT and SVR in accuracy but it is time consuming.
If there would be large dataset then RF will perform badly and DT should be
considered as good.
In second scenario for twelve-hours ahead forecasting, K-NN is good but
here DT’s is better than RF and SVR. Also, DT’s and RF’s accuracy increased
here as compared to their accuracy in previous scenario. In third and fourth
scenario, after K-NN, RF gave good accuracy as compared to others and it
show its highest accuracy in third scenario as compared to its accuracy in other
scenarios. Overall K-NN with k value 3 is good for all scenarios mentioned in
this paper. Here small dataset is considered due to lazy learner this classifier
don’t perform good at large dataset.
Table 4. Model’s comparative analysis
Model Type Time (ms) MAPE Accuracy (%)
DT One-hour ahead 36 10.452 89.548
RF One-hour ahead 360 9.649 90.351
SVR One-hour ahead 203 69.642 30.358
K-NN One-hour ahead 11 3.047 96.952
DT 12-hours ahead 36 10.452 89.548
RF 12-hours ahead 360 9.649 90.351
SVR 12-hours ahead 203 69.642 30.358
K-NN 12-hours ahead 11 3.047 96.952
DT One-day ahead 36 10.452 89.548
RF One-day ahead 360 9.649 90.351
SVR One-day ahead 203 69.642 30.358
K-NN One-day ahead 11 3.047 96.952
DT One-week ahead 36 10.452 89.548
RF One-week ahead 360 9.649 90.351
SVR One-week ahead 203 69.642 30.358
K-NN One-week ahead 11 3.047 96.952
532 M. Nawaz et al.
5 Conclusion and Future Work
In this paper, big data is used for short term load and price forecasting only for a
SH by SG. For electric load and price forecasting, after preprocessing of data we
perform two further process on it, One feature engineering and other forecasting
models on data. RFE with LSVC are used for selecting more important features
from data. Load and price forecasting are performed by four models with four
different forecasting scenarios as one-hour, twelve-hours, one-day and one-weak
ahead forecasting. We notice that our models showed different forecasting accu-
racy at different scenarios and we compare their accuracy against each scenario.
At the end there is model’s comparisons Analysis table which explain each mod-
els accuracy rate against forecasting scenarios. Load and price forecasting for
SH by SG, this paper is best solution for SG to forecast accurate load and price
with best performing model against each scenario because one model can never
gave good results in every scenario. In future, we will do medium and long term
load and price forecasting with best forecasting models for these scenarios which
will make SG efficient to predict and fulfill SH demand.
References
1. Abedinia, O., Amjady, N., Zareipour, H.: A new feature selection technique for load
and price forecast of electrical power systems. IEEE Trans. Power Syst. 32(1), 62–
74 (2017)
2. Aydarous, A., Elshahed, M.A., Hassan, M.M.: Short term load forecasting as a
base core of smart grid integrated intelligent energy management system. In: 2017
International Conference on Modern Electrical and Energy Systems (MEES), pp.
192–195. IEEE, November 2017
3. Din, G.M.U., Mauthe, A.U., Marnerides, A.K.: Appliance-level short-term load
forecasting using deep neural networks. In: 2018 International Conference on Com-
puting, Networking and Communications (ICNC), pp. 53–57. IEEE, March 2018
4. Dong, X., Qian, L., Huang, L.: Short-term load forecasting in smart grid: a com-
bined CNN and K-means clustering approach. In: 2017 IEEE International Confer-
ence on Big Data and Smart Computing (BigComp), pp. 119–125. IEEE, February
2017
5. Gao, J., Asamoah, K.O., Sifah, E.B., Smahi, A., Xia, Q., Xia, H., Zhang, X., Dong,
G.: Gridmonitoring: secured sovereign blockchain based monitoring on smart grid.
IEEE Access 6, 9917–9925 (2018)
6. Jindal, A., Singh, M., Kumar, N.: Consumption-aware data analytical demand
response scheme for peak load reduction in smart grid. IEEE Trans. Ind. Electron.
65(11), 8993–9004 (2018)
7. Lago, J., De Ridder, F., Vrancx, P., De Schutter, B.: Forecasting day-ahead elec-
tricity prices in Europe: the importance of considering market integration. Appl.
Energy 211, 890–903 (2018)
8. Liu, Y., Wang, W., Ghadimi, N.: Electricity load forecasting by an improved fore-
cast engine for building level consumers. Energy 139, 18–30 (2017)
9. Luo, F., Dong, Z.Y., Liang, G., Murata, J., Xu, Z.: A distributed electricity trad-
ing system in active distribution networks based on multi-agent coalition and
blockchain. IEEE Trans. Power Syst. (2018)
Forecasting of Electricity Load and Price 533
10. Lusis, P., Khalilpour, K.R., Andrew, L., Liebman, A.: Short-term residential load
forecasting: impact of calendar effects and forecast granularity. Appl. Energy 205,
654–669 (2017)
11. Moon, J., Kim, K.H., Kim, Y., Hwang, E.: A short-term electric load forecasting
scheme using 2-stage predictive analytics. In: 2018 IEEE International Conference
on Big Data and Smart Computing (BigComp), pp. 219–226. IEEE, January 2018
12. Raviv, E., Bouwman, K.E., Van Dijk, D.: Forecasting day-ahead electricity prices:
utilizing hourly prices. Energy Econ. 50, 227–239 (2015)
13. Sulaiman, S.M., Jeyanthy, P.A., Devaraj, D.: Artificial neural network based day
ahead load forecasting using smart meter data. In: 2016 Biennial International Con-
ference on Power and Energy Systems: Towards Sustainable Energy (PESTSE), pp.
1–6. IEEE, January 2016
14. Tudor, V., Almgren, M., Papatriantafilou, M.: Employing private data in AMI
applications: short term load forecasting using differentially private aggregated
data. In: 2016 Intl IEEE Conferences on Ubiquitous Intelligence & Computing,
Advanced and Trusted Computing, Scalable Computing and Communications,
Cloud and Big Data Computing, Internet of People, and Smart World Congress
(UIC/ATC/ScalCom/CBDCom/IoP/SmartWorld), pp. 404–413. IEEE, July 2016
15. Wang, K., Xu, C., Zhang, Y., Guo, S., Zomaya, A.: Robust big data analytics
for electricity price forecasting in the smart grid. IEEE Trans. Big Data 5, 34–45
(2017)
16. Wang, L., Zhang, Z., Chen, J.: Short-term electricity price forecasting with stacked
denoising autoencoders. IEEE Trans. Power Syst. 32(4), 2673–2681 (2017)
17. Zheng, H., Yuan, J., Chen, L.: Short-term load forecasting using EMD-LSTM neu-
ral networks with a Xgboost algorithm for feature importance evaluation. Energies
10(8), 1168 (2017)
... A seasonal linear regression model outperforms the other two approaches in terms of forecast error and also the objective of the storage control. Nawaz et al. [23] uses Recursive Feature Elimination to select features and then compare various methods, i.e., kNN, RT, RF and SVR. They show that at different forecast horizons different methods perform best. ...
Article
The increased digitalisation and monitoring of the energy system opens up numerous opportunities to decarbonise the energy system. Applications on low voltage, local networks, such as community energy markets and smart storage will facilitate decarbonisation, but they will require advanced control and management. Reliable forecasting will be a necessary component of many of these systems to anticipate key features and uncertainties. Despite this urgent need, there has not yet been an extensive investigation into the current state-of-the-art of low voltage level forecasts, other than at the smart meter level. This paper aims to provide a comprehensive overview of the landscape, current approaches, core applications, challenges and recommendations. Another aim of this paper is to facilitate the continued improvement and advancement in this area. To this end, the paper also surveys some of the most relevant and promising trends. It establishes an open, community-driven list of the known low voltage level open datasets to encourage further research and development.
... ARX models are well-established in electricity price forecasting (EPF), as noted in [10,23]. The k-nearest neighbors algorithm has been found to be successful in the field of electricity market forecasts, mainly in forecasting electricity price and load [2,3,8,13,19,26] and renewable energy sources (RES) generation [25]. The proposed method is applied to the EPEX SPOT market in Germany. ...
Chapter
The ongoing reshape of electricity markets has significantly stimulated electricity trading. Limitations in storing electricity as well as on-the-fly changes in demand and supply dynamics, have led price forecasts to be a fundamental aspect of traders’ economic stability and growth. In this perspective, there is a broad literature that focuses on developing methods and techniques to forecast electricity prices. In this paper, we develop a new hybrid method, called ARHNN, for electricity price forecasting (EPF) in day-ahead markets. A well performing autoregressive model, with exogenous variables, is the main forecasting instrument in our method. Contrarily to the traditional statistical approaches, in which the calibration sample consists of the most recent and successive observations, we employ the k-nearest neighbors (k-NN) instance-based learning algorithm and we select the calibration sample based on a similarity (distance) measure over a subset of the autoregressive model’s variables. The optimal levels of the k-NN parameter are identified during the validation period in a way that the forecasting error is minimized. We apply our method in the EPEX SPOT market in Germany. The results reveal a significant improvement in accuracy compared to commonly used approaches.
... A seasonal linear regression model outperforms the other two approaches in terms of forecast error and also the objective of the storage control. Nawaz et al.[165] uses Recursive Feature Elimination to select features and then compare various methods, i.e., kNN, RT, RF and SVR. They show that at dierent forecast horizons dierent methods perform best. ...
Preprint
Full-text available
The increased digitalisation and monitoring of the energy system opens up numerous opportunities % and solutions which can help to decarbonise the energy system. Applications on low voltage (LV), localised networks, such as community energy markets and smart storage will facilitate decarbonisation, but they will require advanced control and management. Reliable forecasting will be a necessary component of many of these systems to anticipate key features and uncertainties. Despite this urgent need, there has not yet been an extensive investigation into the current state-of-the-art of low voltage level forecasts, other than at the smart meter level. This paper aims to provide a comprehensive overview of the landscape, current approaches, core applications, challenges and recommendations. Another aim of this paper is to facilitate the continued improvement and advancement in this area. To this end, the paper also surveys some of the most relevant and promising trends. It establishes an open, community-driven list of the known LV level open datasets to encourage further research and development.
... ARX models are well-established in electricity price forecasting (EPF), as noted in [10,23]. The k-nearest neighbors algorithm has been found to be successful in the field of electricity market forecasts, mainly in forecasting electricity price and load [2,3,8,13,19,26] and renewable energy sources (RES) generation [25]. The proposed method is applied to the EPEX SPOT market in Germany. ...
Preprint
Full-text available
We extend the \emph{neural basis expansion analysis} (NBEATS) to incorporate exogenous factors. The resulting method, called NBEATSx, improves on a well performing deep learning model, extending its capabilities by including exogenous variables and allowing it to integrate multiple sources of useful information. To showcase the utility of the NBEATSx model, we conduct a comprehensive study of its application to electricity price forecasting (EPF) tasks across a broad range of years and markets. We observe state-of-the-art performance, significantly improving the forecast accuracy by nearly 20\% over the original NBEATS model, and by up to 5\% over other well established statistical and machine learning methods specialized for these tasks. Additionally, the proposed neural network has an interpretable configuration that can structurally decompose time series, visualizing the relative impact of trend and seasonal components and revealing the modeled processes' interactions with exogenous factors.
... We observed that the feature selections influence the forecasting accuracy and the models M4 and M5 are still relatively stable, performing better than other models. Indeed, the RFE-SVR and Lasso regression feature selection methods are applied to various areas in energy finance and achieve good performance for improving forecasting accuracy [78,79,80,81]. Figure 15 shows the ranking of features and their impacts on the predicted price in terms of the selected features of the RFE-SVR model (M4), which is the model with the best performance. From Figure 15a, we can observe that the features from supply/demand sides are more important than the other by the model. ...
Preprint
Full-text available
The availability of accurate day-ahead electricity price forecasts is pivotal for electricity market participants. In the context of trade liberalisation and market harmonisation in the European markets, accurate price forecasting becomes even more difficult to obtain. The increasing power market integration has complicated the forecasting process, where electricity forecasting requires considering features from both the local market and ever-growing coupling markets. In this paper, we apply state-of-the-art deep learning models, combined with feature selection algorithms for electricity price prediction under the consideration of market coupling. We propose three hybrid architectures of long-short term memory (LSTM) deep neural networks and compare the prediction performance, in terms of various feature selections. In our empirical study, we construct a broad set of features from the Nord Pool market and its six coupling countries for forecasting the Nord Pool system price. The results show that feature selection is essential to achieving accurate prediction. Superior feature selection algorithms filter meaningful information, eliminate irrelevant information, and further improve the forecasting accuracy of LSTM-based deep neural networks. The proposed models obtain considerably accurate results.
... A smart grid is an intelligent electric power grid that combines information and communication technology with the existing electric power grid [3]. The smart grid can optimize energy use by sharing electric energy production and consumption information with consumers and suppliers in both directions and in real time [4]. The most fundamental approach for sustainable development of smart grids is electric power generation using renewable energy sources, such as photovoltaic and wind energy [5,6]. ...
Article
Full-text available
An energy-management system requires accurate prediction of the electric load for optimal energy management. However, if the amount of electric load data is insufficient, it is challenging to perform an accurate prediction. To address this issue, we propose a novel electric load forecasting scheme using the electric load data of diverse buildings. We first divide the electric energy consumption data into training and test sets. Then, we construct multivariate random forest (MRF)-based forecasting models according to each building except the target building in the training set and a random forest (RF)-based forecasting model using the limited electric load data of the target building in the test set. In the test set, we compare the electric load of the target building with that of other buildings to select the MRF model that is the most similar to the target building. Then, we predict the electric load of the target building using its input variables via the selected MRF model. We combine the MRF and RF models by considering the different electric load patterns on weekdays and holidays. Experimental results demonstrate that combining the two models can achieve satisfactory prediction performance even if the electric data of only one day are available for the target building.
Article
The availability of accurate day-ahead electricity price forecasts is pivotal for electricity market participants. In the context of trade liberalisation and market harmonisation in the European markets, accurate price forecasting becomes difficult for electricity market participants to obtain because electricity forecasting requires the consideration of features from ever-growing coupling markets. This study provides a method of exploring the influence of market coupling on electricity price prediction. We apply state-of-the-art long short-term memory (LSTM) deep neural networks combined with feature selection algorithms for electricity price prediction under the consideration of market coupling. LSTM models have a good performance in handling nonlinear and complex problems and processing time series data. In our empirical study of the Nordic market, the proposed models obtain considerably accurate results. The results show that feature selection is essential to achieving accurate prediction, and features from integrated markets have an impact on prediction. The feature importance analysis implies that the German market has a salient role in the price generation of Nord Pool.
Conference Paper
One of the most relevant applications today in the smart grid is the use of artificial intelligence and analysis techniques over the data generated, allowing both end-users and utility companies new applications. Energy fraud is considered one of the main non-technical losses for utilities that translate into economic losses, so being able to detect and minimize fraud becomes a vital necessity for utilities. This paper presents a machine learning model that allows the detection of abnormal behaviors in readings of consumption and/or production of electrical energy that can be classified as energy fraud. The proposed model is based on regression and classification techniques over an edge-fog computing architecture, and the results obtained show that its implementation in smart metering systems is adequate.
Article
Full-text available
Electricity is the commonest commodity for most businesses in our world today. The use of electricity has been a breakthrough for the discovery of new technologies and has become the main driving force behind several innovations. With the introduction of smart grid systems, there have been improvements in how utility companies interact with their customers with regards to electricity use. However, since the readings are done via the internet, there is the tendency for the data to be compromised when it gets into the hands of the wrong people. Moreover, customers mostly do not know why they pay huge amounts and which appliances use more electricity, since they are not privy to the readings. The sovereign blockchain technology, which provides transparency and provenance, is utilized in this paper to mitigate these above mentioned problems. A smart contract, which executes laid down procedures to provide a trust-based system between participants on the network is also implemented. Our system proves very efficient as the user can monitor how the electricity is used, and it also provides a platform where there is no manipulation from either parties.
Article
Full-text available
Motivated by the increasing integration among electricity markets, in this paper we propose two different methods to incorporate market integration in electricity price forecasting and to improve the predictive performance. First, we propose a deep neural network that considers features from connected markets to improve the predictive accuracy in a local market. To measure the importance of these features, we propose a novel feature selection algorithm that, by using Bayesian optimization and functional analysis of variance, evaluates the effect of the features on the algorithm performance. In addition, using market integration, we propose a second model that, by simultaneously predicting prices from two markets, improves the forecasting accuracy even further. As a case study, we consider the electricity market in Belgium and the improvements in forecasting accuracy when using various French electricity features. We show that the two proposed models lead to improvements that are statistically significant. Particularly, due to market integration, the predictive accuracy is improved from 15.7% to 12.5% sMAPE (symmetric mean absolute percentage error). In addition, we show that the proposed feature selection algorithm is able to perform a correct assessment, i.e. to discard the irrelevant features.
Article
Full-text available
Accurate load forecasting is an important issue for the reliable and efficient operation of a power system. This study presents a hybrid algorithm that combines similar days (SD) selection, empirical mode decomposition (EMD), and long short-term memory (LSTM) neural networks to construct a prediction model (i.e., SD-EMD-LSTM) for short-term load forecasting. The extreme gradient boosting-based weighted k-means algorithm is used to evaluate the similarity between the forecasting and historical days. The EMD method is employed to decompose the SD load to several intrinsic mode functions (IMFs) and residual. Separated LSTM neural networks were also employed to forecast each IMF and residual. Lastly, the forecasting values from each LSTM model were reconstructed. Numerical testing demonstrates that the SD-EMD-LSTM method can accurately forecast the electric load.
Article
The prevalence of distributed energy resources encourages the concept of an electricity ‘Prosumer (Producer and Consumer)’. This paper proposes a distributed electricity trading system to facilitate the peer-to-peer electricity sharing amongst prosumers. The proposed system includes two layers. In the first layer, a Multi-Agent System (MAS) is designed to support the prosumer network, and an agent coalition mechanism is proposed to ena-ble the prosumers to form coalitions and negotiate electricity trading. In the second layer, a Blockchain (BC) based transaction settlement mechanism is proposed to enable the trusted and secure settlement of electricity trading transactions formed in the first layer. Simulations are conducted based on the Java Agent Development Environment (JADE) to validate the proposed electricity trading process.
Article
The ever-increasing load demand of the residential sector gives rise to concerns such as-decreased quality of service and increased demand-supply gap in the electricity market. To tackle these concerns, the utilities are switching to smart grids (SGs) to manage the demand response (DR) of the connected loads. However, most of the existing DR management schemes have not explored the concept of data analytics for reducing peak load while taking consumer constraints into account. To address this issue, a novel data analytical demand response (DADR) management scheme for residential load is proposed in this paper with an aim to reduce the peak load demand. The proposed scheme is primarily based on the analysis of consumers' consumption data gathered from smart homes (SHs) for which factors such as-appliance adjustment factor, appliance priority index, etc. have been considered. Based on these factors, different algorithms with respect to consumer's and utility's perspective have been proposed to take DR. In addition to it, an incentive scheme has also been presented to increase the consumers' participation in the proposed scheme. The results obtained show that it efficiently reduces the peak load at the grid by a great extent. Moreover, it also increases the savings of the consumers by reducing their overall electricity bills.
Article
Literature is rich in methodologies for “aggregated” load forecasting which has helped electricity network operators and retailers in optimal planning and scheduling. The recent increase in the uptake of distributed generation and storage systems has generated new demand for “disaggregated” load forecasting for a single-customer or even down at an appliance level. Access to high resolution data from smart meters has enabled the research community to assess conventional load forecasting techniques and develop new forecasting strategies suitable for demand-side disaggregated loads. This paper studies how calendar effects, forecasting granularity and the length of the training set affect the accuracy of a day-ahead load forecast for residential customers. Root mean square error (RMSE) and normalized RMSE were used as forecast error metrics. Regression trees, neural networks, and support vector regression yielded similar average RMSE results, but statistical analysis showed that regression trees technique is significantly better. The use of historical load profiles with daily and weekly seasonality, combined with weather data, leaves the explicit calendar effects a very low predictive power. In the setting studied here, it was shown that forecast errors can be reduced by using a coarser forecast granularity. It was also found that one year of historical data is sufficient to develop a load forecast model for residential customers as a further increase in training dataset has a marginal benefit.