Conference PaperPDF Available

Data Analytics for Price Forecasting in Smart Grids: A Survey

Authors:

Abstract and Figures

Electricity Price Forecasting (EPF) plays a significant role in competitive electricity markets. Market participants rely on price forecast for generation, assets scheduling and effective bidding plan formulation. The uncertainty and volatility of energy market makes price forecasting a very challenging task. This paper gives a survey on methods and techniques used in literature for price forecasting in last four years. This work provides general background of price forecasting and brief discussion of the models and approaches implemented in the area of price forecasting. A comparison of price forecasting techniques with respect to different categories is presented that draws out the important aspects of the implemented methodologies.
Content may be subject to copyright.
Data Analytics for Price Forecasting in Smart
Grids: A Survey
Sana Mujeeb
Computer Science Department,
COMSATS University Islamabad,
Islamabad 44000, Pakistan.
akbarech@gmail.com
Nadeem Javaid
Computer Science Department,
COMSATS University Islamabad,
Islamabad 44000, Pakistan.
*Correspondence: nadeemjavaidqau@gmail.com
Website: www.njavaid.com
Sakeena Javaid
Computer Science Department,
COMSATS University Islamabad,
Islamabad 44000, Pakistan.
sakeenajavaid@gmail.com
Abstract—Electricity Price Forecasting (EPF) plays a signifi-
cant role in competitive electricity markets. Market participants
rely on price forecast for generation, assets scheduling and
effective bidding plan formulation. The uncertainty and volatility
of energy market makes price forecasting a very challenging
task. This paper gives a survey on methods and techniques
used in literature for price forecasting in last four years. This
work provides general background of price forecasting and brief
discussion of the models and approaches implemented in the area
of price forecasting. A comparison of price forecasting techniques
with respect to different categories is presented that draws out
the important aspects of the implemented methodologies.
Index Terms—Data Analytics, Big Data, Price Forecasting,
Artificial intelligent Forecasters, Deep Learning.
I. INTRODUCTION
With the advent of smart grid, a revolution occurs in the
field of electricity generation and distribution in last two
decades. Smart grid is an advanced form of conventional grid
as it uses a digital communications technology to identify and
react to consumption changes. The consumption of electric
energy is communicated to smart grid through smart meters
after small time intervals. A huge amount of data relevant to
energy is recorded with the installment of smart meters. Proper
procedures are to be followed for storage and maintenance of
the data recorded by the smart meters. Analysis of that big
amount of data can help in having deep insights that can be
very effective in decision making, planning and management.
Data analytics plays an important role in making the smart
grid more efficient, intelligent and beneficial. It can be said
that data analytics makes smart grid smarter.
The deregulation of the power markets develops a very com-
petitive environment. It allows the involvement of all markets
players, i.e., producers, investors, traders and buyers/end users.
Therefore, the power price is determined by this buying and
selling system. There are three types of participants in power
systems: energy producers, transmission and distribution side
and end users. All the participants get benefit from price
forecasting in different ways.
Electricity is very different from all the other commodities.
It has a few interesting and unique characteristics. Electricity
price time series exhibits the following characteristics:
TABLE I: List of abbreviations and symbols
Abbreviation Description
AEMO Australia Electricity Market Operators
ANN Artificial Neural Networks
ARIMA Auto Regressive Integrated Moving Average
BEC Bulk Electricity Consumers
CNN Convolution Neural Networks
DCANN Dynamic Choice Artificial Neural Networks
EPF Electricity Price Forecast
DNN Deep Neural Networks
DT Decision Tree
DE Differential Evaluation
FFNN Feed Forward Neural Networks
GA Genetic Algorithm
ISO NECA Independent System Operator New England
Control Area
LSTM Long Short Term Memory
MIMO Multi Input Multi Output
MLR Multiple Linear Regression
MRMRMS Maximizing Relevancy, Minimizing Redun-
dancy and Maximizing Synergy
NYISO New York Independent System Operator
PJM Pennsylvania-New Jersey-Maryland (Inter-
connection)
RNN Recurrent Neural Network
SAE Stacked Auto Encoders
SVM Support Vector Machine
VA R Vector Auto Regression
bSVM bias
ccost penalty
ηinsensitive loss function parameter
ρSVM marginal plane
σSVM kernel function parameter
wiANN weights
Whx RNN weights
Seasonality: The repetitive fluctuation of price over the
time is called seasonality. It is due to the weather, holi-
days and economic factors. Price exhibits daily, seasonal
and annual seasonality.
Trend: The general patterns of the electricity signals in a
long term that show an increasing, decreasing or constant
behavior, is known as trend. The trend can have several
different profiles: linear, parabolic or exponential.
Volatility: Price signals have highly volatile nature. Sud-
den and unexpected increase or decrease in price can be
seen in historical price numerous times. These sharp price
978-1-5386-1370-2/18/$31.00 ©2018 IEEE 1
spikes are due to many influential factors like fuel prices,
renewable generation availability, weather conditions and
economic factors.
Several factors influence the electricity price that include
excessive use penalty, government taxes on fuel, etc. The elec-
tricity generation from renewable sources cause a considerable
decrease in price. In a particular case of Germany, the energy
price goes down to negative. The energy consumers are paid to
use the electricity generated through wind turbines. Because
energy produced by windmill forms is environment friendly
as there is no carbon emission.
There are different prediction methods used for different du-
rations of prediction, either short or long [1]- [10]. Prediction
can be one step ahead in future; single value or multi-step
ahead. These prediction durations are known as forecasting
horizons. Price forecasting horizons can be categorized into
four major categories:
1) Dynamic/real-time: Real time price forecast is based on
dynamic data. Dynamic price forecasting is a stream
processing that is used in real-time applications. Low
response latency is essential in this real time pricing, also
known as Very Short Term Price Forecasting (VSTPF).
Real-time pricing enables consumers to be aware of
electricity prices allowing them to control the use of
appliances that consume more power.
2) Short-term: Refers to few minutes or few days ahead price
forecasting.
3) Medium-term: Refers to month or year ahead price fore-
casting.
4) Long-term: Refers to year and more ahead forecasting.
It is essential for long-term investment and political
decisions.
Platform of forecasting also effects the choice of forecasting
method. The testbed or platform for forecasting experiments
can be smart grid, micro grid, smart city or smart buildings.
List of abbreviations are presented in Table 1. and list of
symbols is presented in Table 2. The terms: electricity, power
and energy are used interchangeably throughout this article.
The distribution of rest of the paper is given as: Section 2is
background, related work from existing literature is presented
in Section 3, Section 4comprises of forecasting models and
their comparisons, and the last Section 5concludes the article.
II. BACKGROUND
With the deregulation and liberalization of electricity supply
markets, the electricity price forecasting (EPF) has gained a
revived attention. The restructuring of energy supply systems
brings a very competitive environment by allowing partici-
pation of all market players from producers to end users.
For all the market participants, the EPF has become a very
important task. Electricity supply companies can optimize their
generation strategies and assets allocation according to the
price forecasts. The changes in the prices of energy derivatives
also depend on energy price forecasts. As for the demand side,
the companies can schedule their operation based on low price
zones. Since last two decades, energy price forecasts have
become an essential input in energy utilities decision-making
at a corporate level. About 400 market players participate
in energy market operations annually. And transactions of
approximately 10$ million, are made in ISO NE Control Area.
A1% decrease in Mean Absolute Percentage Error (MAPE)
in short term electricity forecast, results in about 0.1%to
0.4% reduction in cost. This reduction in cost is approximately
1.5$ million annually in a medium sized power utility with
maximum peak load of 5 GW [1]
Electricity prices are very different in nature from all the
other financial markets due to its unique characteristics such
as energy non-storability, requirement of maintaining balance
between demand and supply sides. Electricity price forecasting
is a really challenging task due to its volatility caused by many
factors. These factors include availability of inexpensive power
generation sources, prices of fuel and other unquantifiable
factors; like weather conditions and market needs. Due to the
non storable nature of electricity, a real time equilibrium needs
to established between electricity generation and consumption.
All the generated electricity must be consumed.
The major participants of energy sector are generation com-
panies (GenCO), transmission companies (TransCO), distri-
bution companies (DisCO) and bulk electricity consumers
(BECs) [1]. Now a days, electricity price forecast has become
focus of attention for all market participants. Energy producers
can be benefited from price forecast and formulate profit
maximizing bidding strategies. The electricity buyers can
adjust power consumption in order to reduce the cost of pur-
chased energy. Energy price forecast helps market regulators
in development of a stable market.
The energy price forecasting has a great significance, there-
fore, it is central focus of many researchers. Various prediction
methods have been proposed for price forecasting. However,
due to high volatility of price signal, several prediction
methods are still competing each other with varying success.
Several prediction methods are used for price forecasting
which are classified as single, hybrid and ensemble methods.
This survey is conducted on price forecasting methods for
last four years, specifically from 2015 to 2018. In this survey,
different forecasting methods are discussed in detail.
III. MOTIVATION,RELATED WORK AND OUR
CONTRIBUTION
Accurate electricity price forecasting can be proven as
cost effective for electricity vendors. On the other hand, a
wrong estimate of price can cost the GenCOs minimum or
no profit at all. Therefore, the efficiency of price prediction
methods is crucial. Forecasting methods have gone through
revolutionary improvements in past two decades. Simple sta-
tistical to complex artificial intelligent predictors are applied
for price forecasting. Electricity price is predicted by several
prediction models, i.e., embedded [2], hybrid [3], [4], single
[5], univariate (time series), multivariate [6] [7] and artificial
intelligent methods [8]- [20]. Electricity forecast is focus of
many researchers and a lot of work has been done in this
Fig. 1: Steps of electricity price prediction
field. There are a few surveys, reviews and comparative studies
published on the topic of electricity price forecasting.
In [21], authors perform a bibliometrics analysis of literature
on electricity price forecasting. They also critically reviewed
the complexity, strengths and limitations of prediction methods
proposed in last one and a half decade. Prediction methods are
reviewed with prospective of forecasting horizons. A compara-
tive analysis of prediction techniques is performed with respect
to their category, i.e., statistical or AI. The future directions
for price forecast are also discussed. All the aforementioned
aspects of price forecasting are analyzed briefly. This survey is
conducted as a motivation for researchers who are interested
in electricity price forecasting. It is a very comprehensive
survey for developing basic understanding of price forecasting
techniques.
The focus of authors in [22] is on exploring Short Term Price
Forecasting (STPF) techniques available in existing literature.
The two basic forecast models discussed in this paper are time
series and simulation based models. First of all, the exogenous
factor are explained that influence energy price. Secondly,
the problems and issues of price forecasting are elaborated.
Thirdly, the prediction methods are discussed categorically.
Prediction models based on linear regression, nonlinear heuris-
tics, simulations and game theory are studied in this work.
Prediction methods that lie in aforementioned categories are
explained briefly. Most commonly used performance evalua-
tion techniques for predictors are also mentioned. The paper is
concluded by explaining the importance of suitable predictor
selection according to scope of the forecast problem. In [24],
the four price forecasting models: simulation, time series, unit
commitment and ANN are briefly explained. Variants of above
mentioned methods are also mentioned. The applicability of
these forecasting models in price forecasting is also described.
A comprehensive study of price forecasting in deregulated
energy market is presented in [25]. Applications of price
forecasting methods in various energy markets are explained.
The difficulties faced in the prediction of price and appro-
priate input features selection are described. It is concluded
that every energy market is different, therefore, there is no
universal price predictor. Price prediction methods can be
selected according to the market data. The advantage of hybrid
model over other models is explained. Hybrid models are
combination of two or more techniques for achieving better
prediction accuracy. These hybrid methods can overcome the
TABLE II: Types of forecasting models from existing literature
Ref. No. Linear Non-linear Single Hybrid Ensemble Embedded Uni-variate Multivariate
LASSO [1] - - - - -
DNN [2] - - - -
DE-SVM [3] - ----
SDAE [4] - - - - -
DNN [5] - - - - -
VAR [6] - - - - -
CNN+LSTM [7] - ----
Inverse
optimization
framework [8]
- - - - -
GELM, WNN [9] - - - - -
RCGA [10] - ----
ARIMA+TVABC-
NLSSVM [11]
-- -
ARMAHX [12] - - - - -
HRVM-GA [13] - ----
QOABC+LSSVM
[14]
-----
ANN [15] - ----
ARX [16] - - - - -
ANN [17] - - - - -
GNM [18] - - - - -
DCANN [19] - -- - -
SARIMA [20] - - - - -
weakness of one method separately.
A survey is a strong motivation for the new researchers in
any field to understand everything about the topic of interest.
Therefore, a survey should cover recent and state-of-the-art
work that is going on in a specific domain. Previous price
forecasting surveys mostly focus on just one or two forecast
horizons [26], i.e., short and medium term price forecasting
[22], [23]. Many new and improved forecasting methods are
introduced in recent years. Therefore, there is a need to explore
the most optimal prediction methods. The aforementioned
points are the motivation for this survey. In this survey, the
most recent state-of-the-art price forecasting techniques are
critically analyzed. The analysis is performed for all the fore-
casting horizons, short, medium and long term forecasting. The
strengths and limitations of existing forecasting techniques are
also highlighted. Furthermore, a comprehensive comparative
study of predictors of different categories is presented. This
survey helps the novice learners in this area to develop the
basic understanding of applicability of forecasting techniques
in different scenarios. The basic steps involved in electricity
price prediction are shown in Fig. 1.
IV. FORECASTING MODELS COMPARISON
This section classifies electricity price forecasting models
with respect to techniques as shown in Fig. 2. Due to extremely
volatile nature of electricity, several forecasting methods are
still competing with each other for accurate forecast. Fore-
casting methods can further be classified based on univariate
time series models: GARCH, ARIMA, ARMAX, NARX, and
multivariate models: SVM, ANN, Bayesian, DT that take
multiple exogenous variables as input. This section comprises
of discussion on few widely used forecasting models with
respect to aforementioned categories.
4.1 Price Forecasting based on Single Models
These methods can perform well enough depending on qual-
ity of input data. If the input data is well preprocessed, then
the prediction models can perform well with good accuracy.
Formulation of good input models can also improve the single
forecaster’s performance [4].
4.1.1 Support Vector Machine (SVM)
SVM is a really efficient prediction method. Due to its
computational simplicity and accuracy, it is one of the most
used methods for prediction. SVM was proposed by Vapnik
et al. in 1995. A brief discussion on SVM is as follows:
There are nlabeled examples (x1,y
1),...,(xn,y
n)with
labels yi∈{1,1}. We want to find the hyperplane <w,x>
+b=0(i.e., with parameters (w, b)) satisfying the following
three conditions [27]:
1) The scale of (w, b)is fixed so that the plane is in
canonical position w.r.t. {x1,...,x
n}. i.e., minin|<
w, xi>+b|=1.
2) The plane with parameters (w, b)separate the +1s from
the 1’s, i.e., yi(<w,x
i>+b)0for all in.
3) The plane has maximum margin ρ=1/|w|. i.e., mini-
mum |w|2.
There may or may not be a separating plane for the observed
data. Make an assumption that the data is linearly separa-
ble. Equation 1 and 2 are combined just in one condition:
yi(<w,x
i>+b)1; for all in. To solve the following
optimization problem, minimize 1
2|w|2, over all wRdand
bR, subject to, yi(<w,x
i>+b)10,for all in.
4.1.2 Artificial Neural Networks
ANN is a broadly used technique for price forecasting [11].
The capability of automatically learning feature from input

 

  
 
 





 
 







!" # 


$ 
%



Fig. 2: Prediction methods hierarchy based on computational model type
data is a quality which makes ANN very suitable for price
forecasting. ANN is a network with several interconnected
computational units called neurons. It has typically three types
of layers: the input layer, hidden layer and the output layer. It
can be defined with the following equation [28]:
y(x1,...,x
n)=f(w1x1+w2x2+...+wnxn)(1)
f(v)=1
x0(v)(2)
Where, wiare parameters/weights, fis the activation function
and f(v)is McCulloch/Pitts activation function. For learning,
the addition is used for integrating the inputs.
Study has proven that in last few years, effective forecasting
results are achieved by using DNN [2]. The computational
power of DNN can model any non linear and complex func-
tion. Recurrent Neural Network (RNN) is a type of DNN [28].
RNN is a commonly used predictor. RNN was first introduced
by John Hopfield in 1982. RNN is given in equation 3 and
equation 4 is its activation function [29]:
ht=f(Whxxt+Whh ht1+b)(3)
yt=g(Wyhht+c)(4)
Where, Whx are parameters/weights, bis the bias, fis
activation function, xtis input and ht1is output of forget
unit.
4.1.3 Auto-regressive Models
Auto regression prediction models comprise of linear and
non-linear models. Auto Regressive Integrated Moving Aver-
age (ARIMA) is linear model that is popular for time series
forecasting. Many variants of ARIMA are proposed for non-
linear prediction such as, Seasonal ARIMA (SARIMA), Vector
Auto-regression (VAR) [6] and ARIMA with exogenous (inde-
pendent) variables (ARIMAX) [12]. Linear Regression (LR),
Multi-Linear Regression (MLR) and Quartile Regression (QR)
are also considered as the effective methods of forecasting.
4.2 Price Forecasting based on Hybrid Models
Hybrid forecasting method combines the regression, data
smoothing, and various other techniques, that can overcome
the limitations of the single methods and produce accurate
forecasts. Some forecasters are effective for short-term pre-
diction, however they cannot deal with seasonality. Every
forecasting model has a few hyper parameters. It is not
certain that default hyper parameters perform well on every
dataset. Therefore, hyper parameters are tuned according to
the input data. For selecting suitable hyper parameters for
models, optimization algorithms are used. The objective of
the optimization algorithms is to minimize the prediction error
of the model on the given dataset. Combination of prediction
method with optimization algorithm or combination of two
or more prediction methods make hybrid models. The hybrid
models yield better accuracy as compared to single forecasters.
However, they have higher computational complexity com-
pared to single models. Mostly, nature inspired meta-heuristic
optimization algorithms are used for hyper parameters tuning,
e.g., Genetic Algorithm (GA), Particle Swam Optimization
(PSO), Artificial Bee Colony (ABC), and many others.
4.2.1 Hybrid SVM
SVM has three main hyper parameters that are: cost penalty
c, insensitive loss function parameter ηand sigma kernel
parameter σ. In [3], authors tune hyper parameters of SVM
using modified Differential Evaluation (DE) algorithm and
renamed this hybrid predictor as DE-SVM. DE is a meta
heuristic optimization technique. The objective of the opti-
mization is to select such values of η,σand c that minimizes
the regularized risk function. The optimized SVM achieved
very good accuracy and outperforms Naive Bayesian NB
and Decision Tree DT predictors. Short-term hourly price is
predicted for day and week ahead.
Authors in [11] implement an improved and enhanced Time
Fig. 3: Applications of electricity data analysis.
Varying Artificial Bee Colony (TV-ABC) algorithm for pa-
rameter tuning of Nonlinear Least Square SVM (NLSSVM).
The prediction method is combination of two techniques: first
the inputs are fed to ARIMA predictor and the residuals of
ARIMA is input to optimized NLSSVM. This ARIMA + TV-
ABC NLSSVM is a multi input multi output (MIMO) forecast
engine. It forecasts electricity price and load simultaneously
with great accuracy. Proposed method is tested on three well
known energy markets that are: NYISO, PJM and NSW
AEMO. It yields lesser MAPE, RMSE and MAE in compar-
ison to MIMO ANN and MIMO NLSSVM. Accuracy of this
predictor proves the effectiveness of TV-ABC and NLSSVM
hybrid predictor.Another variant of improved ABC is proposed
for selecting best hyper parameters of LSSVM by authors
of [14]. The predictor LSSVM is based on Bayesian model.
The optimal solution search process of ABC is improved by
introducing Orthogonal Learning (OL). The OL serves as a
tool for global search and avoids the problem of stucking in
local optimum. Day ahead hourly price is predicted for Iran,
Spain and New South Wales. This method results in a good
accuracy and performs better in comparison to several popular
prediction methods.
4.2.2 Hybrid ANN
ANN suffers from the problem of over fitting. The model’s
good prediction accuracy on training data and poor perfor-
mance on test data is known as over fitting that is a very
common problem in ANN predictors. This network is not
generalized enough to perform well on unseen data. To resolve
the over fitting issue, the network parameters need to be
selected carefully. The number of neurons in hidden layers,
number of layers, initial weights, momentum and learning
rate are hyper parameters of ANN. These hyper parameters
are selected using optimization algorithms such as, PSO, GA,
DE, etc. Another way to resolve the issue of over-fitting is the
combining of two or more ANNs. In [15], ANNs are combined
in different topologies: parallel, serial, cascade and hybrid of
cascade and parallel ANNs.
4.2.3 Hybrid DNN
DNN are computationally very expensive, especially their
time and space complexity is much higher than any other
prediction method. Therefore, selection of optimal network
parameters is a crucial task. Initial network parameters deter-
mine the convergence time of DNN. In [8], two DNN models
are combined. CNN is used for feature extraction and LSTM
is used for prediction.
The main purpose of this section is to compare the electricity
forecast models available in recent literature. First of all, the
models are analyzed based on all the major groups of forecast
techniques. Table 3 shows the types of forecasting methods
available in literature.
Features Platform/Dataset Duration Forecast
Horizon
Region/Market Prediction Method Advantages / Per-
formance
Disadvantages
Historic price,
weather data
Day ahead spot
prices, total num-
ber of 11,640 ob-
servations
January
2013-
2030,
April
2014
Short term,
day and week
ahead
Germany, European
Power Exchange
(EPEX SPOT)
LASSO,
random forest,
ARMAX/ARMA
[1]
LASSO is embed-
ded predictor that
perform both fea-
ture extraction and
prediction. MAE =
5.7, RMSE = 7.9
Computationally
expensive
Day ahead load,
forecast of available
generation
Day-ahead mar-
ket data of Bel-
gium, France
2010-
2016
Short term,
day ahead
European power ex-
change (EPEX) Bel-
gium, France
LSTM, GRU, CNN
[2]
High accuracy,
power to model
complex data
pattern. Symmetric
MAPE (SMAPE)
= 12.34
Over fitting, space
and time complex-
ity
System Load, Day
ahead demand,
Weather data
Hourly
consumption,
weather, data of
New England
Control Area
2010-
2015
Short term,
day and week
ahead
ISO NE CA, New Eng-
land, USA
DE-SVM [3] Good accuracy,
fast convergence
of DE. Percentage
Accuracy = 95.6%
Hard to find right
kernel function,
DE unstable
convergence (local
optimum)
Electricity price, ob-
served load, forecast
load
Hourly data
five hubs of the
Mid-continent
ISO (MISO) in
U.S.
2012-
2014
Short term,
online hourly,
day ahead
MISO, Nebraska
Public Power District
(NPPD), Arkansas,
Louisiana, Texas, and
Indiana, U.S.
Stacked De-noising
Auto-encoder
(SDA) [4]
Robust to noisy
data, good accu-
racy. MAPE = 4.67
Computationally
expensive
Day ahead price,
forecast load,
calendar of public
holidays
Day-ahead mar-
ket data of Bel-
gium, France
2010-
2016
Short term,
day ahead
European power ex-
change (EPEX) Bel-
gium, France
DNN [5] Efficient feature
pattern modeling.
SMAPE = 12.5
Space and time
complexity
Historic energy price Hourly electricity
prices of Nord
Pool Spot
exchange
1992-
2010
Short term,
day and week
ahead
Nord Pool Spot ex-
change
Vector Auto
Regression (VAR),
Bayesian VAR,
Reduced Rank
Regression (RRR),
Bayesian RRR [6]
Simple and fast.
RMSE = 0.85,
MAPE = 0.3,
MAE = 0.2
Unable to model
extreme seasonal-
ity of data accu-
rately
Historic price Half hour
regulation market
capacity clearing
price
2017 Short term,
day and week
ahead
Electric power market
of PJM
CNN+LSTM [7] Excellent
accuracy,
robustness. MAE
= 5.6, RMSE =
10.5
Computationally
very expensive,
hard to select
optimal network
parameters
Load consumption,
broadcasted price,
observed weather
variables, outside
temperature, solar
irradiance, wind
speed, Humidity,
dew point and wind
direction
Hourly
measurements
of load
consumption
of Olympic
Peninsula Project
dataset
May
2006 -
March
2007
Short term Washington, Oregon Inverse
optimization
framework [8]
High accuracy.
MAPE = 0.8
Computationally
expensive,
intensive
preprocessing
required
Historic price Hourly price 2014 Short term Ontario electricity
market (IESO),
Australian Electricity
Market (AEMO),
Ontario, Australia
Bootstrapping,
generalized
extreme learning
machine (GELM),
wavelet neural
networks (WNNs)
[9]
Efficient pattern
capturing,
Robustness to
noisy data.
Percentage
Accuracy =
95.8%
Need to resolve
over fitting
Relevancy,
redundancy and
interaction
Real load and
price data of
PJM, Spanish
and New York
electric utility
Not men-
tioned
Short term,
day ahead
New Jersey-Maryland,
Spain, New York
MRMRMS, real-
coded genetic
algorithm [10]
Computationally
simple. Average
DMAPE = 0.8
Unable to capture
extreme nonlinear-
ity
Historic price and
load
Hourly load
and price of: (i)
NYC, (ii) PJM,
(iii) NYC
(i) 2014,
(ii) 2013,
(iii) 2010
Short term,
day and week
ahead
AEMO, PJM, NYISO
electricity markets
ARIMA+TVABC-
NLSSVM [11]
High accuracy,
perform equally
well on different
market data.
MAPE = 7.8
computationally
expensive
Historic price Spanish day-
ahead electricity
price
2007-
2008
Short term,
day and week
ahead
Spain Autoregressive
Moving Average
Hibernian model
with exogenous
variables [12]
Computationally
simple. MAPE =
6.65
Unable to capture
extreme nonlinear-
ity
Historic price,
lagged price, price
on holidays
Hourly price of
four weeks from
every season,
New England
2001 Short term,
Day ahead
ISO NE CA, New Eng-
land, USA
Hybrid relevance
vector machine,
micro Genetic
Algorithm [13]
High accuracy.
MAE = 5.8
Time complexity
Historical load and
price
Hourly price and
load of NYISO,
PJM and New
South Wales
2010,
2014
Short term,
day ahead
NYISO, PJM, NSW
AEMO energy markets
QOABC+LSSVM
[14]
High accuracy, ro-
bust. MAPE = 1.07
Time compexity
Historic price, calen-
dar and weather vari-
ables
Hourly price,
weather data of
AEMO
2005 Medium
term, month
ahead
AEMO, Australia ANN [15] Robustness to
noisy data, good
accuracy. MAE =
7.9
High time com-
plexity
Historical load
and price,
imports/exports,
capacity
excess/shortfall,
historical reserves,
generation types,
calendar effects
Hourly market
operation data of
Ontario, Alberta,
New York, Texas
2015 Short term,
Day ahead
IESO Ontario, AESO
Alberta, NYISO New
York, ERCOT Texas
Autoregressive
model with
exogenous
variables (ARX)
[16]
Computationally
simple. MAE =
5.56
Unable to capture
extreme nonlinear-
ity
Historic load, fuel
price, RES genera-
tion, calendar vari-
ables
Hourly load,
RES power
generation, fuel
prices, fuel
type available
capacities,
calendar data of
Germany
July
2011-
September
2013
Short term,
day ahead
Germany electricity
market
ANN [17] Efficient pattern
modeling
over-fitting
Historic price, load,
lagged price, day
type
Hourly price,
load data of New
South Wales
- Short term,
week ahead
AEMO, New South
Wales, Australia
Generalized
Neuron Model
[18]
Efficient pattern
modeling. RMSE
= 9.65
Over-fitting
Historic price 30-minute market
operation data of
Queensland, total
17520 measure-
ments
2010 Short term,
day ahead
Australia electricity
market (AEM)
Pennsylvania-New
Jersey-Maryland
(PJM)
Dynamic Choice
ANN (DCANN)
[19]
High accuracy,
fast convergence.
MAPE = 8.7,
MAE = 4.2
Over-fitting
Historic price,
weather variables
2016-2017 Hourly
price
EPEX
day-ahead
electricity
market for
Germany and
Austria
Long term, year ahead Seasonal ARIMA
[20]
Capture data sea-
sonality. MAPE =
2.9, MAE = 0.72,
RMSE = 0.89
Unable to model
hidden complex
patterns
TABLE III: Comparison of forecasting methods in existing literature
The classifications and applications of electricity data an-
alytics are shown in Fig. 3. The complete description of
forecasting models is listed in Table 4 that includes input
features used to build the model, testbed or dataset, duration
(in which data is captured), forecast horizon and prediction
methods. The advantages and disadvantages of prediction
methods are mentioned with respect to the scenario of the
referred paper.
V. C ONCLUSION
In the present era, electricity price forecasting plays the
key role in electricity market operations’ planning and man-
agement. This paper presents a survey of EPF with respect
to forecast horizon: short-term, medium-term and long-term
forecasting. The methods used for price forecasting are also
reviewed. The applicability of short, medium and long term
forecasting is highlighted. It is concluded that the hybrid
forecasters perform better as compared to single forecasters.
However, the hybrid methods are computationally more ex-
pensive than single methods. Their better forecasting accuracy
makes them a favorable choice for price forecasting. Recently,
DNN performs considerably well in price forecasting. DNN
performs very well on large amount of data. It well captures
the effect of exogenous variable and models the uncertainties
of price with great accuracy. In conclusion, the artificial
intelligent prediction methods are robust, adaptive and able
to efficiently capture the uncertainties that are highlighted in
state of the art.
REFERENCES
[1] ISO NE Anuall Report, https ://www.iso ne.com/static
assets/documents/2018/05/2017 annual markets report
(Last visited on 30th July 2018)
[2] Ludwig Nicole, Stefan Feuerriegel, and Dirk Neumann. "Putting Big Data
analytics to work: Feature selection for forecasting electricity prices using
the LASSO and random forests." Journal of Decision Systems 24, no. 1
(2015): 19-36.
[3] Lago Jesus, Fjo De Ridder, and Bart De Schutter. "Forecasting spot
electricity prices: deep learning approaches and empirical comparison of
traditional algorithms." Applied Energy 221 (2018): 386-405.
[4] Wang Kun, Chenhan Xu, Yan Zhang, Song Guo, and Albert Zomaya.
"Robust big data analytics for electricity price forecasting in the smart
grid." (Accepted) IEEE Transactions on Big Data (2017).
[5] Wang Long, Zijun Zhang, and Jieqiu Chen. "Short-Term Electricity Price
Forecasting with Stacked Denoising Autoencoders." IEEE Transactions
on Power Systems 32, no. 4 (2017): 2673-2681.
[6] Lago Jesus, Fjo De Ridder, Peter Vrancx, and Bart De Schutter. "Forecast-
ing day-ahead electricity prices in Europe: the importance of considering
market integration." Applied Energy 211 (2018): 890-903.
[7] Raviv Eran, Kees E. Bouwman, and Dick van Dijk. "Forecasting day-
ahead electricity prices: Utilizing hourly prices." Energy Economics 50
(2015): 227-239.
[8] Kuo Ping-Huan, and Chiou-Jye Huang. "An Electricity Price Forecasting
Model by Hybrid Structured Deep Neural Networks." Sustainability 10,
no. 4 (2018): 1280.
[9] Saez-Gallego Javier, Juan M. Morales, Marco Zugno, and Henrik Madsen.
"A data-driven bidding model for a cluster of price-responsive consumers
of electricity." IEEE Transactions on Power Systems 31, no. 6 (2016):
5001-5011.
[10] Rafiei Mehdi, Taher Niknam, and Mohammad-Hassan Khooban. "Prob-
abilistic forecasting of hourly electricity price by generalization of ELM
for usage in improved wavelet neural network." IEEE Transactions on
Industrial Informatics 13, no. 1 (2017): 71-79.
[11] Abedinia Oveis, Nima Amjady, and Hamidreza Zareipour. "A new
feature selection technique for load and price forecast of electrical power
systems." IEEE Transactions on Power Systems 32, no. 1 (2017): 62-74.
[12] Ali Ghasemi, Hossien Shayeghi, Mohammad Moradzadeh, and Mo-
hammad Nooshyar. "A novel hybrid algorithm for electricity price and
load forecasting in smart grids with demand-side management." Applied
energy 177 (2016): 40-59.
[13] Gonzalez Jose Portela, Antonio Munoz San Roque, and Estrella Alonso
Perez. "Forecasting functional time series with a new Hilbertian ARMAX
model: Application to electricity price forecasting." IEEE Transactions on
Power Systems 33, no. 1 (2018): 545-556.
[14] Alamaniotis Miltiadis, Dimitrios Bargiotas, Nikolaos Bourbakis, and
Lefteri Tsoukalas. "Genetic optimal regression of relevance vector ma-
chines for electricity pricing signal forecasting in smart grids." IEEE
Transactions on Smart Grid 6, no. 6 (2015): 2997-3005.
[15] Shayeghi Hossein, Ali Ghasemi, Mohammad Moradzadeh, and Moham-
mad Nooshyar. "Simultaneous day-ahead forecasting of electricity price
and load in smart grids. " Energy Conversion and Management 95 (2015):
371-384.
[16] Mosbah Hossam, and Mohamed El-Hawary. "Hourly Electricity Price
Forecasting for the Next Month Using Multilayer Neural Network. "
Canadian Journal of Electrical and Computer Engineering 39, no. 4
(2016): 283-291.
[17] Chitsaz Hamed, Payam Zamani-Dehkordi, Hamidreza Zareipour, and
Palak Parikh. "Electricity Price Forecasting for Operational Scheduling
of Behind-the-meter Storage Systems." (Accepted) IEEE Transactions on
Smart Grid (2017).
[18] Keles Dogan, Jonathan Scelle, Florentina Paraschiv, and Wolf Fichtner.
"Extended forecast methods for day-ahead electricity spot prices applying
artificial neural networks." Applied energy 162 (2016): 218-230.
[19] Nitin Singh, Soumya Ranjan Mohanty, and Rishabh Dev Shukla. "Short
term electricity price forecast based on environmentally adapted general-
ized neuron." Energy 125 (2017): 127-139.
[20] Wang Jianzhou, Feng Liu, Yiliao Song, and Jing Zhao. "A novel model:
Dynamic choice artificial neural network (DCANN) for an electricity
price forecasting system." Applied Soft Computing 48 (2016): 281-297.
[21] Ziel Florian, and Rick Steinert. "Probabilistic mid-and long-term elec-
tricity price forecasting." Renewable and Sustainable Energy Reviews 94
(2018): 251-266.
[22] Weron Rafal. "Electricity price forecasting: A review of the state-of-the-
art with a look into the future." International journal of forecasting 30,
no. 4 (2014): 1030-1081.
[23] Nitin Singh, and Soumya Ranjan Mohanty. "A review of price forecast-
ing problem and techniques in deregulated electricity markets." Journal
of Power and Energy Engineering 3, no. 09 (2015): 1.
[24] Weron, Rafal. "Electricity price forecasting: A review of the state-of-
the-art with a look into the future. " International journal of forecasting
30, no. 4 (2014): 1030-1081.
[25] Y. Nagaraja, T. Devaraju, M. Vijay Kumar, and Sreedhar Madichetty.
"A survey on wind energy, load and price forecasting: (Forecasting meth-
ods)." In Electrical, Electronics, and Optimization Techniques (ICEEOT),
International Conference on, pp. 783-788. IEEE, 2016.
[26] Pandey Nivedita, and K. G. Upadhyay. "Different price forecasting
techniques and their application in deregulated electricity market: A
comprehensive study." In Emerging Trends in Electrical Electronics and
Sustainable Energy Systems (ICETEESES), International Conference on,
pp. 1-4. IEEE, 2016.
[27] Vardakas John, Nizar Zorba, and Christos Verikoukis. "A survey on de-
mand response programs in smart grids: Pricing methods and optimization
algorithms." IEEE Communications Surveys & Tutorials 17, no. 1 (2015):
152-178.
[28] Pal Mahesh, and P. M. Mather. "Support vector machines for classifica-
tion in remote sensing." International Journal of Remote Sensing 26, no.
5 (2005): 1007-1011.
[29] McCulloch, Warren S., and Walter Pitts. "A logical calculus of the ideas
immanent in nervous activity." The bulletin of mathematical biophysics
5, no. 4 (1943): 115-133.
[30] Hochreiter Sepp, and Jurgen Schmidhuber. "LSTM can solve hard
long time lag problems." In Advances in neural information processing
systems, (1997) pp. 473-479.
10
... Forecasting horizons are classified in three categories [142][143][144][145][146][147]; short-term, medium-term and long-term; although some literature adds a fourth category: very short-term or real-time forecasting [145,[148][149][150]. Table 6 specifies what applications and time range covers each forecasting horizon. Most of the articles focus on short-term forecasting, as [146] also points out. ...
... Forecasting horizons are classified in three categories [142][143][144][145][146][147]; short-term, medium-term and long-term; although some literature adds a fourth category: very short-term or real-time forecasting [145,[148][149][150]. Table 6 specifies what applications and time range covers each forecasting horizon. ...
... To S. Barja-Martinez et al. Short-term 1 h < t < 1 week Deregulated electricity markets [146,[155][156][157] Real-time energy management systems [144,145] Optimal management of power system [157] Medium-term 1 week < t < 12 months Assess environmental impacts, maintenance scheduling [146,158] Long-term t > 1 year Long-term investment and political decisions [145,159] Grid expansion criteria. Decision tool for buying futures of the spot product [159] [199] solve this problem, [213] estimates the aggregated power generation of small-scale rooftops solar sites that are not monitored by system operators. ...
Article
Full-text available
Artificial intelligence techniques lead to data-driven energy services in distribution power systems by extracting value from the data generated by the deployed metering and sensing devices. This paper performs a holistic analysis of artificial intelligence applications to distribution networks, ranging from operation, monitoring and maintenance to planning. The potential artificial intelligence techniques for power system applications and needed data sources are identified and classified. The following data-driven services for distribution networks are analyzed: topology estimation, observability, fraud detection, predictive maintenance, non-technical losses detection, forecasting, energy management systems, aggregated flexibility services and trading. A review of the artificial intelligence methods implemented in each of these services is conducted. Their interdependencies are mapped, proving that multiple services can be offered as a single clustered service to different stakeholders. Furthermore, the dependencies between the AI techniques with each energy service are identified. In recent years there has been a significant rise of deep learning applications for time series prediction tasks. Another finding is that unsupervised learning methods are mainly being applied to customer segmentation, buildings efficiency clustering and consumption profile grouping for non-technical losses detection. Reinforcement learning is being widely applied to energy management systems design, although more testing in real environments is needed. Distribution network sensorization should be enhanced and increased in order to obtain larger amounts of valuable data, enabling better service outcomes. Finally, the future opportunities and challenges for applying artificial intelligence in distribution grids are discussed.
... Prediction of prices is also an important factor in various fields, including energetics, as presented in the study by Mujeeb, Javaid, and Javaid (2018). They note that participants in electric energy market have been dependent on price predictions already for a long time. ...
... They note that participants in electric energy market have been dependent on price predictions already for a long time. The authors compare individual methods and technologies, including Big Data and Deep Learning, assigned for prognosing [21]. ...
Article
Full-text available
CRISP-DM (cross-industry standard process for data mining) methodology was developed as an intuitive tool for data scientists, to help them with applying Big Data methods in the complex technological environment of Industry 4.0. The review of numerous recent papers and studies uncovered that most of papers focus either on the application of existing methods in case studies, summarizing existing knowledge, or developing new methods for a certain kind of problem. Although all of these types of research are productive and required, we identified a lack of complex best practices for a specific field. Therefore, our goal is to propose best practices for the data analysis in production industry. The foundation of our proposal is based on three main points: the CRISP-DM methodology as the theoretical framework, the literature overview as an expression of current needs and interests in the field of data analysis, and case studies of projects we were directly involved in as a source of real-world experience. The results are presented as lists of the most common problems for selected phases (‘Data Preparation’ and ‘Modelling’), proposal of possible solutions, and diagrams for these phases. These recommendations can help other data scientists avoid certain problems or choose the best way to approach them.
... Machine Learning Algorithms (MLAs) gives a systematic way to analyze the data and make appropriate decisions to run the grid. MLA functionalities include price and load forecasting, fault detection, power generation, future optimum schedule during data breach [5]. ...
Conference Paper
Full-text available
Conventional grid moves towards Smart Grid (SG). In conventional grids, electricity is wasted in generation-transmissions-distribution, and communication is in one direction only. SG is introduced to solve prior issues. In SG, there are no restrictions, and communication is bi-directional. Electricity forecasting plays a significant role in SG to enhance operational cost and efficient management. Load and price forecasting gives future trends. In literature many data-driven methods have been discussed for price and load forecasting. The objective of this paper is to focus on literature related to price and load forecasting in last four years. The author classifies each paper in terms of its problems and solutions. Additionally, the comparison of each proposed technique regarding performance are presented in this paper. Lastly, papers limitations and future challenges are discussed.
Thesis
Full-text available
The revolution of power grids from traditional grids to Smart Grids (SGs) requires effective Demand Side Management (DSM) and reliable Renewable Energy Sources (RESs) incorporation in order to maintain demand, supply balance and optimize energy in an environment friendly manner. Data analytics provide solutions to the emerging challenges of power systems, such as DSM, environmental pollution (due to carbon emission), fossil fuel dependency mitigation, RESs incorporation, cost curtailment, grid’s stability and security. To efficiently manage electricity and maximize the profit of power utilities several tasks are focused in this thesis, i.e., prediction of electricity load to avoid demand and generation mismatch, wind power forecasting to satisfy energy demand effectively, electricity price forecasting for regulating market operations, carbon emissions forecasting for reducing payment of carbon tax, Electricity Theft Detection (ETD) for recovering power utilities’ revenue loss caused by electricity theft. In addition to that, a wind power forecast based DSM scheme is proposed. Furthermore, impact of RESs integration level on carbon emissions, electricity price and consumption cost is quantified. Both forecasting and classification techniques are utilized for efficient energy management. Forecasting of electricity load, price, wind power and carbon emissions is performed, whereas, classification of fair and fraudulent electricity consumers is performed. To balance electricity demand and supply, electricity load forecasting is required. Three models are proposed for this purpose, i.e., Deep Long Short-Term Memory (DLSTM), Efficient Sparse Autoencoder Nonlinear Autoregressive eXogenous network (ESAENARX) and Differential Evolution Recurrent Extreme Learning Machine (DE-RELM). DLSTM utilizes univariate data and gives single result, whereas, ESAENARX and DE-RELM model multivariate data and predict electricity load and price simultaneously. Due to adaptive and automatic feature learning mechanism, DLSTM achieves accurate results for separate forecasting of electricity load and price. ESAENARX and DE-RELM models are enhanced by newly proposed efficient feature extractor and model’s parameter tuning, respectively. Real-world datasets of ISO-NE, PJM, NYISO are used for load and price forecasting. The purpose of regulating the electricity market operations is achieved by forecasting of electricity load, price, wind power and carbon emissions. Wind power generation is predicted by an efficient model named Efficient Deep Convolution Neural Network (EDCNN). Moreover, a DSM strategy is also proposed based on predicted wind power generation. Power utilities have to pay carbon emissions tax imposed by government. To pay less carbon emissions tax, carbon emissions prediction is required, which helps in encouraging electricity consumers to shift their consumption load to low carbon price time periods of the day. For accomplishing the carbon emissions forecasting task, an efficient model named as Improved Particle Swarm Optimization based Deep Neural Network (IPSO DNN) is proposed. This model is improved by tunning the parameters of DNN by newly proposed improved optimization technique named as IPSO. ISO-NE dataset is used for wind power and carbon emissions forecasting. To reduce the financial loss of power utilities ETD is very important. For this purpose four models are proposed, named as, Differential Evolution Random Under Sampling Boosting (DE-RUSBoost), Jaya-RUSBoost, RUS Ensemble CNN (RUSE-CNN) and anomaly detection based ETD. In DE-RUSBoost and Jaya-RUSBoost, the parameters of RUSBoost classifier are tunned by DE and Jaya optimization techniques, respectively. In RUSE-CNN, RUS data balancing technique is applied along with ensemble CNN to improve ETD performance. DE-RUSBoost, Jaya-RUSBoost and RUSE-CNN are supervised model that work on labeled electricity theft data. Whereas, anomaly detection based ETD model is capable of identifying electricity theft from unlabeled electricity consumption data. Real-world datasets of SGCC, UMass, PRECON, CER, EnerNOC and LCL are used for ETD. Simulation results show that all the proposed models perform significantly better on real-world dataset as compared to their state-of-the-art counterpart models. The improved feature engineering and model hyper-parameter tuning enhance the performance of the proposed models in terms of prediction and classification results.
Thesis
Full-text available
The revolution of power grids from traditional grids to Smart Grids (SGs) requires effective Demand Side Management (DSM) and reliable Renewable Energy Sources (RESs) incorporation in order to maintain demand, supply balance and optimize energy in an environment friendly manner. Data analytics provide solutions to the emerging challenges of power systems, such as DSM, environmental pollution (due to carbon emission), fossil fuel dependency mitigation, RESs incorporation, cost curtailment, grid’s stability and security. To efficiently manage electricity and maximize the profit of power utilities several tasks are focused in this thesis, i.e., prediction of electricity load to avoid demand and generation mismatch, wind power forecasting to satisfy energy demand effectively, electricity price forecasting for regulating market operations, carbon emissions forecasting for reducing payment of carbon tax, Electricity Theft Detection (ETD) for recovering power utilities’ revenue loss caused by electricity theft. In addition to that, a wind power forecast based DSM scheme is proposed. Furthermore, impact of RESs integration level on carbon emissions, electricity price and consumption cost is quantified. Both forecasting and classification techniques are utilized for efficient energy management. Forecasting of electricity load, price, wind power and carbon emissions is performed, whereas, classification of fair and fraudulent electricity consumers is performed. To balance electricity demand and supply, electricity load forecasting is required. Three models are proposed for this purpose, i.e., Deep Long Short-Term Memory (DLSTM), Efficient Sparse Autoencoder Nonlinear Autoregressive eXogenous network (ESAENARX) and Differential Evolution Recurrent Extreme Learning Machine (DE-RELM). DLSTM utilizes univariate data and gives single result, whereas, ESAENARX and DE-RELM model multivariate data and predict electricity load and price simultaneously. Due to adaptive and automatic feature learning mechanism, DLSTM achieves accurate results for separate forecasting of electricity load and price. ESAENARX and DE-RELM models are enhanced by newly proposed efficient feature extractor and model’s parameter tuning, respectively. Real-world datasets of ISO-NE, PJM, NYISO are used for load and price forecasting. The purpose of regulating the electricity market operations is achieved by forecasting of electricity load, price, wind power and carbon emissions. Wind power generation is predicted by an efficient model named Efficient Deep Convolution Neural Network (EDCNN). Moreover, a DSM strategy is also proposed based on predicted wind power generation. Power utilities have to pay carbon emissions tax imposed by government. To pay less carbon emissions tax, carbon emissions prediction is required, which helps in encouraging electricity consumers to shift their consumption load to low carbon price time periods of the day. For accomplishing the carbon emissions forecasting task, an efficient model named as Improved Particle Swarm Optimization based Deep Neural Network (IPSO DNN) is proposed. This model is improved by tunning the parameters of DNN by newly proposed improved optimization technique named as IPSO. ISO-NE dataset is used for wind power and carbon emissions forecasting. To reduce the financial loss of power utilities ETD is very important. For this purpose four models are proposed, named as, Differential Evolution Random Under Sampling Boosting (DE-RUSBoost), Jaya-RUSBoost, RUS Ensemble CNN (RUSE-CNN) and anomly detection based ETD. In DE-RUSBoost and Jaya-RUSBoost, the parameters of RUSBoost classifier are tunned by DE and Jaya optimization techniques, respectively. In RUSE-CNN, RUS data balancing technique is applied along with ensemble CNN to improve ETD performance. DE-RUSBoost, Jaya- RUSBoost and RUSE-CNN are supervised model that work on labeled electricity theft data. Whereas, anomaly detection based ETD model is capable of identifying electricity theft from unlabeled electricity consumption data. Real-world datasets of SGCC, UMass*, PRECON, CER, EnerNOC and LCL are used for ETD. Simulation results show that all the proposed models perform significantly better on real-world dataset as compared to their state-of-the-art counterpart models. The improved feature engineering and model hyper-parameter tuning enhance the performance of the proposed models in terms of prediction and classification results.
Thesis
Full-text available
The transformation of conventional grid into Smart Grid (SG) requires strategic implementation of the demand-sensitive programs while considering the varying fluctuations in the consumers’ load. The core challenges faced by existing electric system are that how to utilize electrical devices, how to tackle large amount of data generated by end devices and how to meet energy demands of consumers in limited resources. This dissertation is focused on the energy management of residential sector in the SG. For this purpose, we have proposed the Energy Management Controllers (EMCs) at three levels: at home level (including the single and multiple homes), at building level and at regional level. In addition, cloud and fog based environments are integrated to provide on-demand services according to the consumers’ demands and are used to tackle the problems in existing electric system. At first level, heuristic algorithms based EMC is developed for the energy management of single and multiple homes in residential sector. Five heuristic algorithms: genetic algorithm, binary particle swarm optimization algorithm, bacterial foraging optimization algorithm, wind driven optimization algorithm and our proposed hybrid genetic wind driven algorithm are used to develop the EMC. These algorithms are used for scheduling of the residential load during peak and off peak hours in a real time pricing environment for minimizing both the electricity cost and peak to average ratio while maximizing the user comfort. In addition, the advancements in the electrical system, smart meters and implementation of Renewable Energy Sources (RESs) have yielded extensive changes to the current power grid for meeting the consumers’ demand. For integrating RESs and Energy Storage System (ESS) in existing EMCs, we have proposed another Home EMC (HEMC) that manages the residential sector’s load. The proposed HEMC is developed using the earliglow algorithm for electricity cost reduction. At second level, a fuzzy logic based approach is proposed and implemented for the hot and cold regions of the world using the world-wide adaptive thermostat for the residential buildings. Results show that the proposed approach achieves a maximum energy savings of 6.5% as compared to the earlier techniques. In addition, two EMCs: binary particle swarm optimization fuzzy mamdani and binary particle swarm optimization fuzzy sugeno are proposed for energy management of daily and seasonally used appliances. The comfort evaluation of these loads is also performed using the Fanger’s Predicted Mean Vote method. For increasing the system automation and on-demand availability of the resources, we have proposed a cloud-fog-based model for intelligent resource management in SG for multiple regions at next level. To implement this model, we have proposed a new hybrid approach of Ant Colony Optimization (ACO) and artificial bee colony known as Hybrid Artificial Bee ACO (HABACO). Moreover, a new Cloud to Fog to Consumer (C2F2C) based framework is also proposed for efficiently managing the resources in the residential buildings. C2F2C is a three layered framework having cloud, fog and consumer layers, which are used for the efficient resource management in six regions of the world. In order to efficiently manage the computation of the large amount of data of the residential consumers, we have also proposed and implemented the deep neuro-fuzzy optimizer. The simulation results of the proposed techniques show that they have outperformed the previous techniques in terms of energy consumption, user comfort, peak to average ratio and cost optimization in the residential sector.
Chapter
Full-text available
District Heating networks (DHNs) are promising technologies for heat distribution in residential and commercial buildings since they enable high efficiency and low emissions. Within the recently proposed paradigm of smart grids, DHNs have acquired intelligent tools able to enhance their efficiency. Among these tools, there are demand forecasting technologies that enable improved planning of heat production and power station maintenance. In this work we propose a comparative study for heat load forecasting methods on a real case study based on a dataset provided by an Italian utility company. We trained and tested three kinds of models, namely non-autoregressive, autoregressive and hybrid models, on the available dataset of heat load and meteorological variables. The optimal model, in terms of root mean squared error of prediction, was selected. It considers the day of the week, the hour of the day, some meteorological variables, past heat loads and social components, such as holidays. Results show that the selected model is able to achieve accurate 48-hours predictions of the heat load in several conditions (e.g., different days of the week, different times, holidays and workdays). Moreover, an analysis of the parameters of the selected models enabled to identify a few informative variables.
Research Proposal
Full-text available
With the advent of Smart Grid (SG), the data collected by smart meters and Phasor Measurement Units (PMUs) has become a valuable source for grid operators and researchers to perform advanced analytics. In SG, the energy-related data is collected in a very huge volume, at a high velocity from a variety of sources. Data analytics provide solutions to the emerging challenges of power systems, such as: Demand Side Management (DSM), environmental pollution (due to carbon emission), fossil fuel dependency mitigation, reliable Renewable Energy Sources (RESs) incorporation, cost curtailment, grid’s stability and security. The global energy demand is increasing with the increasing population. The trivial power generation source, i.e., fossil fuel is decreasing continuously. Moreover, the environmental pollution is increasing at an alarming rate due to the carbon emission for trivial power generation sources. Therefore, effective DSM and RES incorporation have become important to maintain demand, supply balance and optimize energy in an environment friendly manner. DSM programs are based on the future energy consumption and price predictions. On the other hand, the reliable incorporation of RES is possible if there is a correct estimation of future generation. For this purpose, Deep Learning (DL) combined with data analytics techniques are proposed in this research. The aim of this research is to explore the SG databases and device solutions to the aforementioned problems. First, the predictive modeling is used for learning the consumption pattern from the data, to ensure the uninterrupted power supply. Predictive analytics are performed on energy price that is beneficial in effective DSM programs’ formulation. Moreover, as a popular RES, the wind power is analyzed and predicted. A DSM algorithm is proposed that considers the day-ahead energy price, consumption and wind power forecast for energy demand management. This research applies the data science techniques to the smart grid data as well as elaborates the benefits of this emerging data to the smart grid.
Article
Full-text available
Electricity price is a key influencer in the electricity market. Electricity market trades by each participant are based on electricity price. The electricity price adjusted with the change in supply and demand relationship can reflect the real value of electricity in the transaction process. However, for the power generating party, bidding strategy determines the level of profit, and the accurate prediction of electricity price could make it possible to determine a more accurate bidding price. This cannot only reduce transaction risk, but also seize opportunities in the electricity market. In order to effectively estimate electricity price, this paper proposes an electricity price forecasting system based on the combination of 2 deep neural networks, the Convolutional Neural Network (CNN) and the Long Short Term Memory (LSTM). In order to compare the overall performance of each algorithm, the Mean Absolute Error (MAE) and Root-Mean-Square error (RMSE) evaluating measures were applied in the experiments of this paper. Experiment results show that compared with other traditional machine learning methods, the prediction performance of the estimating model proposed in this paper is proven to be the best. By combining the CNN and LSTM models, the feasibility and practicality of electricity price prediction is also confirmed in this paper.
Article
Full-text available
In this paper, a novel modeling framework for forecasting electricity prices is proposed. While many predictive models have been already proposed to perform this task, the area of deep learning algorithms remains yet unexplored. To fill this scientific gap, we propose four different deep learning models for predicting electricity prices and we show how they lead to improvements in predictive accuracy. In addition, we also consider that, despite the large number of proposed methods for predicting electricity prices, an extensive benchmark is still missing. To tackle that, we compare and analyze the accuracy of 27 common approaches for electricity price forecasting. Based on the benchmark results, we show how the proposed deep learning models outperform the state-of-the-art methods and obtain results that are statistically significant. Finally, using the same results, we also show that: (i) machine learning methods yield, in general, a better accuracy than statistical models; (ii) moving average terms do not improve the predictive accuracy; (iii) hybrid models do not outperform their simpler counterparts.
Article
Full-text available
Motivated by the increasing integration among electricity markets, in this paper we propose two different methods to incorporate market integration in electricity price forecasting and to improve the predictive performance. First, we propose a deep neural network that considers features from connected markets to improve the predictive accuracy in a local market. To measure the importance of these features, we propose a novel feature selection algorithm that, by using Bayesian optimization and functional analysis of variance, evaluates the effect of the features on the algorithm performance. In addition, using market integration, we propose a second model that, by simultaneously predicting prices from two markets, improves the forecasting accuracy even further. As a case study, we consider the electricity market in Belgium and the improvements in forecasting accuracy when using various French electricity features. We show that the two proposed models lead to improvements that are statistically significant. Particularly, due to market integration, the predictive accuracy is improved from 15.7% to 12.5% sMAPE (symmetric mean absolute percentage error). In addition, we show that the proposed feature selection algorithm is able to perform a correct assessment, i.e. to discard the irrelevant features.
Article
Full-text available
Electricity price forecast plays a key role in strategic behavior of participants in competitive electricity markets. With the growth of behind-the-meter energy storage, price forecasting becomes important in energy management and control of such small-scale storage systems. In this paper, a forecasting strategy is proposed for real-time electricity markets using publicly available market data. The proposed strategy uses high-resolution data along with hourly data as inputs of two separate forecasting models with different forecast horizons. Moreover, an intra-hour rolling horizon framework is proposed to provide accurate updates on price predictions. The proposed forecasting strategy has the capability to detect price spikes and capture severe price variations. The real data from Ontario’s electricity market is used to evaluate the performance of the proposed forecasting strategy from the statistical point of view. The generated price forecasts are also applied to an optimization platform for operation scheduling of a battery energy storage system within a grid-connected micro-grid in Ontario to show the value of the proposed strategy from an economic perspective.
Article
Full-text available
The liberalization of electricity markets and the development of renewable energy sources has led to new challenges for decision makers. These challenges are accompanied by an increasing uncertainty about future electricity price movements. The increasing amount of papers, which aim to model and predict electricity prices for a short period of time provided new opportunities for market participants. However, the electricity price literature seem to be very scarce on the issue of medium- to long-term price forecasting, which is mandatory for investment and political decisions. Our paper closes this gap by introducing a new approach to simulate electricity prices with hourly resolution for several months up to three years. Considering the uncertainty of future events we are able to provide probabilistic forecasts which are able to detect probabilities for price spikes even in the long-run. As market we decided to use the EPEX day-ahead electricity market for Germany and Austria. Our model extends the X-Model which mainly utilizes the sale and purchase curve for electricity day-ahead auctions. By applying our procedure we are able to give probabilities for the due to the EEG practical relevant event of six consecutive hours of negative prices. We find that using the supply and demand curve based model in the long-run yields realistic patterns for the time series of electricity prices and leads to promising results considering common error measures.
Article
Full-text available
A short-term forecasting of the electricity price with data-driven algorithms is studied in this research. A Stacked Denoising Autoencoder (SDA) model, a class of deep Neural Networks (DNN), and its extended version are utilized to forecast the electricity price hourly. Data collected in Nebraska, Arkansas, Louisiana, Texas, and Indiana hubs in U.S. are utilized. Two types of forecasting, the online hourly forecasting and day-ahead hourly forecasting, are examined. In on-line forecasting, SDA models are compared with data-driven approaches including the classical neural networks (NN), support vector machine (SVM), multivariate adaptive regression splines (MARS), and least absolute shrinkage and selection operator (Lasso). In day-ahead forecasting, the effectiveness of SDA models is further validated through comparing with industrial results and a recently reported method. Computational results demonstrate that SDA models are capable to accurately forecast electricity prices and the extended SDA model further improves the forecasting performance.
Article
Electricity price forecasting is a significant part of smart grid because it makes smart grid cost efficient. Nevertheless, existing methods for price forecasting may be difficult to handle with huge price data in the grid, since the redundancy from feature selection cannot be averted and an integrated infrastructure is also lacked for coordinating the procedures in electricity price forecasting. To solve such a problem, a novel electricity price forecasting model is developed. Specifically, three modules are integrated in the proposed model. First, by merging of Random Forest (RF) and Relief-F algorithm, we propose a hybrid feature selector based on Grey Correlation Analysis (GCA) to eliminate the feature redundancy. Second, an integration of Kernel function and Principle Component Analysis (KPCA) is used in feature extraction process to realize the dimensionality reduction. Finally, to forecast price classification, we put forward a differential evolution (DE) based Support Vector Machine (SVM) classifier. Our proposed electricity price forecasting model is realized via these three parts. Numerical results show that our proposal has superior performance than other methods.
Article
A functional time series is the realization of a stochastic process where each observation is a continuous function defined on a finite interval. These processes are commonly found in electricity markets and are gaining more importance as more market data become available and markets head toward continuous-time marginal pricing approaches. Forecasting these time series requires models that operate with continuous functions. This paper proposes a new functional forecasting method that attempts to generalize the standard seasonal ARMAX time series model to the L <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> Hilbert space. The structure of the proposed model is a linear regression where functional parameters operate on functional variables. The variables can be lagged values of the series (autoregressive terms), past observed innovations (moving average terms), or exogenous variables. In this approach, the functional parameters used are integral operators whose kernels are modeled as linear combinations of sigmoid functions. The parameters of each sigmoid are optimized using a Quasi-Newton algorithm that minimizes the sum of squared errors. This novel approach allows us to estimate the moving average terms in functional time series models. The new model is tested by forecasting the daily price profile of the Spanish and German electricity markets and it is compared to other functional reference models.
Article
The liberalization of the power markets gained a remarkable momentum in the context of trading electricity as a commodity. With the upsurge in restructuring of the power markets, electricity price plays a dominant role in the current deregulated market scenario which is majorly influenced by the economics being governed. In the deregulated environment price forecasting is an important aspect for the power system planning. The problem of price forecasting can be entirely viewed as a signal processing problem with proper estimation of model parameters, modelling of uncertainties, etc. Among the different existing models the artificial neural network based models have gained wide popularity due their black box structure but it too has its own limitations. In the proposed work in order to overcome the limitations of the classical artificial neural network model, generalized neuron model is used for forecasting the short term electricity price of Australian electricity market. The pre-processing of the input parameters is accomplished using wavelet transform for better representation of the low and high frequency components. The free parameters of the generalized neuron model are tuned using environment adaptation method algorithm for increasing the generalization ability and efficacy of the model.
Conference Paper
The wind energy has an upward trend to continue over several decades, with the improvement of its turbine technology and reduction of energy production costs. Due to its unpredictable wind behavior and weather patterns, it may pose many problems including grid voltage balancing, synchronization, capital cost expenditure etc. So, there would be a definite need in forecasting the wind energy, its load and price. The use of these forecasting techniques may reduce the capital cost, installation and maintenance etc., Hence, in this article an attempt has been made to review the forecasting techniques for wind energy, load and price also tried to explore the best possible methods to forecast the behavior/pattern of the wind.