Experiment FindingsPDF Available
New forecast tools to enhance the
value of VRE on the electricity markets
Deliverable number:
D4.9
Work Package:
WP4 - Development of Open-access Market Simulation
Models and Tools
Lead Beneficiary:
LNEG
Page 2 of 51
Author(s) information (alphabetical)
Organisation
Email
LNEG
antonio.couto@lneg.pt
DLR
christoph.schimeczek@dlr.de
LNEG
hugo.algarvio@lneg.pt
Smartwatt
isabel.preto@smartwatt.pt
DLR
johannes.kochems@dlr.de
Smartwatt
tiago.santos@smartwatt.pt
Acknowledgements/Contributions
Name
Organisation
Email
Kristina Nienhaus
DLR
Kristina.Nienhaus@dlr.de
Ana Estanqueiro
LNEG
ana.estanqueiro@lneg.pt
Document information
Version
Date
Dissemina-
tion Level
Description
1
25.10.2021
Public
In this report the implementation of new forecasting
techniques and time synergies is explained.
Review and approval
Prepared by
Reviewed by
Approved by
See list above
Débora de São José
Ana Estanqueiro
Disclaimer
The views expressed in this document are the sole responsibility of the authors and do not necessarily reflect the views or
position of the European Commission or the Innovation and Network Executive Agency. Neither the authors nor the
TradeRES consortium are responsible for the use which might be made of the information contained in here.
Page 3 of 51
Executive Summary
The present deliverable was developed as part of the research activities of the
TradeRES project Task 4.4 - Enhancing the value of VRE on the electricity markets with
advanced forecasting and ramping tools.
This report presents the first version of deliverable 4.9, which consists on the descrip-
tion and implementation of the forecasting techniques aiming to identify and explore the
time synergies of meteorological effects and electricity market designs in order to maxim-
ize the value of variable renewable energy systems and minimize market imbalances.
An overview of key aspects that characterize a power forecast system is presented in
this deliverable through a literature review. This overview addresses the: i) forecast time
horizon; ii) type of approach (physical, statistical or hybrid); iii) data pre-processing proce-
dures; iv) type of forecast output; and v) the most common metrics used to evaluate the
performance of the forecast systems.
While in the TradeRES project work package 3 the conception of new market designs
and products are presented from a theoretical point of view, in this deliverable, the power
forecast capabilities to address the new designs and products are presented and dis-
cussed. The link between electricity markets time frames and the performance of the dif-
ferent power forecast approaches is analysed in this deliverable focusing on the day-
ahead market. Thus, as an initial step, a non-disruptive change in the day-ahead market is
proposed by simply postponing the gate closure hour according to the meteorological data
availability from the global numerical prediction models while the 24 hours forecast peri-
ods are still used.
Some preliminary results regarding the potential certainty gain effect from changing the
day-ahead market gate closure are presented and analysed in this deliverable. For the
Iberian electricity market, high power forecast errors are still observed, especially for wind
and solar power players. Even using a forecasting data from a forecast provider, limited
improvements are attained with the updated data from numerical global models.
In order to improve the forecasting accuracy for wind and solar power players, a new
forecast method developed within the scope of TradeRES is also presented. A key aspect
embedded in this method is the use of meteorological features that traditionally are not
used in power forecast systems and the definition of specific models according to the
weather conditions. Preliminary results suggest that the use of meteorological parameters
as wind gusts, wind power density, wind shear, and planetary boundary layer should be
used to improve the wind power forecast.
Another aspect regarding the meteorological time synergy and electricity markets ana-
lysed in this deliverable is the identification of extreme events. A wind power ramping
forecast approach implemented in the TradeRES forecast tools is described and aims to
complement the existing deterministic/probabilistic time series. This approach can be
used to increase the transmission system operators’ awareness level and helping them to
better scale the level of reserve required. Market players can also take advantage of this
information to define strategic bidding.
Page 4 of 51
Table of Contents
Executive Summary .................................................................................................. 3
Table of Contents ..................................................................................................... 4
List of Tables ............................................................................................................ 5
List of Figures ........................................................................................................... 5
List of Abbreviations ................................................................................................. 6
1. Introduction .............................................................................................................. 8
2. Power forecasts ......................................................................................................10
2.1 The forecasting process and objectives .........................................................10
2.2 Forecast time horizon .....................................................................................11
2.3 Type of approach ...........................................................................................12
2.3.1. Physical approaches ...............................................................................12
2.3.2. Statistical approaches .............................................................................17
2.3.3. Hybrid approaches ..................................................................................20
2.4 Data pre-processing .......................................................................................22
2.5 Forecast output: deterministic, probabilistic, or ramp events ..........................23
2.6 Metrics to evaluate the performance of the forecast approaches ....................24
3. Electricity markets time frames and power forecasts ..............................................27
3.1 Impact on market modelling ...........................................................................30
4. Forecast approaches developed in TradeRES project ............................................32
4.1 Preliminary results ..........................................................................................32
4.1.1. Wind, solar, small hydro and electricity demand forecast ........................32
4.1.2. Power forecast with feature selection ......................................................35
4.1.3. TradeRES - vRES power forecast approach ...........................................38
4.2 Wind power ramping forecast .........................................................................40
4.2.1. Ramp detection algorithm .......................................................................41
4.2.2. A nested forecast approach ....................................................................43
5. Final remarks ..........................................................................................................44
References ..................................................................................................................45
Page 5 of 51
List of Tables
Table 1. Time horizon, temporal scale and common application of the forecast approaches [17]. ..11
Table 2. Example of global and regional models available. Adapted from [21]. ...............................14
Table 3. Characteristics of the most common forecasting methods [15], [41], [42]. .........................19
Table 4. Key Schematic 2X2 contingency table for power ramp detection. Adapted from: [47]. ......25
Table 5. Scenarios analysed to quantify the certainty gain effect for different electricity market
players. ..............................................................................................................................................34
Table 6. Results for scenario 1 (wind power forecast) in relation to the DAM forecast at 06:00
(UTC). ................................................................................................................................................35
List of Figures
Figure 1. Recommended source of information/approach for solar and wind for the different time
horizons and spatial resolution. Adapted from [18]. ..........................................................................12
Figure 2. ECMWF forecast performance. Monthly wind speed root mean square error at 850 hPa
for one-day (blue) and five-days (red) time horizon forecast. Bold lines represent a 12-month
moving average of the results (figure extracted from [20]). ..............................................................13
Figure 3. From global to mesoscale/regional numerical models (figure extracted from [22]). ..........15
Figure 4. Main steps applied in the physical wind power forecast approaches. The interactions
indicated with the orange arrow represent an alternative approach within the physical forecast
approaches. .......................................................................................................................................16
Figure 6. Example of the main steps applied in the statistical forecast approaches for wind power
applications. .......................................................................................................................................19
Figure 5. Autocorrelation values of electricity demand in Portugal for different hourly delays. ........19
Figure 7. Main steps applied in the hybrid wind power forecast approaches based on a combination
of different forecast approaches. .......................................................................................................21
Figure 8. Different forecast outputs: a) deterministic, b) probabilistic, and c) ramp events (red
background represents periods with severe power ramps and green background represents
periods where power ramps are not expected). ................................................................................23
Figure 9. Forecast errors according to time horizon for different wind power forecast approaches.
HWP approach refers to a physical approach and “HWP/MOS” refers to a hybrid forecast
approach. Figure adapted from [33]. .................................................................................................28
Figure 10. Solar power forecast skills according to time horizon and type of forecast approach.
Figure extracted [79]. ........................................................................................................................28
Figure 11. Identifying the time synergy between the meteorological data availability and DAM
possible designs. D represents the day on which the simulation is carried out (Figure extracted
from [80]). ..........................................................................................................................................29
Figure 12. Histogram of power forecast error levels created in AMIRIS following a normal
distribution with a mean of 0.05 and a standard deviation of 0.1; 8760 hourly data points
representing one year. ......................................................................................................................30
Figure 13. Sample impact of power forecast errors on (non-realistic) DAM clearing prices; black
curve represents prices without power forecast errors, red dots resemble prices that include
modified renewable feed-in estimates based on the same error distribution function as shown in
Figure 12. ..........................................................................................................................................31
Figure 14. Daily average profiles for a) wind, b) solar PV and c) small hydro for the nine clusters
(profiles) for Portugal. ........................................................................................................................33
Figure 15. Average RMSE improvements for the seven wind parks analysed compared with the
benchmarking approach. “PCA-WithoutSFF” – PCA approach without applying SFF algorithm;
“NWPPoint+SFF” data from NWP was extracted to the nearest point of each wind park and the
SFF was applied; “PCA+SFF” – PCA approach and application of the SFF algorithm. ...................38
Figure 16. Power forecast approach implemented in TradeRES project. .........................................38
Figure 17. Example of one event in time t (black line) and one event in time t+1 (green line). The
magenta “*” symbols represent the average geometric center of each candidate, while the magenta
line indicates the trajectory of the meteorological event. ..................................................................42
Figure 18. Example of the outcomes from the nested forecast approach. .......................................43
Page 6 of 51
List of Abbreviations
AMIRIS
Agent-based Market model for the Investigation of Renewable and Integrated
energy Systems
ANN
Artificial neural networks
AR
Autoregressive
ARIMA
Autoregressive integrated moving average
ARMA
Autoregressive Moving Average
Bias
Bias Score
BRPs
Balance responsible parties
CFD
Computational fluid dynamics
D
Deliverable
DAM
Day-Ahead Market
ECMWF
European Centre for Medium-Range Weather Forecasts
FN
False negative
FP
False positive
GFS
Global Forecast System
hPa
Hectopascal
IBC
Initial and Boundary Conditions
IDM
Intraday markets
KNN
K-nearest neighbour
KSS
Hanssen & Kuipers Skill Score
Lat
Latitude
Long
Longitude
MA
Moving Average
MAPE
Mean absolute percentage error
MATREM
Multi-Agent TRading in Electricity Markets
Meso
Mesoscale numerical model
Micro
Microscale numerical model
ML
Machine Learning
NB
Normalized bias
NRMSE
Normalized root mean square error
NWP
Numerical Weather Prediction
OF
Objective function
P
Mean sea level pressure
PCA
Principal component analysis
PCs
Principal components
PDF
Probability density functions
POD
Probability of detection
PV
Photovoltaic
r
Pearson correlation coefficient
RMSE
Root mean square error
SCADA
Supervisory control and data acquisition
SFF
Sequential forward feature
SIDC
Single Intraday Coupling
TCT
Target-circulation types
TN
True negative
TP
True positive
TSO
Transmission System Operator
UTC
Universal time coordinated
vRES
variable Renewable Energy Sources
Page 7 of 51
WP
Work package
WR
Weather regime
WRF
Weather Research and Forecasting
Page 8 of 51
1. Introduction
The present deliverable was developed as part of the research activities of the
TradeRES project’s Task 4.4 - Enhancing the value of VRE on the electricity markets with
advanced forecasting and ramping tools (work package 4). This report presents the deliv-
erable D4.9 which consists of the description of forecasting techniques implementation
aiming to identify and explore the time synergies of meteorological effects and electricity
market designs to maximize the value of variable renewable energy systems and mini-
mize market imbalances.
One of the most important challenges in the energy sector is the large-scale integration
of renewable energy sources (particularly, variable renewable energy sources - vRES
such as wind and solar) into electrical power systems in an economic and environmentally
sustainable way. Transmission system operators (TSOs) must always ensure the balance
between electricity production and consumption. Currently, a safe and robust operation of
a power system needs highly accurate forecasts of both vRES power production and con-
sumption to minimize the need for balancing the energy in the reserve markets, typically
at high costs [1][3]. With near to 100% renewable power systems, the role of the forecast
system and its accuracy will be even more relevant.
Forecasts are also important to electricity markets. To participate in the different prod-
ucts from electricity markets, market players/actors need to rely on forecast systems to
build their bids. With the existing market designs, when power producers do not follow the
scheduled bid, they are penalized and its profits are strongly decreased [4]. Due to the
intrinsic chaotic nature of atmosphere, the participation of vRES players in the existing
markets is still a challenge, especially, when long time horizon forecasts are needed. In
work package (WP) 3 of TradeRES project the shortcomings and alternative designs for a
near 100% renewable electricity system were addressed. For day-ahead market (DAM),
which is the most used and with higher liquidity market, the authors of deliverable 3.5 [5]
suggested a reduction of the time gap between the DAM closure hour and the forecast’s
delivery time while keeping the current organization of wholesale electricity trade. Never-
theless, it is necessary to assess if the new gate closure’s timing are enough to reduce
expressively the vRES power forecast errors, or if it is necessary to replace the existing
designs [5]. Therefore, to design electricity markets that are adequate to vRES trading it is
crucial to understand the capabilities of power forecast systems as well as how they work
in order to i) identify potential new time frames for this specific market, ii) identify the play-
ers that have more challenges to participate in the existing DAM, and iii) develop the fore-
cast tools required for TradeRES project and the electricity markets for 2030 and beyond.
Another aspect regarding the synergy between meteorological timings and electricity
markets refers to the identification of extreme events. Extreme events as wind power
ramps usually have a significant impact on the DAM [6], [7]. In the case of wind power
ramps, the early identification and forecasting of these events triggered by weather condi-
tions can allow to raise the level of Transmission system operators’ (TSOs) awareness
helping them to better scale the level of risk that exists for the power system [7] as well as
commit additional reserves, to minimize operational risks. It should be noted that this type
Page 9 of 51
of information can complement (and does not replace) traditional time-series forecast,
allowing to dynamically allocate the level of reserves needed. Market players can also
take advantage of this information to participate strategically in electricity markets since,
under these conditions, large vRES forecast errors in DAM are expected [7].
Forecast of wind power ramps is a relatively novel research topic and the works al-
ready published highlight that the trigger mechanisms of such events are rarely similar
across the control regions or wind parks [8]. Nevertheless, one of the most successful
approaches to understand and forecast the dynamics of wind power ramps involve the
use of holistic approaches capable of accounting the spatial and temporal development of
atmospheric large-scale circulation [7], [9]. In [7], an automated cyclone detection algo-
rithm was settled to identify challenging weather situations for the TSO. In [9], the authors
also applied an automated cyclone detection algorithm and compared its performance
with a windstorm algorithm. The highest performance to detect wind power ramps is ob-
served with the windstorm detection algorithm. Nevertheless, all the previous algorithms
have a common shortcoming: a wind power ramp is neither always a consequence nor it
is always linked to the existence of extreme wind speed values, being essentially depend-
ent from the previous (historical) state of the atmosphere. In this sense, a new algorithm
that uses a time numerical differentiation to fit the particular case of wind power ramps
events was developed and is presented in this deliverable.
This deliverable is organized as follows: an overview regarding the power forecast sys-
tems is provided in section 2. In section 3, the link between electricity market time frames
and power forecasts capabilities are discussed. Section 4 provides some preliminary re-
sults. These results led to the development of the forecast approaches presented in this
section and that will be applied and tested in regional market case studies (WP 5). Section
5 briefly presents some final remarks.
Finally, it should be highlighted that the work conducted so far in T4.4, which is sum-
marised in this report, paves the way for addressing some of the market designs and
products choices identified in TradeRES. This first version of deliverable will focus on
DAM and a second version of this deliverable will be released at Month 41.
Page 10 of 51
2. Power forecasts
The forecasting problem is transversal to several sectors of activity, such as financial,
scientific, industrial, political, etc. In the energy sector, several systems have been devel-
oped in the recent years to predict the power output from wind or solar power plants as
well as the electricity demand. In general, forecast is performed in an instant, t, for a future
time horizon, t+k. Despite the developments observed in forecast tools/approaches, in the
energy sector, most of the applications for European countries still refer to the average
power, Pt+k, that is expected to be provided to the grid at time t+k. In the literature, sever-
al classifications for the power forecast systems are available [10]. These systems can be
classified according to the forecast time horizon and type of approach. In the next subsec-
tions, these classifications are addressed as well as some of the additional steps to im-
plement a forecast system.
It should be noted that this chapter provides a summary of approaches commonly ap-
plied in the energy sector aiming to highlight the different options available. This back-
ground is important to establish how to proceed to implement the forecast approaches
most suitable to the different needs of the project. Detailed and up-to-date literature re-
views forecast approaches can be found about wind power [1], [2], [10], solar power and
electricity demand [11], [12].
2.1 The forecasting process and objectives
The forecasting process aims to transform one or more independent variables (inputs)
into one or more dependent variables (outputs). This process is characterized by some
key steps [13]:
- Problem definition
- Data collection
- Descriptive data analysis
- Forecast model selection
- Model validation
- Forecast
- Performance evaluation
Problem definition consists of evaluating the forecast period, forecast horizons and the
time step of the outputs. The type of output needed and the establishment of admissible
errors in the results are also established in this step. In the data collection phase, the vari-
ables under study (object of the forecast) and the independent variables necessary to
build the forecast model need to be collected. For the descriptive analysis of the data, it is
necessary, in the first place and when working with a time series, to take into account that
successive observations are not independent events [14], and as such, the order of ob-
servations must be respected. According to [13], to obtain greater sensitivity of the data
under analysis, they should be represented in the form of a temporal graph and a sum-
mary of some statistical parameters should be computed. This procedure makes possible
to identify anomalies in the data, trends and seasonality that otherwise might not be evi-
dent.
Page 11 of 51
After analysing the data, the forecasting method is applied. This task consists of choosing
and adjusting one or more models to the specific case study, that is, reproducing the de-
pendent variable, depending on the independent variable (or variables), within a certain
margin of error. The model selection should take into consideration aspects as the time
horizon and the type of information expected from these models (deterministic, probabilis-
tic, or ramp event forecast). Once selected, the method must be validated. This validation
is done by assessing the performance of the forecast. For this purpose, the method is
normally adjusted to only a part of the available data, with the rest being used for its vali-
dation. Once validated, the method is implemented and its control is carried out continu-
ously by measuring the forecast errors (e.g., bias) to verify the continued validity of the
method, and, if necessary, make all necessary updates to reduce the errors.
2.2 Forecast time horizon
The time duration for which the power output is forecast is known as the forecast time
horizon. The forecast horizon of interest depends on the different applications, and in en-
ergy sector it can be divided into four main time scales: very short-term, short-term, medi-
um-term, and long-term [1], [15], [16] The time frames of this classification can slightly
change among the different authors. The application for the different time frames is de-
picted in Table 1.
Table 1. Time horizon, temporal scale and common application of the forecast approaches [17].
Time horizon
Temporal scale
Applications
Very short-
term
From seconds to 30
minutes
- Real-time dispatch and regulation
operations on the network;
- Forecasting the consumption of build-
ings in the context of micro-grids
Short-term
From 30 minutes to 6
hours
- Impacts on energy price determination
in intraday markets;
- Support decision on the status of net-
work loads;
- Support the decision to turn-on or off
the generator set with quick response;
- Security operations for the energy
market.
Medium-term
Varies between 6 hours
and 1 day
- Support the decision to turn genera-
tors on or off;
- Safety time horizon for the day-ahead
market.
- Impacts on energy price determination
- Allocation of power reserves
Long-term
More than a day
- Planning of maintenance operations;
- Power system adequacy planning
Page 12 of 51
2.3 Type of approach
In this section, the most common types of forecast approaches used in the energy sec-
tor are presented. Figure 1 depicts the recommended source of information/forecast ap-
proach according to the spatial and temporal horizon that will be further analysed in the
next subsections.
Figure 1. Recommended source of information/approach for solar and wind for the different time
horizons and spatial resolution. Adapted from [18].
2.3.1. Physical approaches
The physical power forecast approaches are mainly based on the use of numerical me-
teorological models Numerical Weather Prediction (NWP), which parameterize and sim-
ulate in detail the atmosphere and its circulation mechanisms. NWP models provide me-
teorological parameters as wind components, cloud coverage, air temperature, and pres-
sure, that are used to generate forecasts.
This type of model is being developed since 1950 when NWP models were used to
make weather forecasts with time horizons on the scale of days. They were, however,
very primitive models based on quasi-geostrophic theories where it was impossible, either
due to lack of knowledge or lack of computational resources, to include relevant physical
processes (radiation processes and phase transition) to make reliable predictions[19].
Over the years and with substantial technological improvement, these models and the
parameterizations that govern them were improved and the relevant physical processes
missing were progressively added. Currently, these models are still the core of weather
forecasting and have evolved substantially following the growing knowledge regarding the
physical processes that govern the atmosphere dynamics and its circulation as well as the
computational capabilities.
Spatial Resolution (km)
Time Horizon (h)
0.1 1 10 100 1000
0.1
1
10
100
1000
0.01
Wind
Solar
Physical
Statistical
Satellite images
Sky images
Meso.
Global
Micro.
Page 13 of 51
NWP models are, nowadays, less simplistic and with more detailed and precise physi-
cal parameterizations. Additionally, these models benefited from more efficient and repre-
sentative data acquisition and assimilation systems around the globe, which includes me-
teorological stations, satellite data, radiosondes and measurements performed by air-
planes and ships. All these improvements lead to a significant decrease in the forecast
errors, as reported for the operational European Centre for Medium-Range Weather Fore-
casts (ECMWF) model, Figure 2. This figure also highlights that the errors are significantly
higher for a time horizon of five-days compared to one-day time-horizon.
Figure 2. ECMWF forecast performance. Monthly wind speed root mean square error (RMSE)
at 850 hPa for one-day (blue) and five-days (red) time horizon forecast. Bold lines represent a 12-
month moving average of the results (figure extracted from [20]).
There are two major groups of NWP models: global models (grid covering all the Earth)
and regional/mesoscale models (also known as limited area models) [21]. The main dif-
ferences between these two groups are related to the spatial and temporal resolution of
the model, the geographical area covered and the time horizon. Moreover, region-
al/mesoscale models are calibrated using physical parameterization for specific regions
which can enable to reduce the forecast errors. These differences will have a significant
impact on forecast accuracy and computational effort [21].
Table 2 characterizes the spatial and temporal resolution of some of the existing global
and mesoscale/regional model.
Page 14 of 51
Table 2. Example of global and regional models available. Adapted from [21].
Type
of
model
Provider
Model
Temporal
resolution
(hour)
Horizontal
resolution
(aprox. km)
Runs per
day (UTC)
Global
European Centre
for Medium-Range
Weather Forecasts
(ECMWF)
Integrated Fore-
casting System
1
10
4 (00, 06,
12 and 18)
Canadian Mete-
orological Centre
Global Determinis-
tic Prediction Sys-
tem
3
25
2 (00, 12)
National Centers
for Environmental
Prediction
Global Forecast
System (GFS)
3
25
4 (00, 06,
12 and 18)
Deutscher Wetter-
dienst
Icosahedral Nonhy-
drostatic
1
13
4 (00, 06,
12 and 18)
Regional
Deutscher Wetter-
dienst
Consortium for
Small-scale Model-
ing
1
2.8
8 (00, 02,
…, 18 and
21)
Finnish Meteoro-
logical Institute
High Resolution
Limited Area Model
1
7.5
4 (00, 06,
12 and 18
Weather Research
and Forecasting
(WRF)1
Defined by
user (< 1
hour)
Defined by
user (< 5 km)
User specif-
ic (but lim-
ited to the
global mod-
el availabil-
ity)
Most of power forecast systems use a coupled approach by feeding the regional mod-
els with initial and boundary conditions (IBC) gathered from the global models. This ap-
proach aims to mitigate some of the drawbacks associated with global models that pre-
sent low spatial and temporal resolutions by describing the behavior and evolution of the
air masses, and explicitly treat the atmospheric phenomena that need high spatial and
temporal resolution. On the other hand, numerical mesoscale/regional models always
need to be forced with IBC at the limits of their domains (boundary, surface, and top of the
domain), Figure 3. These IBC can be historical data from reanalysis projects (used to pro-
duce wind or irradiance atlases for example) or by operational global forecasts projects
(as GFS).
1
Example weather forecast providers using WRF model: Climetua (http://climetua.fis.ua.pt/weather) and Me-
teoGalicia (https://www.meteogalicia.gal).
Page 15 of 51
Figure 3. From global to mesoscale/regional numerical models (figure extracted from [22]).
Despite the improvement observed in NWP models, these models still present sys-
tematic errors partly explained by the: 1) inadequate model’s physics parametrizations; 2)
inability to handle sub-grid scale phenomena; 3) stochastic behaviour of the atmosphere
and 4) uncertainty on IBC [21], [23]. Although, the precision of the results of these models
increases proportionally to the number of data assimilated in these models, as well as the
quality of these same data [21].
At the current stage of NWP, the models parameterization and the spatial resolution
from these models are unable to simulate some local effects as the exact location and
extent of cloud fields in the case of solar power forecast. To properly account for these
effects and correct the outputs from NWP, downscaling techniques can be applied to cor-
rect the data providing location-specific forecasts [24], [25]. Thus, downscaling consists of
applying further methods to enhance the data extracted from the NWP with local/regional
effects. This process can be performed through various statistical methods that establish
relationships between local variables (such as wind speed) and variables with large-scale
characteristics (such as pressure fields). Another possibility is the use of other physical
approaches that varies according to the type of technology under consideration. Below,
some examples of downscaling physical approaches for the wind and solar power cases
are provided.
Wind power
Microscale models: This type of model allows working with high spatial resolutions (up to
10 30 meters). With the growing need to estimate accurately the wind resource for dif-
ferent applications, new models for simulation of wind flow were developed. These models
can be classified into linear and non-linear [26]. Linear models as the Wind Atlas Analysis
and Application Program software have the advantage of low need for computing re-
sources and it enables to evaluate, with reasonable accuracy, the wind resource for flat
Page 16 of 51
orography with small elevations, i.e., under non-complex terrain conditions [19]. However,
these models tend to, e.g., miscalculate the wind speed behaviour in the lee side of the
hills [26]. Therefore, these models are unsuitable for complex terrain. The advances in
numerical modelling together with the increase in computational capabilities enabled the
development of non-linear models in the flow simulation industry and in the assessment of
wind potential. Among these non-linear models, in the wind sector, computational fluid
dynamics (CFD) models stand out enabling to increase the accuracy of wind potential
assessments, especially in complex terrain [27]. Results from several authors highlighted
the benefits of this model against the linear models [28]. These benefits are derived from
the inclusion of thermal effects in the vertical stratification of the CFD simulations. This
type of approach was explored in [29] showing higher performance when compared to a
traditional statistical approach, especially for periods with high or low energy levels and in
the ascending wind power ramps.
Conversion to power: This type of approach is based on the power curves from the
wind turbine manufacturers/or estimated based on historical wind speed and power data
to determine the power from the wind turbine or park [30]. Typically, the historical power
curves of wind turbines/parks are defined as a function of additional parameters, such as
wind direction to account for physical features (e.g., wake effect of wind turbines, air den-
sity and terrain) in surrounding regions [30][32]. Based on the results of the NWP, name-
ly, the wind speed and direction for the hub height of the wind turbine, the power curves
are applied, and the forecast is obtained. This type of approach is the most common dur-
ing the initial periods of wind park operation where there is no historical data.
Figure 4 presents the main steps commonly applied to obtain the physical-based wind
power forecast.
Figure 4. Main steps applied in the physical wind power forecast approaches. The interactions
indicated with the orange arrow represent an alternative approach within the physical forecast ap-
proaches.
Microescale models
NWP data
Wind speed Wind direction Atmospheric
pressure
Downscaling to
Wind turbine
Hub height Power Curve Forecasting
Power
Terrain Surface
Roughness Obstacles
Page 17 of 51
Solar power
Cloud and all-sky imagery: In addition to the daily cycle, the other factor with the great-
est impact on ground-level solar irradiation and, consequently, on solar production is the
cloud cover [33]. This parameter presents a high variability that can be induced by local
effects, which are not always correctly described by NWP. In recent years, models that
forecast cloud cover from images from the sky taken from all-sky cameras or by artificial
satellites have been developed. The all-sky cameras, installed at ground level, allow to
obtain images of the sky, being very useful for very short and short-term forecast time
horizons due to the reduced field of view of the camera [34], [35]. On the other hand,
through satellites, it is possible to obtain images with a wide field of view, but with lower
spatial and temporal resolution. In this sense, information obtained through satellites is
more suitable for short and medium-term forecasts [36]. Regardless of the source of in-
formation (cameras or satellite), this type of approach uses algorithms that identify cloud
coverage patterns between sequential images. The information from the images can be
transformed, for instance, into cloud motion vectors, enabling to determine the move-
ments (intensity and direction) of individual clouds [37]. This information regarding the
cloud cover can then be used in different ways: i) correct the NWP outputs, or ii) apply
semi-empirical models to obtain ground level solar irradiance.
The use of these models is crucial for weather-dependent generation technologies as
wind and solar power forecast. Nevertheless, the outputs from these models are also
used by several authors to improve the load forecast for short- and medium-term forecast
horizons [12].
2.3.2. Statistical approaches
To overcome the inefficiencies of the physical methods described above and, at the
same time, obtain operational forecasts with adequate precision to manage the variability
of vRES, several statistical methodologies have been developed. These forecasting ap-
proaches are based on statistical approaches using historical time series (observed or
forecasted) data. More precisely, this type of approach seeks to establish relationships
between historical data series with what is currently observed, at the instant for which the
forecast is to be made. Despite the constant emerging of new forecast techniques that
require deep mathematical knowledge, compared to physical forecasting methods, this
type of method presents reduced complexity and is less costly (whether in terms of time or
resources) since the physical processes are not explicitly expressed. Within the statistical
models, three distinct methods are usually considered: persistence, time series modelling,
and automatic learning methods.
Persistence methods: Persistence-based forecasting methodology is the most basic
and simplest statistical forecasting methodology to be implemented. Despite belonging to
the group of statistical forecasting methods, it is often addressed separately since it is
considered as reference, or benchmark, against all other used methodologies. In this
sense, to study the feasibility of implementing new forecasting methodologies, the results
obtained through these must be matched with the results achieved by the persistence
method. Only methodologies that present more favourable results than those generated
by persistence are likely to be implemented. The persistence method assumes that the
Page 18 of 51
wind/solar power or electricity demand, remains equal, at a future instant, to the value
observed at the instant for which a forecast is made. If the power, at time t, is given by ,
then the power at the future time, t+∆t, will be given by:

(1)
where  corresponds to the time interval for which the forecast is to be performed. For
very short forecast time horizons (Table 1), this model provides results, on average, with
some accuracy. However, and as expected, due to the vRES variability as the forecast
time horizon increases, the accuracy of this methodology decreases. For time series with
high fluctuation in production (e.g., wind parks in areas with complex orography), the ac-
curacy of this method is reduced. For solar power and electricity demand, the persistency
can assume the value of the previous day and/or week.
Methods based on time series modelling: For very short and short forecast time hori-
zons (Table 1), there is the possibility, with a certain degree of reliability, to use methods
based exclusively on the statistical analysis of time series of real data. Specifically, this
type of methodology tries to find out what is the relationship between a historical series of
production or demand data, and its value at the instant for which the forecast is to be
made, in order to obtain predictions for the following instants. Unlike physical models, in
this type of forecasting methodology, only one step is needed to convert the input data
into output data. Among the various statistical methods used in wind forecasting, the auto-
regressive (AR), moving average (MA), autoregressive moving average (ARMA), auto-
regressive integrated moving average (ARIMA), the Box-Jenkins methodology, and the
use of Kalman filters [1], [38].
Machine Learning Methods: A group of statistical data analysis methods that are ap-
plied to problems of this nature is machine learning methods. Machine learning (ML) or by
“gray box” [39] are models essentially characterized by the capacity for self-learning
through experience and training, i.e., ML offers the computational capacity to learn without
explicit programming. As such, ML algorithms present the possibility of learning and mak-
ing predictions on a set of data in an unexplicit way without following a set of static statis-
tical learning instructions. The process of a self-learning model includes a few steps.
Firstly, it is necessary to obtain data referring to the past for a later training phase.
Secondly, a relationship between the input and output data the input and the output
desired through a target function is defined. The third step is to choose the self-learning
model. Then, this model undergoes a training process, using the set of data and previous-
ly determined examples. In most cases, this type of methodology generally presents the
best forecast results. However, the implementation of these methodologies has the disad-
vantage that it is not possible to describe the relationship between the elements of the
model, i.e., it is not possible to describe or understand the relationships found by these
models between the input variables and the output variables [40]. Among the methods of
this group, the artificial neural networks (ANN) stand out for their simplicity and efficiency.
One of the disadvantages of statistical methods (e.g., multiple linear regression) lies in
the inability to deal with the occurrence of events with distinct patterns within the time se-
ries itself, namely, the distinction between weekly, weekend and special days (e.g., holi-
Page 19 of 51
days). ANN-based methods, on the other hand, are capable of accepting these character-
istics as an independent variable and modelling implicit non-linear relationships between
the forecast variable and the variables that affect it.
Figure 5 depicts an example of the main steps used to apply statistical forecast ap-
proaches. As can be seen in the figure for the case of wind power forecast typically only
wind speed and power historical data are needed.
Figure 5. Example of the main steps applied in the statistical forecast approaches for wind pow-
er applications.
This type of approach is also common for load demand forecast [15]. In this case, soci-
oeconomic variables economic growth rates are also frequently considered as well as
historical values (delays) since the autocorrelation between successive events is high at
different temporal scales, as can be observed in Figure 6. This figure depicts the energy
demand autocorrelation in Portugal continental for different. The correlation with a 24 hour
delays is 0.97.
Figure 6. Autocorrelation values of electricity demand in Portugal for different hourly delays.
Table 3 presents a list of the most common statistical and learning approaches and
their advantages/disadvantages.
Table 3. Characteristics of the most common forecasting methods [15], [41], [42].
Power
Speed Statistical
models
Wind
power
Historical data
Page 20 of 51
Forecasting
method
Advantages
Disadvantages
Linear Re-
gression
Simplicity;
You can use only one (Single) or several
(Multiple) independent variables.
Only capture relationships between
linearly correlated variables;
Sensitive to extreme values (outli-
ers);
Variables used in forecasting must
be linearly independent.
Time Series
Analysis
(Box-
Jenkins)
Adaptable, there are many versions of the
method and it has already been exten-
sively studied;
Able to deal with seasonality and non-
stationarity.
Requires only historical data for the
series;
Unlikely to perform well in long-term
forecasting;
It is computationally demanding to
estimate model parameters;
Requires a solid knowledge of sta-
tistics inherent to the time series.
K-nearest
neighbour
(KNN)
It is relatively simple to understand and
implement;
It does not need a training phase, making
its prediction based on observed historical
values;
Non-parametric approach, which does not
imply any assumptions regarding the dis-
tribution of the variables to be predicted.
Requires an extensive period of
historical data;
Computationally demanding for
large datasets.
Artificial
Neural Net-
works
It is not necessary to know the relation-
ship between dependent and independent
variables;
Capable to deal effectively with non-linear
relationships;
Capable to deal with the presence of
noise in the dataset without significantly
affecting the forecast result.
It is computationally demanding to
train the neural network;
Requires a large amount of histori-
cal data from independent variables;
Do not result in a mathematical
model with physical meaning.
Support
Vector Ma-
chine
Adjustment of the adjustment parameter
of the objective function helps to avoid
over-fitting the training data (over-fitting);
Convex optimization problem (there are
no local minima);
Use of the kernel trick, which maps the
variable space to a non-linear vector
space, allowing to capture non-linear rela-
tionships more efficiently.
It is difficult to define a “good” kernel
function;
Computationally demanding for
large datasets.
2.3.3. Hybrid approaches
Hybrid power forecast models are models that combine two or more models of similar
or different nature [40]. The genesis of such approach results from the combination of
both statistical and physical models. The time scales applicable to forecasting methods
can also be different because it is possible to join methods whose forecast horizon is dif-
ferent. In general, hybrid models are composed of a linear model and a non-linear model
to be able to analyse the respective linear and non-linear components of the time series of
data. Hybrid methods can be categorized into four different classes:
I. Weight-based methodologies: These methodologies are based on assigning weights
to the various forecast models used according to their performance. It is a simple method-
ology, easy to implement and has the advantage of adapting to new datasets. It is a suita-
Page 21 of 51
ble methodology for a wide range of forecast time horizons. However, this does not guar-
antee the best forecasting efficiency for the entire forecasting time horizon and has the
need for an additional model to assign the weights.
II. Methodologies based on the combination of different forecast approaches: the pre-
diction is performed via the combination of different types of approaches (combine physi-
cal with statistical approaches). This type of methodology presents a robust behavior due
to the sudden, nature of the vRES generation wind speed. Therefore, it is possible to ob-
tain high forecasting efficiencies. However, this type of methodologies has the disad-
vantage of requiring the user to understand the complex mathematical model that per-
forms the data decomposition and the absence of a dynamic behavior in the sense that
the use of new data series, or the updating of the same, may result in a slow response of
the methodology.
III. Methodologies based on optimization techniques and parameter selection: These
methodologies are based on the optimization of the forecast model parameters mete-
orological parameters such as temperature, wind speed and direction, precipitation,
among others. Despite providing the user with a greater understanding of the impact of
different parameters on forecasting effectiveness, this approach is difficult to implement.
IV. Methodologies based on post-data processing techniques: These methodologies
are based on a post-processing of forecast data. More specifically, this approach studies
the impact of residual errors in the forecasts obtained, through the forecast models used,
on the overall effectiveness. The effectiveness of forecasts considerably increased com-
pared to other approaches. However, due to the need to calculate residual errors, compu-
tational and temporal resources required are much higher than those needed for the other
approaches presented.
Figure 7 provides the main steps typically applied in hybrid power forecast approaches.
Comparing with the previous approaches, it includes both NWP and historical data to feed
the statistical models. As discussed in this section, for this type of approach different vari-
ations can be used.
Figure 7. Main steps applied in the hybrid wind power forecast approaches based on a combi-
nation of different forecast approaches.
Power
Speed Statistical
models
Wind
power
NWP data
Wind speed Wind direction Atmospheric
pressure
Historical data
Page 22 of 51
2.4 Data pre-processing
Before applying statistical forecasting methods, it is common to apply pre-processing
procedures to the data under analysis [15], [43][45]. The most common types of pre-
processing are data cleaning, integration, transformation and dimensional reduction.
These treatments can be used in various combinations or alone. Data cleaning consists of
removing or modifying values from incorrect values and entering missing values. Integra-
tion consists of combining data from different sources. The transformation consists, for
example, in normalizing the data to scale them on a predefined range (e.g., [0, 1]) or in
transforming a value recorded every 15 minutes into an hourly average.
Dimensional reduction consists of reducing the number of existing variables. This re-
duction can be done, for example, through principal component analysis (PCA), discrimi-
nant analysis, empirical mode decomposition or wavelet analysis [46]. In PCA, an orthog-
onal transformation is applied to convert the data into a set of values of linearly uncorre-
lated variables designated as principal components (PCs) [47]. With this procedure, the
number of PCs generated in the process is always equal to or less than the number of
original variables. The transformation applied with this technique allows the first PC to
explain the largest possible variance, i.e., this PC characterizes the maximum possible
variability observed in the data. With the restriction that it is orthogonal, the subsequent
PC has the greatest possible variance that was not explained by the previous one. The
process continues until the number of PCs became equal to the number of original varia-
bles. The resulting vectors enable to obtain an uncorrelated orthogonal basis set and they
are used to feed the statistical forecasting techniques.
Another type of approach to reduce the data dimension is the application of feature se-
lection algorithms [46]. The selection of the most relevant features aims to remove insig-
nificant entries in the forecasting models allowing to reduce model complexity as well as
computational costs [48]. These methods can be classified into three different types [45],
[48]: filter, wrapper and embedded. The filter methods remove the less significant varia-
bles a priori, and then a model is created with the remaining features. Variables are elimi-
nated with a criterion such as Pearson correlation. The wrapping methods involve the en-
tire training algorithm in the variable selection process. The algorithm runs with several
iterations (as many as there are variables) of the model by adding (or removing) variables
and evaluating the performance of the model obtained. For the construction of the final
model, the variables that enable to improve the result are kept and the rest discarded.
Embedded methods introduce the variable selection process directly into the training pro-
cess, in order to avoid the complete search that happens in wrapped methods, thus re-
ducing computational complexity [49]. Additionally, combinations of these methods can be
created, giving rise to the so-called hybrid methods. All these techniques aim to ensure
the robustness of the data, also bringing benefits in improving computational efficiency.
Another type of pre-processing is the decomposition and classification of data by clus-
tering [46]. Decomposition, in the context of the analysis of electricity demand forecast
refers to the data separation according to the seasonal, weekly, and special days (holi-
days) effects. For the vRES case, this classification can refer to the so-called weather
regimes (WR) types [50] or target-circulation types (TCT) [51]. The WRs allow to reduce
the complexity of meteorological variability while enabling the identification of daily recur-
Page 23 of 51
rent patterns in the climate system (top-down approach). On the contrary, TCT are de-
rived from the power system's weather response (down-top approach), which can be the
vRES generation [52].
2.5 Forecast output: deterministic, probabilistic, or ramp
events
The initial focus of power forecast systems was to provide deterministic information, i.e.,
a value for each time step of the temporal horizon. Famous statistical methods are ARMA,
ARIMA, Kalman filtering and Gaussian mixture models [39], [53], [54]. Other robust statis-
tical approaches include ANN, KNN among others [55][57].
As presented in Figure 8, deterministic forecast approaches do not include information
regarding its uncertainty, which can be very useful for utilities or for specific market players
through the definition of strategic bidding [58]. The need to characterize and assess the
uncertainty in the vRES power forecast to better integrate it in decision-making processes
led to the development of various probabilistic forecasting techniques [59][63]. Probabilis-
tic forecasts can allow: i) increase the revenues of market players within the electricity envi-
ronments, e.g., [64], [65], and ii) suitable reserve allocation [66].
Broadly speaking, the probabilistic forecast allows to obtain probability density functions
(PDF) for an specific time providing an interval of uncertainty of future events. The PDF
can be achieved using physical approaches (e.g., NWP ensembles [67], [68]), statistical, or
the combination of both. As described in [67], NWP ensembles forecasts are computation-
ally demanding when compared with statistical methods. Power forecast uncertainty using
statistical/hybrid models can be attained by calculating their distribution parameters based
on: i) nonparametric regression assumptions as quantile regression [61] and kernel density
estimators [69], [70], or ii) upon historical analogous [71][73] or iii) parametric distribution
assumptions.
Figure 8. Different forecast outputs: a) deterministic, b) probabilistic, and c) ramp events (red
background represents periods with severe power ramps and green background represents peri-
ods where power ramps are not expected).
Ramp events refer to the significant changes of power output in a short period. Thus,
the importance of the detection of severe power ramps for TSOs lies on the necessity to
control conventional power plants to balance those ramps in order to ensure the stable
Power
1 2 3 4 5 6 7 8 t+k
Time
1 2 3 4 5 6 7 8 t+k
Time
1 2 3 4 5 6 7 8 t+k
Time
a) b) c)
Page 24 of 51
operation of the power system. Contrary to deterministic and probabilistic that provide time-
series, this type of forecast provides binary information: existence or not of a power ramp
[74]. This information should be integrated into the existing forecasting systems as an addi-
tional feature, but must not substitute the existing forecasting systems [7]. Thus, the main
potential benefit of power ramps forecast is to alert the TSO regarding the existence (or
not) of power (rapid) ramps events for which they should be prepared to commit additional
reserves for safety and guarantee of robustness of the power system, in all meteorological
conditions [7], [47].
2.6 Metrics to evaluate the performance of the forecast
approaches
There is no single metric that can describe or measure the performance of a forecasting
methodology. In existing literature, some new deterministic and probabilistic metrics have
been proposed in the last years [75]. Nevertheless, most of them are not being adopted to
energy sector being difficult to place the results among the values found in the literature.
Taking into account this aspect, the following metrics will be used in TradeRES project to
access the accuracy of time-series forecast - normalized bias (NB), RMSE or the normal-
ized RMSE (NRMSE), the Pearson correlation coefficient (r) and the average value of the
absolute forecast deviation (
):

 
 
(2)
 

(3)


(4)
 

 
 
(5)

  


(6)
 corresponds to the total nominal power of the control region or vRES
power parks under analysis. The bias corresponds to systematic error present in the fore-
cast. This metric denotes an average error value for the forecast time horizon allowing to
assess whether the forecasting methodology tends to underestimate or overestimate
comparing with the observed values. Ideally, a bias is sought, for the time horizon, as
close as possible to zero. RMSE allows to identify the variation of amplitude errors, due to
the squared nature of the differences. NRMSE, as aforementioned discussed, normalizes
the RMSE for an easily comparison between different forecast approaches. The perfect
score of the last two metric is also zero. The correlation coefficient measures the similari-
ties between the obtained forecasts, for a forecast time horizon, and the observed value
Page 25 of 51
for the same time horizon. This coefficient varies between [-1 1]. A value close to zero
means poor predictions, and the unit value represents perfect predictions. A value close to
-1 means that the forecast is in phase opposition. The average value of the absolute fore-
cast deviation allows to illustrate the mean forecast deviation in relation to the observed
power.
To quantify the improvement of using the forecast methods proposed in TradeRES pro-
ject, the approach followed in [73] is used for each metric:


(7)
where represents the results of a specific metric using the forecast
method implemented in TradeRES project, and  represent the forecast
results for the benchmarking approach (that will be defined according to each case study).
A positive value indicates an improvement of the proposed forecast method. A negative
value corresponds to an underperformance of the TradeRES forecast method. It needs to
be mentioned that forecast tool proposed (section 4.3) provides probabilistic information.
Thus, the previous metrics will be applied to the quantiles obtained. The quantile closer to
the ideal score will be used.
Ramps power events refer to a dichotomous case, i.e., the existence or not of a pow-
er ramp. For this type of approach, a contingency table are usually built to derive the re-
sults. In Table 4, true positive (TP) corresponds to the ramps forecasted with the pro-
posed methodology that occurred; false positive (FP) corresponds to the ramps forecast-
ed but do not occur; false negative (FN) corresponds to power ramp events that occurred
but were not forecasted; and true negative (TN) corresponds to power ramps forecasted
and observed.
Table 4. Key Schematic 2X2 contingency table for power ramp detection. Adapted
from: [47].
Event Foreseen
Event Observation
Total
Yes
No
Yes
TP
FP
Foreseen Yes
No
FN
TN
Foreseen No
Total
Observed Yes
Observed No
N=TP+FP+FN+TN
From the contingency table the following metrics can be computed: Bias Score (Bias),
precision, the probability of detection (POD) and the Hanssen & Kuipers Skill Score
(KSS):


(8)
Page 26 of 51


(9)


(10)


(11)
The ideal score for the aforementioned metrics is 1. KSS ranges between 0 and 1 [76].
Bias, precision and POD metrics enables to understand if the power ramps algorithm has
the tendency to over foreseen (precision, POD and Bias Score > 1) or under foreseen
(precision, POD and Bias Score < 1) the number of power ramp events.
Page 27 of 51
3. Electricity markets time frames and power forecasts
The existing designs of most European electricity markets were defined during a con-
ventional energy technology dominated period. These technologies can respond to the
demand variability, they are easily adjustable and, if requested in due time, they can re-
spond efficiently to operational set-points. However, in addition to the negative environ-
mental impacts of using fossil technologies, the marginal cost to operate these technolo-
gies is high. In contrast, vRES are weather dependent and still present significant forecast
errors, especially for long time horizons.
DAMs require the forecast of electricity production 12-36 hours before physical delivery
in central Europe due to coupled DAM auction at noon, or 13-37 hours in Great Britain
2
,
Ireland and Portugal. This time gap between bidding and the first deliverable can jeopard-
ize the profitability of vRES [77]. The DAM shortcomings and alternative designs for a
near 100% renewable electricity system were addressed in Deliverable 3.5 from
TradeRES project [5]. The authors suggested a reduction of the time gap between the
DAM closure and the delivery time. This reduction could facilitate vRES, since it allows
reducing the uncertainty associated to power forecasts, and many flexibility options. The
authors concluded that the choice for European market design is whether to maintain the
current organization of wholesale electricity trade, in which the 24 hours of each day are
traded together at noon the day before, or to replace it with a different wholesale market
design.”
As shown in Figure 9 and Figure 10, for a time horizon above six hours, NWP-based
forecast is the recommended approach for vRES technologies. Nevertheless, the forecast
performance is worse than the one expected for very short and short time horizons. In
[78], the authors quantify the annual value of using solar power forecast in the Iberian
electricity market. The forecast models that use NWP data showed the highest revenue.
The benefits from using a NWP-based forecast approach, with respect to the persistence
prediction, ranges from 1 to 6 kEUR per MW of PV capacity per year.
From a forecasting point of view, the motivation for the DAM closure change is related
to the availability of the IBC conditions used to feed the NWP models. As mentioned in
section 2, the quality of the forecast strongly relies on these data. For the European coun-
tries, IBC availability from global models is limited to updates every 6 hours (at 00, 06, 12
and 18 UTC). For participate in DAM, currently, the NWP-based power forecast systems
use the IBC from 00 or 06 UTC to obtain the expected vRES production or electricity de-
mand. In order to benefit from updated IBC data, postponing in some hours the DAM clo-
sure gate (Figure 11) could allow to keep its overall structure, while it is expected to re-
duce the forecast uncertainties. The new gate closure hour is associated with the IBC
delivery hour plus an additional period of two hours to perform all required steps (down-
2
For Great Britain, the Day-ahead auction for 60 min products has been moved to 9.20 due to the Brexit,
even enlarging the lead times. In addition, there is a 30 min auction held at 15.30.
Page 28 of 51
load the IBC data, run the numerical mesoscale/regional model and apply the different
forecast approach) to obtain the forecast.
Figure 9. Forecast errors according to time horizon for different wind power forecast approach-
es. HWP approach refers to a physical approach and “HWP/MOS refers to a hybrid forecast ap-
proach. Figure adapted from [33].
Figure 10. Solar power forecast skills according to time horizon and type of forecast approach.
Figure extracted [79].
Page 29 of 51
Figure 11. Identifying the time synergy between the meteorological data availability and DAM
possible designs. D represents the day on which the simulation is carried out (Figure extracted
from [80]).
In this energy transition phase toward a near 100% renewable power system, this
postponing does not require any disruptive change in the market designs and, as de-
scribed in Deliverable 3.5 [5], it can allow a compromise between the need to accommo-
date facilities with ramping constraints, which need longer lead times, and variable renew-
able energy sources, for which a short time between market clearing and delivery reduces
weather uncertainty”.
In the current market design, market participants can already make use of short-term
forecasts with high accuracy on intraday markets (IDM). Compared to the DAM design,
intraday market designs show a larger variability across European countries. There are
some (opening) auctions held on the day before delivery for some countries, such as the
IDM auction for Austria, Belgium, Denmark and Netherlands at 15:00 in which 15 min
products are traded. For the continuous trading, a greater degree of harmonization has
been established from the Single Intraday Coupling (SIDC) (see [81] and also TradeRES
D3.5). Continuous intraday markets allow a trading up until real-time for Finland. For other
markets, lead times are rather short and range from 5 mins for Austria, Belgium, Denmark
and Netherlands to 30 mins for France and Switzerland [81].
Improved forecasting accuracy can lead to smaller asymmetry for balance responsible
parties (BRPs), ultimately reducing their imbalance payments. Thus, there is already a
benefit of increasing generation forecast quality which will be further increased with trad-
ing even closer to lead time, potentially also in DAM markets, as well as rising shares of
vRES.
Page 30 of 51
3.1 Impact on market modelling
Following the approach presented in Deliverable 4.1 [80], the Agent-based Market
model for the Investigation of Renewable and Integrated energy Systems (AMIRIS) was
enhanced to consider power forecast errors for agents marketing renewable energy.
Since error distribution functions for different technologies and varying gate closure lead
times are not yet fully integrated, a Gaussian distribution was chosen to represent the
forecast error. The distribution parameters are exemplary and were selected to illustrate
the impact of forecast errors on the market simulations and to demonstrate the basic func-
tionality developed within TradeRES. The data used does, however, not yet represent
realistic data regarding power forecast errors. Such realistic data potentially also con-
sidering error dependency on a given weather situation will be developed in the project
and integrated into AMIRIS in the near future.
In this demonstration of the approach, the renewable energy trading agent was config-
ured to consider power forecast errors with a constant distribution function over time.
Thus, each hour of the day was assumed to follow the same error distribution. It is as-
sumed that the power forecast error follows a normal distribution formulation. Generated
values represent levels with different relative error. To obtain power supply bids that in-
clude these errors, the error level is multiplied with the perfect foresight power infeed (see
also Section 3.3.1.2 in Deliverable 4.1) which can be extracted from historical time series
data. Figure 12 shows the incidence of actual power forecast error levels obtained within
AMIRIS. The normal distribution of the error levels is clearly visible. The positive mean of
the distribution (12) corresponds to an average overestimation of renewable power. Due
to the variance of the distribution (13), however, also negative error values occur, reflect-
ing a lower-than-actual feed-in estimate.

(12)

(13)
Figure 12. Histogram of power forecast error levels created in AMIRIS following a normal distri-
bution with a mean of 0.05 and a standard deviation of 0.1; 8760 hourly data points representing
one year.
Page 31 of 51
The power forecast errors are created during the bid preparation stage of AMIRIS and
propagate through the simulation. Therefore, these errors can impact the day-ahead mar-
ket clearing price, traders’ profits and system costs as well as other subsequent markets
(e.g., intra-day, ancillary services) which, however, are not explicitly modelled in AMIRIS.
Figure 13 demonstrates the possible impact of power forecast errors on the day-ahead
electricity market clearing price using the same error distribution as before. For most sit-
uations, prices found with erroneous forecasts are below the “perfect foresight” prices that
do not contain any forecast errors. This matches the expectation since an on-average
higher renewable feed-in should lead to lower prices due to the merit-order effect [82].
Figure 13. Sample impact of power forecast errors on (non-realistic) DAM clearing prices; black
curve represents prices without power forecast errors, red dots resemble prices that include modi-
fied renewable feed-in estimates based on the same error distribution function as shown in Figure
12.
This simple example highlights the possible impact of power forecast errors on the
market prices. However, it must be noted that several aspects are not yet satisfyingly re-
flected. For instance, real-world forecast errors might depend on the specific type of tech-
nology and the hour of the day. In addition, the errors shown here have no autocorrela-
tion, while real-world error series often have autocorrelative features. Thus, to obtain a
more realistic time series of errors, correlations should also be considered.
Page 32 of 51
4. Forecast approaches developed in TradeRES
project
In this section, some preliminary results are presented as a drive for the first version of
TradeRES vRES power forecast tools. Specifications of the market players/agents that
will benefit from these forecasts are also presented. Detailed results will be presented in
the deliverables from WP 5.
4.1 Preliminary results
4.1.1. Wind, solar, small hydro and electricity demand forecast
The potential benefit of changing the day-ahead market closure gate to an hour near
the time real operation was analysed for different technologies (wind, solar PV, small hy-
dro) and for electricity demand. This benefit is usually referred as the certainty gain ef-
fect and it represents the potential economic surplus that the market players can obtain
by building their offers in the day-ahead market with a highest level of certainty on power
forecast and, consequently, with low-risk exposure [83].
From a forecasting point of view, the motivation for the DAM change is related to the
availability of the IBC conditions used to feed the NWP models that are crucial for fore-
casting systems for the considered time horizon. The work conducted so far intends to
understand which market players can benefit most from this change in the market design.
On the other hand, and since the forecasts are obtained from a partner who is a forecast
provider (including for some players in the Iberian electricity market), it allows to identify
which technologies/players have more challenges to participate in electricity market and,
therefore, require further developments regarding the forecast approaches during the
TradeRES project.
Method and data
The approach followed is based on the work presented in [4]. Although, in this case,
the approach is extended to other technologies and to electricity demand players. The first
step was the identification of representative days. Representative days are a widely used
statistical approach to detect the most typical daily patterns of a dataset under analysis.
This approach, based on a statistical clustering technique, allows, at the same time, to
group days that exhibit identical patterns. With this procedure, it is possible to feed the
agent-based electricity models (in this case the Multi-Agent TRading in Electricity Markets
- MATREM) and identify the profiles that can jeopardize the income of different players /
market agents enabling the adoption of measures to mitigate their exposure to risk. By
applying the K-medois clustering [4] to the Portuguese aggregated wind, solar PV, and
small-hydro power production, nine typical profiles were identified, Figure 14.
Page 33 of 51
a)
b)
c)
Figure 14. Daily average profiles for a) wind, b) solar PV and c) small hydro for the nine clusters
(profiles) for Portugal.
For the days that correspond to the medoids of each cluster, the forecast using differ-
ent IBC was attained. The forecast is based on a combination of machine learning ap-
proaches (ANN and KNN) that uses an input a combination of weather sources from the
ECMWF and GFS global models. Different models use different weather variables depend
on technology. Usually, the models are trained using the last 2 historical years, and when
data is available, we create a model by site. At the end of the process, an aggregated
model is used to combine all sites for each technology. Models run every 6 hours using
the best weather data available. Some of these models consider the last real values col-
lected from the SCADA to improve the very short-term forecast.
Metrics of market performance
The following metrics are used to compare the results between the base and the up-
graded forecast scenarios. Considering each scenario, the total remuneration of each
player, , from the markets is computed as follows:
 

    
    
(14)
where is the period number,  is the DAM price, , is the up deviation cost,
 is the down deviation cost.   is the parcel that consists in
0
10
20
30
40
50
60
70
80
90
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Capacity factor (%)
Hour
0
10
20
30
40
50
60
70
80
90
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Capacity factor (%)
Hour
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Capacity factor (%)
Hour
0
10
20
30
40
50
60
70
80
90
12345678910 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Capacity factor (%)
Hour
Profile 1
Profile 2
Profile 3
Profile 4
Profile 5
Profile 6
Profile 7
Profile 8
Profile 9
Page 34 of 51
the remuneration obtained from the DAM. The other parcel of the equation represents the
remuneration obtained from the deviations. Therefore, considering that scenario 0 is the
current scenario, the remuneration gain effect, , of rolling the gate closure is computed
as follows:


(15)
The cost,, paid in each scenario due to the penalties (difference between the forecast
and observed values) is computed as follows:
    
    
(16)
Scenarios and synthesis of the main results
To quantify the certainty gain effect for different players with the existing electricity
market design and products, the MATREM simulator was feed step-by-step using the
scenarios presented in Table 5.
Table 5. Scenarios analysed to quantify the certainty gain effect for different electricity
market players.
Scenario
Description
1
Wind forecast and observed values for other generation technologies + load
2
Solar forecast and observed values for other generation technologies + load
3
Small hydro forecast and observed values for other generation technologies + load
4
Load forecast + observed for generation technologies
The lowest forecast errors were observed for small hydro and load players. Preliminary
results show that the certainty gain effect of using updated IBCs is relatively small. Table
6 showns the results for the scenario 1 using the forecast data at 6 UTC as a benchmark.
In the case of a wind power producer, the reduction in the RMSE reaches nearly 1%. The
total energy deviation for the representative days analysed reduces 1.47% and 2.17%
comparing with the use of IBC from global models provided at 06:00 UTC. Regarding the
remuneration, an increase of 5.76% is expected when the IBC from 12:00 UTC is used.
For the IBC from the 18:00, the increase in the remuneration only increases 1.68%. Re-
sults will be further analysed in WP 5, in specific, D5.3- Performance assessment of cur-
rent and new market designs and trading mechanisms for National and Regional Markets.
Page 35 of 51
Table 6. Results for scenario 1 (wind power forecast) in relation to the DAM forecast at
06:00 (UTC).
Metrics
IBC forecast data
Forecast at (UTC)
12:00
18:00
RMSE (%)
-0.77
-1.06
Energy deviation (%)
-1.47
-2.17
Penalties to DAM schedule (%)
-2.53
-5.52
Total remuneration (%)
+5.67
+1.68
4.1.2. Power forecast with feature selection
As discussed in the previous sections, and supported by several authors (e.g., [84]),
the vRES power forecast accuracy for short and medium time horizons strongly relies on
the outcomes of NWP models. The main error in the final forecast comes from the mete-
orological input rather than the existing statistical techniques applied in power forecast
systems [84]. For instance, using one source of meteorology for wind speed forecast the
mean absolute percentage error (MAPE) is approximately 15%. Using this wind speed as
input, MAPE for wind power forecast is approximately 23%. However, using the same
algorithm with another source of meteorology, with wind speed MAPE of 24%, the MAPE
error for power forecast has a substantial increase, around 45%. A generic graphical rep-
resentation of a generation modern wind turbine power curve, which presents typically a
cubic dependency of wind speed for the range between 4-11 m/s. For these wind speed
ranges, we can intuitively understand that an absolute error of 1 m/s in wind speed can
represent a high error value in power forecast. However, the full potential of the output of
these models was not fully explored. For instance, most of vRES power system uses a
single point information from the NWP grid and a limited number of meteorological pa-
rameters.
For the specific case of the single point outputs, the variability in generation observed
in a given wind or solar power plant depends not only on the local dynamics, but it is also
influenced by large-scale atmospheric patterns [73]. Although these patterns may be well
simulated, in a specific location, the time series may present deviations [85]. Thus, anoth-
er feature of the NWP that is not usually explored in forecasting systems is the use of the
results of a spatial grid in contrast to the use of only data from a single spatial point or the
midpoints of the NWP domain surrounding a wind power plant/ solar PV [86]. In [73], a
methodology that combines a gradient boosting trees algorithm with feature engineering
techniques, aiming to extract the maximum spatial and temporal information from the
NWP grid to improve wind and solar power, was implemented. The authors identified that
the use of PCA enables to improve the wind power forecast accuracy. Thus, the achieved
results indicate that an adequate extraction of features from the raw data of the NWP can
improve the forecast systems. The authors recommend more investment in the data min-
ing phase as well as the application of statistical downscaling techniques capable of in-
corporating all data.
Page 36 of 51
Regarding the meteorological parameters extracted from NWP, recent works have
shown that a careful selection of input variables for statistical methods can improve the
accuracy of the wind and solar power forecasts [46], [49]. In the case of wind power, the
most common meteorological parameters used as input to feed the downscaling ap-
proaches are the wind speed and direction. The influence of parameters related to the
conversion efficiency as air temperature and pressure are included in some works. In [85]
the authors included parameters from the upper levels of the atmosphere (e.g., the 850
and 500 hPa pressure levels) as input to improve the performance of the statistical mod-
els. Using a PCA, in the work conducted by [87], the following parameters were identified
as the most relevant to forecast the wind power variability: mean sea level pressure, geo-
potential height, and the meridional wind component and humidity relative. A physics-
oriented pre-processing with a NWP feature selection approach had a positive impact on
the model performance from the team that won the European Energy Market Conference
competition [88]. In [46], the author identified that a possible development trend to im-
prove the power forecast systems is to include exogenous meteorological input variables.
For solar power, [45] identified that parameters as precipitation intensity and wind speed
penalized the performance of the forecast. On the other hand, parameters as ultraviolet
index, wind bearing, and dew point could allow to improve the solar power forecast.
Against this background, in TradeRES, a method was implemented to identify if the in-
clusion of meteorological parameters derived directly from a NWP and others with impact
in wind and solar power generation behaviour. These meteorological parameters include
both, surface (e.g., latent flux) and vertical levels information.
Method and data
The method implemented uses a greedy sequential forward feature (SFF) selection al-
gorithm [49] to select iteratively the meteorological features (based on the principal com-
ponents scores), which minimizes an objective function, in this case, the root mean
square error. The SFF starts with an empty set of features. Then, at each iteration, it ex-
tends the previous set with the feature that allows to minimize a pre-determined objective
function (OF). The RMSE was used as an objective function in this case study.
The ANN approach implemented in Matlab toolbox [89] was used to provide the power
forecast due to the features described in section 2.2.2. Estimating with the ANN involves
two main steps: training and learning. One of the most effective learning algorithms in
ANN is the backpropagation algorithm [90][92]. The backpropagation algorithm uses
supervised learning to adjust the weights and biases in each unit [90]. The process of
training the network is the adjustment of the network weights to produce the desired re-
sponse to the given inputs. Consequently, the training begins with random weights that
iteratively are adjusted so that the error (the difference between estimated and expected
results) based on inputs and outputs data will always be minimized to an acceptable val-
ue. The goal of the backpropagation algorithm is to reduce this error until the ANN learns
the training data [90]. The following parameters were imposed during the ANN setup:
a) the number of hidden layers - In this case, it was considered only one hidden layer;
Page 37 of 51
b) different numbers of units in each layer. There is no predefined rule and therefore
a rule-of-thumb for determining the correct number was followed. In this case, the
number of hidden units is always 2/3 the size of the input layer, plus the size of the
output layer [93];
c) different types of transfer functions. In this case, a sigmoid function was consid-
ered for each unit in the hidden layer;
d) learning algorithm. In this case, the Levenberg-Marquardt algorithm was adopted
[91], [93].
As a baseline to assess the benefit of the SFF selection algorithm, an ANN approach
was feed with the meteorological parameters similar to the ones used in [73] azimuthal
wind speed, meridional wind speed and wind speed module. For each wind park ana-
lysed, the data is obtained for the nearest grid point of NWP. Data from the WRF model
provided by MeteoGalicia
3
are used. These data are available, free of charge, and contain
several meteorological historical forecast parameters such as wind speed components
and gust, convective available potential energy, cloud cover at high levels, surface down-
welling shortwave flux and visibility.
The two following case studies were also analysed: i) use the PCA approach without
applying the feature selection algorithm, and ii) single point data with the feature selection
algorithm.
The method was applied to seven wind parks in Portugal located in regions with differ-
ent climatic conditions (coastal and mountains regions) to understand the consistency of
the meteorological features along the different regions.
Synthesis of the main results
The average impact for the seven locations analysed different configurations of the
wind forecast power approach is depicted in Figure 15.
By applying the PCA and the feature selection algorithm the RMSE can reduce nearly
25% comparing to the benchmarking approach (single-point forecast with limited number
of meteorological parameters), Figure 15. If only PCA is applied, the results show a reduc-
tion in the RMSE values of nearly 8%. Although, the use of PCA improves the forecast, an
insightful selection of the meteorological features is paramount to reduce the uncertainty
in the wind power forecasts. Parameters as wind gust, wind power density, wind shear,
and planetary boundary layer should be used to improve the wind power forecast.
3
Data available at: http://mandeo.meteogalicia.es/thredds/catalogos/DATOS/ARCHIVE/WRF
/WRF_hist.html (accessed on 02 November 2020).
Page 38 of 51
Figure 15. Average RMSE improvements for the seven wind parks analysed compared with the
benchmarking approach. “PCA-WithoutSFF PCA approach without applying SFF algorithm;
“NWPPoint+SFF data from NWP was extracted to the nearest point of each wind park and the
SFF was applied; “PCA+SFF PCA approach and application of the SFF algorithm.
Despite the improvements observed, it is noted a tendency to underestimate the days
with a high level of wind power production. On contrary, an overestimation of the observed
wind power production is observed for days with low level of wind power production.
4.1.3. TradeRES - vRES power forecast approach
Based on the preliminary results, the main steps of the vRES power forecast method
applied in TradeRES are presented in Figure 16.
Figure 16. Power forecast approach implemented in TradeRES project.
0
5
10
15
20
25
PCA-WithoutSFF NWPPoint+SFF PCA+SFF
RMSE improvements regarding
benchmarking (%)
Forecast system configuration
Forward feature selection for each Weather Regime
Meteorological
forecast data PCA Analysis Pearson
correlation
Forecast statistical
approach
Meteo
feature
retained
OF
Improv.?
Ranking PCs
Y
N
PCs
Weather Regime
Classification
Page 39 of 51
The forecasting approach is also based on a SFF algorithm but, in this case, a calibra-
tion procedure according to the different weather regimes is performed. The classification
of atmospheric circulation states into distinct types is a common approach used for under-
standing and scrutinizing weather patterns and their impact on a predetermined parame-
ter. For instance, in [94] the weather regimes were used to estimate Europe-wide wind
power generation. Thus, the goal is to improve the drawbacks identified in the previous
subsection by obtaining a forecast configuration for similar weather conditions. As related
by several authors [84], the weather conditions have a strong impact on the wind power
variability as well as in the uncertainty in its forecast. This can be partly explained by the
weather conditions that unleash different responses, e.g., heating and cooling between
land/sea surfaces, and thermal stratification [47], [95], [96]. The performance of solar
power forecast systems depends on the cloud coverage, which can be distinguished using
weather regimes.
To accomplish the specification requested by some market players, probabilistic out-
puts will be provided with this approach. One of the objectives of this approach is to ena-
ble bidding strategically of market players by identifying the most adequate quantile (e.g.,
the quantile that maximizes the revenue in the day-ahead market) [97].
Below, further details of each step in the methodology implemented are provided:
Meteorological data: The NWP data will be provided from the two weather sources
the ECMWF and GFS global models.
Several meteorological parameters from the NWP will be tested to identify meteoro-
logical features that enable to improve the wind or solar power forecast accuracy.
New variables such as mean seal level gradient or atmospheric instability [47] to ac-
count for the energy conversion processes will be computed.
PCA analysis: After obtaining the data, a PCA analysis is implemented for each me-
teorological parameter. Only the PCs that explain 90% of the total variability are re-
tained since the remaining PCs only describe local effects.
Pearson coefficient correlation: is then applied to identify the correlation among the
PCs and the wind/solar PV power (or the combination of both) and the PCs are
ranking in a descending way. In parallel with this process, a weather regime classifi-
cation is computed.
Weather regime classification: each forecast day will be classified into a specific WR
using a Lamb-type approach [50]. This approach uses the mean sea level pressure
from 16 grid points to identify twenty-six dissimilar weather patterns. Two of the
weather patterns are classified as pure low-pressure system or anticyclonic, eight
are defined as directional according to the wind rose (N, NE, E, SE, S, SW, W and
NW), and the remaining sixteen are defined as hybrid.
FSS for each weather regime: It is a greedy algorithm that chooses the “most attrac-
tive” solution in each iteration. In this case, the SFS attempts to find the “optimal”
feature subset by selecting, iteratively, the meteorological PC that improves reduces
the RMSE value. Sensitivity tests will be implemented for each case study and tech-
nology to identify the most adequate OF. The OF will depend on the perspective: i)
for market players as wind and solar power producers (or vRES aggregator), the
RMSE and electricity market revenue (including day-ahead and imbalances) will be
Page 40 of 51
tested and compared, and ii) for the TSOs, the RMSE will be used. The ANN will be
the statistical approach implemented. A quantile spline regression technique will be
applied to obtain the probabilistic forecasts [98].
The power forecast approach will be tested in the regional cases studies defined in WP
5, and the benefit from this approach will be discussed as an outcome of WP 5.
Output: Wind or solar power probabilistic forecast with 15- and 60-minutes time resolu-
tion, according to the needs of the different players. When deterministic forecast is re-
quired, the quantile that minimizes the RMSE or maximizes the producers’ revenues will
be applied.
Agent-based models that will benefit from these data: Wind and solar power pro-
ducers, aggregators/virtual power plants, and TSO.
4.2 Wind power ramping forecast
As the share of wind and solar PV increases in most of the power systems, ramping
alert tools are being implemented by some TSOs [7], [99]. The goal of such tool is to
complement the existing deterministic or probabilistic forecast systems enabling to in-
crease the level of situational awareness available to the TSO by helping them to better
scale the level of risk that exists in the system. This risk can then be managed by taking
into consideration additional factors, such as potential changes in energy consumption,
additional reserves that can be deployed, and additional generators that may be available
for unit commitment. Furthermore, players capable to provide temporal and sectoral flexi-
bility can also take advantage of this information to strategically participate into electricity
markets.
The characterization and definition of wind power ramps are linked to the notion of an
event that is critical enough to deserve special attention[50]. In specific, ramp events
consist of a rapid and substantial change in the wind power during a time interval .
Since no clear definition is available in the literature to classify power ramps, the definition
(17) and principles used in [9] will be followed in TradeRES project as a “first-guess”.

 
(17)
where, t denotes the time and  is the reference value. For these parameters, the
values identified in [9] will be used.
Understanding power ramps events is not an easy task as the weather conditions are
rarely the same for different wind parks. In fact, even when two wind parks are placed in
similar latitudes, these triggering mechanisms can be very different due to local effects as
the terrain characteristics, roughness and topography or phenomena like sea/land breez-
es [74]. Recent works, e.g., [9] state that, in order to understand and forecast the dynam-
ics of wind power ramps, holistic methodologies should be used to account for the spatial
and temporal evolution of atmospheric large-scale circulation. In this sense, in their work,
the authors implemented a windstorm detection algorithm and compared the performance
Page 41 of 51
with a common cyclone detection algorithm [100], [101]. Windstorm algorithm presented a
highest performance. Nevertheless, some issues were identified in the current windstorm
detection methodologies. The most critical one is that a wind power ramp is not always a
consequence or is always linked to the existence of extreme wind speed values, being
essentially dependent from the previous (historical) state of the flow. Moreover, these al-
gorithms are unable to distinguish upward from downward power ramps. For that reason,
information from the previous time step ("memory effect") needs to be included in this type
of fast ramping tool. In the next section, an algorithm that uses a time numerical differenti-
ation in order to fit the particular case of wind power ramps events is described.
4.2.1. Ramp detection algorithm
This algorithm is based on the forecast mean sea level pressure from the NWP. In or-
der to be able to identify areas where there is the highest variation in the meteorological
field, the pressure gradient was calculated as follows:


 
 

(18)
where p is the average pressure at sea level, Long the longitude and Lat the latitude.
Next, and in order to introduce a “memory effect”, the derivative in time of the pressure
gradient is calculated according to the following expression:

 

(19)
The remaining detection algorithm is equal to the windstorm algorithm presented in [9].
Therefore, the major differences between the two methodologies are the following as-
pects: i) use of pressure data, ensuring better identification of the synoptic centres [9]; ii)
identification of extreme events associated with positive/negative power changes in time,
enabling a better relationship with the wind power ramp events. In this sense, it is consid-
ered that the events with negative variations are those with a change in the pressure gra-
dient of less than the 2nd percentile. On the other hand, the positive events are identified
as regions with a variation above the 98th percentile in the pressure gradient.
In the case of upward ramps, the algorithm starts to determine the grid points where
the pressure gradient is above a certain percentile. The spatial percentile calculation is
based on the following formula [9]:


(20)
where, p represents the percentile considered and stands for the cumulative distribu-
tion function weighted by the cosine of the latitude of {W (Long, Lat, t):(Long, Lat)δ} be-
ing δ the spatial domain [102]. For downward power ramps, the 2nd percentile is consid-
ered, and the search is for grid points where the pressure gradient is below this value.
Then, contiguous grid points for which the percentile condition occurs are enclosed into
the same candidate [9]. A convex hull approximation is employed in this step to identify
the convex polygon comprising all the spatial grid points that can belong to the same me-
Page 42 of 51
teorological event (see black line in Figure 17). After, the average geometric center of
each event is computed (magenta “*” symbols in Figure 17). As outcome, this spatial
search algorithm provides a list of the possible location of events associated with synoptic
systems. Only events with a minimum area of 150 000 km2 [103] are considered. This
step is performed for each temporal time-step.
Figure 17. Example of one event in time t (black line) and one event in time t+1 (green line). The
magenta “*” symbols represent the average geometric center of each candidate, while the magenta
line indicates the trajectory of the meteorological event.
Once the synoptic events are identified in the time step t, it becomes necessary to
stitch to the nearest candidate at the time t+1 to build the trajectory. The following as-
sumptions are imposed in this step [9]:
1. The maximum Euclidean distance between the centers of two consecutive time-
steps is 720 km [103];
2. Only events with a lifetime above 2h or with a maximum speed of 120 km/h are re-
tained.
All events with no continuity are eliminated, and when two or more candidates are
found, a cost function is applied in order to determine the most appropriate trajectory. The
cost function applied is similar to the one shown by [104], which is expressed by:
 



(21)
where, are the coordinates of the center for a determined synoptic event at time step t,
 are the coordinates of the center for the jth synoptic event at time step t+1,  is the
intensity observed at the geometric center of the event at time-step t and  is the
intensity observed of the geometric center for the jth synoptic event at time-step t+1.
Page 43 of 51
At the end, the algorithm retains a tracking table with the different trajectories of the ex-
treme events detected and some basic characteristics, e.g., their lifetime, occurrence
dates, speed, area of influence. In real-time operation, an alert will be issued when these
events are nearby the region under analysis.
The power ramp power forecast accuracy will be assessed and analysed in the region-
al cases from WP 5.
4.2.2. A nested forecast approach
Based on the probabilistic and power ramp detection algorithm, a nested forecast ap-
proach will be established aiming to reduce the system cost associated with less commit-
ted reserves. Thus, the power reserves allocation can be dynamically established and
dependent on the probability of existence (or not) of wind ramps. Other market players as
flexibility providers or wind power producers can also benefit from this nested approach
since it can allow for strategic participation in electricity markets. An example of the out-
comes of this tool is shown in Figure 18.
Figure 18. Example of the outcomes from the nested forecast approach.
Output: Hourly binary information regarding the occurrence (or not) of wind power
ramps.
Agent-based models that will benefit from these data: All TradeRES agent-based
models may benefit from these data by incorporating it in TSO agent capabilities or in flex-
ibility provider and wind power producers’ agent behaviour, using this information to stra-
tegically participate in the electricity markets.
Page 44 of 51
5. Final remarks
In this report, a non-extensive review of the different forecasting methods was present-
ed focusing on the technologies under analysis in the TradeRES project. This review
served as the basis for framing the advantages and disadvantages of the different fore-
cast approaches commonly applied in the energy sector. Moreover, it allowed to identify
the synergies among the different approaches and existing electricity market time-frames.
Based on this background and preliminary results from TradeRES, a nested forecasting
approach was presented to feed specific market players.
Preliminary results were presented highlighting that current power forecast approaches
already allow to have a significant level of accuracy in forecasting electricity demand and
in small hydro power plants. However, for the time-frame on which this report is focused
(day-ahead market) wind and solar PV technologies (or vRES aggregator) continue to
show significant errors, even assuming a postponing of their bids to an hour closer to real-
time. To improve the existing forecast systems, a new approach is proposed in
TradeRES. Results showed that the use of numerical weather prediction (NWP) grid data
coupled with a feature selection algorithm could enable to improve the forecast perfor-
mance, when compared with a forecast based only a NWP single-point data. Preliminary
results also show that meteorological parameters as wind gust (traditionally not consid-
ered in the wind power forecast systems) enable to reduce the wind power forecast errors.
In this deliverable, some values and principles were defined as “first-guess” based on a
literature review. Due to the iterative nature of the project, it is expected that some values
can be further improved in the second version of this deliverable at Month 41 based on
the outcomes of the remaining work packages. It should be noted that the focus of this
deliverable was the day-ahead market, which is one of the most important in the current
electricity markets designs and detailed results will be presented in work package 5. In the
second version of deliverable 4.9, the work is expected to focus on the very-short and
short time horizons - in line with the new market electricity designs/products that are being
proposed in work package 3 of the project.
Page 45 of 51
References
[1] S. Hanifi, X. Liu, Z. Lin, and S. Lotfian, “A Critical Review of Wind Power Forecasting
Methods—Past, Present and Future,” Energies, vol. 13, no. 15, p. 3764, Jul. 2020, doi:
10.3390/en13153764.
[2] C. Croonenbroeck and G. Stadtmann, “Renewable generation forecast studies – Review
and good practice guidance,” Renew. Sustain. Energy Rev., vol. 108, no. June 2018, pp.
312322, Jul. 2019, doi: 10.1016/j.rser.2019.03.029.
[3] H. Holttinen et al., “Methodologies to determine operating reserves due to increased wind
power,” IEEE Trans. Sustain. Energy, vol. 3, no. 4, pp. 713723, Oct. 2012, doi:
10.1109/TSTE.2012.2208207.
[4] H. Algarvio, A. Couto, F. Lopes, and A. Estanqueiro, “Changing the Day-Ahead Gate
Closure to Wind Power Integration: A Simulation-Based Study,” Energies, vol. 12, no. 14, p.
2765, Jul. 2019, doi: 10.3390/en12142765.
[5] L. de Vries et al., “D3.5 - Market design for a reliable ~ 100 % renewable electricity system,”
TradeRES project deliverable. p. 62, 2021 [Online]. Available: https://traderes.eu/wp-
content/uploads/2021/04/D3.5_MarketDesignOptions_H2020.pdf
[6] C. Köhler et al., “Critical weather situations for renewable energies – Part B: Low stratus risk
for solar power,” Renew. Energy, vol. 101, pp. 794803, Feb. 2017, doi:
10.1016/j.renene.2016.09.002.
[7] A. Steiner et al., “Critical weather situations for renewable energies – Part A: Cyclone
detection for wind power,” Renew. Energy, vol. 101, pp. 4150, Feb. 2017, doi:
10.1016/j.renene.2016.08.013.
[8] C. Gallego-Castillo, E. Garcia-Bustamante, A. Cuerva, and J. Navarro, “Identifying wind
power ramp causes from multivariate datasets: a methodological proposal and its
application to reanalysis data,” IET Renew. Power Gener., vol. 9, no. 8, pp. 867875, 2015,
doi: 10.1049/iet-rpg.2014.0457.
[9] M. Lacerda, A. Couto, and A. Estanqueiro, “Wind Power Ramps Driven by Windstorms and
Cyclones,” Energies, vol. 10, no. 10, p. 1475, Sep. 2017, doi: 10.3390/en10101475.
[10] M. S. H. Lipu et al., Artificial Intelligence Based Hybrid Forecasting Approaches for Wind
Power Generation: Progress, Challenges and Prospects,” IEEE Access, vol. 9, pp. 102460
102489, 2021, doi: 10.1109/ACCESS.2021.3097102.
[11] A. Al Mamun, M. Sohel, N. Mohammad, M. S. Haque Sunny, D. R. Dipta, and E. Hossain,
“A Comprehensive Review of the Load Forecasting Techniques Using Single and Hybrid
Predictive Models,” IEEE Access, vol. 8, pp. 134911134939, 2020, doi:
10.1109/ACCESS.2020.3010702.
[12] I. K. Nti, M. Teimeh, O. Nyarko-Boateng, and A. F. Adekoya, “Electricity load forecasting: a
systematic review,” J. Electr. Syst. Inf. Technol., vol. 7, no. 1, p. 13, Dec. 2020, doi:
10.1186/s43067-020-00021-8.
[13] D. C. Montgomery, C. L. Jennings, and M. Kulahci, Introduction to Time Series Analysis and
Forecasting, 2nd ed. New Jersey: JohnWiley & Sons, Inc., 2015.
[14] C. Chatfield, Time-Series Forecasting. Florida: Chapman and Hall/CRC; 6th edition (July
31, 2003), 2000.
[15] C. Kuster, Y. Rezgui, and M. Mourshed, “Electrical load forecasting models: A critical
systematic review,” Sustain. Cities Soc., vol. 35, no. August, pp. 257270, 2017, doi:
10.1016/j.scs.2017.08.009.
[16] Y. H. Hsiao, “Household Electricity Demand Forecast Based on Context Information and
User Daily Schedule Analysis From Meter Data,” IEEE Trans. Ind. Informatics, vol. 11, no.
1, pp. 3343, 2015, doi: 10.1109/TII.2014.2363584.
[17] B. Yang et al., “State-of-the-art one-stop handbook on wind forecasting technologies: An
overview of classifications, methodologies, and analysis,” J. Clean. Prod., vol. 283, p.
Page 46 of 51
124628, 2021, doi: 10.1016/j.jclepro.2020.124628.
[18] J. Widén, N. Carpman, V. Castellucci, D. Lingfors, J. Olauson, and et al., “Variability
Assessment and Forecasting of Renewables: A Review for Solar, Wind, Wave and Tidal
Resources,” Renew. Sustain. energy Rev., vol. 44, pp. 356375, 2015.
[19] C. Monteiro, R. Bessa, V. Miranda, A. Botterud, J. Wang, and G. Conzelmann, “Wind Power
Forecasting: State-of-the-art 2009,” 2009 [Online]. Available:
https://publications.anl.gov/anlpubs/2009/11/65613.pdf
[20] T. Haiden et al., “Evaluation of ECMWF forecast, including 2021 upgrade,” ECMWF
technical memo 884. p. 56, 2021 [Online]. Available:
https://www.ecmwf.int/sites/default/files/elibrary/2021/20142-evaluation-ecmwf-forecsts-
including-2021-upgrade.pdf
[21] C. Sweeney, R. J. Bessa, J. Browell, and P. Pinson, “The future of forecasting for
renewable energy,” WIREs Energy Environ., vol. 9, no. 2, Mar. 2020, doi:
https://doi.org/10.1002/wene.365.
[22] Portal do Clima, “Portal do Clima - Climate Change in Portugal,” Programa ADAPT
Alterações Climáticas em Portugal, 2021. [Online]. Available:
http://portaldoclima.pt/en/project/methodology/climate-models/. [Accessed: 01-Sep-2021]
[23] A. Couto, L. Rodrigues, P. Costa, J. Silva, and A. Estanqueiro, “Wind power participation in
electricity markets - The role of wind power forecasts,” in EEEIC 2016 - International
Conference on Environment and Electrical Engineering, 2016, p. 6, doi:
10.1109/EEEIC.2016.7555453.
[24] R. Marujo, P. Costa, M. Fernandes, T. Simoes, and A. Estanqueiro, “Offshore wind field:
Application of statistical models as a spatial validation technique,” in European Offshore
Wind Conference & Exhibition, 2011, p. 8 [Online]. Available:
https://repositorio.lneg.pt/bitstream/10400.9/1501/1/EWEA_Amsterdam_RMarujo_
PCosta_MFernandes_AEstanqueiro.pdf
[25] J. Silva, F. Marques da Silva, A. Couto, and A. Estanqueiro, “A method to correct the flow
distortion of offshore wind data using CFD simulation and experimental wind tunnel tests,”
J. Wind Eng. Ind. Aerodyn., vol. 140, 2015, doi: 10.1016/j.jweia.2015.01.017.
[26] J. M. L. M. Palma, F. A. Castro, L. F. Ribeiro, A. H. Rodrigues, and A. P. Pinto, “Linear and
nonlinear models in wind resource assessment and wind turbine micro-siting in complex
terrain,” J. Wind Eng. Ind. Aerodyn., vol. 96, no. 12, pp. 23082326, 2008, doi:
10.1016/j.jweia.2008.03.012.
[27] C. G. Nunalee, C. Meißner, A. Vignaroli, N. Carolina, and R. Nc, “Downscaling MERRA
mesoscale data for the generation microscale wind fields using CFD,” in Spain: Europe’s
Premier Wind Energy Event, 2014, p. 1.
[28] C. Schmitt, C. Meissner, and D. Weir, “Influence Of Thermal Stratification on CFD
Simulations,” Europe’s Premier Wind Energy Event poster. EWEA, Brussels, p. 1, 2011
[Online]. Available: https://windsim.com/Windsimarchive/library -- papers-
presentations/Influence Of Thermal Stratification on CFD Simulations.pdf
[29] F. Castellani, D. Astolfi, M. Mana, M. Burlando, C. Meißner, and E. Piccioni, “Wind Power
Forecasting techniques in complex terrain: ANN vs. ANN-CFD hybrid approach,” J. Phys.
Conf. Ser., vol. 753, no. 8, p. 082002, Sep. 2016, doi: 10.1088/1742-6596/753/8/082002.
[Online]. Available:
https://repositorio.lneg.pt/bitstream/10400.9/1501/1/EWEA_Amsterdam_RMarujo_
PCosta_MFernandes_AEstanqueiro.pdf
[30] M. Lydia, S. S. Kumar, A. I. Selvakumar, and G. E. Prem Kumar, “A comprehensive review
on wind turbine power curve modeling techniques,” Renew. Sustain. Energy Rev., vol. 30,
pp. 452460, Feb. 2014, doi: 10.1016/j.rser.2013.10.030.
[31] E. Yun and J. Hur, “Probabilistic estimation model of power curve to enhance power output
forecasting of wind generating resources,” Energy, vol. 223, p. 120000, May 2021, doi:
10.1016/j.energy.2021.120000.
Page 47 of 51
[32] M. Yang, C. Shi, and H. Liu, “Day-ahead wind power forecasting based on the clustering of
equivalent power curves,” Energy, vol. 218, p. 119515, Mar. 2021, doi:
10.1016/j.energy.2020.119515.
[33] R. Blaga, A. Sabadus, N. Stefu, C. Dughir, M. Paulescu, and V. Badescu, “A current
perspective on the accuracy of incoming solar energy forecasting,” Prog. Energy Combust.
Sci., vol. 70, pp. 119144, 2019, doi: 10.1016/j.pecs.2018.10.003.
[34] R. A. Rajagukguk and R. Kamil, “A Deep Learning Model to Forecast Solar Irradiance Using
a Sky Camera,” Appl. Sci., vol. 11, no. 5049, p. 15, 2021, doi:
doi.org/10.3390/app11115049.
[35] S. Dev, F. M. Savoy, Y. H. Lee, and S. Winkler, “Estimating solar irradiance using sky
imagers,” Atmos. meas. Tech, vol. 12, pp. 54175429, 2019, doi:
https://doi.org/10.5194/amt-12-5417-2019.
[36] J. Antonanzas, N. Osorio, R. Escobar, R. Urraca, F. J. Martinez-de-pison, and F.
Antonanzas-torres, “Review of photovoltaic power forecasting,” Sol. Energy, vol. 136, pp.
78111, 2016, doi: 10.1016/j.solener.2016.06.069.
[37] M. N. Akhter, S. Mekhilef, H. Mokhlis, and N. Mohamed Shah, “Review on forecasting of
photovoltaic power generation based on machine learning and metaheuristic techniques,”
IET Renew. Power Gener., vol. 13, no. 7, pp. 10091023, May 2019, doi: 10.1049/iet-
rpg.2018.5649.
[38] S. Atique, S. Noureen, V. Roy, V. Subburaj, S. Bayne, and J. Macfie, “Forecasting of total
daily solar energy generation using ARIMA: A case study,” in 2019 IEEE 9th Annual
Computing and Communication Workshop and Conference (CCWC), 2019, pp. 01140119,
doi: 10.1109/CCWC.2019.8666481.
[39] A. M. Foley, P. G. Leahy, A. Marvuglia, and E. J. McKeogh, “Current methods and
advances in forecasting of wind power generation,” Renew. Energy, vol. 37, no. 1, pp. 18,
Jan. 2012, doi: 10.1016/j.renene.2011.05.033.
[40] H. Liu, C. Chen, X. Lv, X. Wu, and M. Liu, “Deterministic wind energy forecasting: A review
of intelligent predictors and auxiliary methods,” Energy Convers. Manag., vol. 195, no. May,
pp. 328345, 2019, doi: 10.1016/j.enconman.2019.05.020.
[41] E. Vinagre, T. Pinto, S. Ramos, Z. Vale, and J. M. Corchado, “Electrical Energy
Consumption Forecast Using Support Vector Machines,” in Proceedings - International
Workshop on Database and Expert Systems Applications, DEXA, 2017, pp. 171175, doi:
10.1109/DEXA.2016.046.
[42] G. Dudek, “Neural networks for pattern-based short-term load forecasting: A comparative
study,” Neurocomputing, vol. 205, pp. 6474, 2016, doi: 10.1016/j.neucom.2016.04.021.
[43] M. Q. Raza and A. Khosravi, “A review on artificial intelligence based load demand
forecasting techniques for smart grid and buildings,” Renew. Sustain. Energy Rev., vol. 50,
pp. 13521372, 2015, doi: 10.1016/j.rser.2015.04.065.
[44] K. Amasyali and N. M. El-gohary, “A review of data-driven building energy consumption
prediction studies,” Renew. Sustain. Energy Rev., vol. 81, no. 2018, pp. 11921205, 2018,
doi: 10.1016/j.rser.2017.04.095.
[45] M. Castangia, A. Aliberti, L. Bottaccioli, E. Macii, and E. Patti, “A compound of feature
selection techniques to improve solar radiation forecasting,” Expert Syst. Appl., vol. 178, no.
March, p. 114979, Sep. 2021, doi: 10.1016/j.eswa.2021.114979.
[46] H. Liu and C. Chen, “Data processing strategies in wind energy forecasting models and
applications: A comprehensive review,” Appl. Energy, vol. 249, no. May, pp. 392408, 2019,
doi: 10.1016/j.apenergy.2019.04.188.
[47] A. Couto, P. Costa, L. Rodrigues, V. V. Lopes, and A. Estanqueiro, “Impact of Weather
Regimes on the Wind Power Ramp Forecast in Portugal,” IEEE Trans. Sustain. Energy, vol.
6, no. 3, pp. 934942, 2015, doi: 10.1109/TSTE.2014.2334062.
[48] G. Chandrashekar and F. Sahin, “A survey on feature selection methods,” Comput. Electr.
Eng., vol. 40, no. 1, pp. 1628, 2014, doi: 10.1016/j.compeleceng.2013.11.024.
Page 48 of 51
[49] S. Salcedo-Sanz, L. Cornejo-Bueno, L. Prieto, D. Paredes, and R. García-Herrera, “Feature
selection in machine learning prediction systems for renewable energy applications,”
Renew. Sustain. Energy Rev., vol. 90, pp. 728741, Jul. 2018, doi:
10.1016/j.rser.2018.04.008.
[50] A. Couto, P. Costa, and T. Simões, “Identification of Extreme Wind Events Using a Weather
Type Classification,” Energies, vol. 14, no. 3944, p. 16, Jul. 2021, doi:
10.3390/en14133944.
[51] H. C. Bloomfield, D. J. Brayshaw, and A. J. Charlton-Perez, “Characterizing the winter
meteorological drivers of the European electricity system using targeted circulation types,”
Meteorol. Appl., no. November 2019, pp. 118, 2019, doi: 10.1002/met.1858.
[52] A. Couto and A. Estanqueiro, “Assessment of wind and solar PV local complementarity for
the hybridization of the wind power plants installed in Portugal,” J. Clean. Prod., vol. 319,
no. August, p. 128728, Oct. 2021, doi: 10.1016/j.jclepro.2021.128728.
[53] F. Cassola and M. Burlando, “Wind speed and wind energy forecast through Kalman
filtering of Numerical Weather Prediction model output,” Appl. Energy, vol. 99, pp. 154166,
2012, doi: 10.1016/j.apenergy.2012.03.054.
[54] I. González-Aparicio and A. Zucker, “Impact of wind power uncertainty forecasting on the
market integration of wind energy in Spain,” Appl. Energy, vol. 159, pp. 334349, Dec.
2015, doi: 10.1016/j.apenergy.2015.08.104.
[55] P. Pinson, N. Siebert, and G. Kariniotakis, “Forecasting of regional wind generation by a
dynamic fuzzy-neural networks based upscaling approach,” Eur. Wind Energy Conf. Exhib.
EWEC, 2003 [Online]. Available: http://hal.archives-ouvertes.fr/hal-00530550/
[56] Z. Liu, W. Gao, and N. Carolina, “Wind Power Plant Prediction by Using Neural Networks,”
in IEEE Energy Conversion Conference and Exposition, 2012, p. 7, doi:
10.1109/ECCE.2012.6342351.
[57] I. Okumus and A. Dinler, “Current status of wind energy forecasting and a hybrid method for
hourly predictions, Energy Convers. Manag., vol. 123, pp. 362371, 2016, doi:
10.1016/j.enconman.2016.06.053.
[58] S. Alessandrini, F. Davò, S. Sperati, M. Benini, and L. Delle Monache, “Comparison of the
economic impact of different wind power forecast systems for producers,” Adv. Sci. Res.,
vol. 11, pp. 4953, May 2014, doi: 10.5194/asr-11-49-2014.
[59] J. Zhang, a Florita, B. Hodge, and J. Freedman, “Ramp Forecasting Performance from
Improved Short-Term Wind Power Forecasting,” ASME 2014 Int. Des. Eng. Conf. Comput.
Inf. Eng. Conf., no. May, 2014, doi: 10.1115/DETC2014-34775.
[60] C. Junk, L. Delle Monache, S. Alessandrini, G. Cervone, and L. von Bremen, “Predictor-
weighting strategies for probabilistic wind power forecasting with an analog ensemble,”
Meteorol. Zeitschrift, vol. 24, no. 4, pp. 361379, 2015, doi: 10.1127/metz/2015/0659.
[61] T. Jónsson, P. Pinson, H. Madsen, and H. Nielsen, “Predictive Densities for Day-Ahead
Electricity Prices Using Time-Adaptive Quantile Regression,” Energies, vol. 7, no. 9, pp.
55235547, 2014, doi: 10.3390/en7095523.
[62] M. Sun, C. Feng, and J. Zhang, “Probabilistic solar power forecasting based on weather
scenario generation,” Appl. Energy, vol. 266, p. 114823, May 2020, doi:
10.1016/j.apenergy.2020.114823.
[63] B. Li and J. Zhang, “A review on the integration of probabilistic solar forecasting in power
systems,” Sol. Energy, vol. 210, pp. 6886, Nov. 2020, doi: 10.1016/j.solener.2020.07.066.
[64] Z. Zhou et al., “Application of probabilistic wind power forecasting in electricity markets,”
Wind Energy, vol. 16, pp. 321338, 2013, doi: 10.1002/we.
[65] H. Holttinen, J. J. Miettinen, A. Couto, H. Algarvio, L. Rodrigues, and A. Estanqueiro, “Wind
power producers in shorter gate closure markets and balancing markets,” in 3th
International Conference on the European Energy Market (EEM), 2016, p. 5, doi:
10.1109/EEM.2016.7521309.
[66] R. J. Bessa, C. Moreira, B. Silva, and M. Matos, “Handling renewable energy variability and
Page 49 of 51
uncertainty in power systems operation,” Wiley Interdiscip. Rev. Energy Environ., vol. 3, no.
2, pp. 156178, May 2014, doi: 10.1002/wene.76.
[67] R. Bessa et al., “Towards Improved Understanding of the Applicability of Uncertainty
Forecasts in the Electric Power Industry,” Energies, vol. 10, no. 9, pp. 14021450, 2017,
doi: 10.3390/en10091402.
[68] N. J. Cutler, H. R. Outhred, I. F. MacGill, and J. D. Kepert, “Predicting and presenting
plausible future scenarios of wind power production from numerical weather prediction
systems: A qualitative ex ante evaluation for decision making,” Wind Energy, vol. 15, no. 3,
pp. 473488, 2012, doi: 10.1002/we.485.
[69] R. J. Bessa, V. Miranda, A. Botterud, J. Wang, and E. M. Constantinescu, “Time adaptive
conditional kernel density estimation for wind power forecasting,” IEEE Trans. Sustain.
Energy, vol. 3, no. 4, pp. 660669, 2012, doi: 10.1109/TSTE.2012.2200302.
[70] P. Li, X. Guan, and J. Wu, “Aggregated wind power generation probabilistic forecasting
based on particle filter,” Energy Convers. Manag., vol. 96, pp. 579587, 2015, doi:
10.1016/j.enconman.2015.03.021.
[71] L. Delle Monache, F. A. Eckel, D. L. Rife, B. Nagarajan, and K. Searight, “Probabilistic
Weather Prediction with an Analog Ensemble,” Mon. Weather Rev., vol. 141, no. 10, pp.
34983516, Oct. 2013, doi: 10.1175/MWR-D-12-00281.1.
[72] F. Davò, S. Alessandrini, S. Sperati, L. Delle Monache, D. Airoldi, and M. T. Vespucci,
“Post-processing techniques and principal component analysis for regional wind power and
solar irradiance forecasting,” Sol. Energy, vol. 134, pp. 327338, 2016, doi:
10.1016/j.solener.2016.04.049.
[73] J. R. Andrade and R. J. Bessa, “Improving Renewable Energy Forecasting With a Grid of
Numerical Weather Predictions,” IEEE Trans. Sustain. Energy, vol. 8, no. 4, pp. 15711580,
Oct. 2017, doi: 10.1109/TSTE.2017.2694340.
[74] C. Gallego-castillo, A. Cuerva-tejero, and O. Lopez-garcia, “A review on the recent history
of wind power ramp forecasting,” Renew. Sustain. Energy Rev., vol. 52, pp. 11481157,
2015, doi: 10.1016/j.rser.2015.07.154.
[75] T. Ouyang, X. Zha, and L. Qin, “A Survey of Wind Power Ramp Forecasting,” Energy Power
Eng., vol. 05, no. 04, pp. 368372, 2013, doi: 10.4236/epe.2013.54B071.
[76] N. Tartaglione, “Relationship between Precipitation Forecast Errors and Skill Scores of
Dichotomous Forecasts,” Weather Forecast., vol. 25, no. 1, pp. 355365, Feb. 2010, doi:
10.1175/2009WAF2222211.1.
[77] A. Estanqueiro and A. Couto, “Chapter 1 - New electricity markets. The challenges of
variable renewable energy,” in Local Electricity Markets, T. Pinto, Z. Vale, and S.
Widergren, Eds. Academic Press, 2021, pp. 320.
[78] B. Bochenek et al., “Day-Ahead Wind Power Forecasting in Poland Based on Numerical
Weather Prediction,” Energies, vol. 14, no. 8, p. 2164, Apr. 2021, doi: 10.3390/en14082164.
[79] SOLARGIS, “How Solargis is improving accuracy of solar power forecasts,” Best practises,
2017. [Online]. Available: https://solargis.com/blog/best-practices/improving-accuracy-of-
solar-power-forecasts
[80] C. Schimeczek et al., “D4.1 - Temporal flexibility options in electricity market simulation
models,” TradeRES project deliverable. p. 58, 2021 [Online]. Available:
https://traderes.eu/wp-
content/uploads/2021/10/D4.1_TemporalFlexibilityOptions_H2020.pdf
[81] EPEX Spot, “Trading at EPEX SPOT 2021,” Technical brochure. p. 8, 2021 [Online].
Available: https://www.epexspot.com/sites/default/files/download_center_files/21-03-
09_Trading Brochure.pdf
[82] F. Sensfuß, M. Ragwitz, and M. Genoese, “The merit-order effect: A detailed analysis of the
price effect of renewable electricity generation on spot market prices in Germany,” Energy
Policy, vol. 36, no. 8, pp. 30863094, Aug. 2008, doi: 10.1016/j.enpol.2008.03.035.
[83] J. M. Morales, A. J. Conejo, and J. Perez-Ruiz, “Short-Term Trading for a Wind Power
Page 50 of 51
Producer,” IEEE Trans. Power Syst., vol. 25, no. 1, pp. 554564, Feb. 2010, doi:
10.1109/TPWRS.2009.2036810.
[84] G. Giebel, R. Brownsword, G. Kariniotakis, M. Denhard, and C. Draxl, “The State-Of-The-
Art in Short-Term Prediction of Wind Power A Literature Overview,” in Technical Report
ANEMOS.plus, 2011, p. 109 [Online]. Available:
http://ecolo.org/documents/documents_in_english/wind-predict-ANEMOS.pdf
[85] N. Ellis, R. Davy, and A. Troccoli, “Predicting wind power variability events using different
statistical methods driven by regional atmospheric model output,” Wind Energy, vol. 18, no.
9, pp. 16111628, Sep. 2015, doi: 10.1002/we.1779.
[86] T. T. Warner, “Quality assurance in atmospheric modeling,” Am. Meteorol. Soc., vol. 92, no.
12, pp. 16011610, 2011, doi: 10.1175/BAMS-D-10-00054.1.
[87] N. Ellis, R. Davy, and A. Troccoli, “Predicting wind power variability events using different
statistical methods driven by regional atmospheric model output,” Wind Energy, vol. 18, no.
9, pp. 16111628, Sep. 2015, doi: 10.1002/we.1779.
[88] K. Bellinguer, V. Mahler, S. Camal, and G. Kariniotakis, “Probabilistic Forecasting of
Regional Wind Power Generation for the EEM20 Competition: a Physics-oriented Machine
Learning Approach,” in 2020 17th International Conference on the European Energy Market
(EEM), 2020, p. 6, doi: 10.1109/EEM49802.2020.9221960.
[89] “MATLAB and Neural Network Toolbox Release 2012b, The MathWorks, Inc., Natick,
Massachusetts, United States.” .
[90] J. P. S. Catalão, H. M. I. Pousinho, and V. M. F. Mendes, “Short-term wind power
forecasting in Portugal by neural networks and wavelet transform,” Renew. Energy, vol. 36,
no. 4, pp. 12451251, Apr. 2011, doi: 10.1016/j.renene.2010.09.016.
[91] G. Li and J. Shi, “On comparing three artificial neural networks for wind speed forecasting,”
Appl. Energy, vol. 87, no. 7, pp. 23132320, Jul. 2010, doi:
10.1016/j.apenergy.2009.12.013.
[92] M. G. V. Marroquín, M. C. A. Cervantes, J. L. M. Flores, and M. Cabrera-Ríos, “Time series:
Empirical characterization and artificial neural network-based selection of forecasting
techniques,” Intell. Data Anal., vol. 13, no. 6, pp. 969982, 2009, doi: 10.3233/IDA-2009-
0403.
[93] M. T. Hagan, H. B. Demuth, M. H. Beale, and O. De Jesús, Neural Network Design (2nd
Edition). Oklahoma State University (ebook), 2014.
[94] B. U. Schyska, A. Couto, L. von Bremen, A. Estanqueiro, and D. Heinemann, “Weather
dependent estimation of continent-wide wind power generation based on spatio-temporal
clustering,” Adv. Sci. Res., vol. 14, no. 2003, pp. 131138, May 2017, doi: 10.5194/asr-14-
131-2017.
[95] C. Draxl, “On the Predictability of Hub Height Winds,” PhdThesis, DTU Wind Energy (Risø-
PhDM No 84(EN)), p. 105, 2012 [Online]. Available:
https://backend.orbit.dtu.dk/ws/portalfiles/portal/10246050/Ris_PhD_84.pdf
[96] M. Ohba, S. Kadokura, and D. Nohara, “Impacts of synoptic circulation patterns on wind
power ramp events in East Japan,” Renew. Energy, vol. 96, pp. 591602, 2016, doi:
10.1016/j.renene.2016.05.032.
[97] G. Strbac et al., “Decarbonization of Electricity Systems in Europe: Market Design
Challenges,” IEEE Power Energy Mag., vol. 19, no. 1, pp. 5363, Jan. 2021, doi:
10.1109/MPE.2020.3033397.
[98] H. A. Nielsen, H. Madsen, and T. S. Nielsen, “Using quantile regression to extend an
existing wind power forecasting system with probabilistic forecasts,” Wind Energy, vol. 9,
pp. 95108, Jan. 2006, doi: 10.1002/we.180.
[99] K. D. Orwig et al., “Recent Trends in Variable Generation Forecasting and Its Value to the
Power System,” IEEE Trans. Sustain. Energy, vol. 6, no. 3, pp. 924933, Jul. 2015, doi:
10.1109/TSTE.2014.2366118.
[100] R. J. Murray and I. Simmonds, “A numerical scheme for tracking cyclone centres from
Page 51 of 51
digital data. Part I: development and operation of the scheme,” Aust. Met. Mag., vol. 39, pp.
155166, 1991 [Online]. Available: http://www.bom.gov.au/jshess/docs/1991/murray1.pdf
[101] R. J. Murray and I. Simmonds, “A numerical scheme for tracking cyclone centres from
digital data. Part II:Application to January and July general circulation model simulations.,”
Aust. Met. Mag., vol. 39, pp. 167180, 1991 [Online]. Available:
http://www.bom.gov.au/jshess/docs/1991/murray2.pdf
[102] P. M. Della-Marta, H. Mathis, C. Frei, M. A. Liniger, J. Kleinn, and C. Appenzeller, “The
return period of wind storms over Europe,” Int. J. Climatol., vol. 29, no. 3, pp. 437459, Mar.
2009, doi: 10.1002/joc.1794.
[103] D. Renggli, “Seasonal predictability of wintertime windstorm climate over the North Atlantic
and Europe,” Dissertation Freie Universität Berlin. p. 151, 2011 [Online]. Available:
https://refubium.fu-berlin.de/handle/fub188/901
[104] E. Flaounas, V. Kotroni, K. Lagouvardos, and I. Flaounas, “Tracking winter extra-tropical
cyclones based on their relative vorticity evolution and sensitivity to prior data filtering
(cycloTRACK v1.0),” Geosci. Model Dev. Discuss., vol. 7, no. 1, pp. 12451276, 2014, doi:
10.5194/gmdd-7-1245-2014.
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Globally, wind energy is growing rapidly and has received huge consideration to fulfill global energy requirements. An accurate wind power forecasting is crucial to achieve a stable and reliable operation of the power grid. However, the unpredictability and stochastic characteristics of wind power affect the grid planning and operation adversely. To address these concerns, a substantial amount of research has been carried out to introduce an efficient wind power forecasting approach. Artificial Intelligence (AI) approaches have demonstrated high precision, better generalization performance and improved learning capability, thus can be ideal to handle unstable, inflexible and intermittent wind power. Recently, AI-based hybrid approaches have become popular due to their high precision, strong adaptability and improved performance. Thus, the goal of this review paper is to present the recent progress of AI-enabled hybrid approaches for wind power forecasting emphasizing classification, structure, strength, weakness and performance analysis. Moreover, this review explores the various influential factors toward the implementations of AI-based hybrid wind power forecasting including data preprocessing, feature selection, hyperparameters adjustment, training algorithm, activation functions and evaluation process. Besides, various key issues, challenges and difficulties are discussed to identify the existing limitations and research gaps. Finally, the review delivers a few selective future proposals that would be valuable to the industrialists and researchers to develop an advanced AI-based hybrid approach for accurate wind power forecasting toward sustainable grid operation.
Article
Full-text available
The identification of extreme wind events and their driving forces are crucial to better integrating wind generation into the power system. Recent work related the occurrence of extreme wind events with some weather circulation patterns, enabling the identification of (i) wind power ramps and (ii) low-generation events as well as their intrinsic features, such as the intensity and time duration. Using Portugal as a case study, this work focuses on the application of a weather classification-type methodology to link the weather conditions with wind power generation, namely, the different types of extreme events. A long-term period is used to assess and characterize the changes in the occurrence of extreme weather events and corresponding intensity on wind power production. High variability is expected under cyclonic regimes, whereas low-generation events are most common in anticyclonic regimes. The results of the work provide significant insights regarding wind power production in Portugal, enabling an increase in its predictability.
Article
Full-text available
The role of renewable energy sources in the Polish power system is growing. The highest share of installed capacity goes to wind and solar energy. Both sources are characterized by high variability of their power output and very low dispatchability. Taking into account the nature of the power system, it is, therefore, imperative to predict their future energy generation to economically schedule the use of conventional generators. Considering the above, this paper examines the possibility to predict day-ahead wind power based on different machine learning methods not for a specific wind farm but at national level. A numerical weather prediction model used operationally in the Institute of Meteorology and Water Management–National Research Institute in Poland and hourly data of recorded wind power generation in Poland were used for forecasting models creation and testing. With the best method, the Extreme Gradient Boosting, and two years of training (2018–2019), the day-ahead, hourly wind power generation in Poland in 2020 was predicted with 26.7% mean absolute percentage error and 4.5% root mean square error accuracy. Seasonal and daily differences in predicted error were found, showing high mean absolute percentage error in summer and during daytime.
Article
This report covers the implementation of temporal flexibility options in TradeRES’ agent-based electricity market simulations models. Within this project, the term “temporal flexibility option” was defined as an asset or measure supporting the power system to balance electric demand and supply and compensate for their stochastic fluctuations stemming from, e.g., weather or consumer behaviour by adjusting demand and or supply as function over time or by reducing their forecast uncertainty. Other reports from the same work package of TradeRES are published almost simultaneously, each focussing on another aspect of market model enhancements. These accompanying reports address sectoral flexibility, spatial flexibility, actor types, and modelling requirements for market designs.