Content uploaded by Teresa Simões Esteves

Author content

All content in this area was uploaded by Teresa Simões Esteves on Jan 31, 2022

Content may be subject to copyright.

New forecast tools to enhance the

value of VRE on the electricity markets

Deliverable number:

D4.9

Work Package:

WP4 - Development of Open-access Market Simulation

Models and Tools

Lead Beneficiary:

LNEG

Page 2 of 51

Author(s) information (alphabetical)

Name

Organisation

Email

António Couto

LNEG

antonio.couto@lneg.pt

Christoph Schimeczek

DLR

christoph.schimeczek@dlr.de

Hugo Algarvio

LNEG

hugo.algarvio@lneg.pt

Isabel Preto

Smartwatt

isabel.preto@smartwatt.pt

Johannes Kochems

DLR

johannes.kochems@dlr.de

Tiago Santos

Smartwatt

tiago.santos@smartwatt.pt

Acknowledgements/Contributions

Name

Organisation

Email

Kristina Nienhaus

DLR

Kristina.Nienhaus@dlr.de

Ana Estanqueiro

LNEG

ana.estanqueiro@lneg.pt

Document information

Version

Date

Dissemina-

tion Level

Description

1

25.10.2021

Public

In this report the implementation of new forecasting

techniques and time synergies is explained.

Review and approval

Prepared by

Reviewed by

Approved by

See list above

Débora de São José

Ana Estanqueiro

Disclaimer

The views expressed in this document are the sole responsibility of the authors and do not necessarily reflect the views or

position of the European Commission or the Innovation and Network Executive Agency. Neither the authors nor the

TradeRES consortium are responsible for the use which might be made of the information contained in here.

Page 3 of 51

Executive Summary

The present deliverable was developed as part of the research activities of the

TradeRES project Task 4.4 - Enhancing the value of VRE on the electricity markets with

advanced forecasting and ramping tools.

This report presents the first version of deliverable 4.9, which consists on the descrip-

tion and implementation of the forecasting techniques aiming to identify and explore the

time synergies of meteorological effects and electricity market designs in order to maxim-

ize the value of variable renewable energy systems and minimize market imbalances.

An overview of key aspects that characterize a power forecast system is presented in

this deliverable through a literature review. This overview addresses the: i) forecast time

horizon; ii) type of approach (physical, statistical or hybrid); iii) data pre-processing proce-

dures; iv) type of forecast output; and v) the most common metrics used to evaluate the

performance of the forecast systems.

While in the TradeRES project work package 3 the conception of new market designs

and products are presented from a theoretical point of view, in this deliverable, the power

forecast capabilities to address the new designs and products are presented and dis-

cussed. The link between electricity markets time frames and the performance of the dif-

ferent power forecast approaches is analysed in this deliverable focusing on the day-

ahead market. Thus, as an initial step, a non-disruptive change in the day-ahead market is

proposed by simply postponing the gate closure hour according to the meteorological data

availability from the global numerical prediction models while the 24 hours forecast peri-

ods are still used.

Some preliminary results regarding the potential certainty gain effect from changing the

day-ahead market gate closure are presented and analysed in this deliverable. For the

Iberian electricity market, high power forecast errors are still observed, especially for wind

and solar power players. Even using a forecasting data from a forecast provider, limited

improvements are attained with the updated data from numerical global models.

In order to improve the forecasting accuracy for wind and solar power players, a new

forecast method developed within the scope of TradeRES is also presented. A key aspect

embedded in this method is the use of meteorological features that traditionally are not

used in power forecast systems and the definition of specific models according to the

weather conditions. Preliminary results suggest that the use of meteorological parameters

as wind gusts, wind power density, wind shear, and planetary boundary layer should be

used to improve the wind power forecast.

Another aspect regarding the meteorological time synergy and electricity markets ana-

lysed in this deliverable is the identification of extreme events. A wind power ramping

forecast approach implemented in the TradeRES forecast tools is described and aims to

complement the existing deterministic/probabilistic time series. This approach can be

used to increase the transmission system operators’ awareness level and helping them to

better scale the level of reserve required. Market players can also take advantage of this

information to define strategic bidding.

Page 4 of 51

Table of Contents

Executive Summary .................................................................................................. 3

Table of Contents ..................................................................................................... 4

List of Tables ............................................................................................................ 5

List of Figures ........................................................................................................... 5

List of Abbreviations ................................................................................................. 6

1. Introduction .............................................................................................................. 8

2. Power forecasts ......................................................................................................10

2.1 The forecasting process and objectives .........................................................10

2.2 Forecast time horizon .....................................................................................11

2.3 Type of approach ...........................................................................................12

2.3.1. Physical approaches ...............................................................................12

2.3.2. Statistical approaches .............................................................................17

2.3.3. Hybrid approaches ..................................................................................20

2.4 Data pre-processing .......................................................................................22

2.5 Forecast output: deterministic, probabilistic, or ramp events ..........................23

2.6 Metrics to evaluate the performance of the forecast approaches ....................24

3. Electricity markets time frames and power forecasts ..............................................27

3.1 Impact on market modelling ...........................................................................30

4. Forecast approaches developed in TradeRES project ............................................32

4.1 Preliminary results ..........................................................................................32

4.1.1. Wind, solar, small hydro and electricity demand forecast ........................32

4.1.2. Power forecast with feature selection ......................................................35

4.1.3. TradeRES - vRES power forecast approach ...........................................38

4.2 Wind power ramping forecast .........................................................................40

4.2.1. Ramp detection algorithm .......................................................................41

4.2.2. A nested forecast approach ....................................................................43

5. Final remarks ..........................................................................................................44

References ..................................................................................................................45

Page 5 of 51

List of Tables

Table 1. Time horizon, temporal scale and common application of the forecast approaches [17]. ..11

Table 2. Example of global and regional models available. Adapted from [21]. ...............................14

Table 3. Characteristics of the most common forecasting methods [15], [41], [42]. .........................19

Table 4. Key Schematic 2X2 contingency table for power ramp detection. Adapted from: [47]. ......25

Table 5. Scenarios analysed to quantify the certainty gain effect for different electricity market

players. ..............................................................................................................................................34

Table 6. Results for scenario 1 (wind power forecast) in relation to the DAM forecast at 06:00

(UTC). ................................................................................................................................................35

List of Figures

Figure 1. Recommended source of information/approach for solar and wind for the different time

horizons and spatial resolution. Adapted from [18]. ..........................................................................12

Figure 2. ECMWF forecast performance. Monthly wind speed root mean square error at 850 hPa

for one-day (blue) and five-days (red) time horizon forecast. Bold lines represent a 12-month

moving average of the results (figure extracted from [20]). ..............................................................13

Figure 3. From global to mesoscale/regional numerical models (figure extracted from [22]). ..........15

Figure 4. Main steps applied in the physical wind power forecast approaches. The interactions

indicated with the orange arrow represent an alternative approach within the physical forecast

approaches. .......................................................................................................................................16

Figure 6. Example of the main steps applied in the statistical forecast approaches for wind power

applications. .......................................................................................................................................19

Figure 5. Autocorrelation values of electricity demand in Portugal for different hourly delays. ........19

Figure 7. Main steps applied in the hybrid wind power forecast approaches based on a combination

of different forecast approaches. .......................................................................................................21

Figure 8. Different forecast outputs: a) deterministic, b) probabilistic, and c) ramp events (red

background represents periods with severe power ramps and green background represents

periods where power ramps are not expected). ................................................................................23

Figure 9. Forecast errors according to time horizon for different wind power forecast approaches.

HWP approach refers to a physical approach and “HWP/MOS” refers to a hybrid forecast

approach. Figure adapted from [33]. .................................................................................................28

Figure 10. Solar power forecast skills according to time horizon and type of forecast approach.

Figure extracted [79]. ........................................................................................................................28

Figure 11. Identifying the time synergy between the meteorological data availability and DAM

possible designs. D represents the day on which the simulation is carried out (Figure extracted

from [80]). ..........................................................................................................................................29

Figure 12. Histogram of power forecast error levels created in AMIRIS following a normal

distribution with a mean of 0.05 and a standard deviation of 0.1; 8760 hourly data points

representing one year. ......................................................................................................................30

Figure 13. Sample impact of power forecast errors on (non-realistic) DAM clearing prices; black

curve represents prices without power forecast errors, red dots resemble prices that include

modified renewable feed-in estimates based on the same error distribution function as shown in

Figure 12. ..........................................................................................................................................31

Figure 14. Daily average profiles for a) wind, b) solar PV and c) small hydro for the nine clusters

(profiles) for Portugal. ........................................................................................................................33

Figure 15. Average RMSE improvements for the seven wind parks analysed compared with the

benchmarking approach. “PCA-WithoutSFF” – PCA approach without applying SFF algorithm;

“NWPPoint+SFF” – data from NWP was extracted to the nearest point of each wind park and the

SFF was applied; “PCA+SFF” – PCA approach and application of the SFF algorithm. ...................38

Figure 16. Power forecast approach implemented in TradeRES project. .........................................38

Figure 17. Example of one event in time t (black line) and one event in time t+1 (green line). The

magenta “*” symbols represent the average geometric center of each candidate, while the magenta

line indicates the trajectory of the meteorological event. ..................................................................42

Figure 18. Example of the outcomes from the nested forecast approach. .......................................43

Page 6 of 51

List of Abbreviations

AMIRIS

Agent-based Market model for the Investigation of Renewable and Integrated

energy Systems

ANN

Artificial neural networks

AR

Autoregressive

ARIMA

Autoregressive integrated moving average

ARMA

Autoregressive Moving Average

Bias

Bias Score

BRPs

Balance responsible parties

CFD

Computational fluid dynamics

D

Deliverable

DAM

Day-Ahead Market

ECMWF

European Centre for Medium-Range Weather Forecasts

FN

False negative

FP

False positive

GFS

Global Forecast System

hPa

Hectopascal

IBC

Initial and Boundary Conditions

IDM

Intraday markets

KNN

K-nearest neighbour

KSS

Hanssen & Kuipers Skill Score

Lat

Latitude

Long

Longitude

MA

Moving Average

MAPE

Mean absolute percentage error

MATREM

Multi-Agent TRading in Electricity Markets

Meso

Mesoscale numerical model

Micro

Microscale numerical model

ML

Machine Learning

NB

Normalized bias

NRMSE

Normalized root mean square error

NWP

Numerical Weather Prediction

OF

Objective function

P

Mean sea level pressure

PCA

Principal component analysis

PCs

Principal components

PDF

Probability density functions

POD

Probability of detection

PV

Photovoltaic

r

Pearson correlation coefficient

RMSE

Root mean square error

SCADA

Supervisory control and data acquisition

SFF

Sequential forward feature

SIDC

Single Intraday Coupling

TCT

Target-circulation types

TN

True negative

TP

True positive

TSO

Transmission System Operator

UTC

Universal time coordinated

vRES

variable Renewable Energy Sources

Page 8 of 51

1. Introduction

The present deliverable was developed as part of the research activities of the

TradeRES project’s Task 4.4 - Enhancing the value of VRE on the electricity markets with

advanced forecasting and ramping tools (work package 4). This report presents the deliv-

erable D4.9 which consists of the description of forecasting techniques implementation

aiming to identify and explore the time synergies of meteorological effects and electricity

market designs to maximize the value of variable renewable energy systems and mini-

mize market imbalances.

One of the most important challenges in the energy sector is the large-scale integration

of renewable energy sources (particularly, variable renewable energy sources - vRES

such as wind and solar) into electrical power systems in an economic and environmentally

sustainable way. Transmission system operators (TSOs) must always ensure the balance

between electricity production and consumption. Currently, a safe and robust operation of

a power system needs highly accurate forecasts of both vRES power production and con-

sumption to minimize the need for balancing the energy in the reserve markets, typically

at high costs [1]–[3]. With near to 100% renewable power systems, the role of the forecast

system and its accuracy will be even more relevant.

Forecasts are also important to electricity markets. To participate in the different prod-

ucts from electricity markets, market players/actors need to rely on forecast systems to

build their bids. With the existing market designs, when power producers do not follow the

scheduled bid, they are penalized and its profits are strongly decreased [4]. Due to the

intrinsic chaotic nature of atmosphere, the participation of vRES players in the existing

markets is still a challenge, especially, when long time horizon forecasts are needed. In

work package (WP) 3 of TradeRES project the shortcomings and alternative designs for a

near 100% renewable electricity system were addressed. For day-ahead market (DAM),

which is the most used and with higher liquidity market, the authors of deliverable 3.5 [5]

suggested a reduction of the time gap between the DAM closure hour and the forecast’s

delivery time while keeping the current organization of wholesale electricity trade. Never-

theless, it is necessary to assess if the new gate closure’s timing are enough to reduce

expressively the vRES power forecast errors, or if it is necessary to replace the existing

designs [5]. Therefore, to design electricity markets that are adequate to vRES trading it is

crucial to understand the capabilities of power forecast systems as well as how they work

in order to i) identify potential new time frames for this specific market, ii) identify the play-

ers that have more challenges to participate in the existing DAM, and iii) develop the fore-

cast tools required for TradeRES project and the electricity markets for 2030 and beyond.

Another aspect regarding the synergy between meteorological timings and electricity

markets refers to the identification of extreme events. Extreme events as wind power

ramps usually have a significant impact on the DAM [6], [7]. In the case of wind power

ramps, the early identification and forecasting of these events triggered by weather condi-

tions can allow to raise the level of Transmission system operators’ (TSOs) awareness

helping them to better scale the level of risk that exists for the power system [7] as well as

commit additional reserves, to minimize operational risks. It should be noted that this type

Page 9 of 51

of information can complement (and does not replace) traditional time-series forecast,

allowing to dynamically allocate the level of reserves needed. Market players can also

take advantage of this information to participate strategically in electricity markets since,

under these conditions, large vRES forecast errors in DAM are expected [7].

Forecast of wind power ramps is a relatively novel research topic and the works al-

ready published highlight that the trigger mechanisms of such events are rarely similar

across the control regions or wind parks [8]. Nevertheless, one of the most successful

approaches to understand and forecast the dynamics of wind power ramps involve the

use of holistic approaches capable of accounting the spatial and temporal development of

atmospheric large-scale circulation [7], [9]. In [7], an automated cyclone detection algo-

rithm was settled to identify challenging weather situations for the TSO. In [9], the authors

also applied an automated cyclone detection algorithm and compared its performance

with a windstorm algorithm. The highest performance to detect wind power ramps is ob-

served with the windstorm detection algorithm. Nevertheless, all the previous algorithms

have a common shortcoming: a wind power ramp is neither always a consequence nor it

is always linked to the existence of extreme wind speed values, being essentially depend-

ent from the previous (historical) state of the atmosphere. In this sense, a new algorithm

that uses a time numerical differentiation to fit the particular case of wind power ramps

events was developed and is presented in this deliverable.

This deliverable is organized as follows: an overview regarding the power forecast sys-

tems is provided in section 2. In section 3, the link between electricity market time frames

and power forecasts capabilities are discussed. Section 4 provides some preliminary re-

sults. These results led to the development of the forecast approaches presented in this

section and that will be applied and tested in regional market case studies (WP 5). Section

5 briefly presents some final remarks.

Finally, it should be highlighted that the work conducted so far in T4.4, which is sum-

marised in this report, paves the way for addressing some of the market designs and

products choices identified in TradeRES. This first version of deliverable will focus on

DAM and a second version of this deliverable will be released at Month 41.

Page 10 of 51

2. Power forecasts

The forecasting problem is transversal to several sectors of activity, such as financial,

scientific, industrial, political, etc. In the energy sector, several systems have been devel-

oped in the recent years to predict the power output from wind or solar power plants as

well as the electricity demand. In general, forecast is performed in an instant, t, for a future

time horizon, t+k. Despite the developments observed in forecast tools/approaches, in the

energy sector, most of the applications for European countries still refer to the average

power, Pt+k, that is expected to be provided to the grid at time t+k. In the literature, sever-

al classifications for the power forecast systems are available [10]. These systems can be

classified according to the forecast time horizon and type of approach. In the next subsec-

tions, these classifications are addressed as well as some of the additional steps to im-

plement a forecast system.

It should be noted that this chapter provides a summary of approaches commonly ap-

plied in the energy sector aiming to highlight the different options available. This back-

ground is important to establish how to proceed to implement the forecast approaches

most suitable to the different needs of the project. Detailed and up-to-date literature re-

views forecast approaches can be found about wind power [1], [2], [10], solar power and

electricity demand [11], [12].

2.1 The forecasting process and objectives

The forecasting process aims to transform one or more independent variables (inputs)

into one or more dependent variables (outputs). This process is characterized by some

key steps [13]:

- Problem definition

- Data collection

- Descriptive data analysis

- Forecast model selection

- Model validation

- Forecast

- Performance evaluation

Problem definition consists of evaluating the forecast period, forecast horizons and the

time step of the outputs. The type of output needed and the establishment of admissible

errors in the results are also established in this step. In the data collection phase, the vari-

ables under study (object of the forecast) and the independent variables necessary to

build the forecast model need to be collected. For the descriptive analysis of the data, it is

necessary, in the first place and when working with a time series, to take into account that

successive observations are not independent events [14], and as such, the order of ob-

servations must be respected. According to [13], to obtain greater sensitivity of the data

under analysis, they should be represented in the form of a temporal graph and a sum-

mary of some statistical parameters should be computed. This procedure makes possible

to identify anomalies in the data, trends and seasonality that otherwise might not be evi-

dent.

Page 11 of 51

After analysing the data, the forecasting method is applied. This task consists of choosing

and adjusting one or more models to the specific case study, that is, reproducing the de-

pendent variable, depending on the independent variable (or variables), within a certain

margin of error. The model selection should take into consideration aspects as the time

horizon and the type of information expected from these models (deterministic, probabilis-

tic, or ramp event forecast). Once selected, the method must be validated. This validation

is done by assessing the performance of the forecast. For this purpose, the method is

normally adjusted to only a part of the available data, with the rest being used for its vali-

dation. Once validated, the method is implemented and its control is carried out continu-

ously by measuring the forecast errors (e.g., bias) to verify the continued validity of the

method, and, if necessary, make all necessary updates to reduce the errors.

2.2 Forecast time horizon

The time duration for which the power output is forecast is known as the forecast time

horizon. The forecast horizon of interest depends on the different applications, and in en-

ergy sector it can be divided into four main time scales: very short-term, short-term, medi-

um-term, and long-term [1], [15], [16] The time frames of this classification can slightly

change among the different authors. The application for the different time frames is de-

picted in Table 1.

Table 1. Time horizon, temporal scale and common application of the forecast approaches [17].

Time horizon

Temporal scale

Applications

Very short-

term

From seconds to 30

minutes

- Real-time dispatch and regulation

operations on the network;

- Forecasting the consumption of build-

ings in the context of micro-grids

Short-term

From 30 minutes to 6

hours

- Impacts on energy price determination

in intraday markets;

- Support decision on the status of net-

work loads;

- Support the decision to turn-on or off

the generator set with quick response;

- Security operations for the energy

market.

Medium-term

Varies between 6 hours

and 1 day

- Support the decision to turn genera-

tors on or off;

- Safety time horizon for the day-ahead

market.

- Impacts on energy price determination

- Allocation of power reserves

Long-term

More than a day

- Planning of maintenance operations;

- Power system adequacy planning

Page 12 of 51

2.3 Type of approach

In this section, the most common types of forecast approaches used in the energy sec-

tor are presented. Figure 1 depicts the recommended source of information/forecast ap-

proach according to the spatial and temporal horizon that will be further analysed in the

next subsections.

Figure 1. Recommended source of information/approach for solar and wind for the different time

horizons and spatial resolution. Adapted from [18].

2.3.1. Physical approaches

The physical power forecast approaches are mainly based on the use of numerical me-

teorological models – Numerical Weather Prediction (NWP), which parameterize and sim-

ulate in detail the atmosphere and its circulation mechanisms. NWP models provide me-

teorological parameters as wind components, cloud coverage, air temperature, and pres-

sure, that are used to generate forecasts.

This type of model is being developed since 1950 when NWP models were used to

make weather forecasts with time horizons on the scale of days. They were, however,

very primitive models based on quasi-geostrophic theories where it was impossible, either

due to lack of knowledge or lack of computational resources, to include relevant physical

processes (radiation processes and phase transition) to make reliable predictions[19].

Over the years and with substantial technological improvement, these models and the

parameterizations that govern them were improved and the relevant physical processes

missing were progressively added. Currently, these models are still the core of weather

forecasting and have evolved substantially following the growing knowledge regarding the

physical processes that govern the atmosphere dynamics and its circulation as well as the

computational capabilities.

Spatial Resolution (km)

Time Horizon (h)

0.1 1 10 100 1000

0.1

1

10

100

1000

0.01

Wind

Solar

Physical

Statistical

Satellite images

Sky images

Meso.

Global

Micro.

Page 13 of 51

NWP models are, nowadays, less simplistic and with more detailed and precise physi-

cal parameterizations. Additionally, these models benefited from more efficient and repre-

sentative data acquisition and assimilation systems around the globe, which includes me-

teorological stations, satellite data, radiosondes and measurements performed by air-

planes and ships. All these improvements lead to a significant decrease in the forecast

errors, as reported for the operational European Centre for Medium-Range Weather Fore-

casts (ECMWF) model, Figure 2. This figure also highlights that the errors are significantly

higher for a time horizon of five-days compared to one-day time-horizon.

Figure 2. ECMWF forecast performance. Monthly wind speed root mean square error (RMSE)

at 850 hPa for one-day (blue) and five-days (red) time horizon forecast. Bold lines represent a 12-

month moving average of the results (figure extracted from [20]).

There are two major groups of NWP models: global models (grid covering all the Earth)

and regional/mesoscale models (also known as limited area models) [21]. The main dif-

ferences between these two groups are related to the spatial and temporal resolution of

the model, the geographical area covered and the time horizon. Moreover, region-

al/mesoscale models are calibrated using physical parameterization for specific regions

which can enable to reduce the forecast errors. These differences will have a significant

impact on forecast accuracy and computational effort [21].

Table 2 characterizes the spatial and temporal resolution of some of the existing global

and mesoscale/regional model.

Page 14 of 51

Table 2. Example of global and regional models available. Adapted from [21].

Type

of

model

Provider

Model

Temporal

resolution

(hour)

Horizontal

resolution

(aprox. km)

Runs per

day (UTC)

Global

European Centre

for Medium-Range

Weather Forecasts

(ECMWF)

Integrated Fore-

casting System

1

10

4 (00, 06,

12 and 18)

Canadian Mete-

orological Centre

Global Determinis-

tic Prediction Sys-

tem

3

25

2 (00, 12)

National Centers

for Environmental

Prediction

Global Forecast

System (GFS)

3

25

4 (00, 06,

12 and 18)

Deutscher Wetter-

dienst

Icosahedral Nonhy-

drostatic

1

13

4 (00, 06,

12 and 18)

Regional

Deutscher Wetter-

dienst

Consortium for

Small-scale Model-

ing

1

2.8

8 (00, 02,

…, 18 and

21)

Finnish Meteoro-

logical Institute

High Resolution

Limited Area Model

1

7.5

4 (00, 06,

12 and 18

Weather Research

and Forecasting

(WRF)1

Defined by

user (< 1

hour)

Defined by

user (< 5 km)

User specif-

ic (but lim-

ited to the

global mod-

el availabil-

ity)

Most of power forecast systems use a coupled approach by feeding the regional mod-

els with initial and boundary conditions (IBC) gathered from the global models. This ap-

proach aims to mitigate some of the drawbacks associated with global models that pre-

sent low spatial and temporal resolutions by describing the behavior and evolution of the

air masses, and explicitly treat the atmospheric phenomena that need high spatial and

temporal resolution. On the other hand, numerical mesoscale/regional models always

need to be forced with IBC at the limits of their domains (boundary, surface, and top of the

domain), Figure 3. These IBC can be historical data from reanalysis projects (used to pro-

duce wind or irradiance atlases for example) or by operational global forecasts projects

(as GFS).

1

Example weather forecast providers using WRF model: Climetua (http://climetua.fis.ua.pt/weather) and Me-

teoGalicia (https://www.meteogalicia.gal).

Page 15 of 51

Figure 3. From global to mesoscale/regional numerical models (figure extracted from [22]).

Despite the improvement observed in NWP models, these models still present sys-

tematic errors partly explained by the: 1) inadequate model’s physics parametrizations; 2)

inability to handle sub-grid scale phenomena; 3) stochastic behaviour of the atmosphere

and 4) uncertainty on IBC [21], [23]. Although, the precision of the results of these models

increases proportionally to the number of data assimilated in these models, as well as the

quality of these same data [21].

At the current stage of NWP, the model’s parameterization and the spatial resolution

from these models are unable to simulate some local effects as the exact location and

extent of cloud fields in the case of solar power forecast. To properly account for these

effects and correct the outputs from NWP, downscaling techniques can be applied to cor-

rect the data providing location-specific forecasts [24], [25]. Thus, downscaling consists of

applying further methods to enhance the data extracted from the NWP with local/regional

effects. This process can be performed through various statistical methods that establish

relationships between local variables (such as wind speed) and variables with large-scale

characteristics (such as pressure fields). Another possibility is the use of other physical

approaches that varies according to the type of technology under consideration. Below,

some examples of downscaling physical approaches for the wind and solar power cases

are provided.

Wind power

Microscale models: This type of model allows working with high spatial resolutions (up to

10 – 30 meters). With the growing need to estimate accurately the wind resource for dif-

ferent applications, new models for simulation of wind flow were developed. These models

can be classified into linear and non-linear [26]. Linear models as the Wind Atlas Analysis

and Application Program software have the advantage of low need for computing re-

sources and it enables to evaluate, with reasonable accuracy, the wind resource for flat

Page 16 of 51

orography with small elevations, i.e., under non-complex terrain conditions [19]. However,

these models tend to, e.g., miscalculate the wind speed behaviour in the lee side of the

hills [26]. Therefore, these models are unsuitable for complex terrain. The advances in

numerical modelling together with the increase in computational capabilities enabled the

development of non-linear models in the flow simulation industry and in the assessment of

wind potential. Among these non-linear models, in the wind sector, computational fluid

dynamics (CFD) models stand out enabling to increase the accuracy of wind potential

assessments, especially in complex terrain [27]. Results from several authors highlighted

the benefits of this model against the linear models [28]. These benefits are derived from

the inclusion of thermal effects in the vertical stratification of the CFD simulations. This

type of approach was explored in [29] showing higher performance when compared to a

traditional statistical approach, especially for periods with high or low energy levels and in

the ascending wind power ramps.

Conversion to power: This type of approach is based on the power curves from the

wind turbine manufacturers/or estimated based on historical wind speed and power data

to determine the power from the wind turbine or park [30]. Typically, the historical power

curves of wind turbines/parks are defined as a function of additional parameters, such as

wind direction to account for physical features (e.g., wake effect of wind turbines, air den-

sity and terrain) in surrounding regions [30]–[32]. Based on the results of the NWP, name-

ly, the wind speed and direction for the hub height of the wind turbine, the power curves

are applied, and the forecast is obtained. This type of approach is the most common dur-

ing the initial periods of wind park operation where there is no historical data.

Figure 4 presents the main steps commonly applied to obtain the physical-based wind

power forecast.

Figure 4. Main steps applied in the physical wind power forecast approaches. The interactions

indicated with the orange arrow represent an alternative approach within the physical forecast ap-

proaches.

Microescale models

NWP data

Wind speed Wind direction Atmospheric

pressure

Downscaling to

Wind turbine

Hub height Power Curve Forecasting

Power

Terrain Surface

Roughness Obstacles

Page 17 of 51

Solar power

Cloud and all-sky imagery: In addition to the daily cycle, the other factor with the great-

est impact on ground-level solar irradiation and, consequently, on solar production is the

cloud cover [33]. This parameter presents a high variability that can be induced by local

effects, which are not always correctly described by NWP. In recent years, models that

forecast cloud cover from images from the sky taken from all-sky cameras or by artificial

satellites have been developed. The all-sky cameras, installed at ground level, allow to

obtain images of the sky, being very useful for very short and short-term forecast time

horizons due to the reduced field of view of the camera [34], [35]. On the other hand,

through satellites, it is possible to obtain images with a wide field of view, but with lower

spatial and temporal resolution. In this sense, information obtained through satellites is

more suitable for short and medium-term forecasts [36]. Regardless of the source of in-

formation (cameras or satellite), this type of approach uses algorithms that identify cloud

coverage patterns between sequential images. The information from the images can be

transformed, for instance, into cloud motion vectors, enabling to determine the move-

ments (intensity and direction) of individual clouds [37]. This information regarding the

cloud cover can then be used in different ways: i) correct the NWP outputs, or ii) apply

semi-empirical models to obtain ground level solar irradiance.

The use of these models is crucial for weather-dependent generation technologies as

wind and solar power forecast. Nevertheless, the outputs from these models are also

used by several authors to improve the load forecast for short- and medium-term forecast

horizons [12].

2.3.2. Statistical approaches

To overcome the inefficiencies of the physical methods described above and, at the

same time, obtain operational forecasts with adequate precision to manage the variability

of vRES, several statistical methodologies have been developed. These forecasting ap-

proaches are based on statistical approaches using historical time series (observed or

forecasted) data. More precisely, this type of approach seeks to establish relationships

between historical data series with what is currently observed, at the instant for which the

forecast is to be made. Despite the constant emerging of new forecast techniques that

require deep mathematical knowledge, compared to physical forecasting methods, this

type of method presents reduced complexity and is less costly (whether in terms of time or

resources) since the physical processes are not explicitly expressed. Within the statistical

models, three distinct methods are usually considered: persistence, time series modelling,

and automatic learning methods.

Persistence methods: Persistence-based forecasting methodology is the most basic

and simplest statistical forecasting methodology to be implemented. Despite belonging to

the group of statistical forecasting methods, it is often addressed separately since it is

considered as reference, or benchmark, against all other used methodologies. In this

sense, to study the feasibility of implementing new forecasting methodologies, the results

obtained through these must be matched with the results achieved by the persistence

method. Only methodologies that present more favourable results than those generated

by persistence are likely to be implemented. The persistence method assumes that the

Page 18 of 51

wind/solar power or electricity demand, remains equal, at a future instant, to the value

observed at the instant for which a forecast is made. If the power, at time t, is given by ,

then the power at the future time, t+∆t, will be given by:

(1)

where corresponds to the time interval for which the forecast is to be performed. For

very short forecast time horizons (Table 1), this model provides results, on average, with

some accuracy. However, and as expected, due to the vRES variability as the forecast

time horizon increases, the accuracy of this methodology decreases. For time series with

high fluctuation in production (e.g., wind parks in areas with complex orography), the ac-

curacy of this method is reduced. For solar power and electricity demand, the persistency

can assume the value of the previous day and/or week.

Methods based on time series modelling: For very short and short forecast time hori-

zons (Table 1), there is the possibility, with a certain degree of reliability, to use methods

based exclusively on the statistical analysis of time series of real data. Specifically, this

type of methodology tries to find out what is the relationship between a historical series of

production or demand data, and its value at the instant for which the forecast is to be

made, in order to obtain predictions for the following instants. Unlike physical models, in

this type of forecasting methodology, only one step is needed to convert the input data

into output data. Among the various statistical methods used in wind forecasting, the auto-

regressive (AR), moving average (MA), autoregressive moving average (ARMA), auto-

regressive integrated moving average (ARIMA), the Box-Jenkins methodology, and the

use of Kalman filters [1], [38].

Machine Learning Methods: A group of statistical data analysis methods that are ap-

plied to problems of this nature is machine learning methods. Machine learning (ML) or by

“gray box” [39] are models essentially characterized by the capacity for self-learning

through experience and training, i.e., ML offers the computational capacity to learn without

explicit programming. As such, ML algorithms present the possibility of learning and mak-

ing predictions on a set of data in an unexplicit way without following a set of static statis-

tical learning instructions. The process of a self-learning model includes a few steps.

Firstly, it is necessary to obtain data referring to the past for a later training phase.

Secondly, a relationship between the input and output data – the input and the output –

desired through a target function is defined. The third step is to choose the self-learning

model. Then, this model undergoes a training process, using the set of data and previous-

ly determined examples. In most cases, this type of methodology generally presents the

best forecast results. However, the implementation of these methodologies has the disad-

vantage that it is not possible to describe the relationship between the elements of the

model, i.e., it is not possible to describe or understand the relationships found by these

models between the input variables and the output variables [40]. Among the methods of

this group, the artificial neural networks (ANN) stand out for their simplicity and efficiency.

One of the disadvantages of statistical methods (e.g., multiple linear regression) lies in

the inability to deal with the occurrence of events with distinct patterns within the time se-

ries itself, namely, the distinction between weekly, weekend and special days (e.g., holi-

Page 19 of 51

days). ANN-based methods, on the other hand, are capable of accepting these character-

istics as an independent variable and modelling implicit non-linear relationships between

the forecast variable and the variables that affect it.

Figure 5 depicts an example of the main steps used to apply statistical forecast ap-

proaches. As can be seen in the figure for the case of wind power forecast typically only

wind speed and power historical data are needed.

Figure 5. Example of the main steps applied in the statistical forecast approaches for wind pow-

er applications.

This type of approach is also common for load demand forecast [15]. In this case, soci-

oeconomic variables economic growth rates are also frequently considered as well as

historical values (delays) since the autocorrelation between successive events is high at

different temporal scales, as can be observed in Figure 6. This figure depicts the energy

demand autocorrelation in Portugal continental for different. The correlation with a 24 hour

delays is 0.97.

Figure 6. Autocorrelation values of electricity demand in Portugal for different hourly delays.

Table 3 presents a list of the most common statistical and learning approaches and

their advantages/disadvantages.

Table 3. Characteristics of the most common forecasting methods [15], [41], [42].

Power

Speed Statistical

models

Wind

power

Historical data

Page 20 of 51

Forecasting

method

Advantages

Disadvantages

Linear Re-

gression

Simplicity;

You can use only one (Single) or several

(Multiple) independent variables.

Only capture relationships between

linearly correlated variables;

Sensitive to extreme values (outli-

ers);

Variables used in forecasting must

be linearly independent.

Time Series

Analysis

(Box-

Jenkins)

Adaptable, there are many versions of the

method and it has already been exten-

sively studied;

Able to deal with seasonality and non-

stationarity.

Requires only historical data for the

series;

Unlikely to perform well in long-term

forecasting;

It is computationally demanding to

estimate model parameters;

Requires a solid knowledge of sta-

tistics inherent to the time series.

K-nearest

neighbour

(KNN)

It is relatively simple to understand and

implement;

It does not need a training phase, making

its prediction based on observed historical

values;

Non-parametric approach, which does not

imply any assumptions regarding the dis-

tribution of the variables to be predicted.

Requires an extensive period of

historical data;

Computationally demanding for

large datasets.

Artificial

Neural Net-

works

It is not necessary to know the relation-

ship between dependent and independent

variables;

Capable to deal effectively with non-linear

relationships;

Capable to deal with the presence of

noise in the dataset without significantly

affecting the forecast result.

It is computationally demanding to

train the neural network;

Requires a large amount of histori-

cal data from independent variables;

Do not result in a mathematical

model with physical meaning.

Support

Vector Ma-

chine

Adjustment of the adjustment parameter

of the objective function helps to avoid

over-fitting the training data (over-fitting);

Convex optimization problem (there are

no local minima);

Use of the kernel trick, which maps the

variable space to a non-linear vector

space, allowing to capture non-linear rela-

tionships more efficiently.

It is difficult to define a “good” kernel

function;

Computationally demanding for

large datasets.

2.3.3. Hybrid approaches

Hybrid power forecast models are models that combine two or more models of similar

or different nature [40]. The genesis of such approach results from the combination of

both statistical and physical models. The time scales applicable to forecasting methods

can also be different because it is possible to join methods whose forecast horizon is dif-

ferent. In general, hybrid models are composed of a linear model and a non-linear model

to be able to analyse the respective linear and non-linear components of the time series of

data. Hybrid methods can be categorized into four different classes:

I. Weight-based methodologies: These methodologies are based on assigning weights

to the various forecast models used according to their performance. It is a simple method-

ology, easy to implement and has the advantage of adapting to new datasets. It is a suita-

Page 21 of 51

ble methodology for a wide range of forecast time horizons. However, this does not guar-

antee the best forecasting efficiency for the entire forecasting time horizon and has the

need for an additional model to assign the weights.

II. Methodologies based on the combination of different forecast approaches: the pre-

diction is performed via the combination of different types of approaches (combine physi-

cal with statistical approaches). This type of methodology presents a robust behavior due

to the sudden, nature of the vRES generation wind speed. Therefore, it is possible to ob-

tain high forecasting efficiencies. However, this type of methodologies has the disad-

vantage of requiring the user to understand the complex mathematical model that per-

forms the data decomposition and the absence of a dynamic behavior in the sense that

the use of new data series, or the updating of the same, may result in a slow response of

the methodology.

III. Methodologies based on optimization techniques and parameter selection: These

methodologies are based on the optimization of the forecast model parameters – mete-

orological parameters such as temperature, wind speed and direction, precipitation,

among others. Despite providing the user with a greater understanding of the impact of

different parameters on forecasting effectiveness, this approach is difficult to implement.

IV. Methodologies based on post-data processing techniques: These methodologies

are based on a post-processing of forecast data. More specifically, this approach studies

the impact of residual errors in the forecasts obtained, through the forecast models used,

on the overall effectiveness. The effectiveness of forecasts considerably increased com-

pared to other approaches. However, due to the need to calculate residual errors, compu-

tational and temporal resources required are much higher than those needed for the other

approaches presented.

Figure 7 provides the main steps typically applied in hybrid power forecast approaches.

Comparing with the previous approaches, it includes both NWP and historical data to feed

the statistical models. As discussed in this section, for this type of approach different vari-

ations can be used.

Figure 7. Main steps applied in the hybrid wind power forecast approaches based on a combi-

nation of different forecast approaches.

Power

Speed Statistical

models

Wind

power

NWP data

Wind speed Wind direction Atmospheric

pressure

Historical data

Page 22 of 51

2.4 Data pre-processing

Before applying statistical forecasting methods, it is common to apply pre-processing

procedures to the data under analysis [15], [43]–[45]. The most common types of pre-

processing are data cleaning, integration, transformation and dimensional reduction.

These treatments can be used in various combinations or alone. Data cleaning consists of

removing or modifying values from incorrect values and entering missing values. Integra-

tion consists of combining data from different sources. The transformation consists, for

example, in normalizing the data to scale them on a predefined range (e.g., [0, 1]) or in

transforming a value recorded every 15 minutes into an hourly average.

Dimensional reduction consists of reducing the number of existing variables. This re-

duction can be done, for example, through principal component analysis (PCA), discrimi-

nant analysis, empirical mode decomposition or wavelet analysis [46]. In PCA, an orthog-

onal transformation is applied to convert the data into a set of values of linearly uncorre-

lated variables designated as principal components (PCs) [47]. With this procedure, the

number of PCs generated in the process is always equal to or less than the number of

original variables. The transformation applied with this technique allows the first PC to

explain the largest possible variance, i.e., this PC characterizes the maximum possible

variability observed in the data. With the restriction that it is orthogonal, the subsequent

PC has the greatest possible variance that was not explained by the previous one. The

process continues until the number of PCs became equal to the number of original varia-

bles. The resulting vectors enable to obtain an uncorrelated orthogonal basis set and they

are used to feed the statistical forecasting techniques.

Another type of approach to reduce the data dimension is the application of feature se-

lection algorithms [46]. The selection of the most relevant features aims to remove insig-

nificant entries in the forecasting models allowing to reduce model complexity as well as

computational costs [48]. These methods can be classified into three different types [45],

[48]: filter, wrapper and embedded. The filter methods remove the less significant varia-

bles a priori, and then a model is created with the remaining features. Variables are elimi-

nated with a criterion such as Pearson correlation. The wrapping methods involve the en-

tire training algorithm in the variable selection process. The algorithm runs with several

iterations (as many as there are variables) of the model by adding (or removing) variables

and evaluating the performance of the model obtained. For the construction of the final

model, the variables that enable to improve the result are kept and the rest discarded.

Embedded methods introduce the variable selection process directly into the training pro-

cess, in order to avoid the complete search that happens in wrapped methods, thus re-

ducing computational complexity [49]. Additionally, combinations of these methods can be

created, giving rise to the so-called hybrid methods. All these techniques aim to ensure

the robustness of the data, also bringing benefits in improving computational efficiency.

Another type of pre-processing is the decomposition and classification of data by clus-

tering [46]. Decomposition, in the context of the analysis of electricity demand forecast

refers to the data separation according to the seasonal, weekly, and special days (holi-

days) effects. For the vRES case, this classification can refer to the so-called weather

regimes (WR) types [50] or target-circulation types (TCT) [51]. The WRs allow to reduce

the complexity of meteorological variability while enabling the identification of daily recur-

Page 23 of 51

rent patterns in the climate system (top-down approach). On the contrary, TCT are de-

rived from the power system's weather response (down-top approach), which can be the

vRES generation [52].

2.5 Forecast output: deterministic, probabilistic, or ramp

events

The initial focus of power forecast systems was to provide deterministic information, i.e.,

a value for each time step of the temporal horizon. Famous statistical methods are ARMA,

ARIMA, Kalman filtering and Gaussian mixture models [39], [53], [54]. Other robust statis-

tical approaches include ANN, KNN among others [55]–[57].

As presented in Figure 8, deterministic forecast approaches do not include information

regarding its uncertainty, which can be very useful for utilities or for specific market players

through the definition of strategic bidding [58]. The need to characterize and assess the

uncertainty in the vRES power forecast to better integrate it in decision-making processes

led to the development of various probabilistic forecasting techniques [59]–[63]. Probabilis-

tic forecasts can allow: i) increase the revenues of market players within the electricity envi-

ronments, e.g., [64], [65], and ii) suitable reserve allocation [66].

Broadly speaking, the probabilistic forecast allows to obtain probability density functions

(PDF) for an specific time providing an interval of uncertainty of future events. The PDF

can be achieved using physical approaches (e.g., NWP ensembles [67], [68]), statistical, or

the combination of both. As described in [67], NWP ensembles forecasts are computation-

ally demanding when compared with statistical methods. Power forecast uncertainty using

statistical/hybrid models can be attained by calculating their distribution parameters based

on: i) nonparametric regression assumptions as quantile regression [61] and kernel density

estimators [69], [70], or ii) upon historical analogous [71]–[73] or iii) parametric distribution

assumptions.

Figure 8. Different forecast outputs: a) deterministic, b) probabilistic, and c) ramp events (red

background represents periods with severe power ramps and green background represents peri-

ods where power ramps are not expected).

Ramp events refer to the significant changes of power output in a short period. Thus,

the importance of the detection of severe power ramps for TSOs lies on the necessity to

control conventional power plants to balance those ramps in order to ensure the stable

Power

1 2 3 4 5 6 7 8 t+k

Time

1 2 3 4 5 6 7 8 t+k

Time

1 2 3 4 5 6 7 8 t+k

Time

a) b) c)

Page 24 of 51

operation of the power system. Contrary to deterministic and probabilistic that provide time-

series, this type of forecast provides binary information: existence or not of a power ramp

[74]. This information should be integrated into the existing forecasting systems as an addi-

tional feature, but must not substitute the existing forecasting systems [7]. Thus, the main

potential benefit of power ramps forecast is to alert the TSO regarding the existence (or

not) of power (rapid) ramps events for which they should be prepared to commit additional

reserves for safety and guarantee of robustness of the power system, in all meteorological

conditions [7], [47].

2.6 Metrics to evaluate the performance of the forecast

approaches

There is no single metric that can describe or measure the performance of a forecasting

methodology. In existing literature, some new deterministic and probabilistic metrics have

been proposed in the last years [75]. Nevertheless, most of them are not being adopted to

energy sector being difficult to place the results among the values found in the literature.

Taking into account this aspect, the following metrics will be used in TradeRES project to

access the accuracy of time-series forecast - normalized bias (NB), RMSE or the normal-

ized RMSE (NRMSE), the Pearson correlation coefficient (r) and the average value of the

absolute forecast deviation (

):

(2)

(3)

(4)

(5)

(6)

corresponds to the total nominal power of the control region or vRES

power parks under analysis. The bias corresponds to systematic error present in the fore-

cast. This metric denotes an average error value for the forecast time horizon allowing to

assess whether the forecasting methodology tends to underestimate or overestimate

comparing with the observed values. Ideally, a bias is sought, for the time horizon, as

close as possible to zero. RMSE allows to identify the variation of amplitude errors, due to

the squared nature of the differences. NRMSE, as aforementioned discussed, normalizes

the RMSE for an easily comparison between different forecast approaches. The perfect

score of the last two metric is also zero. The correlation coefficient measures the similari-

ties between the obtained forecasts, for a forecast time horizon, and the observed value

Page 25 of 51

for the same time horizon. This coefficient varies between [-1 1]. A value close to zero

means poor predictions, and the unit value represents perfect predictions. A value close to

-1 means that the forecast is in phase opposition. The average value of the absolute fore-

cast deviation allows to illustrate the mean forecast deviation in relation to the observed

power.

To quantify the improvement of using the forecast methods proposed in TradeRES pro-

ject, the approach followed in [73] is used for each metric:

(7)

where represents the results of a specific metric using the forecast

method implemented in TradeRES project, and represent the forecast

results for the benchmarking approach (that will be defined according to each case study).

A positive value indicates an improvement of the proposed forecast method. A negative

value corresponds to an underperformance of the TradeRES forecast method. It needs to

be mentioned that forecast tool proposed (section 4.3) provides probabilistic information.

Thus, the previous metrics will be applied to the quantiles obtained. The quantile closer to

the ideal score will be used.

Ramps power events refer to a dichotomous case, i.e., the existence or not of a pow-

er ramp. For this type of approach, a contingency table are usually built to derive the re-

sults. In Table 4, true positive (TP) corresponds to the ramps forecasted with the pro-

posed methodology that occurred; false positive (FP) corresponds to the ramps forecast-

ed but do not occur; false negative (FN) corresponds to power ramp events that occurred

but were not forecasted; and true negative (TN) corresponds to power ramps forecasted

and observed.

Table 4. Key Schematic 2X2 contingency table for power ramp detection. Adapted

from: [47].

Event Foreseen

Event Observation

Total

Yes

No

Yes

TP

FP

Foreseen Yes

No

FN

TN

Foreseen No

Total

Observed Yes

Observed No

N=TP+FP+FN+TN

From the contingency table the following metrics can be computed: Bias Score (Bias),

precision, the probability of detection (POD) and the Hanssen & Kuipers Skill Score

(KSS):

(8)

Page 26 of 51

(9)

(10)

(11)

The ideal score for the aforementioned metrics is 1. KSS ranges between 0 and 1 [76].

Bias, precision and POD metrics enables to understand if the power ramps algorithm has

the tendency to over foreseen (precision, POD and Bias Score > 1) or under foreseen

(precision, POD and Bias Score < 1) the number of power ramp events.

Page 27 of 51

3. Electricity markets time frames and power forecasts

The existing designs of most European electricity markets were defined during a con-

ventional energy technology dominated period. These technologies can respond to the

demand variability, they are easily adjustable and, if requested in due time, they can re-

spond efficiently to operational set-points. However, in addition to the negative environ-

mental impacts of using fossil technologies, the marginal cost to operate these technolo-

gies is high. In contrast, vRES are weather dependent and still present significant forecast

errors, especially for long time horizons.

DAMs require the forecast of electricity production 12-36 hours before physical delivery

in central Europe due to coupled DAM auction at noon, or 13-37 hours in Great Britain

2

,

Ireland and Portugal. This time gap between bidding and the first deliverable can jeopard-

ize the profitability of vRES [77]. The DAM shortcomings and alternative designs for a

near 100% renewable electricity system were addressed in Deliverable 3.5 from

TradeRES project [5]. The authors suggested a reduction of the time gap between the

DAM closure and the delivery time. This reduction could facilitate vRES, since it allows

reducing the uncertainty associated to power forecasts, and many flexibility options. The

authors concluded that the “choice for European market design is whether to maintain the

current organization of wholesale electricity trade, in which the 24 hours of each day are

traded together at noon the day before, or to replace it with a different wholesale market

design.”

As shown in Figure 9 and Figure 10, for a time horizon above six hours, NWP-based

forecast is the recommended approach for vRES technologies. Nevertheless, the forecast

performance is worse than the one expected for very short and short time horizons. In

[78], the authors quantify the annual value of using solar power forecast in the Iberian

electricity market. The forecast models that use NWP data showed the highest revenue.

The benefits from using a NWP-based forecast approach, with respect to the persistence

prediction, ranges from 1 to 6 kEUR per MW of PV capacity per year.

From a forecasting point of view, the motivation for the DAM closure change is related

to the availability of the IBC conditions used to feed the NWP models. As mentioned in

section 2, the quality of the forecast strongly relies on these data. For the European coun-

tries, IBC availability from global models is limited to updates every 6 hours (at 00, 06, 12

and 18 UTC). For participate in DAM, currently, the NWP-based power forecast systems

use the IBC from 00 or 06 UTC to obtain the expected vRES production or electricity de-

mand. In order to benefit from updated IBC data, postponing in some hours the DAM clo-

sure gate (Figure 11) could allow to keep its overall structure, while it is expected to re-

duce the forecast uncertainties. The new gate closure hour is associated with the IBC

delivery hour plus an additional period of two hours to perform all required steps (down-

2

For Great Britain, the Day-ahead auction for 60 min products has been moved to 9.20 due to the Brexit,

even enlarging the lead times. In addition, there is a 30 min auction held at 15.30.

Page 28 of 51

load the IBC data, run the numerical mesoscale/regional model and apply the different

forecast approach) to obtain the forecast.

Figure 9. Forecast errors according to time horizon for different wind power forecast approach-

es. HWP approach refers to a physical approach and “HWP/MOS” refers to a hybrid forecast ap-

proach. Figure adapted from [33].

Figure 10. Solar power forecast skills according to time horizon and type of forecast approach.

Figure extracted [79].

Page 29 of 51

Figure 11. Identifying the time synergy between the meteorological data availability and DAM

possible designs. D represents the day on which the simulation is carried out (Figure extracted

from [80]).

In this energy transition phase toward a near 100% renewable power system, this

postponing does not require any disruptive change in the market designs and, as de-

scribed in Deliverable 3.5 [5], it can allow a “compromise between the need to accommo-

date facilities with ramping constraints, which need longer lead times, and variable renew-

able energy sources, for which a short time between market clearing and delivery reduces

weather uncertainty”.

In the current market design, market participants can already make use of short-term

forecasts with high accuracy on intraday markets (IDM). Compared to the DAM design,

intraday market designs show a larger variability across European countries. There are

some (opening) auctions held on the day before delivery for some countries, such as the

IDM auction for Austria, Belgium, Denmark and Netherlands at 15:00 in which 15 min

products are traded. For the continuous trading, a greater degree of harmonization has

been established from the Single Intraday Coupling (SIDC) (see [81] and also TradeRES

D3.5). Continuous intraday markets allow a trading up until real-time for Finland. For other

markets, lead times are rather short and range from 5 mins for Austria, Belgium, Denmark

and Netherlands to 30 mins for France and Switzerland [81].

Improved forecasting accuracy can lead to smaller asymmetry for balance responsible

parties (BRPs), ultimately reducing their imbalance payments. Thus, there is already a

benefit of increasing generation forecast quality which will be further increased with trad-

ing even closer to lead time, potentially also in DAM markets, as well as rising shares of

vRES.

Page 30 of 51

3.1 Impact on market modelling

Following the approach presented in Deliverable 4.1 [80], the Agent-based Market

model for the Investigation of Renewable and Integrated energy Systems (AMIRIS) was

enhanced to consider power forecast errors for agents marketing renewable energy.

Since error distribution functions for different technologies and varying gate closure lead

times are not yet fully integrated, a Gaussian distribution was chosen to represent the

forecast error. The distribution parameters are exemplary and were selected to illustrate

the impact of forecast errors on the market simulations and to demonstrate the basic func-

tionality developed within TradeRES. The data used does, however, not yet represent

realistic data regarding power forecast errors. Such realistic data – potentially also con-

sidering error dependency on a given weather situation – will be developed in the project

and integrated into AMIRIS in the near future.

In this demonstration of the approach, the renewable energy trading agent was config-

ured to consider power forecast errors with a constant distribution function over time.

Thus, each hour of the day was assumed to follow the same error distribution. It is as-

sumed that the power forecast error follows a normal distribution formulation. Generated

values represent levels with different relative error. To obtain power supply bids that in-

clude these errors, the error level is multiplied with the perfect foresight power infeed (see

also Section 3.3.1.2 in Deliverable 4.1) which can be extracted from historical time series

data. Figure 12 shows the incidence of actual power forecast error levels obtained within

AMIRIS. The normal distribution of the error levels is clearly visible. The positive mean of

the distribution (12) corresponds to an average overestimation of renewable power. Due

to the variance of the distribution (13), however, also negative error values occur, reflect-

ing a lower-than-actual feed-in estimate.

(12)

(13)

Figure 12. Histogram of power forecast error levels created in AMIRIS following a normal distri-

bution with a mean of 0.05 and a standard deviation of 0.1; 8760 hourly data points representing

one year.

Page 31 of 51

The power forecast errors are created during the bid preparation stage of AMIRIS and

propagate through the simulation. Therefore, these errors can impact the day-ahead mar-

ket clearing price, traders’ profits and system costs as well as other subsequent markets

(e.g., intra-day, ancillary services) which, however, are not explicitly modelled in AMIRIS.

Figure 13 demonstrates the possible impact of power forecast errors on the day-ahead

electricity market clearing price using the same error distribution as before. For most sit-

uations, prices found with erroneous forecasts are below the “perfect foresight” prices that

do not contain any forecast errors. This matches the expectation since an on-average

higher renewable feed-in should lead to lower prices due to the merit-order effect [82].

Figure 13. Sample impact of power forecast errors on (non-realistic) DAM clearing prices; black

curve represents prices without power forecast errors, red dots resemble prices that include modi-

fied renewable feed-in estimates based on the same error distribution function as shown in Figure

12.

This simple example highlights the possible impact of power forecast errors on the

market prices. However, it must be noted that several aspects are not yet satisfyingly re-

flected. For instance, real-world forecast errors might depend on the specific type of tech-

nology and the hour of the day. In addition, the errors shown here have no autocorrela-

tion, while real-world error series often have autocorrelative features. Thus, to obtain a

more realistic time series of errors, correlations should also be considered.

Page 32 of 51

4. Forecast approaches developed in TradeRES

project

In this section, some preliminary results are presented as a drive for the first version of

TradeRES vRES power forecast tools. Specifications of the market players/agents that

will benefit from these forecasts are also presented. Detailed results will be presented in

the deliverables from WP 5.

4.1 Preliminary results

4.1.1. Wind, solar, small hydro and electricity demand forecast

The potential benefit of changing the day-ahead market closure gate to an hour near

the time real operation was analysed for different technologies (wind, solar PV, small hy-

dro) and for electricity demand. This benefit is usually referred as the “certainty gain ef-

fect” and it represents the potential economic surplus that the market players can obtain

by building their offers in the day-ahead market with a highest level of certainty on power

forecast and, consequently, with low-risk exposure [83].

From a forecasting point of view, the motivation for the DAM change is related to the

availability of the IBC conditions used to feed the NWP models that are crucial for fore-

casting systems for the considered time horizon. The work conducted so far intends to

understand which market players can benefit most from this change in the market design.

On the other hand, and since the forecasts are obtained from a partner who is a forecast

provider (including for some players in the Iberian electricity market), it allows to identify

which technologies/players have more challenges to participate in electricity market and,

therefore, require further developments regarding the forecast approaches during the

TradeRES project.

Method and data

The approach followed is based on the work presented in [4]. Although, in this case,

the approach is extended to other technologies and to electricity demand players. The first

step was the identification of representative days. Representative days are a widely used

statistical approach to detect the most typical daily patterns of a dataset under analysis.

This approach, based on a statistical clustering technique, allows, at the same time, to

group days that exhibit identical patterns. With this procedure, it is possible to feed the

agent-based electricity models (in this case the Multi-Agent TRading in Electricity Markets

- MATREM) and identify the profiles that can jeopardize the income of different players /

market agents enabling the adoption of measures to mitigate their exposure to risk. By

applying the K-medois clustering [4] to the Portuguese aggregated wind, solar PV, and

small-hydro power production, nine typical profiles were identified, Figure 14.

Page 33 of 51

a)

b)

c)

Figure 14. Daily average profiles for a) wind, b) solar PV and c) small hydro for the nine clusters

(profiles) for Portugal.

For the days that correspond to the medoids of each cluster, the forecast using differ-

ent IBC was attained. The forecast is based on a combination of machine learning ap-

proaches (ANN and KNN) that uses an input a combination of weather sources from the

ECMWF and GFS global models. Different models use different weather variables depend

on technology. Usually, the models are trained using the last 2 historical years, and when

data is available, we create a model by site. At the end of the process, an aggregated

model is used to combine all sites for each technology. Models run every 6 hours using

the best weather data available. Some of these models consider the last real values col-

lected from the SCADA to improve the very short-term forecast.

Metrics of market performance

The following metrics are used to compare the results between the base and the up-

graded forecast scenarios. Considering each scenario, the total remuneration of each

player, , from the markets is computed as follows:

(14)

where is the period number, is the DAM price, , is the up deviation cost,

is the down deviation cost. is the parcel that consists in

0

10

20

30

40

50

60

70

80

90

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Capacity factor (%)

Hour

0

10

20

30

40

50

60

70

80

90

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Capacity factor (%)

Hour

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Capacity factor (%)

Hour

0

10

20

30

40

50

60

70

80

90

12345678910 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Capacity factor (%)

Hour

Profile 1

Profile 2

Profile 3

Profile 4

Profile 5

Profile 6

Profile 7

Profile 8

Profile 9

Page 34 of 51

the remuneration obtained from the DAM. The other parcel of the equation represents the

remuneration obtained from the deviations. Therefore, considering that scenario 0 is the

current scenario, the remuneration gain effect, , of rolling the gate closure is computed

as follows:

(15)

The cost,, paid in each scenario due to the penalties (difference between the forecast

and observed values) is computed as follows:

(16)

Scenarios and synthesis of the main results

To quantify the certainty gain effect for different players with the existing electricity

market design and products, the MATREM simulator was feed step-by-step using the

scenarios presented in Table 5.

Table 5. Scenarios analysed to quantify the certainty gain effect for different electricity

market players.

Scenario

Description

1

Wind forecast and observed values for other generation technologies + load

2

Solar forecast and observed values for other generation technologies + load

3

Small hydro forecast and observed values for other generation technologies + load

4

Load forecast + observed for generation technologies

The lowest forecast errors were observed for small hydro and load players. Preliminary

results show that the certainty gain effect of using updated IBCs is relatively small. Table

6 showns the results for the scenario 1 using the forecast data at 6 UTC as a benchmark.

In the case of a wind power producer, the reduction in the RMSE reaches nearly 1%. The

total energy deviation for the representative days analysed reduces 1.47% and 2.17%

comparing with the use of IBC from global models provided at 06:00 UTC. Regarding the

remuneration, an increase of 5.76% is expected when the IBC from 12:00 UTC is used.

For the IBC from the 18:00, the increase in the remuneration only increases 1.68%. Re-

sults will be further analysed in WP 5, in specific, D5.3- Performance assessment of cur-

rent and new market designs and trading mechanisms for National and Regional Markets.

Page 35 of 51

Table 6. Results for scenario 1 (wind power forecast) in relation to the DAM forecast at

06:00 (UTC).

Metrics

IBC forecast data

Forecast at (UTC)

12:00

18:00

RMSE (%)

-0.77

-1.06

Energy deviation (%)

-1.47

-2.17

Penalties to DAM schedule (%)

-2.53

-5.52

Total remuneration (%)

+5.67

+1.68

4.1.2. Power forecast with feature selection

As discussed in the previous sections, and supported by several authors (e.g., [84]),

the vRES power forecast accuracy for short and medium time horizons strongly relies on

the outcomes of NWP models. The main error in the final forecast comes from the mete-

orological input rather than the existing statistical techniques applied in power forecast

systems [84]. For instance, using one source of meteorology for wind speed forecast the

mean absolute percentage error (MAPE) is approximately 15%. Using this wind speed as

input, MAPE for wind power forecast is approximately 23%. However, using the same

algorithm with another source of meteorology, with wind speed MAPE of 24%, the MAPE

error for power forecast has a substantial increase, around 45%. A generic graphical rep-

resentation of a generation modern wind turbine power curve, which presents typically a

cubic dependency of wind speed for the range between 4-11 m/s. For these wind speed

ranges, we can intuitively understand that an absolute error of 1 m/s in wind speed can

represent a high error value in power forecast. However, the full potential of the output of

these models was not fully explored. For instance, most of vRES power system uses a

single point information from the NWP grid and a limited number of meteorological pa-

rameters.

For the specific case of the single point outputs, the variability in generation observed

in a given wind or solar power plant depends not only on the local dynamics, but it is also

influenced by large-scale atmospheric patterns [73]. Although these patterns may be well

simulated, in a specific location, the time series may present deviations [85]. Thus, anoth-

er feature of the NWP that is not usually explored in forecasting systems is the use of the

results of a spatial grid in contrast to the use of only data from a single spatial point or the

midpoints of the NWP domain surrounding a wind power plant/ solar PV [86]. In [73], a

methodology that combines a gradient boosting trees algorithm with feature engineering

techniques, aiming to extract the maximum spatial and temporal information from the

NWP grid to improve wind and solar power, was implemented. The authors identified that

the use of PCA enables to improve the wind power forecast accuracy. Thus, the achieved

results indicate that an adequate extraction of features from the raw data of the NWP can

improve the forecast systems. The authors recommend more investment in the data min-

ing phase as well as the application of statistical downscaling techniques capable of in-

corporating all data.

Page 36 of 51

Regarding the meteorological parameters extracted from NWP, recent works have

shown that a careful selection of input variables for statistical methods can improve the

accuracy of the wind and solar power forecasts [46], [49]. In the case of wind power, the

most common meteorological parameters used as input to feed the downscaling ap-

proaches are the wind speed and direction. The influence of parameters related to the

conversion efficiency as air temperature and pressure are included in some works. In [85]

the authors included parameters from the upper levels of the atmosphere (e.g., the 850

and 500 hPa pressure levels) as input to improve the performance of the statistical mod-

els. Using a PCA, in the work conducted by [87], the following parameters were identified

as the most relevant to forecast the wind power variability: mean sea level pressure, geo-

potential height, and the meridional wind component and humidity relative. A physics-

oriented pre-processing with a NWP feature selection approach had a positive impact on

the model performance from the team that won the European Energy Market Conference

competition [88]. In [46], the author identified that a possible development trend to im-

prove the power forecast systems is to include exogenous meteorological input variables.

For solar power, [45] identified that parameters as precipitation intensity and wind speed

penalized the performance of the forecast. On the other hand, parameters as ultraviolet

index, wind bearing, and dew point could allow to improve the solar power forecast.

Against this background, in TradeRES, a method was implemented to identify if the in-

clusion of meteorological parameters derived directly from a NWP and others with impact

in wind and solar power generation behaviour. These meteorological parameters include

both, surface (e.g., latent flux) and vertical levels information.

Method and data

The method implemented uses a greedy sequential forward feature (SFF) selection al-

gorithm [49] to select iteratively the meteorological features (based on the principal com-

ponents scores), which minimizes an objective function, in this case, the root mean

square error. The SFF starts with an empty set of features. Then, at each iteration, it ex-

tends the previous set with the feature that allows to minimize a pre-determined objective

function (OF). The RMSE was used as an objective function in this case study.

The ANN approach implemented in Matlab toolbox [89] was used to provide the power

forecast due to the features described in section 2.2.2. Estimating with the ANN involves

two main steps: training and learning. One of the most effective learning algorithms in

ANN is the backpropagation algorithm [90]–[92]. The backpropagation algorithm uses

supervised learning to adjust the weights and biases in each unit [90]. The process of

training the network is the adjustment of the network weights to produce the desired re-

sponse to the given inputs. Consequently, the training begins with random weights that

iteratively are adjusted so that the error (the difference between estimated and expected

results) based on inputs and outputs data will always be minimized to an acceptable val-

ue. The goal of the backpropagation algorithm is to reduce this error until the ANN learns

the training data [90]. The following parameters were imposed during the ANN setup:

a) the number of hidden layers - In this case, it was considered only one hidden layer;

Page 37 of 51

b) different numbers of units in each layer. There is no predefined rule and therefore

a rule-of-thumb for determining the correct number was followed. In this case, the

number of hidden units is always 2/3 the size of the input layer, plus the size of the

output layer [93];

c) different types of transfer functions. In this case, a sigmoid function was consid-

ered for each unit in the hidden layer;

d) learning algorithm. In this case, the Levenberg-Marquardt algorithm was adopted

[91], [93].

As a baseline to assess the benefit of the SFF selection algorithm, an ANN approach

was feed with the meteorological parameters similar to the ones used in [73] azimuthal

wind speed, meridional wind speed and wind speed module. For each wind park ana-

lysed, the data is obtained for the nearest grid point of NWP. Data from the WRF model

provided by MeteoGalicia

3

are used. These data are available, free of charge, and contain

several meteorological historical forecast parameters such as wind speed components

and gust, convective available potential energy, cloud cover at high levels, surface down-

welling shortwave flux and visibility.

The two following case studies were also analysed: i) use the PCA approach without

applying the feature selection algorithm, and ii) single point data with the feature selection

algorithm.

The method was applied to seven wind parks in Portugal located in regions with differ-

ent climatic conditions (coastal and mountains regions) to understand the consistency of

the meteorological features along the different regions.

Synthesis of the main results

The average impact for the seven locations analysed different configurations of the

wind forecast power approach is depicted in Figure 15.

By applying the PCA and the feature selection algorithm the RMSE can reduce nearly

25% comparing to the benchmarking approach (single-point forecast with limited number

of meteorological parameters), Figure 15. If only PCA is applied, the results show a reduc-

tion in the RMSE values of nearly 8%. Although, the use of PCA improves the forecast, an

insightful selection of the meteorological features is paramount to reduce the uncertainty

in the wind power forecasts. Parameters as wind gust, wind power density, wind shear,

and planetary boundary layer should be used to improve the wind power forecast.

3

Data available at: http://mandeo.meteogalicia.es/thredds/catalogos/DATOS/ARCHIVE/WRF

/WRF_hist.html (accessed on 02 November 2020).

Page 38 of 51

Figure 15. Average RMSE improvements for the seven wind parks analysed compared with the

benchmarking approach. “PCA-WithoutSFF” – PCA approach without applying SFF algorithm;

“NWPPoint+SFF” – data from NWP was extracted to the nearest point of each wind park and the

SFF was applied; “PCA+SFF” – PCA approach and application of the SFF algorithm.

Despite the improvements observed, it is noted a tendency to underestimate the days

with a high level of wind power production. On contrary, an overestimation of the observed

wind power production is observed for days with low level of wind power production.

4.1.3. TradeRES - vRES power forecast approach

Based on the preliminary results, the main steps of the vRES power forecast method

applied in TradeRES are presented in Figure 16.

Figure 16. Power forecast approach implemented in TradeRES project.

0

5

10

15

20

25

PCA-WithoutSFF NWPPoint+SFF PCA+SFF

RMSE improvements regarding

benchmarking (%)

Forecast system configuration

Forward feature selection for each Weather Regime

Meteorological

forecast data PCA Analysis Pearson

correlation

Forecast statistical

approach

Meteo

feature

retained

OF

Improv.?

Ranking PCs

Y

N

PCs

Weather Regime

Classification

Page 39 of 51

The forecasting approach is also based on a SFF algorithm but, in this case, a calibra-

tion procedure according to the different weather regimes is performed. The classification

of atmospheric circulation states into distinct types is a common approach used for under-

standing and scrutinizing weather patterns and their impact on a predetermined parame-

ter. For instance, in [94] the weather regimes were used to estimate Europe-wide wind

power generation. Thus, the goal is to improve the drawbacks identified in the previous

subsection by obtaining a forecast configuration for similar weather conditions. As related

by several authors [84], the weather conditions have a strong impact on the wind power

variability as well as in the uncertainty in its forecast. This can be partly explained by the

weather conditions that unleash different responses, e.g., heating and cooling between

land/sea surfaces, and thermal stratification [47], [95], [96]. The performance of solar

power forecast systems depends on the cloud coverage, which can be distinguished using

weather regimes.

To accomplish the specification requested by some market players, probabilistic out-

puts will be provided with this approach. One of the objectives of this approach is to ena-

ble bidding strategically of market players by identifying the most adequate quantile (e.g.,

the quantile that maximizes the revenue in the day-ahead market) [97].

Below, further details of each step in the methodology implemented are provided:

Meteorological data: The NWP data will be provided from the two weather sources

the ECMWF and GFS global models.

Several meteorological parameters from the NWP will be tested to identify meteoro-

logical features that enable to improve the wind or solar power forecast accuracy.

New variables such as mean seal level gradient or atmospheric instability [47] to ac-

count for the energy conversion processes will be computed.

PCA analysis: After obtaining the data, a PCA analysis is implemented for each me-

teorological parameter. Only the PCs that explain 90% of the total variability are re-

tained since the remaining PCs only describe local effects.

Pearson coefficient correlation: is then applied to identify the correlation among the

PCs and the wind/solar PV power (or the combination of both) and the PCs are

ranking in a descending way. In parallel with this process, a weather regime classifi-

cation is computed.

Weather regime classification: each forecast day will be classified into a specific WR

using a Lamb-type approach [50]. This approach uses the mean sea level pressure

from 16 grid points to identify twenty-six dissimilar weather patterns. Two of the

weather patterns are classified as pure low-pressure system or anticyclonic, eight

are defined as directional – according to the wind rose (N, NE, E, SE, S, SW, W and

NW), and the remaining sixteen are defined as hybrid.

FSS for each weather regime: It is a greedy algorithm that chooses the “most attrac-

tive” solution in each iteration. In this case, the SFS attempts to find the “optimal”

feature subset by selecting, iteratively, the meteorological PC that improves reduces

the RMSE value. Sensitivity tests will be implemented for each case study and tech-

nology to identify the most adequate OF. The OF will depend on the perspective: i)

for market players as wind and solar power producers (or vRES aggregator), the

RMSE and electricity market revenue (including day-ahead and imbalances) will be

Page 40 of 51

tested and compared, and ii) for the TSOs, the RMSE will be used. The ANN will be

the statistical approach implemented. A quantile spline regression technique will be

applied to obtain the probabilistic forecasts [98].

The power forecast approach will be tested in the regional cases studies defined in WP

5, and the benefit from this approach will be discussed as an outcome of WP 5.

Output: Wind or solar power probabilistic forecast with 15- and 60-minutes time resolu-

tion, according to the needs of the different players. When deterministic forecast is re-

quired, the quantile that minimizes the RMSE or maximizes the producers’ revenues will

be applied.

Agent-based models that will benefit from these data: Wind and solar power pro-

ducers, aggregators/virtual power plants, and TSO.

4.2 Wind power ramping forecast

As the share of wind and solar PV increases in most of the power systems, ramping

alert tools are being implemented by some TSOs [7], [99]. The goal of such tool is to

complement the existing deterministic or probabilistic forecast systems enabling to in-

crease the level of situational awareness available to the TSO by helping them to better

scale the level of risk that exists in the system. This risk can then be managed by taking

into consideration additional factors, such as potential changes in energy consumption,

additional reserves that can be deployed, and additional generators that may be available

for unit commitment. Furthermore, players capable to provide temporal and sectoral flexi-

bility can also take advantage of this information to strategically participate into electricity

markets.

The characterization and definition of wind power ramps are linked to the notion of an

“event that is critical enough to deserve special attention” [50]. In specific, ramp events

consist of a rapid and substantial change in the wind power during a time interval .

Since no clear definition is available in the literature to classify power ramps, the definition

(17) and principles used in [9] will be followed in TradeRES project as a “first-guess”.

(17)

where, t denotes the time and is the reference value. For these parameters, the

values identified in [9] will be used.

Understanding power ramps events is not an easy task as the weather conditions are

rarely the same for different wind parks. In fact, even when two wind parks are placed in

similar latitudes, these triggering mechanisms can be very different due to local effects as

the terrain characteristics, roughness and topography or phenomena like sea/land breez-

es [74]. Recent works, e.g., [9] state that, in order to understand and forecast the dynam-

ics of wind power ramps, holistic methodologies should be used to account for the spatial

and temporal evolution of atmospheric large-scale circulation. In this sense, in their work,

the authors implemented a windstorm detection algorithm and compared the performance

Page 41 of 51

with a common cyclone detection algorithm [100], [101]. Windstorm algorithm presented a

highest performance. Nevertheless, some issues were identified in the current windstorm

detection methodologies. The most critical one is that a wind power ramp is not always a

consequence or is always linked to the existence of extreme wind speed values, being

essentially dependent from the previous (historical) state of the flow. Moreover, these al-

gorithms are unable to distinguish upward from downward power ramps. For that reason,

information from the previous time step ("memory effect") needs to be included in this type

of fast ramping tool. In the next section, an algorithm that uses a time numerical differenti-

ation in order to fit the particular case of wind power ramps events is described.

4.2.1. Ramp detection algorithm

This algorithm is based on the forecast mean sea level pressure from the NWP. In or-

der to be able to identify areas where there is the highest variation in the meteorological

field, the pressure gradient was calculated as follows:

(18)

where p is the average pressure at sea level, Long the longitude and Lat the latitude.

Next, and in order to introduce a “memory effect”, the derivative in time of the pressure

gradient is calculated according to the following expression:

(19)

The remaining detection algorithm is equal to the windstorm algorithm presented in [9].

Therefore, the major differences between the two methodologies are the following as-

pects: i) use of pressure data, ensuring better identification of the synoptic centres [9]; ii)

identification of extreme events associated with positive/negative power changes in time,

enabling a better relationship with the wind power ramp events. In this sense, it is consid-

ered that the events with negative variations are those with a change in the pressure gra-

dient of less than the 2nd percentile. On the other hand, the positive events are identified

as regions with a variation above the 98th percentile in the pressure gradient.

In the case of upward ramps, the algorithm starts to determine the grid points where

the pressure gradient is above a certain percentile. The spatial percentile calculation is

based on the following formula [9]:

(20)

where, p represents the percentile considered and stands for the cumulative distribu-

tion function weighted by the cosine of the latitude of {W (Long, Lat, t):(Long, Lat)δ} be-

ing δ the spatial domain [102]. For downward power ramps, the 2nd percentile is consid-

ered, and the search is for grid points where the pressure gradient is below this value.

Then, contiguous grid points for which the percentile condition occurs are enclosed into

the same candidate [9]. A convex hull approximation is employed in this step to identify

the convex polygon comprising all the spatial grid points that can belong to the same me-

Page 42 of 51

teorological event (see black line in Figure 17). After, the average geometric center of

each event is computed (magenta “*” symbols in Figure 17). As outcome, this spatial

search algorithm provides a list of the possible location of events associated with synoptic

systems. Only events with a minimum area of 150 000 km2 [103] are considered. This

step is performed for each temporal time-step.

Figure 17. Example of one event in time t (black line) and one event in time t+1 (green line). The

magenta “*” symbols represent the average geometric center of each candidate, while the magenta

line indicates the trajectory of the meteorological event.

Once the synoptic events are identified in the time step t, it becomes necessary to

stitch to the nearest candidate at the time t+1 to build the trajectory. The following as-

sumptions are imposed in this step [9]:

1. The maximum Euclidean distance between the centers of two consecutive time-

steps is 720 km [103];

2. Only events with a lifetime above 2h or with a maximum speed of 120 km/h are re-

tained.

All events with no continuity are eliminated, and when two or more candidates are

found, a cost function is applied in order to determine the most appropriate trajectory. The

cost function applied is similar to the one shown by [104], which is expressed by:

(21)

where, are the coordinates of the center for a determined synoptic event at time step t,

are the coordinates of the center for the jth synoptic event at time step t+1, is the

intensity observed at the geometric center of the event at time-step t and is the

intensity observed of the geometric center for the jth synoptic event at time-step t+1.

Page 43 of 51

At the end, the algorithm retains a tracking table with the different trajectories of the ex-

treme events detected and some basic characteristics, e.g., their lifetime, occurrence

dates, speed, area of influence. In real-time operation, an alert will be issued when these

events are nearby the region under analysis.

The power ramp power forecast accuracy will be assessed and analysed in the region-

al cases from WP 5.

4.2.2. A nested forecast approach

Based on the probabilistic and power ramp detection algorithm, a nested forecast ap-

proach will be established aiming to reduce the system cost associated with less commit-

ted reserves. Thus, the power reserves’ allocation can be dynamically established and

dependent on the probability of existence (or not) of wind ramps. Other market players as

flexibility providers or wind power producers can also benefit from this nested approach

since it can allow for strategic participation in electricity markets. An example of the out-

comes of this tool is shown in Figure 18.

Figure 18. Example of the outcomes from the nested forecast approach.

Output: Hourly binary information regarding the occurrence (or not) of wind power

ramps.

Agent-based models that will benefit from these data: All TradeRES agent-based

models may benefit from these data by incorporating it in TSO agent capabilities or in flex-

ibility provider and wind power producers’ agent behaviour, using this information to stra-

tegically participate in the electricity markets.

Page 44 of 51

5. Final remarks

In this report, a non-extensive review of the different forecasting methods was present-

ed focusing on the technologies under analysis in the TradeRES project. This review

served as the basis for framing the advantages and disadvantages of the different fore-

cast approaches commonly applied in the energy sector. Moreover, it allowed to identify

the synergies among the different approaches and existing electricity market time-frames.

Based on this background and preliminary results from TradeRES, a nested forecasting

approach was presented to feed specific market players.

Preliminary results were presented highlighting that current power forecast approaches

already allow to have a significant level of accuracy in forecasting electricity demand and

in small hydro power plants. However, for the time-frame on which this report is focused

(day-ahead market) wind and solar PV technologies (or vRES aggregator) continue to

show significant errors, even assuming a postponing of their bids to an hour closer to real-

time. To improve the existing forecast systems, a new approach is proposed in

TradeRES. Results showed that the use of numerical weather prediction (NWP) grid data

coupled with a feature selection algorithm could enable to improve the forecast perfor-

mance, when compared with a forecast based only a NWP single-point data. Preliminary

results also show that meteorological parameters as wind gust (traditionally not consid-

ered in the wind power forecast systems) enable to reduce the wind power forecast errors.

In this deliverable, some values and principles were defined as “first-guess” based on a

literature review. Due to the iterative nature of the project, it is expected that some values

can be further improved in the second version of this deliverable at Month 41 based on

the outcomes of the remaining work packages. It should be noted that the focus of this

deliverable was the day-ahead market, which is one of the most important in the current

electricity markets designs and detailed results will be presented in work package 5. In the

second version of deliverable 4.9, the work is expected to focus on the very-short and

short time horizons - in line with the new market electricity designs/products that are being

proposed in work package 3 of the project.

Page 45 of 51

References

[1] S. Hanifi, X. Liu, Z. Lin, and S. Lotfian, “A Critical Review of Wind Power Forecasting

Methods—Past, Present and Future,” Energies, vol. 13, no. 15, p. 3764, Jul. 2020, doi:

10.3390/en13153764.

[2] C. Croonenbroeck and G. Stadtmann, “Renewable generation forecast studies – Review

and good practice guidance,” Renew. Sustain. Energy Rev., vol. 108, no. June 2018, pp.

312–322, Jul. 2019, doi: 10.1016/j.rser.2019.03.029.

[3] H. Holttinen et al., “Methodologies to determine operating reserves due to increased wind

power,” IEEE Trans. Sustain. Energy, vol. 3, no. 4, pp. 713–723, Oct. 2012, doi:

10.1109/TSTE.2012.2208207.

[4] H. Algarvio, A. Couto, F. Lopes, and A. Estanqueiro, “Changing the Day-Ahead Gate

Closure to Wind Power Integration: A Simulation-Based Study,” Energies, vol. 12, no. 14, p.

2765, Jul. 2019, doi: 10.3390/en12142765.

[5] L. de Vries et al., “D3.5 - Market design for a reliable ~ 100 % renewable electricity system,”

TradeRES project deliverable. p. 62, 2021 [Online]. Available: https://traderes.eu/wp-

content/uploads/2021/04/D3.5_MarketDesignOptions_H2020.pdf

[6] C. Köhler et al., “Critical weather situations for renewable energies – Part B: Low stratus risk

for solar power,” Renew. Energy, vol. 101, pp. 794–803, Feb. 2017, doi:

10.1016/j.renene.2016.09.002.

[7] A. Steiner et al., “Critical weather situations for renewable energies – Part A: Cyclone

detection for wind power,” Renew. Energy, vol. 101, pp. 41–50, Feb. 2017, doi:

10.1016/j.renene.2016.08.013.

[8] C. Gallego-Castillo, E. Garcia-Bustamante, A. Cuerva, and J. Navarro, “Identifying wind

power ramp causes from multivariate datasets: a methodological proposal and its

application to reanalysis data,” IET Renew. Power Gener., vol. 9, no. 8, pp. 867–875, 2015,

doi: 10.1049/iet-rpg.2014.0457.

[9] M. Lacerda, A. Couto, and A. Estanqueiro, “Wind Power Ramps Driven by Windstorms and

Cyclones,” Energies, vol. 10, no. 10, p. 1475, Sep. 2017, doi: 10.3390/en10101475.

[10] M. S. H. Lipu et al., “Artificial Intelligence Based Hybrid Forecasting Approaches for Wind

Power Generation: Progress, Challenges and Prospects,” IEEE Access, vol. 9, pp. 102460–

102489, 2021, doi: 10.1109/ACCESS.2021.3097102.

[11] A. Al Mamun, M. Sohel, N. Mohammad, M. S. Haque Sunny, D. R. Dipta, and E. Hossain,

“A Comprehensive Review of the Load Forecasting Techniques Using Single and Hybrid

Predictive Models,” IEEE Access, vol. 8, pp. 134911–134939, 2020, doi:

10.1109/ACCESS.2020.3010702.

[12] I. K. Nti, M. Teimeh, O. Nyarko-Boateng, and A. F. Adekoya, “Electricity load forecasting: a

systematic review,” J. Electr. Syst. Inf. Technol., vol. 7, no. 1, p. 13, Dec. 2020, doi:

10.1186/s43067-020-00021-8.

[13] D. C. Montgomery, C. L. Jennings, and M. Kulahci, Introduction to Time Series Analysis and

Forecasting, 2nd ed. New Jersey: JohnWiley & Sons, Inc., 2015.

[14] C. Chatfield, Time-Series Forecasting. Florida: Chapman and Hall/CRC; 6th edition (July

31, 2003), 2000.

[15] C. Kuster, Y. Rezgui, and M. Mourshed, “Electrical load forecasting models: A critical

systematic review,” Sustain. Cities Soc., vol. 35, no. August, pp. 257–270, 2017, doi:

10.1016/j.scs.2017.08.009.

[16] Y. H. Hsiao, “Household Electricity Demand Forecast Based on Context Information and

User Daily Schedule Analysis From Meter Data,” IEEE Trans. Ind. Informatics, vol. 11, no.

1, pp. 33–43, 2015, doi: 10.1109/TII.2014.2363584.

[17] B. Yang et al., “State-of-the-art one-stop handbook on wind forecasting technologies: An

overview of classifications, methodologies, and analysis,” J. Clean. Prod., vol. 283, p.

Page 46 of 51

124628, 2021, doi: 10.1016/j.jclepro.2020.124628.

[18] J. Widén, N. Carpman, V. Castellucci, D. Lingfors, J. Olauson, and et al., “Variability

Assessment and Forecasting of Renewables: A Review for Solar, Wind, Wave and Tidal

Resources,” Renew. Sustain. energy Rev., vol. 44, pp. 356–375, 2015.

[19] C. Monteiro, R. Bessa, V. Miranda, A. Botterud, J. Wang, and G. Conzelmann, “Wind Power

Forecasting: State-of-the-art 2009,” 2009 [Online]. Available:

https://publications.anl.gov/anlpubs/2009/11/65613.pdf

[20] T. Haiden et al., “Evaluation of ECMWF forecast, including 2021 upgrade,” ECMWF

technical memo 884. p. 56, 2021 [Online]. Available:

https://www.ecmwf.int/sites/default/files/elibrary/2021/20142-evaluation-ecmwf-forecsts-

including-2021-upgrade.pdf

[21] C. Sweeney, R. J. Bessa, J. Browell, and P. Pinson, “The future of forecasting for

renewable energy,” WIREs Energy Environ., vol. 9, no. 2, Mar. 2020, doi:

https://doi.org/10.1002/wene.365.

[22] Portal do Clima, “Portal do Clima - Climate Change in Portugal,” Programa ADAPT

Alterações Climáticas em Portugal, 2021. [Online]. Available:

http://portaldoclima.pt/en/project/methodology/climate-models/. [Accessed: 01-Sep-2021]

[23] A. Couto, L. Rodrigues, P. Costa, J. Silva, and A. Estanqueiro, “Wind power participation in

electricity markets - The role of wind power forecasts,” in EEEIC 2016 - International

Conference on Environment and Electrical Engineering, 2016, p. 6, doi:

10.1109/EEEIC.2016.7555453.

[24] R. Marujo, P. Costa, M. Fernandes, T. Simoes, and A. Estanqueiro, “Offshore wind field:

Application of statistical models as a spatial validation technique,” in European Offshore

Wind Conference & Exhibition, 2011, p. 8 [Online]. Available:

https://repositorio.lneg.pt/bitstream/10400.9/1501/1/EWEA_Amsterdam_RMarujo_

PCosta_MFernandes_AEstanqueiro.pdf

[25] J. Silva, F. Marques da Silva, A. Couto, and A. Estanqueiro, “A method to correct the flow

distortion of offshore wind data using CFD simulation and experimental wind tunnel tests,”

J. Wind Eng. Ind. Aerodyn., vol. 140, 2015, doi: 10.1016/j.jweia.2015.01.017.

[26] J. M. L. M. Palma, F. A. Castro, L. F. Ribeiro, A. H. Rodrigues, and A. P. Pinto, “Linear and

nonlinear models in wind resource assessment and wind turbine micro-siting in complex

terrain,” J. Wind Eng. Ind. Aerodyn., vol. 96, no. 12, pp. 2308–2326, 2008, doi:

10.1016/j.jweia.2008.03.012.

[27] C. G. Nunalee, C. Meißner, A. Vignaroli, N. Carolina, and R. Nc, “Downscaling MERRA

mesoscale data for the generation microscale wind fields using CFD,” in Spain: Europe’s

Premier Wind Energy Event, 2014, p. 1.

[28] C. Schmitt, C. Meissner, and D. Weir, “Influence Of Thermal Stratification on CFD

Simulations,” Europe’s Premier Wind Energy Event poster. EWEA, Brussels, p. 1, 2011

[Online]. Available: https://windsim.com/Windsimarchive/library -- papers-

presentations/Influence Of Thermal Stratification on CFD Simulations.pdf

[29] F. Castellani, D. Astolfi, M. Mana, M. Burlando, C. Meißner, and E. Piccioni, “Wind Power

Forecasting techniques in complex terrain: ANN vs. ANN-CFD hybrid approach,” J. Phys.

Conf. Ser., vol. 753, no. 8, p. 082002, Sep. 2016, doi: 10.1088/1742-6596/753/8/082002.

[Online]. Available:

https://repositorio.lneg.pt/bitstream/10400.9/1501/1/EWEA_Amsterdam_RMarujo_

PCosta_MFernandes_AEstanqueiro.pdf

[30] M. Lydia, S. S. Kumar, A. I. Selvakumar, and G. E. Prem Kumar, “A comprehensive review

on wind turbine power curve modeling techniques,” Renew. Sustain. Energy Rev., vol. 30,

pp. 452–460, Feb. 2014, doi: 10.1016/j.rser.2013.10.030.

[31] E. Yun and J. Hur, “Probabilistic estimation model of power curve to enhance power output

forecasting of wind generating resources,” Energy, vol. 223, p. 120000, May 2021, doi:

10.1016/j.energy.2021.120000.

Page 47 of 51

[32] M. Yang, C. Shi, and H. Liu, “Day-ahead wind power forecasting based on the clustering of

equivalent power curves,” Energy, vol. 218, p. 119515, Mar. 2021, doi:

10.1016/j.energy.2020.119515.

[33] R. Blaga, A. Sabadus, N. Stefu, C. Dughir, M. Paulescu, and V. Badescu, “A current

perspective on the accuracy of incoming solar energy forecasting,” Prog. Energy Combust.

Sci., vol. 70, pp. 119–144, 2019, doi: 10.1016/j.pecs.2018.10.003.

[34] R. A. Rajagukguk and R. Kamil, “A Deep Learning Model to Forecast Solar Irradiance Using

a Sky Camera,” Appl. Sci., vol. 11, no. 5049, p. 15, 2021, doi:

doi.org/10.3390/app11115049.

[35] S. Dev, F. M. Savoy, Y. H. Lee, and S. Winkler, “Estimating solar irradiance using sky

imagers,” Atmos. meas. Tech, vol. 12, pp. 5417–5429, 2019, doi:

https://doi.org/10.5194/amt-12-5417-2019.

[36] J. Antonanzas, N. Osorio, R. Escobar, R. Urraca, F. J. Martinez-de-pison, and F.

Antonanzas-torres, “Review of photovoltaic power forecasting,” Sol. Energy, vol. 136, pp.

78–111, 2016, doi: 10.1016/j.solener.2016.06.069.

[37] M. N. Akhter, S. Mekhilef, H. Mokhlis, and N. Mohamed Shah, “Review on forecasting of

photovoltaic power generation based on machine learning and metaheuristic techniques,”

IET Renew. Power Gener., vol. 13, no. 7, pp. 1009–1023, May 2019, doi: 10.1049/iet-

rpg.2018.5649.

[38] S. Atique, S. Noureen, V. Roy, V. Subburaj, S. Bayne, and J. Macfie, “Forecasting of total

daily solar energy generation using ARIMA: A case study,” in 2019 IEEE 9th Annual

Computing and Communication Workshop and Conference (CCWC), 2019, pp. 0114–0119,

doi: 10.1109/CCWC.2019.8666481.

[39] A. M. Foley, P. G. Leahy, A. Marvuglia, and E. J. McKeogh, “Current methods and

advances in forecasting of wind power generation,” Renew. Energy, vol. 37, no. 1, pp. 1–8,

Jan. 2012, doi: 10.1016/j.renene.2011.05.033.

[40] H. Liu, C. Chen, X. Lv, X. Wu, and M. Liu, “Deterministic wind energy forecasting: A review

of intelligent predictors and auxiliary methods,” Energy Convers. Manag., vol. 195, no. May,

pp. 328–345, 2019, doi: 10.1016/j.enconman.2019.05.020.

[41] E. Vinagre, T. Pinto, S. Ramos, Z. Vale, and J. M. Corchado, “Electrical Energy

Consumption Forecast Using Support Vector Machines,” in Proceedings - International

Workshop on Database and Expert Systems Applications, DEXA, 2017, pp. 171–175, doi:

10.1109/DEXA.2016.046.

[42] G. Dudek, “Neural networks for pattern-based short-term load forecasting: A comparative

study,” Neurocomputing, vol. 205, pp. 64–74, 2016, doi: 10.1016/j.neucom.2016.04.021.

[43] M. Q. Raza and A. Khosravi, “A review on artificial intelligence based load demand

forecasting techniques for smart grid and buildings,” Renew. Sustain. Energy Rev., vol. 50,

pp. 1352–1372, 2015, doi: 10.1016/j.rser.2015.04.065.

[44] K. Amasyali and N. M. El-gohary, “A review of data-driven building energy consumption

prediction studies,” Renew. Sustain. Energy Rev., vol. 81, no. 2018, pp. 1192–1205, 2018,

doi: 10.1016/j.rser.2017.04.095.

[45] M. Castangia, A. Aliberti, L. Bottaccioli, E. Macii, and E. Patti, “A compound of feature

selection techniques to improve solar radiation forecasting,” Expert Syst. Appl., vol. 178, no.

March, p. 114979, Sep. 2021, doi: 10.1016/j.eswa.2021.114979.

[46] H. Liu and C. Chen, “Data processing strategies in wind energy forecasting models and

applications: A comprehensive review,” Appl. Energy, vol. 249, no. May, pp. 392–408, 2019,

doi: 10.1016/j.apenergy.2019.04.188.

[47] A. Couto, P. Costa, L. Rodrigues, V. V. Lopes, and A. Estanqueiro, “Impact of Weather

Regimes on the Wind Power Ramp Forecast in Portugal,” IEEE Trans. Sustain. Energy, vol.

6, no. 3, pp. 934–942, 2015, doi: 10.1109/TSTE.2014.2334062.

[48] G. Chandrashekar and F. Sahin, “A survey on feature selection methods,” Comput. Electr.

Eng., vol. 40, no. 1, pp. 16–28, 2014, doi: 10.1016/j.compeleceng.2013.11.024.

Page 48 of 51

[49] S. Salcedo-Sanz, L. Cornejo-Bueno, L. Prieto, D. Paredes, and R. García-Herrera, “Feature

selection in machine learning prediction systems for renewable energy applications,”

Renew. Sustain. Energy Rev., vol. 90, pp. 728–741, Jul. 2018, doi:

10.1016/j.rser.2018.04.008.

[50] A. Couto, P. Costa, and T. Simões, “Identification of Extreme Wind Events Using a Weather

Type Classification,” Energies, vol. 14, no. 3944, p. 16, Jul. 2021, doi:

10.3390/en14133944.

[51] H. C. Bloomfield, D. J. Brayshaw, and A. J. Charlton-Perez, “Characterizing the winter

meteorological drivers of the European electricity system using targeted circulation types,”

Meteorol. Appl., no. November 2019, pp. 1–18, 2019, doi: 10.1002/met.1858.

[52] A. Couto and A. Estanqueiro, “Assessment of wind and solar PV local complementarity for

the hybridization of the wind power plants installed in Portugal,” J. Clean. Prod., vol. 319,

no. August, p. 128728, Oct. 2021, doi: 10.1016/j.jclepro.2021.128728.

[53] F. Cassola and M. Burlando, “Wind speed and wind energy forecast through Kalman

filtering of Numerical Weather Prediction model output,” Appl. Energy, vol. 99, pp. 154–166,

2012, doi: 10.1016/j.apenergy.2012.03.054.

[54] I. González-Aparicio and A. Zucker, “Impact of wind power uncertainty forecasting on the

market integration of wind energy in Spain,” Appl. Energy, vol. 159, pp. 334–349, Dec.

2015, doi: 10.1016/j.apenergy.2015.08.104.

[55] P. Pinson, N. Siebert, and G. Kariniotakis, “Forecasting of regional wind generation by a

dynamic fuzzy-neural networks based upscaling approach,” Eur. Wind Energy Conf. Exhib.

EWEC, 2003 [Online]. Available: http://hal.archives-ouvertes.fr/hal-00530550/

[56] Z. Liu, W. Gao, and N. Carolina, “Wind Power Plant Prediction by Using Neural Networks,”

in IEEE Energy Conversion Conference and Exposition, 2012, p. 7, doi:

10.1109/ECCE.2012.6342351.

[57] I. Okumus and A. Dinler, “Current status of wind energy forecasting and a hybrid method for

hourly predictions,” Energy Convers. Manag., vol. 123, pp. 362–371, 2016, doi:

10.1016/j.enconman.2016.06.053.

[58] S. Alessandrini, F. Davò, S. Sperati, M. Benini, and L. Delle Monache, “Comparison of the

economic impact of different wind power forecast systems for producers,” Adv. Sci. Res.,

vol. 11, pp. 49–53, May 2014, doi: 10.5194/asr-11-49-2014.

[59] J. Zhang, a Florita, B. Hodge, and J. Freedman, “Ramp Forecasting Performance from

Improved Short-Term Wind Power Forecasting,” ASME 2014 Int. Des. Eng. Conf. Comput.

Inf. Eng. Conf., no. May, 2014, doi: 10.1115/DETC2014-34775.

[60] C. Junk, L. Delle Monache, S. Alessandrini, G. Cervone, and L. von Bremen, “Predictor-

weighting strategies for probabilistic wind power forecasting with an analog ensemble,”

Meteorol. Zeitschrift, vol. 24, no. 4, pp. 361–379, 2015, doi: 10.1127/metz/2015/0659.

[61] T. Jónsson, P. Pinson, H. Madsen, and H. Nielsen, “Predictive Densities for Day-Ahead

Electricity Prices Using Time-Adaptive Quantile Regression,” Energies, vol. 7, no. 9, pp.

5523–5547, 2014, doi: 10.3390/en7095523.

[62] M. Sun, C. Feng, and J. Zhang, “Probabilistic solar power forecasting based on weather

scenario generation,” Appl. Energy, vol. 266, p. 114823, May 2020, doi:

10.1016/j.apenergy.2020.114823.

[63] B. Li and J. Zhang, “A review on the integration of probabilistic solar forecasting in power

systems,” Sol. Energy, vol. 210, pp. 68–86, Nov. 2020, doi: 10.1016/j.solener.2020.07.066.

[64] Z. Zhou et al., “Application of probabilistic wind power forecasting in electricity markets,”

Wind Energy, vol. 16, pp. 321–338, 2013, doi: 10.1002/we.

[65] H. Holttinen, J. J. Miettinen, A. Couto, H. Algarvio, L. Rodrigues, and A. Estanqueiro, “Wind

power producers in shorter gate closure markets and balancing markets,” in 3th

International Conference on the European Energy Market (EEM), 2016, p. 5, doi:

10.1109/EEM.2016.7521309.

[66] R. J. Bessa, C. Moreira, B. Silva, and M. Matos, “Handling renewable energy variability and

Page 49 of 51

uncertainty in power systems operation,” Wiley Interdiscip. Rev. Energy Environ., vol. 3, no.

2, pp. 156–178, May 2014, doi: 10.1002/wene.76.

[67] R. Bessa et al., “Towards Improved Understanding of the Applicability of Uncertainty

Forecasts in the Electric Power Industry,” Energies, vol. 10, no. 9, pp. 1402–1450, 2017,

doi: 10.3390/en10091402.

[68] N. J. Cutler, H. R. Outhred, I. F. MacGill, and J. D. Kepert, “Predicting and presenting

plausible future scenarios of wind power production from numerical weather prediction

systems: A qualitative ex ante evaluation for decision making,” Wind Energy, vol. 15, no. 3,

pp. 473–488, 2012, doi: 10.1002/we.485.

[69] R. J. Bessa, V. Miranda, A. Botterud, J. Wang, and E. M. Constantinescu, “Time adaptive

conditional kernel density estimation for wind power forecasting,” IEEE Trans. Sustain.

Energy, vol. 3, no. 4, pp. 660–669, 2012, doi: 10.1109/TSTE.2012.2200302.

[70] P. Li, X. Guan, and J. Wu, “Aggregated wind power generation probabilistic forecasting

based on particle filter,” Energy Convers. Manag., vol. 96, pp. 579–587, 2015, doi:

10.1016/j.enconman.2015.03.021.

[71] L. Delle Monache, F. A. Eckel, D. L. Rife, B. Nagarajan, and K. Searight, “Probabilistic

Weather Prediction with an Analog Ensemble,” Mon. Weather Rev., vol. 141, no. 10, pp.

3498–3516, Oct. 2013, doi: 10.1175/MWR-D-12-00281.1.

[72] F. Davò, S. Alessandrini, S. Sperati, L. Delle Monache, D. Airoldi, and M. T. Vespucci,

“Post-processing techniques and principal component analysis for regional wind power and

solar irradiance forecasting,” Sol. Energy, vol. 134, pp. 327–338, 2016, doi:

10.1016/j.solener.2016.04.049.

[73] J. R. Andrade and R. J. Bessa, “Improving Renewable Energy Forecasting With a Grid of

Numerical Weather Predictions,” IEEE Trans. Sustain. Energy, vol. 8, no. 4, pp. 1571–1580,

Oct. 2017, doi: 10.1109/TSTE.2017.2694340.

[74] C. Gallego-castillo, A. Cuerva-tejero, and O. Lopez-garcia, “A review on the recent history

of wind power ramp forecasting,” Renew. Sustain. Energy Rev., vol. 52, pp. 1148–1157,

2015, doi: 10.1016/j.rser.2015.07.154.

[75] T. Ouyang, X. Zha, and L. Qin, “A Survey of Wind Power Ramp Forecasting,” Energy Power

Eng., vol. 05, no. 04, pp. 368–372, 2013, doi: 10.4236/epe.2013.54B071.

[76] N. Tartaglione, “Relationship between Precipitation Forecast Errors and Skill Scores of

Dichotomous Forecasts,” Weather Forecast., vol. 25, no. 1, pp. 355–365, Feb. 2010, doi:

10.1175/2009WAF2222211.1.

[77] A. Estanqueiro and A. Couto, “Chapter 1 - New electricity markets. The challenges of

variable renewable energy,” in Local Electricity Markets, T. Pinto, Z. Vale, and S.

Widergren, Eds. Academic Press, 2021, pp. 3–20.

[78] B. Bochenek et al., “Day-Ahead Wind Power Forecasting in Poland Based on Numerical

Weather Prediction,” Energies, vol. 14, no. 8, p. 2164, Apr. 2021, doi: 10.3390/en14082164.

[79] SOLARGIS, “How Solargis is improving accuracy of solar power forecasts,” Best practises,

2017. [Online]. Available: https://solargis.com/blog/best-practices/improving-accuracy-of-

solar-power-forecasts

[80] C. Schimeczek et al., “D4.1 - Temporal flexibility options in electricity market simulation

models,” TradeRES project deliverable. p. 58, 2021 [Online]. Available:

https://traderes.eu/wp-

content/uploads/2021/10/D4.1_TemporalFlexibilityOptions_H2020.pdf

[81] EPEX Spot, “Trading at EPEX SPOT 2021,” Technical brochure. p. 8, 2021 [Online].

Available: https://www.epexspot.com/sites/default/files/download_center_files/21-03-

09_Trading Brochure.pdf

[82] F. Sensfuß, M. Ragwitz, and M. Genoese, “The merit-order effect: A detailed analysis of the

price effect of renewable electricity generation on spot market prices in Germany,” Energy

Policy, vol. 36, no. 8, pp. 3086–3094, Aug. 2008, doi: 10.1016/j.enpol.2008.03.035.

[83] J. M. Morales, A. J. Conejo, and J. Perez-Ruiz, “Short-Term Trading for a Wind Power

Page 50 of 51

Producer,” IEEE Trans. Power Syst., vol. 25, no. 1, pp. 554–564, Feb. 2010, doi:

10.1109/TPWRS.2009.2036810.

[84] G. Giebel, R. Brownsword, G. Kariniotakis, M. Denhard, and C. Draxl, “The State-Of-The-

Art in Short-Term Prediction of Wind Power A Literature Overview,” in Technical Report

ANEMOS.plus, 2011, p. 109 [Online]. Available:

http://ecolo.org/documents/documents_in_english/wind-predict-ANEMOS.pdf

[85] N. Ellis, R. Davy, and A. Troccoli, “Predicting wind power variability events using different

statistical methods driven by regional atmospheric model output,” Wind Energy, vol. 18, no.

9, pp. 1611–1628, Sep. 2015, doi: 10.1002/we.1779.

[86] T. T. Warner, “Quality assurance in atmospheric modeling,” Am. Meteorol. Soc., vol. 92, no.

12, pp. 1601–1610, 2011, doi: 10.1175/BAMS-D-10-00054.1.

[87] N. Ellis, R. Davy, and A. Troccoli, “Predicting wind power variability events using different

statistical methods driven by regional atmospheric model output,” Wind Energy, vol. 18, no.

9, pp. 1611–1628, Sep. 2015, doi: 10.1002/we.1779.

[88] K. Bellinguer, V. Mahler, S. Camal, and G. Kariniotakis, “Probabilistic Forecasting of

Regional Wind Power Generation for the EEM20 Competition: a Physics-oriented Machine

Learning Approach,” in 2020 17th International Conference on the European Energy Market

(EEM), 2020, p. 6, doi: 10.1109/EEM49802.2020.9221960.

[89] “MATLAB and Neural Network Toolbox Release 2012b, The MathWorks, Inc., Natick,

Massachusetts, United States.” .

[90] J. P. S. Catalão, H. M. I. Pousinho, and V. M. F. Mendes, “Short-term wind power

forecasting in Portugal by neural networks and wavelet transform,” Renew. Energy, vol. 36,

no. 4, pp. 1245–1251, Apr. 2011, doi: 10.1016/j.renene.2010.09.016.

[91] G. Li and J. Shi, “On comparing three artificial neural networks for wind speed forecasting,”

Appl. Energy, vol. 87, no. 7, pp. 2313–2320, Jul. 2010, doi:

10.1016/j.apenergy.2009.12.013.

[92] M. G. V. Marroquín, M. C. A. Cervantes, J. L. M. Flores, and M. Cabrera-Ríos, “Time series:

Empirical characterization and artificial neural network-based selection of forecasting

techniques,” Intell. Data Anal., vol. 13, no. 6, pp. 969–982, 2009, doi: 10.3233/IDA-2009-

0403.

[93] M. T. Hagan, H. B. Demuth, M. H. Beale, and O. De Jesús, Neural Network Design (2nd

Edition). Oklahoma State University (ebook), 2014.

[94] B. U. Schyska, A. Couto, L. von Bremen, A. Estanqueiro, and D. Heinemann, “Weather

dependent estimation of continent-wide wind power generation based on spatio-temporal

clustering,” Adv. Sci. Res., vol. 14, no. 2003, pp. 131–138, May 2017, doi: 10.5194/asr-14-

131-2017.

[95] C. Draxl, “On the Predictability of Hub Height Winds,” PhdThesis, DTU Wind Energy (Risø-

PhDM No 84(EN)), p. 105, 2012 [Online]. Available:

https://backend.orbit.dtu.dk/ws/portalfiles/portal/10246050/Ris_PhD_84.pdf

[96] M. Ohba, S. Kadokura, and D. Nohara, “Impacts of synoptic circulation patterns on wind

power ramp events in East Japan,” Renew. Energy, vol. 96, pp. 591–602, 2016, doi:

10.1016/j.renene.2016.05.032.

[97] G. Strbac et al., “Decarbonization of Electricity Systems in Europe: Market Design

Challenges,” IEEE Power Energy Mag., vol. 19, no. 1, pp. 53–63, Jan. 2021, doi:

10.1109/MPE.2020.3033397.

[98] H. A. Nielsen, H. Madsen, and T. S. Nielsen, “Using quantile regression to extend an

existing wind power forecasting system with probabilistic forecasts,” Wind Energy, vol. 9,

pp. 95–108, Jan. 2006, doi: 10.1002/we.180.

[99] K. D. Orwig et al., “Recent Trends in Variable Generation Forecasting and Its Value to the

Power System,” IEEE Trans. Sustain. Energy, vol. 6, no. 3, pp. 924–933, Jul. 2015, doi:

10.1109/TSTE.2014.2366118.

[100] R. J. Murray and I. Simmonds, “A numerical scheme for tracking cyclone centres from

Page 51 of 51

digital data. Part I: development and operation of the scheme,” Aust. Met. Mag., vol. 39, pp.

155–166, 1991 [Online]. Available: http://www.bom.gov.au/jshess/docs/1991/murray1.pdf

[101] R. J. Murray and I. Simmonds, “A numerical scheme for tracking cyclone centres from

digital data. Part II:Application to January and July general circulation model simulations.,”

Aust. Met. Mag., vol. 39, pp. 167–180, 1991 [Online]. Available:

http://www.bom.gov.au/jshess/docs/1991/murray2.pdf

[102] P. M. Della-Marta, H. Mathis, C. Frei, M. A. Liniger, J. Kleinn, and C. Appenzeller, “The

return period of wind storms over Europe,” Int. J. Climatol., vol. 29, no. 3, pp. 437–459, Mar.

2009, doi: 10.1002/joc.1794.

[103] D. Renggli, “Seasonal predictability of wintertime windstorm climate over the North Atlantic

and Europe,” Dissertation Freie Universität Berlin. p. 151, 2011 [Online]. Available:

https://refubium.fu-berlin.de/handle/fub188/901

[104] E. Flaounas, V. Kotroni, K. Lagouvardos, and I. Flaounas, “Tracking winter extra-tropical

cyclones based on their relative vorticity evolution and sensitivity to prior data filtering

(cycloTRACK v1.0),” Geosci. Model Dev. Discuss., vol. 7, no. 1, pp. 1245–1276, 2014, doi:

10.5194/gmdd-7-1245-2014.