Content uploaded by Juan R. Trapero
Author content
All content in this area was uploaded by Juan R. Trapero on Jun 22, 2016
Content may be subject to copyright.
Motivation
Case study
Models
Experimental results
Conclusions
Optimal combination of volatility forecasts to
enhance solar irradiation prediction intervals
estimation
Juan R. Trapero 1Alberto Martin 2
1Universidad de Castilla-La Mancha
2Agencia Estatal de Meteorolog
´
ıa, AEMET
June 2016
Juan R. Trapero , Alberto Martin 1/16
Motivation
Case study
Models
Experimental results
Conclusions
Why is it important?
•Short-term forecasts are required to optimize operational
planning of solar-power plants
Forecast errors involve significant costs
Reduction of 47.6 % of penalty costs by improving the forecasting
technique in a Concentrating solar power plant (Kraas et al., 2013)
•Although stakeholders also require a measure of uncertainty
(prediction interval), most of the works are focused on point
forecasting
•In addition, forecast error is assumed to be normally i.i.d.
Juan R. Trapero , Alberto Martin 3/16
Motivation
Case study
Models
Experimental results
Conclusions
Why is it important?
•Short-term forecasts are required to optimize operational
planning of solar-power plants
Forecast errors involve significant costs
Reduction of 47.6 % of penalty costs by improving the forecasting
technique in a Concentrating solar power plant (Kraas et al., 2013)
•Although stakeholders also require a measure of uncertainty
(prediction interval), most of the works are focused on point
forecasting
•In addition, forecast error is assumed to be normally i.i.d.
Juan R. Trapero , Alberto Martin 3/16
Motivation
Case study
Models
Experimental results
Conclusions
Why is it important?
•Short-term forecasts are required to optimize operational
planning of solar-power plants
Forecast errors involve significant costs
Reduction of 47.6 % of penalty costs by improving the forecasting
technique in a Concentrating solar power plant (Kraas et al., 2013)
•Although stakeholders also require a measure of uncertainty
(prediction interval), most of the works are focused on point
forecasting
•In addition, forecast error is assumed to be normally i.i.d.
Juan R. Trapero , Alberto Martin 3/16
Motivation
Case study
Models
Experimental results
Conclusions
Why is it important?
•Short-term forecasts are required to optimize operational
planning of solar-power plants
Forecast errors involve significant costs
Reduction of 47.6 % of penalty costs by improving the forecasting
technique in a Concentrating solar power plant (Kraas et al., 2013)
•Although stakeholders also require a measure of uncertainty
(prediction interval), most of the works are focused on point
forecasting
•In addition, forecast error is assumed to be normally i.i.d.
Juan R. Trapero , Alberto Martin 3/16
Motivation
Case study
Models
Experimental results
Conclusions
Research objectives
•Extend solar energy forecasting research beyond point
forecasting.
•Explore non-parametric approaches as Kernel density
estimators if forecast errors are not normal.
•Explore time-varying parametric volatility estimators (SES and
GARCH) if forecast errors are not independent.
•Explore combination methods if any of the assumptions is
hold.
Idea
“Optimal” combination based on maximizing conditional coverage
Christoffersen test p-value.
Juan R. Trapero , Alberto Martin 4/16
Motivation
Case study
Models
Experimental results
Conclusions
Research objectives
•Extend solar energy forecasting research beyond point
forecasting.
•Explore non-parametric approaches as Kernel density
estimators if forecast errors are not normal.
•Explore time-varying parametric volatility estimators (SES and
GARCH) if forecast errors are not independent.
•Explore combination methods if any of the assumptions is
hold.
Idea
“Optimal” combination based on maximizing conditional coverage
Christoffersen test p-value.
Juan R. Trapero , Alberto Martin 4/16
Motivation
Case study
Models
Experimental results
Conclusions
Research objectives
•Extend solar energy forecasting research beyond point
forecasting.
•Explore non-parametric approaches as Kernel density
estimators if forecast errors are not normal.
•Explore time-varying parametric volatility estimators (SES and
GARCH) if forecast errors are not independent.
•Explore combination methods if any of the assumptions is
hold.
Idea
“Optimal” combination based on maximizing conditional coverage
Christoffersen test p-value.
Juan R. Trapero , Alberto Martin 4/16
Motivation
Case study
Models
Experimental results
Conclusions
Research objectives
•Extend solar energy forecasting research beyond point
forecasting.
•Explore non-parametric approaches as Kernel density
estimators if forecast errors are not normal.
•Explore time-varying parametric volatility estimators (SES and
GARCH) if forecast errors are not independent.
•Explore combination methods if any of the assumptions is
hold.
Idea
“Optimal” combination based on maximizing conditional coverage
Christoffersen test p-value.
Juan R. Trapero , Alberto Martin 4/16
Motivation
Case study
Models
Experimental results
Conclusions
Case study
•We are going to focus on one-step-ahead uncertainty forecasts
obtained from Global Horizontal Irradiation (GHI)
•This study can be extended to analyse the Direct Normal
Irradiation (DNI)
•Spanish Institute of Concentrated-Photovoltaic Systems
(ISFOC),
•1,1 MW of
Concentrated-Photovoltaic
Energy (CPV)
•Hourly series:
(01/2011)-(12/2011)
Juan R. Trapero , Alberto Martin 5/16
Motivation
Case study
Models
Experimental results
Conclusions
Case study
8400 8450 8500 8550 8600 8650 8700
0
100
200
300
400
500
Wh/m2
4200 4250 4300 4350 4400 4450 4500
0
200
400
600
800
1000
Wh/m2
Hours
Example of hourly solar irradiation for GHI in winter (upper plot) and in
summer (lower plot).
Juan R. Trapero , Alberto Martin 6/16
Motivation
Case study
Models
Experimental results
Conclusions
Literature review
•Prediction intervals.
•Theoretical: [Lt+h,Ut+h] = Ft+h±z(1−p)/2·σt+h
•Lt+h: lower interval
•Ut+h: upper interval
•Ft+h: point forecast
•h: forecasting horizon
•z(1−p)/2: standard normal distribution table for a certain
confidence level 100p%
•σt+h:is the theoretical standard deviation determined by the h
and the model forecasting parameters
Forecasting hazard!
If the model is misspecified, the theoretical prediction intervals tend to be
too narrow
Juan R. Trapero , Alberto Martin 7/16
Motivation
Case study
Models
Experimental results
Conclusions
Literature review
•Prediction intervals.
•Theoretical: [Lt+h,Ut+h] = Ft+h±z(1−p)/2·σt+h
•Lt+h: lower interval
•Ut+h: upper interval
•Ft+h: point forecast
•h: forecasting horizon
•z(1−p)/2: standard normal distribution table for a certain
confidence level 100p%
•σt+h:is the theoretical standard deviation determined by the h
and the model forecasting parameters
Forecasting hazard!
If the model is misspecified, the theoretical prediction intervals tend to be
too narrow
Juan R. Trapero , Alberto Martin 7/16
Motivation
Case study
Models
Experimental results
Conclusions
Literature review
•Prediction intervals: If there are doubts about the validity of
the “true” model, go empirical
•Parametric (Normal): [Lt+h,Ut+h] = Ft+h±z(1−p)/2·σt+h
•σ2
t+h=Pt−1
i=t−n(i+h)2
n
•i+h: forecasting error at i+h
•Non-parametric:
[Lt+h,Ut+h]=[Ft+h+Qh((1 −p)/2),Ft+h+Qh((1 + p)/2)]
•Qh(p): 100p% forecast error quantile
Bonus track :)
Deterministic (Numerical Weather Prediction) models are widely used. As they
do not rely on a statistical model, the are well-suited for empirical approaches.
Juan R. Trapero , Alberto Martin 8/16
Motivation
Case study
Models
Experimental results
Conclusions
Literature review
•Prediction intervals: If there are doubts about the validity of
the “true” model, go empirical
•Parametric (Normal): [Lt+h,Ut+h] = Ft+h±z(1−p)/2·σt+h
•σ2
t+h=Pt−1
i=t−n(i+h)2
n
•i+h: forecasting error at i+h
•Non-parametric:
[Lt+h,Ut+h]=[Ft+h+Qh((1 −p)/2),Ft+h+Qh((1 + p)/2)]
•Qh(p): 100p% forecast error quantile
Bonus track :)
Deterministic (Numerical Weather Prediction) models are widely used. As they
do not rely on a statistical model, the are well-suited for empirical approaches.
Juan R. Trapero , Alberto Martin 8/16
Motivation
Case study
Models
Experimental results
Conclusions
Benchmarks
•Point Forecast (Ft+1): ARIMA(1,0,0) ×(1,1,0)24 (Reikard,
2009)
•Empirical Quantile forecast (Q1(p))
•Parametric-Normal:Q1(p) = z(1−p)/2·σt+1
•Single Exponential Smoothing: σ2
t+1 =a2
t+ (1 −a)σ2
t
•GARCH(1,1): σ2
t+1 =ω+a12
t+β1σ2
t
•Non-parametric: Kernel Density estimator
f(x) = 1
Nh PN
j=1 Kx−Xj
h
•f(x) is probability density function of the forecast errors
•Nis the sample size
•K(·) is the kernel smoothing function
•his the bandwidth.
Juan R. Trapero , Alberto Martin 9/16
Motivation
Case study
Models
Experimental results
Conclusions
Benchmarks
•Point Forecast (Ft+1): ARIMA(1,0,0) ×(1,1,0)24 (Reikard,
2009)
•Empirical Quantile forecast (Q1(p))
•Parametric-Normal:Q1(p) = z(1−p)/2·σt+1
•Single Exponential Smoothing: σ2
t+1 =a2
t+ (1 −a)σ2
t
•GARCH(1,1): σ2
t+1 =ω+a12
t+β1σ2
t
•Non-parametric: Kernel Density estimator
f(x) = 1
Nh PN
j=1 Kx−Xj
h
•f(x) is probability density function of the forecast errors
•Nis the sample size
•K(·) is the kernel smoothing function
•his the bandwidth.
Juan R. Trapero , Alberto Martin 9/16
Motivation
Case study
Models
Experimental results
Conclusions
Benchmarks
•Point Forecast (Ft+1): ARIMA(1,0,0) ×(1,1,0)24 (Reikard,
2009)
•Empirical Quantile forecast (Q1(p))
•Parametric-Normal:Q1(p) = z(1−p)/2·σt+1
•Single Exponential Smoothing: σ2
t+1 =a2
t+ (1 −a)σ2
t
•GARCH(1,1): σ2
t+1 =ω+a12
t+β1σ2
t
•Non-parametric: Kernel Density estimator
f(x) = 1
Nh PN
j=1 Kx−Xj
h
•f(x) is probability density function of the forecast errors
•Nis the sample size
•K(·) is the kernel smoothing function
•his the bandwidth.
Juan R. Trapero , Alberto Martin 9/16
Motivation
Case study
Models
Experimental results
Conclusions
Experimental setup
•Data (8,760 observations) have been split down in two parts.
•First part (4,392 observations)
•to estimate ARIMA-GARCH model (in-sample data)
•With nights (seasonality=24 hours)
•Maximum likelihood based on a t-distribution with the
Econometrics toolbox from MATLAB
•to estimate SES parameter
•Using ARIMA forecasts
•Without nights
•Minimisation of the sum of one-step-ahead prediction errors
•Second part (4,368 observations) as out-of-sample data.
Rolling origin experiment.
•SES slightly outperforms GARCH (RMSE of the variance)
Juan R. Trapero , Alberto Martin 10/16
Motivation
Case study
Models
Experimental results
Conclusions
Experimental setup
•Data (8,760 observations) have been split down in two parts.
•First part (4,392 observations)
•to estimate ARIMA-GARCH model (in-sample data)
•With nights (seasonality=24 hours)
•Maximum likelihood based on a t-distribution with the
Econometrics toolbox from MATLAB
•to estimate SES parameter
•Using ARIMA forecasts
•Without nights
•Minimisation of the sum of one-step-ahead prediction errors
•Second part (4,368 observations) as out-of-sample data.
Rolling origin experiment.
•SES slightly outperforms GARCH (RMSE of the variance)
Juan R. Trapero , Alberto Martin 10/16
Motivation
Case study
Models
Experimental results
Conclusions
Experimental setup
•Data (8,760 observations) have been split down in two parts.
•First part (4,392 observations)
•to estimate ARIMA-GARCH model (in-sample data)
•With nights (seasonality=24 hours)
•Maximum likelihood based on a t-distribution with the
Econometrics toolbox from MATLAB
•to estimate SES parameter
•Using ARIMA forecasts
•Without nights
•Minimisation of the sum of one-step-ahead prediction errors
•Second part (4,368 observations) as out-of-sample data.
Rolling origin experiment.
•SES slightly outperforms GARCH (RMSE of the variance)
Juan R. Trapero , Alberto Martin 10/16
Motivation
Case study
Models
Experimental results
Conclusions
Prediction intervals performance
•Comparison prediction intervals performance
•prediction interval coverage: proportion of times that a
forecast is included within the prediction interval (hit rate)
•average interval width: interval range divided by its midpoint
•Conditional coverage test: (Christoffersen, 1998) as a
combination of tests for unconditional coverage and
independence
In summary
An ideal method should provide a close prediction interval coverage
with regards to the desired confidence level, a low average interval
width and pass the conditional coverage test.
Juan R. Trapero , Alberto Martin 11/16
Motivation
Case study
Models
Experimental results
Conclusions
Prediction intervals performance
•Comparison prediction intervals performance
•prediction interval coverage: proportion of times that a
forecast is included within the prediction interval (hit rate)
•average interval width: interval range divided by its midpoint
•Conditional coverage test: (Christoffersen, 1998) as a
combination of tests for unconditional coverage and
independence
In summary
An ideal method should provide a close prediction interval coverage
with regards to the desired confidence level, a low average interval
width and pass the conditional coverage test.
Juan R. Trapero , Alberto Martin 11/16
Motivation
Case study
Models
Experimental results
Conclusions
Results
Method p( %) Hit Rate ( %) LRuLRiLRcAv. width
KERNEL 80 84.13 0.06 0.00 0.00 1.33
85 89.21 0.03 0.00 0.00 1.46
90 94.60 0.00 0.00 0.00 1.60
95 97.14 0.06 0.24 0.09 1.74
SES 80 73.02 0.00 0.16 0.00 0.86
85 78.10 0.00 0.55 0.00 0.93
90 83.81 0.00 0.77 0.00 1.00
95 87.94 0.00 0.75 0.00 1.09
•Likelihood ratio tests p-values for unconditional coverage (LRu),
independence (LRi) and conditional coverage (LRc)
•P-values lower than 0.05 mean that the null hypothesis of unconditional
coverage, independence and conditional coverage, respectively, are
rejected at the 5 % significance level.
Juan R. Trapero , Alberto Martin 12/16
Motivation
Case study
Models
Experimental results
Conclusions
Results
•KERNEL provides a higher hit rate, although it does not pass
the independence test.
•SES passes the independence test, although it does not pass
the unconditional coverage test.
•In conclusion, none of them pass the conditional coverage test.
•Why don’t we combine them?
•Combined prediction interval [ˆ
Lc,ˆ
Uc] :
ˆ
Lc=w·ˆ
LKernel + (1 −w)·ˆ
LSES
ˆ
Uc=w·ˆ
UKernel + (1 −w)·ˆ
USES
where 0 <w<1 is a constant that maximizes the LRc
Juan R. Trapero , Alberto Martin 13/16
Motivation
Case study
Models
Experimental results
Conclusions
Results
•KERNEL provides a higher hit rate, although it does not pass
the independence test.
•SES passes the independence test, although it does not pass
the unconditional coverage test.
•In conclusion, none of them pass the conditional coverage test.
•Why don’t we combine them?
•Combined prediction interval [ˆ
Lc,ˆ
Uc] :
ˆ
Lc=w·ˆ
LKernel + (1 −w)·ˆ
LSES
ˆ
Uc=w·ˆ
UKernel + (1 −w)·ˆ
USES
where 0 <w<1 is a constant that maximizes the LRc
Juan R. Trapero , Alberto Martin 13/16
Motivation
Case study
Models
Experimental results
Conclusions
Results
Method p( %) Hit Rate ( %) LRuLRiLRcAv. width
KERNEL 80 84.13 0.06 0.00 0.00 1.33
85 89.21 0.03 0.00 0.00 1.46
90 94.60 0.00 0.00 0.00 1.60
95 97.14 0.06 0.24 0.09 1.74
SES 80 73.02 0.00 0.16 0.00 0.86
85 78.10 0.00 0.55 0.00 0.93
90 83.81 0.00 0.77 0.00 1.00
95 87.94 0.00 0.75 0.00 1.09
COMBINED 80 78.41 0.47 0.09 0.18 0.92
85 83.49 0.44 0.09 0.17 0.99
90 89.84 0.91 0.66 0.90 1.15
95 95.24 0.86 0.74 0.93 1.32
Juan R. Trapero , Alberto Martin 14/16
Motivation
Case study
Models
Experimental results
Conclusions
Conclusions
•Kernel (non-parametric) provide a high prediction interval
coverage but it also yields a high average interval width and
lack of independence.
•SES (parametric) pass the independence test but they offer a
prediction interval coverage excessively low.
•This work proposes a novel approach that combines a
non-parametric and a parametric approach.
•The combination weight is obtained by maximizing the
Christoffersen conditional coverage test p-value.
•The results show a good compromise between coverage and
average interval width.
Juan R. Trapero , Alberto Martin 15/16
Motivation
Case study
Models
Experimental results
Conclusions
Thank you for your attention!
e-mail: juanramon.trapero@uclm.es
blog: http://blog.uclm.es/juanramontrapero/
This work has been supported by the Spanish Ministerio de Econom
´
ıa y
Competitividad, under Research Grant no. DPI2015-64133-R
(MINECO/FEDER/UE)
Juan R. Trapero , Alberto Martin 16/16