Available via license: CC BY 4.0

Content may be subject to copyright.

sensors

Article

Probabilistic Load Forecasting for Building

Energy Models

Eva Lucas Segarra †, Germán Ramos Ruiz *,† and Carlos Fernández Bandera

School of Architecture, University of Navarra, 31009 Pamplona, Spain; elucas@unav.es (E.L.S.);

cfbandera@unav.es (C.F.B.)

*Correspondence: gramrui@unav.es; Tel.: +34-948-425-600 (ext. 802751)

† These authors contributed equally to this work.

Received: 12 October 2020; Accepted: 12 November 2020 ; Published: 15 November 2020

Abstract:

In the current energy context of intelligent buildings and smart grids, the use of load

forecasting to predict future building energy performance is becoming increasingly relevant.

The prediction accuracy is directly inﬂuenced by input uncertainties such as the weather forecast,

and its impact must be considered. Traditional load forecasting provides a single expected value for

the predicted load and cannot properly incorporate the effect of these uncertainties. This research

presents a methodology that calculates the probabilistic load forecast while accounting for the

inherent uncertainty in forecast weather data. In the recent years, the probabilistic load forecasting

approach has increased in importance in the literature but it is mostly focused on black-box models

which do not allow performance evaluation of speciﬁc components of envelope, HVAC systems, etc.

This research ﬁlls this gap using a white-box model, a building energy model (BEM) developed in

EnergyPlus, to provide the probabilistic load forecast. Through a Gaussian kernel density estimation

(KDE), the procedure converts the point load forecast provided by the BEM into a probabilistic load

forecast based on historical data, which is provided by the building’s indoor and outdoor monitoring

system. An hourly map of the uncertainty of the load forecast due to the weather forecast is generated

with different prediction intervals. The map provides an overview of different prediction intervals for

each hour, along with the probability that the load forecast error is less than a certain value. This map

can then be applied to the forecast load that is provided by the BEM by applying the prediction

intervals with their associated probabilities to its outputs. The methodology was implemented and

evaluated in a real school building in Denmark. The results show that the percentage of the real

values that are covered by the prediction intervals for the testing month is greater than the conﬁdence

level (80%), even when a small amount of data are used for the creation of the uncertainty map;

therefore, the proposed method is appropriate for predicting the probabilistic expected error in load

forecasting due to the use of weather forecast data.

Keywords:

probabilistic load forecasting; white-box models; building energy models; weather

forecast; uncertainty analysis; monitoring; reliability

1. Introduction

In the era of the Internet of Things (IoT), virtual and physical environments are being closely

linked and widely used in various areas of the industry. Among these areas, building operation and

management is receiving more attention due to the growing development of intelligent buildings

and smart grids [

1

,

2

]. Smart buildings integrate connected objects through systems that monitor

and control a great variety of variables, such as indoor temperature, weather data, airﬂow rates and

CO

2

concentration. Sensors are fundamental devices in this new generation of buildings because

they connect the simulation model with the real world and they enable appropriate management

Sensors 2020,20, 6525; doi:10.3390/s20226525 www.mdpi.com/journal/sensors

Sensors 2020,20, 6525 2 of 20

and adequate decision making [

3

]. In this ﬁeld, the energy efﬁciency of buildings is one of the most

important research areas: buildings represent almost 40% of the world’s total energy consumption

and thus hold great energy-saving potential [

4

]. Load forecasting is one of the key elements of this

new intelligent building and smart grid environment, where network solutions are used to optimize

energy sources.

Accurate load forecasts for buildings allow the optimum management of buildings’ energy

systems and low-voltage networks in different contexts, such as energy management systems [

5

,

6

],

energy storage system control [

7

], demand response (DR) and demand-side management (DSM) [

8

]

and the integration of distributed energy resources [9].

The prediction of building energy is key for the optimization of its management, and it falls into

three general categories in the literature [

10

]: black-box or data-driven models, white-box or physical

models and gray-box or hybrid models. Black-box models are mathematical models constructed

from historical data and lack explicit link between model inputs and physical building parameters.

In recent years, several reviews articles have studied the growing use of this type of prediction models

for building energy prediction [

11

–

13

]. Following the classiﬁcation proposed in the most recent

review from Sun et al. [

13

], data driven approaches can be divided into statistical, which derive

correlations between the variable of study and inﬂuential parameters, such as linear regression (LR) or

time series analysis (ARMA and ARIMA) [

14

,

15

]; and machine learning (ML) approaches, which are

a more advanced statistical methods and use prediction algorithms. ML includes, among others,

support vector machine (SVM) [

16

], ensemble methods [

17

], deep learning [

5

] and the increasing and

most common in the recent literature [

13

,

18

], artiﬁcial neural networks (ANN) [

19

–

21

]. White-box or

physical models are based on physical principles and predict loads with detailed heat and mass transfer

equations using simulation software such as EnergyPlus and TRNSYS. These software packages

calculate building energy prediction based on building construction details, heating, ventilation and

air conditioning (HVAC) design information, operation schedules and climate information

[22–24].

Finally, gray-box or hybrid models, which are a combination of data-driven and physics-based

models, use simpliﬁed physical descriptions but also require parameter estimation based on measured

data [25,26].

Among these building energy prediction models, black-box-based models lack an understanding

of the underlying parameters of the energy prediction and its behavior so they are not transparent [

18

].

However, white-box models allow to monitor the modeling and analyze the process step by step

interpreting the results for different scales (whole building, thermal zones, etc.) and link them with

the physics and architectural parameters of the buildings. While black-box models, such as ANNs,

are commonly used for small-scale modeling tasks or assuming that the zone temperature distribution

is uniform [

27

], white-box models are able to characterize large multi-thermal zones buildings. On the

other hand, white-box models are more ﬂexible to changes in the buildings characteristics or operation

since they do not require the re-training of the model, avoiding problems of input data quality

[28,29]

.

Once the model is developed, it can be more easily used for other applications like retroﬁt analysis or

fault detection and diagnostics [

30

], or conversely, it is easy to exploit a white-box model to provide

load forecast previously developed for other purposes.

The common factor among building load forecasting techniques is that their accuracy depends not

only on the accuracy of the model itself but also on the accuracy of the predicted external inputs [

31

].

Among these driving factors, weather parameter forecasting is a fundamental element because it

has a great inﬂuence on the building’s actual energy consumption but has inherent uncertainty.

The literature recognizes the signiﬁcant inﬂuence of the weather forecast on a building’s energy

performance, especially outdoor temperature [

32

]. However, the impact of the uncertainty due to

forecast weather data on building load forecasting is not well represented in the literature [

33

–

35

],

and few studies have directly investigated its effect [31,32,36–38].

On the other hand, traditionally, a load forecast is generated with a point or deterministic approach,

which means that a single expected value for the predicted load is provided. The problem is that this

Sensors 2020,20, 6525 3 of 20

point load forecast is not able to properly consider and quantify the effect of its inherent uncertainties.

Therefore, it is necessary to develop a tool to quantitatively describe the uncertainties of load prediction

and to assess the risk of relying on these forecasts. Uncertainties in load forecasting can be addressed

through probabilistic load forecasting (PLF), an approach that can provide future predictions with the

associated prescribed probabilities. This probabilistic approach is more adaptable to the current energy

context, where the dependence of load forecasting on its inherent uncertainties complicates reliable

and efﬁcient energy management [39].

Probabilistic load forecasting is gradually increasing in importance in the literature, especially

after the Global Energy Forecasting Competition 2014 [

40

], since it can provide more comprehensive

information for the energy management decision-making process. Hong and Fan [

39

] provided a

review of the state-of-the-art in probabilistic electric load forecasting where they stated that it can

be implemented in practically the same cases in which single-valued load forecasts are applied.

For example, it has been used for electricity consumption prediction in buildings [

41

–

43

], but also for

distributed renewable energy production forecasts, such as photovoltaic power generation or wind

speed forecasts [

44

,

45

]; applications related to electric vehicles [

46

]; and the quantiﬁcation of the power

reserve of a microgrid [47].

The literature includes several studies with different approaches that incorporate weather

uncertainty into PLF development to forecast the building’s load. Xu et al. developed a probabilistic

load forecasting model using an artiﬁcial neural network (ANN) and probabilistic temperature

forecasts. Their results showed that the probabilistic normal load forecasts had satisfactory accuracies,

and the load forecasts based on one-day-ahead probabilistic weather forecasts were the best [

48

].

Dahl et al. presented an autoregressive heat forecast model with weather prediction input and

concluded that ensemble weather predictions could improve supply temperature control in district

heating area substations [

49

]. Zhao et al. used the Monte Carlo method (MCM) to pre-process

meteorological forecast data to improve the accuracy of load forecasts provided by a support

vector machine (SVM) model, and the forecasting results became closer to the actual data [

34

].

Similarly, Fan et al. also employed the MCM to calibrate the input variables of their proposed

SVM cooling load prediction model with the aim of reducing the inﬂuence of the uncertainty of the

input variables (weather parameters, among others). With calibrated inputs, this approach produced

a more accurate prediction, which was closer to the load prediction based on measured data [

50

].

Although probabilistic load forecasting studies are increasing in the literature, they are mainly based

on black-box models which cannot clarify the link between inputs and the forecasted building loads.

This research aims to ﬁll this gap by providing a probabilistic load forecasting methodology that

considers the weather prediction uncertainty using white-box models (building energy models, BEMs)

instead of black-box models. The proposed methodology converts the point load forecast provided by

a BEM into a probabilistic load forecast using historical data based on indoor and outdoor building

monitoring. An hourly map of the uncertainty of the load forecast due to the weather forecast is

provided for a speciﬁc building and weather forecast source. After applying the uncertainty map

on the BEM outputs, the hourly load forecast is obtained with the probabilistic error due to weather

forecast data, providing a tool that can help the building’s energy managers and network operators to

make more informed decisions.

The load forecast is provided by a BEM (physics-based model), which is fed weather ﬁles that

are generated using a methodology that respects the thermal history of the building. Different time

horizons, from 1 day to 6 days ahead, were employed in this study to assess their inﬂuence on the

probabilistic load prediction. The methodology was applied to a real case study, a building located in

Gedved (Denmark), which is equipped with indoor temperature sensors and an on-site weather station.

Following the recommendations of Agüera-Pérez [

35

], this research used a real weather forecast from

an external provider instead of synthetic data [

51

] and six weather parameters instead of using only

temperature [52] or temperature and humidity [36].

Sensors 2020,20, 6525 4 of 20

The main contributions of this research are: (1) a probabilistic load forecasting approach is

provided based on white-box models, instead of black-box models; (2) an hourly uncertainty map

is provided as an easy tool to represent the expected hourly probabilistic load forecast error due to

weather forecast for a speciﬁc building and weather prediction source; and (3) a dedicated script is

developed to generate the daily weather ﬁles that feed the building energy model.

This paper is organized as follows: Section 2shows the proposed PLF methodology, including the

simulation and the probabilistic process performed on the data generated by the BEM. Section 3focuses

on the description of the case study for which the methodology was implemented, and Section 4shows

the results, including the evaluation of the methodology. Finally, Sections 5and 6present the discussion

and conclusions, respectively.

2. Methodology

This section presents the methods for the proposed probabilistic load forecast technique,

which uses a BEM and accounts for weather forecast uncertainty. This technique requires two

procedures, which are detailed in the following sections: the simulation process to determine the

historical impact of the weather forecast data on the load provided by the BEM and the probabilistic

processing of the simulation outputs. The overall approach of the proposed probabilistic load

forecasting methodology is illustrated in the Figure 1.

Combined

weather files

Building

Energy Model

Kernel

density

estimation

(KDE)

Hourly

Uncertainty

Map

Measured

weather data

Forecast

weather data

Actual

building's load

Forecast

building's load

Indoor measured

temperature

Data from on-site sensors Probabilistic load forecast (PLF)

Simulation Process Probabilistic Process

(Point load forecast)

Figure 1.

Components and steps of the proposed probabilistic load forecasting methodology based on

white-box models (building energy model (BEM)).

Since the effect of the weather forecast on the energy model is considered in this methodology,

all uncertainties related to the simulation must be minimized so that the results of the probabilistic

analysis are valid. Section 2.1 discusses aspects such as the thermal history, the creation of weather

ﬁles, the accuracy of the building energy model used, which accounts for the building’s internal

loads and the preparation of outputs for the probabilistic analysis. Then, Section 2.2 details the

probabilistic processing of the historical differences between the loads obtained from the observed and

forecast weather data employed to obtain the uncertainty map, which is subsequently applied to the

point forecast.

2.1. Simulation Process through a BEM

In the ﬁeld of forecasting building loads, many studies have used statistical or machine learning

approaches and reported low errors and good prediction accuracy. However, since these models were

trained using historical data, their ability to adapt to changes in the building is limited. Physical models,

which are based on physical and universal laws and equations, are more adaptive to changes [

53

].

These models facilitate an understanding of the thermodynamic aspects and interactions with the

internal and external environments. The proposed methodology employs calibrated building energy

models (BEMs) based on EnergyPlus simulation software [

54

,

55

] as the best representation of the

behavior of the real building.

In order to compare the load differences using observed and forecast weather data, it is

necessary to explain how the forecast and observed data are implemented in the BEM. In this regard,

important roles are played by weather ﬁles, generated with both observed and forecast information;

Sensors 2020,20, 6525 5 of 20

the indoor temperature, as the best representation of the internal loads of the building; and the

simulation periods, which are very relevant when processing the thermal history in the simulation.

To ensure the appropriate initial conditions in the simulation process, the weather ﬁles must be

created with respect to the thermal history of the building for both the internal and external conditions.

In this research, a procedure was developed to generate weather ﬁles in

EPW

(EnergyPlus weather

ﬁle) format to meet this requirement. The Weather Converter [

56

] tool, provided as an auxiliary

program by EnergyPlus, was used for the creation of all of these ﬁles by translating and extending

typical weather data into the

EPW

format and making the necessary calculations for unavailable

data. The source of the observed weather data is an on-site weather station installed in the building

surroundings. Following the recommendations made by Agüera et al. in [

35

], real weather forecast

data supplied by an external provider were used in this methodology, instead of using arbitrary

forecasts based on synthetic data [

57

] or historical forecasts, which can be treated as perfect forecasts or

modiﬁed by adding variations [

36

] that do not provide the real context of the building’s performance.

The process of creating the weather ﬁle started with the collection of daily forecast weather data from

the external provider. Then, one weather ﬁle was generated for each day with the measured weather

information (historical data) and forecast data. The resulting ﬁle is called the combined weather

ﬁle and contains both observed and forecast data. The process was implemented using a dedicated

script that injects the forecast weather data into the historical weather data. Therefore, the combined

daily weather ﬁles contain

n

days of forecast weather data, and the rest of the data correspond to the

observed data provided by an on-site weather station. Figure 2depicts the methodology.

day 1

day 2

day 3

day 4

day 5

day n

forecast

data 1

Forecast data from external provider

MEASURED DATA

Script to

inject data

Weather creation

day 1

day 2

day 3

day 4

day 5

day n

forecast data 1

weather file

MEASURED

*.epw 1

day 1

day 2

day 3

day 4

day 5

day n

forecast

data 2

day 1

day 2

day 3

day 4

day 5

day n

forecast

data n

day 1

day 2

day 3

day 4

day 5

day n

forecast data 2

weather file

MEASURED

*.epw 2

day 1

day 2

day 3

day 4

day 5

day n

forecast data n

weather file

MEASURED

*.epw n

WEATHER FILE

Combined weather files - One per day of analysis

Figure 2. Weather ﬁle creation methodology.

It is crucial that the BEM correctly reﬂects the past thermal behavior by taking into account all

loads of the building (HVAC system, people, lighting, electric equipment, etc.). The best way to

account for these loads is to use the building’s indoor temperature measurements via the sensors of

the building management system (BMS). An external ﬁle with the actual indoor temperature is used

by the simulation model as a dynamic set-point for the HVAC system. The simulation output is the

energy demand required by the model to follow it. This is shown in Figure 3.

Figure 3. Simulation process methodology.

The probabilistic analysis was carried out by comparing the energy differences in the models

when using the observed and forecast weather data. To obtain accurate energy differences, the use of

the correct simulation period is very important. One simulation per day of analysis was executed since

each day has its own weather ﬁle. For each day, the simulation was conﬁgured to run 15 days before

the baseline day (day 0) with the measured weather data to capture the thermal history of the model.

Sensors 2020,20, 6525 6 of 20

The loads on day 0 and

n

days of the forecast were obtained as the outputs of each simulation.

These results (ordered and classiﬁed) were subsequently used in the probabilistic analysis.

The error of the point load prediction provided by the BEM when using weather forecast can be

evaluated using three error metrics commonly employed in the forecasting literature: mean absolute

error (MAE) (Equation

(1)

), which measures the average magnitude of the error in the units of

the variable of interest; mean absolute percentage error (MAPE) (Equation

(2)

), which is a relative

error measure that allows comparing the forecasts performance on different data sets; and the

coefﬁcient of determination (

R2

) (Equation

(3)

), which allows to measure the linear relationship

of the two patterns [58].

MAE =1

n

n

∑

i=1|yi−ˆ

yi|, (1)

MAPE =1

n

n

∑

i=1

|yi−ˆ

yi|

ˆ

yi×100%, (2)

R2=

n·∑n

i=1yi·ˆ

y−∑n

i=1yi·∑n

i=1ˆ

y

q(n·∑n

i=1y2

i−(∑n

i=1yi)2)·(n·∑n

i=1ˆ

y2−(∑n

i=1ˆ

y)2)

2

(3)

2.2. Probabilistic Load Forecast

The probability load forecast (PLF) proposed for this methodology was produced by applying the

probability density function of residuals to the point forecast. This approach to producing the PLF

was classiﬁed as output by Hong et al. in [

39

]. The point forecast provided by the BEM was converted

to the PLF using historical data based on the observed and forecast weather data and the building’s

indoor temperature.

First, the distribution of the residuals, which are the energy load differences provided by the BEM

when it is fed the observed and forecast weather data, was studied through a probabilistic histogram.

This is useful because it provides a straightforward visualization of the spread and the skewness of

the data, the presence of outliers and the presence of multiple modes in the data. Second, to obtain a

smooth curve that represents the data, a probability density estimation was performed. Many studies

have used the normal distribution to estimate the density function, but it performs well only when the

data follow a bell-shaped distribution. To avoid making assumptions about the distribution of the data,

kernel density estimate (KDE) method was employed in this methodology, which is a non-parametric

representation of the probability density function [

45

]. If sequence Xconsists of N1-dimensional

observations

x1

,

x2

,

. . .

,

xN

, the KDE method estimates the actual probability density function fthrough

the following function (4):

ˆ

fh(x) = 1

n

n

∑

i=1

K(x,xi)(4)

where Kis the kernel function, which is a non-negative symmetric function that integrated to one and

has mean zero. There are many types of kernels but empirically is it not very relevant which one is

employed [

59

]. In this case, Gaussian kernel is used to realize the KDE. It replaces each sample point

with a Gaussian-shaped kernel and then obtains the resulting estimate for the density by adding these

Gaussians. It can be expressed by (5) [45]:

K(x1,x2,σ) = 1

√2πσ

e−(x1−x2)2

2σ2(5)

The

σ>

0 is the bandwidth, a smoothing parameter that inﬂuences the shape of the distribution.

There are many bandwidth selection methods [

60

] and the advantages of using Gaussian kernel KDE

is that it can calculate the bandwidth by a rule of thumb automatically [61].

Sensors 2020,20, 6525 7 of 20

The objective of this methodology is to obtain the expected probability that the load forecast

error is below a certain value. The cumulative distribution function (CDF) or S-curve is an easily

interpretable representation of the probability that the variable (here, the load forecast error due to

the weather forecast) will be less than or equal to a certain value. Finally, from the CDF plot of each

forecast hour, prediction intervals (PIs) of load forecast errors are extracted and transformed into an

hourly map of uncertainty. The schema in Figure 4shows the complete process.

Probability of occurrence

Energy differences (kWh)

Probability density

Energy differences (kWh)

Kernel Density

Estimation (KDE)

Cumulative probability

1.0

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0.0

Energy differences (kWh)

Quantiles

Hourly

Uncertainty Map

Hours of forecast day

Energy demand uncertainty

Prediction

Intervals (PI)

Probability

Histogram

Probability Density

Function (PDF)

Cumulative Distribution

Function (CDF)

Figure 4. Process of the probabilistic load forecast.

The map of the uncertainty of the load forecast due to weather forecast data shows an overview

of the probability that the load error is below a certain value for each hour. The map was constructed

with the available hours ahead of the forecast time, and it is read as follows: for hour

n

, there is an

x

% probability that the energy demand error is less than

y

kWh. This map of uncertainty can then

be applied to the load forecast provided by the BEM by applying the intervals of the energy demand

error with their probabilities to the load forecast outputs of the model. In this way, the load forecast

is obtained with the probability error due to weather forecast data, similar to a risk map. It can be

used, for example, by the building’s energy manager to make a more informed choice according to the

bearable risk.

Finally, for the evaluation of the methodology, two indicators commonly used in the related

literature were employed for the prediction interval assessment: the prediction interval coverage

probability (PICP) and the mean prediction interval width (MPIW) [

62

]. PICP measures the reliability

of the predictions and shows the percentage of the real values that will be covered by the upper and

lower bounds. The larger the PICP, the more likely that the real values are within the prediction interval.

It can be deﬁned as:

PI CP =1

H

H

∑

i=1

Ci, (6)

in which His the number of samples, and Ciis a Boolean variable deﬁned as follows:

Ci=(1, yi∈[Li,Ui]

0, yi/∈[Li,Ui],(7)

where

Li

and

Ui

are the lower and upper PI bounds of target

yi

, respectively. PICP ranges between 0

and 100%. The prediction interval is considered valid if the PICP value is greater than the prediction

interval nominal conﬁdence (PI NC =100(1−α)%), where αrepresents the probability of error.

High PICP values can be easily reached when the width of the prediction intervals (PIs) is large.

However, large PIs have higher levels of uncertainty, and, thus, they are useless for decision making.

Therefore, a complementary metric is required to assess the prediction interval widths: this metric is

the Mean Prediction Interval Width (MPIW), which is deﬁned as:

MPIW =1

H

H

∑

i=1

(Ui−Li). (8)

In conclusion, to make a suitable decision, small MPIW and high PICP values are desirable.

Sensors 2020,20, 6525 8 of 20

As mentioned in the methodology explanation, the validity of the results of the probabilistic

analysis is closely related to the data selection and processing used to create the uncertainty map.

The following section describes a case study in which the methodology was implemented, and it

illustrates the importance of sensors for obtaining both the weather ﬁle and the indoor temperatures.

3. Description of the Case Study

In this section, the case study that was used to apply the proposed methodology is presented.

The test site is a public elementary school in Gedved, Denmark. This building is part of the EU-funded

H2020 research and innovation project SmArt BI-directional multi eNergy gAteway (SABINA) [

63

].

Gedved School consists of 6 buildings and was built in 1979 and renewed in 2007. For this case study,

the library was selected, which is a one-story building with a total surface area of 1138 m

2

. The main

characteristics of the building are as follows: there are two brick layers with 150 mm insulation in

between them for the facades; the windows are two-layer double-glazed with cold frames; the ceiling

is insulated with 200–250 mm mineral wool for sloping and ﬂat ceilings, respectively; and the ﬂoor

is made up of concrete and contains 150 mm insulation underneath. Regarding HVAC systems,

only heating is provided to the building, and it is connected for 24 h every day from October to May.

Figure 5shows an outdoor photograph of the library.

Figure 5.

Library building from Gedved School, Denmark. Left: Outdoor image.

Middle: Weather station

installed on the building’s roof. Right: The building energy model

(OpenStudio plugin for SketchUp [64]).

The building energy model used in this study was provided by the SABINA project and was

developed using the EnergyPlus engine. In the load forecasting ﬁeld, when using BEMs, it is necessary

to take into account three main sources of uncertainty: BEM accuracy, building use and external

conditions. In order to minimize the ﬁrst uncertainty, this case study employed a calibrated BEM,

obtained using a calibration methodology explained in the authors’ previous papers [

58

,

65

–

68

].

Regarding the building’s use, no uncertainty was consider in the indoor conditions since the model

used indoor temperatures measured in each thermal zone by the BMS.

For the creation of weather ﬁles, the external conditions are required for both the observed

and the forecast weather data. The observed weather data were obtained from a weather station

installed on the building’s roof, which provides measurements with hourly intervals for atmospheric

pressure, temperature, humidity, direct and diffuse irradiation, wind speed, wind direction and rainfall.

Figure 5shows the weather station location. The forecast weather information used in this study is a real

forecast supplied by the commercial service Meteoblue [

69

]. This company uses a multimodel/machine

learning approach to calculate the forecast weather data using both Meteoblue weather models

(Nonhydrostatic Meso-Scale Modeling) and third-party models for the simulations. More information

about Meteoblue’s forecast weather data process is available on its web page [

69

]. For this study,

the weather forecast data were gathered at 09:00 on each day. For the time horizon, 6 days ahead of the

weather forecast data were available for the present study.

The period in which all of the required data were available is from December 2018 to April

2020. The summer months (from June 2019 to September 2019) were not useful for the present study

since no cooling system is installed in the building, and, therefore, only heating load forecasting was

Sensors 2020,20, 6525 9 of 20

considered. Thus, the ﬁnal period of the study is composed of 13 months, from December 2018 to

May 2019 and from October 2019 to April 2020, so two complete winter campaigns (2018–2019 and

2019–2020) were analyzed.

First, the methodology is illustrated using a map of uncertainty generated from all 13 months

of data to show how this probabilistic load forecast (PLF) method is implemented using a BEM.

Then, the application of the uncertainty map to the heating load forecast provided by the BEM is

presented for a random day. Finally, it is evaluated the ability of the proposed PLF methodology to

predict the expected error in the heating load using the 12 ﬁrst months of the period of the study to

generate the map of uncertainty and the last month (April 2020) to test the method.

4. Results

In this section, the results of applying the proposed probabilistic load forecasting (PLF)

methodology to a real test site are presented. As mentioned before, the methodology converts the

point load forecast provided by a BEM, which is a single-value, into a probabilistic load forecast.

Table 1presents the quantitative errors of the point load forecast provided by the BEM for the different

time horizons (from 1 to 6 day-ahead). It shows the error metrics MAE, MAPE and

R2

between

the forecast and real heating load when using the forecast and actual weather data, respectively.

The results showed, as expected, the increase in the error as the day ahead grows (MAE and MAPE

values increased and R2decreased).

Table 1.

Error metrics for the point load forecast. Comparison between forecast and real heating load.

Index Forecast Day 1 Forecast Day 2 Forecast Day 3 Forecast Day 4 Forecast Day 5 Forecast Day 6

MAE (kWh) 9.91 10.50 11.04 11.57 12.43 13.74

MAPE (%) 38.99 41.26 43.56 47.39 51.10 55.84

R2(%) 74.48 70.82 65.99 59.65 49.72 45.66

In order to show how the methodology is implemented, the whole period of study (13 months of

2 winter campaigns) was employed for the construction of the map of uncertainty. The differences in the

hourly heating load between simulations with observed and forecast weather data were transformed

into a probability histogram. This process was ﬁrst performed for each full day-ahead in order to show

the inﬂuence of the forecast time horizon on the heating load provided by the BEM. Figure 6shows

the histograms for the 6 forecast days. The highlighted gray area shows the spread of the variables

for each forecast day. The graphs show that this gray area grows, which means that there are larger

errors, as the forecast days increase. The errors in all the forecast days do not follow a symmetrical

distribution with respect to the zero, and they all tend to skew toward the right, which means that the

heating energy demand simulated with the forecast weather data is mainly overestimated. The graphs

also present the probability density functions (PDFs) based on the Gaussian kernel density estimation

(KDE) (red line).

The information in Figure 6was transformed into a cumulative density function (CDF),

which depicts the probability that an error in the heating load (in kWh) is below a particular value.

Figure 7shows the representation of the CDF for the six forecast days. For example, in this case,

forecast day 1 has a 90% probability that the hourly load forecast error is below 18.5 kWh or a 10%

probability that it is below 1.2 kWh (blue lines). The plot also shows that the heating load error

increases as the forecast days increase. In this example, for the same 90% probability, the heating load

error for forecast day 6 grows to 27 kWh.

Sensors 2020,20, 6525 10 of 20

30 20 10 0 10 20 30 40 50 60 70 80 90 100

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.07

Probability

30 20 10 0 10 20 30 40 50 60 70 80 90 100

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.07

Probability

30 20 10 0 10 20 30 40 50 60 70 80 90 100

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.07

Probability

Forecast day 1 Forecast day 2

Forecast day 3 Forecast day 4

Forecast day 5 Forecast day 6

Heating load difference (kWh) Heating load difference (kWh)

Heating load difference (kWh) Heating load difference (kWh)

Heating load difference (kWh) Heating load difference (kWh)

30 20 10 0 10 20 30 40 50 60 70 80 90 100

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.07

Probability

30 20 10 0 10 20 30 40 50 60 70 80 90 100

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.07

Probability

30 20 10 0 10 20 30 40 50 60 70 80 90 100

0.00

0.01

0.02

0.03

0.04

0.05

0.06

0.07

Probability

N: 7493

Bandwidth: 1.4

N: 7493

Bandwidth: 1.14

N: 7493

Bandwidth: 1.84

N: 7493

Bandwidth: 1.52

N: 7493

Bandwidth: 1.25

N: 7493

Bandwidth: 2.05

Figure 6.

Probability histogram of the 6 forecast days and the Gaussian kernel density estimation

(red line).

30 25 20 15 10 5 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Cumulative probability F(x)

CDF: Forecast day: 1

CDF: Forecast day: 2

CDF: Forecast day: 3

CDF: Forecast day: 4

CDF: Forecast day: 5

CDF: Forecast day: 6

Heating load difference (kWh)

90% = 18.5 kWh

10% = 1.2 kWh

Figure 7. Cumulative distribution function (CDF) for the 6 forecast days.

The previous plots graphically show the hourly heating load error probability as a function of

the full forecast day. In many applications of load forecasting, such as demand response or model

predictive control, an hour-by-hour load error approach is more useful since most applications are

implemented on an hourly basis. Thus, the same procedure was performed but based on an hourly

approach from hour 1 to hour 144 with respect to the forecast time (09:00 h). The probability histogram,

Sensors 2020,20, 6525 11 of 20

the ﬁtted PDFs (KDE), and the CDF were calculated for all of the data that correspond to the ﬁrst

forecast hour, the second forecast hour and so on, with respect to the forecast time.

With the hourly CDF, the heating load error probability can be extracted as prediction intervals

(in this case, intervals of 10%), making the probabilistic load forecast information more useful

and applicable. For this purpose, a map that summarizes all of the hourly information was computed

in order to present an overview of the heating load error due to the weather forecast as a function of

the hours ahead of the forecast time. This generates the hourly uncertainty map of the heating load

forecast due to weather data uncertainty for this speciﬁc building and weather forecast provider.

The bound margins are obtained by a normal inverse cumulative distribution function for

a desired probability index

x

, which, in this case, ranges from 10% to 90%. Figure 8shows the

uncertainty map for the ﬁrst 24 h ahead of the forecast hour (09:00 h). For each hour, the expected

heating load error due to weather forecast uncertainty can be extracted. For example, at 10:00 on

forecast day 1, there is a 50% probability that the error in the heating load is less than 8.1 kWh, or,

being more conservative, there is a 90% probability that the error is less than 17.1 kWh. In other

words, the use of the forecast weather data leads to an overestimation of the heat load of 8.1 kWh

and 17.1 kWh, respectively. The map shows that the heating load predicted with the forecast weather

is likely to be overestimated with respect to the observed weather data for all of the forecast hours

since the prediction intervals are mainly above 0. When the map is applied to the forecast heat load

provided by the BEM, this overestimation must be considered inverted: there is a 50% probability that

the simulation with forecast weather data provides 8.1 kWh in excess.

This map can be computed for as many forecast hours as there are available. Figure 9shows

the uncertainty map for the 144 h forecast hours available in this case study, divided into the

six forecast days. As expected, the load uncertainty increases as the forecast hours increase, and this

is reﬂected in these maps: for example, at 10:00 on forecast day 6 (121 h ahead of the forecast time),

there is a 50% probability that the heat load error is lower than 10 kWh, which is 1.9 kWh higher than

that for the same probability at 10:00 on forecast day 1 (1 h ahead of the forecast time), and there is a

90% probability that the heat load error is lower than 27.9 kWh, which is 10.8 kWh higher than that for

the same probability at 10:00 on forecast day 1. The graphs also show that the width of the prediction

intervals in the results grows as the forecast hours increase since, as seen in the histograms for the

daily analysis (see Figure 6), as the forecast days increase, the dispersion of the results also increases.

Once the map of uncertainty is obtained, it can be applied to the heating point load forecast

provided by the BEM. To illustrate this, the previous map is shown for a random day (2 January 2020).

Figure 10 shows the heating load forecast with the uncertainty map for prediction intervals from 10%

to 90% implemented for 2 January 2020 and for 1 day ahead (from 1 to 24 h ahead). In the graph,

the red dots represent the load forecast provided by the BEM for this speciﬁc day. The load uncertainty

is represented as prediction intervals around the heating load forecast. When the predictions intervals

(PIs) are below the heating load forecast, the energy demand is overestimated, and if they are above,

it is underestimated. For the case study, the graph shows that the prediction intervals tend to be below

the forecast energy demand, which means that it is much more likely that the model overestimates

the heating load due to the weather forecast. In order to clarify how the graph is read, we provide

the following example. At 10:00, which is 1 h ahead of the forecast time, the forecast heating load

provided by the BEM is 47.6 kWh (red dot), and there is a 90% probability that the real heating load

(with the observed weather data) is more than 30.5 kWh; as another way to analyze it, there is an

80% probability (prediction intervals between 10% and 90%) that the real heating load is higher than

30.5 kWh and lower than 46.1 kWh. According to the risk that the energy manager can accept, higher or

lower probability intervals can be used for decision making. In the graph, the blue dots represent

the real heating load, which is the result of the simulation with the observed weather data. For this

example, all the dots are within the PI, which means that, for this case, this procedure provided a good

prediction of the error in the heating load forecast due to the weather forecast data.

Sensors 2020,20, 6525 12 of 20

The evaluation of the ability of the methodology to predict the load forecast error due to the

weather forecast was performed for the 1 day-ahead data since it is the most commonly used time

horizon in load forecasting. PICP and MPIW indicators were used to perform the assessment,

and prediction intervals from 10% to 90% were considered. The validation of the methodology

was performed using the ﬁrst 12 months of the data (from December 2018 to May 2019 and from

October 2019 to March 2020) to generate the uncertainty map and the most recent month, April 2020,

to test the prediction interval accuracy. This month has a range of actual hourly heating loads from 0

on the mild days to 45.8 kWh on the coolest days. The PICP result is 83.1%, which is greater than the

conﬁdence level (PINC = 80%), and the MPIW is 17.5 kWh. Figure 11 shows the graphs of the results of

the testing period (April 2020). In order to facilitate its interpretation, the graph for the whole month is

divided into three parts. The actual loads are indicated by the blue line, the heating load forecast is the

red line and the gray area represents the 80% prediction interval (from 10% to 90%). It can be seen that

the prediction intervals generated with the proposed method cover the actual value most of the time.

Previous evaluation is performed using the 12 previous months prior to the testing month

(April 2020) to generate the uncertainty map, but, how does the amount of data affect the uncertainty

map and the results? In order to analyze this inﬂuence an analysis is performed gradually increasing

the amount of data used for the creation of the uncertainty map. Maintaining April 2020 as the testing

period, the uncertainty map is created using only the previous month (March 2020), the previous two

(February 2020–March 2020) and so on. Table 2presents the results. It shows that in all the cases the

PICP values are higher than the conﬁdence level (80%). It is remarkable that PICP values increase as

months closer to the testing period are used for the creation of the uncertainty map, despite having a

lesser amount of data, but MPIW values increase as well providing higher prediction intervals widths.

Therefore, the results are robust and similar despite the number of months employed to build the

uncertainty map.

Table 2.

Prediction interval coverage probability (PICP) and mean prediction interval width (MPIW)

results when using a gradually increasing amount of data for the creation of the uncertainty map.

April 2020 is maintained as the testing period.

Months

Uncertainty

Map

March 2020 February 2020

March 2020

January 2020

March 2020

December 2019

March 2020

November 2019

March 2020

October 2019

March 2020

May 2019

March 2020

April 2019

March 2020

March 2019

March 2020

February 2019

March 2020

January 2019

March 2020

December 2018

March 2020

Nº of months 1 2 3 4 5 6 7 8 9 10 11 12

PICP (%)91.5 89.9 84.4 82.2 83.1 84.2 83.8 84.3 84.4 84.2 84.0 83.1

MPIW (kWh) 27.1 23.1 18.8 18.4 18.3 17.8 17.5 17.2 17.7 17.4 17.7 17.5

-5

0

5

10

15

20

25

30

Energy demand uncertainty (kWh)

Hours

Hours

ahead

10

+1

11

+2

12

+3

13

+4

14

+5

15

+6

16

+7

17

+8

18

+9

19

+10

20

+11

21

+12

22

+13

23

+14

24

+15

01

+16

02

+17

03

+18

04

+19

05

+20

06

+21

07

+22

08

+23

09

+24

PI 90% = 17.1 KWh

PI 50% = 8.1 KWh

80–90%

70–80%

60–70%

50–60%

40–50%

30–40%

20–30%

10–20%

Prediction intervals

Figure 8.

Probabilistic heating load forecast results: Hourly uncertainty map of the heating load

forecast due to weather forecast data for the ﬁrst 24 h ahead of the forecast hour (09:00 h).

Sensors 2020,20, 6525 13 of 20

-5

0

5

10

15

20

25

30

Energy demand uncertainty (kWh)

10

+1

11

+2

12

+3

13

+4

14

+5

15

+6

16

+7

17

+8

18

+9

19

+10

20

+11

21

+12

22

+13

23

+14

24

+15

01

+16

02

+17

03

+18

04

+19

05

+20

06

+21

07

+22

08

+23

09

+24

10

+25

11

+26

12

+27

13

+28

14

+29

15

+30

16

+31

17

+32

18

+33

19

+34

20

+35

21

+36

22

+37

23

+38

24

+39

01

+40

02

+41

03

+42

04

+43

05

+44

06

+45

07

+46

08

+47

09

+48

Hours

Hours

ahead

Hours

Hours

ahead

-5

0

5

10

15

20

25

30

Energy demand uncertainty (kWh)

10

+49

11

+50

12

+51

13

+52

14

+53

15

+54

16

+55

17

+56

18

+57

19

+58

20

+59

21

+60

22

+61

23

+62

24

+63

01

+64

02

+65

03

+66

04

+67

05

+68

06

+69

07

+70

08

+71

09

+72

Hours

Hours

ahead

10

+73

11

+74

12

+75

13

+76

14

+77

15

+78

16

+79

17

+80

18

+81

19

+82

20

+83

21

+84

22

+85

23

+86

24

+87

01

+88

02

+89

03

+90

04

+91

05

+92

06

+93

07

+94

08

+95

09

+96

Hours

Hours

ahead

-5

0

5

10

15

20

25

30

Energy demand uncertainty (kWh)

10

+97

11

+98

12

+99

13

+100

14

+101

15

+102

16

+103

17

+104

18

+105

19

+106

20

+107

21

+108

22

+109

23

+110

24

+111

01

+112

02

+113

03

+114

04

+115

05

+116

06

+117

07

+118

08

+119

09

+120

Hours

Hours

ahead

-5

0

5

10

15

20

25

30

Energy demand uncertainty (kWh)

10

+121

11

+122

12

+123

13

+124

14

+125

15

+126

16

+127

17

+128

18

+129

19

+130

20

+131

21

+132

22

+133

23

+134

24

+135

01

+136

02

+137

03

+138

04

+139

05

+140

06

+141

07

+142

08

+143

09

+144

Hours

Hours

ahead

-5

0

5

10

15

20

25

30

Energy demand uncertainty (kWh)

-5

0

5

10

15

20

25

30

Energy demand uncertainty (kWh)

Uncertainty map forecast day 2 (25–48 h)

Uncertainty map forecast day 4 (73–96 h)

Uncertainty map forecast day 6 (121–144 h)

Uncertainty map forecast day 1 (1–24 h)

Uncertainty map forecast day 3 (49–72 h)

Uncertainty map forecast day 5 (97–120 h)

80–90%

70–80%

60–70%

50–60%

40–50%

30–40%

20–30%

10–20%

Prediction intervals

Figure 9.

Probabilistic heating load forecast results: Hourly uncertainty map of the heating load

forecast due to weather forecast data for 1 to 6 days ahead.

10

15

20

25

30

35

40

45

50

55

Energy demand (kWh)

10

+1

11

+2

12

+3

13

+4

14

+5

15

+6

16

+7

17

+8

18

+9

19

+10

20

+11

21

+12

22

+13

23

+14

24

+15

01

+16

02

+17

03

+18

04

+19

05

+20

06

+21

07

+22

08

+23

09

+24

Hours

Hours

ahead

Forecast hours

Real energy demand

Forecast energy demand

10–20%

20–30%

30–40%

40–50%

50–60%

60–70%

70–80%

80–90%

PI 10% = 46.1 KWh

PI 90% = 30.5 KWh

Figure 10.

Application of the map of the heating load forecast uncertainty due to forecast weather data

using the whole period of the study for a random day: 2 January 2020.

Sensors 2020,20, 6525 14 of 20

0

5

10

15

20

25

30

35

40

45

50

55

60

65

70

1 2 3 4 5 6 7 8 9 10

0

5

10

15

20

25

30

35

40

45

50

55

60

65

70

11 12 13 14 15 16 17 18 19 20

0

5

10

15

20

25

30

35

40

45

50

55

60

65

70

21 22 23 24 25 26 27 28 29 30

Heating Load (kWh)

Heating Load (kWh)

Heating Load (kWh)

Date

Forecast heating load (kWh)

Actual heating load (kWh)

Prediction interval PINC 80

% (10%–90%)

Figure 11.

Results for the validation of the probabilistic load forecasting methodology for the testing

period (April 2020). Above: from 1 to 10 April 2020; middle: from 11 to 20 April 2020; and below:

from 21 to 30 April 2020.

5. Discussion

In the current energy context, load forecasting is becoming increasingly widespread, and the

inﬂuence of the uncertainties that affect the prediction of future energy demand must be taken

into consideration. One of the most important uncertainties is the weather forecast, but traditional

point load forecast cannot properly reﬂect the effect of this uncertainty. This paper aims to provide a

methodology that calculates the probabilistic load forecast while accounting for the inherent uncertainty

in forecast weather data. Recently, PLF approaches have gained importance in the literature but focused

mainly in black-box models. In order to ﬁll this gap, this research provides a PLF methodology based

on white-box models (BEMs). This methodology generates an hourly map of the uncertainty of

the load forecast, which allows the point load forecasting provided by the BEM to be converted

into a probabilistic load forecast. The map of uncertainty is created through a probabilistic analysis

of the impact that the weather forecast has on the building’s load, which is provided by the BEM

using data from the past. This analysis is possible thanks to the use of an accurately calibrated BEM

and data gathered from different sources: the indoor conditions are provided by the sensors of the

Sensors 2020,20, 6525 15 of 20

building’s monitoring; the observed outdoor conditions are supplied by an on-site weather station;

and the weather forecast is provided by an external service.

A case study in a real school building in Gedved, Denmark, is presented. Thirteen months

of data from the building’s monitoring, weather station and weather forecast were available,

which corresponds to two complete winter campaigns. As a result that this building lacks a cooling

system, the study was performed for the heating load forecast, and only the months in which the

systems were actuated were considered.

Unlike other studies from the literature, this research employed a physics-based model, which is

a calibrated BEM developed with EnergyPlus. Following the recommendations of Agüera et al.

in [

35

], this research used the real weather forecast from an external provider instead of synthetic

data. Agüera et al. also found that many papers that employ weather forecast for load forecasting

employed test periods of one day or shorter, and they recommended the use of longer testing periods.

In the present study, the methodology was validated using a whole month. Regarding the load

forecasts, Agüera et al. [

35

] recommended the introduction of at least outdoor temperature forecasts,

and they considered humidity estimations to be valuable. Studies that have accounted for weather

forecasts have usually incorporated only outdoor temperature [

52

], outdoor temperature and relative

humidity [

34

,

36

] or temperature and solar irradiation [

32

] but not all inﬂuential weather parameters.

Previous studies from the authors have shown that weather parameters such as wind speed can be

very inﬂuential on the building’s load [

70

]. For this reason, this research introduced forecast data

for outdoor temperature, relative humidity, direct normal irradiation, diffuse horizontal irradiation,

wind speed and wind direction.

Regarding the results from the case study, ﬁrst, the probabilistic load forecast was generated

based on all 13 months of data. The hourly map of the uncertainty of the heating load for this building

and weather provider is presented, and an example of its application is shown. This map shows the

probability of having

x

error in the load forecast for each forecast hour (from 1 to 144 h) for different

prediction intervals from 10% to 90%. In this case, the map shows clearly that the forecast weather

data of the weather provider and location of the study generate an overestimation of the heating load.

The validation of the methodology was performed for the one-day-ahead weather forecast,

which is the most common time horizon used, using the PICP and MPIW indicators. The evaluation

was performed using April 2020 as the testing period, and the previous months were used to generate

the prediction intervals. The results reveal a good performance: the PICP is 83.1%, which was greater

than the conﬁdence level (PINC = 80%), with an MPIW of 17.5 kWh. In order to evaluate the inﬂuence

in the results of the size of the sample used for the creation of the uncertainty map, an analysis was

performed gradually increasing the number of months employed and maintaining April 2020 as the

testing period. It showed robust and similar results despite the amount of data employed in the

uncertainty map creation. In all the cases, PICP values were above the conﬁdence level ranging from

82.2 to 91.5%.

6. Conclusions

The present study shows a probabilistic load forecasting methodology based on the use of white-box

models developed in EnergyPlus instead of data-driven or black-box models, which are the commonly

used prediction models employed in the literature for PLF approaches. White-box models allow to link

the results to the physical and architectural building parameters, evaluate the thermal indoor conditions in

different time and spatial scales and are more flexible and robust to changes in the building or operation

schedules. The proposed methodology allows to build an uncertainty map of the building load prediction,

which considers the uncertainty due to the use of weather forecast data, and then apply it to the point load

forecast provided by the building simulation model.

The proposed methodology was applied in a case study showing, through the uncertainty map, an easy

way to represent the expected hourly probabilistic load forecast error, which is very useful for interpretation

by the building’s manager, aggregators, microgrids operators, etc. The results of the evaluation of this

Sensors 2020,20, 6525 16 of 20

methodology showed that for the testing period (April 2020), morethan 80% of the actual hourly heating

loads values were within the prediction intervals. The influence of the amount of data used for the creation

of the uncertainty map was analyzed gradually increasing the number of months employed. It showed

that the methodology is robust since similar results, always with PICP values above the confidence level,

were obtained despite the amount of data employed. Therefore, the proposed methodology and the resulting

map of uncertainty shown in this research are useful tools for applications in which the future load forecast is

required, since future predictions always imply weather forecast uncertainties. For example, when predictive

control strategies are implemented in a microgrid and the consumer has to provide a day-ahead demand plan

with an hourly resolution, which is supposed to be granted within a given confidence interval, the probabilistic

load forecast of the building allows the network operators to better make decisions and plan for future smart

grids; or when the building participates in a demand response (DR) program, the aggregator that manages

the DR events can use a probabilistic load forecast that accounts for uncertainties such as the weather forecast

to make an informed decision about which buildings to rely on for the event.

A limitation of the study was that only heating load forecast, and therefore only cold months,

were considered since no cooling system was available in the building. In future research, this methodology

will be applied and tested in other buildings with both heating and cooling systems in order to assess how

the seasonality of the weather data influences the results when the whole year is employed. Furthermore,

we are currently analyzing whether the proposed map of uncertainty can be generated for a cluster of

buildings with similar architectural characteristics, use patterns and weather conditions. This could be a

simple and useful tool for network managers or aggregators in the DR context.

Author Contributions:

Conceptualization, methodology, formal analysis and visualization, E.L.S. and G.R.R.;

writing—original draft, E.L.S.; writing—review and editing, G.R.R. and C.F.B.; software, G.R.R.; and supervision, C.F.B.

All authors have read and agreed to the published version of the manuscript.

Funding:

This project received funding from the European Union’s Horizon 2020 research and innovation

program under Grant Agreement No. 731211, project SABINA.

Acknowledgments:

We would like to thank the Insero for providing the data of the Gedved school located in

Gedved (Denmark).

Conﬂicts of Interest: The authors declare no conﬂicts of interest.

Abbreviations

The following abbreviations are used in this manuscript:

ANN Artiﬁcial neural network

ARIMA Autoregressive integrated moving average

ARMA Autoregressive moving average

BEM Building energy model

BMS Building management system

CDF Cumulative distribution function

DR Demand response

DSM Demand-side management

EPW EnergyPlus weather ﬁle

HVAC Heating, ventilation, and air conditioning

Iot Internet of things

KDE Kernel density estimation

kWh Kilowatt hour

LR Linear regression

MAE Mean absolute error

MAPE Mean absolute percentage error

MCM Monte Carlo method

ML Machine learning

MPC Model predictive control

Sensors 2020,20, 6525 17 of 20

MPIW Mean prediction interval width

PDF Probability density function

PICP Prediction interval coverage probability

PINC Prediction interval nominal conﬁdence

PLF Probabilistic load forecast

PI Prediction intervals

SVM Support vector machine

R2Coefﬁcient of determination

References

1.

Marinakis, V.; Doukas, H. An advanced IoT-based system for intelligent energy management in buildings.

Sensors 2018,18, 610. [CrossRef] [PubMed]

2.

Jia, M.; Komeily, A.; Wang, Y.; Srinivasan, R.S. Adopting Internet of Things for the development of smart

buildings: A review of enabling technologies and applications. Autom. Constr.

2019

,101, 111–126. [CrossRef]

3.

Lu, X.; O’Neill, Z.; Li, Y.; Niu, F. A novel simulation-based framework for sensor error impact analysis

in smart building systems: A case study for a demand-controlled ventilation system. Appl. Energy

2020,263, 114638. [CrossRef]

4.

2019 Global Status Report for Buildings and Construction: Towards a Zero Emissions, Efﬁcient and Resilient

Buildings and Construction Sector. Available online: https://wedocs.unep.org/bitstream/handle/20.500.

11822/30950/2019GSR.pdf?sequence=1&isAllowed=y (accessed on 15 November 2020).

5.

Khan, Z.A.; Hussain, T.; Ullah, A.; Rho, S.; Lee, M.; Baik, S.W. Towards Efﬁcient Electricity Forecasting in

Residential and Commercial Buildings: A Novel Hybrid CNN with a LSTM-AE based Framework. Sensors

2020,20, 1399. [CrossRef] [PubMed]

6.

El Jaouhari, S.; Jose Palacios-Garcia, E.; Anvari-Moghaddam, A.; Bouabdallah, A. Integrated Management of

Energy, Wellbeing and Health in the Next Generation of Smart Homes. Sensors 2019,19, 481. [CrossRef]

7.

Lee, S.; Choi, D.H. Energy Management of Smart Home with Home Appliances, Energy Storage System and

Electric Vehicle: A Hierarchical Deep Reinforcement Learning Approach. Sensors

2020

,20, 2157. [CrossRef]

8.

Kerk, S.G.; Hassan, N.U.; Yuen, C. Smart Distribution Boards (Smart DB), Non-Intrusive Load Monitoring

(NILM) for Load Device Appliance Signature Identiﬁcation and Smart Sockets for Grid Demand Management.

Sensors 2020,20, 2900. [CrossRef]

9.

Lee, S.; Choi, D.H. Reinforcement learning-based energy management of smart home with rooftop solar

photovoltaic system, energy storage system, and home appliances. Sensors 2019,19, 3937. [CrossRef]

10.

Foucquier, A.; Robert, S.; Suard, F.; Stéphan, L.; Jay, A. State of the art in building modelling and energy

performances prediction: A review. Renew. Sustain. Energy Rev. 2013,23, 272–288. [CrossRef]

11.

Amasyali, K.; El-Gohary, N.M. A review of data-driven building energy consumption prediction studies.

Renew. Sustain. Energy Rev. 2018,81, 1192–1205. [CrossRef]

12.

Bourdeau, M.; Zhai, X.; Nefzaoui, E.; Guo, X.; Chatellier, P. Modeling and forecasting building energy

consumption: A review of data-driven techniques. Sustain. Cities Soc. 2019,48, 101533. [CrossRef]

13.

Sun, Y.; Haghighat, F.; Fung, B.C. A Review of the-State-of-the-Art in Data-driven Approaches for Building

Energy Prediction. Energy Build. 2020,221, 110022. [CrossRef]

14.

Chou, J.S.; Ngo, N.T. Time series analytics using sliding window metaheuristic optimization-based machine

learning system for identifying building energy consumption patterns. Appl. Energy

2016

,177, 751–770.

[CrossRef]

15.

Nepal, B.; Yamaha, M.; Yokoe, A.; Yamaji, T. Electricity load forecasting using clustering and ARIMA model

for energy management in buildings. Jpn. Archit. Rev. 2020,3, 62–76. [CrossRef]

16.

Moradzadeh, A.; Mansour-Saatloo, A.; Mohammadi-Ivatloo, B.; Anvari-Moghaddam, A. Performance Evaluation

of Two Machine Learning Techniques in Heating and Cooling Loads Forecasting of Residential Buildings. Appl. Sci.

2020,10, 3829. [CrossRef]

17.

Khoshrou, A.; Pauwels, E.J. Short-term scenario-based probabilistic load forecasting: A data-driven approach.

Appl. Energy 2019,238, 1258–1268. [CrossRef]

18.

Runge, J.; Zmeureanu, R. Forecasting energy use in buildings using artiﬁcial neural networks: A review.

Energies 2019,12, 3254. [CrossRef]

Sensors 2020,20, 6525 18 of 20

19.

Cox, S.J.; Kim, D.; Cho, H.; Mago, P. Real time optimal control of district cooling system with thermal energy

storage using neural networks. Appl. Energy 2019,238, 466–480. [CrossRef]

20.

Kim, J.H.; Seong, N.C.; Choi, W. Forecasting the Energy Consumption of an Actual Air Handling Unit and

Absorption Chiller Using ANN Models. Energies 2020,13, 4361. [CrossRef]

21.

Sadeghian Broujeny, R.; Madani, K.; Chebira, A.; Amarger, V.; Hurtard, L. Data-driven living spaces’ heating

dynamics modeling in smart buildings using machine learning-based identiﬁcation. Sensors 2020,20, 1071.

[CrossRef]

22.

Kwak, Y.; Huh, J.H. Development of a method of real-time building energy simulation for efﬁcient

predictive control. Energy Convers. Manag. 2016,113, 220–229. [CrossRef]

23.

Kwak, Y.; Huh, J.H.; Jang, C. Development of a model predictive control framework through real-time

building energy management system data. Appl. Energy 2015,155, 1–13. [CrossRef]

24.

Kampelis, N.; Papayiannis, G.I.; Kolokotsa, D.; Galanis, G.N.; Isidori, D.; Cristalli, C.; Yannacopoulos, A.N.

An Integrated Energy Simulation Model for Buildings. Energies 2020,13, 1170. [CrossRef]

25.

Ghosh, S.; Reece, S.; Rogers, A.; Roberts, S.; Malibari, A.; Jennings, N.R. Modeling the thermal dynamics of

buildings: A latent-force-model-based approach. ACM Trans. Intell. Syst. Technol.

2015

,6, 1–27. [CrossRef]

26.

Gray, F.M.; Schmidt, M. A hybrid approach to thermal building modelling using a combination of Gaussian

processes and grey-box models. Energy Build. 2018,165, 56–63. [CrossRef]

27.

Huang, H.; Chen, L.; Hu, E. A neural network-based multi-zone modelling approach for predictive control

system design in commercial buildings. Energy Build. 2015,97, 86–97. [CrossRef]

28.

Luo, J.; Hong, T.; Fang, S.C. Benchmarking robustness of load forecasting models under data integrity attacks.

Int. J. Forecast. 2018,34, 89–104. [CrossRef]

29.

Zhang, Y.; Lin, F.; Wang, K. Robustness of Short-Term Wind Power Forecasting Against False Data

Injection Attacks. Energies 2020,13, 3780. [CrossRef]

30.

Henze, G. Model predictive control for buildings: A quantum leap? J. Build. Perform. Simul.

2013

,

doi:10.1080/19401493.2013.778519. [CrossRef]

31.

Petersen, S.; Bundgaard, K.W. The effect of weather forecast uncertainty on a predictive control concept for

building systems operation. Appl. Energy 2014,116, 311–321. [CrossRef]

32.

Sandels, C.; Widén, J.; Nordström, L.; Andersson, E. Day-ahead predictions of electricity consumption in

a Swedish ofﬁce building from weather, occupancy, and temporal data. Energy Build.

2015

,108, 279–290.

[CrossRef]

33.

Thieblemont, H.; Haghighat, F.; Ooka, R.; Moreau, A. Predictive control strategies based on weather forecast

in buildings with energy storage system: A review of the state-of-the art. Energy Build.

2017

,153, 485–500.

[CrossRef]

34.

Zhao, J.; Duan, Y.; Liu, X. Uncertainty analysis of weather forecast data for cooling load forecasting based on

the Monte Carlo method. Energies 2018,11, 1900. [CrossRef]

35.

Agüera-Pérez, A.; Palomares-Salas, J.C.; González de la Rosa, J.J.; Florencias-Oliveros, O. Weather forecasts

for microgrid energy management: Review, discussion and recommendations. Appl. Energy

2018

,

228, 265–278. [CrossRef]

36.

Wang, Z.; Hong, T.; Piette, M.A. Building thermal load prediction through shallow machine learning and

deep learning. Appl. Energy 2020,263, 114683. [CrossRef]

37.

Henze, G.P.; Kalz, D.E.; Felsmann, C.; Knabe, G. Impact of forecasting accuracy on predictive optimal control

of active and passive building thermal storage inventory. HVAC R Res. 2004,10, 153–178. [CrossRef]

38.

Oldewurtel, F.; Parisio, A.; Jones, C.N.; Gyalistras, D.; Gwerder, M.; Stauch, V.; Lehmann, B.; Morari, M.

Use of model predictive control and weather forecasts for energy efﬁcient building climate control.

Energy Build. 2012,45, 15–27. [CrossRef]

39.

Hong, T.; Fan, S. Probabilistic electric load forecasting: A tutorial review. Int. J. Forecast.

2016

,32, 914–938.

[CrossRef]

40.

Hong, T.; Pinson, P.; Fan, S.; Zareipour, H.; Troccoli, A.; Hyndman, R.J. Probabilistic energy forecasting:

Global energy forecasting competition 2014 and beyond. Int. J. Forecast. 2016,32, 896–913. [CrossRef]

41.

Gerossier, A.; Girard, R.; Kariniotakis, G.; Michiorri, A. Probabilistic day-ahead forecasting of household

electricity demand. CIRED-Open Access Proc. J. 2017,2017, 2500–2504. [CrossRef]

Sensors 2020,20, 6525 19 of 20

42.

van der Meer, D.W.; Shepero, M.; Svensson, A.; Widén, J.; Munkhammar, J. Probabilistic forecasting of

electricity consumption, photovoltaic power generation and net demand of an individual building using

Gaussian Processes. Appl. Energy 2018,213, 195–207. [CrossRef]

43.

Rouleau, J.; Ramallo-González, A.P.; Gosselin, L.; Blanchet, P.; Natarajan, S. A uniﬁed probabilistic model

for predicting occupancy, domestic hot water use and electricity use in residential buildings. Energy Build.

2019,202, 109375. [CrossRef]

44.

El-Baz, W.; Tzscheutschler, P.; Wagner, U. Day-ahead probabilistic PV generation forecast for buildings

energy management systems. Sol. Energy 2018,171, 478–490. [CrossRef]

45.

Zhao, X.; Liu, J.; Yu, D.; Chang, J. One-day-ahead probabilistic wind speed forecast based on optimized

numerical weather prediction data. Energy Convers. Manag. 2018,164, 560–569. [CrossRef]

46.

Huber, J.; Dann, D.; Weinhardt, C. Probabilistic forecasts of time and energy ﬂexibility in battery electric

vehicle charging. Appl. Energy 2020,262, 114525. [CrossRef]

47.

Yan, X.; Abbes, D.; Francois, B. Uncertainty analysis for day ahead power reserve quantiﬁcation in an urban

microgrid including PV generators. Renew. Energy 2017,106, 288–297. [CrossRef]

48.

Xu, L.; Wang, S.; Tang, R. Probabilistic load forecasting for buildings considering weather forecasting

uncertainty and uncertain peak load. Appl. Energy 2019,237, 180–195. [CrossRef]

49.

Dahl, M.; Brun, A.; Andresen, G.B. Using ensemble weather predictions in district heating operation and

load forecasting. Appl. Energy 2017,193, 455–465. [CrossRef]

50.

Fan, C.; Liao, Y.; Zhou, G.; Zhou, X.; Ding, Y. Improving cooling load prediction reliability for HVAC system

using Monte-Carlo simulation to deal with uncertainties in input variables. Energy Build.

2020

,226, 110372.

[CrossRef]

51.

Luna, A.C.; Meng, L.; Diaz, N.L.; Graells, M.; Vasquez, J.C.; Guerrero, J.M. Online energy management

systems for microgrids: Experimental validation and assessment framework. IEEE Trans. Power Electron.

2017,33, 2201–2215. [CrossRef]

52.

Lamoudi, M.Y.; Béguery, P.; Alamir, M. Use of simulation for the validation of a model predictive

control strategy for energy management in buildings. In Proceedings of the Building Simulation 2011,

11th international IBPSA conference, Sydney, Australia, 14–16 November 2011; pp. 2703–2710.

53.

Lazos, D.; Sproul, A.B.; Kay, M. Optimisation of energy management in commercial buildings with weather

forecasting inputs: A review. Renew. Sustain. Energy Rev. 2014,39, 587–603. [CrossRef]

54.

Crawley, D.B.; Lawrie, L.K.; Winkelmann, F.C.; Buhl, W.F.; Huang, Y.J.; Pedersen, C.O.; Strand, R.K.;

Liesen, R.J.; Fisher, D.E.; Witte, M.J.; et al. EnergyPlus: Creating a new-generation building energy

simulation program. Energy Build. 2001,33, 319–331. [CrossRef]

55.

Crawley, D.B.; Lawrie, L.K.; Pedersen, C.O.; Winkelmann, F.C.; Witte, M.J.; Strand, R.K.; Liesen, R.J.;

Buhl, W.F.; Huang, Y.J.; Henninger, R.H.; et al. EnergyPlus: An update. Proc. Simbuild 2004,1, 1.

56.

DOE, E. Auxiliary Programs: EnergyPlus

TM

Version 8.9.0 Documentation; US Department of Energy:

Washington, DC, USA, 2018.

57.

Romero-Quete, D.; Cañizares, C.A. An afﬁne arithmetic-based energy management system for isolated

microgrids. IEEE Trans. Smart Grid 2018,10, 2989–2998. [CrossRef]

58.

González, V.G.; Colmenares, L.Á.; Fidalgo, J.F.L.; Ruiz, G.R.; Bandera, C.F. Uncertainy’s Indices Assessment

for Calibrated Energy Models. Energies 2019,12, 2096. [CrossRef]

59. Rao, B.P. Nonparametric Function Estimation; Academic Press: London, UK, 1983.

60.

Bashtannyk, D.M.; Hyndman, R.J. Bandwidth selection for kernel conditional density estimation.

Comput. Stat. Data Anal. 2001,36, 279–298. [CrossRef]

61.

Scott, D.W. Multivariate Density Estimation: Theory, Practice, and Visualization; John Wiley & Sons: New York,

NY, USA, 2015.

62.

Shrivastava, N.A.; Panigrahi, B.K. Point and prediction interval estimation for electricity markets with

machine learning techniques and wavelet transforms. Neurocomputing 2013,118, 301–310. [CrossRef]

63.

SABINA SmArt BI-directional multi eNergy gAteway. Available online: https://sabina-project.eu/

(accessed on 20 April 2020).

64.

Guglielmetti, R.; Macumber, D.; Long, N. OpenStudio: An Open Source Integrated Analysis Platform;

Technical Report; National Renewable Energy Laboratory (NREL): Golden, CO, USA, 2011.

65.

Ruiz, G.R.; Bandera, C.F.; Temes, T.G.A.; Gutierrez, A.S.O. Genetic algorithm for building

envelope calibration. Appl. Energy 2016,168, 691–705. [CrossRef]

Sensors 2020,20, 6525 20 of 20

66.

Ruiz, G.R.; Bandera, C.F. Analysis of uncertainty indices used for building envelope calibration. Appl. Energy

2017,185, 82–94. [CrossRef]

67.

Fernández Bandera, C.; Ramos Ruiz, G. Towards a new generation of building envelope calibration. Energies

2017,10, 2102. [CrossRef]

68.

Gutiérrez González, V.; Ramos Ruiz, G.; Fernández Bandera, C. Empirical and Comparative Validation for a

Building Energy Model Calibration Methodologya. Sensors 2020,20, 5003. [CrossRef] [PubMed]

69. Meteoblue. Available online: https://meteoblue.com/ (accessed on 20 April 2020).

70.

Segarra, E.L.; Ruiz, G.R.; González, V.G.; Peppas, A.; Bandera, C.F. Impact Assessment for Building Energy

Models Using Observed vs. Third-Party Weather Data Sets. Sustainability 2020,12, 6788. [CrossRef]

Publisher’s Note:

MDPI stays neutral with regard to jurisdictional claims in published maps and institutional

afﬁliations.

©

2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access

article distributed under the terms and conditions of the Creative Commons Attribution

(CC BY) license (http://creativecommons.org/licenses/by/4.0/).