Content uploaded by Simone Baldi
Author content
All content in this area was uploaded by Simone Baldi on Jan 18, 2020
Content may be subject to copyright.
Passive versus active learning in operation and adaptive
maintenance of Heating, Ventilation, and Air Conditioning
Simone Baldib,a,∗, Fan Zhangc,b, Thuan Le Quangd,a, Petr Endele, Ondrej Holube
aDelft Center for Systems and Control, Delft University of Technology (TU Delft), Delft 2628CD, The Netherlands
bSchool of Mathematics, Southeast University, Jiangsu Province, Nanjing 210096, China
cSchool of Aeronautics and Astronautics (Shenzhen Campus), Sun Yat-sen University, Guangzhou 510275, China
dDepartment of Mathematics, Quy Nhon University, Binh Dinh province, Quy Nhon city, Viet Nam
eHoneywell Prague Laboratory, V Parku 2326/18, 148 00 Prague 4, Czech Republic
Abstract
In smart buildings, the models used for energy management and those used for maintenance scheduling differ in scope and struc-
ture: while the models for energy management describe continuous states (energy, temperature), the models used for maintenance
scheduling describe only a few discrete states (healthy/faulty equipment, and fault typology). In addition, models for energy man-
agement typically assume the Heating, Ventilation, and Air Conditioning (HVAC) equipment to be healthy, whereas the models for
maintenance scheduling are rarely human-centric, i.e. they do not take possible human factors (e.g. discomfort) into account. As a
result, it is very difficult to integrate energy management and maintenance scheduling strategies in an efficient way. In this work, a
holistic framework for energy-aware and comfort-driven maintenance is proposed: energy management and maintenance schedul-
ing are integrated in the same optimization framework. Continuous and discrete states are embedded as hybrid dynamics of the
system, while considering both continuous controls (for energy management) and discrete controls (for maintenance scheduling).
To account for the need to estimate the equipment efficiency online, the solution to the problem is addressed via an adaptive dual
control formulation. We show, via a zone-boiler-radiator simulator, that the best economic cost of the system is achieved by active
learning strategies, in which control interacts with estimation (dual control design).
Keywords: Energy management, maintenance scheduling, adaptive learning-based control, smart buildings.
1. Introduction
The economic cost of buildings is largely dependent on con-
trol and maintenance of Heating, Ventilating and Air Condition-
ing (HVAC) equipment [1]. For example, neglecting any per-
formance degradation or even faults in HVAC will irredeemably
lead to increased costs for facility managers and building own-
ers. While control decisions have a direct impact on energy
consumption [2], literature has shown that the effect of perfor-
mance degradation is more complex and essentially twofold:
firstly, increased consumption of resources in order to compen-
sate for the system inefficiency [3]; secondly, failure to meet
the given set points leading to decreased comfort within the
building (with loss of productivity, complaints, etc.) [4]. This
human-centered impact adds to the cost of maintenance, which
should be scheduled optimally in such a way to minimize the
overall adverse economic effects. In a nutshell, joint control
and predictive maintenance is a complex and largely unsolved
optimization problem involving the joint design of estimation
∗Corresponding author. Tel.: +31 15 2781823
Email addresses: s.baldi@tudelft.nl (Simone Baldi),
zhangfan6@mail.sysu.edu.cn (Fan Zhang),
lequangthuan@qnu.edu.vn (Thuan Le Quang),
petr.endel@Honeywell.com (Petr Endel),
ondrej.holub@Honeywell.com (Ondrej Holub)
and control. In this work, we formulate such problem as a dual
control problem: the term ‘dual’ refers to the twofold action
of the control action, which is in charge of both running the
HVAC system toward optimal performance and of reducing the
uncertainty when estimating HVAC degradation. We address
two types of actions: a continuous action involving the selec-
tion of the HVAC set point (e.g. water temperature set point for
boilers), and a discrete action determining maintenance (e.g.
repairing or not the boiler). Due to the thermostatic mechanism
of HVAC operation, we embed such mechanism in a hybrid dy-
namical system with continuous and discrete dynamics, thus
requiring the solution of a hybrid control problem.
Because it involves the monetization of HVAC performance
and discomfort, the formulation and the solution of such control
problem is relevant and, to the best of the authors’ knowledge,
novel. Previous researches have analyzed smaller aspects of
the global problem, namely: (a) optimization of HVAC energy
consumption and thermal comfort (with no focus on mainte-
nance); (b) performance monitoring, i.e. fault detection and
identification of HVAC equipment (with no focus on schedul-
ing maintenance); (c) scheduling HVAC maintenance (with no
focus on the human-centered impact of decreased comfort). In
the following, we review these three research directions.
Preprint submitted to Applied Energy June 24, 2019
1.1. Related works in optimization of energy/comfort
With respect to optimization of energy consumption and
thermal comfort, several strategies can be found in literature
to predict the effect of changing the control strategy on indoor
comfort [5], or energy consumption [6], or both [7]: typical
criteria driving the optimization include maximizing economy
while satisfying power demand [8], optimizing components siz-
ing [9], maximizing self-consumption [10], balancing natural
ventilation and air conditioning [11], and many more techno-
economic criteria [12] (see also references therein). The terms
‘human-in-the-loop optimization’ [13], or ‘comfort-driven op-
timization’ [14], or ‘ occupancy-based optimization’ [15] are
sometimes adopted, referring to the fact that the energy demand
is ultimately driven by human needs [16]. See also the recent
review [17]. In [18], a data-driven approach for minimization
of HVAC energy consumption and room temperature ramp rate
is presented. Intelligent glazed facades is the subject of [19],
with emphasis on the influence of different control policies on
energy and comfort performance. The authors in [20] apply
particle swarm optimization to optimize the set points based on
some comfort zones. In [21] the operation of variable air vol-
ume HVAC is optimized with respect to comfort and indoor air
quality. The influence of thermostat operation on energy con-
sumption and thermal comfort is studied in [22]; [23] focuses
on integration of multiple HVAC systems, [24] studies how
to optimize simultaneously several HVAC set points, and [25]
studies cooperation among intelligent HVAC systems. Coop-
erative HVAC control has lead to studying the effect of HVAC
operation at the grid level, such as demand response [26] or
other ancillary services [27]. All these approaches show, some-
times also via real-life experiments, that relevant energy savings
can be achieved without compromising thermal comfort. How-
ever, in these and other related works the degradation of HVAC
components is neglected to a large extent: the HVAC system is
assumed to work as good as new, thus neglecting the possible
waste of energy and loss of comfort due to HVAC degradation.
1.2. Related works in performance monitoring
On the other hand, much literature has been focusing on
HVAC performance monitoring, both at a system-level or at
a component-level [28]. System-level approaches describe the
HVAC system as a network of interconnected subsystems [29]:
for every subsystem, a monitoring agent is designed that com-
bines local and transmitted information from its neighboring
agents in order to provide a decision on the type and location of
the faults [30]. In the presence of uncertainty, decisions can be
based on stochastically robust thresholds [31], adaptive thresh-
olds [32], or on state estimation techniques [33]. Centralized
(in place of distributed) strategies are also possible, like the
data-driven automated building HVAC fault detection methods
in [34] and the system identification-based method in [35]. At
a component-level, mainly boilers and air handling units have
been studied. For boilers, in [36] a model was developed to pre-
dict the seasonal efficiency based on the efficiency at full load
evaluated at return water mean temperature. In [37] heat and
mass transfer analytical models of a condensing heat exchanger
system were developed to predict the boiler efficiency accord-
ing to design parameters choices: the model in [38] includes
flue gas outlet temperature, supply water temperature, water
vapor mole fraction, and condensation rate of water vapor. A
dynamic relation between boiler efficiency and state of the heat
exchange can be derived from the model in [39]. In [40] algo-
rithms for real-time monitoring of condensing boilers have been
developed. For air handling units, the work in [41] focuses on
monitoring techniques as part of the on-going commissioning
process. The set of expert rules derived from mass and energy
balances in [42] is able to detect faults in air handling units,
whereas [43] adopts Kalman filtering techniques instead of ex-
pert rules. A detailed overview of fault detection and diagnosis
methodologies on air-handling units is given in [44]. What is
missing in current fault detection and diagnosis methodologies
is a complete monetization analysis taking into account the bal-
ance between costs due to loss of performance and costs due
to maintenance actions. A work partly going in this direction
is [45], which adopts a hybrid approach utilizing expert rules,
performance indexes and statistical process control models: in
this way it is possible to include increased energy consumption
due to HVAC degradation. Summarizing, most works on fault
detection and diagnosis do not investigate the whole economic
aspects of degraded HVAC operation.
1.3. Related works in scheduling maintenance
In the category of maintenance, the authors in [4] develop
commissioning strategies to identify cost-effective operational
and maintenance measures in buildings to bring them up to the
optimum operation. The aim of [46] is to early plan mainte-
nance interventions for a multi-components system based on
stoppages characteristics, system remaining useful life and com-
ponents criticalities. Retrofitting is the focus of [47]. The ap-
proach in [48] focuses on operational and cleaning costs of a
biomass boiler. In [49] the energy and economic performance
of energy recovery ventilators is studied as a function of pa-
rameters such as climate, building design and HVAC system
parameters. An overview of procedures about continuous com-
missioning in office buildings is given in [50]: interestingly, this
work discusses how to select good models not only for mainte-
nance, but also for model-based control. However, rarely these
two aspects are connected into a human-centric (e.g. comfort-
driven) maintenance strategy: notable exceptions are [51], where
it is recognized that discomfort plays an important role in de-
termining when the maintenance is performed, and [52], that
investigates the maintenance characteristics of HVAC system
that affect occupants’ satisfaction. However, what is missing in
these works is recognizing the role of control in reducing un-
certainty (e.g. uncertainty around efficiency parameters). To
clarify this point let us observe this: the use of identification
techniques as in [52] to establish relationships among quanti-
ties, e.g. regression models, is a passive learning method; on
the other hand, the use of the control action to improve the fi-
delity of the regression models while minimizing HVAC oper-
ational costs (dual control action) is an active learning frame-
work, whose formulation and solution is still missing.
2
1.4. Main contribution and originality
In this work we address the gaps in the state of the art
by considering an active learning framework which is relevant
to the maintenance optimization problem. The monetization
model we propose will incorporate in a comprehensive cost
function the operational costs of HVAC equipment subject to
degradation, the human-centered costs of the fault occurring in
the system, and the costs of maintenance actions. The following
points are covered in this work that, to the best of the authors’
knowledge, have not been covered in the state of the art:
•The control, the monitoring, and the maintenance prob-
lems are recast in the same optimization framework via
a dual control formulation (joint design of control and
estimation);
•The thermostat hysteretic behavior is embedded as hy-
brid dynamics of the system (with continuous and dis-
crete states). In addition, both continuous and discrete
control actions are considered;
•Comparisons between passive learning strategies and ac-
tive learning strategies are provided.
In order to keep the optimization problem tractable, assump-
tions and simplifications have been made when describing the
joint control/estimation problem: such assumptions and sim-
plifications, have been studied by the authors in such a way to
retain the main features of the HVAC problem. It is worth men-
tioning that the proposed framework is validated (cf. Sect. 7) on
a zone-boiler-radiator simulation environment studied within
the European Union project ‘Advanced Methods in Building
Diagnostics and Maintenance (AMBI)’ (FP7-PEOPLE-2012-
IAPP - Industry-Academia Partnerships and Pathways).
The rest of the paper is organized as follows: in Section
2 the HVAC and room models are given; in Section 3 the ef-
ficiency model is given, whereas in Section 4 all the continu-
ous/discrete dynamics are recast as a hybrid system. The role
of uncertainty is covered in Section 5, and the proposed adap-
tive approach is in Section 6. Section 7 gives the simulation
results, and Section 8 concludes the work.
Notation: The notation is quite standard as explained in Ta-
ble 1. The subscripts B,R,Zrefer to boiler, radiator, and zone,
respectively. The subscripts rw,sw stand to return and supply
water. The numerical values of such parameters used for simu-
lation purposes are reported in the Appendix.
2. HVAC and room models
In the following we provide the details of the model used
for synthesis of the maintenance strategy. We consider a sin-
gle zone whose HVAC consists of a radiator driven by a boiler.
The model has been selected as a trade-off between depth of
description of the thermal/energy dynamics and computational
feasibility of the maintenance strategy. Dynamics of boiler, ra-
diator and zone are presented in order.
Explanation Symbol Unit
Boiler supply water temperature Tsw [oC]
Boiler return water temperature Trw [oC]
Radiator return water temperature TrwR[oC]
Dew point temperature Tdew [oC]
Room (zone) temperature TZ[oC]
Neighbor room temperature Tn[oC]
Outside temperature To[oC]
Desired temperature Td[oC]
Boiler mass flow rate wwB[kg/s]
Radiator mass flow rate wwR[kg/s]
Shunt mass flow rate wwS[kg/s]
Percentage shunt mass flow rate cS[%]
Input power of boiler (gas side) pin [kW]
Output power of boiler (water side) pout [kW]
Boiler efficiency curve
η
(Trw)-
Specific heat of water cw[kJ/kgoC]
Density of water
ρ
w[kg/m3]
Specific heat of air ca[kJ/kgoC]
Density of air
ρ
a[kg/m3]
Volume of the radiator VR[m3]
Area of the radiator section AR[m2]
Radiator heat transfer coeff. KR[kW/oC]
Convection heat transfer coeff. hR[kW/m2oC]
Room area with outside Ao[m2]
Room area with neighbors An[m2]
Room-to-out. heat transfer coeff. ho[kW/m2oC]
Room-to-room heat transfer coeff. hn[kW/m2oC]
Thermostat hysteretic threshold h[oC]
Valve quantization
δ
-
Boiler degradation rate
α
[s−1]
Forgetting factor
κ
[s−1]
Efficiency parameter
θ
1[oC−1]
Efficiency parameter
θ
2-
Efficiency parameter
θ
3[oC−1]
Variance radiator noise
σ
2
R[kW2]
Variance zone noise
σ
2
Z[kW2]
Variance efficiency noise
σ
2
B-
Variance
θ
1noise
σ
2
1[oC−2]
Variance
θ
2noise
σ
2
2-
Variance
θ
3noise
σ
2
3[oC−2]
Table 1: Table of symbols
2.1. Boiler
We will focus on a condensing boiler which, whenever the
return temperatures from the heating system is below the dew
temperature of the flue gas, can recover the latent heat of wa-
ter vapor in the flue gas so as to achieve higher efficiency than
traditional boilers. Above the dew temperature, no latent heat
is recovered and the boiler will operate in a non-condensing
mode [53]. We assume that the boiler has no dynamics, which
amounts to assuming being in steady-state operation. This is
a reasonable assumption, since most boiler models available
in literature are static models [40]. The input provided to the
3
Figure 1: Condensing boiler efficiency curve. As a condensing boiler can re-
cover the latent heat of water vapor in the flue gas, its efficiency is higher low
water temperature.
boiler is the supply temperature set point Tsw, which determines
the power (on the water side) necessary to reach the set point,
according to
pout =cwwwB(Tsw −Trw)(1)
where cwis the specific heat of water in [kJ/kgoC], wwBis the
boiler mass flow rate in [kg/s], Tsw and Trw are the temperatures
in [oC] of the supply and return water exiting and entering the
boiler, respectively. Let us distinguish between the power on
the water side and the power on the gas side, by calling them
pout and pin, respectively. The output power of the boiler in
[kW] (on the water side) is pout =
η
(Trw)pin , where pin is the
input power to the boiler in [kW] (on the gas side) and
η
(Trw)
is the (dimensionless) efficiency curve depending on Trw (an
example of this curve is shown in Figure 1)1. The boiler mass
flow rate wwBand the firing rate will be assumed to be constant.
2.2. Radiator
The radiator is modeled as a first-order system as follows
cw
ρ
wVR˙
TrwR=
cwwwR(Tsw −TrwR)
+KRTZ−TrwR+Tsw
2+
ξ
Rif VALV E =1
ξ
Rif VALV E =0
(2)
where
ρ
wis the density of the water in [kg/m3], VRis the vol-
ume of the radiator in [m3], wwRis the water mass flow rate
into the radiator in [kg/s], KR=hRARis the heat transfer coef-
ficient in [kW/oC], where ARis the surface area of the radiator
and hRis the convection heat transfer coefficient of the radiator
in [kW/m2oC]. In addition, TrwRrepresents the temperature in
[oC] of the return water out of the radiator, and TZis the temper-
ature in [oC] of the air in the zone. The quantity
ξ
Rrepresents
a stochastic process noise. According to (2), the heat exchange
with the room occurs via the difference between the zone tem-
perature and the radiator mean temperature.
1The reason which the x-axis in Figure 1 is given in Fahrenheits, is that most
boiler manufactures provide the efficiency curve with this unit. Nevertheless,
all the calculations in this work are carried out in Celsius.
Figure 2: Radiator shunt. The radiator operation depends on its valve which in
turn is open/closed via the thermostatic control.
Assuming a heating setting (late fall or winter), we model
the thermostatic control in the radiator as follows:
VALV E =
1 if ˙
TZ>0 and TZ≤Td+hor
if ˙
TZ≤0 and TZ≤Td−h
0 otherwise
(3)
where CLOSED =0, OPEN =1. In other words, the valve can
be fully open or fully closed, leading to raising or decreasing
temperature (thermostat hysteretic behavior).
Remark 1. It is also possible to model multiple valve positions
(not only open or closed), at the expense of complicating the
model, i.e. VALVE =
1 if ˙
TZ>0 and Td+(
δ
−1)h/
δ
≤TZ≤Td+hor
if ˙
TZ≤0 and Td−h≤TZ≤Td−(
δ
−1)h/
δ
.
.
.
2/
δ
if ˙
T>0 and Td+h/
δ
≤TZ≤Td+2h/
δ
or
if ˙
T≤0 and Td−2h/
δ
≤TZ≤Td−h/
δ
1/
δ
if ˙
T>0 and Td≤TZ≤Td+h/
δ
or
if ˙
T≤0 and Td−h/
δ
≤TZ≤Td
0 otherwise
(4)
where
δ
is the discretization step of the valve. This model is not
considered to keep the control setting as simple as possible.
The radiator receives water from the boiler: the presence of
a radiator shunt splits the mass flow rate as
wwS=wwB−wwR
wwS= (1−VALV E(t) + cSV ALV E (t))wwB
wwR= (1−cS)VALV E (t)wwB(5)
where wwSis the water mass flow rate of into the shunt in [kg/s]
and cSis the minimum percentage of flow circulating in the
shunt. In other words, even if the radiator valve is fully open,
a mass flow rate cSwwBwill circulate in the shunt. As shown in
Figure 2, the return water to the boiler is given by the mixing
between the return water from the radiator and the supply water,
Trw =wwRTrwR+wwSTsw
wwB
(6)
4
2.3. Zone
The zone is modeled as a first-order system interacting with
the outside air, with the neighbor zones, and with the radiator
ca
ρ
aVZ˙
TZ=
hoAo(To−TZ)+ hnAn(Tn−TZ)
+KRTrwR+Tsw
2−TZ+
ξ
Zif VALV E =1
hoAo(To−TZ)
+hnAn(Tn−TZ)+
ξ
Zif VALV E =0
(7)
where
ρ
ais the density of the air in [kg/m3], VZis the volume
of the zone in [m3], cais the specific heat of air in [kJ/kg oC],
hoand hnare the convection heat transfer coefficient in [kW/m2
oC] of the indoor air with outside and neighbor zones, respec-
tively, Aoand Anare the surface in [m2] of the zone with out-
side and neighbor zones, respectively. In addition, Toand Tnare
the temperatures [oC] of the outside and neighbor zone air, re-
spectively, while the quantity
ξ
Zrepresents a stochastic process
noise.
3. Boiler efficiency
The efficiency of the boiler is approximated as a piecewise
affine function of Trw , similarly to what shown in Figure 3. The
approximation range is 30-71 oC (corresponding to 90-160 oF):
η
(Trw) = ¯
θ
1Trw +¯
θ
2if Trw ≤Tdew
¯
θ
3Trw +¯
θ
4if Trw >Tdew
(8)
with ¯
θ
1Tdew +¯
θ
2=¯
θ
3Tdew +¯
θ
4for continuity of the efficiency
curve. In order to reduce the number of parameters from four
to three, we explicitly make use of the continuity condition, so
that the previous expression can be written as
η
(Trw) =
θ
1Trw +
θ
2if Trw ≤Tdew
θ
1Trw +
θ
2+
θ
3(Trw −Tdew)if Trw >Td ew
(9)
The curve (9) has not only less parameters, but it is also con-
tinuous by construction. The dew point Tdew is the temperature
at which the condensing process will occur: as commonly done
by all boiler manufacturers, this temperature is given in terms
of the return water temperature even if, from a physical point of
view, it should be calculated in terms of the flue gas temperature
[39]. The dew point is commonly in the range Trw ≈54-58oC
(slightly depending on the flue gas composition). The tempera-
ture of the return water Tr w is calculated as in (6). In the follow-
ing, it is described how the parameters of the curve (9) change
with time as a consequence of performance degradation.
3.1. Boiler degradation and maintenance
In order to include performance degradation we consider
the following multiplicative degradation
η
(Trw)(t) = (1−d(t))
η
(Trw)(0)(10)
where d(t)∈[0,1]indicates the level of degradation from ‘new’
(d(t) = 0)to ‘completely faulty’(d(t) = 1)condensing boiler.
Figure 3: Piecewise affine approximation of the efficiency curve (in the range
90-160 oF)
The multiplicative degradation (10) is used to model the delete-
rious effects of processes like deposition, erosion and corrosion.
The formulation (10) leads to the degradation model
η
(Trw)(t) =
θ
1(1−d(t))Trw +
θ
2(1−d(t)) if Trw ≤Tdew
θ
1(1−d(t))Trw +
θ
2(1−d(t))
+
θ
3(1−d(t))(Trw −Tdew )if Tr w >Td ew
(11)
Remark 2. In other words, the linear-in-the-parameter efficiency
model (9) leads to a degradation model (11) whose parameters
θ
1(1−d(t)),
θ
2(1−d(t)),
θ
3(1−d(t)) evolve linearly with the
degradation d(t). So, these parameters (which are typically un-
known) can be estimated using standard state estimation tech-
niques (cf. Section 5).
We assume an exponential incipient degradation
d(t) = 1−e−
α
t(12)
where
α
>0 is the degradation rate, i.e. the rate of deposi-
tion, erosion and corrosion deteriorating the efficiency of the
boiler. The relation (12) implies that the half-life of the boiler,
i.e. the time necessary for the efficiency to fall to one half of its
initial value is
α
ln(2). Most condensing boilers have half-life
time constant of several months or a few years. The exponential
incipient degradation (12) results in the following degradation
model for the parameters
˙
θ
1(t) = −
αθ
1(t) +
ζ
1,
θ
1(0) =
θ
1new
˙
θ
2(t) = −
αθ
2(t) +
ζ
2,
θ
2(0) =
θ
2new
˙
θ
3(t) = −
αθ
3(t) +
ζ
3,
θ
3(0) =
θ
3new (13)
which is a set of first-order filters driven by stochastic noises.
The parameters
θ
1new ,
θ
2new and
θ
3new describe the efficiency for
a new boiler.
Remark 3. The stochastic noises
ζ
1,
ζ
2,
ζ
3account for model
inaccuracies, for example if the degradation is not exactly ex-
ponential. Similarly to
ζ
Rand
ζ
Z, by setting appropriate co-
variances for such disturbances, the designer will be able to
set to which extent the model is an approximation of the ac-
tual system (the larger the covariance, the larger the modelling
inaccuracies).
5
3.2. Actions
Two possible actions, namely control and maintenance ac-
tions, can be taken on the system:
•The first type of action is the local continuous control,
i.e. setting the set point for building equipment: in the
case at hand, this amounts to properly setting the supply
hot water temperature set point Tsw for the boiler.
•The second type of action is the maintenance discrete
action, i.e. the repair at a certain time ¯
tof the build-
ing equipment. In this work we consider an ideal repair
restoring its performance to the initial performance:
θ
1(¯
t) =
θ
1new
θ
2(¯
t) =
θ
2new
θ
3(¯
t) =
θ
3new
if x(¯
t)∈MAINT (14)
where xand MAINT are the variables and the set to be
used by the maintenance strategy, as they will be defined
later. In other words, whenever it occurs, the mainte-
nance action is supposed to restore the state of the boiler
to its initial value
θ
new = [
θ
1new
θ
2new
θ
3new ]′.
4. Automaton formulation
The boiler-radiator-zone system can be described by a par-
ticular class of stochastic hybrid system. A hybrid dynamical
system is an indexed collection of dynamical systems along
with some map for jumping among them (switching dynamical
system and/or resetting the state). This jumping occurs when-
ever the state satisfies certain conditions, given by its member-
ship in a specified subset of the state space. The hybrid dynam-
ical system can be described as
Hc= [Q,Σ,A,G,V,C,F](15)
with constituent parts as follows
•Qis the set of index states or discrete states.
•Σ=Σqq∈Qis the collection of controlled dynamical
systems, where each Σq= [Xq,fq,Uq]is a controlled dy-
namical system. Here, Xqare the continuous state spaces,
and fqare the continuous dynamics; Uqis the set of con-
tinuous controls.
•A=Aqq∈Q,Aq⊂Xqfor each q∈Q, is the collection
of autonomous jump sets.
•G=Gqq∈Q, where Gq:Aq×Vq→Sis the autonomous
jump transition map, parameterized by the transition con-
trol set Vq, a subset of the collection V=Vqq∈Q; they
are said to represent the discrete dynamics and controls.
•C=Cqq∈Q,Cq⊂Xqis the collection of controlled
jump sets.
•F=Fqq∈Q, where Fq:Cq→2Swhere is the collec-
tion of controlled jump destination maps.
Figure 4: Automaton associated to the joint energy/maintenance problem: Tit
comprises two regimes (on/off) driven by the thermostat. Each regime con-
tains continuous dynamics for evolution of temperature and degradation. The
maintenance action is a discrete control that restores the efficiency of the boiler.
•Finally, S=∪q∈QXq×{q}is the hybrid state space of
the dynamical system.
For the problem at hand we have:
•The continuous state space Xqarises from the room tem-
perature (7), radiator return water temperature (2) (which
can be observed) and the boiler performance parameters
(13) (which have to be estimated);
•The discrete state space Qarises from the states of the
valve (closed/open state {0,1}or, in case of valve with
quantized openings, states {0,1/
δ
,...,1});
•The autonomous transitions Aare driven by the ther-
mostat behavior (3) or (4). This defines the map G: the
vector field changes regime, when the state (room tem-
perature) hits the hysteresis boundaries;
•The controlled transitions Care driven by the repair of
the boiler (11). This defines the map F: the continu-
ous state (boiler performance parameters) changes im-
pulsively on hitting prescribed regions of the state space
(maintenance action region).
The hybrid dynamical system can be represented as an au-
tomaton as in Figure 4: note that the room temperature and
radiator return water temperature evolve continuously, but the
performance parameter evolve discontinuously after repair. The
different regimes of the automaton are formally defined as:
Regime valve open
cw
ρ
wVR˙
TrwR=cwwwR(Tsw −TrwR)
+KRTZ−TrwR+Tsw
2+
ξ
R
ca
ρ
aVZ˙
TZ=hoAo(To−TZ) + hnAn(Tn−TZ)
+KRTrwR+Tsw
2−TZ+
ξ
Z
˙
θ
1(t) = −
αθ
1(t) +
ζ
1
˙
θ
2(t) = −
αθ
2(t) +
ζ
2
˙
θ
3(t) = −
αθ
3(t) +
ζ
3(16)
6
Regime valve closed
cw
ρ
wVR˙
TrwR=
ξ
R
ca
ρ
aVZ˙
TZ=hoAo(To−TZ) + hnAn(Tn−TZ) +
ξ
Z
˙
θ
1(t) = −
αθ
1(t) +
ζ
1
˙
θ
2(t) = −
αθ
2(t) +
ζ
2
˙
θ
3(t) = −
αθ
3(t) +
ζ
3(17)
In addition, some limits are selected taking into account typ-
ical operating conditions:
Operating conditions
θ
1≤0,
θ
2≥0,
θ
3≤0
30 ≤Trw ≤70
40 ≤Tsw ≤85
Tz,Tn>To,Tsw >Trw (18)
where the constraints have been selected taking into account
typical operating conditions. It is clear that the control actions
influence the transitions and thus the behavior of the hybrid dy-
namical system. In the following we will introduce a cost to
quantify the performance associated to a certain behavior.
4.1. Cost
The operation of the automaton presented in Figure 4 must
be optimized taking into account the following cost:
Ctot (t) = Coper (t) +Cf ail (t) + Cmaint(t) + Ccycl (t).(19)
The four terms are all monetized in AC, as explained in the fol-
lowing.
•Operational costs Coper : the first term in (19) is related
to the costs of system operation, which in most cases is
simply the cost of energy consumed by the HVAC system
for given time step as function of observations and inputs.
In the boiler case, the energy consumption is given by pin,
i.e. the energy (at the gas side) necessary to reach the set-
point temperature, based on the efficiency of the boiler.
As the boiler degrades its performance, more and more
energy will be necessary to achieve the same set point
(for the same return water temperature). The economic
value of this term is derived from the natural gas price
statistics in EU-28 [54], which is 0.07 AC/kWh
Coper (t) = 0.07pin(t)(20)
•Failure costs Cf ail : the second term in (19) is related to
the costs due to improper behavior of the system, which
in our case is not following the desired temperature Td.
We focus on zone set points, since in building domain
these are crucial constraints to occupant comfort and thus
productivity. It has been estimated by the Federation
of European Heating, Ventilation and Air Conditioning
(REHVA) that improving indoor environment in office
buildings would result in a direct increase in productivity
of 0.5% to 5%: reduction in performance is around 4% at
cooler temperatures and 6% at warmer temperatures [55].
To the purpose of this study, the failure cost is taken as
the squared distance from the desired temperature
Cf ail (t) = 0.2(Td(t)−TZ(t))2/dt (21)
where dt is the sample time. The following estimate is
made: losses of 0.2 AC per sample time for one degree far
from the desired one and 0.8 AC per sample time for two
degrees far from the desired one, and so on.
•Maintenance costs Cmaint : the third term in (19) is related
to the costs of maintenance actions. We will focus on the
maintenance action of reparation.
Cmaint(t) = 500AC at controlled jumps (22)
as the cost of condensing boiler reparation vary in the
range 500 - 2000 AC depending on the brand or output
[56].
•Cycling costs Ccycl : finally, one last term should be con-
sidered in (19) mainly due to well-posedness reasons
Ccycl(t) = 0.15AC at autonomous jumps (23)
In fact, in order to have a well-posed formulation with no
chattering phenomena (high frequency control actions),
it is necessary to penalize any transition caused by au-
tonomous switches (i.e. changes in the valve). Roughly
speaking, assigning a cost to such switches has an inter-
pretation in terms of avoiding fast cycling, which could
potentially wear out the equipment.
5. Uncertainty
Let us specify which coefficients can be measured and which
ones must be estimated. The known coefficients are:
•The thermostat hysteretic threshold h;
•Properties of fluids (density and heat capacitance of air
and water);
•Volumes of boiler, radiator and zone;
•The heat transfer coefficients;
•The exponential decay of the boiler efficiency
α
;
Uncertainty arises from the unknown coefficients in the boiler
efficiency curve (the efficiency of the boiler cannot be known
perfectly and it is thus subject to uncertainty):
•The coefficients
θ
1,
θ
2and
θ
3are unknown and must be
estimated.
Let us consider a least-squares estimator for the following
linear-in-the-parameter model
pin
η
(Trw) = cwwwB(Tsw −Trw)(27)
7
where pin is assumed to be measured (from gas side measure-
ments). The least-squares estimator takes the form
˙
¯
θ
=P
θεφ
˙
P
θ
=
κ
P
θ
−P
θφφ
′P
θ
(28)
where ¯
θ
= [
θ
1
θ
2
θ
3]′,
ε
=pin ˆ
η
(Trw)−cwwwB(Tsw −Trw)is the
estimation error, ˆ
η
(Trw)is the estimate of
η
(Trw),
κ
>0 is a
forgetting factor,
φ
= [Trw 1Trw −Tdew]′is the regressor vector,
and P
θ
is the covariance matrix of the uncertainty. In order to
take into account the error arising from neglecting the boiler
transient, the following stochastic model is taken
η
(Trw) = cwwwB(Tsw −Tr w)
pin
+
ζ
B(29)
where
ζ
Bis a stochastic noise. Because the management/ main-
tenance algorithm is ultimately implemented on a digital con-
troller, we consider the estimator in discrete time in place of
(28). After discretization of (13) using backward Euler and
sample time dt, the parameters in
η
(Trw)are estimated using
the stochastic Kalman filter at the end of the page.
Remark 4. It has to be noted that, despite the fact that the con-
densing boiler operates in two modes, a unique estimate ¯
θ
and
a unique covariance matrix P
θ
are updated. Because
θ
1and
θ
2are shared in both modes, but
θ
3can be observed only in
the non-condensing mode, the following estimation strategy is
adopted:
•in the non-condensing mode, full-order update (for the
three components from measurements) and full-order pre-
diction (based on dynamics for the three components) is
performed;
•in the non-condensing mode, reduced-order update (for
the first two components from measurements) and full-
order prediction (based on dynamics for the three com-
ponents) is performed.
This amounts to assuming that when
θ
3is not observed its esti-
mate is only based on the evolution of its dynamics. In fact,
θ
3will decay exponentially also when it is not observed (in
the condensing mode). Also, every time a maintenance oc-
curs, we reset the covariance matrix P
θ
to some initial value,
i.e. P
θ
(¯
t) = P
θ
new , where ¯
t is the same instant as in (14). This
is done in order to reset the a priori knowledge of the Kalman
estimator.
6. Optimization approach
The idea is to adopt the unified framework for optimal con-
trol in hybrid systems [57, 58]. The following total discounted
cost is considered
Z∞
0
e−
λτ
Coper (
τ
) + Cfail (
τ
)d
τ
+∑
i
e−
λ
tiCcycl(ti) + ∑
i
e−
λ
¯
tiCmaint(¯
ti)(30)
where
λ
is the discounting factor for the future cost. The deci-
sion variables over which this cost has to be minimized are: the
continuous control Tsw (boiler set point), and the maintenance
strategy, which will both be defined in the following.
According to the way the automaton in Fig. 4 is operated,
the cost (30) can be closer to or more distant from the optimum.
It is clear that the continuous control Tsw (supply water temper-
ature) is a crucial parameter in operating the automaton. Let
us define a simple base case strategy for operating the supply
water temperature:
Tsw(t) =
Trw if V ALV E =0 or
if VALV E =1 and TZ>Td
Trw +kp(Td−TZ)if V ALV E =1 and TZ≤Td
(31)
with kp>0 a proportional gain to be designed. The strategy
(31) amounts to setting the boiler off when the radiator valve is
Backward Euler discretization:
Degradation
θ
1(k+1) =
θ
1(k) + dt
ζ
1
1+dt
α
,
θ
2(k+1) =
θ
2(k) + dt
ζ
2
1+dt
α
,
θ
3(k+1) =
θ
3(k) + dt
ζ
3
1+dt
α
(24)
Update (from measurements):
Innovation or measurement residual
ε
=z−
φ
′
θ
k|k−1
Innovation (or residual)covariance S=
φ
′P
θ
k|k−1
φ
+
σ
B
Optimal Kalman gain K=P
θ
k|k−1
φ
S−1
Updated (a posteriori)state estimate
θ
k|k=
θ
k|k−1+K
ε
Updated (a posteriori)estimate covariance P
θ
k|k= (I−K
φ
′)P
θ
k|k−1(25)
Predict (based on dynamics):
Predicted (a priori)state estimate
θ
k+1|k=A
θ
k|k
Predicted (a priori)estimate covariance P
θ
k+1|k=AP
θ
k|kA′+E′
σ
2
DE(26)
where
σ
2
Bis the covariance of
ζ
B,
σ
2
Dis the covariance of
ζ
D= [
ζ
1
ζ
2
ζ
3],A=1
1+dt
α
I3and E=dt
1+dt
α
I3.
8
Table 2: Summary of the tasks to be accomplished (energy management and maintenance schedule) and of the policies to accomplish them (certainty equivance,
cautious and dual policy).
Task Type of control Required measurements Variable Optimization parameter
Energy
management
Continuous (water
temperature)
Room temperature
Supply water temperature
Hot water
supply temperature
Proportional
gain kp
Maintenance
scheduling
Discrete (maintain
- not maintain)
Boiler input power (gas)
Return water temperature
Minimum allowed
efficiency
Degradation
threshold rth
Policy Energy management Maintenance scheduling Type of learning
Certainty
equival.
Strategy (32): nothing is done
to reduce estimation error
Strategy (33): deals with estimated
efficiency as the true one Passive
Cautious Strategy (32): nothing is done
to reduce estimation error
Strategy (35): uncertainty bound is
added to estimated efficiency Passive
Dual Strategy (37): tries to reduce
estimation error via probing
Strategy (35): uncertainty bound is
added to estimated efficiency Active
closed or when the valve is open but the temperature is above
the desired temperature (note that when Tsw =Trw then pin =0).
When the valve is open and the temperature of the zone is below
the desired temperature, the supply water set point is increased
proportionally to the difference Td−TZ.
In order to reformulate (31) as a function of the continuous
state (TrwR,TZ), we equivalently rewrite it as
Tsw(t) =
TrwRif V ALV E =0 or
if VALV E =1 and TZ>Td
wwRTrwR+wwBkp(Td−TZ)
wwR
if VALV E =1 and TZ≤Td
(32)
For the maintenance strategy, a simple idea is to perform main-
tenance once the efficiency falls below a certain percentage r%
of the efficiency of a new boiler. However, in view of uncer-
tainty in the parameters
θ
1,
θ
2and
θ
3, the efficiency must be
estimated. Therefore, we propose two different strategies de-
pending on how the estimated efficiency is used.
6.1. Certainty equivalence strategy
The simplest idea for maintenance strategy is to perform
maintenance once the estimated efficiency falls below a cer-
tain percentage rth% of the efficiency of a new boiler. In other
words, in this certainty equivalence framework the control ac-
tion is calculated as if the estimate ˆ
θ
(k)were exact. This amounts
to neglecting any uncertainty in the estimate
MAI NT (t) = (1 if ˆ
η
(Trw)(t)
η
new(Trw)≤rth
0 otherwise (33)
where
η
new(Trw)is the efficiency curve of a new boiler (with
parameters
θ
1new ,
θ
2new and
θ
3new ), and it has to be noted that we
take into account that the efficiency curve depends on Trw .
Remark 5. Two things must be noticed. The first one is that
it is not difficult to include the firing rate, call it f r in the ef-
ficiency model, i.e.
η
(Trw,f r ): this can be simply achieved by
a linear in the parameter model not only with respect to Trw ,
but with respect to Trw and f r. In fact, most efficiency curves
for boilers, heat pumps etc are given as linear in the param-
eter models with respect to two or more parameters (cf. [23]
for more details). The second one is that, according to (14), we
have x = [Trw
θ
1
θ
2
θ
3]′and that the strategy MAINT is defined
by the parameters Td ew, rth ,
θ
1new ,
θ
2new and
θ
3new .
6.2. Cautious strategy
The maintenance strategy (33) does not take into account
any uncertainty in the estimation of the efficiency. A simple
design to take into account the uncertainty is a ‘cautious’ con-
trol action that adds a measure of caution depending on the un-
certainty: to this purpose we first define the covariance of the
efficiency
σ
2
η
(t) =
φ
′(t)P
θ
(t)
φ
(t)(34)
and then we define the ‘cautious’ threshold
MAI NT (t) = (1 if ˆ
η
(Trw)(t)−3
ση
(t)
η
new(Trw)≤rth
0 otherwise (35)
In other words, for the same threshold rth , the cautions con-
troller will tend to do maintenance more often if
ση
is large.
Basically, we notice that the numerator decreases if the un-
certainty associated to ˆ
η
(Trw)increases. The uncertainty on
ˆ
η
(Trw)is measured as the square root of the covariance
σ
2
η
(t).
If
ση
reduces to zero, then the cautious strategy converges to
the certainty equivalence strategy. Therefore, a crucial question
arises: is it possible to actively reduce uncertainty by means of
the control action? The next strategy tries to address this ques-
tion.
6.3. Dual strategy
Both the certainty equivalence and the cautious strategy are
adaptive because they depend on an efficiency curve that is
estimated, and thus adapted, online. However, both the cer-
tainty equivalence and the cautious strategies are passive learn-
ing policies because they do not involve any active probing sig-
nal generated to improve the estimation of the efficiency curve.
9
Figure 5: Features of the zone-boiler-radiator test case used to test the proposed framework. The building test case (of around 150 m2) contains a boiler driving a
zone with radiators. The boiler uses a proportional controller to set the water supply temperature, while the thermostatic controller determines on and off regimes.
The efficiency of the boiler degrades with time, so that maintenance is needed. The flow diagrams of energy/maintenance controls are in the middle. The overall
energy/maintenance scenario can be managed according to three policies (certainty equivalence/cautious/dual).
In the following, we want to create a more active learning mech-
anism. Let us define the proportional gain:
¯
kp=kp+3k
σση
,kp,k
σ
>0 (36)
In other words, the control gain increases if the uncertainty in
ˆ
η
(Trw)increases. This is because when the uncertainty is large,
larger control actions might help in reducing the estimation er-
ror (and thus in reducing
σ
2
η
(t)). It has to be remarked that pin
enters the linear-in-the-parameters model, thus the selection of
Tsw has an effect on reducing the uncertainty. Note that the op-
posite mechanism (the gain ¯
kpis decreased as the uncertainty is
increased) is not desirable: since the effect of a control action
on estimation is not taken into account, this can lead to turning
off the controller if the uncertainty becomes too large.
In view of these considerations, the dual strategy becomes
Tsw(t) =
TrwRif V ALV E =0 or
if VALV E =1 and TZ>Td
wwRTrwR+wwB¯
kp(Td−TZ)
wwR
if VALV E =1 and TZ≤Td
(37)
Summarizing, the proposed strategies are illustrated in Table 2:
their performance will be compared via numerical simulations.
7. Simulation experiments
Simulation experiments are performed on a ‘smart building’
simulator developed by the authors. The simulator environ-
ment is based on the zone-boiler-radiator dynamics described
in the previous sections, implemented in Matlab in a similar
way as previously done by some of the authors in [23]. A vi-
sualization of the features of the simulator can be seen in Fig.
5, with the available measurements, the flow diagrams of en-
ergy/maintenance controls and the different policies. The sim-
ulator also comprises a few additional modules, such as ther-
mostat features and testing criteria which are proprietary and
0 100 200 300 400 500 600 700 800 900
TO [o C]
-20
-10
0
10
20
time [h]
0 100 200 300 400 500 600 700 800 900
SO [kW/m2]
0
0.2
0.4
0.6
0.8
1
Figure 6: Weather data (outside temperature and solar radiation) during 36 win-
ter days. The data are used to drive the ‘smart building’ simulator.
cannot be disclosed due to intellectual property agreement. The
weather data used for the simulations represent outside temper-
ature and solar radiation for 36 days; as it can be seen from the
trend of the weather, initially we have a quite rigid winter that
evolves into a milder one. In order to consider a longer sim-
ulation horizon, we repeated these 36 days for 60 times, with
random perturbations on the values. In this way we are able to
simulate around 2200 days of winter season. The simulations
are run to optimize the following parameters:
•For the certainty equivalence strategy: kpand rth ;
•For the cautious strategy: kpand rth ;
•For the dual strategy: kp,k
σ
and rth.
10
50
40
30
kp
20
10
0
0
0.2
rth
0.4
1.5
0
1
2
0.5
0.6
Cost/h
Figure 7: Certainty equivalence strategy: Average hourly cost as a function of
kpand rth .
50
40
30
kp
20
10
0
0
0.2
rth
0.4
0
0.5
1
1.5
2
0.6
Cost/h
Figure 8: Cautious strategy: Average hourly cost as a function of kpand rth.
50
40
30
kp
20
10
0
0
0.2
rth
0.4
1.2
1
0.8
0.6
0.4
1.6
1.4
0.6
Cost/h
Figure 9: Dual strategy: Average hourly cost as a function of kpand rth (with
k
σ
=0.5).
0.8
0.6
Oper
0.4
0.2
0
0
0.5
Fail
1
0.35
0.05
0.1
0.15
0.2
0.25
0.3
1.5
Maint
Figure 10: Certainty equivalence strategy: Pareto front for Coper,Cf ail and
Cmaint.
0.5
0.4
0.3
Oper
0.2
0.1
0
0
0.5
Fail
1
0.3
0.2
0.1
0
0.4
0.6
0.5
1.5
Maint
Figure 11: Cautious strategy: Pareto front for Coper,Cf ail and Cmaint .
0.5
0.4
0.3
Oper
0.2
0.1
0
0
0.5
Fail
1
0
0.4
0.3
0.6
0.2
0.1
0.5
1.5
Maint
Figure 12: Dual strategy: Pareto front for Coper ,Cfail and Cmaint.
11
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Tsw, Trw [ o C]
40
60
80
time [days]
0 200 400 600 800 1000 1200 1400 1600 1800 2000
pin [kW]
0
10
20
30
Average pin =4.86 kW/h
Figure 13: Certainty equivalence strategy: water temperatures and boiler power.
The average power input is 4.86 kW/h.
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Tsw, Trw [ o C]
40
60
80
time [days]
0 200 400 600 800 1000 1200 1400 1600 1800 2000
pin [kW]
0
10
20
30
Average pin =4.75 kW/h
Figure 14: Cautious strategy: water temperatures and boiler power. The aver-
age power input is 4.75 kW/h.
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Tsw, Trw [ o C]
40
60
80
time [days]
0 200 400 600 800 1000 1200 1400 1600 1800 2000
pin [kW]
0
10
20
30
Average pin =4.10 kW/h
Figure 15: Dual strategy: water temperatures and boiler power. The average
power input is 4.10 kW/h.
0 200 400 600 800 1000 1200 1400 1600 1800 2000
θ1, θ2, θ3
-0.5
0
0.5
time [days]
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Trace Pθ
×10-3
0
0.5
1
1.5
Average error = 5.91× 10-4
Figure 16: Certainty equivalence strategy: estimation of efficiency parameters.
The average estimation error is 5.91·10−4(adimensional).
0 200 400 600 800 1000 1200 1400 1600 1800 2000
θ1, θ2, θ3
-0.5
0
0.5
time [days]
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Trace Pθ
×10-3
0
0.5
1
1.5
Average error = 3.98× 10-4
Figure 17: Cautious strategy: estimation of efficiency parameters. The average
estimation error is 3.98·10−4(adimensional).
0 200 400 600 800 1000 1200 1400 1600 1800 2000
θ1, θ2, θ3
-0.5
0
0.5
time [days]
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Trace Pθ
×10-3
0
0.5
1
1.5
Average error = 1.97× 10-4
Figure 18: Dual strategy: estimation of efficiency parameters. The average
estimation error is 1.97·10−4(adimensional).
12
Because of the low number of parameters, we can optimize
the parameters using a brute force approach over a grid. The
following initial grid has been chosen for optimization
kp: 0.1,0.25,0.5,1,2,5,10,20,50
k
σ
: 0.1,0.25,0.5,1,2.5
rth : 0.1,0.15,0.2,0.25,0.3,0.35,0.4,0.45,0.5,0.55
(38)
whose meaning is the following: the proportional gain kpis
selected from low gain (shallow control) to high gain (aggres-
sive control); the threshold rth goes from 10% degradation to
55% degradation with steps of 5%; finally, the probing gain k
σ
goes from low probing to high probing. All experiments run
on a Dell OptiPlex 7060 MT, Intel Core i5-8500, 8GB RAM
DDR4, Windows 10. The simulation platform is developed
in Matlab R2016b. Spanning the complete grid (450 possible
policies) takes around 4 hours (around half a minute to simu-
late one policy over a 2200 day horizon). Then, when a rough
estimate of the optimal point has been found, the grid can be
refined and reduced in size to further improve the performance.
Spanning this smaller grid takes around 30 minutes. Therefore,
we verified that the computational complexity of the proposed
approach is relatively low, and it is due to the fact that one pol-
icy can be simulated over a 2200 day horizon in less than half
a minute. Key to such a fast simulation is the relatively sim-
ple nature of the proposed hybrid modelling, which can signifi-
cantly simplify the model while still retaining the main features
of the HVAC maintenance problem. We believe that the pro-
posed hybrid modelling reaches quite a good trade-off in terms
of realistic maintenance scenario and complexity of the formu-
lation. It is inevitable that, when increasing the number of states
and actions, the proposed optimization would become more and
more complex and cumbersome: such an issue can be studied
in future work.
The results of the optimization are shown in Table 3, where
it is shown that the dual strategy can improve the cost of around
0.02 AC/h with respect to the certainty equivalence strategy, and
of around 0.01 AC/h with respect to the cautious strategy (note
that this amounts to around 175 AC/year and 87.5 AC/year, which
are non negligible savings for a zone of 150 m2). The first thing
to be noticed is that apparently the certainty equivalence strat-
egy is not the best strategy to be adopted: this is because the
controller is designed independently from the estimator, there-
fore learning is passive and uncertainty leads to non optimal
maintenance decisions. By selecting a lower threshold, the cau-
tious strategy adds some caution in the maintenance which helps
in improving the cost: however, even in this case the controller
Table 3: Parameters and costs for each strategy
kprth k
σ
Hourly cost
Certainty 12 0.32 - 0.494 AC/h
Cautious 5 0.17 - 0.486 AC/h
Dual 6 0.24 0.5 0.475 AC/h
is designed independently from the estimator, therefore learn-
ing is passive. The dual strategy is the one for which the con-
troller is co-designed with the estimator (active learning): this
dual role of control is apparently really necessary to improve
the performance even more. This is clearly an interesting re-
sult, as it shows that passive learning (certainty equivalence or
cautious strategies) on the long run does not pay. Reducing un-
certainty (active learning) can lead to economic benefits on the
long run. This might stimulate future study and implementation
of active learning policies.
In Figs. 7, 8 and 9 we plot how average hourly cost depends
on the parameters kpand rth. Figs. 10, 11 and 12 show how
the different components in the cost (Coper ,Cf ail and Cmaint)
change by changing the parameters. It can be seen that a Pareto
front exists in all cases: there are tradeoffs regarding optimiza-
tion of the different costs. In order to better investigate such
Pareto from, let us compare Figs. 17 and 14 (cautious strat-
egy), with Figs. 18 and 15 (dual strategy). As indicated in
Figs. 13, 14 and 15, it can be seen that the dual strategy is the
one that saves more on energy (followed by the cautious and
the certainty equivalence): therefore, the dual strategy seems
to be the one with the best tradeoff between energy costs and
maintenance costs. Figs. 16, 17 and 18 show the estimation of
the efficiency parameters and the covariance of the efficiency
curve: the peaks that can be noticed in the figures correspond
to a maintenance action restoring the efficiency to their initial
values (this also requires to reset the covariance matrix of the
estimator). The boiler has to be maintained several times be-
cause the degradation rate of the boiler has been set a bit higher
than usual on purpose, in such a way to have a richer scenario
in which maintenance is required quite often. As indicated in
the figures, the dual strategy is the one with the smallest es-
timation error (this can be seen also by noticing that the dual
strategy is the one whose covariance matrix decreases faster),
followed by the cautious strategy and by the certainty equiv-
alence strategy. Therefore, the estimation of the dual strategy
is more accurate, which is due to the presence of the term k
σ
.
Note that the certainty equivalence strategy is the one with the
largest gain kp, therefore one would expect somehow a better
estimation performance due to high gain: however, this does
not happen and it is really the term k
σ
in the dual strategy that,
by making the controller interact with the estimator, contributes
in a sensible reduction of the estimation error. Overall, the sim-
ulations demonstrate that the best economic cost of the system
is achieved by active learning, i.e. the dual strategy, in which
control interacts with estimation.
8. Conclusions
In smart buildings, the models used for Heating, Ventila-
tion, and Air Conditioning energy management and for mainte-
nance scheduling differ in scope and structure: while the mod-
els for energy management describe continuous quantities (en-
ergy, temperature), the models used for maintenance scheduling
describe only a few discrete states (healthy/faulty equipment,
and fault typology). In addition, models for energy manage-
ment typically assume the Heating, Ventilation, and Air Condi-
13
tioning equipment to be healthy, whereas the models for main-
tenance scheduling do not take possible human factors (e.g. dis-
comfort) into account. In this work, a framework for human-
centric optimal maintenance is proposed: energy management
and maintenance scheduling for Heating, Ventilation, and Air
Conditioning are recast in the same optimization framework.
Both continuous and discrete states are embedded as hybrid dy-
namics of the system: in addition, both continuous controls (for
energy management) and discrete controls (for maintenance
scheduling) are considered. Because of the presence of uncer-
tainty (the status of the equipment) the solution to the problem
is addressed via an adaptive dual control formulation, where
control occurs jointly with estimation. Numerical examples ob-
tained via a zone-boiler-radiator test case demonstrate the ef-
fectiveness of the approach.
This work can further proceed in many directions: (1) con-
sidering more complex models can give a more realistic mea-
sure of human comfort, such as the Predicted Mean Vote and the
Predicted Percentage of Dissatisfaction; (2) studying trade-offs
between complexity of human comfort models (requiring mea-
surements of metabolic rate, ratio of clothed/nude surface area,
surface temperature of clothing, air velocity relative to human
body, etc.) and most commonly available sensors (temperature
and humidity); (3) studying the feasibility and the effectiveness
of considering simplified models, i.e. studying how the sim-
plifications in the human comfort model introduce additional
uncertainties to be estimated on line and to be embedded in the
optimization; (4) extending the maintenance actions by consid-
ering non-ideal reparation (that would decrease the degradation
without restoring the initial state) or inspection (whose effect
would be to improve the estimate of the efficiency, possibly re-
setting the estimate and the covariance matrix).
Acknowledgment
The research leading to these results has been partially funded
by the European Commission FP7-PEOPLE-2012-IAPP - Marie
Curie Action: Industry-Academia Partnerships and Pathways,
under contract #324432 (Advanced Methods in Building Di-
agnostics and Maintenance, AMBI). The first author acknowl-
edges the support by ”the Fundamental Research Funds for The
Central Universities” under the project RECON-STRUCT. The
second author appreciates the financial supports by the State
Key Laboratory of Intelligent Control and Decision of Com-
plex Systems, the Young Scientist Fund of the National Natural
Science Foundation of China (Grant No. 61703099), and the
China Postdoctoral Science Foundation Funded Project (Grant
No. 2017M621589).
References
[1] Luis Perez-Lombard, Jose Ortiz, and Ismael R. Maestre. The map of
energy flow in hvac systems. Applied Energy, 88(12):5020 – 5031, 2011.
ISSN 0306-2619.
[2] D.H. Blum, K. Arendt, L. Rivalin, M.A. Piette, M. Wetter, and C.T. Veje.
Practical factors of envelope model setup and their effects on the perfor-
mance of model predictive control for building heating, ventilating, and
air conditioning systems. Applied Energy, 236:410 – 425, 2019.
[3] Rongpeng Zhang and Tianzhen Hong. Modeling of hvac operational
faults in building performance simulation. Applied Energy, 202:178 –
188, 2017.
[4] L. Wang, S. Greenberg, J. Fiegel, A. Rubalcava, S. Earni, X. Pang, R. Yin,
S. Woodworth, and J. Hernandez-Maldonado. Monitoring-based HVAC
commissioning of an existing office building for energy efficiency. Ap-
plied Energy, 102:1382 – 1390, 2013. Special Issue on Advances in sus-
tainable biofuel production and use - XIX International Symposium on
Alcohol Fuels - ISAF.
[5] S. Baldi, A. Karagevrekis, I. Michailidis, and E. B. Kosmatopoulos.
Joint energy demand and thermal comfort optimization in photovoltaic-
equipped interconnected microgrids. Energy Conversion and Manage-
ment, 101:352–363, 2015.
[6] E. H. Mathews, D. C. Arndt, C. B. Piani, and E. Van Heerden. Developing
cost efficient control strategies to ensure optimal energy use and sufficient
indoor comfort. Applied Energy, 66(2):135–159, 2000.
[7] S. Baldi, I. Michailidis, C. Ravanis, and E. B. Kosmatopoulos. Model-
based and model-free plug-and-play building energy efficient control. Ap-
plied Energy, 154:829–841, 2016.
[8] Xiaohua Wu, Xiaosong Hu, Xiaofeng Yin, Caiping Zhang, and Shide
Qian. Optimal battery sizing of smart home via convex programming.
Energy, 140:444 – 453, 2017.
[9] Xiaohua Wu, Xiaosong Hu, Yanqiong Teng, Shide Qian, and Rui Cheng.
Optimal integration of a hybrid solar-battery power source into smart
home nanogrid with plug-in electric vehicle. Journal of Power Sources,
363:277 – 283, 2017.
[10] J.C. Solano, L. Olivieri, and E. Caama˜
no-Mart´
ın. Assessing the potential
of pv hybrid systems to cover hvac loads in a grid-connected residen-
tial building through intelligent control. Applied Energy, 206:249 – 266,
2017.
[11] Shuangping Duan, Zhiwen Luo, Xinyan Yang, and Yuguo Li. The im-
pact of building operations on urban heat/cool islands under urban densi-
fication: A comparison between naturally-ventilated and air-conditioned
buildings. Applied Energy, 235:129 – 138, 2019.
[12] Dashamir Marini. Optimization of hvac systems for distributed generation
as a function of different types of heat sources and climatic conditions.
Applied Energy, 102:813 – 826, 2013.
[13] C. D. Korkas, S. Baldi, and E. B. Kosmatopoulos. 9 - grid-connected
microgrids: Demand management via distributed control and human-in-
the-loop optimization. In I. Yahyaoui, editor, Advances in Renewable
Energies and Power Technologies, pages 315 – 344. Elsevier, 2018.
[14] A. Ghahramani, F. Jazizadeh, and B. Becerik-Gerber. A knowledge based
approach for selecting energy-aware and comfort-driven HVAC tempera-
ture set points. Energy and Buildings, 85:536 – 548, 2014.
[15] Siddharth Goyal, Prabir Barooah, and Timothy Middelkoop. Experimen-
tal study of occupancy-based control of hvac zones. Applied Energy, 140:
75 – 84, 2015.
[16] P. Carreira, A. Aguiar Costa, V. Mansur, and A. Ars´
enio. Can HVAC
really learn from users? a simulation-based study on the effectiveness of
voting for comfort and energy use optimization. Sustainable Cities and
Society, 41:275 – 285, 2018.
[17] Wooyoung Jung and Farrokh Jazizadeh. Human-in-the-loop hvac opera-
tions: A quantitative review on occupancy, comfort, and energy-efficiency
dimensions. Applied Energy, 239:1471 – 1508, 2019.
[18] X. He, Z. Zhang, and A. Kusiak. Performance optimization of HVAC sys-
tems with computational intelligence algorithms. Energy and Buildings,
81:371 – 380, 2014.
[19] M. Liu, K. B. Wittchen, and P. K. Heiselberg. Control strategies for in-
telligent glazed fac¸ade and their influence on energy and comfort perfor-
mance of office buildings in Denmark. Applied Energy, 145:43–51, 2015.
[20] Z. Wang, L. Wang, A. I. Dounis, and R. Yang. Multi-agent control system
with information fusion based comfort model for smart buildings. Applied
Energy, 99:247–254, 2012.
[21] M. Mossolly, K. Ghali, and N. Ghaddar. Optimal control strategy for a
multi-zone air conditioning system using a genetic algorithm. Energy, 34
(1):58–66, 2009.
[22] C. Tzivanidis, K. A. Antonopoulos, and F. Gioti. Numerical simulation
of cooling energy consumption in connection with thermostat operation
mode and comfort requirements for the Athens buildings. Applied Energy,
88(8):2871–2884, 2011.
[23] H. Satyavada and S. Baldi. An integrated control-oriented modelling for
14
HVAC performance benchmarking. Journal of Building Engineering, 6:
262 – 273, 2016.
[24] Hussain Syed Asad, Richard Kwok Kit Yuen, and Gongsheng Huang.
Multiplexed real-time optimization of hvac systems with enhanced con-
trol stability. Applied Energy, 187:640 – 651, 2017.
[25] I. T. Michailidis, T. Schild, R. Sangi, P. Michailidis, C. Korkas, J. Futterer,
D. Muller, and E. B. Kosmatopoulos. Energy-efficient HVAC manage-
ment using cooperative, self-trained, control agents: A real-life german
building case study. Applied Energy, 211:113 – 125, 2018.
[26] T. Samad and A. M. Annaswamy. Controls for smart grids: Architectures
and applications. Proceedings of the IEEE, 105(11):2244–2261, 2017.
[27] Huilong Wang, Shengwei Wang, and Rui Tang. Development of grid-
responsive buildings: Opportunities, challenges, capabilities and applica-
tions of hvac systems in non-residential buildings in providing ancillary
services by fast demand responses to smart grids. Applied Energy, 250:
697 – 712, 2019.
[28] S. Wang, Q. Zhou, and F. Xiao. A system-level fault detection and di-
agnosis strategy for HVAC systems involving sensor faults. Energy and
Buildings, 42:477–490, 2010.
[29] S. Wu and J.-Q. Sun. Cross-level fault detection and diagnosis of building
HVAC systems. Energy and Environment, 46:1558–1566, 2011.
[30] P. M. Papadopoulos, V. Reppa, M. M. Polycarpou, and C. G. Panayiotou.
Distributed diagnosis of actuator and sensor faults in HVAC systems.
IFAC-PapersOnLine, 50(1):4209 – 4215, 2017. 20th IFAC World
Congress.
[31] R. Ferrari, H. Dibowski, and S. Baldi. A message passing algorithm for
automatic synthesis of probabilistic fault detectors from building automa-
tion ontologies. IFAC-PapersOnLine, 50(1):4184 – 4190, 2017. 20th
IFAC World Congress.
[32] Dian ce Gao, Shengwei Wang, Kui Shan, and Chengchu Yan. A system-
level fault detection and diagnosis method for low delta-t syndrome in the
complex hvac systems. Applied Energy, 164:1028 – 1038, 2016.
[33] Marco Bonvini, Michael D. Sohn, Jessica Granderson, Michael Wetter,
and Mary Ann Piette. Robust on-line fault detection diagnosis for hvac
components based on nonlinear state estimation techniques. Applied En-
ergy, 124:156 – 166, 2014.
[34] W.J.N. Turner, A. Staino, and B. Basu. Residential HVAC fault detection
using a system identification approach. Energy and Buildings, 151:1 –
17, 2017.
[35] C. Yang, W. Shen, Q. Chen, and B. Gunay. A practical solution for HVAC
prognostics: Failure mode and effects analysis in building maintenance.
Journal of Building Engineering, 15:26 – 32, 2018.
[36] L. Rosa and R. Tosato. Experimental evaluation of seasonal efficiency of
condensing boilers. Energy and Buildings, 14:237–241, 1990.
[37] K. Jeong, M. J. Kessen, H. Bilirgen, and E. K. Levy. Analytical mod-
eling of water condensation in condensing heat exchanger. International
Journal of Heat and Mass Transfer, 53:2361–2368, 2010.
[38] C. Di Perna, G. Magri, G. Giuliani, and G. Serenelli. Experimental as-
sessment and dynamic analysis of a hybrid generator composed of an air
source heat pump coupled with a condensing gas boiler in a residential
building. Applied Themal Engineering, 76:86–97, 2015.
[39] H. Satyavada and S. Baldi. Monitoring energy efficiency of condensing
boilers via hybrid first-principle modelling and estimation. Energy, 142:
121 – 129, 2018.
[40] S. Baldi, T. Le Quang, O. Holub, and P. Endel. Real-time monitoring en-
ergy efficiency and performance degradation of condensing boilers. En-
ergy Conversion and Management, 136:329 – 339, 2017.
[41] K. Bruton, P. Raftery, B. Kennedy, M. M. Keane, and D. T. J. O’Sullivan.
Review of automated fault detection and diagnostic tools in air handling
units. Energy efficiency, 7:335–351, 2014.
[42] J. Schein, S. T. Bushby, N. S. Castro, and J. M. House. A rule-based
fault detection method for air handling units. Energy and Buildings, 38:
1485–1492, 2006.
[43] S. Baldi, S. Yuan, P. Endel, and O. Holub. Dual estimation: Constructing
building energy models from data sampled at low rate. Applied Energy,
169:81 – 92, 2016.
[44] Y. Yu, D. Woradechjumroen, and D. Yu. A review of fault detection and
diagnosis methodologies on air-handling units. Energy and Buildings, 82:
550–562, 2014.
[45] J. Qin and S. Wang. A fault detection and diagnosis strategy of vav air-
conditioning systems for improved energy and control performances. En-
ergy and Buildings, 37:1035–1048, 2005.
[46] C. Nzukam, A. Voisin, E. Levrat, D. Sauter, and B. Iung. A dynamic
maintenance decision approach based on maintenance action grouping
for HVAC maintenance costs savings in non-residential buildings. IFAC-
PapersOnLine, 50(1):13722 – 13727, 2017. 20th IFAC World Congress.
[47] Syed Aftab Rashid, Zeeshan Haider, S.M. Chapal Hossain, Kashan
Memon, Fazil Panhwar, Momoh Karmah Mbogba, Peng Hu, and Gang
Zhao. Retrofitting low-cost heating ventilation and air-conditioning sys-
tems for energy management in buildings. Applied Energy, 236:648 –
661, 2019.
[48] S. Haesaert, N. Cauchi, and A. Abate. Certified policy synthesis for gen-
eral markov decision processes: An application in building automation
systems. Performance Evaluation, 117:75 – 103, 2017.
[49] M. Rasouli, G. Ge, C. J. Simonson, and R. W. Besant. Uncertainties
in energy and economic performance of HVAC systems and energy re-
covery ventilators due to uncertainties in building and HVAC parameters.
Applied Thermal Engineering, 50(1):732 – 742, 2013. ISSN 1359-4311.
[50] J. Verhelst, G. Van Ham, D. Saelens, and L. Helsen. Model selection
for continuous commissioning of HVAC-systems in office buildings: A
review. Renewable and Sustainable Energy Reviews, 76:673 – 686, 2017.
[51] N. Cauchi, K. Macek, and A. Abate. Model-based predictive maintenance
in building automation systems with user discomfort. Energy, 138:306 –
315, 2017.
[52] A.-Y. Cheong Peng, A. Azlan Shah, and A. Faizah. Improving occupants’
satisfaction with effective maintenance management of HVAC system in
office buildings. Automation in Construction, 43:31 – 37, 2014.
[53] R. M. Lazzarin. Condensing boilers in buildings and plants refurbish-
ment. Energy and Buildings, 47:61–67, 2012.
[54] Natural gas price statistics. URL \url{http://ec.europa.eu/
eurostat/statistics-explained/index.php/Natural_
gas_price_statistics}.
[55] Optimal thermal environment improves performance
of office work. URL \url{https://www.
rehva.eu/fileadmin/hvac-dictio/01- 2012/
optimal-thermal- environment-improves-performance\
-of- office-work_rj1201.pdf}.
[56] Condensing boilers prices. URL \url{https://www.
boilersprices.co.uk/condensing-boilers- prices/}.
[57] M. S. Branicky, V. S. Borkar, and S. K. Mitter. A unified framework for
hybrid control: model and optimal control theory. IEEE Transactions on
Automatic Control, 43(1):31–45, 1998.
[58] M. Rungger and O. Stursberg. Optimal control for deterministic hybrid
systems using dynamic programming. IFAC Proceedings Volumes, 42
(17):316 – 321, 2009. 3rd IFAC Conference on Analysis and Design of
Hybrid Systems.
15
Appendix A. Parameters used for simulation
Symbol Value Unit
Tdew 57.2 [oC]
Td21 [oC]
wwB0.15 [kg/s]
cS0.5 [%]
cw4.179 [kJ/kgoC]
ρ
w992.3 [kg/m3]
ca1.005 [kJ/kgoC]
ρ
a1.205 [kg/m3]
VR0.971 [m3]
AR14.63 [m2]
KR0.015 [kW/oC]
hR0.005 [kW/m2oC]
Ao45 [m2]
An405 [m2]
ho0.002 [kW/m2oC]
hn0.001 [kW/m2oC]
h1 [oC]
α
1/10368000 [s−1]
κ
1/10368000 [s−1]
θ
1-0.360 [oC−1]
θ
20.359 -
θ
30.276 [oC−1]
σ
2
R200 [kW2]
σ
2
Z10 [kW2]
σ
2
B0.001 -
σ
2
10.001
α
[oC−2]
σ
2
20.001
α
-
σ
2
30.001
α
[oC−2]
Table A.4: Numerical parameters
16
