Prabir Barooah’s research while affiliated with Indian Institute of Technology Guwahati and other places

We consider the problem of optimal control of district cooling energy plants (DCEPs) consisting of multiple chillers, a cooling tower, and a thermal energy storage (TES), in the presence of time-varying electricity price. A straightforward application of model predictive control (MPC) requires solving a challenging mixed-integer nonlinear program (MINLP) because of the on/off of chillers and the complexity of the DCEP model. Reinforcement learning (RL) is an attractive alternative since its real-time control computation is much simpler. But designing an RL controller is challenging due to myriad design choices and computationally intensive training. In this paper, we propose an RL controller and an MPC controller for minimizing the electricity cost of a DCEP and compare them via simulations. The two controllers are designed to be comparable in terms of objective and information requirements. The RL controller uses a novel Q-learning algorithm that is based on least-squares policy iteration. We describe the design choices for the RL controller, including the choice of state space and basis functions, that are found to be effective. The proposed MPC controller does not need a mixed integer solver for implementation, but only a nonlinear program (NLP) solver. A rule-based baseline controller is also proposed to aid in comparison. Simulation results show that the proposed RL and MPC controllers achieve similar savings over the baseline controller, about 17%.

We present an open-source wireless network and data management system for collecting and storing indoor environmental measurements and perceived comfort via participatory sensing in commercial buildings. The system, called a personal comfort and indoor environment measurement (PCIEM) platform, consists of several devices placed in office occupants’ work areas, a wireless network, and a remote database to store the data. Each device, called a PCFN (personal comfort feedback node), contains a touchscreen through which the occupant can provide feedback on their perceived comfort on-demand, and several sensors to collect environmental data. The platform is designed to be part of an indoor climate control system that can enable personalized comfort control in real-time. We describe the design, prototyping, and initial deployment of a small number of PCFNs in a commercial building. We also provide lessons learned from these steps. Application of the data collected from the PCFNs for modeling and real-time control will be reported in future work. We use hardware components that are commercial and off-the-shelf, and our software design is based on open-source tools that are freely and publicly available to enable repeatability.

This paper aims to quantify the performance of Model Predictive Control (MPC) for a typical commercial building heating, ventilation and air conditioning (HVAC) system across a wide range of climate and weather conditions. The motivation of the study comes from the fact that although there is a large body of work on MPC for HVAC systems, there is a lack of studies that examine the range of possible performance of MPC, in terms of both energy savings and maintaining indoor climate (temperature and humidity) as a function of outdoor weather. A challenge in conducting such a study is developing an MPC controller that can be used in a wide range of weather. The root cause of this challenge is the need for a tractable cooling and dehumidification coil model that can be used by the MPC controller, since the coil may operate in quite distinct modes depending on weather. We present such an MPC controller, and then leverage it to conduct an extensive simulation campaign for fourteen climate zones in the United States and four weather conditions (winter, spring, summer, and fall) in each climate zone. The performance of the proposed controller is compared with not only a rule-based baseline controller but also with a simpler MPC controller that ignores humidity and latent heat considerations. There are several results the arise from this comparative study. One such result is that energy savings from MPC over baseline can vary dramatically based on climate and season. Another is that the effect of ignoring humidity in the MPC formulation can lead to poor indoor humidity control more in milder weather rather than in hot weather. The results from this study can help practitioners and researchers assess costs and benefits of proposed MPC formulations for HVAC control.

District cooling energy plants (DCEPs) consisting of chillers, cooling towers, and thermal energy storage (TES) systems consume a considerable amount of electricity. Optimizing the scheduling of the TES and chillers to take advantage of time-varying electricity price is a challenging optimal control problem. The classical method, model predictive control (MPC), requires solving a high dimensional mixed-integer nonlinear program (MINLP) because of the on/off actuation of the chillers and charging/discharging of TES, which are computationally challenging. RL is an attractive alternative to MPC: the real time control computation is a low-dimensional optimization problem that can be easily solved. However, the performance of an RL controller depends on many design choices. In this paper, we propose a Q-learning based reinforcement learning (RL) controller for this problem. Numerical simulation results show that the proposed RL controller is able to reduce energy cost over a rule-based baseline controller by approximately 8%, comparable to savings reported in the literature with MPC for similar DCEPs. We describe the design choices in the RL controller, including basis functions, reward function shaping, and learning algorithm parameters. Compared to existing work on RL for DCEPs, the proposed controller is designed for continuous state and actions spaces.

Thermostatically Controlled Loads (TCLs), e.g., an air conditioner, typically maintain their temperature within a preset range using on/off actuation. These types of loads are inherently flexible: many different power consumption trajectories exist that can keep the temperature within range. Grid operators need tools for quantifying demand flexibility of a collection of TCLs in order to use them as a resource. However, computationally tractable characterization of flexibility capacity obtained so far has considered temperature constraints alone, ignoring their cycling constraints: the length of time a TCL must stay in ‘'on’' stage before switching to ‘'off’', or vice versa. In this work, we present a tractable characterization of the flexibility capacity of a collection of TCLs that incorporates not only temperature but also cycling and total energy consumption constraints. Unlike prior attempts at capacity characterizations incorporating cycling constraints, our results are independent of the algorithm used to coordinate the TCLs. The characterization leads to a set of convex constraints. A grid operator can use this characterization to compute a power consumption trajectory for an ensemble of TCLs that comes closest to what the operator needs to maintain demand-supply balance. Numerical results are provided to showcase the effectiveness of the proposed characterization.