PreprintPDF Available

Abstract and Figures

Labelled human comfort data can be a valuable resource in optimising the built environment, and improving the wellbeing of individual occupants. The acquisition of labelled data however remains a challenge. This paper presents a methodology for the collection of in-situ occupant feedback data using a Fitbit smartwatch. The clock-face application cozie can be downloaded free-of-charge on the Fitbit store and tailored to fit a range of occupant comfort related experiments. In the initial trial of the app, fifteen users were given a smartwatch for one month and were prompted to give feedback on their thermal preferences. In one month, with minimal administrative overhead, 1460 labelled responses were collected. This paper demonstrates how these large data sets of human feedback can be analysed to reveal a range of results from building anomalies, occupant behaviour, occupant personality clustering, and general feedback related to the building. The paper also discusses limitations in the approach and the next phase of design of the platform.
Content may be subject to copyright.
Is your clock-face cozie? A smartwatch methodology
for the in-situ collection of occupant comfort data
P Jayathissa1, M Quintana1, T Sood1, N. Narzarian2, C. Miller 1 3
1Building and Urban Data Science Group, National University of Singapore (NUS), Singapore
2University of New South Wales (UNSW), Australia
E-mail: p.jayathissa@nus.edu.sg
Abstract. Labelled human comfort data can be a valuable resource in optimising the built
environment, and improving the wellbeing of individual occupants. The acquisition of labelled
data however remains a challenge. This paper presents a methodology for the collection of
in-situ occupant feedback data using a Fitbit smartwatch. The clock-face application cozie can
be downloaded free-of-charge on the Fitbit store and tailored to fit a range of occupant comfort
related experiments. In the initial trial of the app, fifteen users were given a smartwatch for
one month and were prompted to give feedback on their thermal preferences. In one month,
with minimal administrative overhead, 1460 labelled responses were collected. This paper
demonstrates how these large data sets of human feedback can be analysed to reveal a range
of results from building anomalies, occupant behaviour, occupant personality clustering, and
general feedback related to the building. The paper also discusses limitations in the approach
and the next phase of design of the platform.
1. Introduction
In the Ma¯ori legends of old, there was a time when the sun would travel quickly across the sky,
leaving people without sufficient light and warmth. aui, a great hero of the time, observed
this discomfort amongst the village and went on a quest to tame the sun. Armed with his magic
jawbone of Murirangawhenua and a lot of flax rope, he succeeded in tying down the sun and
beating it, until it slowed down to the speeds we have today. What M¯aui effectively did was
categorise everyone in a one-size-fits-all thermal comfort model, and based on this assumption,
took action to change the environment he lived in.
The way we control our buildings today is similar to the method that M¯aui used to tame the
sun. We make an assumption of the general population based on a survey of a few people, and
change the environment we live in based on these few data points. From these methodologies,
traditional thermal comfort research has produced comfort models based on indices such as the
predicted mean vote (PMV). A recent review of data from dozens of such studies has shown that
PMV is accurate only 34% of the time [1]. The issue with these models is that they assume that
all occupants within a building zone share the same comfort preferences. In reality, variations
in metabolic rates and the preferences or tolerances of each person presents a challenge when
attempting to condition a workspace to meet the requirements of all occupants. These personal
attributes of comfort create a comfort personality that is unique to the individual and could
3Present address: Department of Building, National University of Singapore, 4 Architecture Drive, Singapore
Experiment
Settings
Feedback
Data
Client Side
Sensor
Data
Influx
Time-Series
Database
Query
(a) (b) (c) (d)
Figure 1: Overview of the cozie app platform: (a) the Fitbit mobile app is used to set
experimental settings, (b) the flow of questions based on settings selected in (a), (c) photos
of the watch-face in action with the strap-pack sensor box, (d) overview of data communication.
be shaped by culture, personal background, acclimatization, and general personality traits, in
addition to physiology.
Understanding the preferences of individuals presents a significant challenge, both in the
times of M¯aui and now. The state of the art of human comfort data collection is in the form of
surveys, either as an online form, or paper-based. This format presents three major challenges.
The methodology cannot be scaled to large sample sets due to the administrative overhead
in preparing these studies.
The studies are often conducted outside of the test subjects natural working environment.
Users suffer from survey fatigue [2] due to the number of data points required to conduct
a thorough assessment. Even when willing to participate, there is a concern about how
accurately their responses are [3].
This paper presents cozie, a publicly available clock-face designed for the Fitbit smartwatch
which can be used for tailored, scalable, in-situ human comfort studies. We will show how the
watch-face can be deployed for a range of tailored experimental scenarios, and be evaluated using
modern data analytics to infer behavioural patterns of the test participants. This information
can be used to optimise human comfort through spatial recommendation, or combined with
building sensor data to create a labelled data-set for the comfort optimisation of the building
management system.
2. The cozie clock-face
Cozie is built as a clock-face for Fitbit, a smartwatch with 25 million active users [4]. The
application is publicly available for download from the cozie website4. In this paper, we define
user as the test participant who is wearing the Fitbit, and manager as the person coordinating
the experiment. The default status of the clock-face is a simple binary question: Comfy or Not
Comfy, as seen in Figure 1. By simply clicking one of the icons, information about the users’
4cozie.app
location (GPS), heart-rate, steps walked since the last log, and the comfort data is anonymously
sent to an Influx time series cloud database 5. Data from this database can be simply queried
with an API key that can be provided to the manager.
If the manager is interested as to why the user is feeling discomfort, then there is a range
of additional questions that can be configured using the cellphone that the Fitbit is paired
with. The optional questions include: thermal preference, light preference, noise preference,
indoor/outdoor, mood, and whether the user is in office. These settings, along with a unique
user-id for each user, and a unique experiment-id can be configured by the manager using the
cellphone that the Fitbit is paired with. The watch-face also has the ability to prompt the user
and force them to provide feedback at custom intervals set by the manager.
3. Experimental methodology
An experiment was deployed as part of Project Coolbit6, an international effort that focusses
on the use of smartwatches for human comfort analysis [5]. Fifteen participants residing in
Singapore were recruited for the experiment and were equipped with Fitbit Versa or Ionic
watches. Most of the participants were working at the National University of Singapore (NUS) at
several flexible workspaces around campus. The cozie clock-face was installed and set to request
thermal preference (prefer warmer, prefer cooler, comfy), and the set to request feedback at the
hours of 9:00, 11:00, 13:00, 15:00, and 17:00. These feedback requests take the form of a gentle
vibration of the watch to prompt the user to provide feedback.
The watch was further complimented with Internet-of-Things (IoT) connected on-body and
environmental sensors. The on-body sensor consists of a temperature and light sensor from
mbient-labs7that had been modified to fit the watch strap with a custom 3D printed case. An
off-body sensor measuring temperature and humidity was attached to the participant’s bag.
The sensors communicate via Bluetooth to Raspberry-Pi gateways that had been positioned
throughout the working space. Data from the cozie clock-face, and the environmental sensors
were aggregated in an Influx cloud time-series database, which served as a platform for data
acquisition and fault detection.
4. Results
The experiment, consisting of fifteen users each equipped with a Fitbit over a month, produced
a data set of 1,460 data points. Each data point is effectively a survey of the user at a particular
time. The results presented in this section is a demonstration of the type of analysis that can
be conducted using data acquired from the cozie clock-face.
4.1. Overview of spatio-temporal data
Figure 2a details the spatial distribution of data throughout Singapore. Each of these data
points is tagged with the users heart-rate, response, and local temperature which can be used to
infer faults or issues within the building. Figure 2b is a simple heat map plotting the number of
responses based on the hour of the day, and day of the week. It is interesting to note that there
are more responses in the hours of 9:00, 11:00, 13:00, 15:00, and 17:00 when the occupant is
buzzed and forced to give feedback. Nevertheless there are still significant amounts of responses
made outside these times through the motivation of the participants themselves. Figure 2c
details the daily responses from the participants, and no observable decrease in responses can
be made, indicating no effects of survey fatigue. Dips in responses naturally occur during the
5https://github.com/influxdata/influxdb
6http://www.projectcoolbit.com
7https://mbientlab.com/
Prefer Cooler
Comfortable
Prefer Warmer
co-working space 1
(a)
(b) (c)
(d) (e) (f)
days
responses
hour
daynormalised distribution
Heart Rate (bpm) Temperature (C) Relative Humidity (%)
SpatialTemporalSensor
Figure 2: Overview of raw data extracted from the cozie clock-face and additional sensors. (a)
spatial distribution of responses throughout Singapore, (b) temporal distribution of responses,
(c) number of responses per day over the course of the experiment, (d-f) rug plots detailing
the normalised distribution of responses based on the Fitbit heart rate sensor, wrist mounted
temperature sensor, and off-body humidity sensor
weekend. A normal distribution of heart-rate data from the Fitbit heart rate sensor can be
found in Figure 2d.
4.2. Merging with environmental sensor data
Combining the cozie clock-face, with additional environmental sensors opens further dimensions
of analysis. User responses are mapped to the environmental condition at which they are
exposed, which can provide a high quality labelled data set for training data driven models.
Figure 2e-f detail the distribution of temperature and humidity data. Unfortunately, due to
communication issues from these sensors, not all data points could be recorded. The temperature
of the temperature sensor is on average 0.8 C warmer than the surrounding environment due
to the influence of body temperature.
4.3. Clustering of thermal comfort personality
Individual user feedback can be clustered using unsupervised learning techniques. In this
example, we use a hierarchical k-means clustering based on euclidean distance using the
nearest-point-algorithm. The results, shown in Figure 3, show four distinct clusters of users.
Understanding and defining these differences in user preferences can be used to recommend
spaces that may better suit the needs of the occupant. For example, Group A, which seems to
primarily be working off-site can be recommended alternative workspaces that are on average
A
C
B
D
Temperature
Prefer Cooler Comfortable Prefer Warmer
Heart Rate
co-working space 1
Location
National University
of Singapore
Prefer Cooler
Comfortable
Prefer Warmer
Figure 3: Heiracheal cluster-map of user feedback using k-means with euclidean distance. The
numbers within the cluster-map detail the normalised number of responses. Four distinct groups
can be observed. (A) two users that generally prefer cooler environments to their norm, (B)
users that are comfortable 50% of the time, (C) user that is almost always comfortable, (D)
users that are comfortable on average 70% of the time. To the right are breakdowns of the
respective groups via sensor data and location. Below the cluster plot are spatial distributions
of feedback responses at the university.
cooler. Group C on the other hand is a single highly satisfied user, who works from a single
work-space within a narrow temperature range, and relatively low resting heart-rate. Group D
represents our conventional occupant that may be comfortable 70% of the time.
Below the cluster plot are spatial distributions of responses that can be used to identify
different building climates. The majority of Prefer Warmer responses occur in co-working
space 1. This area can be labelled as a cooler working space for users that would prefer cooler
working environments. Alternatively, if facilities management wishes to save energy, increasing
the set-point temperature of these over-cooled spaces may be a low effort solution which may
simultaneously improve occupant well-being.
5. Discussion
5.1. Large uncontrolled data vs. small controlled data
Placing a group of participants in a controlled experimental space and conducting feedback
surveys is a trusted traditional method for human comfort surveys. Giving each participant a
smartwatch, and analysing the patterns of hundreds of data points per user would be more akin
to modern data analytics employed in industries outside the building sector. Both methods are
useful for reaching different types of conclusions. The traditional method can derive conclusions
such as: 4 of the 15 users felt warm at temperatures higher than 25.6 C. This insight is
useful in the context of the previously mentioned generalizable thermal comfort models that
are traditionally created in built environment research, but have poor accuracy. On the other
hand, the uncontrolled, large data method can draw conclusions such as: 4 of 15 users can be
categorized as a user type that prefers cooler working environments. This paper focuses on the
use of clustering to show the groups of comfort personality types that correspond with different
behaviour in giving feedback and interacting with the spaces.
5.2. In-situ benefits and limitations
Uncontrolled experiments have minimal management overhead, which means that it can be
easily scaled to larger groups by purchasing more devices. Furthermore, the users are analysed
in their natural work environments and give feedback with a simple click on their watch. This
reduction of effort results in no fatigue in the number of voluntary responses given as shown in
Figure 2c. While users generally work from their office, they sometimes work from home, or at
a local cafe. This presents a limitation in the context of traditional small controlled data. Large
enough data needs to be obtained to filter out these scenarios and interpret meaningful results.
5.3. Indoor Localisation
The clock-face collects GPS data from the Fitbit, however GPS data indoors is not always
reliable, and often not accessible. Only 30% of all data points were tagged with GPS data.
The team is currently investigating other methods of localization, which includes continuous
logging of GPS data to infer an entry and exit of building spaces, Bluetooth based localisation
from Steerpath, integration with applications focused on space use optimization, and pattern
matching of wearable sensor data to indoor sensors.
6. Conclusion
First trial runs of the cozie application for occupant comfort data collection have proven
successful. Within just four weeks 1460 data points of thermal comfort were obtained from
the 15 test participants, with minimal administrative overhead. This rich data set provides new
opportunities in analysing occupant comfort behaviour through data driven methods. Within
this paper, we have demonstrated how the data can be manipulated and clustered to group
people into various comfort profiles. In our case there were 4 distinct groups, which can be
recommended spaces that better suit their comfort profile. The data can also be clustered via
time to display building defects, or anomalies in occupant behaviour. Finally, we demonstrate
how the app can be combined with wearable environmental sensors to cross-reference a users
preference to the environment that they were in. The clock-face is publicly available for download
at cozie.app.
Next steps in this research involve using cozie for the exploration of occupant clustering and
spatial recommendation. We will explore elements of sound, light, and thermal comfort to de-
termine whether spatial recommendation can serve as an alternative to individualised adaptive
buildings control. Furthermore, the development of a strap-pack, a smartwatch environmental
sensor that can be adapted to the watch strap is underway.
References
[1] T. Cheung, S. Schiavon, T. Parkinson, P. Li, G. Brager, Analysis of the accuracy on PMV–PPD model using
the ashrae global thermal comfort database ii, Building and Environment.
[2] S. R. Porter, M. E. Whitcomb, W. H. Weitzer, Multiple surveys of students and survey fatigue, New Directions
for Institutional Research 2004 (121) (2004) 63–73.
[3] A. K. Clear, S. Mitchell Finnigan, P. Olivier, R. Comber, ThermoKiosk: Investigating Roles for Digital
Surveys of Thermal Experience in Workplace Comfort Management, Proc. of CHI (2018) 1–12doi:
10.1145/3173574.3173956.
[4] Fitbit 2018 investor report (2018).
URL https://investor.fitbit.com
[5] N. Nazarian, C. Miller, L. Norford, M. Kohler, W. Chow, J. Lee, S. Alhadad, M. Quintana, L. Suden,
A. Martilli, Project coolbit updates: Personal thermal comfort assessments using wearable devices,
Geophysical Research Abstracts, EGU General Assembly.
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
The predicted mean vote (PMV) and predicted percentage of dissatisfied (PPD) are the most widely used thermal comfort indices. Yet, their performance remains a contested topic. The ASHRAE Global Thermal Comfort Database II, the largest of its kind, was used to evaluate the prediction accuracy of the PMV/PPD model. We focused on: (i) the accuracy of PMV in predicting both observed thermal sensation (OTS) or observed mean vote (OMV) and (ii) comparing the PMV-PPD relationship with binned OTS – observed percentage of unacceptability (OPU). The accuracy of PMV in predicting OTS was only 34%, meaning that the thermal sensation is incorrectly predicted two out of three times. PMV had a mean absolute error of one unit on the thermal sensation scale and its accuracy decreased towards the ends of the thermal sensation scale. The accuracy of PMV was similarly low for air-conditioned, naturally ventilated and mixed-mode buildings. In addition, the PPD was not able to predict the dissatisfaction rate. If the PMV model would perfectly predict thermal sensation, then PPD accuracy is higher close to neutrality but it would overestimate dissatisfaction by approximately 15–25% outside of it. Furthermore, PMV-PPD accuracy varied strongly between ventilation strategies, building types and climate groups. These findings demonstrate the low prediction accuracy of the PMV–PPD model, indicating the need to develop high prediction accuracy thermal comfort models. For demonstration, we developed a simple thermal prediction model just based on air temperature and its accuracy, for this database, was higher than PMV.
Conference Paper
Full-text available
Thermal comfort in shared workplaces is often contested and impacts productivity, wellbeing, and energy use. Yet, subjective and situated comfort experiences are rarely captured and engaged with. In this paper, we explore roles for digital surveys in capturing and visualising subjective experiences of comfort in situ for comfort management. We present findings from a 3-week field trial of our prototype system called ThermoKiosk, which we deployed in an open plan, shared office with a history of thermal comfort complaints. In interviews with occupants and members of facilities management, we find that the data and interactions can play an important role in initiating dialogue to understand and handle tensions, and point to design considerations for more systematically integrating them into workplace comfort practices.
Article
This chapter reviews the literature on survey fatigue and summarizes a research project that indicates that administering multiple surveys in one academic year can significantly suppress response rates in later surveys.
Project coolbit updates: Personal thermal comfort assessments using wearable devices
  • N Nazarian
  • C Miller
  • L Norford
  • M Kohler
  • W Chow
  • J Lee
  • S Alhadad
  • M Quintana
  • L Suden
  • A Martilli
N. Nazarian, C. Miller, L. Norford, M. Kohler, W. Chow, J. Lee, S. Alhadad, M. Quintana, L. Suden, A. Martilli, Project coolbit updates: Personal thermal comfort assessments using wearable devices, Geophysical Research Abstracts, EGU General Assembly.