Sensor Data and the City: Urban Visualisation and Aggregation of Well-Being Data

Abstract

The growth of mobile sensor technologies have made it possible for city councils to understand peoples' behaviour in urban spaces which could help to reduce stress around the city. We present a quantitative approach to convey a collective sense of urban places. The data was collected at a high level of granularity, navigating the space around a highly popular urban environment. We capture people's behaviour by leveraging continuous multi-model sensor data from environmental and physiological sensors. The data is also tagged with self-report, location coordinates as well as the duration in different environments. The approach leverages an exploratory data visualisation along with geometrical and spatial data analysis algorithms, allowing spatial and temporal comparisons of data clusters in relation to people's behaviour. Deriving and quantifying such meaning allows us to observe how mobile sensing unveils the emotional characteristics of places from such crowd-contributed content.

Full text

Sensor Data and the City: Urban Visualisation and Aggregation
of Well-Being Data
Thomas Johnson
thomas.johnson@ntu.ac.uk
Nottingham Trent University
Nottingham, UK
Eiman Kanjo
eiman.kanjo@ntu.ac.uk
Nottingham Trent University
Nottingham, UK
Kieran Woodward
kieran.woodward@ntu.ac.uk
Nottingham Trent University
Nottingham, UK
ABSTRACT
The growth of mobile sensor technologies have made it possible
for city councils to understand peoples’ behaviour in urban spaces
which could help to reduce stress around the city. We present a
quantitative approach to convey a collective sense of urban places.
The data was collected at a high level of granularity, navigating the
space around a highly popular urban environment. We capture peo-
pleâĂŹs behaviour by leveraging continuous multi-model sensor
data from environmental and physiological sensors. The data is also
tagged with self-report, location coordinates as well as the duration
in dierent environments. The approach leverages an exploratory
data visualisation along with geometrical and spatial data analysis
algorithms, allowing spatial and temporal comparisons of data clus-
ters in relation to peopleâĂŹs behaviour. Deriving and quantifying
such meaning allows us to observe how mobile sensing unveils the
emotional characteristics of places from such crowd-contributed
content.
CCS CONCEPTS
•Human-centered computing →Geographic visualization.
KEYWORDS
Voronoi, Visualisation, Mental Well-Being, Sensors, Urban Environ-
ment
ACM Reference Format:
Thomas Johnson, Eiman Kanjo, and Kieran Woodward. 2020. Sensor Data
and the City: Urban Visualisation and Aggregation of Well-Being Data .
In Proceedings of UbiComp ’20. ACM, New York, NY, USA, 4 pages. https:
//doi.org/10.1145/1122445.1122456
1 INTRODUCTION
Stress-related illnesses are drastically increasing around the world.
Stress is often dicult to detect and is commonly associated with a
stigma of embarrassment and humiliation preventing people from
receiving help or treatment. However, the impact of stress is pro-
found as it costs the UK economy Âč2.4 billion per year due to work
absence [
16
]. To help alleviate stress, many people restore their
attention and seek relief through meditation or outdoor recreation.
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specic permission and/or a
fee. Request permissions from permissions@acm.org.
UbiComp ’20, September 12–16, 2020, Cancun, Mexico
©2020 Association for Computing Machinery.
ACM ISBN 978-1-4503-XXXX-X/18/06.. .$15.00
https://doi.org/10.1145/1122445.1122456
Nature and urban environments oer a restorative experience to
improve an individualsâĂŹ well-being. However, some environ-
ments are busy, hectic and stressful resulting in a negative impact
in well-being and not relieving stress or exhaustion. With recent
technological advances, several studies have emerged which can
be utilized to assess the eects of built environments on humans
using physiological sensors. For example, Heart Rate (HR) monitors
and Electrodermal Activity (EDA) sensors, have shown enhanced
results following the exposure to restorative environments.
When comparing walking in the countryside compared with ur-
ban environments the results illustrated how places are constructed
through dierent senses and peopleâĂŹ s bodies impact whether
the place is perceived as welcoming and pleasant or hostile and ag-
gressive [
19
]. Positive impressions about places encourage people
to visit, these places are most commonly quiet, restful and tranquil
allowing people to reduce their stress levels by providing a pallia-
tive to the nonstop attentional demands of typical, city streets. It
is important to use objective sensor data to study the relationship
between dierent places and the mental states of people who visit
these places. Kaplan et al. [
10
], Kohlleppel et al. [
11
] have studied
the power of natural environments to provide a restful experience
encouraging a quick and strong recovery from any stress encoun-
tered. In addition to natural settings, coee shops, health clubs,
video arcades and some retail shops were shown to encourage the
restoration from stress and can promote positive emotions [17].
Many studies have been conducted to study the eects of environ-
mental noise on mental health and human well-being. The research
results proved that noise can impair productivity and cause serious
health problems such as chronic stress and heart diseases [
18
]. Phys-
iological signals contain useful patterns that aim to help identify
individual’s mental state. For example, using physiological sensors
such as Electrocardiogram, Electromyogram, skin conductance and
respiration has found to determine a driver’s overall stress levels
[
7
]. Furthermore, recent studies have begun to use physiological
sensors to identify an individual’s mood and emotions [
22
] [
21
]
[20].
There are many applications emerging in the area of environ-
mental monitoring and the impact the environment has on health
and well-being [
9
] [
4
]. For instance, a wearable, low power, air
quality and environmental monitoring sensor sampling a range of
air pollutants (CO, NO2 and O3). The sensors in the system using
real-time information helped those suering from health related
problems such as asthma which in turn helped those individual’s
to avoid highly-polluted environments [
24
]. Furthermore, the ap-
plication NoiseSpy is a low-cost sound measurement device that
monitors environmental noise levels, allowing users to explore an
environment while visualising noise levels in real time [8].
arXiv:2007.02674v1 [cs.HC] 3 Jul 2020

UbiComp ’20, September 12–16, 2020, Cancun, Mexico Johnson, Kanjo and Woodward
The relationship between emotional stability and an urban envi-
ronment is especially important and challenging in cities. This is
because the environments are typically highly dynamic and densely
populated. The ubiquitous nature of smartphones coupled with sen-
sors and increased computational power has allowed them to be
considered as serious competitors to dedicated sensor platforms.
Mobile phones and physiological sensors present many opportuni-
ties to observe human emotion in urban environments. [1].
Previous research on visualising sensor-based data spatially is
focused primarily on mapping, heat-maps or grid overlay. The com-
plexity of a spatial interpolation has a signiïňĄcant impact on how
much sensor data can be placed over a map. For example, Kriging
can be used to visualise and map spatial data [
12
]. Alternatively,
heat maps can be utilised for a quicker visualisation. This has been
used to track Electrodermal activity (EDA) in supermarkets and
around shopping centres to understand how these environments
have a negative impact on an individual’s well-being [
1
] [
6
]. Figure
1 presents a noise visualisation map whereby noise was recorded
over a two week period. The darker colours on the grid view indi-
cate a higher noise level in that particular location [8].
Figure 1: Visualising environmental noise data using grid
overlay around Cambridge City.
There has been limited research focused on understanding how
human emotion impacts when exploring an urban environment.
A study [
14
] has started to use this method by mapping human
emotion around a place, by visualising on a map individual feelings
(fear and comfort) in Los Angeles. However, although this is an
early study in this area, the data was recorded prior and did not
report in real-time which would have given a deeper understanding
into the overall impact. The use of self-reporting to record well-
being is becoming increasingly popular [
23
] especially through
mobile systems such as Mappiness [
13
] and WiMO [
15
] because of
their ability to link the individual’s emotion to a particular location.
We explore the use of physiological and environmental sensors
in an urban city context to monitor how the environment impacts
stress. The built environments considered in this work consist of
both green spaces and urban built area allowing us to understand
how the exposure to natural green spaces may promote greater
attention restoration and stress recovery than visiting built envi-
ronments. We propose a new method of spatial exploration and
visualisation to help reduce stress within urban environments.
2 SPATIAL AGGREGATION AND
VISUALISATION
To visualise the dynamic trends, the urban area needs to be divided
in smaller areas. Binning is a frequently used method to visualise
geographical patterns which use heat maps of the data. Figure 1
illustrates the heat map of the physiological sensor data from one
user data along within the city environment. The heat map shows
that stressful hotspots are scattered along the path and not conned
to one particular area.
While Figure 2 shows the level of intensity of dierent urban
environments based on GPS coordinates, the sensor data on these
heat maps do not show the real distribution of sensor data. One
option to solve this challenge is to divide the study area into grid
cells [
8
]; however, it is dicult to allocate a cell for each sensor
reading, moreover, it is not possible to decide on the cell size, since
the density of the sensor mobility traces can be of dierent density
distributions.
Figure 2: Map showing heat maps of various sensor data in
urban settings.
The biggest challenge in metric layer visualisation lies in scala-
bility. There can be large number of records to map in a single city.
Drawing all these records at their accurate locations simultaneously
will be impossible, due to the large amount of rendering cost and
the required visual bandwidth.
To address these issues, we use a combinatorial computational
geometry algorithm called âĂIJVoronoiâĂİ, which is a diagram
partitioning a plane into regions based on distance to points in a
specic subset of the plane [
5
]. Voronoi diagrams divide the space
into a set of regions called Voronoi cells, including the space that
is closest to the object (location). The size of these cells give an
indication of the density of the area a certain object is in or the size
of an object.
The cell structure also shows the Delaunay triangulation, which
enables the calculation of an object’s immediate set of neighbours.
The denition of a Voronoi cell is given by the following equation,
where x is a planar metric space; p is the set of generator points in
the metric space; and d is the distance between all points in x and
a specic generator point.
Vori ={x|d(x,pi) ≤ d(x,pj ),j,i}(1)
The creation of a Voronoi tessellations is a dynamic procedure
that is repeated until all of the points are represented in adjacent
polygons. If there are not a sucient number of particles to satisfy

Sensor Data and the City: Urban Visualisation and Aggregation of Well-Being Data UbiComp ’20, September 12–16, 2020, Cancun, Mexico
Equation (1), then the Voronoi diagram is only partially lled and
the data is redistributed. The Voronoi diagram is composed of a
collection of tessellations dened as Vor, where:
Vor ={Vor1,Vor2. . . Vorn}(2)
By giving each polygon a value that corresponds to the sensor
value collected in a particular GPS coordinate, it is possible to divide
the space into adjacent polygons with dierent sensor reading
which are represented in colours. This type of spatial analysis,
helps in understanding the degree to which a place is similar to
other nearby places, the correlation among dierent spatial objects
needs to be deïňĄned. We have mapped users to dierent regions
based on their mobility and form several user groups for further
exploration.
3 EXPERIMENTAL SETUP
We use the âĂIJEnvBodySensâĂİ dataset consisting of various sen-
sor and self-report data from a smart phone application, and a
Microsoft wristband 2. Collected data is logged and stamped with
the time and date and GPS location. In the EnvBodySens applica-
tion, an interface is implemented for continuous and quick labelling
of user emotions while walking and collecting data. When the user
launches the application, mobile interface appears with ve iconic
facial expressions ranging from very negative to very positive. Users
were asked to constantly select one of the aective categories (in the
form of buttons) as they walk around the city centre. We adopted
the 5-step SAM Scale for Valence taken from [
2
] to simplify the
continuous labelling process.
Forty participants took part in the study all female with an aver-
age age of 28. Participants were scheduled to take part in the study
in order to collect data around Nottingham city centre. The street
we selected for the experiment is a pedestrianised shopping street in
the centre of Nottingham. The street hosts some of the stylish shops
and boutiques housed inside elegant buildings along with a number
of cafes for shoppers to rest in. Data was collected under similar
weather conditions (average 20
◦
degrees), at around 11am.During
the data collection process 550,432 data lines of locations traces
were collected as well as 5,345 self-report responses.
4 DISCUSSION
We explore the use of Voronoi diagrams to map the impact phys-
iological and environmental sensor data have within an urban
environment. Voronoi is a computational geometry algorithm that
allows the visualisation of data [
3
]. It works by dening a set of
regions called cells. Once the diagram has been created the size
of the overall cells give an indication of the density of the area an
object is in or the size of the object itself.
Figure 3 demonstrates self-reported emotions data using the
EnvBodySens app within a city environment. To represent the
metric value of each record, we apply a linear mapping from the
value to the color hue, which forms a blue to red rainbow palette. In
particular, the red polygons highlight there was negative sentiment
reported by the user. The green polygons aim to highlight where
there was a positive sentiment.
Within the diagram, the negative sentiments, depicted in red
polygons, demonstrates where participants’ needed to respond to an
Figure 3: Voronoi showing self-report emotions data within
a fragrance and Beauty Shop in Nottingham.
assistant within a shop. This correlation demonstrates the negative
impact on an individual’s well-being. We see these results being
of particular importance to city planners, to help them design and
understand what individual’s needs in city centres to help reduce
stress and improve individual well-being. Our visualisation tools
have enabled the mapping of all collected variables (physiological
and environmental) as separate layers.
Figure 4: Voronoi presenting the heart rate of participants
whilst visiting a popular clothes shop. in Nottingham.
Figure 4 shows two Voronoi diagrams displaying the heart rate
(HR) when participants entered a range of dierent urban environ-
ments. The darker colours used indicate higher levels of heat rate.
Through analysis of sensor data we have discovered a trend when
shopping, in that heart rate frequently increases when encountering
discounted items in the shop.
Using Voronoi diagrams to express data visually oers many
opportunities to plot correlations and observe patterns. Figure 5
presents a map of a street in Nottingham City Centre where Voronoi
diagrams have been created using the heart rate and location sensor
data. By using location tracking and the utilisation of Voronoi

UbiComp ’20, September 12–16, 2020, Cancun, Mexico Johnson, Kanjo and Woodward
diagrams plotted on to maps has clearly shown the areas where
there is an increase in heart rate and stress.
Figure 5: Heart rate Voronoi diagram overlays of dierent
shops in popular Nottingham shopping street.
We have found that these Voronoi charts can be utilised by city
planners to explore areas that induce the most stress such as a
busy shopping environment and create calming spaces to develop
a restorative experience. By enabling people to reduce stress and
improve their well-being will have positive impacts on cities ur-
ban environments by creating a positive experience, potentially
helping to increase the number of visitors and improve shopping
experiences.
5 CONCLUSION
We have shown how Voronoi diagrams demonstrate great potential
to spatially visualise and explore emotional and physiological sen-
sor data. This spatial exploration enables city planners to develop
urban environments that can help to reduce stress and improve
well-being. It is hoped that city centres could utilise this data to add
calming areas such as green spaces in environments which are con-
sidered as high stress. By understanding the environments which
induce stress and those that aid relaxation we hope future urban
developments can use this method of data visualisation to create
restorative experiences in high stress environments to improve
mental well-being. In the future, we envision the use of Voronoi
charts should continue to be explored in urban environments. With
more participants, additional sensor data such as additional envi-
ronmental measurements (gases, particulate matter, noise) and in
more urban city environments will help to evaluate the full poten-
tial this form of spatial exploration enables and further understand
how urban environments impact mental well-being.
REFERENCES
[1]
Nouf Alajmi, Eiman Kanjo, Nour El Mawass, and Alan Chamberlain. 2013. Shop-
mobia: An Emotion-Based Shop Rating System. In 2013 Humaine Association
Conference on Aective Computing and Intelligent Interaction. IEEE, 745–750.
[2]
Ellen Banzhaf, Francisco De La Barrera, Annegret Kindler, Sonia Reyes-Paecke,
Uwe Schlink, Juliane Welz, and Sigrun Kabisch. 2014. A conceptual framework
for integrated analysis of environmental quality and quality of life. Ecological
Indicators (2014).
[3]
Adam Dobrin. 2005. A Review of Properties and Variations of Voronoi Diagrams.
Whitman.Edu (2005).
[4]
David Donaire-Gonzalez, Antònia Valentín, Erik van Nunen, Ariadna Curto, Al-
bert Rodriguez, Mario Fernandez-Nieto, Alessio Naccarati, Sonia Tarallo, Ming Yi
Tsai, Nicole Probst-Hensch, Roel Vermeulen, Gerard Hoek, Paolo Vineis, John
Gulliver, and Mark J. Nieuwenhuijsen. 2019. ExpoApp: An integrated system to
assess multiple personal environmental exposures. Environment International
(2019).
[5]
Qiang Du, Vance Faber, and Max Gunzburger. 1999. Centroidal Voronoi tessella-
tions: Applications and algorithms. SIAM Rev. (1999).
[6]
Nour El Mawass and Eiman Kanjo. 2013. A supermarket stress map. In UbiComp
2013 Adjunct - Adjunct Publication of the 2013 ACM Conference on Ubiquitous
Computing.
[7]
J.A. Healey and R.W. Picard. 2005. Detecting Stress During Real-World Driving
Tasks Using Physiological Sensors. IEEE Transactions on Intelligent Transportation
Systems 6, 2 (jun 2005), 156–166.
[8]
Eiman Kanjo. 2010. NoiseSPY: A real-time mobile phone platform for urban noise
monitoring and mapping. Mobile Networks and Applications (2010).
[9]
Eiman Kanjo, Eman M.G. Younis, and Nasser Sherkat. 2018. Towards unravelling
the relationship between on-body, environmental and emotion data using sensor
information fusion approach. Information Fusion (2018).
[10]
Stephen Kaplan. 1995. The restorative benets of nature: Toward an integrative
framework. Journal of Environmental Psychology (1995).
[11]
Tammy Kohlleppel, Jennifer Campbell Bradley, and Steve Jacob. 2002. A walk
through the garden: Can a visit to a botanic garden reduce stress? HortTechnology
(2002).
[12]
D G Krige. 1952. A Statistical Approach to Some Basic Mine Valuation Problems
on the Witwatersrand. Journal of the Chemical, Metallurgical and Mining Society
of South Africa (1952).
[13] George MacKerron. 2012. Mappiness.
[14]
Sorin Matei, Sandra J. Ball-Rokeach, and Jack Linchuan Qiu. 2001. Fear and misper-
ception of Los Angeles urban space: A spatial-statistical study of communication-
shaped mental maps. Communication Research (2001).
[15]
Ruturaj N. Mody, Katharine S. Willis, and Roland Kerstein. 2009. WiMo: Location-
based emotion tagging. In MUM 2009 - Proceedings of the 8th International Con-
ference on Mobile and Ubiquitous Multimedia.
[16] Perkbox. 2018. THE 2O18 UK WORKPLACE STRESS SURVEY. (2018).
[17]
Mark S. Rosenbaum. 2009. Restorative servicescapes: Restoring directed attention
in third places. Journal of Service Management (2009).
[18]
Stephen A. Stansfeld and Mark P. Matheson. 2003. Noise pollution: Non-auditory
eects on health.
[19]
Nigel Thrift. 2003. Space: the fundamental stu of geography. Key concepts in
geography (2003).
[20]
Kieran Woodward, Eiman Kanjo, David Brown, and T.M. McGinnity. 2020. On-
Device Transfer Learning for Personalising Psychological Stress Modelling Using
a Convolutional Neural Network. In On-device Intelligence Workshop, MLSys ,
Austin, Texas.
[21]
Kieran Woodward, Eiman Kanjo, David Brown, T. M. McGinnity, Becky Inkster,
Donald J Macintyre, and Athanasios Tsanas. 2019. Beyond Mobile Apps: A Survey
of Technologies for Mental Well-being. IEEE Transactions on Aective Computing
(2019).
[22]
Kieran Woodward, Eiman Kanjo, Andreas Oikonomou, and Samuel Burton. 2018.
Emoecho: A tangible interface to convey and communicate emotions. In Ubi-
Comp/ISWC 2018 - 2018 ACM International Joint Conference on Pervasive and
Ubiquitous Computing and Proceedings of the 2018 ACM International Symposium
on Wearable Computers.
[23]
Kieran Woodward, Eiman Kanjo, Andreas Oikonomou, and Alan Chamberlain.
2020. LabelSens: enabling real-time sensor data labelling at the point of col-
lection using an articial intelligence-based approach. Personal and Ubiquitous
Computing (jun 2020), 1–14.
[24]
Piero Zappi, Elizabeth Bales, Jing Hong Park, William Griswold, and Tajana
Šimuni. 2012. The CitiSense Air Quality Monitoring Mobile Sensor Node. In
Proceedings of the 11th ACM/IEEE Conference on Information Processing in Sensor
Networks.