Content uploaded by Derek Lomas
Author content
All content in this area was uploaded by Derek Lomas on Feb 11, 2020
Content may be subject to copyright.
Designing an Escape Room in the City
for Public Engagement with
AI-enhanced Surveillance
Tomo Kihara
TU Delft, Industrial Design Engineering
Netherlands, Delft
playful.intervention@gmail.com
Roy Bendor
TU Delft, Industrial Design Engineering
Netherlands, Delft
r.bendor@tudelft.nl
Derek Lomas
TU Delft, Industrial Design Engineering
Netherlands, Delft
j.d.lomas@tudelft.nl
ABSTRACT1
Escape the Smart City is a critical pervasive game that uses an escape room format to help players
develop an understanding of the implications of urban surveillance technologies. Set in downtown
Amsterdam, players work together as a team of hackers to stop the mass deployment of an all-
seeing AI-enhanced surveillance system. In order to defeat the system players need to understand
its attributes and exploit its weaknesses. Novel gameplay elements include locating hidden
surveillance cameras in the city, discovering and exploiting algorithmic biases in computer vision,
and exploring new techniques to avoid facial recognition systems. This work makes two distinct
contributions to the CHI community: first, it introduces critical pervasive games as an approach to
engage the public in complex sociotechnical issues, and second, it experiments with the escape
room format as a platform for critical play.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee
provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the
full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses,
contact the owner/author(s).
CHI'19 Extended Abstracts, May 4–9, 2019, Glasgow, Scotland UK
© 2019 Copyright is held by the owner/author(s).
ACM ISBN 978-1-4503-5971-9/19/05.
https://doi.org/10.1145/3290607.3313003
CHI 2019 Late-Breaking Work
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
LBW1618, Page 1
KEYWORDS
Smart city; pervasive game; critical design;
speculative design; computer vision; artificial
intelligence; surveillance technology;
1 INTRODUCTION
1.1 Why raise awareness of AI-enhanced urban surveillance?
In recent years, the smart city has emerged as a new urban development paradigm. Its aim is to
transform cities into more efficient, responsive, and sustainable environments [10]. As a
sociotechnical imaginary [15], the smart city relies heavily on technological innovation, including
expansive networks of sensors, large data acquisition mechanisms, and algorithmic optimization of
complex urban systems [14]. Computer based, AI-enhanced surveillance plays a central role in this.
Where once humans were tasked with keeping an eye on the surveillance cameras that watched the
city, new AI-enhanced surveillance systems are taking over. Urban surveillance cameras are no longer
merely devices for recording images, but are becoming active agents that can evaluate what they see
and trigger intervention if they deem it necessary.
When integrated into law enforcement activities and deployed worldwide, these technologies raise
important ethical concerns [5][7][16]. First, since these systems are software-based, they may
potentially be installed on existing surveillance infrastructures without the public being aware of
their augmented capacities and potentially negative implications. This is amplified by the fact that
the large majority of smart city technologies are developed by large corporations (IBM, Cisco,
Alphabet, Alibaba), and in increasingly blurred jurisdictions [3]. Second, the datasets that are the
lifeblood of AI are vulnerable to the biases of those who collect and curate them [9]. Such biases may
lead to systematic racist, sexist, or other forms of discrimination. Lastly, the complexity of the
technologies seems to deter informed public discussion about their design and deployment. A UK
based data analytics firm has found that 96% of British people surveyed online were not aware of any
“smart city” initiatives being run by their local city council [6]. Our goal in this project is to raise
awareness of these issues to the wide public by designing and deploying a playful, non-didactic
experience: a critical, pervasive game.
2 APPROACH
2.1 Critical pervasive games
We approached the design of the game through the notions of critical design [2], critical play
[4], and pervasive gaming [11]. We wanted the game to provoke critical reflection about the
direction smart city technology is taking, while blurring the physical and temporal boundaries
between the game world and the ‘real’ world. Accordingly, the game blends relatively “safe” and
more riskier settings, and is set in a futuristic, speculative city. In this sense, the critical pervasive
game we propose here aims to immerse the players in a future alternative reality in order to
communicate latent socio-technical trends in relatable, experiential ways.
Drawing from previous research on the use of location-based games to raise awareness about
urban issues [e.g., 8], we hypothesized that a critical pervasive game will make the experience more
accessible for non-expert audiences, make the issues relatable via first-hand experience, and help
elicit in situ speculations about the future.
CHI 2019 Late-Breaking Work
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
LBW1618, Page 2
Figure 1: In-game footage of the Watcher system
recognizing individuals and showing their trust
score based on data from bank accounts and social
media.
Figure 2: The hall of the hacker guild where the
players are first invited to. The central screen
works as one of the UI during the game.
2.2 The escape room as a critical pervasive game
Despite their commercial popularity, there has been relatively little academic engagement with
the escape room genre in the field of serious games [1]. Our choice of the escape room as a game
format was based on three factors. First, since escape rooms are quite popular they provide a
familiar, almost intuitive set of game rules. Players, in other words, wouldn’t require a lengthy
introduction to the game’s structure and rules. Second, escape rooms provide a balanced mix of
linear and rhizomatic narrative structures [12]. While the overall flow of the game is linear, twists
and turns allow players to wander and reach different outcomes. This helps players feel like
protagonists in their own story – an experience with the potential to create a meaningful
emotional impact. Lastly, the escape room provides a provocative homology to the smart city.
Insofar as AI-enhanced surveillance seeks to tightly regulate urban behaviour, the game structure
offered players the opportunity to first experience, and then break free from such control, allowing
players to rehearse a new kind of activism.
Based on our approach, we formulated a central research question: Would a critical pervasive game
help citizens become critically aware of the implications of AI-enhanced surveillance in the city? Our
design objective was to increase such awareness by letting ordinary citizens play with the
technologies ‘behind’ AI-enhanced surveillance so they might become acquainted with the specific
technical, and broader societal implications. Derived from our research question, we formulated 3
design goals:
Design Goal 1. The game should help people engage with the biased and opaque nature of AI
(as a black box);
Design Goal 2. The game should sensitize people about existing surveillance infrastructures;
Design Goal 3. The game should help people become concerned about the social implications
of AI-enhanced surveillance in the city.
3. GAME DESIGN: ESCAPE THE SMART CITY
3.1 Narrative and Objective
This game takes place in a future Amsterdam in which Watcher, a full AI-enhanced surveillance
system, has been installed and is ready to be activated. Watcher has access to data from various
sensors and surveillance cameras, social media activity, and bank accounts. The system synthesizes
this data and assigns all citizens a social trust score Fig. 1. Important for gameplay, the system’s
computer vision is capable of advanced emotion and violence detection. The players work as a
team of hackers, trying to take down Watcher and prevent the city from entering a state of total
surveillance. Players are told they are the last remaining members of a Hacker-guild, an
underground organization dedicated to taking down Watcher Fig. 2. They are guided by Gan, a
member of the Hacker-guild, who explains the plight of fellow hacker Stanley who was caught
developing a virus that would take down Watcher. Before he disappeared, Stanley hid an SD card
with the virus somewhere in the city. The players need to hack into 3 layers of Watcher’s firewall,
find the SD card and upload the virus in order to delete the whole system. They have 60 minutes to
complete the task.
CHI 2019 Late-Breaking Work
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
LBW1618, Page 3
Figure 3: The trust score shown to the players
Figure 4: Players mask the faces in order to avoid
emotion recognition.
Figure 5: Players enact fake violence to an AI
which returns the probability of violence.
3.2 Game Progression
Introducing the trust score
: First, players learn that a social trust score ranging from 0 to 100 is
assigned to them by Watcher Fig. 3. The score, as the game narrator explains, dynamically changes
based on the in-game actions of the players. The participants start with a relatively low score of
around 30 and have to accomplish several tasks in order to increase the score. The integration of a
trust score helped make the game more realistic as it is similar to how equivalent systems are
currently deployed in China [17].
Encountering computer vision
: Players were exposed to the mechanics of computer vision early
in the game, when they were asked to present positive emotions to a camera (as part of raising
their trust score). The camera used the Affdex framework for realtime emotion recognition [13] to
detect seven emotions: happiness, surprise, contempt, sadness, smirk, disgust, and anger. Each
player had to go in front of the camera and demonstrate “happiness” until the screen indicated
that “All citizens in this area are trustworthy”.
Face masking
: After players increased the trust score to above 50 and were acknowledged by the
system as “trustworthy citizens”, they were asked to try and avoid facial recognition. Using a pair
of scissors and black tape, players were instructed to create their own mask. The masking exercise
was complete when the facial recognition system failed to identify players’ face Fig. 4.
Enacting fake violence
: Players learn that the Watcher system is vulnerable to hacking when the
system detects the act of extreme violence. In order to do this, players had to discover what the
system perceives as extreme violence by actually enacting it. Each frame taken by the camera was
analysed by Google’s Cloud Vision feature to detect inappropriate images depicting violence.
Players used a toy gun and additional masks to try and enact violence until they managed to
convince the system that extreme violence was indeed taking place Fig. 5.
Hunting surveillance cameras in the city
: In this phase, players ventured outside the safe
confinements of the room and into the city, looking for 4 surveillance cameras in the viscinity.
Surveillance cameras were located on existing utility poles, and each one had a numeric code
located somewhere on the post, to be used as a password later in the game Fig. 6.
Plot Twist/Reveal
: At the end of the game when players succeed in deleting the Watcher system,
Gan reveals that he was actually the Watcher system itself. Players learn that masking their face
and going out into the city was all part of Watcher’s attempts to train its machine learning model
to improve the detection of criminals that try to avoid facial detection.
CHI 2019 Late-Breaking Work
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
LBW1618, Page 4
Figure 6: Players searching for surveillance camera
outside with a mask on.
Limitations of the initial test-run
1.The number of players was small, thus limiting
our ability to draw more general conclusions.
2.The cultural and geographical background of
players filtered the way they experienced the
game, but, more importantly, influenced the
believability of the game’s narrative.
3.The game incorporated other existing tools
such as the Affdex emotion recognition
framework and Google’s Cloud Vision. The
availability and consistency of these may create
future problems.
4 EVALUATION & REFLECTION
Although the pilot run of the game was played by only four players, we find the initial results
encouraging. Players were selected based on two criteria: (1) they had no previous knowledge
about AI; and (2) they were not aware of the contentious nature of smart city technologies. Players
completed pre- and post-game surveys that included questions answerable on a 5-point Likert
scale. Questions targeted the social context of the game (surveillance in the smart city), and the
game’s design. In what follows we report on the survey results, adding qualitative data derived
from post-game interviews with players.
Design Goal 1
: The game should help people engage with the biased and opaque nature of AI (as a
black box). Post-game average scores rose from 2.75/5.0 to 3.25/5.0 (+0.5). During interviews players
indicated that having direct feedback on how the system sees the world created a deeper
understanding of the opaque nature of AI. This was evident when one of the players smiled, yet his
emotion was read as a “smirk”. This stirred a discussion among players on how unpredictable the
system is. Also, during the play-test, players were quite confused about how the AI system
classified things. When players were asked to enact fake violence, for instance, they were confused
about how the violence classifier changed depending on the gender of those appearing on camera.
Design Goal 2
: The design should sensitize people about existing surveillance infrastructures. Post-
game average scores rose from 3.25/5.0 to 4.25/5.0 (+1.0). Players indicated that using existing
surveillance infrastructures as part of the game worked well to create heightened awareness of the
extent to which surveillance cameras were already present in the city. After the play session, three
participants mentioned that they now recognize more surveillance cameras in their everyday life
than before. Players also told us that after playing the game they were much more concerned
about the possibility that existing surveillance cameras will be equipped with the kind of facial
recognition and advanced violence detection featured in the game.
Design Goal 3
: The design should help people become concerned about the implications of AI-
enhanced surveillance in the city. Post-game average scores rose from 2.75/5.0 to 4.25/5.0 (+1.5).
Working against time created a sense of urgency that resulted in increased player attentiveness,
and foregrounded the importance of the social issue. At the same time, considerations of AI as a
threat varied based on the players’ geographic background. A player from Germany considered the
threat to be a problem in a distant future and in a different place. This was because the future
Amsterdam depicted in the narrative resembled too much a totalitarian government which, the
player reasoned, would unlikely exist in the EU. On the other hand, other participants from East
Asian nations showed strong signs of concern.
Based on these initial results we have identified several limitations to the work reported here on
the left column.
CHI 2019 Late-Breaking Work
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
LBW1618, Page 5
Recommendations for successfully
designing a critical pervasive game.
1. Illustrating the common enemy as believable
as possible helped to motivate players and
attribute the game with an aura of realism.
2. Forcing players to wear a uniform or a mask
helped to separate the game world from the real
world.
3. Situating every individual in-game task within
the larger narrative helped make the experience
more cohesive.
4. Creating a ‘sensitizing’ phase at the beginning
of the game helped players make sense of the
narrative and get into role playing faster.
5. Adding a time trial aspect can create
immediate urgency and allow players to quickly
become immersed in the game's narrative.
Acknowledgement
This project was done in collaboration with
Waag. The authors would like to express their
gratitude to Tom Demeyer and Stefano Bocconi
and the team at CODE for the extensive support
in research and development.
5 CONCLUSIONS
In this paper we presented a critical pervasive game based on an escape room format. Although
we are only at the pilot stage, early results are encouraging. Players found the game both
entertaining and thought-provoking, and by engaging with actual AI systems, players appeared to
develop an improved understanding of the “black box” nature of AI and its potential consequences.
Also, by playing the game in the actual city, players developed a heightened awareness of existing
surveillance infrastructures while raising concerns about the future implications of urban AI-
enhanced surveillance. Our initial results lead us to conclude that critical pervasive games show
promise for generating awareness about complex sociotechnical issues because they render the
issues more accessible, legible, and relatable. Making use of the escape room format proved
successful not only because it expedited the game learning process, but also because it provided us
with room to experiment with game mechanics in a modular way while keeping the overall
experience intact and intuitive. Over the next few months we plan to run the game with more
participants in order to validate our initial results and improve the game design.
REFERENCES
[1] Clarke, S., Peel, D. J., Arnab, S., Morini, L., Keegan, H., & Wood, O. (2017). escapED: a framework for creating educational
escape rooms and Interactive Games For Higher/Further Education. International Journal of Serious Games, 4(3), 73-86.
[2] Dunne, A., & Raby, F. (2001). Design Noir: The secret life of electronic objects. Basel: August/Birkhäuser.
[3] Easterling, K. (2016). Extrastatecraft: The power of infrastructure space. London Dunne, A., & Raby,& NY: Verso.
[4] Flanagan, M. (2009). Critical play: radical game design. Cambridge, MA: MIT press.
[5] Greenfield, A. (2017). Radical technologies: The design of everyday life. Verso Books.
[6] Hatcher, J. (2015, July 1). UK citizens unaware of ‘smart cities’.Smart Buildings. Retrieved http://bit.do/eEMJg
[7] Hollands, R.G.(2008).Will the real smart city please stand up?Intelligent,progressive or entrepreneurial?.City,12(3).
[8] Korte, G.,& Ferri, G. (2018). Research Through Game Design : Interactive Stories from a Submerged Amsterdam, Ocula,
vol.19, no.19, pp.110-128.
[9] Leese, M. (2014). The new profiling: Algorithms, black boxes, and the failure of antidiscriminatory safeguards in the
European Union. Security Dialogue, 45(5), 494-511.
[10] Neirotti, P., De Marco, A., Cagliano, A. C., Mangano, G., & Scorrano, F. (2014). Current trends in Smart City initiatives:
Some stylised facts. Cities, 38, 25-36.
[11] Montola, M., Stenros, J., & Waern, A. (2009). Pervasive games: theory and design. CRC Press.
[12] Murray, J. H. (1997). Hamlet on the holodeck: the future of narrative in cyberspace. Cambridge, Mass.: MIT Press.
[13] McDuff, D., Mahmoud, A., Mavadati, M., Amr, M., Turcot, J., & Kaliouby, R. E. (2016, May). AFFDEX SDK: a cross-
platform real-time multi-face expression recognition toolkit. In Proceedings of the 2016 CHI Conference Extended
Abstracts on Human Factors in Computing Systems (pp. 3723-3726).
[14] Ratti, C & Claudel, M. (2016). The city of tomorrow: Sensors, networks, hackers, and the future of urban life.
[15] Sadowski, J. and R. Bendor. (2018). ‘Selling Smartness: Corporate Narratives and the Smart City as a Sociotechnical
Imaginary’. Science, Technology, & Human Values.
[16] Sadowski, J., & F. Pasquale (2015). The spectrum of control: A social theory of the smart city. First Monday, 20(7).
[17] Xu,V., & Xiao, B. (2018, Apr 2). China's Social Credit System seeks to assign citizens scores, engineer social behaviour,
ABC News. Retrieved http://bit.do/eEMSY
CHI 2019 Late-Breaking Work
CHI 2019, May 4–9, 2019, Glasgow, Scotland, UK
LBW1618, Page 6
