No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Designing an Escape Room in the City for Public Engagement with AI-enhanced Surveillance
Abstract
Escape the Smart City is a critical pervasive game that uses an escape room format to help players develop an understanding of the implications of urban surveillance technologies. Set in downtown Amsterdam, players work together as a team of hackers to stop the mass deployment of an all-seeing AI-enhanced surveillance system. In order to defeat the system players need to understand its attributes and exploit its weaknesses. Novel gameplay elements include locating hidden surveillance cameras in the city, discovering and exploiting algorithmic biases in computer vision, and exploring new techniques to avoid facial recognition systems. This work makes two distinct contributions to the CHI community: first, it introduces critical pervasive games as an approach to engage the public in complex sociotechnical issues, and second, it experiments with the escape room format as a platform for critical play.
... System helper, information assistant (Ahuja & Khosla, 2019;Amaro & Oliveira, 2019;Casals et al., 2017;Cherner et al., 2019;Fahlquist et al., 2011;Fischöder et al., 2018;Fraternali et al., 2017;Innocent, 2016;Kihara et al., 2019;Laine & Sedano, 2015;Miglino et al., 2014;Monge & Postolache, 2018;Mylonas et al., 2019;Oliveri et al., 2019;Õunapuu, 2015;Palakvangsa-Na-Ayudhya et al., 2017;Papaioannou et al., 2018;Pokric et al., 2015;Rock Zou et al., 2015;Rowland, 2015;Spyrou et al., 2018;Spyrou et al., 2018;Tziortzioti et al., 2018;Wang & Hu, 2017;Williams et al., 2019) 25 ...
... Tactile, olfactory, gustatory stimulus; physical movement based control (Agyeman & Al-Mahmood, 2019;Alexandre et al., 2019;Amaro & Oliveira, 2019;Ardito et al., 2018;Ben-Moussa et al., 2017;Butgereit & Martinus, 2016;Chen et al., 2017;Fischöder et al., 2018;Gabrielli et al., 2014;Gawley et al., 2016;Hong & Cho, 2018;Hwang et al., 2012;Karime et al., 2012;Kihara et al., 2019;Kobeissi et al., 2017;Konstantinidis et al., 2014;Krommyda et al., 2018;Laine & Sedano, 2015;Madar et al., 2014;Miglino et al., 2014;Monge & Postolache, 2018;Oliver et al., 2018;Palakvangsa-Na-Ayudhya et al., 2017;Pargman et al., 2017;Postolache et al., 2019;Pozzi & Sgardelis, 2016;Radeta et al., 2019;Song et al., 2016;Spyrou et al., 2018;Van Der Helm, 2008;Wang & Hu, 2017;Wilkowska et al., 2015) 32 (Continued ) individual engagement predictably involved less social incentive compared with multi-user engagement. Surprisingly, public engagement appeared to involve the least social incentive. ...
... As technology matures, IeG that can support public engagement may become a new hot area. In this literature review, we identified three papers that have researched (Alexandre et al., 2019;Ben-Moussa et al., 2017;Casals et al., 2017;Chen et al., 2017;Cherner et al., 2019;Fahlquist et al., 2011;Fischöder et al., 2018;Garcia-Garcia et al., 2017;Henry et al., 2018;Kihara et al., 2019;Konstantinidis et al., 2014;Krommyda et al., 2018;Lu, 2018;Madar et al., 2014;Oliveri et al., 2019;Õunapuu, 2015;Pargman et al., 2017;Postolache et al., 2019;Rock Zou et al., 2015;Rowland, 2015;Song et al., 2016;Wang & Hu, 2017;Williams et al., 2019) 23 ...
Engagement is a common goal pursued by most social and technical systems, because of its widely acknowledged effects on enhancing user acceptance and performance. Previous research has shown that a system’s ability to engage users involves two known aspects: the technology foundation that determines the interactive paths for engaging users and the design methodology that determines the atop user experience to be conveyed through those paths. In recent years, an emerging and promising engagement approach that integrates both an advanced technology stack and novel design methodology, i.e. IoT-enabled Gamification (IeG), has attracted wide interest from both public and private sectors. This article aims to conduct a systematic review to answer some fundamental questions. 75 papers were reviewed under a 3-axis analysis framework of user engagement, the majority of which indicated that IeG is linked to increased engagement in a variety of application domains, stages, and population scales.
... Similarly, artist David Rokeby's media installation "Sorting Daemon" [37] in Toronto adopt a surveillance system composed of a computer, a LCD screen and a camera, to extract image patches of pedestrian walking in a public street, then the person patches are separated into coloured regions which are sorted and reassembled into composite pictures for the display. Kihara et al. [21] design a critical pervasive game based on surveillance cameras with an escape room theme, using surveillance cameras to raise public awareness about the socio-technical implications of AI-enhanced urban surveillance. In contrast, Albrechtslund and Dubbeld [3] propose an positive view of surveillance, suggesting it can be protective as well as playful and enjoyable. ...
... However, the public nature of body language performance in public areas, along with the video clips recorded by the surveillance system, will effectively deter such attempts. (3) Previous studies have raised various criticism and questions regarding the widespread use of surveillance technologies [39] [21]. However, surveillance technology is inherently two-faced, embodying both "care-and-control" aspects [28]. ...
In this paper, we create "Love in Action" (LIA), a body language-based social game utilizing video cameras installed in public spaces to enhance social relationships in real-world. In the game, participants assume dual roles, i.e., requesters, who issue social requests, and performers, who respond social requests through performing specified body languages. To mediate the communication between participants, we build an AI-enhanced video analysis system incorporating multiple visual analysis modules like person detection, attribute recognition, and action recognition, to assess the performer's body language quality. A two-week field study involving 27 participants shows significant improvements in their social friendships, as indicated by self-reported questionnaires. Moreover, user experiences are investigated to highlight the potential of public video cameras as a novel communication medium for socializing in public spaces.
... A few studies have taken this challenge up by simulating smart city surveillance technologies, processes, and practices to participants. Examples include visual and verbal vignettes based on different kinds of surveillance technologies (Mariën and Poels 2020) or scenarios of smart city futures (Butot et al. 2020;Jameson, Richter, and Taylor 2019), gamified surveys challenging players to identify surveillant objects in a virtual smart city (Rijshouwer, Leclercq, and van Zoonen 2022), and escape rooms with surveillance storylines (Kihara, Lomas, and Bendor 2019). Although these studies have slightly different epistemological approaches to the phenomenon of "subjective experience," all highlight feelings of exposure to smart city surveillance, which is experienced as opaque in its functioning (Butot et al. 2020;Jameson, Richter, and Taylor 2019;Rijshouwer, Leclercq, and van Zoonen 2022;Mariën and Poels 2020). ...
Smart cities are commonly seen as places that are defined by surveillance because of their reliance on vast amounts of digital data to improve urban management challenges. Although the infrastructures and technologies that enable smart city surveillance pervade multitudinous urban spaces and everyday practices, they are often “hiding in plain sight,” going unnoticed in the bustle of everyday life. Hence, fostering research settings where citizens can productively reflect on their everyday surveillance constitutes a major challenge for the interrelated projects of doing empirical research about subjective experiences of smart city surveillance and the inclusion of citizens in smart city discussions. Drawing on walking as a method, this study attempts to meet this challenge by developing and empirically testing a methodology of purposive “data walking.” Situating the research in Rotterdam, the Netherlands, participants are instructed to identify data points for public safety purposes on a short walk through the city and reflect on their experiences. Observations and experiences of smart city surveillance are documented with photos, text descriptions, and audio notes, which are shared in real-time with researchers and provide the basis for group reflections. These walks and reflections generate rich visual and textual data that yield insights into embodied and situated constructions of smart city surveillance as an object of subjective inquiry, experiences of visibility, considerations of agency and evaluations of public safety implications. The study considers these empirical results in conjunction with reflections on the methodology, contributing to further methodological explorations for including citizens in smart city discussions and surveillance subjectivity research.
... (Some examples of artificial intelligence models in a similar domain are given in: Ai & Kwon (2020), Kihara et al. (2019)) The applications of graceful degradation can be split into two sections, one where the system deals with out-of-distribution data in a self-contained manner, and one where the system can take additional inputs to improve its performance in the observed distribution. These two approaches can be split into passive and active approaches. ...
When machine learning models encounter data which is out of the distribution on which they were trained they have a tendency to behave poorly, most prominently over-confidence in erroneous predictions. Such behaviours will have disastrous effects on real-world machine learning systems. In this field graceful degradation refers to the optimisation of model performance as it encounters this out-of-distribution data. This work presents a definition and discussion of graceful degradation and where it can be applied in deployed visual systems. Following this a survey of relevant areas is undertaken, novelly splitting the graceful degradation problem into active and passive approaches. In passive approaches, graceful degradation is handled and achieved by the model in a self-contained manner, in active approaches the model is updated upon encountering epistemic uncertainties. This work communicates the importance of the problem and aims to prompt the development of machine learning strategies that are aware of graceful degradation.
... [7] In addition, Death on the Nile was designed as a pervasive game because it is a role-playing game where the experience is extended out in the real world, or where the fictive world in which the game takes place blends with the physical world. [14] The story plot took place in Egypt, in the Kings' Valley, where visitors were called to solve the mystery around a murder in the tomb of a Pharaoh. More in detail, participants were invited to exploit interactive technologies to help the researchers to identify the assassin. ...
In a world where public engagement is increasingly important, where there is the urge of leaving the research laboratories to tell people what is being done, and where the effort from Academy is still limited, we propose Death on the Nile, a novel experience to get people in touch with innovative interactive technologies. In our exhibition, visitors are invited to solve a crime while they get in touch with the new frontiers of Human-Computer Interaction. Also, participants can experience the different stages of the research process through the metaphor of the investigation. An empirical study (N=969) shows that Death on the Nile is engaging and effective as a method through which to present HCI research. Finally, we conceptualize the experience into a framework that can be used with potentially any interactive technology.
... They also give opportunities to researchers to study group dynamics under the locked conditions of a room and study the behavior of participants. The escape rooms have been used to raise awareness about complex sociotechnical issues, like the use of surveillance technology in cities [11] and they have been also used as testbeds for different learning settings [12]. ...
A peripheral museum with a nevertheless important collection of ancient items, wished to increase its visibility and digital presence in order to become more known and attract new visitors. Among the different solutions designed and implemented, we also designed a series of games of different kinds (i.e. mini vs complex games) for different purposes, like profiling of potential visitors and advertising the venue in social media. Based on lessons learnt from physical games played at museums of the same type in the same region, we designed and implemented a museum escape game. The game is presented here, to provide implementation details regarding the concept, the game mechanics, the interface design, the technical details, as well as the game art. We conclude with primary user testing and future steps that include plans for creating museum escape games with different technologies, like augmented reality and virtual reality.
When machine learning models encounter data which is out of the distribution on which they were trained they have a tendency to behave poorly, most prominently over-confidence in erroneous predictions. Such behaviours will have disastrous effects on real-world machine learning systems. In this field graceful degradation refers to the optimisation of model performance as it encounters this out-of-distribution data. This work presents a definition and discussion of graceful degradation and where it can be applied in deployed visual systems. Following this a survey of relevant areas is undertaken, novelly splitting the graceful degradation problem into active and passive approaches. In passive approaches, graceful degradation is handled and achieved by the model in a self-contained manner, in active approaches the model is updated upon encountering epistemic uncertainties. This work communicates the importance of the problem and aims to prompt the development of machine learning strategies that are aware of graceful degradation.
Deciding what kind of smart cities do we, as a society, want is not only a political question but also a matter of envisioning possible futures. The speculative narratives that designers produce to support their imagination are called "Design Fictions." We share SUBMERGED, a cross-platform project that combines game design, interactive narrative, and urban exploration with the objective of empowering citizens to produce their design fictions. Following a "Research through Design" practice, we describe our process for creating SUBMERGED, we synthesize some critical insights from our experience, and we urgently call for a dialogue between semioticians and design researchers on these topics.
Game-based learning (GBL) is often found to be technologically driven and more often than not, serious games for instance, are conceptualised and designed solely for digital platforms and state of the art technologies. To encourage a greater discussion on the potential benefits and challenges of a more holistic approach to developing GBL that promote human centered interactions and play for learning, the authors present the escapED programme. The escapED programme was conceived following the recent entertainment trend of escape rooms and is used for developing non-digital GBL approaches within education. escapED aids the design and creation of educational Escape Rooms and Interactive Gaming Experiences for staff and students in further/higher education settings. The paper first presents a pilot study that was used to assess the feasibility and acceptance of University teaching staff of embedding interactive GBL into a higher education environment. The authors then present the escapED theoretical framework that was used to create the prototype game for the pilot study as a tool to aid future design and development of on-site interactive experiences. The paper also presents an external developer report of using the escapED framework to develop a prototype game for teaching research methods to Southampton University students. Finally, the authors present a discussion on the use of the escapED framework so far and plans for future work and evaluation in order to provide engaging alternatives for learning and soft skills development amongst higher education staff andstudents.
Debates about the future of urban development in many Western countries have been increasingly influenced by discussions of smart cities. Yet despite numerous examples of this 'urban labelling' phenomenon, we know surprisingly little about so-called smart cities, particularly in terms of what the label ideologically reveals as well as hides. Due to its lack of definitional precision, not to mention an underlying self-congratulatory tendency, the main thrust of this article is to provide a preliminary critical polemic against some of the more rhetorical aspects of smart cities. The primary focus is on the labelling process adopted by some designated smart cities, with a view to problematizing a range of elements that supposedly characterize this new urban form, as well as question some of the underlying assumptions/contradictions hidden within the concept. To aid this critique, the article explores to what extent labelled smart cities can be understood as a high-tech variation of the 'entrepreneurial city', as well as speculates on some general principles which would make them more progressive and inclusive.
The first book to be published on the work of their partnership (in 2001), Design Noir is the essential primary source for understanding the theoretical and conceptual underpinnings for Dunne & Raby's work.
Consisting of three elements - a 'manifesto' on the possibilities of designing with and for the 'secret life' of electronic objects; notes for an embryonic network of critical designers and, most famously, the presentation of the Placebo Project – a prototype for a critical design poetics enacted around electronic furniture-objects – Design Noir offers an in-depth exploration of one of the most seminal design projects of the last two decades, one that arguably initiated speculating through design in its contemporary forms.
By detailing the logic and character of the objects that were constructed; the involvement of users with these objects over-time, and in the creation of a new kinds of spatially and temporally distributed moments of critique and engagement with things, Design Noir presents the case-study of the Placebo projectas a far more complex and subtler project than is often thought.
As a bold and in many ways unprecedented experiment in design writing and book designing, Design Noir is itself an instance of the speculative propositional design it expounds.
An examination of subversive games—games designed for political, aesthetic, and social critique.
For many players, games are entertainment, diversion, relaxation, fantasy. But what if certain games were something more than this, providing not only outlets for entertainment but a means for creative expression, instruments for conceptual thinking, or tools for social change? In Critical Play, artist and game designer Mary Flanagan examines alternative games—games that challenge the accepted norms embedded within the gaming industry—and argues that games designed by artists and activists are reshaping everyday game culture.
Flanagan provides a lively historical context for critical play through twentieth-century art movements, connecting subversive game design to subversive art: her examples of “playing house” include Dadaist puppet shows and The Sims. She looks at artists' alternative computer-based games and explores games for change, considering the way activist concerns—including worldwide poverty and AIDS—can be incorporated into game design.
Arguing that this kind of conscious practice—which now constitutes the avant-garde of the computer game medium—can inspire new working methods for designers, Flanagan offers a model for designing that will encourage the subversion of popular gaming tropes through new styles of game making, and proposes a theory of alternate game design that focuses on the reworking of contemporary popular game practices.
This article argues for engaging with the smart city as a sociotechnical imaginary. By conducting a close reading of primary source material produced by the companies IBM and Cisco over a decade of work on smart urbanism, we argue that the smart city imaginary is premised in a particular narrative about urban crises and technological salvation. This narrative serves three main purposes: (1) it fits different ideas and initiatives into a coherent view of smart urbanism, (2) it sells and disseminates this version of smartness, and (3) it crowds out alternative visions and corresponding arrangements of smart urbanism. Furthermore, we argue that IBM and Cisco construct smart urbanism as both a reactionary and visionary force, plotting a model of the near future, but one that largely reflects and reinforces existing sociopolitical systems. We conclude by suggesting that breaking IBM’s and Cisco’s discursive dominance over the smart city imaginary requires us to reimagine what smart urbanism means and create counter-narratives that open up space for alternative values, designs, and models.
An internationally renowned architect, urban planner, and scholar describes the major technological forces driving the future of cities Since cities emerged ten thousand years ago, they have become one of the most impressive artifacts of humanity. But their evolution has been anything but linear—cities have gone through moments of radical change, turning points that redefine their very essence. In this book, a renowned architect and urban planner who studies the intersection of cities and technology argues that we are in such a moment. The authors explain some of the forces behind urban change and offer new visions of the many possibilities for tomorrow’s city. Pervasive digital systems that layer our cities are transforming urban life. The authors provide a front-row seat to this change. Their work at the MIT Senseable City Laboratory allows experimentation and implementation of a variety of urban initiatives and concepts, from assistive condition-monitoring bicycles to trash with embedded tracking sensors, from mobility to energy, from participation to production. They call for a new approach to envisioning cities: Futurecraft, a symbiotic development of urban ideas by designers and the public. With such participation, we can collectively imagine, examine, choose, and shape the most desirable future of our cities. © 2016 by Carlo Ratti and Matthew Claudel. All rights reserved.
We present a real-time facial expression recognition toolkit that can automatically code the expressions of multiple people simultaneously. The toolkit is available across major mobile and desktop platforms (Android, iOS, Windows). The system is trained on the world's largest dataset of facial expressions and has been optimized to operate on mobile devices and with very few false detections. The toolkit offers the potential for the design of novel interfaces that respond to users' emotional states based on their facial expressions. We present a demonstration application that provides real-time visualization of the expressions captured by the camera.
There is a certain allure to the idea that cities allow a person to both feel at home and like a stranger in the same place. That one can know the streets and shops, avenues and alleys, while also going days without being recognized. But as elites fill cities with "smart" technologies - turning them into platforms for the "Internet of Things" (IoT): sensors and computation embedded within physical objects that then connect, communicate, and/or transmit information with or between each other through the Internet - there is little escape from a seamless web of surveillance and power. This paper will outline a social theory of the "smart city" by developing our Deleuzian concept of the "spectrum of control." We present two illustrative examples: biometric surveillance as a form of monitoring, and automated policing as a particularly brutal and exacting form of manipulation. We conclude by offering normative guidelines for governance of the pervasive surveillance and control mechanisms that constitute an emerging critical infrastructure of the "smart city.".
This article argues that with increasingly large databases and computational power, profiling as a key part of security governance is experiencing major changes. Targeting mobile populations in order to enact security via controlling and sifting the good from the bad, profiling techniques accumulate and process personal data. However, as advanced algorithmic analytics enable authorities to make sense of unprecedented amounts of information and derive patterns in a data-driven fashion, the procedures that bring risk into being increasingly differ from those of traditional profiling. While several scholars have dealt with the consequences of black-boxed and invisible algorithmic analytics in terms of privacy and data protection, this article engages the effects of knowledge-generating algorithms on anti-discriminatory safeguards. Using the European-level efforts for the establishment of a Passenger Name Record (PNR) system as an example, and on the theoretical level connecting distinct modes of profiling with Foucauldian thought on governing, the article finds that with pattern-based categorizations in data-driven profiling, safeguards such as the Charter of Fundamental Rights of the European Union or the EU data-protection framework essentially lose their applicability, leading to a diminishing role of the tools of the anti-discrimination framework.
Obra que analiza las propiedades, ventajas, reacciones y significados que ofrece la narrativa interactiva frente a la narrativa lineal para entender cómo las historias median nuestra forma de pensar el mundo.
China's Social Credit System seeks to assign citizens scores engineer social behaviour ABC News
- V Xu
- B Xiao
Xu,V., & Xiao, B. (2018, Apr 2). China's Social Credit System seeks to assign citizens scores, engineer social behaviour,
ABC News. Retrieved http://bit.do/eEMSY
Radical technologies: The design of everyday life
- A Greenfield
Greenfield, A. (2017). Radical technologies: The design of everyday life. Verso Books.
UK citizens unaware of 'smart cities
- J Hatcher
Hatcher, J. (2015, July 1). UK citizens unaware of 'smart cities'.Smart Buildings. Retrieved http://bit.do/eEMJg