No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Crafting Trajectories of Smart Phone Use at the Opera
Abstract
Losing Her Voice is a new opera which highlights the challenges of subtly interweaving digital technologies into established cultural forms. Audience members were encouraged to use their own mobile phones to interact with on-stage projections before, during and after the performance. We chart the trajectories of participation that were designed into the premiere performances of this work, and how these unfolded in practice. We identify the strategies that were effective to: encourage adoption of the mobile app; interweave use with the other elements of the opera performance; make it consistent with the content; and ensure it complemented but was not essential to the show. We highlight the way in which three canonical trajectories must be woven together in this experience: the standard trajectory of “a night at the opera”; the dramatic arc of the specific work; and the audience member's use of their own device.
ResearchGate has not been able to resolve any citations for this publication.
This paper presents a case study of a long-term collaboration between a physical performance company and interactive digital artists. The collaboration has resulted in the creation of five major performance works which have toured internationally over several years. We argue that the interactive systems can be considered a 'material' which changes over time, shaping performer actions and being shaped by them in return. Based on detailed interviews with key stakeholders and our own personal reflections, we have identified several 'trajectories' that have evolved over the duration of each individual production and the entire body of work. These trajectories address a number of perspectives including the way performers interact with the system, the relationship between the dramaturgy and the interaction palette and the way the stakeholders conceive of the interactive system. The evolution of the technology itself has also been examined in terms of aesthetic capability, performance robustness, operational cost and complexity across the entire duration of the collaboration.
We report on the design, premiere and public evaluation of a multifaceted audience interface for a complex non-linear musical performance called Climb! which is particularly suited to being experienced more than once. This interface is designed to enable audiences to understand and appreciate the work, and integrates a physical instrument and staging, projected visuals, personal devices and an online archive. A public premiere concert comprising two performances of Climb! revealed how the audience reoriented to the second performance through growing understanding and comparison to the first. Using trajectories as an analytical framework for the audience 'journey' made apparent: how the trajectories of a single performance are embedded within the larger trajectories of a concert and the creative work as a whole; the distinctive demands of understanding and interpretation; and the potential of the archive in enabling appreciation across repeated performances.
Immersive theatre is a growing trend within theatre entertainment: audience members can now wander around performances and choose how the story unfolds in front of them. Technology can be used to create novel, multi-modal experiences for audiences in these performances; but when the rules of such an experience are ill-defined, how do users react to this technology? We present an evaluation of 25 performances of an immersive, in the dark performance. Issues of control can arise in situations where technology becomes an important part of such a performance. Participants take and relinquish control in three key areas: navigation, exploration and attention during the performance, and this affects their perception of both technology and the piece itself. We discuss how technology can play a positive role in immersive theatre and other cultural settings, yet its use must be carefully choreographed to ensure the audience experience matches the intended goal.
Television viewers interacting with second screens has become a common sight in the modern living room. Such activities are a mixture of related, semi-related, and non-related browsing of content. This growing trend is revolutionising the way that broadcasters think about their content. Through the envisioned connected home, driven by end-to-end IP connected networks, television content creators and app developers are now considering the design space for multi-device, interactive experiences. In this review paper, we consider the pre-digital beginnings of such scenarios and progress to discuss how the introduction of mobile devices has affected the TV viewing experience. We discuss dual-screen usage over a variety of contexts in the connected home, with a focus on ‘designed’ dual-screen experiences such as companion applications. We conclude with reflections on the future of this area so that app developers, broadcasters, and academics may push further the space and improve future dual- and multi-screen experiences.
Since a couple of years, opera houses have started transmitting selected performances via live streaming over the Internet, in order to make them accessible to a broader audience. In this context, the "Salome Experience" project, carried out at the University of Vienna in collaboration with Vienna State Opera, investigates innovative approaches for further closing the gap between stage and audience via modern communication technology. This paper discusses the prototypical realization of some of these ideas in the framework of a dedicated live streaming transmission of the opera "Salome" by Richard Strauss during a symposium celebrating the 25th anniversary of the Internet in Austria. Moreover we present a novel smartphone app which has evolved from this initiative and aims at providing the audience with "smart subtitles" to make stage performances more accessible, with a specific focus on the audiences of large open air events or streaming transmissions.
So many people are now using handheld second screens whilst watching TV that application developers and broadcasters are designing companion applications – second screen content that accompanies a TV programme. The nature of such dual-screen use cases inherently causes attention to be split, somewhat unpredictably. Dual-screen complexity, a clear factor in this attention split, is largely unexplored by the literature and will have an unknown (and likely negative) impact on user experience (UX). Therefore, we use empirical techniques to investigate the objective and subjective effect of dual-screen visual complexity on attention distribution in a companion content scenario. Our sequence of studies culminates in the deployment of a companion application prototype that supports adjustment of complexity (by either content curator or viewer) to allow convergence on optimum experience. Our findings assist the effective design of dual-screen content, informing content providers how to manage dual second screen complexity for enhanced UX through a more blended, complementary dual-screen experience.
The Smartphone Project (TSP) is an interactive dance-performance in a professional setting that exploits the communication channels provided by smartphone-apps as a new material in the dance-theatre domain. We present an account of the experience and its staging. Based on an initial study with 36 participants from the audience, we present results and discuss lessons learned from this project that might guide similar future work.
This article presents an empirical study investigating audience participation in an interactive theater performance. During the performance, audience members were enticed to act upon and contribute to the performance by sharing their opinions, emotions, values and other thoughts, by means of text messages that were integrated into the performance itself. The study aimed at understanding the main characteristics of audience participation in the interactive performance, as well as the role of communication technology as a medium enabling social participation. The results draw attention to the immediate and reflective facets of audience participation, both unfolding at two different but interrelated levels of interactions: an individual and collective one.
We explore the approach of performance-led research in the wild in which artists drive the creation of novel performances with the support of HCI researchers that are then deployed and studied at public performance in cultural settings such as galleries, festivals and on the city streets. We motivate the approach and then describe how it consists of three distinct activities -- practice, studies and theory -- that are interleaved in complex ways through nine different relationships. We present a historical account of how the approach has evolved over a fifteen-year period, charting the evolution of a complex web of projects, papers, and relationships between them. We articulate the challenges of pursuing each activity as well as overarching challenges of balancing artistic and research interests, flexible management of relationships, and finally ethics.
Design-oriented research practices create opportunities for constructing knowledge that is more abstracted than particular instances, without aspiring to be at the scope of generalized theories. We propose an intermediate design knowledge form that we name strong concepts that has the following properties: is generative and carries a core design idea, cutting across particular use situations and even application domains; concerned with interactive behavior, not static appearance; is a design element and a part of an artifact and, at the same time, speaks of a use practice and behavior over time; and finally, resides on an abstraction level above particular instances. We present two strong concepts—social navigation and seamfulness—and discuss how they fulfil criteria we might have on knowledge, such as being contestable, defensible, and substantive. Our aim is to foster an academic culture of discursive knowledge construction of intermediate-level knowledge and of how it can be produced and assessed in design-oriented HCI research.
An introduction to the special issue of 'The Turn to The Wild' is provided. The introduction reveals that the phrase 'in-the-wild' is becoming popular again in the field of human-computer interaction (HCI), describing approaches to HCI research and accounts of user experience phenomena that differ from those derived from other lab-based methods. Researchers are shifting into the wild, carrying out in-situ development and engagement, sampling experiences and probing people in their homes and on the streets. The aim of this special issue is to examine what this new direction entails and what it means for HCI theory, practice, and design. Researchers are evaluating prototypes in the context of how they are used and integrated within people's lives. This involves observing and recording what people do and how this changes over suitable periods of time.
In this essay, I explore several facets of research through design in order to contribute to discussions about how the approach should develop. The essay has three parts. In the first, I review two influential theories from the Philosophy of Science to help reflect on the nature of design theory, concluding that research through design is likely to produce theories that are provisional, contingent, and aspirational. In the second part, I discuss three possible interpretations for the diversity of approaches to research through design, and suggest that this variation need not be seen as a sign of inadequate standards or a lack of cumulative progress in the field, but may be natural for a generative endeavour. In the final section, I suggest that, rather than aiming to develop increasingly comprehensive theories of design, practice based research might better view theory as annotation of realised design examples, and particularly portfolios of related pieces. Overall, I suggest that the design research community should be wary of impulses towards convergence and standardisation, and instead take pride in its aptitude for exploring and speculating, particularising and diversifying, and - especially - its ability to manifest the results in the form of new, conceptually rich artefacts.
A long-term naturalistic study reveals how artists designed, visitors experienced, and curators and technicians maintained
a public interactive artwork over a four year period. The work consisted of a collaborative augmented reality game that ran
across eleven networked displays (screens and footpads) that were deployed along a winding ramp in a purpose-built gallery.
Reflections on design meetings and documentation show how the artists responded to this architectural setting and addressed
issues of personalisation, visitor flow, attracting spectators, linking real and virtual, and accessibility. Observations
of visitors reveal that while their interactions broadly followed the artists’ design, there was far more flexible engagement
than originally anticipated, especially within visiting groups, while interviews with curators and technicians reveal how
the work was subsequently maintained and ultimately reconfigured. Our findings extend discussions of ‘interactional trajectories’
within CSCW, affirming the relevance of this concept to describing collaboration in cultural settings, but also suggesting
how it needs to be extended to better reflect group interactions at multiple levels of scale.
The idea of interactional trajectories through interfaces has emerged as a sensitizing concept from recent studies of tangible interfaces and interaction in museums and galleries. We put this concept to work as a lens to reflect on published studies of complex user experiences that extend over space and time and involve multiple roles and interfaces. We develop a conceptual framework in which trajectories explain these user experiences as journeys through hybrid structures, punctuated by transitions, and in which interactivity and collaboration are orchestrated. Our framework is intended to sensitize future studies, help distill craft knowledge into design guidelines and patterns, identify technology requirements, and provide a boundary object to connect HCI with Performance Studies.
Mobile experiences that take place in public settings such as on city streets create new opportunities for interweaving the fictional world of a performance or game with the everyday physical world. A study of a touring performance reveals how designers generated excitement and dramatic tension by implicating bystanders and encouraging the (apparent) crossing of normal boundaries of behaviour. The study also shows how designers dealt with associated risks through a process of careful orchestration. Consequently, we extend an existing framework for designing spectator interfaces with the concept of performance frames, enabling us to distinguish audience from bystanders. We conclude that using ambiguity to blur the frame can be a powerful design tactic, empowering players to willingly suspend disbelief, so long as a safety-net of orchestration ensures that they do not stray into genuine difficulty.
Interactive displays are increasingly being distributed in a broad spectrum of everyday life environments: they have very diverse form factors and portability characteristics, support a variety of interaction techniques, and can be used by a variable number of people. The coupling of multiple displays creates an interactive “ecosystem of displays”. Such an ecosystem is suitable for particular social contexts, which in turn generates novel settings for communication and performance and challenges in ownership. This paper aims at providing a design space that can inform the designers of such ecosystems. To this end, we provide a taxonomy that builds on the size of the ecosystem and on the degree of individual engagement as dimensions. We recognize areas where physical constraints imply certain kinds of social engagement, versus other areas where further work on interaction techniques for coupling displays can open new design spaces.
Interaction is increasingly a public affair, taking place in our theatres, galleries, museums, exhibitions and on the city streets. This raises a new design challenge for HCI, questioning how a performer s interaction with a computer experienced is by spectators. We examine examples from art, performance and exhibition design, comparing them according to the extent to which they hide, partially reveal, transform, reveal or even amplify a performerts manipulations. We also examine the effects of these manipulations including movements, gestures and utterances that take place around direct input and output. This comparison reveals four broad design strategies: `secretive,' where manipulations and effects are largely hidden; `expressive,' where they are revealed, enabling the spectator to fully appreciate the performer's interaction; `magical,' where effects are revealed but the manipulations that caused them are hidden; and finally `suspenseful,' where manipulations are apparent, but effects only get revealed when the spectator takes their turn.
The gap between research and design practice has long been a concern for the HCI community. In this article, we explore how different translations of HCI knowledge might bridge this gap. A literature review characterizes the gap as having two key dimensions—one between general theory and particular artefacts and a second between academic HCI research and professional UX design practice. We report on a 5-year engagement between HCI researchers and a major media company to explore how a particular piece of HCI research, the trajectories conceptual framework , might be translated for and with UX practitioners. We present various translations of this framework and fit them into the gap we previously identified. This leads us to refine the idea of translations , suggesting that they may be led by researchers, by practitioners or co-produced by both as boundary objects . We consider the benefits of each approach.
We present a case study of how Human-Computer Interaction (HCI) theory is reused within the field. We analyze the HCI literature in order to reveal the impact of one particular theory, the trajectories framework that has been cited as an example of both contemporary HCI theory and a strong concept that sits between theory and design practice. Our analysis of 60 papers that seriously engaged with trajectories reveals the purposes that the framework served and which parts of it they used. We compare our findings to the originally stated goals of trajectories and to subsequent claims of its status as both theory and strong concept. The results shed new light on what we mean by theory in HCI, including its relationship to practice and to other disciplines.
Most contemporary Western performing arts practices restrict creative interactions from audiences. Open Symphony is designed to explore audience-performer interaction in live music performances, assisted by digital technology. Audiences can conduct improvising performers by voting for various musical "modes." Technological components include a web-based mobile application, a visual client displaying generated symbolic scores, and a server service for the exchange of creative data. The interaction model, app, and visualization were designed through an iterative participatory design process. The system was experienced by about 120 audience and performer participants (35 completed surveys) in controlled (lab) and real-world settings. Feedback on usability and user experience was overall positive, and live interactions demonstrate significant levels of audience creative engagement. The authors identified further design challenges around audience sense of control, learnability, and compositional structure. This article is part of a special issue on multimedia for enriched music.
Tools for supporting performer-audience interaction have been gaining increasing interest in HCI community recently. They encompass a wide range of systems from simple polls to live tweeting and backchannel chats. However, a lack of unifying conceptual framework hampers their efficient development and deployment in events. In this paper, we develop a notion of live participation and present a live participation system that aims to capture performer-audience interaction systems' salient design-relevant characteristics. With user studies, we identify central characteristics of live participation (RQ1) and explore the diversity of types of live participation situations (RQ2). The identified concepts-extended performance, integration work, and episodes-provide the groundwork for further design of live participation systems and more engaging audience interactions.
We introduce ‘The interactive Audience Sensor Kit’ (TASK). This modular system of wirelessly networked sensors facilitates the augmentation of artistic performances, in particular, music events or visual art. It was conceived to enable low-level and low-cost audience interaction by offering a set of tracker-nodes to be arranged across the venue as demanded by the corresponding performance event. In this paper, we present two modules from TASK, which are currently under development, and provide early insights from a field study. (1) TASKswitch reacts to the presence of bodies acting as an on/off switch and thereby can be used to modify particular aspects of a performance. (2) TASKvector, on the other hand, enables more complex input by tracking movement among the audience. For example, as we will show in the paper, we used the modules to create interactive audio-visual experiences for the audience where projections were modulated by TASK.
Bootlegger is a system for creating multi-camera films of live music events using mobile devices. Using readily available technology and a synthesis of film-making conventions, the system coordinates music fans at live shows into an improvised film crew, suggesting shots, collating footage and generating rich metadata in real time. Bootlegger is part of a research project exploring adapting professional media workflows to amateur contexts in order to lower the bar to entry for media production. By enabling concert-goers to contribute to high-quality concert films, the system leverages mobile phone 'bootlegging' practices to support emerging musicians.
We define the concept of the Hyperaudience and a unique approach towards designing real-time interactive performance systems: the design of these systems encourages audience participation and augments the experience of audience members through interconnected networks. In doing so, it embraces concepts found in ubiquitous computing, affective computing, interactive arts, music, theatrical tradition, and pervasive gaming. In addition, five new systems are demonstrated to develop a framework for thinking about audience participation and orchestrating social co-presence in and beyond the performance space. Finally, the principles and challenges that shaped the design of these five systems are defined by measuring, comparing, and evaluating their expressiveness and communicability.
Media Scores provide a theoretical and technical means to orchestrate multiple modalities in the creation of expressive works of art and performance. New technologies afford numerous opportunities to tell stories and create expressive artworks through a variety of media. However, the tools of planning, composition, design, and creation of these works remain disjoint with respect to the artwork's constituent disciplines and from the final experience. Media Scores extend the metaphor of a musical score to other modalities in order to facilitate the process of authoring and performing multimedia compositions, providing a medium through which to realize a modern-day Gesamtkunstwerk. Research into the representation and encoding of expressive intent provides the conceptual underpinnings for the development of novel interfaces for composing and performing with Media Scores. Using such a tool, the composer is able to shape an artistic work that may be performed through human and technological means in a variety of media and utilizing various modalities of expression. Media Scores offer the possibility of authoring content that incorporates live performance data and the potential for audience participation and interaction. This paradigm bridges the extremes of the continuum from composition to performance, allowing for improvisatory compositional acts at performance-time. The Media Score also provides a common point of reference in collaborative productions, as well as the infrastructure for the real-time show control of any technologies used during a live performance.
An ethnographic study reveals how professional artists created a spectator interface for the interactive game Day of the Figurines, designing the size, shape, height and materials of two tabletop interfaces before carefully arranging them in a local setting. We also show how participants experienced this interface. We consider how the artists worked with a multi-scale notion of interactional trajectory that combined trajectories through individual displays, trajectories through a local ecology of displays, and trajectories through an entire experience. Our findings shed light on discussions within HCI concerning interaction with tangible and tabletop displays, spectator interfaces, ecologies of displays, and trajectories through cultural experiences.
Interactive mobile technologies have become part of audience experiences of live performances in terms of both general media sharing and specific (sometimes official) extra content. At the same time, high bandwidth affords streaming of live events to mobile devices. We take advantage of these technologies in our high resolution, panoramic image video stream and study a scenario of audience members viewing the very same live event they are watching on a tablet. The video stream on the tablet is navigational and enables audience members to pan and zoom in the real-time video feed. We studied audience interaction and impressions in three performances of a dance and music show and found distinct uses of the second screen video stream. We emphasize that despite initial reluctance, the observed utilization of the technology opened up for new potential practices. Our study shows how working with perceived conflict in technology can still open up design space for interactive technologies.
Theatre Engine is a three-phase project exploring the integration of mobile devices such as smart phones and tablet computers as user interaction mechanisms for live theatrical performances. This paper describes the technical details for the first performance in the series: Transitions, including system design, user interface details, and lessons learned. The user activities are described in the context of manipulations and the effects of those manipulations. The concept of blurring the distinction between real and virtual is explored. The prototype integrates Android-based mobile devices, servers, projection screens, and modern theatrical lighting with live musicians and dancers, allowing the audience to become a part of the presentation.
The partnership of theater and digital media shows much potential for new means of storytelling. Digital scenery can be joined to the performer's action on stage; algorithmic influences can blur the linearity of a drama; interactive technology offers novel means of involving the audience in the creation of the piece. Interaction can thus enhance the dramaturgical possibilities of traditional theater. However, the narrative task also accompanies various new challenges for the designers of such a play. On the basis of our dance performance Parcival XX-XI, we define requirements for making an audience interact in a theatrical play and introduce four interaction-enabling criteria for theatrical performances that use gestural interfaces.
We apply the HCI concept of trajectories to the design of a sculpture trail. We crafted a trajectory through each sculpture, combining textual and audio instructions to drive directed viewing, movement and touching while listening to accompanying music. We designed key transitions along the way to oscillate between moments of social interaction and isolated personal engagement, and to deliver official interpretation only after visitors had been given the opportunity to make their own. We describe how visitors generally followed our trajectory, engaging with sculptures and making interpretations that sometimes challenged the received interpretation. We relate our findings to discussions of sense-making and design for multiple interpretations, concluding that curators and designers may benefit from considering \'18trajectories of interpretation'.
Moori is a hybrid form of an audience participatory dynamic narrative and an audio-visual performance system. By incorporating a smart phone's dynamic interface and SMS, users share their thoughts to proposed questions by a performer. Through text inputs, buttons, and multi-touch pads, user-data is processed to generate algorithmic audio and visuals. The result is a collaboration between performer and audience members, a real-time audio-visual composition and a dramatic narrative. As authors, viewers transform the content in a collective narrative space.
In this work-in-progress we explore audience behavior at large musical events and present our prototype, a "cheering meter" developed to aid voting at rap competitions. We report from use of the cheering meter at eight concerts and conclude by highlighting how augmented interaction can increase the sense of participation among the audience at large-scale performances.
Video imaging method and apparatus for audience participation. Google Patents. Loren C. Carpenter. 1994. Video imaging method and apparatus for audience participation
- C Loren
- Carpenter
Performing mixed reality
- Steve Benford
- Gabriella Giannachi
- Benford Steve
On Site Opera To Launch Mobile App For Supertitles
- Bww News Desk
- News Desk BWW
Dialtones (A Telesymphony) - Mobile Phone Concert
- Golan Levin
- Levin Golan
Techniques for Interactive Audience Participation
- Dan Maynes-Aminzade
- Randy Pausch
- Steve Seitz
Powers live: a global interactive opera simulcast
- A Peter
- Benjamin Torpey
- Bloomberg
- Peter
Digital Live Art: mediating wittingness in playful arenas
- Jennifer Gayle Sheridan
- Sheridan Jennifer Gayle
Anthony Tommasini. 1995. Reinventing Supertitles: How the Met Did It
- Anthony Tommasini
- Tommasini Anthony
Opera Philadelphia hosts sci-fi simulcast with audience-app interaction
- David Ziegler
- Chris Montgomery
- Ziegler David