Figure 2 - uploaded by Joris Weijdom
Content may be subject to copyright.
Source publication
IETM, international network for contemporary performing arts, launches the 6th issue of its Fresh Perspectives publication series Mixed Reality and the Theatre of the Future, which explores the issues at stake when designing a mixed reality experience in the context of theatre and performance. The publications features a selection of projects descr...
Similar publications
This paper will present the role of the loops and the peculiarities of the mixed reality experience in the case of the performance of Illegitimate (stage adaptation by Adrian Sitaru, based on an original text by Adrian Sitaru and Alina Grigore). The author argues that the loops, defined by Manovich as “a new narrative form appropriate for the compu...
Citations
... In [13] multiple projections with moving screens are employed in order to re-create the idea of VR performance. Many other experiences used similar solutions [14]. ...
Studi sulla Realtà Nuova (in English, Etudes on the New Real) is a cycle of etudes created to explore the possibilities of XR in the context of live multimedia performance. Although many etudes have been created, only three etudes have been performed in public. Two of them are for augmented piano. The performer wears an Augmented Reality (AR) headset and hand trackers. This setup allows them to interact with Virtual Objects (VOs) and control visuals and audio. The environment used for the XR performance allows one to control up to 7 different video output and an n-speaker 3D audio configuration. VOs, as well as audiovisual outputs, react to the performer's gesture. The different aspects of performance develop following a pseudo-narrative outline. One of the key points of interest is the relationship between performer and audience, specifically in terms of transfer of the immersive experience: while the performer is immersed in an AR space, the audience can only see 2D screens. The experimented solution consisted in providing multiple projections and spatial audio. The paper will describe the etudes realized for augmented piano , from a technical and artistic point of view. Space will be also given to the discussion of issues in instrumental performance in an XR setting.
... Leite, 2022;L. M. Leite & Lafontana, 2016), as well as virtual reality and mixed reality technologies (Geigel, 2018;Rostami et al., 2017;Weijdom, 2017;Q. Wu, Boulanger, Kazakevich, & Taylor, 2010). ...
For recorded video content, researchers have proposed advanced concepts and approaches that enable the automatic composition and personalised presentation of coherent videos. This is typically achieved by selecting from a repository of individual video clips and concatenating a new sequence of clips based on some kind of model. However, there is a lack of generic concepts dedicatedly enabling such video mixing functionality for scenarios based on live video streams. This thesis aims to address this gap and explores how a live vision mixing process could be automated in the context of live television production, and, consequently, also extended to other application scenarios. This approach is coined the 'Virtual Director' concept. The name of the concept is inspired by the decision making processes which human broadcast TV directors are conducting when vision mixing live video streams stemming from multiple cameras. Understanding what is currently happening in the scene, they decide which camera view to show, at what point in time to switch to a different perspective, and how to adhere to cinematographic and cinematic paradigms while doing so. While the automation of vision mixing is the focus of this thesis, it is not the ultimate goal of the underlying vision. To automate for many viewers in parallel in a scalable manner allows taking decisions for each viewer or groups of viewers individually. To successfully do so allows moving away from a broadcast model where every viewer gets to see the same output. Particular content adaptation and personalisation features may provide added value for users. Preferences can be expressed dynamically, enabling interactive media experiences. In the course of this thesis, Virtual Director research prototypes are developed for three distinct application domains. Firstly, for distributed theatre performance, a script-based approach and a set of software tools are designed. A basic approach for the decision making process and a pattern how to decouple it into two core components are proposed. A trial validates the technology which does not implement full automation, yet successfully enables a theatre play. The second application scenario is live event 'narrowcast', a term used to denote the personalised equivalent to a 'broadcast'. In the context of this scenario, several computational approaches are considered for the implementation of an automatic Virtual Director with the conclusion to use and recommend a combination of (complex) event processing engines and event-condition-action (ECA) rules to model the decision making behaviour. Several content genres are subject to experimentation. Evaluation interviews provide detailed feedback on the specific research prototypes as well as the Virtual Director concept in general. In the third application scenario, group video communication, the most mature decision making behaviour is achieved. This behaviour needs to be defined in what can be a challenging process and is formalised in a model that is referred to as the 'production grammar'. The aforementioned pattern is realised such that a 'Semantic Lifting' process is processing low-level cue information in order to derive in more abstract, higher-level terms what is currently happening in the scene. The output of the Semantic Lifting process is informing and triggering the second process which is called the 'Director' decision making and eventually takes decisions on how to present the available content on screens. Overall, the exploratory research on the Virtual Director concept resulted in its successful application in the three domains, validated by stakeholder feedback and a range of informal and formal evaluation efforts. As a synthesis of the research in the three application scenarios, the thesis includes a detailed description of the Virtual Director concept. This description is contextualised by many detailed learnings that are considered relevant for both scholars and practitioners regarding the development of such technology.
The Nucleo de Dança Redes at the State University of Campinas has been developing multimodal artistic research in mixed reality in an interactive, immersive, dialogical, and in nets. In this article we present some research focus of the group referencing two works “De uma Margem a Outra” and “Jardim das Cartas”. Researchers in the areas of dance, music and audio visual have been raising discussions about an “expanded sound body” that stems from the interaction between movement, sound, music, video and technology in a mixed environment. A body that extends in sound, and an embodied sound in movement in an expanded perspective of the body in “presence” with more complex contours.
Another point is in the observation and understanding of the event of multimodal improvisation in a mixed environment as a path and an interactive and connective articulation between the communication agent that participate in the net as an expressive path that is shown in the interactive exploration. Based on interactive practices and on the concept of net, we work methodologically towards a “poetic interaction” in the field of complexity. A fertile ground in which improvisation and composition become coordinated paths for the unfolding of a dance which starts from the connectivity and from the interaction between sound, body movement, image and technology.
Досліджено специфіку втілення арт-практик змішаної реальності в новому медійному мистецтві. Проаналізовано основні відмінності між MR як техно-логічною інновацією та суто мистецькою практикою. Розглянуто визначальні художні засоби, що оформлюють образну виразність творів змішаної реальності. Виявлено аудіовізуальну природу арт-практик MR у площині нового медійного мистецтва. Встановлено трансгресивний характер взаємодії учасників арт-практик змішаної реальності з відповідними технологіями та виявлено основні естетичні принципи, притаманні артефактам MR. Визначено роль інтерактивності, віртуалізації та ігрової направленості як засобів художньої виразності у мистецьких практиках змішаної реальності.
Created for the 2019 Prague Quadrennial’s 36Q°, Blue Hour VR was a site-responsive mixed reality performative installation that placed the spectator, as experiencer, within a hybrid landscape of real-time three-dimensional computer graphics and 360-degree video. This article describes the design process, staging and experience of Blue Hour VR from the vantage point of its creators. Using a phenomenological perspective, the article discusses how Blue Hour VR staged presence and embodiment within an intermedial haptic experience. Blue Hour VR demonstrates how virtual reality technology can be harnessed by a mixed reality performance design, which includes both the material and virtual environment, creating a complex stratigraphy of intermedial textures and visual dramaturgies that co-exist inside, outside and in between perceptual realities. In doing so, the article aims to contribute to the limited body of work on mixed and virtual reality in the context of theatre and performance design.
This research project analyses the relationship between performing arts and new media
technologies. The project explores this interchange through the lens of a mobile application
protocol which, we submit, could be employed to augment drama performances through an
interactive approach. The research problem rests on the question of how modern technology has
shaped the perception of performing arts in the contemporary techno-culture. The research is
supported by expert interviews, all of which have bolstered my main finding that technology
acts as an extension of the outer-world, as well as society. Thus, the overarching contribution of
this research posits that performing arts should acknowledge this finding in order to reflect shifts
within the techno-culture.
