No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Beckett in VR: exploring narrative using free viewpoint video
Abstract
This poster describes a reinterpretation of Samuel Beckett's theatrical text Play for virtual reality (VR). It is an aesthetic reflection on practice that follows up an a technical project description submitted to ISMAR 2017 [O'Dwyer et al. 2017]. Actors are captured in a green screen environment using free-viewpoint video (FVV) techniques, and the scene is built in a game engine, complete with binaural spatial audio and six degrees of freedom of movement. The project explores how ludic qualities in the original text help elicit the conversational and interactive specificities of the digital medium. The work affirms the potential for interactive narrative in VR, opens new experiences of the text, and highlights the reorganisation of the author-audience dynamic.
... In the theatre, the audience passively observe the interaction; but, "in our VR version we acknowledge the role of the user as active; we recognise new opportunities for narrative and give the power of activation over to the end user, whose gaze becomes the spotlight. The user thus embodies the 'interrogator' and is empowered to independently discover the story, merely by looking at the characters" [15]. The user is placed in the centre and is surrounded by the three characters in urns, which are spaced far enough apart to allow the user to experience a natural sensation of movement, whilst exploring the three monologues. ...
... Volumetric video (VV) is a logical progression of photogrammetry technology, in the same way that film was the logical progression of the photograph. A VV shoot captures a performance in three dimensions using an array of video cameras surrounding the subject and simultaneously capturing physical live-action gestures from multiple angles (O'Dwyer and Johnson 2019;O'Dwyer et al. 2018), see Figure 5. The footage from all cameras is merged in postproduction processes that use a combination of advanced computer vision techniques, including 'a novel, multi-source shape-from-silhouette (MS-SfS) approach' and multi-view stereo (MVS) algorithms . ...
Site-specific performances are shows created for a specific location and can occur in one or more areas outside the traditional theatre. Social gathering restrictions during the Covid-19 lockdown demanded that these shows be shut down. However, site-specific performances that apply emergent and novel mobile digital technologies have been afforded a compelling voice in showing how performance practitioners and audiences might proceed under the stifling constraints of lockdown and altered live performance paradigms, however they may manifest. Although extended reality (XR) technologies have been in development for a long time, their recent surge in sophistication presents renewed potentialities for site-specific performers to explore ways of bringing the physical world into the digital to recreate real-world places in shared digital spaces. In this research, we explore the potential role of digital XR technologies, such as volumetric video, social virtual reality (VR) and photogrammetry, for simulating site-specific theatre, thereby assessing the potential of these content creation techniques to support future remote performative events. We report specifically on adapting a real-world site-specific performance for VR. This case study approach provides examples and opens dialogues on innovative approaches to site-specific performance in the post-Covid-19 era.
... As a new form of visual media, VV has started to be used more frequently in MR applications, including: remote communication and collaboration schemes [18], live-action reenactments for educational museum-guide applications for cultural heritage [17], and an AR version of the aforementioned Play by Samuel Beckett [16]. Play lends itself to MR because of Beckett's deep engagement of the notion of play. ...
Mixed reality (MR) technology is currently growing in popularity for applications in the cultural heritage domain. Furthermore, with the ability to be viewed with six degrees of freedom, volumetric video (VV) is presently being explored as a viable approach to content creation within this area. When combined, MR technology and VV present both practitioners and audiences with innovative approaches to the creation and consumption of both tangible and intangible representations of cultural significance. While there are some existing quantitative studies appraising these new technologies, the precise effects of MR in a cultural heritage context have yet to be fully explored. Here we show the results of a systematic evaluation of MR technology as applied in a cultural heritage context, where subject matter expert interviews were conducted to identify how virtual reality and augmented reality technologies are influencing the creative practices of domain experts and audience engagements with modern dramatic literature. Gathered from high-level stakeholders within the cultural heritage domain, our results highlighted the problems, concerns, and desires of users who must consider this technology in practice. We found that MR and VV content were considered by many to be disruptive technologies for the future of film, theater, and performance practice from the perspectives of both practitioners and audiences. We anticipate that these results will help future MR and VV projects to create meaningful content that is sympathetic to the needs and requirements of creators and audiences.
... Recent user studies have amplified the need for spatial audio to achieve presence in VR films [3][4][5]. A spatial audio signal (also referred to as 3D audio or 360 audio) is considered as a powerful way *These authors contributed equally to this work. of directing viewers' attention [6,7], however, requiring expensive sound-field microphones, professional sound recording and production tools [8,9]. ...
Ambisonics i.e., a full-sphere surround sound, is quintessential with 360-degree visual content to provide a realistic virtual reality (VR) experience. While 360-degree visual content capture gained a tremendous boost recently, the estimation of corresponding spatial sound is still challenging due to the required sound-field microphones or information about the sound-source locations. In this paper, we introduce a novel problem of generating Ambisonics in 360-degree videos using the audio-visual cue. With this aim, firstly, a novel 360-degree audio-visual video dataset of 265 videos is introduced with annotated sound-source locations. Secondly, a pipeline is designed for an automatic Ambisonic estimation problem. Benefiting from the deep learning-based audio-visual feature-embedding and prediction modules, our pipeline estimates the 3D sound-source locations and further use such locations to encode to the B-format. To benchmark our dataset and pipeline, we additionally propose evaluation criteria to investigate the performance using different 360-degree input representations. Our results demonstrate the efficacy of the proposed pipeline and open up a new area of research in 360-degree audio-visual analysis for future investigations.
... Recent user studies have amplified the need for spatial audio to achieve presence in VR films [3][4][5]. A spatial audio signal (also referred to as 3D audio or 360 audio) is considered as a powerful way *These authors contributed equally to this work. of directing viewers' attention [6,7], however, requiring expensive sound-field microphones, professional sound recording and production tools [8,9]. ...
Ambisonics i.e., a full-sphere surround sound, is quintessential with $360^\circ$ visual content to provide a realistic virtual reality (VR) experience. While $360^\circ$ visual content capture gained a tremendous boost recently, the estimation of corresponding spatial sound is still challenging due to the required sound-field microphones or information about the sound-source locations. In this paper, we introduce a novel problem of generating Ambisonics in $360^\circ$ videos using the audio-visual cue. With this aim, firstly, a novel $360^\circ$ audio-visual video dataset of 265 videos is introduced with annotated sound-source locations. Secondly, a pipeline is designed for an automatic Ambisonic estimation problem. Benefiting from the deep learning based audio-visual feature-embedding and prediction modules, our pipeline estimates the 3D sound-source locations and further use such locations to encode to the B-format. To benchmark our dataset and pipeline, we additionally propose evaluation criteria to investigate the performance using different $360^\circ$ input representations. Our results demonstrate the efficacy of the proposed pipeline and open up a new area of research in $360^\circ$ audio-visual analysis for future investigations.
... The acquired volumetric video can easily be used within various AR and VR applications using the head-mounted displays (HMD). Thus, within both industry and the scientific community, the interest to the effective acquisition and display of volumetric video is increasing [3][4][5]. ...
Volumetric video is becoming easier to capture and display with the recent technical developments in the acquisition, and display technologies. Using point clouds is a popular way to represent volumetric video for augmented or virtual reality applications. This representation, however, requires a large number of points to achieve a high quality of experience and needs compression before storage and transmission. In this paper, we study the subjective and objective quality assessment results for volumetric video compression, using a state-of-the-art compression algorithm: MPEG Point Cloud Compression Test Model Category 2 (TMC2). We conduct subjective experiments to find the perceptual impacts on compressed volumetric video with different quantization parameters and point counts. Additionally, we find the relationship between the state-of-the-art objective quality metrics and the acquired subjective quality assessment results. To the best of our knowledge, this study is the first to consider TMC2 compression for volumetric video represented as coloured point clouds and study its effects on the perceived quality. The results show that the effect of input point counts for TMC2 compression is not meaningful, and some geometry distortion metrics disagree with the perceived quality. The developed database is publicly available to promote the study of volumetric video compression.
This paper draws upon the primary research of an interdepartmental collaborative practice-as-research project that took place at Trinity College during 2017, in which a Samuel Beckett play, entitled Play, was reinterpreted for virtual reality. It included contributions from the Departments of Computer Science, Drama and Electrical and Electronic Engineering. The goal of this article is to offer some expanded philosophical and aesthetic reflections on the practice, now that the major production processes are completed. The primary themes that are dealt with in this paper are the reorganised rules concerning: (1) making work in the VR medium and (2) the impact of the research on viewership and content engagement in digital culture. In doing so we draw on the technological philosophy of Bernard Stiegler, who extends the legacy of Gilles Deleuze and Gilbert Simondon, to reflect on the psychic, sociopolitical and economic impacts of VR technology on cognition, subjectivity and identity in the contemporary digitalised world.
Building on a poster presentation at Siggraph 2018, this article describes an investigation of interactive narrative in virtual reality (VR) through Samuel Beckett’s theatrical text Play. Actors are captured in a green screen environment using free-viewpoint video (FVV). Built in a game engine, the scene is complete with binaural spatial audio and six degrees of freedom of movement. The project explores how ludic qualities in the original text elicit the conversational and interactive specificities of the digital medium. The work affirms potential for interactive narrative in VR, opens new experiences of the text, and highlights the reorganisation of the author–audience dynamic.
This contribution to the Samuel Beckett Critical Companions series (from Methuen Drama), designed for students, teachers, scholars, and practitioners, explores the heritage of as well as the contemporary challenges of directing Beckett.
We present a scalable pipeline for Free-Viewpoint Video (FVV) content creation, considering also visualisation in Augmented Reality (AR) and Virtual Reality (VR). We support a range of scenarios where there may be a limited number of handheld consumer cameras, but also demonstrate how our method can be applied in professional multi-camera setups. Our novel pipeline extends many state-of-the-art techniques (such as structure-from-motion, shape-from-silhouette and multi-view stereo) and incorporates bio-mechanical constraints through 3D skeletal information as well as efficient camera pose estimation algorithms. We introduce multi-source shape-from-silhouette (MS-SfS) combined with fusion of different geometry data as crucial components for accurate reconstruction in sparse camera settings. Our approach is highly flexible and our results indicate suitability either for affordable content creation for VR/AR or for interactive FVV visualisation where a user can choose an arbitrary viewpoint or sweep between known views using view synthesis.
Networked and programmable media are part of a rapidly developing me diascape transforming how citizens of developed countries do business, conduct their social lives, communicate with one another, and?perhaps most significant?think. This essay explores the hypothesis that we are in the midst of a generational shift in cognitive styles that poses challenges to education at all levels, including colleges and universities. The younger the age group, the more pronounced the shift; it is already apparent in present-day college students, but its full effects are likely to be realized only when youngsters who are now twelve years old reach our institutions of higher education. To prepare, we need to become aware of the shift, understand its causes, and think creatively and innovatively about new educational strategies appropriate to the coming changes. The shift in cognitive styles can be seen in the contrast between deep attention and hyper attention. Deep attention, the cognitive style tradi tionally associated with the humanities, is characterized by concentrating on a single object for long periods (say, a novel by Dickens), ignoring out side stimuli while so engaged, preferring a single information stream, and having a high tolerance for long focus times. Hyper attention is character ized by switching focus rapidly among different tasks, preferring multiple information streams, seeking a high level of stimulation, and having a low tolerance for boredom. The contrast in the two cognitive modes may be captured in an image: picture a college sophomore, deep in Pride and
The complete dramatic works of Samuel Beckett
- Samuel Beckett
Samuel Beckett. 2012. The complete dramatic works of Samuel Beckett. Faber & Faber.
Virtual Play in Free-Viewpoint Video: Reinterpreting Samuel Beckett for Virtual Reality
- O' Néill
- Nicholas Dwyer
- Enda Johnson
- Rafael Bates
- Jan Pagés
- Konstantinos Ondřej
- David Amplianitis
- Aljoša Monaghan
- Smolić
Néill O'Dwyer, Nicholas Johnson, Enda Bates, Rafael Pagés, Jan Ondřej, Konstantinos
Amplianitis, David Monaghan, and Aljoša Smolić. 2017. Virtual Play in Free-Viewpoint Video: Reinterpreting Samuel Beckett for Virtual Reality. In Mixed and
Augmented Reality (ISMAR-Adjunct), 2017 IEEE International Symposium on. IEEE,
262-267.