Conference Paper

Vanishing Importance: Studying Immersive Effects of Game Audio Perception on Player Experiences in Virtual Reality

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

Sound and virtual reality (VR) are two important output modalities for creating an immersive player experience (PX). While prior research suggests that sounds might contribute to a more immersive experience in games played on screens and mobile displays, there is not yet evidence of these effects of sound on PX in VR. To address this, we conducted a within-subjects experiment using a commercial horror-adventure game to study the effects of a VR and monitor-display version of the same game on PX. Subsequently, we explored, in a between-subjects study, the effects of audio dimensionality on PX in VR. Results indicate that audio has a more implicit influence on PX in VR because of the impact of the overall sensory experience and that audio dimensionality in VR may not be a significant factor contributing to PX. Based on our findings and observations, we provide five design guidelines for VR games.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... For example, prior work has shown most bystander-VR user interactions involve verbal communication in some capacity [33], however, how a VR user's auditory awareness of their surrounding reality can be increased has been largely ignored. Furthermore, recent work has shown that a VR user's application audio (in-VR audio) can be manipulated (e.g. by removing audio such as background music or sound effects) without influencing the user's presence in VR [35,37,38]. Therefore, we designed a user study (N=15) to explore how in-VR audio might be manipulated to facilitate a verbal bystander-VR user interactions. ...
... Therefore, the effects of audio are particularly relevant for VR, where a goal of the technology is to create as immersive an experience as possible for users. Surprisingly, however, recent empirical work has shown minimal effect of background music on user experience in VR in terms of presence [38]. While older literature (late 1990s / early 2000s) strongly suggested ambient noises, sound effects and music all increased immersion/presence in VR [4,9] studies of modern VR (2013 onward) have failed to replicate this [35,37,38]. ...
... Surprisingly, however, recent empirical work has shown minimal effect of background music on user experience in VR in terms of presence [38]. While older literature (late 1990s / early 2000s) strongly suggested ambient noises, sound effects and music all increased immersion/presence in VR [4,9] studies of modern VR (2013 onward) have failed to replicate this [35,37,38]. Instead, recent work has indicated audio may be less prominent in creating immersive VR experiences than previously thought and recent work has shown ambient noises, sound effects and background music can be removed from a VR scene without altering the user's presence [35,37,38]. ...
... Previous research suggests that audio has strong ties to inducing player emotions [21,68,70] and is capable of both enhancing and disrupting immersion [35,63,68,75]. Music listening, specifically, is capable of inducing emotions through a variety of mechanisms including brain stem reflexes, evaluative conditioning, emotional contagion, visual imagery, episodic memory, and musical expectancy [8,41,65]. ...
... A recent study by Rogers et al. [68] confirmed effects of music on players' immersion and affective state. However, a previous study exploring effects of music in virtual reality games showed that the addition of background music did not significantly affect psychometric qualities of player experience, although it did affect more qualitative measures [70] (similar results occurred for another recent study on music effects in VR [69]). ...
... Music has been described as a way to signify a continuous atmosphere of danger, and keep players tense [67]. However, music perception is also a strongly subjective experience [5,70]. There are individual differences, particularly in the enjoyment of music that evokes negative emotions-this enjoyment is further positively correlated with absorption (a similar concept to immersion), and music empathy [30]. ...
Conference Paper
Game atmosphere and game audio are critical factors linked to the commercial success of video games. However, game atmosphere has been neither operationalized nor clearly defined in games user research literature, making it difficult to study. We define game atmosphere as the emerging subjective experience of a player caused by the strong audiovisual thematic cohesion (i.e., the harmonic fit of sounds and graphics to a shared theme) of video game elements. We studied players' experience of thematic cohesion in two between-subjects, independent-measures experiments (N=109) across four conditions differing in their level of audiovisual thematic fit. Participants' experiences were assessed with physiological and psychometric measurements to understand the effect of game atmosphere on player experience. Results indicate that a lack of thematic fit between audio and visuals lowers the degree of perceived atmosphere, but that while audiovisual thematic dissonance may lead to higher-intensity negative-valence facial events, it does not impact self-reported player experience or immersion.
... The results of that literature show that increasing DF coincides with an improvement in user experience (UX), particularly presence. Yet for auditory DF in particular, recent work with modern VR has shown little impact on users in a game context [38]. In terms of IF, research has yielded evidence that there may be an uncanny valley for IF: moderate IF can negatively impact UX, but both low and high IF can reach comparably good results. ...
... In terms of display fidelity, prior research has shown that high fidelity VR display systems (e.g., graphics and audio quality) facilitate immersion and presence [6,11,33,51]. However, there are indications that the addition of ambient noises to VR game audio does not improve PX (including immersion) [38]. This suggests that bodily and sensory experiences may override effects of audio fidelity in VR; similar effects have been observed for music in the exercise context [23]. ...
... In comparison to traditional video games, players in VR can experience less control, but increased flow in VR; in general, the intensity of experiences are heightened [46]. Further, specific interactions affect players differently, particularly passive game elements such as exploration should be emphasized in VR game design [38]. In games, many other aspects beyond realism constitute łgoodž player experience, e.g., enjoyment, engagement, immersion, challenge or relaxation [13,16,25,52]. ...
Conference Paper
High degrees of interaction fidelity (IF) in virtual reality (VR) are said to improve user experience and immersion, but there is also evidence of low IF providing comparable experiences. VR games are now increasingly prevalent, yet we still do not fully understand the trade-off between realism and abstraction in this context. We conducted a lab study comparing high and low IF for object manipulation tasks in a VR game. In a second study, we investigated players' experiences of IF for whole-body movements in a VR game that allowed players to crawl underneath virtual boulders and łdanglež along monkey bars. Our findings show that high IF is preferred for object manipulation, but for whole-body movements, moderate IF can suffice, as there is a trade-off with usability and social factors. We provide guidelines for the development of VR games based on our results.
... It has been shown to affect presence in VR [11,44]. Yet in modern VR games, audio beyond what is necessary for user feedback (i.e., ambient noises and BGM, as opposed to feedback-based SFX) is perceived less prominently, possibly due to the greater impact of the sensory experience as a whole, as well as novelty bias [72]. Thus, for mixed-reality exergames, audio may be particularly important in masking perceived exertion, but its effects might also be overpowered by the sensory wholebody experience. ...
... However, we must also point out that the non-adaptive ExerCube was otherwise the same as the adaptive version in terms of visual and auditory design, and featured the same basic physicality in its interaction concept, which increases engagement and intensifies affective experiences [6]. As such, this could indicate that the system's attractiveness and effectiveness was sufficient to induce positive player experiences [72,74], and that beyond that, the importance of adaptivity (i.e., higher standards of effectiveness) becomes prominent only over longer periods of use. Nevertheless, we can report that players clearly appreciated the speed-up and slow-down balancing elements, particularly in the adaptive condition, and related to both the physical and cognitive challenges (i.e., the dual flow experience). ...
... Participants' audio perception indicates that auditory feedback signals through SFX may be more important than visual ones for a large portion of players. In terms of BGM, opinions were divided; while most thought that it was an important part of the system, this aspect was clearly less important than the SFX-a noticeable portion of participants did not consciously perceive it [72]. For those that did, however, it was important in facilitating motivation and immersion [15,44,72], and for some functioned as a signal of the system's adaptivity. ...
Conference Paper
Today's spectrum of playful fitness solutions features systems that are clearly game-first or fitness-first in design; hardly any sufficiently incorporate both areas. Consequently, existing applications and evaluations often lack in focus on attractiveness and effectiveness, which should be addressed on the levels of body, controller, and game scenario following a holistic design approach. To contribute to this topic and as a proof-of-concept, we designed the ExerCube, an adaptive fitness game setup. We evaluated participants' multi-sensory and bodily experiences with a non-adaptive and an adaptive ExerCube version and compared them with personal training to reveal insights to inform the next iteration of the ExerCube. Regarding flow, enjoyment and motivation, the ExerCube is on par with personal training. Results further reveal differences in perception of exertion, types and quality of movement, social factors, feedback, and audio experiences. Finally, we derive considerations for future research and development directions in holistic fitness game setups.
... Another user study by Zhang & Fu showed that background music increased immersion in a game, however in this study background music was conflated with sound effects [63]. Sound effects have a different role in game audio, as they are more closely tied to immediate feedback functionality [3,44,48]. ...
... A user study by Jørgensen showed that taking away music in-game decreases control, but can also facilitate concentration, depending on genre [27]. Unexpectedly, a study exploring effects of game audio in virtual reality showed that there was no difference in immersion or affective state whether background music was present or not [48]. It is difficult to compare these studies, as effects of music in games are dependent on genre, medium of display, and the individual; moreover musical attributes such as tempo, arousal, and dynamics also appear to have different effects [11,41,48]. ...
... Unexpectedly, a study exploring effects of game audio in virtual reality showed that there was no difference in immersion or affective state whether background music was present or not [48]. It is difficult to compare these studies, as effects of music in games are dependent on genre, medium of display, and the individual; moreover musical attributes such as tempo, arousal, and dynamics also appear to have different effects [11,41,48]. For instance, we have shown that audio is perceived differently depending on whether the game is played in virtual reality or on a PC [48]. ...
Conference Paper
Music affects our emotions and behaviour in real life, yet despite its prevalence in games, we have a limited understanding of its potential as a tool to explicitly influence player experience and behaviour in games. In this work, we investigate whether we can affect players' risk-taking behaviour through the presence and attributes of background music. We built a game that operationalizes risk behaviour by repeatedly giving players the choice between a safe but less rewarding course, and a risky but potentially more rewarding course. In a mixed-design user study (N=60), we explored the impact of music presence, tempo, and affective inflection on players' in-game risk behaviour and overall player experience. We found an effect of music presence on risk behaviour in the first playthrough, i.e., in the absence of other prior knowledge about the game. Further, music affect and tempo affected player immersion, as well as experienced mastery and challenge. Based on these findings, we discuss implications for game design and future research directions.
... The recently released VR headmounted display provides a new interface for playing digital games. Many traditional desktop games, especially the firstperson shooter [48,51] and horror-adventure types [49,41], have quickly been ported to VR. This has aroused new efforts to learn PX in VR but the main attention is kept on PX of traditional types of game challenge. ...
... Meuleman and Rudrauf [28] explored the potential of using several VR games to evoke multi-componential emotions whereas the emotional types were finally limited to joy and fear clusters. More recently, by asking participants to play the horror-adventure game "The Vanishing of Ethan Carter", Rogers et al. [41] found that VR group showed more emotional involvement than monitor-display group and audio dimensionality had little impact on PX in VR. Although PX in VR has aroused increasing attention, main efforts were put on investigating very limited emotional experiences induced by traditional types of challenge. ...
... Although PX in VR has aroused increasing attention, main efforts were put on investigating very limited emotional experiences induced by traditional types of challenge. One important fact is that increasing current VR games, such as Fallout 4 [52] and "The Vanishing of Ethan Carter" [41], include quite a number of emotionally challenging characteristics: narrative materials, strong characters, emotional ambiguities and etc. However, none of the aforementioned work has explicitly explored emotional challenge in VR games. ...
... 2. What is the process from raw geodata to a rendered frame using these technologies? 3. Is it possible to realistically render landscapes purely based on open GIS data without custom location-specific data, reference photos or manual additions? ...
... [2] suggests that ambient sounds can further enhance realism of landscape visualizations. Some literature suggests this could be even more important in VR, although [3] questions that that is the case. ...
... Self-created data, if necessary and feasible, is an exception. 3. The performance must be good enough to offer an immersive VR experience. ...
Thesis
Full-text available
Within the last years, there have been major developments in the fields of virtual reality, open source software, and open geospatial data. This offers new opportunities for landscape visualization which have not yet been explored. This thesis gives an overview of the data, the theory, and the technology required for developing a modern landscape visualization tool with free software and open data. Through real-time rendering, this visualization can be viewed with virtual reality devices such as the Oculus Rift. As a proof-of-concept, a prototypical landscape visualization tool is developed using the open source game engine Godot. The issue of utilizing common geospatial data within a game engine is solved with a custom native plugin, which uses the GDAL library for loading data and converting it to resources usable by the engine. Techniques for shaping and texturing the terrain, as well as populating it with vegetation and other assets such as streets and buildings, are implemented. Via screenshots and a performance evaluation, real-time rendering of freely traversable large-scale landscapes in VR with minimal preprocessing is shown to be viable using free software and open data.
... Older research points towards strong potential contributions of audio in eliciting presence and immersion in virtual environments [8,18,38,41]. Yet surprisingly, recent research has indicated that effects of audio (including music) may be less prominent in VR than with traditional monitor display [67]. This makes it important to substantiate prior work on the lack of effects of music on player experience in VR, in addition to exploring its effects on time perception in VR. ...
... The findings also constitute a replication of the surprising lack of effects of music on general PX in VR [67]. To provide a counterpoint to this phenomenon, we also briefly report on an exploratory case study (n=12) that was conducted as a pilot study overlapping with this paper's main study; here, preliminary qualitative results suggest that adaptive music can affect players' experience of pressure / tension in VR. ...
... Effects of music are then particularly relevant in VR, where obtrusiveness of the medium is considered to detract from feelings of presence (i.e., the degree to which individuals feel as if they are in the virtual world [41]). Yet recent empirical work showed no effect of music on player experience in terms of immersion or affective state in modern VR [67]. Furtherwhile older literature strongly suggested (with empirical support) that ambient noises and sound effects increase presence in virtual environments [8,16,18,38,41]-studies in modern VR have not yet been able to replicate this [65,67]. ...
Conference Paper
How much music contributes to player experience (PX) in virtual reality (VR) games remains unclear in the games user research literature. A core factor of PX in VR games that has not been studied before (in relation to audio or otherwise) is time perception. Thus, we provide the first empirical exploration of how music affects time perception in a VR game. In a user study (N=64), we investigated the effects of music on PX and time perception (operationalized as retrospective time estimation). Participants retrospectively perceived time to pass significantly quicker in the VR game when music was present, but reported no difference in PX components, including immersion. This contributes to ongoing discourse on the surprising lack of music effects in VR games. Moreover, our results highlight the need to re-conceptualize our understanding of the relationship between time perception and immersion in games.
... Sound effects are critical elements in the process of educational applications design. They can be used to give a feedback to the students about their own actions (Rogers et al., 2018). Sound effects can be used as a primary source for in-game information (Ekman, 2005), like the player-character's health status (Robb et al., 2017). ...
... Background music in learning is a sound element that was defined by Lehmann and Seufert (2017; P, 2) as "music that plays in the background while studying, i.e., when reading a text". Sound effects and background music have a positive impact on presence and engagement (Rogers et al., 2018). Sound effects and background music also affect time perception and user's performance and behavior (Cassidy & MacDonald, 2010;Ekman (2005);. ...
... Some researchers have even found that sound effects were adequate by itself to engage users. Meaning that, adding other sound's elements does not improve learning (Robb et al., 2017;Rogers et al., 2018). ...
Article
Full-text available
Students’ engagement in E-learning applications is considered an important factor for learning. There is an evidence in the literature on the influence of students’ engagement on their learning outcomes and achievement. Sound utilization in E-learning applications is expected to influence the students’ engagement in such applications. However, research is required to provide more evidences on such influence. The current study adopted the arousal theory, to check whether adding sound’s elements (voiceovers, background music and sound effects) to the E-learning applications would improve the students’ engagement in such applications. The participants are non-English speaking undergraduate students (N = 272) from a public university in Jordan. They were randomly assigned to the study groups to use one of the eight E-learning applications with different levels of sound’s elements to measure the students’ engagement in these groups. The study has employed the Analysis of Covariance, ANCOVA, to investigate the proposed hypotheses, and compare the students’ engagement among the study groups. The results showed that the sound’s elements, especially voiceovers, positively influence the students' engagement, when controlling for students’ age, gender, and their prior experience with E-learning applications. These results contribute to theory by confirming the arousal theory in the context of students' engagement in E-learning applications. Practitioners and designers can also be informed, by our results, in designing more engaging educational applications.
... Supported by embodiment and sense of presence, VR experiences can evoke relatively intense primary psychological responses given their realistic sensory motor cues [32,61,72]. Some studies have even shown evidence that VR games can induce heightened emotional responses when compared to desktop games [38,72,73,77,88]. ...
... Their experiences were observed to be quite visceral in nature, heightening the emotional intensity of their gaming experience. This supports prior research on VR PXs showing players' increased emotional state in VR [38,72,73,77,88]. ...
... They include metrics like Interaction Modality [52], Locomotion Index [7] [19], Haptic based systems include Realism, control, interface quality, ability to examine, performance, and haptic subscales [4], Speech, and language-based therapy applications used Electrodermal Activity, Heart Rate, Miss Clicks [35], Rehabilitation Game employed Hand Movement Velocity as a customized metric [43]. Heuristic and Survey based evaluations were conducted as part of studies [28], [54], [80], [65] where metrics like perceptual ration, estimated path length, recall time and immersion score were used as metrics. Andreea et al. conducted a quality evaluation of the effect of thermal visual representation on users grasping Interaction in Virtual Reality application [6]. ...
... [38], Kinect skeletal model [43], Kennedy-Lane Simulator Sickness Questionnaires [74] based empirical methods are used in the Healthcare domain. Single-Stimulus [75] and Immersive Experience Questionnaire [65] [81] [52] are followed in Assembly-simulation based VR applications. Susanne et al. conducted the effectiveness of Questionnaires in VR User Studies as a quality aspect and found that questionnaires reduces Break in Presence (BIP) and theoretical bias [59]. ...
... Researchers have also investigated the effects of game controllers (Ke and Wagner (2019)) on immersion and game experiences across different game genres (Johnson, Nacke, and Wyeth (2015)). Factors such as audio may seem important, but have actually been found to have little impact on game immersion (Rogers et al. (2018)). ...
... Another limitation is that the gameplay occurred in a laboratory setting and lasted only for 10-15 min. Although this is relatively short, similar lengths of exposure are adopted in studies on game immersion for digital games (Hicks et al. (2019); Martin-Niedecken et al. (2019); Rogers et al. (2018)). ...
Article
This study explores the effects of the perspective-taking of non-player characters (NPCs) on enhancing game immersion in prosocial virtual reality (VR) games. Prosocial games are games focusing on helping others. Game researchers have been keen to investigate factors that influence the immersive experience in digital games. Previous studies show that VR allows people to take the perspective of others, inducing empathy and prosocial behaviour in the real world. In this lab-based study, we explore whether and how taking the perspective of other game characters – NPCs in a prosocial VR game – influences players’ in-game empathy towards NPCs and game immersion. Participants first experienced either a robot’s perspective of being destroyed by fire in VR or read a text description about the same event. Then, they participated a prosocial VR game in which they saved robots. The findings show that perspective-taking experiences indirectly enhance participants’ game immersion via the effects of closeness with the destroyed robot and empathy towards the four robots protected by the player. This indirect effect is moderated by players’ weekly exposure to video games. These results suggest that VR-based perspective-taking of NPCs can indirectly enhance gameplay experiences in prosocial VR games. Theoretical and game design implications are discussed.
... • "Choro/lágrima", que antes estava codificado junto com tristeza, passou a ser compreendido como um aspecto fisiológico decorrente de aspectos emocionais diversos. Pode-se chorar de tristeza ou chorar de rir; • "Enjoo/Tontura" fazia parte dos Aspectos técnicos, assim definido a partir de Rogers et al. [31]. Esses autores relacionam imersão a possível sensação de tontura. ...
... Outro fator que pode contribuir a isso é o nível de imersão proporcionado, como Rogers et al. [31] afirma, que que o nível de imersão pode afetar a percepção em relação ao realismo das cenas, causando efeitos fisiológicos. ...
... Game sound effects, often an important source of feedback [53,91,96,147,161], affect immersion [73] and performance [32]. Additionally, the effects of audio are often contextually dependent on game genre [95], device type [164], and preferences [170]. Other studies, though, have found that audio has little effect [164]. ...
... Additionally, the effects of audio are often contextually dependent on game genre [95], device type [164], and preferences [170]. Other studies, though, have found that audio has little effect [164]. Our goal in this paper is to study self-similar avatar voices. ...
Article
Full-text available
Avatar identification is one of the most promising research areas in games user research. Greater identification with one's avatar has been associated with improved outcomes in the domains of health, entertainment, and education. However, existing studies have focused almost exclusively on the visual appearance of avatars. Yet audio is known to influence immersion/presence, performance, and physiological responses. We perform one of the first studies to date on avatar self-similar audio. We conducted a 2 x 3 (similar/dissimilar x modulation upwards/downwards/none) study in a Java programming game. We find that voice similarity leads to a significant increase in performance, time spent, similarity identification, competence, relatedness, and immersion. Similarity identification acts as a significant mediator variable between voice similarity and all measured outcomes. Our study demonstrates the importance of avatar audio and has implications for avatar design more generally across digital applications.
... Music is in games is often designed to elicit specific emotions [5,10,22], although which emotions are evoked by listening to music can be highly subjective [3]. Further, music is often perceived only subconsciously, and there are conflicting reports on its effects on player experience components, as it appears music can be largely overshadowed, e.g., by sensory overload in virtual reality games [24], or when music is perceived as unsuitable [11]. ...
Conference Paper
Game music is increasingly being explored in terms of empirical effects on players, but we know very little about how players actually perceive and use background music in games, and why. We conducted a survey (N=737) to gain an understanding of players' in-the-wild audio habits and motivations, which can inform future research and industry practices regarding game audio design. The results indicate a wide variance in players' estimation of the importance of music in games. Further, we identify and provide evidence for the comparatively new multitasking phenomenon: a substantial number of players who do not listen to games' provided background music, often in favour of additional/parallel media usage. Based on these findings, we discuss implications for game audio design, ground existing common knowledge with empirical support, and delineate future research directions.
... Initially this involved helping in and supervising the design and development of games that explored themes of realism and unrealism implicitly. Viewed from a more nuanced perspective at a later time after much reading of the literature, they then explicitly researched (un-)realism in games in several studies [139,141,178]. As such, they may have a greater interest than others in the ways that realism can be constrained to benefit PX. ...
Conference Paper
Researchers reference realism in digital games without sufficient specificity. Without clarity about the dimensions of realism, we cannot assess how and when to aim for a higher degree of realism, when lower realism suffices, or when purposeful unrealism is ideal for a game and can benefit player experience (PX). To address this conceptual gap, we conducted a systematic review using thematic synthesis to distinguish between types of realism currently found in the digital games literature. We contribute qualitative themes that showcase contradictory design goals of realism/unrealism. From these themes, we created a framework (i.e., a hierarchical taxonomy and mapping) of realism dimensions in digital games as a conceptual foundation. Our themes and framework enable a workable specificity for designing or analyzing types of realism, equip future work to explore effects of specific realism types on PX, and offer a starting point for similar efforts in non-game applications.
... Furthermore, audio cues carry the connotation of haptic and visual sensations which, if executed with a sufcient degree of congruity between actions and audio cues, helps players extend their body schema, i.e., sensory sense of self, to include virtual characters as well [7:102]. This congruency between actions and audio cues has further implications for the design of audio cues, such as sound efects, music, and ambience, in the context of its diegesis [3:55,10], however, the extent to which aspects of diegesis might afect players' sense of embodiment is not clear [21,27]. While the sense of embodiment is part of the experience of a wide variety of media experiences [27], it is a crucial component of VR experiences, since it provides a point of reference for users to experience the VRE as believable and place-like [25]. ...
Conference Paper
Full-text available
While static media makes the distinction between diegetic and non-diegetic audio, the use of these concepts in interactive media, such as games, presents problems of classification. This is especially true for virtual reality where players embody virtual characters through their own bodily movements. This PhD research focuses on the diegetic aspect of audio in virtual reality and the effect that different approaches might have on players’ sense of embodiment and willingness to suspend disbelief.
... On the audio side, Finnegan et al. made use of audio perception to compress the virtual space, in addition to conventional visual-only approaches [5]. Rogers et al. designed and conducted a user study on sound and virtual reality (VR) in games, exploring the player experience infuenced by various factors [25]. Schoop et al. proposed HindSight that can detect objects in real-time, therefore greatly enhancing the awareness and safety with notifcations [27]. ...
Conference Paper
Full-text available
As 360 deg cameras and virtual reality headsets become more popular, panorama images have become increasingly ubiquitous. While sounds are essential in delivering immersive and interactive user experiences, most panorama images, however, do not come with native audio. In this paper, we propose an automatic algorithm to augment static panorama images through realistic audio assignment. We accomplish this goal through object detection, scene classification, object depth estimation, and audio source placement. We built an audio file database composed of over $500$ audio files to facilitate this process. We designed and conducted a user study to verify the efficacy of various components in our pipeline. We run our method on a large variety of panorama images of indoor and outdoor scenes. By analyzing the statistics, we learned the relative importance of these components, which can be used in prioritizing for power-sensitive time-critical tasks like mobile augmented reality (AR) applications.
... In terms of how VRNF is constructed, a multi-disciplinary group of researchers are working to identify and develop best practices across a range of HCI relevant topics that complement our work here. Such topics include subtitle placement and management [7], how viewers transition to and from VR [24], the role of sound [34], the legibility of text in VR environments [12] and interpersonal communication within embodied VR experiences [40]. ...
Conference Paper
Virtual Reality nonfiction (VRNF) is an emerging form of immersive media experience created for consumption using panoramic "Virtual Reality" headsets. VRNF promises nonfiction content producers the potential to create new ways for audiences to experience "the real"; allowing viewers to transition from passive spectators to active participants. Our current project is exploring VRNF through a series of ethnographic and experimental studies. In order to document the content available, we embarked on an analysis of VR documentaries produced to date. In this paper, we present an analysis of a representative sample of 150 VRNF titles released between 2012-2018. We identify and quantify 64 characteristics of the medium over this period, discuss how producers are exploiting the affordances of VR, and shed light on new audience roles. Our findings provide insight into the current state of the art in VRNF and provide a digital resource for other researchers in this area.
... With the release of the first consumer VR head-mounted display in 2016 [6], user experience in VR games has aroused new discussion and attention. Rogers et al. [8] studied the effects of game audio perception on player experiences in VR by using the horror-adventure game "The Vanishing of Ethan Carter". Porter III et al. [6] investigated how the use of motion controls in VR altered user experience when playing "Minecraft". ...
Conference Paper
Digital gameplay experience depends not only on the type of challenge that the game provides, but also on how the challenge be presented. With the introduction of a novel type of emotional challenge and the increasing popularity of virtual reality (VR), there is a need to explore player experience invoked by emotional challenge in VR games. We selected two games that provides emotional challenge and conducted a 24-subject experiment to compare the impact of a VR and monitor-display version of each game on multiple player experiences. Preliminary results show that many positive emotional experiences have been enhanced significantly with VR while negative emotional experiences such as horror and fear have less been influenced; participants' perceived immersion and presence were higher when using VR than using monitor-display. Our finding of VR's expressive capability in emotional experiences may encourage more design and research with regard to emotional challenge in VR games.
... In the gaming context, immersion generates emotion and movement to the narrative and can be explored by different computer resources and interaction strategies. According to Rogers et al. [18], the level of immersion that the game offers may, moreover, affect the perception of the player in relation to scene authenticity, cause nausea or dizziness, and influence the decision of the user to continue to play the game or to quit. ...
Chapter
Digital empathy games have attracted the attention of players since they present topics associated with human frailty, such as death and mortality. Thus this type of game invites users to engage in reflection and they identify with scenes and images of the narrated world. This study presents an analysis of the perception of users of this type of game, in order to identify relevant aspects of its design and evaluation. The object studied by this article is the empathy game Valiant Hearts: The Great War, and user reviews were collected and analyzed through comments found in internet forums. These discussion boards contain user opinions ranging from technical issues and quality to matters related to mortality and death. The research is based on netnography, by means of qualitative and quantitative manual analysis of user comments. In general, the study presented multiple aspects involved in this type of game and generated considerable reflection and future studies.
... These technologies have evolved in recent years at a record pace, bringing a wide range of interests to companies and scientists, due to their flexibility to adapt to various problems in various fields (Edler et al. 2021). In video games, virtual reality is presented as broadly understood recreational games, adventure games or virtual walks containing many navigation and cartographic aspects (Smith 2017;Rogers et al. 2018;Schofield et al. 2018;Lütjens et al. 2019;Foxman et al. 2020). On the other hand, augmented reality on the gaming market can be seen in city games and board games that use many map services and the presentation of virtual-real space as a gameplay world (Alha et al. 2018;Youm et al. 2019;Thevin et al. 2021;Huei-Tse et al. 2021). ...
Article
Cartography in video games has rarely been a topic of scientific research. The authors, through a literature review of this issue, intend to draw attention to and emphasize the aspects of a rather specific and still narrow topic in cartography. They focused their literature review on publications that cover games with particular emphasis on video games with regard to cartography. Cartography has played a significant role in video games for a very long time, whether it reveals as the main user interface, mini-map, element of navigation, visual-spatial component or as orientation tool for gamers. However, in scientific research, cartography in video games is discussed in many aspects, such as cognitive cartography, performative cartography, map symbols, spatial knowledge or map design. The authors want to highlight scientific research on cartography in video games, and at the same time show the possibilities of research directions in future. The article also highlights the specific cartographic aspects of video games with techniques of Augmented Reality (AR) and Virtual Reality (VR).
... No contexto de jogos, a imersão proporciona emoção e movimento à narrativa, e pode ser explorada por diferentes recursos computacionais e estratégias de interação. De acordo com Rogers et al. [34], o nível de imersão que o jogo oferece pode afetar, ainda, a percepção do jogador em relação ao realismo das cenas, causar enjoo ou tontura e influenciar na decisão do jogador em continuar ou não a usar o jogo. ...
Conference Paper
Empathic games present, among other characteristics, the use of narratives about strong themes, which invite users to reflect and identify themselves with scenes and images from the narrated world. Among other themes, death and human frailty are recurrent. We herein analyze users' perceptions of the empathic game The Dragon Cancer in order to identify useful aspects to conceive and evaluate empathic games. Our research is based on netnography, by means of manual qualitative and quantitative analyses of users' comments in forums in Steam's page. We analyzed comments where users expressed their opinions about technical issues of the game, as well as how it addressed themes such as death and mortality. Our results are useful for the production and evaluation of empathic games.
... While we've considered standard desktop displays, it would be worthwhile to investigate other displays, such as head-mounted displays (HMDs). Work by Rogers et al. suggests that in virtual reality (VR), players focus more on the sensory experience [57]. As such, juiciness may be more impactful in VR. ...
Article
Full-text available
“Juiciness” is a term that has been widely used to describe the positive feedback (both visual/audial) present in digital games. However, few empirical investigations have looked at how juiciness concretely impacts players. In this paper, we perform a study (N = 3018) in which we compare four identical versions of an action role-playing game with varying amounts of juiciness: (1) None; (2) Medium; (3) High; and (4) Extreme. We find that both None and Extreme amounts of juiciness lead to significantly decreased play time, significantly decreased player experience, significantly decreased intrinsic motivation, and significantly decreased performance relative to both Medium and High. This is, to the best of our knowledge, the largest study to date on juiciness. Our results have implications for designers, developers, and researchers.
... Effects of VR Experience. Related work has suggested that novelty may be an initial factor in VR experiences [64,72,76], we therefore tested to see if participants' prior VR experience had an effect on their PX. For this between-participants comparison (with vs. without VR experience), we conducted Mann-Whitney U tests. ...
... However, further research is needed to understand whether these results can generalize to additional game types; the interaction effects between game type and controls complexity; and additional varied implementations of text, diagram, and tooltip instructions. For example, prior work has shown that in a VR game (as opposed to a monitor-display game) players may focus more on the sensory experience [91]. Moreover, VR games can induce a higher level of immersion and flow compared to monitor-display games [41]. ...
Article
Full-text available
Virtual reality (VR) has disrupted the gaming market and is rapidly becoming ubiquitous. Yet differences between VR and traditional mediums, such as controllers that are visible in the virtual world, enable entirely new approaches to instruction. In this paper, we present four studies, each using a different VR game. Within each study, we compared three different modalities of tutorials: Text (text-only), Text+Diagram (text with controller diagrams), and Text+Spatial (text with controller tooltips appearing on top of the player's virtual controllers). Data from our studies show that the importance of tutorial modality depends greatly on game type. In a third-person shooter, Text+Spatial led to significantly higher controls learnability than Text and Text+Diagram, and also led to significantly higher performance, player experience, and intrinsic motivation than Text. In a puzzle game, Text+Spatial led to significantly higher controls learnability and performance than Text. Additionally, Text+Diagram led to significantly higher controls learnability than Text. However, in a wave shooter and a rhythm game, differences between conditions were negligible on all measures. Our studies show that game type is an important factor to consider when designing tutorial modality.
... Finally, to achieve a fully integrated scene, additional relevant geographical and ecological information was added to the sub-panel, along with sound simulation of the changes in the sound of waves rising and falling with the sea level, which further enhanced the user's sense of immersion. The importance of sound in human-computer-interaction is often ignored but critical when you are fully immersing a user within a virtual space [13]. ...
Preprint
Full-text available
Virtual Reality (VR) technology has been shown to achieve remarkable results in multiple fields. Due to the nature of the immersive medium of Virtual Reality it logically follows that it can be used as a high-quality educational tool as it offers potentially a higher bandwidth than other mediums such as text, pictures and videos. This short paper illustrates the development of a climate change educational awareness application for virtual reality to simulate virtual scenes of local scenery and sea level rising until 2100 using prediction data. The paper also reports on the current in progress work of porting the system to Augmented Reality (AR) and future work to evaluate the system.
... This suggests that creepiness is connected to not precisely knowing the nature of the artefact. In a similar way, unconventional audio interactions [31,65] led to not knowing what to expect from a technology and thus experiencing creepiness. For many of these interactive technologies, creepiness may not necessarily be a negative property. ...
Preprint
Full-text available
Interactive technologies are getting closer to our bodies and permeate the infrastructure of our homes. While such technologies offer many benefits, they can also cause an initial feeling of unease in users. It is important for Human-Computer Interaction to manage first impressions and avoid designing technologies that appear creepy. To that end, we developed the Perceived Creepiness of Technology Scale (PCTS), which measures how creepy a technology appears to a user in an initial encounter with a new artefact. The scale was developed based on past work on creepiness and a set of ten focus groups conducted with users from diverse backgrounds. We followed a structured process of analytically developing and validating the scale. The PCTS is designed to enable designers and researchers to quickly compare interactive technologies and ensure that they do not design technologies that produce initial feelings of creepiness in users.
... This suggests that creepiness is connected to not precisely knowing the nature of the artefact. In a similar way, unconventional audio interactions [31,65] led to not knowing what to expect from a technology and thus experiencing creepiness. For many of these interactive technologies, creepiness may not necessarily be a negative property. ...
Conference Paper
Full-text available
Interactive technologies are getting closer to our bodies and permeate the infrastructure of our homes. While such technologies offer many benefits, they can also cause an initial feeling of unease in users. It is important for Human-Computer Interaction to manage first impressions and avoid designing technologies that appear creepy. To that end, we developed the Perceived Creepiness of Technology Scale (PCTS), which measures how creepy a technology appears to a user in an initial encounter with a new artefact. The scale was developed based on past work on creepiness and a set of ten focus groups conducted with users from diverse backgrounds. We followed a structured process of analytically developing and validating the scale. The PCTS is designed to enable designers and researchers to quickly compare interactive technologies and ensure that they do not design technologies that produce initial feelings of creepiness in users.
Thesis
Full-text available
Virtual reality (VR) has become an interesting tool in many research fields, and benefits have already emerged. Thus, in recent years VR is also being used in sport science and sports practice. With this, controversial results and often existing preliminary study designs make it hard to explain why differences in user’s actions still occur within VR compared to real-world (RW) conditions. One of the reasons might be the artificial presentation of the virtual environment leading to differences since VR does not fully measure up with natural conditions. Hereby, the most stimulated sense within VR is the visual one, although others can be stimulated. Accordingly, the question is whether differences in user’s visual perception occur within virtual environments and how this may affect the users’ actions. Therefore, more research should be done to address this problem and narrow down the influencing factors. In the current work, basic skills in VR are compared with those from RW to recognize more specific differences and recommend how VR can be used nowadays within the sports sector, also enabling transferable performances. Hereby, the focus is predominantly comparing the visual perception between the conditions (RW and VR). The comparisons relate to 1) the measurement quality of gaze behavior on static and dynamic visual stimuli, 2) the spatial orientation including distance estimations, route recreation, and actively walking tasks and 3) the completion of motoric tasks (balancing, grasping, and throwing) accompanied with different body visualization types to examine which body parts need to be visualized during the VR experience to fulfill adequate performances. Summarized, there occur marginally differences within the parameter collected in both conditions. For the gaze behavior, less precision is provided of the integrated Eye-Tracking system in the head-mounted display. Furthermore, the participants could not observe the dynamic stimuli in VR as accurately as in RW (p<.05). Within the spatial orientation, only in route recreation has been found an impact in VR, however, the actual task demand was equally fulfilled. Examining the body visualization, the worst performance occurred when no body was visualized during task completion. However, the remaining body visualization types did not significantly impact participants’ performance, so whole-body visualization is not essential for completing motor tasks in VR. Overall, the participants’ performances are comparable to those from RW. Although slight differences have been shown in VR in terms of longer movement execution time and increased subjective estimation of tasks’ difficulties during the motoric tasks, there are no limitations affecting participants’ performances within basic skills. Further investigations could reveal whether VR can already serve as an additional or alternative training tool in sports.
Chapter
We introduce Time Expansion Coefficient (that is, α), which refers to the ratio of the actual time to the VR image time, in this paper. VR film designers adjust time expansion coefficient according to different narrative types in VR films to optimize users’ immersive perception. We evaluated five types of narrative which include linear narrative with fixed shot, linear narrative with moving shot, circular narrative, multi-view narrative and interactive narrative. The results show that circular narrative and multi-view narrative are most affected by the time expansion coefficient. Interactive narrative and multi-view narrative are almost equally affected while linear narrative is less affected. In addition, when time expansion coefficient is greater than 1 (that is, α > 1), immersion in all five narrative types we refer to is improved.
Article
Full-text available
This review is designed to systematically examine the available evidence about virtual reality exposure therapy’s (VRET) efficacy for phobias, critically describe some of the most important challenges in the field and discuss possible directions. Evidence reveals that virtual reality (VR) is an effective treatment for phobias and useful for studying specific issues, such as pharmacological compounds and behavioral manipulations, that can enhance treatment outcomes. In addition, some variables, such as sense of presence in virtual environments, have a significant influence on outcomes, but further research is needed to better understand their role in therapeutic outcomes. We conclude that VR is a useful tool to improve exposure therapy and it can be a good option to analyze the processes and mechanisms involved in exposure therapy and the ways this strategy can be enhanced. In the coming years, there will be a significant expansion of VR in routine practice in clinical contexts.
Article
Full-text available
Background. Understanding how sound functions on informational and emotional levels within video games is critical to understanding player experience of games. User interface sounds, such as player-character health, are a pivotal component of gameplay across many video game genres, yet have not been studied in detail. Method. To address this research gap in user interface sounds, we present two studies: The first study examines the impact of the presence or absence of player-health sounds on player experience. The second study explores the impact of the types of sound used to indicate player health. We use mixed methods with qualitative and physiological measures. Results. Our results reveal that despite the presence of visual cues, sound is still important to game design for conveying health-related information and that the type of sound affects player experience.
Conference Paper
Full-text available
Persuasive games exist for a wide variety of objectives, from marketing, to healthcare and activism. Some of the more socially-aware ones cast players as members of disenfranchised minorities, such as migrants, prompting them to 'see what they see'. In parallel, a growing number of designers has recently started to leverage immersive technologies to enable the public to temporarily inhabit another person, to 'sense what they sense'. From these two converging perspectives, we hypothesize a still-uncharted space of opportunities at the crossroads of games, empathy, persuasion, and immersion. Following a Research through Design approach, we explored this space by designing A Breathtaking Journey, an embodied and multisensory mixed-reality game providing a first-person perspective of a refugee's journey. A qualitative study was conducted with a grounded theory/open coding methodology to tease out empathy-arousing characteristics, and to chart this novel game design space. As we elaborate on our analysis, we provide insights on empathic mixed-reality experiences, and conclude with offering three design opportunities: visceral engagement, reflective moments and affective appeals, to spur future research and design.
Conference Paper
Full-text available
Digital games require players to learn various skills, which is often accomplished through play itself. In multiplayer games, novices can feel overwhelmed if competing against better players, and can fail to improve, which may lead to unsatisfying play and missed social play opportunities. To help novices learn the requisite skills, we first determined how experts accomplish an important task in multiplayer FPS games -- locating their opponent. After determining that an understanding of audio cues and how to leverage them was critical, we designed and evaluated two systems for introducing this skill of locating opponents through audio cues -- a training system, and a modified game interface. We found that both systems improved accuracy and confidence, but that the training system led to more audio cues being recognized. Our work may help people of disparate skill play together, by scaffolding novices to learn and use a strategy commonly employed by experts.
Conference Paper
Full-text available
Compared to previous head-mounted displays, the compact and low-cost Oculus Rift has claimed to offer improved virtual reality experiences. However, how and what kinds of user experiences are encountered by people when using the Rift in actual gameplay has not been examined. We present an exploration of 10 participants' experiences of playing a first-person shooter game using the Rift. Despite cybersickness and a lack of control, participants experienced heightened experiences , a richer engagement with passive game elements, a higher degree of flow and a deeper immersion on the Rift than on a desktop setup. Overly demanding movements, such as the large range of head motion required to navigate the game environment were found to adversely affect gaming experiences. Based on these and other findings, we also present some insights for designing games for the Rift.
Conference Paper
Full-text available
Digital games are a wide, diverse and fast developing art form, and it is important to analyse games that are pushing the medium forward to see what design lessons can be learned. However, there are no established criteria to determine which games show these more progressive qualities. Grounded theory methodology was used to analyse language used in games reviews by critics of both 'core gamer' titles and those titles with more avant-garde properties. This showed there were two kinds of challenge being discussed --- emotional and functional which appear to be, at least partially, mutually exclusive. Reviews of 'core' and 'avant-garde' games had different measures of purchase value, primary emotions, and modalities of language used to discuss the role of audiovisual qualities. Emotional challenge, ambiguity and solitude are suggested as useful devices for eliciting emotion from the player and for use in developing more 'avant-garde' games, as well as providing a basis for further lines of inquiry.
Article
Full-text available
Learner engagement has been considered to be an important factor for successful learning in multimedia learning environments, which include digital game-based learning (DGBL). Nonetheless, there have been few empirical studies and very little consensus on what elements critically affect learners’ engagement in a DGBL environment. Visual and audial stimuli are often the only means to communicate learning materials to the learners because of the virtual environment found in DGBLs. This means that learners’ engagement in DGBL will need to occur via audial or visual stimuli – sans other specialty devices to immerse players in virtual reality (such as Occulus Rifts and Virtusphere). Yet, very few researchers have directly measured the effect of sensory stimuli on learners’ engagement during (game-based) learning. This study examined the effects of non-player characters’ (NPC) voiceovers on play-learners’ engagement in DGBL through a role-playing game (RPG). A randomized control-group post-test only design was used to collect data from 74 participants. Data analysis revealed that the participants who played the game with voiceovers were more engaged than participants who played without voiceovers. The results of this study will guide practitioners to identify more effective ways of adopting, developing, and modifying digital games for educational purposes.
Article
Full-text available
This article proposes a scale for measuring player identification in massively multiplayer online games (MMOGs). Three dimensions have been derived from the literature, avatar identification, group identification, and game identification, whereby avatar identification is a second-order factor consisting of similarity identification, wishful identification, and embodied presence. Based on the results of a cross-sectional survey of 544 World of Warcraft players, the measurement instrument's proposed factorial structure was confirmed and the constructs were successfully tested for convergent validity. Subsequently, support for nomological validity was gathered by testing nine theoretically rooted hypotheses linking the identification constructs to motivations for playing MMOGs. The results show that avatar identification is positively associated with roleplay, customization, and escapism; group identification with socializing and relationship; and game identification with escapism, discovery, advancement, and mechanics. These findings indicate that the Player Identification Scale provides a reliable measure of identification in online games ready to be used and further validated in subsequent research.
Article
Full-text available
In this article, the authors describe how they used a hybrid process of inductive and deductive thematic analysis to interpret raw data in a doctoral study on the role of performance feedback in the self-assessment of nursing practice. The methodological approach integrated data-driven codes with theory-driven ones based on the tenets of social phenomenology. The authors present a detailed exemplar of the staged process of data coding and identification of themes. This process demonstrates how analysis of the raw data from interview transcripts and organizational documents progressed toward the identification of overarching themes that captured the phenomenon of performance feedback as described by participants in the study.
Chapter
Full-text available
Presence, the “perceptual illusion of non-mediation,” is often a central goal in mediated and mixed environments, and sound is believed to be crucial for inducing high-presence experiences. This chapter provides a review of the state of the art within presence research related to auditory environments. Various sound parameters such as externalization and spaciousness and consistency within and across modalities are discussed in relation to their presence-inducing effects. Moreover, these parameters are related to the use of audio in mixed realities and example applications are discussed. Finally, we give an account of the technological possibilities and challenges within the area of presence-inducing sound rendering and presentation for mixed realities and outline future research aims. KeywordsPresence–Auditory–Auralization–Sound–Acoustics–Virtual environments–Mixed reality–Augmented reality
Conference Paper
Full-text available
In past years, computer game designers have tried to increase player engagement by improving the believability of characters and environment. Today, the focus is shifting toward improving the game controller. This study seeks to understand engagement on the basis of the body movements of the player. Initial results from two case-studies suggest that an increase in body movement imposed, or allowed, by the game controller results in an increase in the player’s engagement level. Furthermore, they lead us to hypothesize that an increased involvement of the body can afford the player a stronger affective experience. We propose that the contribution of full-body experience is three-fold: (a) it facilitates the feeling of presence in the digital environment (fantasy); (b) it enables the affective aspects of human-human interaction (communication); and (c) it unleashes the regulatory properties of emotion (affect).
Conference Paper
Full-text available
322 subjects participated in an experimental study to investigate the effects of tactile, olfactory, audio and visual sensory cues on a participant's sense of presence in a virtual environment and on their memory for the environment and the objects in that environment. Results strongly indicate that increasing the modalities of sensory input in a virtual environment can increase both the sense of presence and memory for objects in the environment. In particular, the addition of tactile, olfactory and auditory cues to a virtual environment increased the user's sense of presence and memory of the environment. Surprisingly, increasing the level of visual detail did not result in an increase in the user's sense of presence or memory of the environment.
Conference Paper
Full-text available
In this media theoretical and critical elaboration we intend to close the gap between music in static linear and non-linear interactive media. We will give a brief overview on media music perception and its historical development that enables us to recognize parallels between music in films and games and, thereby, uncover future perspectives of interactive media music. According to this, we will elaborate its narrative functions and potentials in order to widen the scope of the field where music meets interactivity.
Article
Full-text available
Game technology has been widely used for educational applications, however, despite the common use of background music in games, its effect on learning has been largely unexplored. This paper discusses how music played in the background of a computer-animated history lesson affected participants’ memory for facts. A virtual history lesson was presented to participants with different background stimuli (music or no-music) to test the effect of music on memory. To test the role of immersion on memory and its possible relationship to the music, two different display systems (3-monitor display system or immersive Reality Center) were used in the study. Overall, participants remembered a signif- icantly higher number of facts using the 3-monitor display system, particularly if no background music was played in the second half of the history lesson. Conversely, for participants using the Reality Center, significantly higher recall of facts was found when participants listened to music in the second half of the history lesson. Cognitive load/overload and (un-)familiarity with the technology are offered as explanations.
Article
Full-text available
The use of sound in an interactive media environment has not been advanced, as a technology, as far as graphics or artificial intelligence. This discussion will explore the use of sound as a way to influence the player of a computer game, will show ways that a game can use sound as input, and will describe ways that the player can influence sound in a game. The role of sound in computer games will be explored some practical design ideas that can be used to improve the current state of the art will be given.
Article
Full-text available
This paper presents an experiment testing which sound parameters, in a survival horror game context, most warrant further investigation as a means to control the level of fear in such games. The experiment is part of a long-term study ultimately designed to support the development of a biofeedback procedural audio engine for computer games. By this means, it is hoped to provide an enhanced gaming experience whereby sound synthesis and audio processing is conducted in real-time according to the player's affect responses and emotional state. Results indicate that coarse manipulation of audio parameters has the potential to influence the intensity of the player’s fear response whilst playing a survival horror game. Evidence is also presented that supports the integration of event logging and realtime participant vocal response into an experimental design to gather unbiased, quantitative data that can be associated with qualitative emotional response.
Article
Full-text available
This chapter treats computer game playing as an affective activity, largely guided by the audio-visual aesthetics of game content (of which, here, we concentrate on the role of sound) and the pleasure of gameplay. To understand the aesthetic impact of game sound on player experience, definitions of emotions are briefly discussed and framed in the game context. This leads to an introduction of empirical methods for assessing physiological and psychological effects of play, such as the affective impact of sonic playergame interaction. The psychological methodology presented is largely based on subjective interpretation of experience, while psychophysiological methodology is based on measurable bodily changes, such as context-dependent, physiological experience. As a means to illustrate both the potential and the difficulties inherent in such methodology we discuss the results of some experiments that investigate game sound and music effects and, finally, we close with a discussion of possible research directions based on a speculative assessment of the future of player-game interaction through affective sound.
Article
Full-text available
The 1970 proposition that there is an Uncanny Valley which man-made characters inhabit as their human-likeness (both appearance and movement) increases has been a growing topic of debate in the fields of robotics, animation and computer games particularly since the turn of the century. However, what the theory and subsequent related writings do not account for is the role of sound in creating perceptions of uncanniness and fear, a particularly useful attribute in computer game genres such as survival horror. This paper has a dual purpose: to explore diverse writings on the uncanny as they relate to sound and to prepare the groundwork for future work investigating the possible relationship between sound and the Uncanny Valley. The paper comprises, in large part, a survey of selected works on the uncanny and the Uncanny Valley from a variety of disciplines. It emphasizes the link between uncanniness and negative emotions, such as fear and apprehension, and discusses the genesis of the term uncanny in early psychoanalytical writings, relating this to more modern theories on human emotion. Writings on the uncanny, or related emotional states, from psychoacoustics, textiles research, films and computer games are assessed as to their validity and potential application to the fostering of an aural climate of fear in computer games and, where such writings do not explicitly deal with sound, attempts are made to apply the ideas contained within to sound as it exists within computer games. In dealing with the theory of the Uncanny Valley, the paper points out the theory‟s focus on appearance and movement to the exclusion of sound and suggests that there is an uncanny in sound that might, in future, be used to modify the Uncanny Valley theory. Throughout, there is the suggestion that the uncanny (and any future theory of an audio or audiovisual Uncanny Valley) can be harnassed to the design of horror computer games. Ultimately, it is hoped, such work will be of use to computer game sound designers who wish to create a greater perception of fear and apprehension through the canny use of uncanny sound. Some of the design tips presented at the end of the discussion are already used instinctively by sound designers across a range of media, including computer games, whereas others are less obvious in their origin and affect. Recently published empirical data is provided to strengthen the case for the latter. In some cases, the design tips must await the coming of procedural audio to computer games.
Article
Full-text available
One of the aims of modern First-Person Shooter (FPS) design is to provide an immersive experience to the player. This paper examines the role of sound in enabling such immersion and argues that even in ‘realism’ FPS games, it may be achieved sonically through a focus on caricature rather than realism. The paper utilizes and develops previous work in which a conceptual framework for the design and analysis of run and gun FPS sound is developed and the notion of the relationship between player and FPS soundscape as an acoustic ecology is put forward (Grimshaw and Schott 2007a; Grimshaw and Schott 2007b). Some problems of sound practice and sound reproduction in the game are highlighted and a conceptual solution is proposed.
Article
Full-text available
Virtual reality exposure therapy (VRET) is an increasingly common treatment for anxiety and specific phobias. Lacking is a quantitative meta-analysis that enhances understanding of the variability and clinical significance of anxiety reduction outcomes after VRET. Searches of electronic databases yielded 52 studies, and of these, 21 studies (300 subjects) met inclusion criteria. Although meta-analysis revealed large declines in anxiety symptoms following VRET, moderator analyses were limited due to inconsistent reporting in the VRET literature. This highlights the need for future research studies that report uniform and detailed information regarding presence, immersion, anxiety and/or phobia duration, and demographics.
Conference Paper
Full-text available
This paper presents a study that explores the use of mediation in virtual environments (VEs) used for cultural storytelling. A VE was created in which a traditional San story was told. Two forms of mediation were used: visual mediation and audio mediation. Four versions of the VE were implemented, differentiated by the type and amount of mediation included. 77 subjects were tested, each experiencing only one of the versions. Measurements of presence, story involvement and enjoyment of each user were taken. A factorial analysis of variance, with a between-subjects design was used. Audio mediation was found to have an effect on presence(F = 138.8, p < 0.002). Visual mediation was found to increase story involvement (F = 9.49, p < 0.003). Both the interaction between the mediations, and audio mediation increased enjoyment (F = 5.87, p < 0.02 and F = 4.01, p < 0.05 respectively). Therefore, the use of subtle mediation that appears natural in the VE setting was shown to be effective. The effects of audio mediation on presence suggests that it is an important addition to any VE. And, in the context of virtual storytelling, visual mediation is valuable in conveying a narrative more successfully.
Article
Full-text available
A between-group experiment was carried out to assess whether two different presence questionnaires can distinguish between real and virtual experiences. One group of 10 subjects searched for a box in a real office environment. A second group of 10 subjects carried out the same task in a virtual environment simulating the same office. Immediately after their experience subjects were given two different presence questionnaires in randomised order: the Witmer and Singer Presence (WS), and the questionnaire developed by Slater, Usoh and Steed (SUS). The paper argues that questionnaires should be able to pass a `reality test' - whereby under current conditions the presence scores should be higher for real experiences than for virtual ones. Nevertheless, there was only a marginally higher mean presence score for the SUS score for the real compared to the virtual, and no significant difference at all in the case of the WS mean score. It is concluded that though such questionnaires may be usef...
Conference Paper
Many people want to learn a language but find it difficult to stay engaged. Ideally, we would have language learning tools that can make language learning more enjoyable by simulating immersion in a foreign language environment. Therefore, we adapted Crystallize, a 3D video game for learning Japanese, so that it can be played in virtual reality with the Oculus Rift. Specifically, we explored whether we could leverage virtual reality technology to teach embodied cultural interaction, such as bowing in Japanese greetings. To evaluate the impact of our virtual reality game designs, we conducted a formative user study with 68 participants. We present results showing that the virtual reality design trained players how and when to bow, and that it increased participants' sense of involvement in Japanese culture. Our results suggest that virtual reality technology provides an opportunity to leverage culturally-relevant physical interaction, which can enhance the design of language learning technology and virtual reality games.
Article
This study explores players’ fright reactions and coping strategies in an immersive virtual reality (VR) horror game. Based on Slater’s theory of virtual reality, two dimensions of fear elements in the VR game−the fear of place illusion (PI) and the plausibility illusion (PSI) −were identified by playing a virtual reality survival horror game with a sample of 145 students. Participants reported greater fear toward PSI elements than toward PI elements. Fear of PSI elements positively and strongly predicted disengagement coping strategies and overall fear. Among coping strategies, players mainly adopted approach strategies, followed by avoidance (disengagement and denial), and self-help strategies. A “self-talk” strategy, newly identified in this study, has been reported as an effective means to cope with mediated threat in VR games. Regarding individual differences, sensation seeking and neuroticism influenced participants’ coping strategies and fear. Additionally, males and females employed different coping strategies. Very few students experienced next-day fright, which consists mostly of cognitive reactions and VR-related reactions, such as the Tetris effect and the fear of being attacked from the back. Theoretical frameworks regarding fear elements and coping reactions are proposed to aid future research. Implications for academia, fear conditioning for training, and marketing campaigns are discussed.
Article
The background music has been proved to be an indispensable part in every successful video game. At the same time, all the video game designers and players hope video games could be as immersive as possible. Present research aimed to make an investigation about the role of video games background music in the influence of players’ subjective immersion. Eighty participants were employed according to their game experience to join the experiment. All of them were assigned in pairs to finish a same video game synchronously with local area network. One was with earphone while the other not. After 20 minutes’ gameplay, they needed to complete three missions (questionnaire, task after game and time distortion) with the purpose of detecting their immersion during video game. The results showed that the participants with background music got significant higher scores from questionnaire, performed worse in after-game task and expressed more serious time distortion than those participants without hearing background music. But these changes just happened to low gamers in the questionnaire and time distortion. Besides, correlations were made to find that only the questionnaire scores and time distortion were significantly related. These results of present research showed that background music did increase participants’ immersion. But this improvement was likely just happened in low gamers.
Article
Sound is an integral part of our everyday lives. Sound tells us about physical events in the environment, and we use our voices to share ideas and emotions through sound. When navigating the world on a day-to-day basis, most of us use a balanced mix of stimuli from our eyes, ears and other senses to get along. We do this totally naturally and without effort. In the design of computer game experiences, traditionally, most attention has been given to vision rather than the balanced mix of stimuli from our eyes, ears and other senses most of us use to navigate the world on a day to day basis. The risk is that this emphasis neglects types of interaction with the game needed to create an immersive experience. This chapter summarizes the relationship between sound properties, GameFlow and immersive experience and discusses two projects in which Interactive Institute, Sonic Studio has balanced perceptual stimuli and game mechanics to inspire and create new game concepts that liberate users and their imagination.
Article
The concept of presence, or “being there” is a frequently emphasized factor in immersive mediated environments. It is often assumed that greater levels of immersive quality elicit higher levels of presence, in turn enhancing the effectiveness of a mediated experience. To investigate this assumption the current meta-analysis synthesizes decades of empirical research examining the effect of immersive system technology on user experiences of presence. Aggregating 115 effect sizes from 83 studies, it finds that technological immersion has a medium-sized effect on presence. Additionally, results show that increased levels of user-tracking, the use of stereoscopic visuals, and wider fields of view of visual displays are significantly more impactful than improvements to most other immersive system features, including quality of visual and auditory content. These findings are discussed in light of theoretical accounts of the presence construct as well as practical implications for design.
Article
This study presents the results of a survey of 269 undergraduate students conducted to examine fright experiences caused by video games. Over half of the participants reported game-induced fear. Sex, sensation-seeking, and empathy all emerged as important individual differences in terms of enjoyment of frightening content, consumption of frightening content, and frequency of fright experience. Interactivity and presentations of realism also predicted fear. This work identifies titles, stimuli, and features that caused fright experience.
Article
Aiming to immerse players into a new realm of drama experience, a growing number of video games utilize interactive, 'dynamic' music that reacts adaptively to game events. Though little is known about the involved perceptual processes, the design rationale of enhanced immersive experiences is taken over by public discussion including scientific accounts, despite lacking empirical validation. The present paper intends to fill this gap by hypothesizing facilitatory effects of dynamic music on attention allocation in the matching of expected and incoming expressive characteristics of concurrent stimuli. Moreover, personality constructs are investigated in mediating the decoding and sensing of experiences linked to immersion, presence, and emotion. The experiment explored experiential states of immersion, emotional valence/arousal as well as trait music empathizing and emotional involvement in the context of dynamic and non-dynamic music. 60 subjects answered self-report questionnaires each time after playing a 3rd-person action-adventure in one of three conditions accounting for (1) dynamic music, (2) non-dynamic music/low arousal potential and (3) non-dynamic music/high arousal potential, in this way aiming to manipulate structural-temporal alignment, emotional arousal and resulting congruency of nondiegetic music. Shedding light on the implications of music dramaturgy within a semantic ecology, different layers of mind sets between the player, avatar, and game environment are assumed to moderate a continuous regulatory modulation of emotional response achieved by context effects of dynamic music.
Article
Emotional responses to music are discussed in the context of a constructivist theory of emotion, which postulates evaluative cognitions that generate the quality of the experience, and autonomic (sympathetic) arousal that influences its intensity. Arousal is frequently generated by cognitive, pcrceptual, and behavioural discrepancies. Schema theory is used to describe the listener's knowledge of the complex structure of music. Schemas serve to interpret music as well as to produce expectations about its likely form and progression. When music is discrepant from the listener's expectations (as it often is), the concatenation of arousal and evaluation produces emotional experiences. This theoretical framework guides an interpretation of emotional responses to familiar and (physically) complex music, and suggests ways of understanding structural evaluations as well as effects of music that have their source outside of musical structure.
Article
In recent studies of the structure of affect, positive and negative affect have consistently emerged as two dominant and relatively independent dimensions. A number of mood scales have been created to measure these factors; however, many existing measures are inadequate, showing low reliability or poor convergent or discriminant validity. To fill the need for reliable and valid Positive Affect and Negative Affect scales that are also brief and easy to administer, we developed two 10-item mood scales that comprise the Positive and Negative Affect Schedule (PANAS). The scales are shown to be highly internally consistent, largely uncorrelated, and stable at appropriate levels over a 2-month time period. Normative data and factorial and external evidence of convergent and discriminant validity for the scales are also presented. (PsycINFO Database Record (c) 2010 APA, all rights reserved)
Article
This paper addresses the unique characteristics of emerging Virtual Reality (VR) technology and the potential of virtual worlds as learning environments. I describe several key attributes of VR environments and discuss them in relationship to educational theory and pedagogical practice. I then identify three challenges that must be met before VR can be integrated into educational settings: cost, usability, and fear of the technology.
Article
Deep engagement in video game-playing has the potential to be to be one important determinant of the impact of playing violent video games, but there are currently no reliable measures of this subjective experience. To fill this gap, the Game Engagement Questionnaire (GEQ) was developed using both classical and Rasch analyses. In Study 1 Rasch analyses provide support for the reliability and functionality of the GEQ scores. Rasch analyses also demonstrate that the GEQ has adequate separation, fit, rating scale functioning, and dimensionality, suggesting that one’s tendency to become engaged in video game-playing is a quantifiable construct. In Study 2, behavioral and questionnaire data supported the reliability and validity of the GEQ for predicting engagement in violent video games. The GEQ provides a psychometrically strong measure of levels of engagement specifically elicited while playing video games, and thus shows promise for future research examining risk and protective factors for negative game impact.
Article
The combination of psychophysiological and psychometric methods provides reliable measurements of affective user experience (UX). Understanding the nature of affective UX in interactive entertainment, especially with a focus on sonic stimuli, is an ongoing research challenge. In the empirical study reported here, participants played a fast-paced, immersive first-person shooter (FPS) game modification, in which sound (on/off) and music (on/off) were manipulated, while psychophysiological recordings of electrodermal activity (EDA) and facial muscle activity (EMG) were recorded in addition to a Game Experience Questionnaire (GEQ). Results indicate no main or interaction effects of sound or music on EMG and EDA. However, a significant main effect of sound on all GEQ dimensions (immersion, tension, competence, flow, negative affect, positive affect, and challenge) was found. In addition, an interaction effect of sound and music on GEQ dimension tension and flow indicates an important relationship of sound and music for gameplay experience. Additionally, we report the results of a correlation between GEQ dimensions and EMG/EDA activity. We conclude subjective measures could advance our understanding of sonic UX in digital games, while affective tonic (i.e., long-term psychophysiological) measures of sonic UX in digital games did not yield statistically significant results. One approach for future affective psychophysiological measures of sonic UX could be experiments investigating phasic (i.e., event-related) psychophysiological measures of sonic gameplay elements in digital games. This could improve our general understanding of sonic UX beyond affective gaming evaluation.
Article
Despite the word's common usage by gamers and reviewers alike, it is still not clear what immersion means. This paper explores immersion further by investigating whether immersion can be defined quantitatively, describing three experiments in total. The first experiment investigated participants’ abilities to switch from an immersive to a non-immersive task. The second experiment investigated whether there were changes in participants’ eye movements during an immersive task. The third experiment investigated the effect of an externally imposed pace of interaction on immersion and affective measures (state anxiety, positive affect, negative affect). Overall the findings suggest that immersion can be measured subjectively (through questionnaires) as well as objectively (task completion time, eye movements). Furthermore, immersion is not only viewed as a positive experience: negative emotions and uneasiness (i.e. anxiety) also run high.
Article
A number of emerging technologies including virtual reality, simulation rides, video conferencing, home theater, and high definition television are designed to provide media users with an illusion that a mediated experience is not mediated, a perception defined here as presence. Traditional media such as the telephone, radio, television, film, and many others offer a lesser degree of presence as well. This article examines the key concept of presence. It begins by noting practical and theoretical reasons for studying this concept. Six conceptualizations of presence found in a diverse set of literatures are identified and a detailed explication of the concept that incorporates these conceptualizations is presented. Existing research and speculation about the factors that encourage or discourage a sense of presence in media users as well as the physiological and psychological effects of presence are then outlined. Finally, suggestions concerning future systematic research about presence are presented.
Article
Cassidy, G.G. & MacDonald, R.A.R. (2010). The effects of music on time perception and performance of a driving game. Scandinavian Journal of Psychology 51, 455–464. There is an established and growing body of evidence highlighting that music can influence behavior across a range of diverse domains (Miell, MacDonald, & Hargreaves 2005). One area of interest is the monitoring of “internal timing mechanisms”, with features such as tempo, liking, perceived affective nature and everyday listening contexts implicated as important (North & Hargreaves, 2008). The current study addresses these issues by comparing the effects of self-selected and experimenter-selected music (fast and slow) on actual and perceived performance of a driving game activity. Seventy participants completed three laps of a driving game in seven sound conditions: (1) silence; (2) car sounds; (3) car sounds with self-selected music, and car sounds with experimenter-selected music; (4) high-arousal (70 bpm); (5) high-arousal (130 bpm); (6) low-arousal (70 bpm); and (7) low-arousal (130 bpm) music. Six performance measures (time, accuracy, speed, and retrospective perception of these), and four experience measures (perceived distraction, liking, appropriateness and enjoyment) were taken. Exposure to self-selected music resulted in overestimation of elapsed time and inaccuracy, while benefiting accuracy and experience. In contrast, exposure to experimenter-selected music resulted in poorest performance and experience. Increasing the tempo of experimenter-selected music resulted in faster performance and increased inaccuracy for high-arousal music, but did not impact experience. It is suggested that personal meaning and subjective associations connected to self-selected music promoted increased engagement with the activity, overriding detrimental effects attributed to unfamiliar, less liked and less appropriate experimenter-selected music.
Article
This study provides one of very few experimental investigations into the impact of a musical soundtrack on the video gaming experience. Participants were randomly assigned to one of three experimental conditions: game-with-music, game-without-music, or music-only. After playing each of three segments of The Lord of the Rings: The Two Towers (Electronic Arts, 2002)--or, in the music-only condition, listening to the musical score that accompanies the scene--subjects responded on 21 verbal scales. Results revealed that some, but not all, of the verbal scales exhibited a statistically significant difference due to the presence of a musical score. In addition, both gender and age level were shown to be significant factors for some, but not all, of the verbal scales. Details of the specific ways in which music affects the gaming experience are provided in the body of the paper.
Article
Solid evidence of virtual reality's benefits has graduated from impressive visual demonstrations to producing results in practical applications. Further, a realistic experience is no longer immersion's sole asset. Empirical studies show that various components of immersion provide other benefits - full immersion is not always necessary. The goal of immersive virtual environments (VEs) was to let the user experience a computer-generated world as if it were real - producing a sense of presence, or "being there," in the user's mind.
Article
The effectiveness of virtual environments (VEs) has often been linked to the sense of presence reported by users of those VEs. (Presence is defined as the subjective experience of being in one place or environment, even when one is physically situated in another.) We believe that presence is a normal awareness phenomenon that requires directed attention and is based in the interaction between sensory stimulation, environmental factors that encourage involvement and enable immersion, and internal tendencies to become involved. Factors believed to underlie presence were described in the premier issue of Presence: Teleoperators and Virtual Environments. We used these factors and others as the basis for a presence questionnaire (PQ) to measure presence in VEs. In addition we developed an immersive tendencies questionnaire (ITQ) to measure differences in the tendencies of individuals to experience presence. These questionnaires are being used to evaluate relationships among reported presenc...
The Vanishing of Ethan Carter Game Audio Review
  • Alyx Jones
Alyx Jones. 2015. The Vanishing of Ethan Carter Game Audio Review. The Sound Architect, http://www.thesoundarchitect.co.uk/ the-vanishing-of-ethan-carter-game-audio-review/.
Command and Control Division Information Sciences Laboratory. Defence Science & Technology Organisation
  • Judy Barrett
Judy Barrett. 2004. Side Effects of Virtual Environments: A Review of the Literature. Technical Report. Command and Control Division Information Sciences Laboratory. Defence Science and Technology Organisation, Canberra, Australia.
On the Design of an Empathy-Arousing Mixed-Reality Game
  • J L Martijn
  • Gabriele Kors
  • Erik D Ferri
  • Van Der Spek
Martijn J.L. Kors, Gabriele Ferri, Erik D. van der Spek, Cas Ketel, and Ben A.M. Schouten. 2016. A Breathtaking Journey. On the Design of an Empathy-Arousing Mixed-Reality Game. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play (CHI PLAY '16). ACM, New York, NY, USA, 91-104. DOI:http://dx.doi.org/10.1145/2967934.2968110
Sound and the diegesis in survival-horror games
  • Daniel Kromand
Daniel Kromand. 2008. Sound and the diegesis in survival-horror games. Proceedings of the Audio Mostly Conference (2008).
VR arcades need to be social to succeed
  • Daniel Cooper
Daniel Cooper. 2017. VR arcades need to be social to succeed. Engadget, https://www.engadget.com/2017/06/ 01/vr-arcades-social-htc-viveland-taiwan/. Accessed: 15th September 2017. (January 2017).
Time for new terminology? Diegetic and nondiegetic sounds in computer games revisited
  • Kristine Jørgensen
Kristine Jørgensen. 2010. Time for new terminology? Diegetic and nondiegetic sounds in computer games revisited. Grimshaw, Mark (ed.): Game Sound Technology and Player Interaction: Concepts and Developments. IGI global (2010), pp. 78-97.