Alexander Raake

Technische Universität Ilmenau | TUI · Institut für Medientechnik

In this paper, we investigated the influence of video resolution and perceived conversational quality on nonverbal behaviors during dyadic videoconferencing (VC) conversations. We analyzed nonverbal behaviors at the individual and interpersonal level. At the individual level, we considered body motion, facial expressions, and gaze directivity. At t...

Demand for streaming services, including satellite, continues to exhibit unprecedented growth. Internet Service Providers find themselves at the crossroads of technological advancements and rising customer expectations. To stay relevant and competitive, these ISPs must ensure their networks deliver optimal video streaming quality, a key determinant...

One of the elements influencing users’ perception of video quality is the viewing distance. Although TVs have grown in size and resolution, viewers’ watching behavior and living room conditions have not changed equally and therefore viewers sit at distances farther than optimal. This gives rise to the question whether higher resolutions are percept...

In this paper, we investigated the effects of transmission delay on individual nonverbal behavior and interpersonal coordination during dyadic video conferencing conversations. For that, we assessed individual-level nonverbal behaviors, including body motion and gaze patterns, and examined participants' interpersonal coordination of body movement....

Virtual Reality (VR) has become an important tool for conducting behavioral studies in realistic, reproducible environments. In this paper, we present ISA, an Immersive Study Analyzer system designed for the comprehensive analysis of social VR studies. For in-depth analysis of participant behavior, ISA records all user actions, speech, and the cont...

Telepresence robots can enhance communication experiences by providing a sense of physical presence, embodiment and may evoke co-presence. In spite of that, telepresence robots have not made it fully to consumer markets. In this paper, we investigate how different levels of controlling a telepresence robot (teleoperation, shared control, and no con...

More and more screens with ultra high resolution and High Dynamic Range (HDR) are available on the market. Furthermore, video streaming providers like YouTube already offer contents in HDR with resolutions up to 8K (UHD-2) and a further increase in available material is conceivable. Therefore, manufacturers and providers need video quality models t...

Virtual Reality (VR) enables users to meet, converse, and collaborate in shared virtual environments. For such commu- nication systems, many system factors can affect user experience and perception. To effectively allocate system resources, under- standing of the relative influence of such factors is required. One important factor is a spatial aura...

Given the profound impact of staying socially connected on the well-being of older adults, this study explores the potential of augmented reality (AR) systems to enrich their social lives. A wearable AR communication system prototype was developed and tested in a user study involving N = 16 older adults from Germany. Participants wore an AR headset...

Audio rendering engines are a cornerstone in offering a plausible and immersive experience for interactive virtual environments (IVEs). For virtual reality IVEs, a culmination of visuals, audio, interactive, and behavioral cues blend to form a user’s perception and cognition. However, implementing such IVEs incurs additional costs and resources bey...

In recent years, there has been a growing effort to create experiments that mimic real-life situations, aiming to improve the ecological validity of research. From an acoustic point of view, one step in this direction is to use binaural reproduction to create a lifelike acoustic environment for participants. Previous work found that white noise inf...

In the project ECoClass-VR we successively increase the realism of an existing test paradigm referred to as ”audio- visual scene analysis” by Ahrens et al. Here, subjects listen to stories told by a set of talkers surrounding them. The listeners are asked to identify the content of the talkers and their specific locations, as measures of cognitive...

The appearance of virtual humans (avatars and agents) has been widely explored in immersive environments. However, virtual humans’ movements and associated sounds in real-world interactions, particularly in Augmented Reality (AR), are yet to be explored. In this paper, we investigate the influence of three distinct movement patterns (circle, side-t...

Spatial audio representations have been shown to positively impact user experience in traditional, non-immersive communication media. While spatial audio also contributes to presence in single-user immersive VR, its impact in virtual communication scenarios has not yet been fully understood. This work aims to further investigate which communication...

Little is known about the expectations of older adults (60+ years old) in robot-mediated communication when leaving aside care-related activities. To bridge this gap, we carried out 30 semi-structured interviews with older adults to explore their experiences and expectations related to technology-mediated communication. We present the results of th...

One of the elements influencing users’ perception of video quality is the viewing distance. Although TVs have grown in size and resolution, viewers’ watching behavior and living room conditions have not changed equally and therefore viewers sit at distances farther than optimal. This gives rise to the question whether higher resolutions are percept...

Ongoing research in the field of capture, coding and display technology and human vision has explored the advantages of high resolution up to 8K (UHD-2) considering perceived quality. One of the crucial elements impacting users’ perception of video quality is the viewing distance. As a result, the presented study employs a subjective evaluation to...

In this column, we reflect on the environmental impact and broader sustainability implications of resource-demanding digital applications and services such as video streaming, VR/AR/XR and videoconferencing. We put emphasis not only on the experiences and use cases they enable but also on the "cost" of always striving for high Quality of Experience...

In the past years, more and more effort has been made to bring the reality in the lab to investigate cognitive performance in a close-to real life manner and, thus, increase the validity of current research. A first step towards being ecologically valid is using a dynamic binaural reproduction, which offers an accurate representation and allows the...

The investigation of listening comprehension in auditory and visually complex classroom settings is a promising method to evaluate children's cognitive performance in a realistic setting. Many studies were able to show that children are more susceptible to noise than adults. However, it has recently been suggested that established monaural listenin...

This paper presents a proof-of-concept study conducted to analyze the effect of simple diotic vs. spatial, position-dynamic binaural synthesis on social presence in VR, in comparison with face-to-face communication in the real world, for a sample two-party scenario. A conversational task with shared visual reference was realized. The collected data...

Previous research from well before the Covid-19 pandemic had indicated little effect of delay on integral quality but a measurable one on user behavior, and a significant effect of resolution on quality but not on behavior in a two-party communication scenario. In this paper, we re-investigate the topic, after the times of the Covid-19 pandemic and...

The benefits of high resolutions in displays, such as 8K (UHD-2), have been the subject of ongoing research in the field of display technology and human perception in recent years. Out of several factors influencing users’ perception of video quality, viewing distance is one of the key aspects. Hence, this study uses a subjective test to investigat...

There is an increased interest in understanding users' behavior when exploring omnidirectional (360°) videos, especially in the presence of spatial audio. Several studies demonstrate the effect of no, mono, or spatial audio on visual saliency. However, no studies investigate the influence of higher-order (i.e., 4t h- order) Ambisonics on subjective...

In ECoClass-VR we investigate the suitability of audiovisual virtual environments to assess the influence of complex visual and acoustic scenes on cognitive performance in classroom-type scenarios. Until now, most studies have investigated rather simple acoustic and visual representations, which do not reflect the reality of school children. To imp...

This paper uses a crowdsourced dataset of online video streaming sessions to investigate opportunities to reduce the power consumption while considering QoE. For this, we base our work on prior studies which model both the end-user's QoE and the end-user device's power consumption with the help of high-level video features such as the bitrate, the...

Augmented Reality (AR) and Virtual Reality (VR) are pushing from the labs towards consumers, especially with social applications. These applications require visual representations of humans and intelligent entities. However, displaying and animating photo-realistic models comes with a high technical cost while low-fidelity representations may evoke...

In many research fields, human-annotated data plays an important role as it is used to accomplish a multitude of tasks. One such example is in the field of multimedia quality assessment where subjective annotations can be used to train or evaluate quality prediction models. Lab-based tests could be one approach to get such quality annotations. They...

Since virtual reality (VR) offers great opportunities to investigate auditory phenomena in more close to real-life scenarios, it is important to translate and extend established paradigms to consider the new technology. One major novelty in VR as opposed to classical laboratory conditions is the spatial audio setup and the added immersive visual co...

Laymen are not trained in camera control and how to use a vision mixing desk to switch between different cameras in a video production. Specific training and expert knowledge are required. In the non-professional environment, multi-camera recordings of theater performances or other stage performances are therefore difficult to realize. This can be...

AI-generated images have gained in popularity in recent years due to improvements and developments in the field of artificial intelligence. This has led to several new AI generators, which may produce realistic, funny, and impressive images using a simple text prompt. DALL-E-2, Midjourney, and Craiyon are a few examples of the mentioned approaches....

There are more and more photographic images uploaded to social media platforms such as Instagram, Flickr, or Facebook on a daily basis. At the same time, attention and consumption for such images is high, with image views and liking as one of the success factors for users and driving forces for social media algorithms. Here, "liking" can be assumed...

In a non-professional environment, multi-camera recordings of theater performances or other stage shows are difficult to realize, because amateurs are usually untrained in camera work and in using a vision mixing desk that mixes multiple cameras. This can be remedied by a production process with high-resolution cameras where recordings of image sec...

Research into multi-modal perception, human cognition, behavior, and attention can benefit from high-fidelity content that may recreate real-life-like scenes when rendered on head-mounted displays. Moreover, aspects of audiovisual perception, cognitive processes, and behavior may complement questionnaire-based Quality of Experience (QoE) evaluation...

The ability to focus ones attention in different acoustical environments has been thoroughly investigated in the past. However, recent technological advancements have made it possible to perform laboratory experiments in a more realistic manner. In order to investigate close-to-real-life scenarios, a classroom was modeled in virtual reality (VR) an...

Communication technologies play an important role in maintaining the grandparent-grandchild (GP-GC) relationship. Based on Media Richness Theory, this study investigates the frequency of use (RQ1) and perceived quality (RQ2) of established media as well as the potential use of selected innovative media (RQ3) in GP-GC relationships with a particular...

Most studies investigating the effects of environmental noise on children’s cognitive performance examine the impact of monaural noise (i.e., same signal to both ears), oversimplifying multiple aspects of binaural hearing (i.e., adequately reproducing interaural differences and spatial information). In the current study, the effects of a realistic...

In the past decade, we have witnessed an enormous growth in the demand for online video services. Recent studies estimate that nowadays, more than 1% of the global greenhouse gas emissions can be attributed to the production and use of devices performing online video tasks. As such, research on the true power consumption of devices and their energy...

Instruction at school relies heavily on oral discourse. Listening comprehension is thus of major importance for successful learning. However, in many classrooms, children’s listening is impaired by unfavourable acoustic conditions such as indoor noise and reverberation. Most studies on the effects of environmental noise on children’s speech percept...

Background: Loneliness and social isolation in older age are considered major public health concerns and research on technology-based solutions is growing rapidly. This scoping review of reviews aims to summarize the communication technologies (CTs) (review question RQ1), theoretical frameworks (RQ2), study designs (RQ3), and positive effects of t...

Telemeetings such as audiovisual conferences or virtual meetings play an increasingly important role in our professional and private lives. For that reason, system developers and service providers will strive for an optimal experience for the user, while at the same time optimizing technical and financial resources. This leads to the discipline of...

During the COVID-19 pandemic, many smaller conferences have moved entirely online and larger ones are being held as hybrid events. Even beyond the pandemic, hybrid events reduce the carbon footprint of conference travel and makes events more accessible to parts of the research community that have difficulty traveling long distances, while preservin...

An audio-only paradigm for investigating auditory selective attention has previously been transferred into a classroom-type audio-visual virtual reality (VR) environment. However, directly translating such a paradigm into VR does not promise a close-to-real life scenario, as possible benefits of the virtual environment are not utilized. In the audi...

Videoconferencing (VC) is a type of online meeting that allows two or more participants from different locations to engage in live multi-directional audio-visual communication and collaboration (e.g., via screen sharing). The COVID-19 pandemic has induced a boom in both private and professional videoconferencing in the early 2020s that elicited con...

The paper presents a conceptual, multidimensional approach to understand the technological factors that are assumed to or even have been proven to contribute to what has been coined as Zoom Fatigue (ZF) or more generally Videoconferencing Fatigue (VCF). With the advent of the Covid-19 pandemic, the usage of VC services has drastically increased, le...

The paper presents $AVQBits$ , a versatile, bitstream-based video quality model. It can be applied in several contexts such as video service monitoring, evaluation of video encoding quality, of gaming video QoE, and even of omnidirectional video quality. In the paper, it is shown that $AVQBits$ predictions closely match video quality ratings ob...

The grandparent-grandchild (GP-GC) relationship is a relevant factor for the wellbeing of both grandchildren and grandparents. Digital communication technologies play an important role in maintaining it, especially when face-to-face interactions are not possible, e.g., due to living far from each other or pandemic contact restrictions. The aim of t...

In the project ECoClass-VR, which is part of the AUDIC-TIVE priority programme, we investigate the suitability of audiovisual Immersive Virtual Environments (IVEs) for a "real-world" assessment of cognitive performance of adults and children in classroom-type environments under different visuospatial and acoustic conditions. Existing knowledge in t...

As part of the AUDICTIVE priority program, the overall objective of the ECoClass-VR project is to investigate the cognitive behavior of children in virtual, but close to real-life settings. In the first stage of this project, three validated paradigms focusing on different aspects of auditory cognition are examined and extended towards using virtua...

Recently an impressive development in immersive technologies, such as Augmented Reality (AR), Virtual Reality (VR) and 360 video, has been witnessed. However, methods for quality assessment have not been keeping up. This paper studies quality assessment of 360 video from the cross-lab tests (involving ten laboratories and more than 300 participants...

—Spatial Information (SI) and Temporal Information (TI) are frequently-used metrics to classify the spatiotemporal complexity of video content. However, they are mostly used on original video sources, and their impact on actual encoding efficiency is not known. In this paper, we propose a method to determine the compressibility of video sources, th...

Groove in music is a fundamental part of why humans entrain to it and enjoy it. Smartphones have become an important medium to listen to music. Especially when being with others, loudspeaker playback may be the method of choice. However, due to the physical limits of acoustics, for loudspeaker playback, smartphones are equipped with sub-optimal aud...