Daniel Roth

Friedrich-Alexander-University of Erlangen-Nürnberg | FAU · Department Artificial Intelligence in Biomedical Engineering & Department of Computer Science

Novel view synthesis using neural radiance fields (NeRF) is the state-of-the-art technique for generating high-quality images from novel viewpoints. Existing methods require a priori knowledge about extrinsic and intrinsic camera parameters. This limits their applicability to synthetic scenes, or real-world scenarios with the necessity of a preproc...

Estimating the 6D pose of objects is one of the major fields in 3D computer vision. Since the promising outcomes from instance-level pose estimation, the research trends are heading towards category-level pose estimation for more practical application scenarios. However, unlike well-established instance-level pose datasets, available category-level...

Alignment between real and virtual objects is a challenging task required for the deployment of Mixed Reality (MR) into manufacturing, medical, and construction applications. To face this challenge, a series of methods have been proposed. While many approaches use dynamic augmentations such as animations, arrows, or text to assist users, they requi...

p>This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible Objective : In the last two decades, there has been a growing interest in exploring surgical procedures with statistical models to analyze operations at different semantic levels. Th...

The development of embodied Virtual Reality (VR) systems involves multiple central design choices. These design choices affect the user perception and therefore require thorough consideration. This article reports on two user studies investigating the influence of common de- sign choices on relevant intermediate factors (sense of embodiment, presen...

Objective: In the last decades, there has been a growing interest in exploring surgical procedures with statistical models to analyze operations in different se?mantic levels. This information is necessary for developing context-aware intelligent systems, which can assist the physicians during operations, evaluate procedures after?ward or help the...

p>This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible Objective : In the last two decades, there has been a growing interest in exploring surgical procedures with statistical models to analyze operations at different semantic levels. Th...

Error-free surgical procedures are crucial for a patient's health. However , with the increasing complexity and variety of surgical instruments , it is difficult for clinical staff to acquire detailed assembly and usage knowledge leading to errors in process and preparation steps. Yet, the gold standard in retrieving necessary information when prob...

Persons affected by blindness or visual impairments are challenged by spatially understanding unfamiliar environments. To obtain such understanding, they have to sense their environment closely and carefully. Especially objects outside the sensing area of analog assistive devices, such as a white cane, are simply not perceived and can be the cause...

Autism - also known as Autism Spectrum Disorders or Autism Spectrum Conditions - is a neurodevelopmental condition characterized by repetitive behaviours and differences in communication and social interaction. As a consequence, many autistic individuals may struggle in everyday life, which sometimes manifests in depression, unemployment, or addict...

The visual depth perception is composed of monocular and binocular depth cues. Studies show that in absence of binocular depth cues the performance of visuomotor tasks like pointing to or grasping objects is limited. Thus, binocular depth cues are of great importance for motor control required in everyday life. However, binocular depth cues like re...

Strabismus is a visual disorder characterized by eye misalignment. The extent of ocular misalignment is denoted as the deviation angle. With the advent of Virtual Reality (VR) Head-Mounted-Displays (HMD) and eye tracking technology, new possibilities measuring strabismus arise. Major research addresses the novel field of VR strabismus assessment by...

Modulating emotional responses to virtual stimuli is a fundamental goal of many immersive interactive applications. In this study, we leverage the illusion of illusory embodiment and show that owning a virtual body provides means to modulate emotional responses. In a single-factor repeated-measures experiment, we manipulated the degree of illusory...

Current teleconsultation solutions for preclinical emergencies can transmit knowledge from a remote expert to a local paramedic using audio and 2D video channels. Such technology lacks precision and efficiency for medical diagnostic tasks, and visual feedback is often missing between participants. We investigate a mixed reality 3D teleconsultation...

A 3D Telepresence system allows users to interact with each other in a virtual, mixed, or augmented reality (VR, MR, AR) environment, creating a shared space for collaboration and communication. There are two main methods for representing users within these 3D environments. Users can be represented either as point cloud reconstruction-based avatars...

Maximizing performance success in sports is about continuous learning and adaptation processes. Aside from physiological, technical and emotional performance factors, previous research focused on perceptual skills, revealing their importance for decision-making. This includes deriving relevant environmental information as a result of eye, head and...

Estimating the depth of virtual content has proven to be a challenging task in Augmented Reality (AR) applications. Existing studies have shown that the visual system uses multiple depth cues to infer the distance of objects, occlusion being one of the most important ones. Generating appropriate occlusions becomes particularly important for AR appl...

Human motion tracking is an active field of research driven by its diverse applications in areas such as health care, daily activity recognition, sports, etc. In sports applications, tracking rowing motion is performed to meet several goals, including prevention of injuries, improvement of performance or provision of virtual coaching. Different est...

[This corrects the article DOI: 10.3389/fpsyg.2020.599703.].

Intensive care units (ICUs) that host patients infected with COVID-19 through the SARS-CoV-2 Coronavirus are typically separated from other care units in the hospital. In typical ICU ward rounds, a number of multidisciplinary experts from different medical areas and expertise levels are present to discuss the situation and care of the patient. Howe...

When users create hand-drawn annotations in Virtual Reality they often reach their physical limits in terms of precision, especially if the region to be annotated is small. One intuitive solution employs magnification beyond natural scale. However, scaling the whole environment results in wrong assumptions about the coherence between physical and v...

This paper examines the impact of implicit and explicit feedback in Virtual Reality (VR) on performance and user experience during motor rehabilitation. In this work, explicit feedback consists of visual and auditory cues provided by a virtual trainer, compared to traditional feedback provided by a real physiotherapist. Implicit feedback was genera...

The current paper addresses two methodological problems pertinent to the analysis of observer studies in non-verbal rapport and beyond. These problems concern: (1) the production of standardized stimulus materials that allow for unbiased observer ratings and (2) the objective measurement of non-verbal behaviors to identify the dyadic patterns under...

Embodying users through avatars based on motion tracking and reconstruction is an ongoing challenge for VR application developers. High quality VR systems use full-body tracking or inverse kinemat-ics to reconstruct the motion of the lower extremities and control the avatar animation. Mobile systems are limited to the motion sensing of head-mounted...

A recurrent problem in egocentric Augmented Reality (AR) applications is the misestimation of depth. Providing alternative views from non-egocentric perspectives can convey useful information for applications that require the correct judgment of depth as it is in the case of placement and alignment of virtual and real content, but also for explorat...

imals to move from one place to another without getting lost. In neurological patients this skill can be impaired, when neural structures that form the brain networks important for spatial learning and navigation are impaired. Thus, spatial navigation represents an important measure of cognitive health that is impossible to test in a clinical exami...

Autism Spectrum Disorders (ASD) are neurodevelopmental disorders that are associated with characteristic difficulties to express and interpret nonverbal behavior, such as social gaze behavior. The state of the art in diagnosis is the clinical interview that is time intensive for the clinicians and does not take into account any objective measures o...

User embodiment is important for many virtual reality (VR) applications, for example, in the context of social interaction, therapy, training, or entertainment. However, there is no data-driven and validated instrument to empirically measure the perceptual aspects of embodiment, necessary to reliably evaluate this important phenomenon. To provide a...

Virtual Reality (VR) allows us to embody any possible appearance using avatars. Previous work found that the visual appearance of an avatar can affect the user's behavior-a phenomenon known as the Proteus effect. The illusory feeling of owning a virtual avatar-the body ownership illusion-modulates the Proteus effect. Prior investigations revealed t...

Body tracking systems often are used to capture body movements of users and to realistically animate humanoid figures in virtual reality (VR). Such systems are based on different techniques, e.g. marker-based and markerless tracking, and consequently have different advantages and disadvantages. In this paper, we highlight different quality aspects...

Virtual reality and related media and communication technologies have a growing impact on professional application fields and our daily life. Virtual environments have the potential to change the way we perceive ourselves and how we interact with others. In comparison to other technologies, virtual reality allows for the convincing display of a vir...

Emotions of other people cannot be experienced directly but are often inferred from a variety of verbal and nonverbal information, including expressive body movement (EBM). Inferring emotional states is critical in social interaction, and questions remain about the factors contributing to ambiguity of EBM. In addressing this issue, researchers have...

User embodiment is important for many Virtual Reality (VR) applications, for example, in the context of social interaction, therapy, training, or entertainment. Yet, there is no commonly agreed-on or validated way to empirically measure the perception of embodiment necessary to evaluate this important quality of user experience (UX). We present a v...

Technologies for Virtual, Mixed, and Augmented Reality (VR, MR, and AR) allow to artificially augment social interactions and thus to go beyond what is possible in real life. Motivations for the use of social augmentations are manifold, for example, to synthesize behavior when sensory input is missing, to provide additional affordances in shared en...

In this demonstration we present a prototype for an avatar-mediated social interaction interface that supports the replication of head- and eye movement in distributed virtual environments. In addition to the retargeting of these natural behaviors, the system is capable of augmenting the interaction based on the visual presentation of affective sta...

Others' movements inform us about their current activities as well as their intentions and emotions. Research on the distinct mechanisms underlying action recognition and emotion inferences has been limited due to a lack of suitable comparative stimulus material. Problematic confounds can derive from low-level physical features (e.g., luminance), a...

This paper presents a framework to develop and investigate location-based Virtual Reality (VR) applications. We demonstrate our framework by introducing a novel type of VR museum, designed to support a large number of simultaneous co-located users. These visitors are walking in a hangar-scale tracking zone (600 m2), while sharing a ten times bigger...

Avatars are our digital embodied alter egos. Virtual embodiment by avatars allows social interaction with others using the full spectrum of verbal and non-verbal behaviour. Still, one's avatar appearances is elective. Hence, avatars make it possible for users to discuss and exchange sensible or even problematic personal topics potentially hiding th...

We explore motion parameters, more specifically gait parameters, as an objective indicator to assess simulator sickness in Virtual Reality (VR). We discuss the potential relationships between simulator sickness, immersion, and presence. We used two different camera pose (position and orientation) estimation methods for the evaluation of motion task...

Movements can inform us about what people are doing and also about how they feel. This phenomenologically evident distinction has been suggested to correspond functionally with differential neural correlates denoted as mirror neuron system (MNS) and mentalizing system (MENT). To separate out the roles of the underlying systems we presented identica...

Games sind die einflussreichste audiovisuelle Ausdrucks- und Erzählform digitaler Kultur. Die aktuell entstehenden staatlichen Bildungsangebote gewinnen daher künstlerisch und kulturell, aber auch wissenschaftlich und ökonomisch eine Bedeutung, wie sie beispielsweise im 20. Jahrhundert Filmstudiengängen zukam. Dieser Band gibt erstmals einen vergle...

Humans communicate to a large degree through nonverbal behavior. Nonverbal mimicry, i.e., the imitation of another's behavior can positively affect the social interactions. In virtual environments, user behavior can be replicated to avatars, and agent behaviors can be artificially constructed. By combining both, hybrid avatar-agent technologies aim...

Virtual characters are widely used in digital media and entertainment, and become more frequent in Virtual Reality (VR) applications. Multiple previous works assessed the perception of virtual characters. Yet, a comparison of the effects of non-immersive and immersive settings on character perception and affect recognition has not been presented. T...

Human gaze is a crucial element in social interactions and therefore an important topic for social Augmented, Mixed, and Virtual Reality (AR,MR,VR) applications. In this paper we systematically compare four modes of gaze transmission: (1) natural gaze, (2) hybrid gaze, which combines natural gaze transmission with a social gaze model, (3) synthesiz...

Recent improvements in virtual reality (VR) technology promise the opportunity to redesign established game genres, such as real-time strategy (RTS) games. In this work we have a look at a taxonomy of RTS games and apply it to RTS titles for VR. Hereby, we identify possible difficulties such as the need for novel means of navigation in VR. We discu...

Nonverbal behavior is the “social glue” of social interactions and mimicry as one phenomenon of nonverbal behavior in social interactions is related to positive effects such as the facilitation of liking and rapport. This extended abstract presents a social artificial intelligence that we call “Mimicry Injector” as a technological development for t...

This paper presents a novel approach for the augmentation of social behaviors in virtual reality (VR). We designed three visual transformations for behavioral phenomena crucial to everyday social interactions: eye contact, joint attention, and grouping. To evaluate the approach, we let users interact socially in a virtual museum using a large-scale...

This article reports the impact of the degree of personalization and individualization of users' avatars as well as the impact of the degree of immersion on typical psychophysical factors in embodied Virtual Environments. We investigated if and how virtual body ownership (including agency), presence, and emotional response are influenced depending...