Susana Castillo

Technische Universität Braunschweig · Department of Computer Science

This article investigates neural and physiological correlates of simulator sickness (SS) through a controlled experiment conducted within a fully immersive dome projection system. Our goal is to establish a reliable, objective, and in situ measurable predictive indicator of SS. SS is a problem common to all types of visual simulators consisting of...

Video-based human motion transfer creates video animations of humans following a source motion. Current methods show remarkable results for tightly-clad subjects. However, the lack of temporally consistent handling of plausible clothing dynamics, including fine and high-frequency details, significantly limits the attainable visual quality. We addre...

In this paper, we report on perceptual experiments indicating that there are distinct and quantitatively measurable differences in the way we visually perceive genuine versus face-swapped videos. Recent progress in deep learning has made face-swapping techniques a powerful tool for creative purposes, but also a means for unethical forgeries. Curren...

We present an efficient and effective computational frame-work for the inverse rendering problem of reconstructing the3D shape of a piece of glass from its caustic image. Our approach is motivated by the needs of 3D glass printing,a nascent additive manufacturing technique that promises to revolutionize the production of optics elements, from light...

Cybersickness often taints virtual experiences. Its source can be associated to the perceptual mismatch happening when our eyes tell us we are moving while we are, in fact, at rest. We reconcile the signals from our senses by using omnidirectional galvanic vestibular stimulation (GVS), stimulating the vestibular canals behind our ears with low-curr...

In this research work, we present our open-source Geant4-based Monte-Carlo simulation application, called RadField3D, for generating threedimensional radiation field datasets for dosimetry. Accompanying, we introduce a fast, machine-interpretable data format with a Python API for easy integration into neural network research, that we call RadFiled3...

3D Gaussian Splats (3DGS) have proven a versatile rendering primitive, both for inverse rendering as well as real-time exploration of scenes. In these applications, coherence across camera frames and multiple views is crucial, be it for robust convergence of a scene reconstruction or for artifact-free fly-throughs. Recent work started mitigating ar...

Dynamic reconstruction and spatiotemporal novel-view synthesis of non-rigidly deforming scenes recently gained increased attention. While existing work achieves impressive quality and performance on multi-view or teleporting camera setups, most methods fail to efficiently and faithfully recover motion and appearance from casual monocular captures....

This paper presents a novel method for accelerating the computationally intensive process of point-based holography using consumer grade hardware. By leveraging the parallel processing capabilities of graphics processing units (GPUs) and implementing optimization techniques, the proposed method significantly reduces the time required to generate co...

The reconstruction and novel view synthesis of dynamic scenes recently gained increased attention. As reconstruction from large-scale multi-view data involves immense memory and computational requirements, recent benchmark datasets provide collections of single monocular views per timestamp sampled from multiple (virtual) cameras. We refer to this...

In this paper, we propose a wavelet-based video codec specifically designed for VR displays that enables real-time playback of high-resolution 360° videos. Our codec exploits the fact that only a fraction of the full 360° video frame is visible on the display at any time. To load and decode the video viewport-dependently in real time, we make use o...

In this paper, we tackle the challenging problem of rendering real‐world 360° panorama videos that support full 6 degrees‐of‐freedom (DoF) head motion from a prerecorded omnidirectional stereo (ODS) video. In contrast to recent approaches that create novel views for individual panorama frames, we introduce a video‐specific temporally‐consistent mul...

In industrial CT, objects are often reconstructed from a circular scan trajectory. This reconstruction is possible even if only half of the object is visible in the detector at any given time, but at the cost of redundancies that can be used for the correction of miscalibrations. We propose a scheme for recalibration using only a sinogram of projec...

Ours Reference HEVC AV1 Fig. 1: In this paper, we propose a wavelet-based video codec that is able to load and decode the data of 360°videos viewport-dependently by exploiting the properties of the wavelet transform. While the quality is on par with state-of-the-art video codecs, we can achieve significantly faster playback times. Here, a visual co...

In computed tomography (CT) reconstruction, ring artifacts emerge from incorrectly normalized or defective detector elements. Correction algorithms often introduce blur or do not correctly accommodate the behaviour of those artifacts. Normalization errors stem from noise in the detector images during the normalization process and are always present...

3D reconstruction and novel view synthesis of dynamic scenes from collections of single views recently gained increased attention. Existing work shows impressive results for synthetic setups and forward-facing real-world data, but is severely limited in the training speed and angular range for generating novel views. This paper addresses these limi...

In this paper we propose a wavelet-based video codec specifically designed for VR displays that enables real-time playback of high-resolution 360{\deg} videos. Our codec exploits the fact that only a fraction of the full 360{\deg} video frame is visible on the display at any time. To load and decode the video viewport-dependently in real-time, we m...

We present a novel framework for the evaluation of eye tracking data in portrait videos including the automatic generation of customized areas of interest (AOIs) based on facial landmarks. In contrast to previous work, our framework allows the user to flexibly create AOIs by grouping the detected landmarks. Moreover, their shape and size can be mod...

To achieve better quality and generation speed, we generate holograms from adaptively sized and placed points, modeled as 2D Gaussians. We compute them with a Rayleigh-Sommerfeld convolution directly in a compact, radially symmetric look-up table.

We present a novel fast method to compute point-based Computer Generated Holograms with occlusion on the GPU, using modern hardware capabilities. Our method offers a promising new route towards real-time calculation of high-resolution holograms.

This paper deals with the highly challenging problem of reconstructing the shape of a refracting object from a single image of its resulting caustic. Due to the ubiquity of transparent refracting objects in everyday life, reconstruction of their shape entails a multitude of practical applications. The recent Shape from Caustics (SfC) method casts t...

The articles in this special section focus on real virtual reality (VR). Since the times of the Lumiere brothers, the way we watch movies has not fundamentally changed: whether in movie theaters, on mobile devices, or on TV at home, we still experience movies as outside observers, watching the action through a proscenium defined by the size of the...

We investigate the mitigation of cybersickness (CS) in 360° videos, a phenomenon caused by the visually induced impression of ego-motion while being physically at rest. We evaluate the effectiveness of scene modulations to reduce motion in the peripheral visual field by deliberately blurring or opaque occluding eccentric view areas of up to ten deg...

Cybersickness is a unpleasant phenomenon caused by the visually induced impression of ego-motion while in fact being seated. To reduce its negative impact in virtual reality experiences, we analyze the effectiveness of two techniques-peripheral blurring and field of view reduction-through an experiment in an interactive race game environment displa...

Videos obtained by current face swapping techniques can contain artifacts potentially detectable, yet unobtrusive to human observers. However, the perceptual differences between real and altered videos, as well as properties leading humans to classify a video as manipulated, are still unclear. Thus, to support the research on perceived realism and...

Current facial reenactment techniques are able to generate results with a high level of photo-realism and temporal consistency. Although the technical possibilities are rapidly progressing, recent techniques focus on achieving fast, visually plausible results. Further perceptual effects caused by altering the original facial expressivity of the rec...

We present a new pipeline to enable head-motion parallax in omnidirectional stereo (ODS) panorama video rendering using a neural depth decoder. While recent ODS panorama cameras record short-baseline horizontal stereo parallax to offer the impression of binocular depth, they do not support the necessary translational degrees-of-freedom (DoF) to als...

Current facial reenactment techniques are able to generate results with a high level of photo-realism and temporal consistency. Although the technical possibilities are rapidly progressing, recent techniques focus on achieving fast, visually plausible results. Further perceptual effects caused by altering the original facial expressivity of the rec...

The peak of virtual reality offers new exciting possibilities for the creation of media content but also poses new challenges. Some areas of interest might be overlooked because the visual content fills up a large portion of viewers' visual field. Moreover, this content is available in 360° around the viewer, yielding locations completely out of si...

This thesis focuses on establishing a - theoretically founded and empirically derived - novel methodological pipeline to provide Embodied Conversational Agents (ECAs) with natural facial expressions and desired personality traits. After giving an overview on the content of this thesis, we dedicate its second part to derive the Semantic Space (SSp)...

Creating convincing representations of humans is a fundamental problem in both traditional arts and modern media. In our digital world, virtual avatars allow us to simulate and render the human body for a variety of applications, including movie production, sports, human-computer interaction, and medical sciences. However, capturing digital represe...

It has been widely proven among several research disciplines that the information conveyed through the visual channel can be critical for communication. Altering either the motion or the appearance of our interlocutor can change the opinion we form about them or lead us to drastically different conclusions on what is being communicated. In this pap...

It has been widely proven among several research disciplines that the information conveyed through the visual channel can be critical for communication. Altering either the motion or the appearance of our interlocutor can change the opinion we form about them or lead us to drastically different conclusions on what is being communicated. In this pap...

People tend to personify machines. Giving machines the ability to actually produce social information can help improve human-machine interactions. Embodied Conversational agents (ECAs) are virtual software agents that can process and produce speech, facial expressions, gestures and eye gaze, enabling natural, multimodal, human-machine communication...

We are increasingly approaching the point where computer-based technology is truly ambient and omnipresent. People tend to personify their technical servants, including giving them human names as well as attributing personality traits and intentions to them. The more those devices advance from simple tools to intelligent assistants the more serious...

Inpainting is the process of replacing areas in an image with a perceptually plausible substitution. A common technique is to iteratively match and fill small patches at the edge of the target region making use of similar patches from the same image. Nearly all inpainting algorithms based on this approach use a single patch size for the entire imag...

The majority of virtual agents have adult bodies. There are, however, a number of reasons for using younger avatars. For example, an adult interface agent usually leads users to expect adult-level communicational and social skills. As a result, users tend to be rather intolerant when the interface agent makes obvious mistakes (e.g., incorrect gramm...

Normally, when people talk to other people, they communicate not only using specific words, but also with intentional changes in their voice melody, facial expressions, and gestures. Not only is human communication inherently multimodal, it is also multi-layered. That is, it conveys more than simple semantic information, but also passes on a wide v...

We cannot not communicate! During our daily lives, we convey information verbally and nonverbally. Most of the affective meaning of a message is transferred with the help of facial expressions, and thereby, when trying to establish a realistic human‐like virtual character, we should pay close attention to the animation. Motion capture is one of the...

We convey a tremendous amount of information vocally. In addition to the obvious exchange of semantic information, we unconsciously vary a number of acoustic properties of the speech wave to provide information about our emotions, thoughts, and intentions. [Cahn 1990] Advances in understanding of human physiology combined with increases in the comp...

Many tasks such as driving or rapidly sorting items can be best achieved by direct actions. Other tasks such as giving directions, being guided through a museum, or organizing a meeting are more easily solved verbally. Since computers are increasingly being used in all aspects of daily life, it would be of great advantage if we could communicate ve...

We can learn a lot about someone by watching their facial expressions and body language. Harnessing these aspects of non-verbal communication can lend artificial communication agents greater depth and realism but requires a sound understanding of the relationship between cognition and expressive behaviour. Here, we extend traditional word-based met...

Assessing media retargeting is not a trivial issue. When resizing one image to a particular percentage of its original size, some content has to be removed, which may affect the image's original meaning and/or composition. We examine the impact of the retargeting process on human fixations, by gathering eye-tracking data for a representative benchm...

Motion blur is a frequent requirement for the rendering of high-quality animated images. However, the computational resources involved are usually higher than those for images that have not been temporally antialiased. In this article we study the influence of high-level properties such as object material and speed, shutter time, and antialiasing l...