Jens Herder

Hochschule Düsseldorf University of Applied Sciences · Faculty of Media

Journal of Virtual Reality and Broadcasting ISSN: 1860-2037 Editor-in-Chief: Jens Herder Established: 2004 Language: English Peer-Review: Peer-Review Aims The “Journal of Virtual Reality and Broadcasting” is an open access E-journal covering advanced media technology for the integration of human computer interaction and modern information system...

Live video streaming is becoming increasingly popular as a form of interaction in social applications. One of its main advantages is an ability to immediately create and connect a community of remote users on the spot. In this paper we discuss how this feature can be used for crowdsourced completion of simple visual search tasks (such as finding sp...

This article describes the possibilities and problems that occur using the SteamVR tracking 2.0 system as a camera tracking system in a virtual studio and explains an approach for implementation and calibration within a professional studio environment. The tracking system allows for cost effective deployment. Relevant application fields are also mi...

Acquiring human motion data from video images plays an important role in the field of computer vision. Ground truth tracking systems require markers to create high quality motion data. But in many applications it is desired to work without markers. In recent years affordable hardware for markerless tracking systems was made available at a consumer...

We describe an application of augmented reality technology for non-destructive testing of products in the metal-industry. The prototype is created with hard- and software, that is usually employed in the gaming industry, and delivers positions for creating ultra- sonic material scans (C-scans). Using a stereo camera in combination with an hmd enabl...

In this paper we propose an integrated immersive augmented reality solution for a software tool supporting spacecraft design and verification. The spacecraft design process relies on expertise in many domains, such as thermal and structural engineering. The various subsystems of a spacecraft are highly interdependent and have differing requirements...

Der fünfzehnte Workshop "Virtuelle und Erweiterte Realität" der Fachgruppe VR/AR der Gesellschaft für Informatik e.V. wurde an der Hochschule Düsseldorf vom 10.-11.10.2018 durchgeführt. Als etablierte Plattform für den Informations- und Ideenaustausch der deutschsprachigen VR/AR-Szene bot der Workshop den idealen Rahmen, aktuelle Ergebnisse und Vo...

Mixed reality telepresence is becoming an increasingly popular form of interaction in social and collaborative applications. We are interested in how created virtual spaces can be archived, mapped, shared, and reused among different applications. Therefore, we propose a decentralized blockchain-based peer-to-peer model of distribution, with virtual...

The article discusses the question of “How to convey the experience in a virtual environment to third parties?” and explains the different technical implementations which can be used for live streaming and recording of a mixed reality experience. The real-world applications of our approach include education, entertainment, e- sports, tutorials, and...

We present a system that exploits mobile rotational tracking and photospherical imagery to allow users to share their environment with remotely connected peers “on the go.” We surveyed related interfaces and developed a unique groupware application that shares a mixed reality space with spatially-oriented live video feeds. Users can collaborate thr...

Mobile live video streaming is becoming an increasingly popular form of interaction both in social media and remote collaboration scenarios. However, in most cases the streamed video does not take mobile devices' spatial data into account (e.g., the viewers do not know the spatial orientation of a streamer), or use such data only in specific scenar...

More than three decades of ongoing research in immersive modelling has revealed many advantages of creating objects in virtual environments. Even though there are many benefits, the potential of immersive modelling has only been partly exploited due to unresolved problems such as ergonomic problems, numerous challenges with user interaction and the...

In order to improve the interactivity between users and computers, recent technologies focus on incorporating gesture recognition into interactive systems. The aim of this article is to evaluate the effectiveness of using a Myo control armband and the Kinect 2 for recognition of gestures in order to interact with virtual objects in a weather report...

Live video broadcasting requires a multitude of professional expertise to enable multi-camera productions. Robotic systems allow the automation of common and repeated tracking shots. However, predefined camera shots do not allow quick adjustments when required due to unpredictable events. We introduce a modular automated robotic camera control and...

The major challenge in virtual studio technology is the interaction between actors and virtual objects. Virtual studios differ from other virtual environments because there always exist two concurrent views: The view of the tv consumer and the view of the talent in front of the camera. This paper illustrates the interaction and feedback in front of...

The interpretation process of complex data sets makes the integration of effective interaction techniques crucial. Recent work in the field of human-computer interaction has shown that there is strong evidence that multimodal user interaction, i.e. the integration of various input modalities and interaction techniques into one comprehensive user in...

A visual and spatial feedback system for orientation in virtual sets of virtual TV studios was developed and evaluated. It is based on a green proxy object, which moves around in the acting space by way of four transparent wires. A separate unit controls four winches and is connected to an engine, which renders the virtual set. A new developed plug...

The major challenge in virtual studio technology is the interaction between the actor and virtual objects. Within a distributed live production, two locally separated markerless tracking systems where used simultaneously alongside a virtual studio. The production was based on a fully tracked actor, cyborg (half actor, half graphics), avatar, and a...

Virtual (tv) studios gain much more acceptance through improvements in computer graphics and camera tracking. Still commercial studios cannot have full interaction between actors and virtual scene because actors data are not completely digital available as well as the feedback for actors is still not sufficient. Markerless full body tracking might...

The late immersion of multi-touch sensitive displays enables the use of tangibles on multi-touch screens. There a several wide spread and/or sophisticated solutions to fulfill this need but they seem to have some flaws. One popular system at the time of writing is an overlay frame that can be placed on a normal display with the corresponding size....

Das Buch Christian Geiger, Jens Herder, Tom Vierjahn (Hg.) - Virtuelle und Erweiterte Realität 9. Workshop der GI-Fachgruppe VR/AR (ISBN: 978-3-8440-1309-2) wurde im Shaker Verlag veröffentlicht.

This paper presents an approach to integrate non-visual user feedback in today's virtual tv studio productions. Since recent studies showed that systems providing vibro-tactile feedback are not sufficient for replacing the common visual feedback, we developed an audio-based solution using an in ear headphone system, enabling a talent to move, avoid...

In this paper we introduce a system for tracking persons walking or standing on a large planar surface and for using the acquired data to easily configure position based interactions for virtual studio productions. The tracking component of the system, radarTRACK, is based on a laser scanner device capable of delivering interaction points on a larg...

This paper presents a mobile approach of integrating tangible user feedback in today's virtual TV studio productions. We describe a tangible multitouch planning system, enabling a single user to prepare and customize scene flow and settings. Users can view and interact with virtual objects by using a tangible user interface on a capacitive multitou...

In this paper we describe a prototypical system for live musical performance in a virtual studio environment. The performer stands in front of the studio camera and interacts with an infrared-laser-based multi-touch device. The final TV image shows the performer interacting with a virtual screen which is augmented in front of herself. To overcome t...

This article presents a new approach of integrating tangible user feedback in todays virtual TV studio productions. We describe a tangible multitouch planning system, enabling multiple users to prepare and customize scene flow and settings. Users can collaboratively view and interact with virtual objects by using a tangible user interface on a shar...

Shadows in computer graphics are an important rendering aspect for spatial objects. For realtime computer applications such as games, it is essential to represent shadows as accurate as possible. Also, various tv stations work with virtual studio systems instead of real studio sets. Especially for those systems, a realistic impression of the render...

For this study, an experimental vibrotactile feedback system was developed to help actors with the task of moving their arm to a certain place in a virtual tv studio under live conditions. Our intention is to improve interaction with virtual objects in a virtual set, which are usually not directly visible to the actor, but only on distant displays....

In this paper we describe the design of a virtual reality simulator for traditional intuitive archery. Traditional archers aim without a target figure. Good shooting results require an excellent body-eye coordination that allows the user to perform identical movements when drawing the bow. Our simulator provides a virtual archery experience and sup...

Actors in virtual studio productions are faced with the challenge that they have to interact with invisible virtual objects because these elements are rendered separately and combined with the real image later in the production process. Virtual sets typically use static virtual elements or animated objects with predefined behavior so that actors ca...

Two high dynamic range HDR environments maps based on video streams from fish-eye lens cameras are used for generating virtual lights in a virtual set renderer. The task of realistic virtual light setup of scenes using captured environment maps might be eased as well as visual quality improves. We discuss the light setting problem for virtual studi...

Einfach benutzbare VR-Anwendungen erfordern andere Interaktionstechniken als konventionelle Desktop-Anwendungen mit Maus, Tastatur und Desktop-Metapher zur Verfügung stellen. Da solche Ansätze in Konzeption und Realisierung deutlicher komplexer sind, müssen diese mit Sorgfalt ausgewählt werden. Folgt man der Argumentation, dass VR eine natürliche I...

Neue Formen der wissenschaftlichen Kommunikation basieren auf Fortschritten in den Informations- und Kommunikationstechnologien. Das dadurch mögliche kollaborative wissenschaftliche Arbeiten liefert Ergebnisse, die in vielfältigen Formaten, als Text, Simulationsdaten oder multimediale Elemente vorliegen. Daraus ergeben sich besondere Anforderungen...

Die interaktive Echtzeit 3D-Visualisierung "Mobilisierung und Homing von Blutstammzellen" wurde konzipiert, um ein sehr komplexes medizinisches Wissen mit den Mitteln der 3-dimensionalen Visualisierung in Echtzeit und des Internets sowie der daraus resultierenden Interaktivität aufzubereiten. Dies musste auf einer Ebene geschehen, die es hinterher...

High dynamic range (HDR) environments maps based on still images or video streams are used for computer animation or interactive systems. The task of realistic light setup of scenes using captured environment maps might be eased as well as the visual quality improves. In this article, we discuss the light setting problem for virtual studio (tv) pro...

Durch den vermehrten Einsatz von multimedialen Technologien werden in der Marktforschung die Möglichkeiten der Durchführung flexibler und kostengünstiger Studien gegeben. In sehr frühen Phasen des Innovationsprozesses als Teil der Marktforschung können durch Einsatz von Virtuellen Umgebungen die Markteinführungskonzepte für neue Produkte getestet w...

The presentation of virtual environments in real time has always been a demanding task. Specially designed graphics hardware is necessary to deal with the large amounts of data these applications typically produce. For several years the chipsets that were used allowed only simple lighting models and fixed algorithms. But recent development has prod...

With the virtual environment developed here, the characteristic sound radiation patterns of musical instruments can be experienced in real-time. The user may freely move around a musical instrument, thereby receiving acoustic and visual feedback in real-time. The perception of auditory and visual effects is intensified by the combination of acousti...

Die Visualisierung von Produkten in Echtzeit ist in vielen Bereichen ein hilfreicher Schritt, um potentiellen Kunden eine Vorstellung vom Einsatzgebiet und einen Überblick über die finale Anwendung zu erlauben. In den letzten Jahren haben neue Technologien in der Grafikkartenindustrie dazu geführt, dass früher nur auf teuren Grafikworkstations verf...

In today's market research mechanisms multi modal technologies are significant tools to perform flexible and price efficient studies for not only consumer products but also consumer goods. Current appraisal mechanisms in combination with applied computer graphics can improve the assessment of a product's launch in the very early design phase or an...

Virtual set environments for broadcasting become more sophisticated as well as the visual quality improves. Realtime interaction and production-specific visualization implemented through plugin mechanism enhance the existing systems like the 3DK. This work presents the integration of the Intersense IS-900 SCT camera tracking and 3D interaction into...

Multimediale Technologien werden in der Marktforschung immer stärker eingesetzt, um flexible und kostengünstige Studien durchzuführen. Im Innovationsprozess kann dabei auf die langjährigen Erfahrungen zurückgegriffen werden, die durch den Einsatz der Computersimulation in der technischen Produktentwicklung zustande gekommen sind. In sehr frühen Pha...

Virtual environment walkthrough applications are generally enhanced by a user's interactions within a simulated architectural space, but the enhancement that stems from changes in spatial sound that are coupled with a user's behavior are particularly important, especially within regard to creating a \"sense of place.\" When accompanied by stereosco...

Virtual set environments for broadcasting become more sophisticated as well as the visual quality improves. Realtime interaction and production-specific visualization implemented through plugin mechanism enhance the existing systems like the virtual studio software 3DK. This work presents an algorithm which can dynamically manage textures of high r...

Auditory displays with the ability to dynamically spatialize virtual sound sources under real-time conditions enable advanced applications for art and music. A listener can be deeply immersed while interacting and participating in the experience. We review some of those applications while focusing on the Helical Keyboard project and discussing the...

Virtual environments can create a realistic impression of an architectural space during the architectural design process, providing a powerful tool for evaluation and promotion during a project's early stages. In comparison to pre-rendered animations, such as walkthroughs based on \sc cad models, virtual environments can offer intuitive interaction...

Digital broadcasting enables interactive \sc tv, which presents new challenges for interactive content creation. Besides the technology for streaming and viewing, tools and systems are under development that extend traditional \sc tv studios with virtual set environments. This presentation reviews current technology and describes the requirements f...

Virtual sets have evolved from computer-generated, prerendered 2D backgrounds to realtime, responsive 3D computer graphics and are nowadays standard repertoire of broadcasting divisions. The graphics, which are combined with real video feed becoming more sophisticated, real looking and more responsive. We will look at the recent developments and su...

The audio design for virtual environments includes simulation of acoustical room properties besides specifing sound sources and sinks and their behavior. Virtual environments supporting room reverberation not only gain realism but also provide additional information to the user about surrounding space. Catching the different sound properties by the...

A chatspace was developed that allows conversation with \sc 3d sound using networked streaming in a shared virtual environment. The system provides an interface to advanced audio features, such as a "whisper function" for conveying a confided audio stream. This study explores the use of spatial audio to enhance a user's experience in multiuser virt...

The Pioneer Sound Field Controller (PSFC), a loudspeaker array system, features realtime configuration of an entire sound field, including sound source direction, virtual distance, and context of simulated environment (room characteristics: room size and liveness) for each of two sound sources. In the PSFC system, there is no native parameter to sp...

Level-of-detail is a concept well-known in computer graphics to reduce the number of rendered polygons. Depending on the distance to the subject (viewer), the objects' representation is changed. A similar concept is the clustering of sound sources for sound spatialization. Clusters can be used to hierarchically organize mixels and to optimize the u...

Sound spatialization is a technology which puts sound into the three dimensional space, so that it has a perceivable direction and distance. Interactive means mutually or reciprocally active. Interaction is when one action (e.g., user moves mouse) has direct or immediate influence to other actions (e.g., processing by a computer: graphics change in...

A chatspace was developed that allows conversation with 3D sound using networked streaming in a shared virtual environment. The system provides an interface to advanced audio features, such as a “whisper function” for conveying a confided audio stream. This study explores the use of spatial audio to enhance a user's experience in multiuser virtual...

Cyberspatial audio applications are distinguished from the broad range of spatial audio applications in a number of important ways. Most significant is that cyberspatial audio is most often designed to be responsive to user inputs. In contrast to non-interactive auditory displays, cyberspatial auditory displays typically allow active exploration of...

A module for soundscape monitoring and visualizing resource management processes was extended for presenting clusters, generated by a novel sound source clustering algorithm. This algorithm groups multiple sound sources together into a single representative source, considering localization errors depending on listener orientation. Localization erro...

Thesis (Ph. D. in Engineering)--University of Tsukuba, (B), no. 1549, 1999.7.23 "Author's publications: p. 129-132 Includes bibliographical references and index In a virtual reality environment, users are immersed in a scene with objects which might produce sound. The responsibility of a VR environment is to present these objects, but a practical s...

The cycle of abstraction-reconstruction, which occurs as a fundamental principle in the development of culture and in cognitive processes, is described and analyzed. This approach leads to recognition of boundary conditions for and directions of probable development of cognitive tools. It is shown how the transition from a conventional Japanese-Eng...

Given limited computational resources available for the rendering of spatial sound imagery, we seek to determine effective means for choosing what components of the rendering will provide the most audible differences in the results. Rather than begin with an analytic approach that attempts to predict audible differences on the basis of objective pa...

Rendering realistic spatial sound imagery for complex virtual environments must take into account the effects of obstructions such as reflectors and occluders. It is relatively well understood how to calculate the acoustical consequence that would be observed at a given observation point when an acoustically opaque object occludes a sound source. B...

The Sound Spatialization Framework is a C++ toolkit and development environment for providing advanced sound spatialization for virtual reality and multimedia applications. The Sound Spatialization Framework provides many powerful display and user-interface features not found in other sound spatialization software packages. It provides facilities t...

Broader use of virtual reality environments and sophisticated animations spawn a need for spatial sound. Until now, spatial sound design has been based very much on experience and trial and error. Most effects are hand-crafted, because good design tools for spatial sound do not exist. This paper discusses spatial sound authoring and its application...

Shared virtual environments require generalized control of user-dependent media streams. Traditional audio mixing idioms for enabling and disabling various sources employ and functions, which, along with selectively disable or focus on respective channels. Exocentric interfaces which explicitly model not only spatial audio sources, but also locatio...

Spatial sound plays an important role in virtual reality environments, allowing orientation in space, giving a feeling of space, focusing the user on events in the scene, and substituting missing feedback cues (e.g., force feedback). The sound spatialization framework of the University of Aizu, which supports number of spatialization backends, has...

The \sc psfc, or \bf Pioneer \bf Sound \bf Field \bf Controller, is a \sc dsp-driven hemispherical loudspeaker array, installed at the University of Aizu Multimedia Center. The \sc psfc features realtime manipulation of the primary components of sound spatialization for each of two audio sources located in a virtual environment, including the conte...

The cycle of abstraction-reconstruction, which occurs as a fundamental principle in the development of culture and in cognitive processes, is described and analyzed. This approach leads to recognition of boundary conditions for and directions of probable development of cognitive tools. It is shown how the transition from a conventional Japanese-Eng...

In an information-rich virtual reality (VR) environment, the user is immersed in a world containing many objects providing that information. Given the finite computational resources of any computer system, optimization is required to ensure that the most important information is presented to the user as clearly as possible and in a timely fashion....

In a virtual reality environment users are immersed in a scene with objects which might produce sound. The responsibility of a VR environment is to present these objects, but a system has only limited resources, including spatialization channels (mixels), MIDI/audio channels, and processing power. The sound spatialization resource manager controls...

Education at the University of Aizu is focussed upon computer science. Besides being the subject matter of many courses, however, the computer also plays a vital role in the educational process itself, both in the distribution of instructional media, and in providing students with valuable practical experience. All students have unlimited access (2...

Broader use of virtual reality environments and sophisticated animations spawn a need for spatial sound. Until now, spatial sound design has been based very much on experience and trial and error. Most effects are hand-crafted, because good design tools for spatial sound do not exist. This paper discusses spatial sound authoring and its application...

The PSFC, or Pioneer sound field control system, is a DSP-driven hemispherical 14-loudspeaker array, installed at the University of Aizu Multimedia Center. Collocated with a large screen rear-projection stereographic display the PSFC features realtime control of virtual room characteristics and direction of two separate sound channels, smoothly ste...

Inspired by the cyclical nature of octaves and helical structure of a scale (Shepard, '82 and '83), we prepared a model of a piano-style keyboard (prototyped in Mathematica), which was then geometrically warped into a left-handed helical configuration, one octave/revolution, pitch mapped to height. The natural orientation of upper frequency keys hi...

Dynamic characteristics of occlusion during lower jaw motion are useful in the diagnosis of jaw articulation problems and in computer-aided design/ manufacture of teeth restorations. The Functionally Generated Path (FGP), produced as a surface which envelops the actual occlusal surface of the moving opponent jaw, can be used for compact representat...

The design and fabrication of teeth restorations in dentistry rely increasingly on CAD/CAM techniques. We present an approach for interactive design of the occlusal surface of teeth based on simulation of jaw articulation and computer-aided diagnosis of occlusal disorders. To bridge the cognitive gap between the dentist and the computer system, we...

Dynamic characteristics of occlusion during lower jaw motion are useful in the diagnosis of jaw articulation problems and in computer-aided design/manufacture of teeth restorations. The Functionally Generated Path (FGP), produced as a surface which envelops the actual occlusal surface of the moving opponent jaw, can be used for compact representati...

The task of the Center for Language Research is to provide content-based English language instruction for students of computer science and engineering. As such, we find ourselves at the confluence of many of the streams currently running through the English Language Teaching profession, including English for Science and Technology (EST), English fo...

CAD/CAM techniques are increasingly used in dentistry for the design and fabrication of teeth restorations. Important concerns are the correction of articulation problems that existed before treatment and the prevention of treatment-generated problems. These require interactive evaluation of the occlusal surfaces of teeth during mastication. Tradit...

CAD/CAM techniques are used increasingly in dentistry for design and fabrication of teeth restorations. An important issue is preserving occlusal contacts of teeth after restoration. Traditional techniques based on the use of casts with mechanical articulators require manual adjustment of the occlusal surface, which becomes impractical when hard re...

The Common Lisp Interface Manager (CLIM) is used to develop graphical user interfaces for Lisp-based applications. With the prototype of the CLIM interface Builder (CLIB) the programmer can generate code for CLIM interactively. The developing process will be fast and less prone to errors. With this new tool, the interactive rapid prototyping reduce...

Der Programmier System Generator - PSG - des Fachgebiets Praktische Informatik in Darmstadt erzeugt aus einer Sprachdefinition eine sprachspezifische Programmierumgebung. Diese besteht u. a. aus einem Editor, welcher syntaktische und semantische Fehler von Programmfragmenten, die nicht vollständig sein müssen, erkennen kann. Dem Benutzer werden per...

Texturen können als Oberflächenstrukturen realer Objekte aufgefasst werden und sind Variationen in Farbe, Geometrie, Transparenz, usw.. Im Gegensatz zu Algorithmen für die künstliche Generierung von Texturen gibt es nur wenige Ansätze zu Textur-Synthese-Sprachen oder zu Hilfsmittel für die Textur-Beschreibung. Die bekannten Werkzeuge decken zudem j...

Dynamic characteristics of occlusion during lower jaw motion are useful in the diagnosis of jaw articulation problems and in computer-aided design/ manufacture of teeth restorations. The Functionally Generated Path (FGP), produced as a surface which envelops the actual occlusal surface of the moving opponent jaw, can be used for compact representat...