High-Quality Illumination of Virtual Objects Based on an Environment Estimation in Mixed Reality Applications

Abstract

Visualizations of virtual objects in the real environment is often done by a simplified representation with simple surfaces and without reference to the surrounding environment. The seamless fusion of the virtual and real environment is, however, an essential factor in many areas, which is of particular importance when calculating lighting in mixed realities on mobile devices. Current approaches focus on approximations, which allow the calculation of diffuse lighting, whereby the rendering of glossy reflection properties is often neglected. The aim of this book is to enable the visualization of mirror-like reflective surfaces in mixed reality. In order to achieve this goal, various approaches are explored enabling high-quality visualization of virtual objects in realtime with a focus on the use of common hardware such as cameras, sensors in mobile devices, and partially depth sensors. Complete ambient lighting can be estimated, which enables detailed reflections. The results provide a novel way to embed complex and simple geometric shapes with glossy surfaces in the real world which offers a higher level of detail in the reflections without using additional hardware.

About the author
Tobias Schwandt´s professional and personal focus at the TU Ilmenau is the area of Mixed-Reality (MR). Within his dissertation, he particularly concerned himself with the topic of illumination of virtual content in AR, its influence on the real environment, the reconstruction of the environment light, and the manipulation of real geometry by virtual content.