Yvonne Jung’s research while affiliated with Darmstadt University of Applied Sciences and other places

We present an approach for visualizing deviations between a 3d printed object and its digital twin. The corresponding 3d visualization for instance allows to highlight particularly critical sections that indicate high deviations along with corresponding annotations. Therefore, the 3d printing thus needs to be reconstructed in 3d, again. However, since the original 3d model that served as blueprint for the 3d printer typically differs topology-wise from the 3d reconstructed model, the corresponding geometries cannot simply be compared on a per-vertex basis. Thus, to be able to easily compare two topologically different geometries, we use a multi-level voxel-based representation for both data sets. Besides using different appearance properties to show deviations, a quantitative comparison of the voxel-sets based on statistical methods is added as input for the visualization. These methods are also compared to determine the best solution in terms of the shape differences and how the results differ, when comparing either voxelized volumes or hulls. The application VoxMesh integrates these concepts into an application and provides the possibility to save the results in form of voxel-sets, meshes and point clouds persistently, that can either be used by third party software or VoxMesh to efficiently reproduce and visualize the results of the shape analysis.

This paper presents a system to determine differences between 3D reconstructed interiors and their corresponding 3D planning data with the aim of correcting identified differences and updating the 3D planning data based on these deviations. Therefore, a point-based comparison algorithm was developed with which deviations can be recognized regardless of the topology of the data used. Usually, resolution and topology of a 3D reconstruction do not match the CAD data. Here, our solution overcomes this problem by segmenting and extracting objects relevant for comparison (e.g., doors, windows) from the reconstruction and planning data separately with a subsequent analysis of the proximity of these objects to connected walls within the corresponding data set. Starting from the connection points of a segmented object to its walls, adjacent spatial data is located for a correction of detected differences to update the 3D planning data. The quality of the result of the developed process is shown in different examples localizing doors and windows to find deviations. In addition, detected differences between the planning and the measurement data are visualized and compared with the ground truth state of the building interior.

This paper presents a system to determine differences between 3D reconstructed interiors and their corresponding3D planning data with the aim of correcting identified differences and updating the 3D planning data based onthese deviations. Therefore, a point-based comparison algorithm was developed with which deviations can berecognized regardless of the topology of the data used. Usually, resolution and topology of a 3D reconstruction donot match the CAD data. Here, our solution overcomes this problem by segmenting and extracting objects relevantfor comparison (e.g., doors, windows) from the reconstruction and planning data separately with a subsequentanalysis of the proximity of these objects to connected walls within the corresponding data set. Starting from theconnection points of a segmented object to its walls, adjacent spatial data is located for a correction of detecteddifferences to update the 3D planning data. The quality of the result of the developed process is shown in differentexamples localizing doors and windows to find deviations. In addition, detected differences between the planningand the measurement data are visualized and compared with the ground truth state of the building interior.

In this paper, we present and evaluate a web service that offers cloud-based machine learning services to improve Augmented Reality applications on mobile and web clients with special regards to tracking quality and registration of complex scenes that require an application-specific coordinate frame. Specifically, our service aims at reducing camera drift that still occurs in modern AR frameworks as well as helps with the initial camera alignment in a known scene by estimating the absolute camera pose using a configurable context-based image segmentation in combination with an adaptive image classification. We demonstrate real-world applications that utilize our web service and evaluate the performance and accuracy of the underlying image segmentation and the camera pose estimation. We also discuss the initial configuration along with the semi-automatic process of generating training data, and the training of the machine learning models for the corresponding tasks.

In this paper, we show how the quality of augmentation in mobile Mixed Reality applications can be improved using a cloud-based image segmentation approach with synthetic training data. Many modern Augmented Reality frameworks are based on visual inertial odometry on mobile devices and therefore have limited access to tracking hardware (e.g., depth sensor). Consequently, tracking still suffers from drift that makes it difficult to utilize in use cases that require a higher precision. To improve tracking quality, we propose a cloud tracking approach that uses machine learning based image segmentation to recognize known objects in a real scene, which allows us to estimate a precise camera pose. Augmented Reality applications that utilize our web service can use the resulting camera pose to correct drift from time to time, while still using local tracking between key frames. Moreover, the device's position in the real world, when starting the application, is usually used as reference coordinate system. Therefore, we simplify the authoring of MR applications significantly due to a well-defined coordinate system, which is context-based and not dependend on the starting position of a user. We present all steps from web-based initialization over the generation of synthetic training data up to usage in production. In addition, we describe the underlying algorithms in detail. Finally, we show a mobile Mixed Reality application, which is based on this novel approach and discuss its advantages.