Leif Oppermann

Fraunhofer Institute for Applied Information Technology | FIT · Department of Cooperation Systems

In Kap. 1 haben wurde schon VR und AR als innovative Formen der Mensch-Computer-Interaktion vorgestellt. Dieses Kapitel behandelt die Gestaltung und Realisierung von Interaktionen und der daraus resultierenden Benutzungsschnittstelle (engl. User Interface) eines VR/AR-Systems detailliert. Ein Nutzer interagiert mit einer Virtuellen Welt, um virtuel...

Over the decades, technologies envisioned by pioneers such as Douglas Engelbart are becoming a reality. AI has become an important driver for technological progress, posing questions for the future of human-computer interaction. Our research group at Fraunhofer FIT looks back on a 51 year long research tradition that started with Engelbart’s vision...

Augmented/Virtual Reality (AR/VR) is still a fragmented space to design for due to the rapidly evolving hardware, the interdisciplinarity of teams, and a lack of standards and best practices. We interviewed 26 professional AR/VR designers and developers to shed light on their tasks, approaches, tools, and challenges. Based on their work and the art...

Will there only be one Metaverse in the future? Since October 2021, many virtual worlds suddenly called themselves “Metaverse”, but is that justified? What is the meaning behind the pervasive term Metaverse? How can it be approached from a pragmatic and research-oriented point of view, and maybe separated a bit from pure marketing? This paper tries...

In construction, defect management represents a crucial yet manually intensive task, demanding meticulous documentation. Our work introduces an Augmented Reality (AR) application enhancing defect management through digital documentation and real-time comparison with the construction site's digital twin. We collected the requirements for our app in...

The integration of the metaverse into the real estate and construction industry reveals various potentials, but also challenges. The increasing digitization in the architecture, engineering, construction, and operation (AECO) sector requires a critical examination of aspects such as the metaverse. This paper is dedicated to examining the impact of...

This paper outlines an ongoing three-year project aimed at enhancing remote maintenance with XR-technologies in the industrial sector, progressing from single-user to multi-user collaboration within the emerging industrial metaverse. With multiple iteration-cycles, this study addresses different sociological and technical challenges. The first year...

In this research article we introduce an Augmented Reality (AR) application specifically designed to use in supermarkets. Our primary goal is to incorporate interactive AR mini-games that bring joy and digital engagement to customers during their visit. Additionally, we aim to investigate the impact of this campaign on brand awareness for both the...

This application integrates Building Information Modeling (BIM) with interactive eXtended Reality (XR) visualizations, leveraging 5G, to enhance construction. Our campus features a 5G stand-alone network covering indoor, outdoor, and currently a construction area. 5G's speed and latency improvements facilitate collaboration using BIM data and exped...

In this chapter, we present a research project that we conducted from March 2020 to October 2021. We developed XR prototypes in a human-centred, iterative procedure, which help bridge inspectors to perform their task. Whilst we found that the AR in situ prototype offers great potential and added value to our target group, the VR prototype in the of...

Eine moderne und leistungsfähige Verkehrsinfrastruktur ist das Rückgrat für eine wettbewerbsfähige Volkswirtschaft. Der Neubau und die Instandhaltung von Infrastruktur erzeugen substanzielle Kosten. Im Straßenwesen besteht für die Erhaltung der Brückenbauwerke ein großer Finanzierungsbedarf, der im internationalen Standortwettbewerb kontinuierlich...

Bridge inspectors work for the safety of our infrastructure and mobility. In regular intervals, they conduct structural inspections – a manual task with a long-lasting and firmly normed analogue tradition. In our collaborative research project, we developed Mixed Reality (MR) and Virtual Reality (VR) prototypes to support that work digitally. We pr...

Bodies of water can be a hostile environment for both humans and technology, yet they are increasingly becoming sources, sites and media of interaction across a range of academic and practical disciplines. Despite the increasing number of interactive systems that can be used in-, on-, and underwater, there does not seem to be a coherent approach or...

Im Projekt „IndustrieStadtpark“ wird ein Koffer mit mobiler Hardware zur Realisierung von 5G-Anwendungen zusammengestellt, die Mixed-Reality und Remote-Support ermöglichen. Hierdurch werden technische Fachkräfte von Reisen und Noteinsätzen entlastet und gleichzeitig Produktionsmitarbeitende qualifiziert. In drei Iterationen werden nach dem MTO-Prin...

In Chap. 1, VR and AR have already been introduced as innovative forms of human–computer interaction. This chapter deals in detail with the design and realization of interaction and the resulting user interface of a VR/AR system. A user interacts with a virtual world to select (selection) and change (manipulation) virtual objects and to control the...

We describe an interactive Augmented and Virtual Reality (AR/VR) system that supports bridge inspectors during in-office preparation, on-site inspection, and follow-up work back in the office using Building Information Modelling (BIM) data. Originating from the process steps required to perform a digitally supported bridge inspection, we focus on o...

Building Information Modeling (BIM), a method of working with digital models of buildings throughout their life cycle, is in the process of being established in the construction industry and becoming a regulatory requirement in many countries. New wireless technologies, such as recently 5G, promise to untangle the users even further and Mixed Reali...

The emergence of high-capable standalone Mixed Reality (MR) consumer hardware and the high-speed high-bandwidth communication technologies, such as 5G, al- low to approach the existing maintenance and work processes from a novel perspective. In this poster, we introduce our 5G-enabled MR suitcase, which can be sent to remote locations and includes...

Kooperationsmöglichkeiten mit Mixed Reality. Vortrag und Artikel zum Innovationspotential für Ingenieurs- und Prüfarbeiten an Bauwerken und an der Verkehrsinfrastruktur. Zeigt die Möglichkeiten auf, die Digitalisierung in diesem Bereich nicht auf der grünen Wiese beginnen zu müssen, sondern sich auf reiche, jahrzehntelange Forschungstradition in de...

Die Digitalisierung erreicht das Bauhandwerk und betrifft nun bald den kompletten Lebenszyklus von Gebäuden, Bau-werken und der gesamten Verkehrsinfrastruktur. Neuartige Begriffe machen die Runde: vom digitalen Zwilling, über künstliche Intelligenz bis hin zu Virtual, Augmented und Mixed Reality, sowie kollaborativem Arbeiten auf Basis von BIM-Date...

Die Bauwerksprüfung bildet die Grundlage zur Erhaltung der Zukunftsfähigkeit von Ingenieurbauwerken. Es ist es von entscheidender Bedeutung, dass relevante Daten z.B. zu festgestellten Schäden vollständig, ortsgenau identifiziert sowie reproduzierbar und einheitlich bewertet werden. Im gleichen Zuge sind eine erhebliche Anzahl an Dokumenten, Bezeic...

We present how to use Mixed Reality (MR) technology to enhance bridge inspections using Building Information Modeling (BIM) process and geometry data. Originating from identifying the processes relevant for digital MR-supported bridge inspections, we describe our plan to implement this support for the preparation and debriefing in the office, and t...

This paper presents the concept, a system-overview, and the evaluation of EnArgus, the central information system for energy research funding in Germany. Initiated by the German Federal Ministry for Economic Affairs and Energy (BMWi), EnArgus establishes a one-stop information system about all recent and ongoing energy research funding projects in...

We present an Augmented Reality (AR) application intended for use in supermarkets, with the primary goal to bring fun and digital engagement through AR mini-games to the customers while shopping. We believe that our approach can be extended and scaled up by integrating mini-games into existing shopping apps in the future.

Reporting on a BASt/BMVI pilot-project on Mixed Reality Supported Bridge Inspections. Introducing the domain, our project's work to link on-site and in the office using VR/AR based on interaction and collaboration with 3D BIM-models. Presenting related work in the domains of CSCW and Mixed Reality to draw a joint vision for IT to leave the office a...

Dieses Kapitel enthält eine Sammlung von ausgewählten erfolgreichen Fallbeispielen für VR/AR aus Forschung und Praxis.

The presented demonstrator showcases the potential of augmented reality (AR) smartglasses in the sterile supply chain. In the scope of the project "Smartglasses in der Sterilgutver-sorgung", we developed an application to investigate on the applicability of smartglasses and AR paradigms during the reprocessing of used operational tools. The demonst...

In this article, we compare a Google Tango tablet with the Microsoft HoloLens smartglasses in the context of the visualisation and interaction with Building Information Modeling data. A user test was conducted where 16 participants solved four tasks, two for each device, in small teams of two. Two aspects are analysed in the user test: the visualis...

In this paper, we present the results of a measurement on the accuracy of Google Tango Development Kit tablets in comparison to ARCore under real conditions. With our results, we can show that the Motion Tracking concept accumulates errors over time and distance to up to several meters, while the Area Learning concept is able to eliminate those err...

Educational location-based games provide a link between content and its real-life relevance in a physical environment. Location-based activities for authentic learning provide multiple opportunities, but educators still per-ceive technological and organisational barriers. There is a need for easy-to-use tools to facilitate the design of playful loc...

Being able to explore vast outdoor scenery, cityscapes, or the interior of virtual buildings is the state of the art for video gamers. But city planners and building owners usually still have to settle with blueprints , rendered movies, and miniature models. We present "Auto AR", a mobile Augmented Reality system that allows the user to experience...

Building Information Modeling (BIM) and Industry 4.0 are complex application domains. Doing interdisciplinary research with architects, engineers and computer scientists at the intersection of the domains is both challenging and promising, since many relevant research problems need multi-perspective views and cooperatively designed solution strateg...

With six degrees of freedom, Google Tango devices differ from ordinary tablets or smartphones, enabling a more precise indoor localisation. We propose a design concept for mobile AR and VR applications visualising Building Information Modeling (BIM) data-stored in one central digital model-on site in an interactive manner using Project Tango develo...

This paper reports on a study of AREEF, a multi-player Underwater Augmented Reality (UWAR) experience for swimming pools. Using off-the-shelf components combined with a custom made waterproof case and an innovative game concept, AREEF puts computer game technology to use for recreational and educational purposes in and under water. After an experie...

Presenting six different visualisation ideas for exploring architectural and BIM (building information modeling) data using mobile Mobile Mixed Reality. This is the large poster made for the poster-paper "Mobile cross-media visualisations made from building information modelling data", as presented at Mobile HCI 2016 in Florence, Italy. Links to...

An easy-to-use tool is presented that supports educators to create geogames for smartphones with media. The research questions are addressed whether educators use audios to support learning, and if they do, how do they use them? First results of the actual usage indicate that the potential of audio has not been fully exploited in all cases.

The advent of Building Information Modelling (BIM) provides geometry data that can be easily used for visualisations. We present six demonstrators made from the same data using similar workflows. They cover different categories of mobile devices, ranging from head-mounted displays to smartphones and tablets with inside-out positional tracking. They...

Wir beschreiben unseren Blended Learning Ansatz zum selbständigen Erlernen der Funktionsweise einer Produktionsmaschine in der Textiltechnik mittels Augmented Reality auf Smartphones. Das System wird von Mitarbeitern und Studenten des Maschinenbaus im Technikum des Instituts für Textiltechnik der RWTH Aachen University eingesetzt. Mit dem System is...

Pervasive games are bridging the physical and digital worlds through the use of mobile devices, positioning technology, and mobile networks. They received an uptake since the Global Positioning System (GPS) became available for public use in May 2000. Since then, pervasive games have been studied in a number of research projects and have reached th...

The idea of augmented or virtual reality in combination with head mounted display is being discussed already since at least 1968. However, for a long time, this topic was discussed mainly within the academic research area with only limited effect or uptake in the work place. Primary reason for this was the missing availability of robust and afforda...

In 2020, German companies will spend close to €850 million on Virtual and Mixed Reality hardware and solutions based on smart glasses. The lion’s share will be spent on innovative applications. This text provides some guidance in this rising market by categorizing the available smart glasses and head-mounted displays into five categories and the po...

Mit EnArgus stellen wir ein interdisziplinäres Projekt von Energieforschern, Informatikern und Linguisten vor: die Erstellung eines zentralen Informationssystems für die Energieforschungsförderung in Deutschland. EnArgus stellt über ein neues Energieforschungs-Wiki für ausgewählte Themenbereiche der Energieforschung eine Fachontologie bereit, die d...

We present a user-orientated design of a motor rehabilitation program for stroke patients. Motor rehabilitation is an important step for stroke patients on their way back into living an independent life. A special software design is needed to suit the patients' needs for an eective and motivating therapy. In a gamied therapy, as we propose, it must...

The Smartphone Project (TSP) is an interactive dance-performance in a professional setting that exploits the communication channels provided by smartphone-apps as a new material in the dance-theatre domain. We present an account of the experience and its staging. Based on an initial study with 36 participants from the audience, we present results a...

This paper presents a suite for configuring and deploying mobile, location-based educational games for natural history museums. We propose a novel approach that uses structured museum-data as game-content and present an initial study thereof. The overall aim is to enable domain-experts, like teachers or museum educators, to create educational games...

This paper presents a user-study of the TOTEM.Scout mobile tool that has been developed for in-situ creation of location-based content for games and edutainment applications based on configurable data templates. It works in cooperation with the TOTEM.Designer web-based tool and enables convenient structuring, management and export of the created co...

This paper reports on AREEF, the world's first multi-player Under Water Augmented Reality (UWAR) experience. The underlying mission of this work was to bring computer games and entertainment applications from traditional settings into the water using Augmented Reality (AR) technology. We provide an application overview and present findings from our...

In Kap. 1 haben wir schon VR als innovative Form der Mensch-Computer-Interaktion kennengelernt. In diesem Kapitel behandeln wir die Gestaltung und Realisierung von Interaktionen und der daraus resultierenden Benutzungsschnittstelle (engl. User Interface ) eines VR-Systems detailliert. Ein Nutzer interagiert mit einem VR-System, um virtuelle Objekte...

Over the last few years we have witnessed the smartphone dominating the market, the rapid growth of mobile apps, a surge in mobile augmented reality and location-based apps, and burgeoning mobile communities. While mobile topics continue to provide rich research challenges, people and companies outside academia already use these apps regularly. Thi...

This paper presents the TOTEM framework for supporting the content creation and structuring for location-based games. Shapes (templates), Marbles (instances) and Pebbles (raw data) build the conceptual background while two integrated tools support the overall workflow: TOTEM.Designer, a web-application that allows for desktop-based authoring and TO...

Presence is usually assessed via a variety of subjective and objective measures. However, constraints often result in subjective measurements using questionnaires as a key method of data collection. In this paper we present a study of 44 participants of a collaborative augmented reality game known as TimeWarp which used both subjective and objectiv...

Designing Augmented Reality location aware games requires an understanding of how form and content issues impact on presence. A study of 60 players was conducted using questionnaires, video analysis and interviews. The results indicate that content including: moral dilemmas, strong narratives, using real locations effectively and applying simple ph...

This paper presents the location-based game Tidy City and the accompanying authoring tools that enable users to create new scavenger hunt like missions. In the game the players need to physically explore their city by interpreting clues to find the correct target destination. The in-situ authoring tool Tidy City Scout enables designers to likewise...

Touring location-based experiences is challenging, as both content and underlying location services must be adapted to each new setting. A study of a touring performance called Rider Spoke as it visited three different cities reveals how professional artists developed a novel approach to these challenges in which users drove the co-evolution of con...

Tidy City is a mobile location-based game concept that is easy to appropriate. The game has been designed with simplicity in mind so that everyone can implement their own designs within the given structure and share it online. The system comprises a game-client for Android devices and a pair of dedicated authoring tools that combine in-situ mobile...

In the gaming area, augmented reality (AR) and especially mobile augmented reality provides unique opportunities. Unlike traditional video games, mobile augmented reality games are not imprisoned in the screen-space but provide for interaction with the world that surrounds us. They incorporate real locations and objects into the game, therefore tap...

Touring location-based experiences is challenging as both content and underlying location-services must be adapted to each new setting. A study of a touring performance called Rider Spoke as it visited three different cities reveals how professional artists developed a novel approach to these challenges in which users drove the co-evolution of cont...

This paper charts the distinctive challenges of designing mobile experiences for cyclists and presents two studies of mobile cycle- based experiences: one a heritage tour; the other an exploration of a city at dusk involving recording and listening to personal stories. 7RXQGHUVWDQGWKHF\FOLVWV∂�H(SHULHQFHVTXHVWLRQQDLUHV��LQWHUYL HZV� and observation...

Location-based experiences depend on the availability and reliability of wireless infrastructures such as GPS, Wi-Fi, or mobile phone networks; but these technologies are not universally available everywhere and anytime. Studies of deployed experiences have shown that the characteristics of wireless infrastructures, especially their limited coverag...

This invited chapter describes the development of a prototype digital artwork/game which was designed to make players more aware of the health consequences of their behaviours. A world first, the Wellcome-funded project described has been demonstrated internationally and gone on, after further development by Nottingham University, to win an interna...

Previous research has proposed that authoring tools for location- aware mobile experiences should be extended to reveal the usually hidden ubiquitous computing infrastructure to designers so that they can take account of its characteristics when placing digital content. This paper explores this idea in practice, describing how a suite of such autho...

In this chapter we provide an overview of the area of mobile location-based gaming and its relation to maps. Digital maps of various forms are essential to enable the implementation of mobile location-based games and support various key tasks from content authoring and interaction scripting over game state management and location-based interaction...

Mobile phones are a widely-available class of device with supporting communications infrastructure which can be appropriated and exploited to support ubicomp experiences. However mobile phones vary hugely in their capabilities. We explore how a single dimension of phone application type embodies the critical trade-off between capability and availab...

With 3D information space we denote a three-dimensional visualization that contains 2D visualizations and puts them in a semantic context. In this paper, we show how Mixed Reality (MR) can be exploited as a technology in order to realize better interaction in a 3D information space and, as a result, to develop new interactive visualization techniqu...