No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Intergrating UAV Development Technology with Augmented Reality toward Landscape Tele-Simulation
... Moreover, they play a crucial role in recognizing objects on construction sites [34]. Additionally, UASs have been selectively employed in the domain of Augmented Reality (AR) to visualize three-dimensional models of buildings, facilitating project analysis and discussion [35][36][37][38]. ...
... Notably, the developed system offered unique advantages not found in the reviewed literature [36,37]. It facilitated the visualization of real-scale models at predefined positions, which is a feature that had not been previously explored. ...
... However, the developed system for the Windows platform had limitations in terms of UAV control during flight and the execution of programmed flights, which had been achieved in previous research [35,37,38]. These limitations were observed within the context of the utilized resources, including the Mavic Air UAV, DJI SDK for Windows, Unity engine, and Microsoft Visual Studio. ...
Unmanned aerial systems (UASs) offer a less invasive solution for accessing remote areas and sites, making them valuable in Architecture, Engineering, Construction, and Operation (AECO). Their ease of use, ability to reach previously inaccessible areas, and sensor integration provide new project perspectives. Augmented Reality (AR), which allows for the real-time insertion of virtual elements into physical spaces, is also being explored in the AECO industry. Recognizing the potential of these technologies, this research aims to integrate them for on-site building model visualization. This article presents the development of resources to visualize building design implementation in AR, which is supported by UASs through georeferencing. The system development process included establishing the software architecture, creating interface prototypes, and constructing the model. It was possible to visualize the building model in AR within the real world; however, limitations were identified regarding the UAS used and its Application Programming Interface, which affected the aircraft’s programmed trajectory. The contribution of this paper lies in exploring the utilization of georeferenced AR enabled by UAS for visualizing architectural designs on site, detailing the steps and strategies employed to achieve this, highlighting the limitations of the chosen approach, and proposing potential solutions to the issues identified in the research.
... For example, the 3D as-built model obtained from UAV photogrammetry was compared to the 4D as-planned model in a BIM environment [34,42]. UAV integration with XR for purposes such as smart historic tourism [30], flight training simulators [43], and virtual site visits [44,45] followed, with eight articles (22%). BIM acted as an intermediary between UAVs and XR technologies, taking the third position with four articles (11%). ...
... This proposed UAV-AR system can assist decision-makers and site planners in observing simulated projects at real sites. Yan et al. [44] performed a similar procedure, using AR technology to visualize real-time video from UAVs for off-site participants. The simulation system uses an AR algorithm named Simultaneous Localization and Mapping (SLAM) to render virtual building models in real-time and real-world environments. ...
... Some studies have tackled this limitation by integrating UAVs and XR to develop real-time video streaming, which can offer many benefits. First, it allows more users to participate in simulations by transmitting the simulated scene to off-site devices in realtime [44]. Second, off-site users can interact with on-site construction workers and make informed and quick decisions [25]. ...
Unmanned Aerial Vehicles (UAVs) have been employed in the construction industry in the last decade for various purposes such as progress monitoring and building inspection. Recently, there has been a rising trend of employing UAVs with other digital technologies (DTs), such as Building Information Modeling and Extended Reality. The integration of these technologies encourages automation and digitization toward better project performance. However, little is known about the implementation of UAVs in conjunction with other DTs. Therefore, this study performs a systematic literature review to determine application areas and technology trends regarding UAVs’ integration with other DTs. The search yielded 287 articles, of which 36 satisfied the established inclusion criteria and formed the foundation of this systematic review. Seven application areas of UAV integration with other DTs were identified: progress monitoring, historic building conservation, information management, construction safety, construction education, structural and infrastructure inspection, and transportation. This study also revealed UAV technology trends encouraging automation and digitization: automated progress monitoring, automated UAV inspection planning, real-time video streaming, and parametric model development of historicbuildings. This study is expected to be a starting point of future in-depth research by providing a general understanding of the current applications of UAVs integration with other DTs.
... Utilized technological attributes: Kudan uses Simultaneous Localization and Mapping (SLAM)-based markerless AR technology [24]. ...
... The system employs the SLAM-based markerless AR algorithm named KUDAN SLAM AR to accomplish the required outdoor review function [24]. ...
Technology is employed in the fields of architecture, engineering, and construction (AEC) for characteristics like producing visual representations and offering assistance during the building phase. Both users and creators of these tools are able to immediately take advantage of the technology's potential as well as create a variety of workarounds for its drawbacks. Both viewpoints will be looked at in this study with regard to mobile extended reality SDKs (software development kit). By excluding the articles that did not provide the relevant information, this research concentrates solely on the papers that discuss the technological aspects of the SDK that were used, the opportunities the SDK offers, and/or the flaws of the SDK. The study's main objective is to compare the technological contributions made by the SDKs employed in the scope of the examined literature to the AEC disciplines and to the contexts in which such contributions are made. Through applications in literature research, the study aims to highlight the contributions of mobile extended reality SDKs to the fields of architecture, engineering, and construction. An entry-level developer can use the SDKs in accordance with his work by using the comparison diagrams, produced in this study, to see the relationships and comparisons between them, as well as to build a framework for what uses should be made in which domains. The technological capabilities and constraints of SDKs have an impact on how research is designed. Making relationality diagrams on the SDK to use and the effects it will have throughout the research phase is also crucial. As a result of the research, SDKs permit flexible uses in a variety of sectors, and their use also financially and logistically supports literature studies.
... Drones provide new perspectives that are not visible from the user's range of action (Gallacher, 2017). To visualize nonexistent landscapes, both past and future, from the air, a method that integrates AR and drones has been proposed (Koch et al., 2011;Wen & Kang, 2014;Unal et al., 2018Unal et al., , 2020Yan et al., 2019). This integrated AR-drone method enables the perception of augmented macro-level information by adding digital information to the aerial video acquired from the drone. ...
... In a subsequent paper, Unal et al. (2020) proposed two methods for integrating drones and AR, one involving location-based positioning using GPS and accelerometers and the other involving vision-based positioning using videos from a monocular camera attached to the drone. Yan et al. (2019) proposed a landscape visualization method that involved AR-drone integration and that used simultaneous localization and mapping. However, the method has not yet been implemented because the drone's captured videos cannot be handled by the AR engine. ...
The integration of augmented reality and drones allows past and future landscapes to be visualized from an aerial perspective. However, these visualizations still suffer from the occlusion problem, where the three-dimensional (3D) virtual model displayed in the real world is in front of a real-world object. Currently, city digital twins are essential for the sustainable development of cities and the development of detailed 3D models of cities. By visualizing the city digital twin, augmented reality can facilitate the participation of nonexpert citizens in the decision-making process of urban design, but research examples are limited. Here, using detailed city 3D models, we develop a digital-twin approach to outdoor augmented reality with occlusion handling for both first-person and bird’s-eye views. In a verification experiment, the occlusion handling accuracy of the prototype system was measured to be about 0.8 using intersection over union. The frame rate of the entire prototype system was about 30 fps, and the delay between the controller and the augmented reality device was about 3 s. The internet-based system architecture was developed to integrate augmented reality and drone systems. Our system allows multiple stakeholders involved in building construction projects to observe aerial perspectives of those projects, both on-site and off-site via an internet browser, using augmented reality with occlusion handling.
... Zollmann et al. (2014) used UAV to conduct 3D reconstruction of site information, providing on-site visualization on mobile. Yan et al. (2019) combined UAV and AR to run landscape simulations. Commercial AR-UAVs for construction could be the next important component of the AR market. ...
The 4th industrial revolution started in 2016 and referred to a new phase in the industrial revolution. One of the most significant technological evolvements during the 4th industrial revolution is Augmented Reality (AR) technology. AR superimposes interactional virtual objects/images to real environments. Because of the interaction and see-through characteristics, AR is better applied to engineering than Virtual Reality (VR). The application of AR in civil infrastructure can avoid artificial mistakes, improve efficiency, and saves budget. This article reviews AR applications in civil infrastructure, focusing on research studies in the latest five years (2016–2020) and their milestone developments. More than half of the AR research and implementation studies have focused on the construction domain in the last five years. Researchers deploy AR technologies in on-site construction to assist in discrepancy checking, collaborative communication, and safety checking. AR also uses building information models (BIMs) to produce detailed 3D structural information for visualization. Additionally, AR has been studied for structural health monitoring (SHM), routine and damage detection, energy performance assessment, crack inspection, excavation, and underground utility maintenance. Finally, AR has also been applied for architecture design, city plan, and disaster prediction as an essential part of smart city service. This article discusses the challenges of AR implementation in civil infrastructure and recommends future applications.
The agricultural sector is evolving with the adoption of smart farming technologies , where Digital Twins (DTs) offer new possibilities for real-time monitoring, simulation, and decision-making. While previous research has explored the Internet of Things (IoT), UAVs, machine learning (ML), and remote sensing (RS) in enhancing agricultural efficiency, a systematic approach to integrating these technologies within a DTs ecosystem remains underdeveloped. This paper presents a systematic review of 167 studies published between 2018 and 2025. The objective of this study is to examine recent advancements in DTs-enabled precision agriculture and propose a comprehensive framework for designing, integrating, and optimizing DTs in smart farming. The study systematically examines the current state of DT adoption, identifies key barriers, and computational efficiency challenges, and provides a step-by-step methodology for DT implementation. The review sheds light on potential future research direction and implications for policy, with the aim to speed up the adoption of DTs-based farm management systems in their operational success and commercial viability through analysis of practical applications and future perspectives. This study presents an innovative strategy for integrating digital and physical systems into agriculture and is an important contribution to existing literature.
Während mobile Erweiterte Realität (mAR) bisher überwiegend auf handelsüblichen Mobilgeräten (Smartphones, Tablets) oder auf speziellen AR-Brillen umgesetzt wird, wäre es auch möglich, unbemannte Luftfahrzeuge (UAVs) zur Realisierung von mAR einzusetzen. Aus der Kombination dieser beiden innovativen Technologien ergäbe sich eine Vielzahl neuer Anwendungen, bei der das Kamerabild des UAVs durch AR-Informationen in Echtzeit erweitert und auf einem separaten Bildschirm dargestellt wird. Da aber Wissen über die Einsatzmöglichkeiten und die technische Umsetzung von AR-UAVs bislang nicht bekannt sind, wird das Potenzial der Kombination aus mAR und UAV bisher nicht genutzt. Daher werden in diesem Beitrag zum einen potenzielle AR-UAV-Anwendungsbereiche auf Basis einer systematischen Literaturanalyse identifiziert sowie zum anderen unterschiedliche Software-Architekturen, mit denen AR-UAV-Anwendungen software-technisch umgesetzt werden können, auf Basis von verschiedenen Tracking-Technologien entworfen und bewertet, um es insgesamt zu ermöglichen, das Potenzial von AR-UAVs stärker auszuschöpfen.
Trotz bestimmungsgemäßer und sachgerechter Pflanzenschutzmittelanwendung kann es aufgrund von ungünstigen Umweltbedingungen, wie undurchlässigen Böden oder Flächen mit starker Hangneigung in Kombination mit hohen Regenmengen, zu einem erhöhten Risiko durch Pflanzenschutzmitteleinträge in Gewässern kommen. Die Landwirtschaftskammer NRW hat in Zusammenarbeit mit dem Julius Kühn-Institut und dem Land NRW das webbasierte Beratungswerkzeug H2Ot-Spot Manager NRW entwickelt. Der H2Ot-Spot Manager unterstützt die zielgerichtete risikomindernde Pflanzenschutzberatung, indem ein nutzerfreundliches und fachlich validiertes Werkzeug zur Unterstützung des vorbeugenden Gewässerschutzes zur Verfügung steht. Das von einer Pflanzenschutzmittelanwendung ausgehende Umweltrisiko kann mit Hilfe anerkannter Modelle analysiert werden. In der Beratung können mit diesen flächenbezogenen Ergebnissen das Problembewusstsein für Risiken durch Pflanzenschutzmittelanwendungen geschärft, Risikominderungsmöglichkeiten aufgezeigt und Handlungsempfehlungen zur Eintragsvermeidung von Pflanzenschutzmittel gegeben werden. Dies unterstützt den Landwirt dabei, seiner Sorgfaltspflicht nachzukommen und in sensiblen Gebieten eine gewässerschonende Bewirtschaftung durchzuführen.
Soil sampling is an important tool to gather information for making proper decisions regarding the fertilization of fields. Depending on the national regulations, the minimum frequency may be once per five years and spatially every ten hectares. For precision farming purposes, this is not sufficient. In precision farming, the challenge is to collect the samples from such regions that are internally consistent while limiting the number of samples required. For this purpose, management zones are used to divide the field into smaller regions. This article presents a novel approach to automatically determine the locations for soil samples based on a soil map created from drone imaging after ploughing, and a wearable augmented reality technology to guide the user to the generated sample points. Finally, the article presents the results of a demonstration carried out in southern Finland.
A visuo-haptic augmented reality (VHAR) interface is presented enabling an operator to teleoperate an unmanned aerial vehicle (UAV) equipped with a custom CdZnTe-based spectroscopic gamma-ray detector in outdoor environments. The task is to localize nuclear radiation sources, whose location is unknown to the user, without the close exposure of the operator. The developed detector also enables identification of the localized nuclear sources. The aim of the VHAR interface is to increase the situation awareness of the operator. The user teleoperates the UAV using a 3DOF haptic device that provides an attractive force feedback around the location of the most intense detected radiation source. Moreover, a fixed camera on the ground observes the environment where the UAV is flying. A 3D augmented reality scene is displayed on a computer screen accessible to the operator. Multiple types of graphical overlays are shown, including sensor data acquired by the nuclear radiation detector, a virtual cursor that tracks the UAV and geographical information, such as buildings. Experiments performed in a real environment are reported using an intense nuclear source.
Over the past few years, the application of camera-equipped Unmanned Aerial Vehicles (UAVs) for visually monitoring construction and operation of buildings, bridges, and other types of civil infrastructure systems has exponentially grown. These platforms can frequently survey construction sites, monitor work-in-progress, create documents for safety, and inspect existing structures, particularly for hard-to-reach areas. The purpose of this paper is to provide a concise review of the most recent methods that streamline collection, analysis, visualization, and communication of the visual data captured from these platforms, with and without using Building Information Models (BIM) as a priori information. Specifically, the most relevant works from Civil Engineering, Computer Vision, and Robotics communities are presented and compared in terms of their potential to lead to automatic construction monitoring and civil infrastructure condition assessment.
The need for visual landscape assessment in large-scale projects for the evaluation of the effects of a particular project on the surrounding landscape has grown in recent years. Augmented reality (AR) has been considered for use as a landscape simulation system in which a landscape assessment object created by 3D models is included in the present surroundings. With the use of this system, the time and the cost needed to perform a 3DCG modeling of present surroundings, which is a major issue in virtual reality, are drastically reduced. This research presents the development of a 3D map-oriented handheld AR system that achieves geometric consistency using a 3D map to obtain position data instead of GPS, which has low position information accuracy, particularly in urban areas. The new system also features a gyroscope sensor to obtain posture data and a video camera to capture live video of the present surroundings. All these components are mounted in a smartphone and can be used for urban landscape assessment. Registration accuracy is evaluated to simulate an urban landscape from a short- to a long-range scale. The latter involves a distance of approximately 2000 m. The developed AR system enables users to simulate a landscape from multiple and long-distance viewpoints simultaneously and to walk around the viewpoint fields using only a smartphone. This result is the tolerance level of landscape assessment. In conclusion, the proposed method is evaluated as feasible and effective.
Augmented reality (AR) allows for an on-site presentation of information that is registered to the physical environment. Applications from civil engineering, which require users to process complex information, are among those which can benefit particularly highly from such a presentation. In this paper, we will describe how to use AR to support monitoring and documentation of construction site progress. For these tasks, the responsible staff usually requires fast and comprehensible access to progress information to enable comparison to the as-built status as well as to as-planned data. Instead of tediously searching and mapping related information to the actual construction site environment, our AR system allows for the access of information right where it is needed. This is achieved by superimposing progress as well as as-planned information onto the user's view of the physical environment. For this purpose, we present an approach that uses aerial 3-D reconstruction to automatically capture progress information and a mobile AR client for on-site visualization. Within this paper, we will describe in greater detail how to capture 3-D, how to register the AR system within the physical outdoor environment, how to visualize progress information in a comprehensible way in an AR overlay, and how to interact with this kind of information. By implementing such an AR system, we are able to provide an overview about the possibilities and future applications of AR in the construction industry.
The extent of effectiveness of real-time communication within BIM environment is somehow restrained due to the limited sense of immersion into virtual environments. The objective of this paper highlights the need for a structured methodology of fully integrating Augmented Reality (AR) technology in BIM. Based on the generic review of BIM in construction, this paper forms the rationales for the onsite information system for construction site activities, and then formulates the methods of configuring BIM + AR prototypes. It is demonstrated that, extended to the site via the “hand” of AR, the BIM solution can address more real problems, such as low productivity in retrieving information, tendency of committing error in assembly, and low efficiency of communication and problem solving.
Unmanned Aerial Vehicle (UAV) systems as a data acquisition platform and as a measurement instrument are becoming attractive for many surveying applications in civil engineering. Their performance, however, is not well understood for these particular tasks. The scope of the presented work is the performance evaluation of a UAV system that was built to rapidly and autonomously acquire mobile three-dimensional (3D) mapping data. Details to the components of the UAV system (hardware and control software) are explained. A novel program for photogrammetric flight planning and its execution for the generation of 3D point clouds from digital mobile images is explained. A performance model for estimating the position error was developed and tested in several realistic construction environments. Test results are presented as they relate to large excavation and earth moving construction sites. The experiences with the developed UAV system are useful to researchers or practitioners in need for successfully adapting UAV technology for their application(s). Published by Elsevier B.V.
DroneAR: augmented reality supported unmanned aerial vehicle (UAV) in agriculture for farmer perspective
- Y Wang
- H B L Duh
- T Taketomi
- H Kato
Wang, Y., Duh, H.B.L., Taketomi, T. and Kato, H.: 2015, DroneAR: augmented
reality supported unmanned aerial vehicle (UAV) in agriculture for farmer perspective,
Proceedings of the IEEE 2015.
UAVAR: an augmented reality supported unmanned aerial vehicle application
- Y Wang
- W Huang
- H B L Duh
Wang, Y., Huang, W. and Duh, H.B.L.: 2016, UAVAR: an augmented reality supported
unmanned aerial vehicle application, 1st International Workshop on Mixed and Augmented
Reality (MARI 2016), Tasmania, 1-2.
A Design of Landscape Tele-simulation System Using Unmanned Aerial Vehicle and Augmented Reality
- L Yan
- T Fukuda
- N Yabuki
Yan, L., Fukuda, T. and Yabuki, N.: 2018, A Design of Landscape Tele-simulation System
Using Unmanned Aerial Vehicle and Augmented Reality, The 41st Symposium on Computer
Technology of Information, Systems and Applications, Architectural Institute of Japan,
35-38.