Article

Review of Robotic Infrastructure Inspection Systems

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

In order to minimize the costs, risks, and disruptions associated with structural inspections, robotic systems have increasingly been studied as an enhancement to current inspection practices. Combined with the increasing variety of commercially available robots, the last two decades have seen dramatic growth in the application of such systems. The use of these systems spans the breadth of civil infrastructure works, and the variety of implemented robotic systems is growing rapidly. However, the highly interdisciplinary nature of research in this field means that results are disseminated across a broad variety of publications. This review paper aggregates these studies in an effort to distill the state of the art in inspection robotics, as well as to assess outstanding challenges in the field and possibilities for the future. Overall, analysis of these studies illustrates that the design of inspection robots is often a case-specific compromise between competing needs for sophisticated inspection sensing and for flexible locomotion in challenging field environments. This review also points toward the growing use of robots as a platform to deploy advanced nondestructive evaluation (NDE) technologies, as well as the expanded use of commercially available robotic systems. Two key outstanding challenges for future researchers are suggested as well. The first is the need for more sophisticated, and inspection-driven, robot autonomy. The other is the need to process and manipulate the massive data sets that modern robots generate.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... For example, machine learning (ML) models have been utilized to classify cracks based on engineered features, such as texture, shape, and intensity [15]. Image processing techniques, such as contrast enhancement, improve crack visibility by filtering out irrelevant background noise [9,10,16]. While these systems have demonstrated potential in improving detection efficiency, they face several challenges that hinder their effectiveness in real-world applications [17]. ...
... Fuzzy systems have been integrated with image processing techniques to improve crack visibility and contrast while reducing noise and background clutter [9,10]. These methods, however, typically lack the ability to learn directly from data, limiting their application to new road conditions or unseen crack patterns [8]. ...
... Robotic inspection of infrastructure has seen an increasing interest over the past decade as it promises to mitigate risk to human life, minimize costs and reduce other disruptions frequently encountered during structural inspection tasks [1]. In particular, aerial robots, because of their advanced agility and flexibility have been applied to a variety of such inspection tasks [2][3][4][5][6][7]. ...
... However, such methods still require quality camera data to localize the known markers [8,9]. As noted in [1,10], not uncommon instances of inspection tasks are carried out in previously known environments in post-disaster conditions such as in the aftermath of a fire. These environments present a challenge for robot operations as they can be GPS-denied in nature and contain visual obscurants such as smoke. ...
Preprint
For robotic inspection tasks in known environments fiducial markers provide a reliable and low-cost solution for robot localization. However, detection of such markers relies on the quality of RGB camera data, which degrades significantly in the presence of visual obscurants such as fog and smoke. The ability to navigate known environments in the presence of obscurants can be critical for inspection tasks especially, in the aftermath of a disaster. Addressing such a scenario, this work proposes a method for the design of fiducial markers to be used with thermal cameras for the pose estimation of aerial robots. Our low cost markers are designed to work in the long wave infrared spectrum, which is not affected by the presence of obscurants, and can be affixed to any object that has measurable temperature difference with respect to its surroundings. Furthermore, the estimated pose from the fiducial markers is fused with inertial measurements in an extended Kalman filter to remove high frequency noise and error present in the fiducial pose estimates. The proposed markers and the pose estimation method are experimentally evaluated in an obscurant filled environment using an aerial robot carrying a thermal camera.
... Critical infrastructure is traditionally inspected by periodic visual evaluations [21,22], but 20 there is increasing interest in the use of autonomous drones [21,23] or sensor-based data-driven mechanisms for structural damage detection (SDD) and SHM. The latter approach can detect damages that may not be apparent through visual inspections [24] and can support real-time decision-making. ...
... Critical infrastructure is traditionally inspected by periodic visual evaluations [21,22], but 20 there is increasing interest in the use of autonomous drones [21,23] or sensor-based data-driven mechanisms for structural damage detection (SDD) and SHM. The latter approach can detect damages that may not be apparent through visual inspections [24] and can support real-time decision-making. ...
... Similarly, in biomedical research, it facilitates accurate manipulation of biological specimens with minimal disruption [39], addressing limitations of existing techniques. Furthermore, the flexible design of the outer feedback loop enables seamless integration into automated inspection systems [40], ensuring reliable performance in industrial robotics. These capabilities underscore the method's broad applicability, effectively bridging the gap between theoretical innovation and practical implementation in Industry 4.0 environments. ...
Article
Full-text available
The control of closed architecture industrial optical tweezers systems presents substantial challenges due to the limited knowledge of inner controller configurations and control gain structures. Conventional methods, such as the computed-torque approach, often prove inadequate for closed architecture robots because the torque terminal is inaccessible to end-users. This creates a disconnect between advanced control algorithms and practical industrial needs. To tackle these challenges, we propose a novel approach utilizing adaptive neural network-based control methodology. Our method employs neural networks to approximate the low-level controller, thereby bypassing the constraints of traditional closed-loop architectures. By integrating an external feedback loop that operates independently of the inner control architecture, our approach offers flexibility for design adjustments and facilitates advanced control actions. Furthermore, we conduct a comprehensive stability analysis of external feedback control systems interacting with a fixed inner loop. In summary, this study represents a pioneering effort in addressing the complexities of industrial optical tweezers systems with closed uncertain architectures, providing an innovative solution aimed at enhancing control capabilities for applications in industrial settings.
... This paper aims to survey the use of LiDAR and HSI in tank inspection, highlighting their integration and the advantages of using advanced sensing techniques over traditional methods [17,18]. This application is not specific to maritime tanks, as many other structures that are impacted by corrosion need constant maintenance and analysis (also known as Structural Health Monitoring or SMH), which could be positively affected by such a technique [19,20]. ...
Article
Full-text available
This paper reviews various sensor technologies for tank inspection, focusing on Light Detection and Ranging (LiDAR) and Hyperspectral Imaging (HSI) as advanced solutions for corrosion detection. These technologies are evaluated alongside traditional methods such as ultrasonic, electromagnetic, and thermographic inspections. This review highlights their potential to enhance inspection accuracy, reduce the limitations of manual inspection, and support integrated data analysis for comprehensive asset management. Additionally, this paper proposes a pathway for automating these techniques to streamline inspection processes and improve implementation in practical applications.
... Additionally, crack detection relies solely and directly on the inspector so the experience level of the inspector determines the level of crack detection accuracy. To tackle this issue, non-invasive techniques are exercised to identify and segment cracks in concrete structures incorporating image processing, machine learning, and deep learning techniques [11][12][13]. These non-invasive and automated techniques are faster, economical, and secure for human inspection. ...
Article
Full-text available
The You Only Look Once (YOLO) network is considered highly suitable for real-time object detection tasks due to its characteristics, such as high speed, single-shot detection, global context awareness, scalability, and adaptability to real-world conditions. This work introduces a comprehensive analysis of various YOLO models for detecting cracks in concrete structures, aiming to assist in the selection of an optimal model for future detection and segmentation tasks. The YOLO models are initially trained on a dataset containing both images with and without cracks, producing a generalized model capable of extracting abstract features beneficial for crack detection. Subsequently, transfer learning is employed using a dataset that reflects real-world conditions, such as occlusions, varying crack sizes, and rotations, to further refine the model. Crack detection in concrete remains challenging due to the wide variation in crack sizes, aspect ratios, and complex backgrounds. To achieve optimal performance, we test different versions of YOLO, a state-of-the-art single-shot detector, and aim to balance inference speed and mean average precision (mAP). Our results indicate that YOLOv10 demonstrates superior performance, achieving a mean average precision (mAP) of 74.52% with an inference time of 19.5 milliseconds per image, making it the most effective among the models tested.
... Autonomous drones can follow pre-planned flight paths, avoid obstacles, and adapt to changing weather conditions, while human operators maintain the authority to intervene in situations that require human judgment [33]. Studies have shown that HAT systems can perform effectively in unmanned settings for search and rescue [34], [35], infrastructure inspection [36], [37], and agriculture and traffic monitoring [38], [39]. ...
Preprint
Full-text available
Artificial Intelligence (AI) techniques, particularly machine learning techniques, are rapidly transforming tactical operations by augmenting human decision-making capabilities. This paper explores AI-driven Human-Autonomy Teaming (HAT) as a transformative approach, focusing on how it empowers human decision-making in complex environments. While trust and explainability continue to pose significant challenges, our exploration focuses on the potential of AI-driven HAT to transform tactical operations. By improving situational awareness and supporting more informed decision-making, AI-driven HAT can enhance the effectiveness and safety of such operations. To this end, we propose a comprehensive framework that addresses the key components of AI-driven HAT, including trust and transparency, optimal function allocation between humans and AI, situational awareness, and ethical considerations. The proposed framework can serve as a foundation for future research and development in the field. By identifying and discussing critical research challenges and knowledge gaps in this framework, our work aims to guide the advancement of AI-driven HAT for optimizing tactical operations. We emphasize the importance of developing scalable and ethical AI-driven HAT systems that ensure seamless human-machine collaboration, prioritize ethical considerations, enhance model transparency through Explainable AI (XAI) techniques, and effectively manage the cognitive load of human operators.
... In confined spaces, conventional large machines encounter difficulties performing tasks such as inspections, unclogging, and transportation without compromising structural integrity (1)(2)(3)(4)(5)(6)(7)(8). To address these challenges, microrobots with excellent maneuverability (9)(10)(11)(12)(13)(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24), such as fast running speeds (9,10,19,21,25,26) and short turning radius (17,20,27), have emerged as a promising solution due to their ability to navigate in narrow spaces (13,(28)(29)(30)(31)(32)(33)(34). ...
Article
Full-text available
The ability to move backward is crucial for millimeter-scale microrobots to navigate dead-end tunnels that are too narrow to allow for turning maneuvers. In this study, we introduce a 15-mm-long legged microrobot, BHMbot-B, which is capable of rapid forward and backward locomotion through vibration mode transition control. By properly arranging the vibratory motions of the magnet, cantilever, and linkages, the pitching movement of the body and the vibration of the forelegs are in phase during the first-order vibration mode of the cantilever and in antiphase during the second-order mode, which induces the forward and backward movement of the microrobot. Owing to its outstanding load-bearing capacity, the BHMbot-B equipped with dual electromagnetic actuators, an onboard battery, and a control circuit, can execute complex running trajectories under wireless command. Its maximum untethered running speeds are evaluated as 18.0 BL/s (360 mm/s) in the forward direction and 16.9 BL/s (338 mm/s) in the backward direction.
... The innovative robotic implementation offers a compelling solution to overcome the mentioned limitations and ensure inspector safety and data objectivity [19], [20]. Robots can be specifically designed to be deployed in hazardous environments, confined spaces, at altitudes, or even submerged underwater minimising the risks encountered by human inspectors while enabling a remarkably comprehensive inspection [21]. ...
... With the invention of forthcoming sensors including mobile sensors, unmanned ground vehicles (UGVs), digital and high-speed cameras, there has been a significant change to non-contact sensing techniques in SHM (Dabous & Feroz, 2020;Sony et al., 2019). They are simpler to implement, require less labour, more economical and enabling more trusted data acquisition from structures with excellent quality spatial and temporal data, (Lattanzi & Miller, 2017). As opposed to conventional contact sensors, non-contact sensors produce images and videos that depends on major developments in image processing, computer vision, robotics and deep learning algorithms, where structural engineers still face number of difficulties. ...
Article
Full-text available
The joint is one of the most critical parts of a building structure. In steel buildings extended shear tab (EST) connection is becoming an attractive alternative for light to moderate end shear connections due to its simplicity and economy. Detection and prediction of cracks in connections perform an important job in the maintenance of steel structures. Currently, the structural inspections of such joints are conducted by manually which is a tedious and expensive. It is essential to detect damages in joints in order to ensure structural safety. In this study a two-phase convolutional neural network (CNN) using transfer learning is presented for detection and segmentation of cracks and damages in component parts of an EST connection. Different pretrained networks using transfer learning, including AlexNet, GoogLeNet, ResNet-101 and VGG-16 are considered for crack and damage detection. The undamaged, cracked and damaged images of component parts of sixteen EST connections were generated through the finite-element simulation which were used to develop the CNN model. Segmentation and detection results show that VGG-16 model appears to give the best results with 100% precision and accuracy, followed by ResNet-101, and AlexNet, and finally GoogLeNet gives the least performance among the four methods selected. This study will be resourceful for quick reference for those who are working in structural health monitoring field.
... Monitoring, prevention, and mitigation [76,77] Pollution detection and cleaning [19,78] Marine infrastructures Designing, constructing, and maintaining infrastructures [79,80] Infrastructure inspection and maintenance [81,82] Marine natural resources Identification and exploration [1,83] Deep sea mining exploration [84,85] Oceanographic applications Scientific study of the ocean's physical, chemical, biological, and geological aspects [86,87] Monitoring, exploration, and research [88,89] Autonomous vehicles in ocean engineering, like other complex systems, rely on a combination of scientific and technological approaches to effectively operate in the various domains-air, surface, and subsurface. In underwater operations, these vehicles typically use sonar and acoustic communication systems to navigate and gather data, thus overcoming the challenges faced in the underwater environment [90,91]. ...
Article
Full-text available
In ocean engineering, engineering principles are applied to the ocean domain. Advanced technology is facilitating efficient exploration of oceanic regions with minimal human intervention, and autonomous vehicles are increasingly used to automate various ocean engineering tasks. However, using fully autonomous vehicles raises ethical and legal concerns that must be properly regulated. Nowadays, the most common applications of autonomous vehicles in the ocean domain include infrastructure maintenance, underwater mapping, resource exploration, environmental monitoring, and various military operations such as mine warfare (MW) and intelligence, surveillance, and reconnaissance (ISR). This article explores the prevalent applications of autonomous vehicles in ocean engineering, analyzing existing regulations, liability and accountability issues, data privacy, cybersecurity challenges, and interoperability. Through a Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis, it was possible to better understand the current state of using autonomous vehicles in ocean engineering and develop a possible future strategy in the field. To make the usage of autonomous vehicles more reliable in ocean engineering, it is essential to advance them technologically and update the existing laws that deal with these kinds of applications.
... They are often selected for bridge inspection purposes because of their excellent payload capacity. 73 These robots can access areas of the bridge that are difficult or impossible for humans to reach, such as the underside of the bridge, or high and narrow sections. This technology and platform are commonly used for NDT solutions in tunnels, where cameras and other sensors are installed on a mobile robot. ...
Article
Recent scientific and technological advancements have enabled a more efficient structural condition assessment of bridges, mainly through the implementation of intelligent inspection strategies. These intelligent strategies can provide early identification of critically damaged components before failure, and will therefore play a key role in extending the life of infrastructure. The latest inspection technologies can provide inspection plans with damage conditions, create a damage report as well as provide statistics and comparisons to the previous inspection findings. The new and existing inspection technologies are directed to help with the digitalization of the Bridge Management System (BMS). The complexity of maintenance/inspection requires organized, automated, open and transparent digital processes, which should consider both-structure and asset management data. Further, the inspection/monitoring findings serve as a source for decision-making models. The digitalized aspects of autonomous inspection provide better performance prediction models and guarantee safety for the users. This paper presents the latest findings in the field of remote inspection of bridges. In particular, the main technologies for inspection and geometrical assessment are depicted, especially those based on computer vision systems installed in UAVs and robots, LiDAR, radar, satellites and other non-contact systems including on-board monitoring. The accompanying article entitled: "Methodologies for remote bridge inspection" deals with the methodologies used for data processing based on Artificial Intelligence (AI).
... This task, known as an object-goal navigation problem, is one of the central problems in embodied AI research [3]. This capability is crucial in a wide range of real-world applications, including urban inspection [50,7,35], home robots [58], monitoring of oil and gas sites [24], and exploration of subterranean environments [1,52,28]. This work addresses the object-goal navigation problem for autonomous inspection in unseen real-world environments (Fig. 1). ...
... The robotic task must be performed in such a way that the process considers reliability, repeatability, and safety. Therefore, it is necessary to enhance operational consistency in the inspection environment [13]. Robotic systems' capabilities have progressed over time, and these systems have become dependent on multiple components with diverse functions. ...
Article
Full-text available
The availability of inspection robots in the construction and operation phases of buildings has led to expanding the scope of applications and increasing technological challenges. Furthermore, the building information modeling (BIM)-based approach for robotic inspection is expected to improve the inspection process as the BIM models contain accurate geometry and relevant information at different phases of the lifecycle of a building. Several studies have used BIM for navigation purposes. Also, some studies focused on developing a knowledge-based ontology to perform activities in a robotic environment (e.g., CRAM). However, the research in this area is still limited and fragmented, and there is a need to develop an integrated ontology to be used as a first step towards logic-based inspection. This paper aims to develop an ontology for BIM-based robotic navigation and inspection tasks (OBRNIT). This ontology can help system engineers involved in developing robotic inspection systems by identifying the different concepts and relationships between robotic inspection and navigation tasks based on BIM information. The developed ontology covers four main types of concepts: (1) robot concepts, (2) building concepts, (3) navigation task concepts, and (4) inspection task concepts. The ontology is developed using Protégé. The following steps are taken to reach the objectives: (1) the available literature is reviewed to identify the concepts, (2) the steps for developing OBRNIT are identified, (3) the basic components of the ontology are developed, and (4) the evaluation process is performed for the developed ontology. The semantic representation of OBRNIT was evaluated through a case study and a survey. The evaluation confirms that OBRNIT covers the domain’s concepts and relationships, and can be applied to develop robotic inspection systems. In a case study conducted in a building at Concordia University, OBRNIT was used to support an inspection robot in navigating to identify a ceiling leakage. Survey results from 33 experts indicate that 28.13% strongly agreed and 65.63% agreed on the usage of OBRNIT for the development of robotic navigation and inspection systems. This highlights its potential in enhancing inspection reliability and repeatability, addressing the complexity of interactions within the inspection environment, and supporting the development of more autonomous and efficient robotic inspection systems.
... In the field of civil engineering and infrastructure (Greenwood et al., 2019) and particularly in the field of inspection and monitoring, UAVs can represent a non-invasive approach in rapid mapping applications (Gaspari et al., 2022) flexible and low-cost (Lattanzi & Miller, 2017) through the use of optical and LiDAR sensors, for disaster assessment after calamitous events or for transport infrastructure damage reduction strategies (Mandirola et al., 2022). The possibility of creating 3D models from UAV surveys (Pepe et al., 2022) through the algorithms of Structure from Motion (SfM) and Multi View Stereo (MVS), has also become commonplace in the monitoring of construction progress on building sites (Teizer, 2015) with a comprehensive and efficient collection of images and the possibility of generating multi-temporal BIM models (Lin et al., 2015). ...
... These systems have the potential to mitigate risks associated with human inspectors, enhance data collection through advanced sensing technologies, and provide a cost-effective and efficient means of evaluating the structural integrity of vital infrastructure components. This shift towards robotic inspection solutions is underpinned by the need for innovations in mobility, autonomy, and sensing, as well as a growing emphasis on purpose-driven designs and multidisciplinary collaborations [14], [15]. ...
... Robots for inspection, both in industry and research, feature a wide range of architectures, traction, and navigation [1]. The differences stem from the specific tasks for which they are designed. ...
Article
Full-text available
Robotic inspection is advancing in performance capabilities and is now being considered for industrial applications beyond laboratory experiments. As industries increasingly rely on complex machinery, pipelines, and structures, the need for precise and reliable inspection methods becomes paramount to ensure operational integrity and mitigate risks. AI-assisted autonomous mobile robots offer the potential to automate inspection processes, reduce human error, and provide real-time insights into asset conditions. A primary concern is the necessity to validate the performance of these systems under real-world conditions. While laboratory tests and simulations can provide valuable insights, the true efficacy of AI algorithms and robotic platforms can only be determined through rigorous field testing and validation. This paper aligns with this need by evaluating the performance of one-stage models for object detection in tasks that support and enhance the perception capabilities of autonomous mobile robots. The evaluation addresses both the execution of assigned tasks and the robot’s own navigation. Our benchmark of classification models for robotic inspection considers three real-world transportation and logistics use cases, as well as several generations of the well-known YOLO architecture. The performance results from field tests using real robotic devices equipped with such object detection capabilities are promising, and expose the enormous potential and actionability of autonomous robotic systems for fully automated inspection and maintenance in open-world settings.
... This task, known as an object-goal navigation problem, is one of the central problems in embodied AI research [3]. This capability is crucial in a wide range of real-world applications, including urban inspection [50,7,35], home robots [58], monitoring of oil and gas sites [24], and exploration of subterranean environments [1,52,28]. This work addresses the object-goal navigation problem for autonomous inspection in unseen real-world environments (Fig. 1). ...
Preprint
This paper addresses the problem of object-goal navigation in autonomous inspections in real-world environments. Object-goal navigation is crucial to enable effective inspections in various settings, often requiring the robot to identify the target object within a large search space. Current object inspection methods fall short of human efficiency because they typically cannot bootstrap prior and common sense knowledge as humans do. In this paper, we introduce a framework that enables robots to use semantic knowledge from prior spatial configurations of the environment and semantic common sense knowledge. We propose SEEK (Semantic Reasoning for Object Inspection Tasks) that combines semantic prior knowledge with the robot's observations to search for and navigate toward target objects more efficiently. SEEK maintains two representations: a Dynamic Scene Graph (DSG) and a Relational Semantic Network (RSN). The RSN is a compact and practical model that estimates the probability of finding the target object across spatial elements in the DSG. We propose a novel probabilistic planning framework to search for the object using relational semantic knowledge. Our simulation analyses demonstrate that SEEK outperforms the classical planning and Large Language Models (LLMs)-based methods that are examined in this study in terms of efficiency for object-goal inspection tasks. We validated our approach on a physical legged robot in urban environments, showcasing its practicality and effectiveness in real-world inspection scenarios.
... Camera-equipped UAVs can effectively monitor the construction and condition of civil infrastructure systems, including high-rise buildings, bridges, and public utilities. They have the potential for automatic construction monitoring and condition assessment, efficiently identifying maintenance needs or structural issues [171,172]. UGVs can then perform detailed inspections and minor repairs at the ground level. ...
... It goes on by documenting the defects with an agreed format for later exchange or analytics, which is finally followed by rehabilitation work to alleviate the impacts of the defects [7]. This "inspect-document-rehabilitate (IDR)" workflow has long been in place and provides a useful guiding framework to plan maintenance work [8]. However, given the immense size of the built environment, its thorough implementation is daunting. ...
Article
Full-text available
Keywords built environment sustainability facility management defect information modeling building information modeling The built environment is subject to various defects as it ages. A well-maintained built environment depends on surveying activities to inspect, document, and rehabilitate the defects that occurred. The advancement of digital technologies paves the pathway towards (1) comprehensive defect inspection by systematic mapping, (2) their consistent documentation by digital modeling, and (3) timely retrofitting by proactive management. However, the three steps of defect mapping , modeling, and management (D3M) remain largely fragmented and have yet to be synergized. Exploiting the pivotal role of building information modeling (BIM) in built asset management, this paper puts forward a cohesive framework for integrated D3M. It leverages the rich geometric-semantic information in BIM to assist defect mapping and enriches the BIM by industry foundation classes (IFCs)-represented defect information. The defect-enriched BIM facilitates defect management in a data-driven manner. The framework was applied in multiple real-life infrastructure and civil works projects. It demonstrates how the BIM-based D3M framework can enhance the maintenance of those that have been built, and ultimately contribute to a safe and sustainable built environment. Future studies are called for to substantiate each of the 3Ms by leveraging BIM as both an enabler and a beneficiary.
... For enabling RVI, the deployment of advanced telecommunications systems is crucial. Unmanned aerial vehicles (UAVs) equipped with cameras or sensors are a promising tool for conducting RVI, especially in situations where physical access may be time-consuming or dangerous, such as when climbing or crawling is required (Lattanzi and Miller 2017). However, managing and analyzing large volumes of data transmitted from UAVs is challenging (He et al. 2018). ...
Conference Paper
To assess the condition of infrastructure such as bridges, highways, and pipelines, trained inspectors typically perform visual inspections with site visits. However, this conventional method can interrupt the regular functioning of the infrastructure as well as be time-consuming, laborious, expensive, and dangerous. To address these problems, this study proposes an immersive metaverse-based infrastructure remote virtual inspection platform. Based on a 3D digital twin model created through digital photogrammetry, this study generates an immersive metaverse to support collaborations between stakeholders for remote inspections. In addition, this study synchronizes a virtual reality (VR) headset with an unmanned aerial vehicle to remotely inspect infrastructure conditions at the target location using augmented real-time images mapped onto the digital twin model. The conducted experiments provided evidence that users equipped with VR headsets were able to see the 3D digital twin model of the infrastructure as if they were physically present on the sites. Within the metaverse space, users could easily navigate to specific areas and conduct inspection tasks by viewing both the digital twin model and real-time images through the proposed platform. The outcomes of this study hold the potential to enhance the efficiency and safety of infrastructure inspection.
... Technological devices are not only necessary to detect special defect types but also important to reduce the inspection effort (Lattanzi and Miller, 2017). For example, a terrestrial laser scanner was used in Mill et al. (2013), and an InfraRed thermography camera on a tripod was used in Garrido et al. (2022) for defect detection. ...
... In Equation 6, i from 1 to 4 denotes the four damage types of crack, spalling, exposed reinforcement, and corrosion, and ij denotes the j th damage prediction box for the i th damage in the images. Since the four kinds of damages often appear in different periods during the service of the underwater concrete structures, it is necessary to limit t i within a range, so that the initial value t i0 is distinguished, and the final value converges to the upper threshold. ...
Article
Full-text available
Large-scale complex underwater concrete structures have structural damage and the traditional damage detection method mostly uses manual identification, which is inaccurate and inefficient. Therefore, robotic detection systems have been proposed to replace manual identification for underwater concrete structures in ocean engineering. However, the highly corrosive and disruptive environment of the ocean poses great difficulties for the application. Here, we develop a manta ray-inspired underwater robot with well controllability to establish the damage datasets of underwater concrete structures, proposing the YOLOX-DG algorithm to improve the damage detection accuracy, and integrating the model into the robotic detection systems for underwater concrete damages. Eventually, the system is used for ocean testing in real applications (i.e., underwater marine harbors around the East China Sea), and satisfactory detection performance is obtained. The reported manta ray-inspired robotic detection system can be used to accurately monitor and analyze the underwater regions.
... It goes on by documenting the defects with an agreed format for later exchange or analytics, which is finally followed by rehabilitation work to alleviate the impacts of the defects [7]. This "inspect-document-rehabilitate (IDR)" workflow has long been in place and provides a useful guiding framework to plan maintenance work [8]. However, given the immense size of the built environment, its thorough implementation is daunting. ...
Article
Full-text available
The built environment is subject to various defects as it ages. A well-maintained built environment depends on surveying activities to inspect, document, and rehabilitate the defects that occurred. The advancement of digital technologies paves the pathway towards (1) comprehensive defect inspection by systematic mapping, (2) their consistent documentation by digital modeling, and (3) timely retrofitting by proactive management. However, the three steps of defect mapping, modeling, and management (D3M) remain largely fragmented and have yet to be synergized. Exploiting the pivotal role of building information modeling (BIM) in built asset management, this paper puts forward a cohesive framework for integrated D3M. It leverages the rich geometric-semantic information in BIM to assist defect mapping and enriches the BIM by industry foundation classes (IFCs)-represented defect information. The defect-enriched BIM facilitates defect management in a data-driven manner. The framework was applied in multiple real-life infrastructure and civil works projects. It demonstrates how the BIM-based D3M framework can enhance the maintenance of those that have been built, and ultimately contribute to a safe and sustainable built environment. Future studies are called for to substantiate each of the 3Ms by leveraging BIM as both an enabler and a beneficiary.
... These state-of-the-art approaches differ in their selection of NDT methods, the method of locomotion (vortex, wheels, air pads, etc.), and the type of navigation. Locomotion limitations can reduce the application area to certain structural parts [16][17][18][19][20]. For example, wheeled robots, which are limited to inspecting horizontally flat surfaces (e.g., bridge decks) with several NDT methods, are described in [21,22]. ...
Article
This study presents an autonomous inspection system for underground pits using an articulated mobile robot. The underground pit is composed of several rooms surrounded by concrete connected to each other by winding pipes. Based on an action list created in advance and environmental maps, the robot autonomously inspects the underground pit by switching between three actions: planar motion, winding pipe passing motion, and image capturing. In planar motion, the robot moves around the room while avoiding obstacles and crosses ditches through distinctive behaviors, switching the allocation of the grounded/ungrounded wheels. In the winding pipe-passing motion, the target path is autonomously generated based on the parameters of the winding pipe. Laboratory and field tests were conducted to demonstrate the effectiveness of the proposed system.
Article
This paper presents a simple, low-cost teleoperation system. The leader device is a handheld camera integrated with an Inertial Measurement Unit (IMU), making it feasible to use a modern smartphone for this purpose. Existing leader devices require hardware and sensors to measure both the user interactions, and to control the follower device. By contrast, the proposed method uses the handheld camera both as a leader device and as a sensor to control the position of the follower device through visual servoing. To the best of the author's knowledge, this visual servoing scenario where the camera is held by a user has not been thoroughly studied. The measurements from the handheld device and the follower are fused together in an Extended Kalman Filter (EKF) to improve further the pose estimation. A Virtual Camera and IMU (VCI) concept is introduced to filter hand tremors for teleoperation efficiency without hindering the bandwidth of the relative pose control loop. The EKF and the VCI performance are assessed experimentally by teleoperating a Suspended Aerial Manipulator (AMES) prototype.
Article
Background The food manufacturing industry faces increasing challenges such as escalating demands for quality and traceability, labor shortages, and rising operational costs. These issues necessitate innovative solutions to enhance productivity, ensure product quality, and maintain competitiveness. Scope and approach Thiswork examines the integration of robotic technology in food manufacturing, focusing on current technologies, practical applications through case studies, and an analysis of emerging trends. The study aims to assess how robotics can address the industry's critical challenges, exploring its role in improving efficiency, ensuring product safety, and contributing to sustainable practices. Key findings and conclusions Robotic systems offer unprecedented precision and consistency, automating complex tasks from ingredient handling to packaging. By compensating for labor shortages and elevating product quality, these systems set new safety standards. With AI and IoT integration, robotics enhance real-time monitoring and data analytics, thereby boosting traceability and regulatory compliance across global supply chains. Furthermore, by optimizing resource use and reducing waste, robotics drive sustainability. This study underscores the need for stakeholders and policymakers to promote robotic adoption through targeted workforce training, supportive regulations, and technology incentives. To secure the future of food manufacturing, strategic integration of robotics is essential, aligning technological advances with traditional practices to effectively meet global food demands. Keywords Robotic automationFood manufacturingQuality assuranceSupply chain traceabilitySustainability in productionTechnological innovation
Article
The metro is susceptible to disruption risks and requires a system response capability to build resilience to manage disruptions. Achieving such resilient response state requires readiness in both the technology side, e.g., utilizing digital technologies (DTs) to monitor system components, and the human factors side, e.g., fostering positive human coping capabilities; however, these two sides are usually considered independently, without sufficient integration. This paper aims to develop and empirically test a model in which monitoring-enabled DTs, employees' reactions, and their positive capabilities are simultaneously considered in terms of their interplay and impact on system response capability. The results showed that while DTs for monitoring physical components enhanced perceived management commitment and fostered collective efficacy, DTs for monitoring human components increased psychological strain and inhibited improvisation capability, creating a "double-edged sword" effect on system response capability. Additionally, explicit management commitment buffered the adverse effect of DTs-induced psychological strain on individual improvisation.
Article
Full-text available
Accurate modeling and simulation of underwater vehicles is essential for autonomous control. In this paper, we present a dynamic model of the VideoRay Pro III microROV, in which the hydrodynamic derivatives are determined both theoretically and experimentally, based on the assumption that the motions in different directions are decoupled. The experiments show that this assumption is reasonable within operating conditions of the VideoRay Pro III. A computer simulation with 3D graphics is also developed to help user to visualize the vehicle's motion.
Article
Full-text available
CREA robot is designed to climb up concrete walls. The robot uses the suction mechanism to provide adhesion and wheel mechanism for locomotion. Eleven chambers which are connected to one common reservoir are responsible to produce adhesion force. A controller is developed to independently control each chamber while satisfying certain criteria on the safety of the robot. It is also designed to reach minimum friction between active inflatable seals and wall. In conclusion, the controller is able to successfully meet the conditions of stability, minimum friction and safety.
Article
Full-text available
This paper reports on the investigations made at Fraunhofer Institute for Non-Destructive Testing (IZFP) where different rotary wing micro UAS have been used to scan infrastructures including bridges and monuments at high resolutions for remote damage assessment and monitoring purposes. The aerial pictures taken at high speed and frequency have then been stitched together to obtain full 2D and 3D building reconstructions at a resolution allowing damages and cracking to be observed still in the millimeter range. With these ultra hi-res building reconstruction models a specific data base could be created for each object in order to provide extensive information for long term evaluation and life cycle management. The UAS also have been equipped with sensors for damage size estimation, which combined with an image processing software developed to allow automatic cracking pattern recognition could be used for further analysis.
Article
Full-text available
Infrastructure is subject to continuous aging. This has given life-cycle management of the infrastructure an increasing role. Therefore, reliable inspection and monitoring tools are in increasing demand. The paper provides a short overview of the major activities in this area in Europe, followed by a presentation of current projects in the field of civil engineering in which topics related to combinations of nondestructive testing (NDT) methods and modular control and data acquisition approaches are discussed. In this paper, the two major construction materials in civil engineering, steel and concrete, are discussed. Regarding reinforced as well as prestressed concrete, a combination of various NDT methods is often required, which allow reliable results for material characterization, flaw detection, and the determination of component-specific geometry parameters. Therefore, a multisensor measurement approach with a high degree of automation is required. Two recently developed measurement devices (OSSCAR and BetoScan) will be described. Regarding steel constructions, an example from the field of pipeline inspection is shown where the combination of NDT with fracture mechanical assessment is directly possible. Both fields deliver the required interfaces in which the fracture risk is evaluated in probabilistic terms. Therefore, this procedure can be used as a tool for inspection-based lifetime conceptions. The paper concludes with an outlook on future trends for degradation monitoring. DOI: 10.1061/(ASCE)EM.1943-7889.0000317. (C) 2013 American Society of Civil Engineers.
Article
Full-text available
This paper describes the use of robotic and other mobile sensor systems for adaptive structural health monitoring. Sensing parameters on large structures poses many challenges due to size, geometry, and the ability to measure important events and parameters in a timely manner. Damage is often localized, can occur in unexpected places and at unexpected times. Mobile robotic and deployable sensor systems offer the opportunity to adapt to structural changes and events as they emerge and to measure where it is impractical or dangerous to humans. Some of the design issues for mobile sensor systems are discussed. Results from a study of four different beam crawling robots will be presented.
Article
Full-text available
Accurately mapping the course and vegetation along a river is challenging, since overhanging trees block GPS at ground level and occlude the shore line when viewed from higher altitudes. We present a multimodal perception system for the active exploration and mapping of a river from a small rotorcraft. We describe three key components that use computer vision, laser scanning, inertial sensing and intermittant GPS to estimate the motion of the rotorcraft, detect the river without a prior map, and create a 3D map of the riverine environment. Our hardware and software approach is cognizant of the need to perform multi-kilometer missions below tree level with size, weight and power constraints. We present experimental results along a 2 km loop of river using a surrogate perception payload. Overall we can build an accurate 3D obstacle map and a 2D map of the river course and width from light onboard sensing.
Article
Full-text available
In recent years, there has been an increasing interest in the adoption of emerging sensing technologies for instrumentation within a variety of structural systems. Wire- less sensors and sensor networks are emerging as sensing paradigms that the structural engineering field has begun to consider as substitutes for traditional tethered monitoring systems. A benefit of wireless structural monitoring systems is that they are inexpensive to install because extensive wir- ing is no longer required between sensors and the data acquisition system. Researchers are discovering that wire- less sensors are an exciting technology that should not be viewed as simply a substitute for traditional tethered monitor- ing systems. Rather, wireless sensors can play greater roles in the processing of structural response data; this feature can be utilized to screen data for signs of structural damage. Also, wireless sensors have limitations that require novel system architectures and modes of operation. This paper is intended to serve as a summary review of the collective experience the structural engineering community has gained from the use of wireless sensors and sensor networks for monitoring structural performance and health.
Article
Full-text available
This research investigates the field performance of flexure-based mobile sensing nodes (FMSNs) developed for system identification and condition monitoring of civil structures. Each FMSN consists of a tetherless magnetic wall-climbing robot capable of navigating on steel structures, measuring structural vibrations, processing measurement data, and wirelessly communicating information. The flexible body design of the FMSN allows it to negotiate with sharp corners on a structure, and attach/detach an accelerometer onto/from structural surface. Our previous research investigated the performance of the FMSNs through laboratory experiments. The FMSNs were deployed to identify minor structural damage, illustrating a high sensitivity in damage detection enabled by flexible mobile deployment. This paper investigates the field performance of the FMSNs with a pedestrian bridge on the Georgia Tech campus. Multiple FMSNs navigate to different sections of the steel bridge and measure structural vibrations at high spatial resolution. Using data collected by a small number of FMSNs, detailed modal characteristics of the bridge are identified. A finite element (FE) model for the bridge is constructed. The FE model is updated based on the modal characteristics extracted from the FMSN data.
Article
Full-text available
Purpose The aim of this research is to design a wall climbing robot (WCR) for the non‐destructive inspection (NDT) of the above‐ground storage tanks (ASTs) autonomously making the industrial inspection and maintenance tasks safer. Design/methodology/approach A WCR is designed that can be equipped with any NDT sensor. It uses permanent magnets as an adhesion mechanism to crawl over the steel tank walls. A surface coverage algorithm is proposed for the WCR to scan the AST wall surfaces autonomously with the NDT sensors to perform the necessary inspection tasks. Findings The proposed surface coverage algorithm performs the complete coverage of the AST walls under different obstacle configurations. It has been tested and demonstrated in simulations. Originality/value A surface coverage algorithm is proposed for the WCR to perform the non‐destructive inspection of the ASTs autonomously. It can also be used in applications like cleaning glass building and painting ship hulls, etc.
Article
Full-text available
This article describes an unprecedented alternative to manual procedures for the application of advanced composite materials, such as Fiber Reinforced Polymer (FRP) and epoxy resins. A complete mobile integrated system is presented for the inspection and maintenance of concrete surfaces in tunnels. It allows performance of operations with minimum interference on passing traffic. The core of this system resides in a specially designed light-weight robotic tool, which is sensed and automated for processes. Sensing includes vision and a laser telemeter to assure precise inspection, superficial preparation, and composite application. The designed interconnection flange allows simple and robust attachment of the tool to a robotic arm’s tip. The robot-tool set is to be mounted on a standard articulated lift platform. Therefore, an operator can direct the platform and the robot-tool set’s operations from a control station placed at ground-level, in a wheeled vehicle on which the articulated lift platform is mounted. A graphical Human-Machine Interface (HMI) has been developed for the system. It allows the operator to identify fissures for the injection of epoxy resin, and weakened surfaces for FRP adhesion. Actual procedures are planned and performed by the system’s automatic components.
Article
Full-text available
RGB-D cameras provide both a color image and per-pixel depth esti-mates. The richness of their data and the recent development of low-cost sensors have combined to present an attractive opportunity for mobile robotics research. In this paper, we describe a system for visual odometry and mapping using an RGB-D camera, and its application to autonomous flight. By leveraging results from recent state-of-the-art algorithms and hardware, our system enables 3D flight in cluttered environments using only onboard sensor data. All computation and sensing required for local position control are performed onboard the vehicle, eliminating its depen-dence on unreliable wireless links. We evaluate the effectiveness of our system for stabilizing and controlling a quadrotor micro-air vehicle, demonstrate its use for constructing detailed 3D maps of an indoor environment, and discuss its limitations.
Article
Full-text available
Early and accurate detection, location and assessment of damage in reinforced concrete bridge decks may be beneficial in the scheduling and performance of maintenance and rehabilitation activities. This article presents the results of a multiple sensor study of the condition of the reinforced concrete deck of the Van Buren Road Bridge in Dumfries, VA. The tests compared the following five different methods: (1) visual inspection and photographic recording; (2) half-cell electrochemical potential; (3) impulse type multipoint scanning ground penetrating radar; (4) chain drag and (5) impact echo. The bridge was tested on two separate occasions. The results of the tests were that each instrument nominally performed and collected data as expected, but that the condition assessments did not necessarily agree. The data are registered, overlaid and compared. The potential for developing automated multisensor systems that fuse data for efficient and effective bridge deck measurements is discussed.
Article
Full-text available
Structural sensing systems that employ adaptive strategies have the potential to achieve superior performance over nonadaptive counterparts. Adaptation is a means that many systems use to improve performance to compensate for varying and uncertain circumstances. Structural sensing systems can use several different adaptive sensing techniques including adaptive signal processing, adaptive data acquisition protocols, rapidly deployable sensors and mobile sensors. Adaptive signal processing is where the processing that is applied to a given set of data is modified according to the information content in the data or external circumstances. Adaptive data acquisition can be implemented through modifying the acquisition parameters and through the modification of sensor location and spatial sensitivity. The adaptive deployment of sensors requires mobile sensors and mechanisms, such as robots, to move the sensors into more advantageous positions. While the enabling technologies of adaptive sensing systems are readily emerging, optimal and effective adaptive strategies remain largely unexplored. This paper will also discuss some of the issues related to developing effective adaptation strategies for sensing systems and present example applications of adaptive signal processing and mobile robotic sensing applications.
Article
Full-text available
Wireless sensor networks (WSNs) for structural health monitoring (SHM) applications can provide the data collection necessary for rapid structural assessment after an event such as a natural disaster puts the reliability of civil infrastructure in question. Technical challenges affecting deployment of such a network include ensuring power is maintained at the sensor nodes, reducing installation and maintenance costs, and automating the collection and analysis of data provided by a wireless sensor network. In this work, a new "mobile host" WSN paradigm is presented. This architecture utilizes nodes that are deployed without resident power. The associated sensors operate on a mechanical memory principle. A mobile host, such as a robot or unmanned aerial vehicle, is used on an as-needed basis to charge the node by wireless power delivery and subsequently retrieve the data by wireless interrogation. The mobile host may be guided in turn to any deployed node that requires interrogation. The contribution of this work is the first field demonstration of a mobile host wireless sensor network. The sensor node, referred to as THINNER, capable of collecting data wirelessly in the absence of electrical power was developed. A peak displacement sensor capable of interfacing with the THINNER sensor node was also designed and tested. A wireless energy delivery package capable of being carried by an airborne mobile host was developed. Finally, the system engineering required to implement the overall sensor network was carried out. The field demonstration took place on an out-of-service, full-scale bridge near Truth-or-Consequences, NM.
Article
Full-text available
This paper describes the current development status of a mobile robot designed to inspect the outer surface of large oil ship hulls and floating production storage and offloading platforms. These vessels require a detailed inspection program, using several nondestructive testing techniques. A robotic crawler designed to perform such inspections is presented here. Locomotion over the hull is provided through magnetic tracks, and the system is controlled by two networked PCs and a set of custom hardware devices to drive motors, video cameras, ultrasound, inertial platform, and other devices. Navigation algorithm uses an extended-Kalman-filter (EKF) sensor-fusion formulation, integrating odometry and inertial sensors. It was shown that the inertial navigation errors can be decreased by selecting appropriate Q and R matrices in the EKF formulation.
Conference Paper
Full-text available
The Center for Robot-Assisted Search and Rescue (CRASAR®) deployed the custom Sea-RAI man-portable unmanned surface vehicle and two commercially available underwater vehicles (the autonomous YSI EcoMapper and the tethered VideoRay) for inspection of the Rollover Pass bridge in the Bolivar peninsula of Texas in the aftermath of Hurricane Ike. A preliminary domain analysis with the vehicles identified two key tasks in subsurface bridge inspection (mapping of the debris field and inspecting the bridge footings for scour), three research challenges (navigation under loss of GPS, underwater obstacle avoidance, and stable positioning in high currents without GPS), and possible improvements to human-robot interaction (having additional display units so that mission specialists can view and operate on imagery independently of the operator control unit, incorporating 2-way audio to allow operator and field personnel to communicate while launching or recovering the vehicle, and the inclusion of teleoperation as the backup mode for autonomy). An additional research question is the cooperative use of surface and underwater vehicles.
Article
Full-text available
This paper proposes a new type of low-cost climbing robot which is able to climb up vertical, cylindrical painted iron surfaces by means of eight suction cups, to change its climbing direction, and using an appropriate system, to carry an ultrasonic probe or other non-destructive tests (NDT) equipment to evaluate the thickness and the integrity of the metal in the inspection of storage tanks. The paper presents a discussion of the design and construction of the robot, the solution adopted for the problem of control and a simulation of the kinematic forward problem for a given trajectory. Finally, the authors give the results of various motion trials performed on walls with different inclinations and of different materials to verify the correct functioning of the movement and the adhesion system used.
Conference Paper
Long term operation and maintenance of steam and hydropower driven electric power plants requires periodic inspection of the concrete cooling, containment and impoundment structures to assure safety, and reliability. Currently, inspections of large, vertical concrete structures are performed manually which is time-consuming, costly, and exposes workers to potentially dangerous work environments. To address this inspection technology gap, EPRI researched and evaluated various types of robotic technologies; and developed a crawler device that could be adapted to perform a multitude of concrete inspection tasks. Its payload of advanced NDE instrumentation, computer encoded data and imaging provide unprecedented capabilities to interrogate the interior of concrete structures; and locate and characterize voids, rebar corrosion, and other internal defects. The concrete crawler obviates the need to use scaffolding and supports long-term operation of generating assets by enabling automated, safe, and comprehensive inspections of structures such as cooling towers, hydro dams, and reactor containments.
Article
In this paper, we discuss an intelligent bridge inspection system by using a Robot and IT technology which enables acquiring images of the bridge condition for managing the safety of its structure. The purpose of this study is composed of two parts: 1) the image acquisition of a bridge structure with the application of a vision-based robot, the name of the robot system is U-BIROS(Ubiquitous Bridge Inspection Robot System) which is remote controlled. 2) Development of digital image processing software for assessing condition of the bridge, and measuring the size of any crack captured in the images. The robot arm is designed to operate on the underside of superstructure during inspections, and equipped with vision devices (digital camera, lens, and lights). The domain of a superstructure has been divided into appropriate segments to facilitate the image acquiring task by making its total domain irrelevant to the procedure. The system's software detects, measures the width and length of a crack semi-automatically, and also composes a drawing map of the whole area of the bridge from the crack's data for a user's convenience in determining the status of bridge. Through field experiments, the application of this inspection system with specialty software has proven to be much faster, safer, and reliable than the inspections carried out by the naked eyes in managing safety of the bridges.
Article
Unmanned aerial vehicles (UAVs) allow remote imaging which can be useful in infrastructure condition evaluation. Furthermore, emerging noncontact sensing techniques such as digital imaging correlation (DIC) and other photogrammetric and visual approaches, including simultaneous localization and mapping (SLAM), can compute three-dimensional (3D) coordinates and perform deformation measurements as in the case of DIC/photogrammetry. A quantitative assessment of ways remote sensing in conjunction with UAVs could be implemented in practical applications is critically needed to leverage such capabilities in structural health monitoring (SHM). A comparative investigation of the remote sensing capabilities of a commercially availabl e UAV, as well as both an optical metrology system known by the acronym TRITOP and the X-Box Kinect, is presented in this paper. The evidence provided demonstrates that red-green-blue cameras on UAVs could detect, from varying distances, cracks of sizes comparable to those currently sought in visual inspections. In addition, mechanical tests were performed on representative bridge structural components to attempt, for the first time to the writers' best knowledge, deformation measurements using an aerial vehicle; displacements and corresponding accuracies were quantified in static and flying conditions. Finally, an outdoor feasibility test with the UAV was accomplished on a pedestrian bridge to test the marker identification algorithm.
Article
This paper provides a comparative study of two methods for reconstructing three-dimensional (3D) scenes from monocular two-dimensional (2D) images with respect to their applicability to robotic civil infrastructure inspection. The two methods considered are dense structure from motion (DSfM) and image mosaicing (IM). The primary metrics used in the comparison include: (1) the accuracy of the reconstructions for inspection measurement purposes, (2) the nature and prevalence of reconstruction artifacts, and (3) resolution demands for adequate reconstruction accuracy. A series of small-scale model reconstructions using both approaches were developed and compared. Findings from the small-scale tests were then applied to a full-scale field test of the robotic reconstruction system. Both methods generally produced geometrically accurate reconstructions, but the DSfM method was found to be more adaptable to complex field scenarios. There were also key differences in the frequency and nature of artifacts introduced by each method.
Conference Paper
This paper exposes the Navigation and Control technology embedded in a recently commercialized micro Unmanned Air Vehicle (UAV), the AR.Drone, which cost and performance are unprecedented among any commercial product for mass markets. The system relies on state-of-the-art indoor navigation systems combining low-cost inertial sensors, computer vision techniques, sonar, and accounting for aerodynamics models.
Conference Paper
Natural disasters can compromise the structural integrity of buildings. Three dimensional post-disaster building models, created using image-based 3D reconstruction techniques, would help early responders assess the extent of the damage. Image-based 3D reconstructions are advantageous as they only require a camera and tens of pictures to create a 3D mesh model. Under some circumstances responders can collect these images; however, if a building is suspected to have sustained significant damage, it may be neither safe nor straightforward to examine its structural integrity. Therefore, an approach is needed which allows responders to remain away from compromised buildings while gathering images for a 3D model. We propose the use of a small ground robot to take pictures of building elements remotely. Once the images are collected, an element model is created using a pipeline of Structure from Motion, dense reconstruction, and surface modeling. This model is compared to a pre-existing CAD model or examined to assess the likelihood of any structural damage. We also demonstrate a new automatic 3D crack-detection algorithm to assist examiners in identifying structural defects. Our preliminary results show that this process can successfully identify numerous cracks in the 3D reconstruction of a sample concrete column. The perceived benefits of the proposed method in a post-disaster situation are also discussed in detail.
Article
Accurate and rapid assessment of the as-built status on any construction site provides the opportunity to understand the current performance of a project easily and quickly. Rapid project assessment further identifies discrepancies between the as-built and as-planned progress, and facilitates decision making on the necessary remedial actions. Currently, manual visual observations and surveying are the most dominant data capturing techniques but they are time-consuming, error-prone, and infrequent, making quick and reliable decision-making difficult. Therefore, research on new approaches that allow automatic recognition of as-built performance and visualization of construction progress is essential. This paper presents and compares two methods for obtaining point cloud models for detection and visualization of as-built status for construction projects: (1) A new method of automated image-based reconstruction and modeling of the as-built project status using unordered daily construction photo collections through analysis of Structure from Motion (SfM); (2) 3D laser scanning and analysis of the as-built dense point cloud models. These approaches provide robust means for recognition of progress, productivity, and quality on a construction site. In this paper, an overview of the newly developed automated image-based reconstruction approach and exclusive features which distinct it from other image-based or conventional photogrammetric techniques is presented. Subsequently the terrestrial laser scanning approach carried out for reconstruction and comparison of as-built scenes is presented. Finally the accuracy and usability of both of these techniques for metric reconstruction, automated production of point cloud models, 3D CAD shape modeling, and as-built visualizations is evaluated and compared on eight different case studies. It is shown that for precise defect detection or alignment tasks, image-based point cloud models may not be as accurate and dense as laser scanners' point cloud models. Nonetheless image-based point cloud models provide an opportunity to extract as-built semantic information (i.e., progress, productivity, quality and safety) through the content of the images, are easy to use, and do not need add burden on the project management teams by requiring expertise for data collection or analysis. Finally image-based reconstruction automatically provides photo alignment with point cloud models and enables image-based renderings which can remarkably impact automated performance monitoring and as-built visualizations.
Article
The information presented in this report provides a detailed assessment of the condition of the Arlington Memorial Bridge (AMB) deck. The field-data collection was obtained by both the RABIT™ Bridge Inspection Tool and a number of semi-automated non-destructive evaluation (NDE) tools. The deployment of the semi-automated NDE tools was performed to inspect the AMB deck condition and also to validate data obtained by the RABIT™ Bridge Inspection Tool. Data mining and analysis were accomplished through enhanced data interpretation and visualization capabilities using advanced data integration, fusion, and 2D rendering. One of the major challenges that the research team had to overcome in assessing the condition of the AMB deck was the presence of an asphalt overlay on the entire bridge deck.
Conference Paper
This paper introduces a climbing robot for corrosion monitoring of reinforced concrete structures such as cooling towers, dams or bridges. The robot combines a vortex adhesion mechanism with a wheel electrode sensor for potential mapping of the concrete surface. A detailed description of the system is presented first. A special effort was made during the design in order to develop a lightweight device. The climbing robot is well suited for rough surfaces and can climb on vertical surfaces or move upside-down. The experiments that have been done to validate the concept are presented afterwards. They show that the climbing robot has several advantages over the traditional corrosion monitoring technique. This robot will therefore provide engineers in charge of infrastructure maintenance with the means to do their job much better than they can today. It offers them a way to circumvent all present barriers and brings a radical innovation in this area.
Article
In today's economic climate, it is crucial for manufacturing plants to run smoothly and with as little disruption as possible. With regular inspections imperative in maintaining the health and safety of workers, this can be difficult. There is, however, another way of doing things that goes far beyond traditional inspection techniques. By using remotely operated aerial vehicles (ROAVs) tall and hard-to-reach structures can be accessed safely and without disruption. Until now, the primary use for these miniature flying vehicles has been visual inspection of assets using HD video and HD still cameras, where the highly detailed images are ideal for understanding maintenance and repair issues. The ROAV technique leaves plants fully operational during the inspection. There is no disruption to output, which creates a major commercial as a well as a health and safety advantage, as an asset can be inspected live.
Article
A nondestructive testing technique based on magnetic flux leakage is presented to inspect automatically the stay cables with large diameters of a cable-stayed bridge. Using the proposed inspection method, an online nondestructive testing (NDT) modular sensor is developed. The wreath-like sensor is composed of several sensor units that embrace the cable at equal angles. Each sensor unit consists of two permanent magnets and a hall sensor to detect the magnetic flux density. The modular sensor can be installed conveniently on cables with various diameters by increasing the number of sensor units and adjusting the relative distances between adjacent sensor units. Results of the experiments performed on a man-made cable with faults prove that the proposed sensor can inspect the status signals of the inner wires of the cables. To filter the interfering signals, three processing algorithms are discussed, including the moving average method, improved detrending algorithm, and signal processing based on a digital filter. Results show that the developed NDT sensor carried by a cable inspection robot can move along the cable and monitor the state of the stay cables.
Article
The advent of RGB-D cameras which provide synchronized range and video data creates new opportunities for exploiting both sensing modalities for various robotic applications. This paper exploits the strengths of vision and range measurements and develops a novel robust algorithm for localization using RGB-D cameras. We show how correspondences established by matching visual SIFT features can effectively initialize the generalized ICP algorithm as well as demonstrate situations where such initialization is not viable. We propose an adaptive architecture which computes the pose estimate from the most reliable measurements in a given environment and present thorough evaluation of the resulting algorithm against a dataset of RGB-D benchmarks, demonstrating superior or comparable performance in the absence of the global optimization stage. Lastly we demonstrate the proposed algorithm on a challenging indoor dataset and demonstrate improvements where pose estimation from either pure range sensing or vision techniques perform poorly.
Article
This paper describes an automated inspection robot for detecting tile exfoliation and a new diagnostic method for determining its existence and extent. The robot moves quickly along a vertical wall and stops to detect a tile's inner condition using a hammering sound. Tile separation commonly comprises outer exfoliation where the tile separates from the mortar concrete and inner exfoliation where the space between the substrate and the mortar concrete deteriorates. In order to detect these two exfoliations, we focused attention on wavelet analysis, which enables us to analyze the frequency element of the sound waveform on time phase continuously. By comparing the wavelet volume rate expressing the characteristics of tile deterioration for several scale tiles and striking hammers, the quantitative detection and its scale effects of visually distinguishing the two exfoliation modes was established. The automated robot and the diagnostics method were used to perform a fast and highly accurate inspection of the outer tile wall.
Article
Inspecting and monitoring oil-gas pipelines, roads, rivers, and canals are very important in ensuring the reliability and life expectancy of these civil systems. An autonomous unmanned aerial vehicle (UAV) can decrease the operational costs, expedite the monitoring process, and be used in situations where a manned inspection is not possible. This paper addresses the problem of monitoring these systems using an autonomous UAV based on visual feedback. A single structure detection algorithm that can identify and localize various structures including highways, roads, and canals is presented in the paper. A fast learning algorithm that requires minimal supervision is applied to obtain detection parameters. The real time detection algorithm runs at 5Hz or more with the onboard video collected by the UAV. Both hardware simulations and flight results of the vision-based control algorithm are presented in this paper. A fixed wing UAV equipped with a camera onboard was able to track a 700m canal based on vision several times with an average cross-track error of around 10m.
Article
A mobile manipulator imaging system is developed for the automation of bridge crack inspection. During bridge safety inspections, an eyesight inspection is made for preliminary evaluation and screening before a more precise inspection. The inspection for cracks is an important part of the preliminary evaluation. Currently, the inspectors must stand on the platform of a bridge inspection vehicle or a temporarily erected scaffolding to examine the underside of a bridge. However, such a procedure is risky. To help automate the bridge crack inspection process, we installed two CCD cameras and a four-axis manipulator system on a mobile vehicle. The parallel cameras are used to detect cracks. The manipulator system is equipped with binocular Charge Coupled Devices (CCD) for examining structures that may not be accessible to the eye. The system also reduces the danger of accidents to the human inspectors. The manipulator system consists of four arms. Balance weights are placed at the ends of Arms 2 and 4, respectively, to maintain the center of gravity during operation. Mechanically, Arms 2 and 4 can revolve smoothly. Experiments indicated that the system could be useful for bridge crack inspections.
Article
Serpentine robots are snake like devices that can use their internal degrees of freedom to thread through tightly packed volumes accessing locations that people or conventional machinery cannot. These devices are ideally suited for minimally invasive inspection tasks where the surrounding areas do not have to be disturbed. Applications for these devices are therefore inspection of underground tanks and other storage facilities for classification purposes. This work deals with the design, construction, and control of a serpentine robot. The challenges lie in developing a device that can lift itself in three dimensions, which is necessary for the inspection tasks. The other challenge in control deals with coordinating all of the internal degrees of freedom to exact purposeful motion.
Article
Routine inspection is the most common form of highway bridge inspection to satisfy the requirements of the National Bridge Inspection Standards. The accuracy and reliability of documentation generated during these inspections are critical to the allocation of Department of Transportation construction, maintenance, and rehabilitation resources. Routine inspections are typically completed using only the visual inspection technique and rely heavily on subjective assessments made by bridge inspectors. In light of this, and given the fact that visual inspection may have other limitations that influence its reliability, the Federal Highway Administration initiated an investigation to examine the reliability of visual inspection as it is currently applied to bridges in the United States. This paper will summarize results from this study related to the accuracy and reliability of routine inspection documentation. A number of important conclusions were developed from the experimental study. Generally, it was found that all structural condition documentation is collected with significant variability. Specifically, 95% of primary element condition ratings for individual bridge components will vary within two rating points of the average and only 68% will vary within one point. Documentation generally collected to support condition ratings also has significant variability as exemplified by the number and types of field notes and photographs taken by inspectors. With respect to the use of element-level inspections, it was found that element usage was generally consistent with the Commonly Recognized Element Guide. However, there is significant variability in the condition state assignments of those elements and in some cases the condition states are not applied correctly to particular elements.
Chapter
The concept of a flying machine originates in ancient Greece and China, whereas the first modern unmanned aircraft was demonstrated less than 15 years after the first flight by the Wright brothers. For several years these unmanned systems were used as target drones and it wasn’t until the 1950s that the first reconnaissance drones were developed, leading to today’s UAS. Today, a large number of countries are developing an equally large number of different systems for reconnaissance, surveillance, hunter-killer, weather monitoring, earth monitoring and other roles. Besides fixed-wing aircraft, there are also UAS based on helicopter designs, as well as back-packable, shrouded rotor, twin-rotor, lighter-than-air, tiltwing, canard rotor wing and other systems.
Article
The distress survey is an important task for pavement maintenance and rehabilitation (M&R) activities. As distress surveys require tremendous human resources, many investigators have begun to develop automatic inspection methods with the aim of increasing the efficiency and accuracy of inspections. After assessment of distress surveys on pavements using an autonomous robot (P3-AT), this research aims at developing motion strategies for executing distress surveys using robots under project-level practices. Three motion strategies were specifically developed: (1) Strategy I: random survey (R); (2) Strategy II: random survey with map recording (R + M); (3) Strategy III: random survey with map recording and vision guidance (R + M + V). To validate these three strategies, we developed a test field in a virtual environment. The test field included five distress types, including an alligator crack, a small patching, a pothole, a rectangular manhole and a circular manhole. We also developed a virtual robot to navigate the test field autonomously. The three survey strategies were then implemented by the virtual robot and their performances were compared with the current traffic-directional survey strategy.Research Highlights► This research aimed to develop strategies for robotics pavement survey. ► Three strategies, (1) random survey (R strategy), (2) random survey with map recording (R+M strategy) and (3) random survey with map recording and vision guidance (R+M+V strategy), have been developed. ► After testing the strategies in a realistic virtual environment, we found strategies can identify more pavement distresses than traditional survey method does. ► The performance of the strategies rank from the best is R+M+V, R+M and R strategy.
Conference Paper
This paper proposes design and control methods for a bridge inspection robot system. A "bridge inspection robot" has been developed with the aim of checking the safety status of a real bridge, gathering accurate data and performing maintenance. The developed robot system is composed of the specially designed car for bridge inspection, the guide rail and the inspection robot. Especially, this paper emphasizes the system integration method to design and control the entire robot system. As a result, this robot has both automatic and manual modes for inspecting a real bridge. In other words, the robot is automatically able to inspect beneath the bridges and manually to inspect them by long-distance user. Also, the users can see the operating conditions of long-distance robot with 3-dimensional graphical user interface. Moreover, the users can feel the operating status of long-distance robot through haptic device.
Conference Paper
Thousands of storage tanks in oil refineries have to be inspected manually to prevent leakage and/or any other potential catastrophe. A wall climbing robot with permanent magnet adhesion mechanism equipped with nondestructive sensor has been designed. The robot can be operated autonomously or manually. In autonomous mode the robot uses an ingenious coverage algorithm based on distance transform function to navigate itself over the tank surface in a back and forth motion to scan the external wall for the possible faults using sensors without any human intervention. In manual mode the robot can be navigated wirelessly from the ground station to any location of interest. Preliminary experiment has been carried out to test the prototype.
Conference Paper
This paper presents a vision-based method to estimate the * real motion of a single camera from views of a planar patch. Projective techniques allow to estimate camera motion from pixel space apparent motion without explicit 3-D reconstruction. In addition, the paper will present the HELINSPEC project, the framework where the proposed method has been tested, and will detail some applications in external building inspection that make use of the proposed techniques.
Conference Paper
Cables are the most important parts of cable-stayed bridges. Safety of the cables on a bridge is among the most important public concerns. This paper proposes an ameliorated wheel-based cable inspection robotic system of the authors, which is able to climb up and down the cables to detect cable defections of a bridge. The robotic system is consisted of three modules equally spaced circularly, which are joined together by six connecting boards to form a closed hexagonal body for clasping a cable. Each module consists of two wheels for climbing, one CCD camera for visual inspection, 2 pairs of actuating permanent magnets and 5 Holl sensors for the detection magnetic flux leakage. The kinematics and the statics of the robot are detailedly analyzed. The amelioration is the newly designed electric circuit which is employed to limit the descending speed of the robot during its sliding down after finishing detection. The principle of detection the cable defection through magnetic flux leakage is also discussed.
Article
To assess the safety of concrete structures, cracks are periodically measured and recorded by inspectors who observe cracks with their naked eye. However, manual inspection is slow and yields subjective results. Therefore, this study proposes a system for inspecting and measuring cracks in concrete structures to provide objective crack data to be used in evaluating safety. The system consists of a mobile robot system and a crack detection system. The mobile robot system is controlled to maintain a constant distance from walls while acquiring image data with a Charged Couple Device (CCD) camera. The crack detection system extracted crack information from the acquired image using image processing. To ensure accurate crack recognition, the geometric properties and patterns of cracks in a structure were applied to the image processing routine. The proposed system was verified with laboratory and field experiments.
Article
Wheeled-tracked vehicles are mainly suitable for relatively flat terrain. Legged-vehicles, on the other hand, have the potential to handle wide variety of terrain. Robug IIs is a legged climbing robot designed to work in relatively unstructured and rough terrain. It has the capability of walking, climbing vertical surfaces and performing autonomous floor to wall transfer. The sensing technique used in Robug IIs is mainly tactile and ultrasonic sensing. A set of reflexive rules has been developed for the robot to react to the uncertainty of the working environment. The robot also has the intelligence to seek and verify its own foot-holds.
Article
A robotic system for inspecting the safety status of bridges is proposed in this paper. Currently, most bridge inspections have been done manually by counting the number of cracks, measuring their lengths and widths and taking pictures of them. Thus the quality and reliability of diagnosis reports on bridges are greatly dependent upon the diligence and education of inspection workers. The robotic inspection system to be proposed consists of three parts: a specially designed car, a robot mechanism and control system for mobility and a machine vision system for automatic detection of cracks. Ultimately, this robot system has been developed for gathering accurate data in order to record the biennial changes of the bridge's safety circumstances as well as check the safety status of bridges. We also demonstrate the effectiveness of the suggested crack detecting and tracing algorithms through experiments on a real bridge crack inspection.
Article
This paper describes the dynamics of an unmanned aerial vehicle (UAV) for monitoring of structures and maintenance of bridges. It presents a novel control law based on computer vision for quasi-stationary flights above a planar target. The first part of the UAV's mission is the navigation from an initial position to a final position in an unknown 3D environment. The new control law uses the homography matrix computed from the information obtained from the vision system. The control scheme will be derived with backstepping techniques. In order to keep the camera's field of view, the control law uses saturation functions for bounding the UAV orientation and limiting it to very small values.