Conference PaperPDF Available

A Study AR Based Smart Device for Work Management at Plant Construction Sites



Content may be subject to copyright.
35th International Symposium on Automation and Robotics in Construction (ISARC 2018)
A Study AR Based Smart Device for Work Management
at Plant Construction Sites
Kyuhyup Leea , Soonwook Kwon*, Taekyu Koa and Youngsuk Kimb
aDepartment of Convergence Engineering for Future, Sungkyunkwan University, 2066, Seoburo, Jangan-gu,
Suwon-si, Gyeonggi-do, Republic of Korea
*School of Civil & Architectural Engineering, Sungkyunkwan University, 2066, Seoburo, Jangan-gu, Suwon-si,
Gyeonggi-do, Republic of Korea
bDepartment of Architectural Engineering, Inha University, Incheon, 22212, Republic of Korea
One of the difficulties encountered at a plant
construction site is frequent changes to design, which
causes work overload. This can lead to increased
construction costs, delayed progress, inferior work
quality and poor productivity due to redundant work
among project participants or omission of work.
In this study, we propose a work management
system that uses smart devices based on augmented
reality technology, which provides real-time
information on plant construction so that site workers
and managers can reduce wastes and errors, as well
as check maintenance information more easily.
Augmented Reality; Smart-Device; Pipe-assembly;
DBS(Device Breakdown Structure)
1 Introduction
1.1 Research Background and Purpose
A plant construction project is large in scale than
general construction site, and involves complicated
planning and processes. At a plant construction site, work
data are generated for different stages of work, including
architectural work, pipe installation, electricity and other
facilities. During construction, massive amounts of data
are generated, and management of this data is
challenging. Thus, plant construction projects are harder
to implement compared to other construction projects.
Various research has been undertaken for efficient
management of plant construction, including studies on
MIS using QR codes, barcodes, RFID and cloud systems.
However, these technologies entail the problem of
redundant work due to delayed processing of real-time
data and duplicated systems. As a result, construction
time is stretched, and the same work is repeated
unnecessarily. Productivity also declines in the
maintenance and management stage, as a delay in the
resolution of troubled tasks spreads to other related tasks.
In this study, we gather construction site managers’
opinions to identify work processes that need
improvement and design a plant site information
management system accordingly, which can be useful for
site workers and managers. At a plant site, a manager
often works inefficiently due to poor processing of real-
time data and lack of communication between the
company and the site office. Moreover, construction
workers have difficulty communicating with a manager
due to complicated plant design. To address these issues,
we apply an AR (augmented reality)-based smart device
for pipe installation and construction management, which
account for the largest part of a plant project (43%), to
identify how to effectively install pipes and manage
object data.
1.2 The Scope and Methods of Research
As mentioned above, we propose a management
system for plant construction workers and managers that
uses an AR-based smart device. At a plant site, workers
install pipes based on drawings generated prior to the
implementation. However, there is a wide discrepancy
between drawings and the actual site, which makes it
hard to install pipes. Thus, it is important to establish an
information-sharing system between workers and
managers to deliver accurate information on the site
situation when installing pipes. To this end, we propose
using an AR-based smart device.
The method and process of this research is:
(1) Preceding Research Analysis
- Existing review of literature
- Analysis of management characteristic of plant
construction and process
(2) Requirement Analysis and Function Derivation
- Figuring out smart device applicability in the
35th International Symposium on Automation and Robotics in Construction (ISARC 2018)
plant field
- Requirement analysis to solve problems
(3) Establishing DBS (Device Breakdown
Structure) for optimal plant construction
- Process Analysis at Plant Site
- Analysis of maintenance Work at Plant Site
- DBS Proposal by Different Work Processes
(4) AR Technology at Plant Construction Site
2 Research Trend
2.1 Literature Review
Since the mid-2000s, active research has been
undertaken to identify how to use various IT technologies
to improve plant construction, specifically, to shorten
construction time, reduce costs, ensure flawless
construction, engage in accident-proof planning, enhance
work predictability, eliminate wastes, improve
productivity, and reduce maintenance costs.
Most studies on plant construction and management
have been limited to proposing conceptual
methodologies or solving logistics issues (e.g. delivery of
materials). There are few studies on how to generate and
utilize real-time information on pipe work or how to
discover errors.
Table 1 Existing research
Area of
Research subjects
Ontology-based BIM modeling;
AR-based system framework
4D tools for greater efficiency in
site management
Use of smart phone to improve
management process; visualization
of project information using real-
time data sharing and management,
wireless communication; and
augmented reality
New 4D safety management and
monitoring system ‘C-RTICS2’ for
more efficient construction and
communication among work
Behavior-based preventive safety
system, visualized management
with VCS (Virtual Construction
Simulation System)
Marker-based AR: Research on
identification of objects, selection of
marker type, marker detection, and
marker-less AR technology
Site condition
Wearable device for real-time work
Wearable device; Cloud system for
massive data management
Set apart in terms of device and
visualization method, while similar in
technology for communication between
office and construction site
Partly applicable to safety management
in terms of carrying out preventive
safety management in visualized form
2.2 Plant Construction Management
Figure 1. Pipe construction process
At a plant construction site, pipe installation can be
divided into three stages: pre-installation, installation and
In the first stage, materials are delivered and detailed
2D drawings are produced, based on which the site is
surveyed, and 3D drawings are produced.
In the installation stage, workers install pipes based on
35th International Symposium on Automation and Robotics in Construction (ISARC 2018)
the drawings. In the after-installation stage, managers
and workers carry out maintenance work.
In this process, workers and managers do not fully
share information, and it is hard for a manager to
accurately explain the installation process to workers
based on drawings. To resolve this difficulty, we propose
a new process and system.
3 Analysis of plant workers’ needs
Moon [16] conducted an interview with and surveyed
the project sites of S company, one of the country’s
leading construction companies, and found out that
workers needed smart applications for construction
assistance, data confirmation and checks and guidelines,
particularly for communication support and progress
management (Figure 2.).
He showed that, at plant construction sites, workers
need to have a smart device system to share information
during the design and implementation processes.
Table 2 Applicable work territories
Construction Assistance App.
Data confirmation and check App.
Data input App.
Guideline App.
Communication support App.
Progress management App.
At a plant site, workers need drawings and related
data to handle materials and make necessary installations.
Figure 2. Overall result
These data are generated in different forms and
communicated to workers in visualized forms. The
alteration of certain data affects other data and generates
new information. Using smart devices and applications
could support this process
4 Establishing DBS(Device Breakdown
Structure) for optimal plant
4.1 Process Analysis at Plant Site
Figure 3. Construction work process of plant site
We analyzed the construction process at a plant site
to identify areas of work in which the use of a smart
device can be applied. The analysis was done for earth
work and architectural work, specifically for processes
such as architectural work and electricity, pipe and
facility installation.
The analysis results showed that, for earth work, it is
important to convert materials data and drawings into a
3D model to help workers’ understanding of the process.
For architectural work, it is important to produce
information on materials and processes that are in
progress or completed, and to provide a 3D model based
on augmented reality for review.
35th International Symposium on Automation and Robotics in Construction (ISARC 2018)
Figure 4. Smart device & IT convergence
technology in plant site
4.2 Analysis of maintenance Work at Plant
The analysis results showed that facilities
management companies, inspectors and facilities
managers need a maintenance system that uses an AR-
based smart device to inspect, maintain and repair plant
More specifically, tasks that could use an AR-based
smart device included planning and review of facilities
maintenance, updating drawings/records for the facilities
management company, managing records and site
locations confirming maintenance results for inspectors
and conducting site check and maintenance work for
facilities managers. Smart devices can be used to track
records of materials and visualize related information.
Figure 5. Maintenance process of plant site
4.3 DBS by Different Work Processes
Based on interviews and analysis of plant processes,
we identified smart devices suitable for various
participants of a plant construction project, that is, plant
constructors, architectural workers, facilities supervisors
and facilities managers. Then we examined various
functions of smart wearable devices that can be used to
process data generated by site workers.
Our investigation showed that watch-type, band-type,
and glass-type devices are suitable for a plant
construction project. As for device sensors, bio
information, GPS/NFC, accelerometer, visual
information and Bluetooth are applicable to plant
construction. We matched site workers’ needs with
different types and functions of smart devices so that the
workers could choose an efficient device.
35th International Symposium on Automation and Robotics in Construction (ISARC 2018)
Figure 6. Device breakdown structure
5 AR Technology at Plant Construction
Figure 7. Augmented Reality System Diagram
To apply AR technology at a plant construction site,
we conducted an experiment (See Figure 7). First, image
input was made to use marker. Second, to check that
images were recognized as markers by the smart device.
If the image was recognized as a marker, save it in DB.
or if not, input the image again. Third, image data of
useful information were collected to generate an AR
marker file. Fourth, the stored image data is used to
improve the object recognition rate through machine
learning techniques. Fifth, the marker was linked with 3D
modelling/images, and sixth, the position of the object
was set on a smart device display. Lastly, AR data were
generated accordingly for use at a plant site
Through this process, the worker uses the camera of
the smart device at the plant construction site to
recognize objects from various angles as markers, and
links objects and 3D modeling information.
35th International Symposium on Automation and Robotics in Construction (ISARC 2018)
Figure 8. Images for using marker
6 Conclusions
In this study, we analyzed traditional processes of
plant construction, conducted interviews to identify
difficulties and problems at plant construction sites, and
explored the applicability of smart device at plant sites.
To this end, we analyzed work processes at plant sites,
examined maintenance work, established DBS (Device
Breakdown Structure) for various participants of a plant
construction project and examined how to apply AR
Regarding work processes at a plant site, we
examined different stages of a construction project and
identified technological elements that can be applied to
earth work and architectural work. For analysis of
maintenance work, we identified areas of work that need
visualized information. Additionally, we established
DBS for site participants and examined how to adopt
marker-based AR technology at a plant construction site.
Based on the analysis of plant construction processes
and maintenance work, smart devices can be adopted at
a plant construction site according to DBS of different
work areas, and AR data can be generated through
marker-based image links. Using these data, workers can
check information and acquire better understanding of
the work process. Moreover, a Cloud system may be
introduced for real-time information sharing. Based on
these findings, we plan to conduct further research on
marker-less image tracking using deep-learning
This work is financially supported by Korea Ministry
of Land, Infrastructure and Transport(MOLIT) as
Smart City Master and Doctor Course Grant Program
"This work was supported by the National Research
Foundation of Korea(NRF) grant funded by the Korea
government(MSIP) (No. NRF-
This research was supported by a grant(17CTAP-
C133224-01) from Infrastructure and transportation
technology promotion research program funded by
Ministry of Land, Infrastructure and Transport of Korean
[1] Yeo, U. Y., Yoon, Y. S., Suh, S. W., and Lim, B. W.
Improvement of Construction Management for
Building Remodeling Projects. Korean Journal of
Construction Engineering and Management, 5(4):
122-129, 2004.
[2] Tang, P., Huber, D., Akinci, B., Lipman, R., and
Lytle, A. Automatic reconstruction of as-built
building information models from laser-scanned
point clouds: A review of related
techniques. Automation in construction, 19(7):
829-843, 2010.
[3] Lee, Augmented reality-based computational
fieldwork support for equipment operations and
maintenance, Automation in Construction 20, 2011
[4] Journal of Information Technology in Construction
2011, 259278
[5] Y.-C. Chen, H.-L. Chi, et al., Use of tangible and
augmented reality models in engineering graphics
courses, in: Journal of Professional Issues in
Engineering Education & Practice, 137, American
Society of Civil Engineers, ASCE, 2011, pp. 267
[6] Park J. K. Safety Management Information System
in Plants Construction Work. The Journal of Korea
Safety Management & Science, 14(4):23-29, 2012.
[7] Chen Y. and Kamara John M. Using Mobile
Computing for Construction Site Information
Management. Engineering, Construction and
Architectural Management, 15(1):7-20, 2008.
[8] Lertlakkhanakul J. and Han S. Y. Construction
Management System Using the Mobile Augmented
Reality Techniques. Journal of the Architectural
Institute of Korea, 22(2):139-145, 2012.
[9] Moon, Improved Method for Increasing
Maintenance Efficiency of Construction Structure
Using Augmented Reality by Marker-Less Method,
35th International Symposium on Automation and Robotics in Construction (ISARC 2018)
Journal of the Korean Society of Civil Engineers
Vol. 35, No. 4: 961-968
[10] Song Y. W. and Cho H. M. Information Breakdown
Structure of Engineering Phase for Plant Project
through Business Analysis. Korean Journal of
Construction Engineering and Management,
10(5):3-15, 2009.
[11] Hyojoon Bae, and Mani Golparvar-Fard. High-
precision vision-based mobile augmented reality
system for context-aware architectural, engineering,
construction and facility management (AEC/FM)
applications. Visualization in Engineering, 1(1):1-
13, 2013.
[12] Hyojoon Bae, and Mani Golparvar-Fard. nhanced
HD4AR (hybrid 4-dimensional augmented reality)
for ubiquitous context-aware AEC/FM applications.
In Proceedings of 12th international conference on
construction applications of virtual reality (CONVR
2012), pp. 253-26, 2012.
[13] Hyojoon Bae, and Mani Golparvar-Fard. High-
precision and infrastructure-independent mobile
augmented reality system for context-aware
construction and facility management applications.
In Proc., 2013 ASCE Int. Workshop on Computing
in Civil Eng, pp. 637-644, 2013.
[14] Koch C. Matthias N. Markus K. and Michael A.
Natural markers for augmented reality-based indoor
navigation and facility maintenance. Automation in
Construction 48: 18-30, 2014.
[15] R. Akhavian, A.H. Behzadan, Construction
equipment activity recognition for simulation input
modeling using mobile sensors and machine
learning classifiers, Adv.Eng. Inform. 29 (4) (2015)
[16] D.Y. Moon and S.W. Kwon and T. Bock and H.L
Ko, Augmented Reality- Based On-site Pipe
Assembly Process Management Using Smart
Glasses, International Symposium on Automation
and Robotics in Construction, 2015
[17] M. D'Orazio, S. Longhi, P. Olivetti, G. Bernardini,
Design and experimental evaluation of an
interactive system for pre-movement time reduction
in case of fire, Autom. Constr. 52 (2015) 1628
[18] Hongling Guo, Yantao Yua, Martin Skitmore,
Visualization technology-based construction safety
management: A review, Automation in
Construction 73 (2017) 135144
[19] Kevin K. Han, Potential of big visual data and
building information modeling for construction
performance analytics: An exploratory study,
Automation in Construction 73 (2017) 184-198
... The use of AR technology in the construction industry has been slow to develop and the current industry use is focused on the following areas: virtual site visits, collaboration opportunity enhancement, project planning and training, monitoring construction progress and managing construction activities (Rankohi and Waugh, 2013), construction simulation (Behzadan et al., 2015), assembly guidance (Shin and Dunston, 2008) and operational training of heavy equipment (Wang and Dunston, 2007). Lee et al. (2018) also proposed an augmented reality-based work management system that generates AR data and shares construction information in real-time, effectively improving pipe installation and construction management, reducing waste and errors. The implementation of AR has been proven in practice to bring considerable benefits to the construction industry. ...
Conference Paper
Full-text available
The adoption of innovative technologies in the construction industry has provided significant benefits to the industry. In Australia, the use of innovative construction technologies has grown increasingly over the past decade and received a significant increase in the level of adoption in construction site activities due to the recent pandemic. One technology that is receiving additional attention to be beneficial to the industry is Augmented Reality (AR). However, its current level of adoption is still very limited due to several reasons the theoretically advantages are not yet well received by the practitioners. To address this gap, this study aimed at investigating the low-level adoption and widespread use of AR technology in the Australian construction industry. To this end, semi-structured interviews were conducted to explore perceptions of the use of AR technology and its limitations from end-users perspectives and managers of construction companies in particular. As a result, a model is presented to map the influential factors and barriers to the adoption of AR. Findings revealed several concepts including lack of knowledge about AR, conservative management attitudes, absence of feasibility analysis, operability and work culture as factors influencing AR adoption in the industry. Moreover, the model provides a basis to direct future research on the topic.
Augmented Reality (AR) is fast becoming an established tool for the construction industry. Previous research reports on the conversion of BIM geometric models and the implementation of these with marker-based AR, or the use of more wide area AR taking positional input from GPS. Much of this focused on the use of AR in an individual context, so there is need to align AR with the more collaborative nature of BIM. By implementing marker-based AR, and connecting to a cloud-based database, the presented BIM-AR system provides the ability to view, interact and collaborate with 3D and 2D BIM data via AR with geographically dispersed teams. An Agile Scrum Method was used to develop the prototype system including a mobile AR application and a Large Touch Screen application based on and a Model, View, Controller (MVC) approach. Finally, the system was tested and verified using a focus group of construction practitioners.
Full-text available
Construction safety management has been a popular issue in research and practice in recent years due to the high accident and death rates in the construction industry. The complexity and variability of construction sites makes safety management more difficult to implement than in other industries. As a promising technology, visualization has been extensively explored to aid construction safety management. However, a comprehensive critical review of the visualization technology in construction safety management is absent in the literature.
Full-text available
The longest phase in a facility's lifecycle is its maintenance period, during which operators perform activities to provide a comfortable living and working environment as well as to upkeep equipment to prevent functional failures. In current practice operators need a considerable amount of time to manually process dispersed and unformatted facility information to perform an actual task. Existing research approaches rely on expensive hardware infrastructure or use artificial, thus unesthetic Augmented Reality (AR) markers. In this paper we present a natural marker based AR framework that can digitally support facility maintenance (FM) operators when navigating to the FM item of interest and when actually performing the maintenance and repair actions. Marker detection performance experiments and case studies on our university campus indicate the feasibility and potential of natural markers for AR-based maintenance support.
Full-text available
Engineering graphics courses are typically a requirement for engineering students around the world. Besides understanding and depicting graphic representation of engineering objects, the goal of these courses is to provide students with an understanding of the relationship between three-dimensional (3D) objects and their projections. However, in the classroom, where time is limited, it is very difficult to explain 3D geometry using only drawings on paper or at the blackboard. The research presented herein aims to develop two teaching aids; a tangible model and an augmented reality (AR) model, to help students better understand the relationship between 3D objects and their projections. Tangible models refer to the physical objects which are comprised of a set of differently shaped pieces. The tangible model we developed includes eight wooden blocks that include all the main geometrical features with respect to their 3D projections. The AR models are the virtual models which can superimpose 3D graphics of typical geometries on real-time video and dynamically vary view perspective in real-time to be seen as real objects. The AR model was developed using the ARToolKitPlus library and includes all the geometrical features generally taught in engineering graphics courses or technical drawing courses. To verify the effectiveness and applicability of the models we developed, we conducted a user test on 35 engineering-major students. The statistical results indicated that the tangible model significantly increased the learning performance of students in their abilities to transfer 3D objects onto two-dimensional (2D) projections. Students also demonstrated higher engagement with the AR model during the learning process. Compared to using the screen-based orthogonal and pictorial images, the tangible model and augmented reality model were evaluated to be more effective teaching aids for engineering graphics courses. © 2011 American Society of Civil Engineers.
The ever increasing volume of visual data due to recent advances in smart devices and camera-equipped platforms provides an unprecedented opportunity to visually capture actual status of construction sites at a fraction of cost compared to other alternatives methods. Most efforts on documenting as-built status, however, stay at collecting visual data and updating BIM. Hundreds of images and videos are captured but most of them soon become useless without properly being localized with plan document and time. To take full advantage of visual data for construction performance analytics, three aspects (reliability, relevance, and speed) of capturing, analyzing, and reporting visual data are critical. This paper 1) investigates current strategies for leveraging emerging big visual data and BIM in construction performance monitoring from these three aspects, 2) characterizes gaps in knowledge via case studies and structures a road map for research in visual sensing and analytics.
As BIM has been increasingly applied to building project recently, its application to civil engineering project is also on the rise. As construction structures have been expanded and complicated in a size and type, the information for handling maintenance process has also increased. Thus, to actively utilize the BIM information created at the design stage, this study has been carried out to establish a maintenance system using a marker-less based augmented reality method, by presenting a maintenance system for the construction structures using augmented reality. A SURF algorithm is used to link the 3D objects in the design and construction phases to the maintenance phase. The presented method in this study can increase the utilization of 3D information created at the design stage, by offering an augmented reality technology at the maintenance stage. The method could also improve the efficiency of visual inspection on construction structures by augmenting 3D model of a bridge structure.
Despite recent advancements, the time, skill, and monetary investment necessary for hardware setup and calibration are still major prohibitive factors in field data sensing. The presented research is an effort to alleviate this problem by exploring whether built-in mobile sensors such as global positioning system (GPS), accelerometer, and gyroscope can be used as ubiquitous data collection and transmission nodes to extract activity durations for construction simulation input modeling. Collected sensory data are classified using machine learning algorithms for detecting various construction equipment actions. The ability of the designed methodology in correctly detecting and classifying equipment actions was validated using sensory data collected from a front-end loader. Ultimately, the developed algorithms can supplement conventional simulation input modeling by providing knowledge such as activity durations and precedence, and site layout. The resulting data-driven simulations will be more reliable and can improve the quality and timeliness of operational decisions.
Conference Paper
This paper presents a new context-aware mobile augmented reality system that provides rapid and robust on-site access to construction project information such as drawings, specifications, schedules, and budgets. The mobile augmented reality system does not need any RF-based location tracking (e.g., GPS or WLAN) or optical fiducial markers for tracking a user's position. Rather, the user's location and orientation are automatically and purely derived by comparing photographs from the user's phone to a 3D point cloud model created from a set of site photographs. After generating a 3D point cloud model of construction site, field personnel can use mobile devices to take pictures of building elements and be presented on-site with a detailed list of project information related to the visible construction elements in an augmented reality format. The experimental results show that (1) the underlying 3D reconstruction module of the system generates more complete 3D point cloud models, and faster than other state-of-the-art Structure-from-Motion(SfM) algorithms; (2) the localization method is an order of magnitude more accurate than the state-of-the-art solutions, and can provide acceptable tolerances of most on-site engineering applications. Using an actual construction case study, the perceived benefits and limitations of the proposed method for on-site context-aware applications are discussed in detail.
Recently, it has increased projects in the Oil and Gas industry as growing plant of overseas construction market. Therefore, domestic companies need to set some aggressive plans up that secure competitiveness of international plant business such as enhancing productivity, technological ability and statistical management. Specifically in the Oil and Gas industry of overseas plant business requires implying information in EPC (Engineering, Procurement and Construction, below EPC) more, but it has not been suggested an appropriate breakdown system for contents that manages the information each EPC phase, which lead to problems such as revision of design document, occurring duplicated work and omissions of each participant. For these problems, we find market share decreasing and competitiveness running down. Thus, for increasing practical use of contents management system and construction industry's information, this paper suggests a contents breakdown system based on BPM (Business Process Management), defines business process and systematizing knowledge needed, and finally analyzing contents information that establish information among industrial facilities in Design phase which is the weakest part of domestic companies.