A Decision Support System Proposal on the Usage of Extended Reality SDKs in AEC Disciplines

Full text

A Decision Support System Proposal on the Usage of Extended Reality SDKs in AEC Disciplines | Karacif, Esranur; Gürer, Ethem
1
MSTAS 2022
AEC Disiplinlerinde Genişletilmiş Gerçeklik SDK’larının Kullanım Alanları
Üzerinden Bir Karar Destek Sistemi Önerisi
Esranur Karacif 1; Dr. Öğr. Üye. Ethem Gürer2
1İstanbul Teknik Üniversitesi; 2İstanbul Teknik Üniversitesi
1https://orcid.org/0000-0002-3173-8687, 1demirtase17@itu.edu.tr;
2https://akademi.itu.edu.tr/egurer/Ethem-Gürer/, 2egurer@itu.edu.tr
Özet
Mimarlık, mühendislik ve inşaat disiplinlerinde (ing. kısaltma AEC) bilgisayar destekli tasarım (ing. kısaltma CAD), sanal gerçeklik,
artırılmış gerçeklik gibi teknolojik araçlar görsel temsil oluşturma, yapının inşa aşamasında destek sağlama gibi özellikleri nedeniyle
kullanılırlar. Bu araçların kullanıcıları ve geliştiricileri teknolojinin mevcut olanaklarını doğrudan kullanabilecekleri gibi, teknolojinin
sınırlılıkları üzerinden çeşitli çözümler geliştirebilirler. Bu çalışmada her iki perspektif de mobil genişletilmiş gerçeklik
uygulamalarının SDK’ları (yazılım geliştirme kiti) özelinde incelenecektir. Yazılım geliştirme kiti (SDK), yazılım geliştirmek için
kullanılan bir programlama aracı ve/veya yazılımlar için bir kütüphane koleksiyonudur. Mobil cihazlar yaygın olarak kullanılan,
kullanıcı dostu, ekonomik araçlardır. Mobil cihazlar yardımıyla genişletilmiş gerçeklik teknolojilerinin AEC disiplinlerindeki yüksek
potansiyelde kullanım olanaklarına rağmen bu alanlarda beklenilenin altında bir kullanım görülmektedir. Bu disiplinlerde farklı
genişletilmiş gerçeklik uygulamalarının mobil kullanımı neticesinde çeşitli teknikler geliştirilmiştir. Bu çalışma kapsamında
mimarlık, mühendislik ve inşaat disiplinlerinde SDK’larla yapılan mobil genişletilmiş gerçeklik uygulamalarının kullanıldığı 184
makale incelenmiş olup, bunlardan 28 makale araştırma kapsamında ele alınmıştır. Bu makaleler artırılmış gerçeklik ve karma
gerçeklik anahtar kelimelerinin yanı sıra, mimari ve kentsel tasarım, inşaat ve dijital fabrikasyon anahtar kelimelerini içermektedir.
İncelenen literatür kapsamında kullanılan SDK’ların teknolojik açıdan sözü geçen disiplinlere sağladıkları katkılar ve bunların hangi
durumlarda kullanıldıklarını akış diyagramları üzerinden ele almak çalışmanın odağını oluşturur. Çalışmanın amacı, mobil
genişletilmiş gerçeklik SDK’larının mimarlık, mühendislik ve inşaat disiplinlerine katkılarını literatür çalışmalarındaki uygulamalar
üzerinden ortaya çıkarmak ve bu veriler üzerinden karar destek sistemi bileşenleri geliştirmektir. Bu sistem, giriş seviyesinde bir
geliştiricinin çalışması doğrultusunda yararlanabileceği yazılım geliştirme araçlarının ilişkiselliklerini, karşılaştırmalarını
görebilmesi açısından ve hangi alanlarda ne tür kullanımlara uygun olabileceğine yönelik bir çerçevenin oluşturulması açısından
önemlidir. SDK’ların teknolojik açıdan potansiyelleri ve sınırlılıkları araştırmaların tasarımına etki etmektedir. SDK’ların odaklanılan
alana uygun olarak seçimi bu nedenle önemlidir. Hangi SDK’nın ne amaçla kullanıldığında nasıl bir sonuç vereceğinin ilişkisini
kurgulamak da araştırma aşamasında önem kazanmaktadır. Araştırma sonucunda, yazılım geliştirme kitlerinin çeşitli alanlarda
değişken kullanımlara olanak sağladıkları ve bu kitlerin kullanımının literatür çalışmalarına süre ve bütçe açısından da destek
sağladığı gözlenmiştir.
Anahtar Kelimeler: Genişletilmiş Gerçeklik, Karar Destek Sistemi, Mimarlık, Mühendislik ve İnşaat (ing. AEC), Mobil Artırılmış
Gerçeklik, Yazılım Geliştirme Kiti (ing. SDK).

A Decision Support System Proposal on the Usage of Extended Reality SDKs in AEC Disciplines | Karacif, Esranur; Gürer, Ethem
2
MSTAS 2022
A Decision Support System Proposal on the Uses of Extended Reality SDKs in
AEC Disciplines
Esranur Karacif 1; Assist. Prof. Dr. Ethem Gürer2
1Istanbul Technical University; 2Istanbul Technical University
1https://orcid.org/0000-0002-3173-8687, 1demirtase17@itu.edu.tr;
2https://akademi.itu.edu.tr/egurer/Ethem-Gürer/, 2egurer@itu.edu.tr
Abstract
In architecture, engineering and construction disciplines (AEC), technological tools such as computer aided design (CAD), virtual
reality, augmented reality are used for their features such as creating visual representation and providing support during the
construction phase of the building. The users and developers of these tools can directly utilize the existing possibilities of the
technology, as well as develop various solutions over the limitations of these technology. In this study, both perspectives will be
examined in terms of SDKs (software development kit) of mobile extended reality applications. 184 articles using mobile extended
reality (both augmented reality and mixed reality) applications made with SDKs in architecture, engineering and construction
disciplines have been examined, and 28 of these articles have been discussed within the scope of the research. These articles include
keywords as augmented reality and mixed reality, as well as construction, architectural design, urban design, digital production
and assembly. This broad scope of literature review phase is a requirement, as some of the papers do not include the utilized
software or SDK. This research studies only the papers with the information such as utilized technological properties of the SDK,
the opportunities the SDK provides and/or the shortcomings of the SDK. It eliminates the papers not giving these specific
information. This study examines Vuforia, Fologram, ARCore, ARKit, ARToolKit, Gamma AR, Kudan and xBIMToolKit in the field of
AEC. Each of these kits offers divergent technological solutions such as visual inertial odometry, scene recognition or rendering
optimizations. The contributions of the SDKs used within the scope of the studied literature to the disciplines mentioned before in
terms of their technological capabilities and the situations in which they are used, through comparison flowcharts constitute the
focus of the study. The aim of the study is to reveal the contributions of mobile extended reality SDKs to architecture, engineering
and construction disciplines through the applications in literature studies and to develop a decision support system based on these
data. This system is important in terms of being able to see the relationality and comparisons of the software development kits
that an entry-level developer can use in line with his work, and in terms of creating a framework for what kind of uses in which
areas. The technological potentials and limitations of SDKs affect the design of research. This is why it is important to select the
SDKs according to the focus area. It is also important during the research phase to create the relationality-diagrams about which
SDK should be used and how it will result. As a result of the research, software development kits allow variable uses in various
fields and the use of these kits also supports the literature studies in terms of time and budget.
Keywords: Architecture, Engineering, Construction (AEC), Decision Support System (DSS), Extended Reality (XR), Mobile Augmented
Reality (MAR), Software Development Kit (SDK).

A Decision Support System Proposal on the Usage of Extended Reality SDKs in AEC Disciplines | Karacif, Esranur; Gürer, Ethem
3
MSTAS 2022
1. Introduction
Emergent technologies effect the field of architecture, engineering and construction (AEC) in terms of design practices,
production and construction techniques, representation methods. Despite the potential of augmented reality (AR) in
this field, the professionals in this field did not adopted it completely with regard to its dynamic concept. By
collaboration between different stakeholders AR increases work efficiency.
Although AR has a potential to enhance the real world objects, it also has some limitations. The quality of the software
limits and effects the quality of features used for the model integrated with AR. By being aware of the potentials and
limitations of the software used in AR applications, solutions can serve the purpose. At this point, integrated use of
the software allows effectivity, because of using the opportunity of each software in specific field or supporting the
shortcomings of software with others. More complex AR Applications can be created by utilizing different Software
Development Kits (SDKs). A software development kit (SDK) is a collection of programming tools and program libraries
that is used to develop software. Mobile devices are wide-spread, user-friendly, and affordable so that mass and
democratic usage of mobile augmented reality applications is possible for everyone. Because of these feature of
mobile devices, the focus of the paper is their production environment, namely SDKs.
SDKs differ from each other with their features such as tracking, recognition and rendering. AR technology can be
roughly classified into two types by the location registration method: sensor-based AR that is registered by a sensor
such as GPS, and image-based AR that recognizes feature points from image. Image-based AR is divided into two types
whether uses special markers or not, which is named marker-based AR by using it, another one is marker-less AR (Liang
Yan, 2019). Simultaneous Localization and Mapping (SLAM) technology allows for the generation of an environmental
map while estimating the self-position without using the map at the same time. The SLAM algorithm here stands for
visual SLAM which only uses the camera as a sensor. Widely used in robotics engineering, this tool improves the
stability of the AR tracking algorithm. Therefore, the designed tele-simulation system uses a markerless AR algorithm
(Liang Yan, 2019).
The research questions are how these environments can be used in AEC field; in which cases are they utilized most;
what potential contributions have been focused on in the literature. Besides, there are sub-questions such as which
SDK can be used effectively in which situations or fields; which kind of usage of the SDK do the authors prefer in their
researches; what are the technological properties and shortcomings of the SDKs; what opportunities the SDKs provide
in AEC fields; which hardware, software and programming languages are used together with the SDKs.
Answering these questions helps revealing the potential contributions and the limitations of the SDKs in the field of
AEC. This provides a framework and components for a decision support system for the user searching for a plausible
software for a specific purpose at the beginning of their study. This research compares the SDKs due to their common
and dissociating features.
The aim of the study is to reveal the contributions of mobile extended reality SDKs to architecture, engineering and
construction disciplines through the applications in literature studies and to develop a decision support system based
on these data. This system is important in terms of being able to see the relationality and comparisons of the software
development tools that an entry-level developer can use in line with his work, and in terms of creating a framework
for what kind of uses in which areas.
The contributions of the SDKs used within the scope of the studied literature to the disciplines mentioned before in
terms of their technological capabilities and the situations in which they are used, through flowcharts constitute the
focus of the study.
There are various kinds of software development kits (SDK) used in AR applications such as ARCore, Vuforia, ARToolKit,
ARKit, WikiTude, LayAR, Kudan, FaceSDK, SLARToolKit, FLARToolKit, OsgART, Droid AR, Augment, Aurasma, Metaio,

A Decision Support System Proposal on the Usage of Extended Reality SDKs in AEC Disciplines | Karacif, Esranur; Gürer, Ethem
4
MSTAS 2022
BazAR, D'Fusion, Gamma AR, Tango, Firebase, Estimote Indoor Location SDK, Fologram, Metaio and xBIMToolkit. In
this study, some SDKs such as Metaio or Tango are eliminated as they are withdrawn from the market. Thus, this study
only includes Vuforia, Fologram, ARCore, ARKit, ARToolkit, Gamma AR, Kudan AR and xBIMTools respectively.
One of the most utilized of these SDKs are Vuforia, ARCore, ARKit and ARToolKit. Vuforia is mostly utilized because of
its features such as recognition of different types of visual objects, text and environment recognition, scanning and
creating objects. ARCore’s prominent features are motion tracking, environmental recognition, light estimation. ARKit
is used especially for light estimation, TrueDepth camera, visual inertial odometry, scene recognition, rendering
optimizations. ARToolKit’s special features are single or stereo camera for position/orientation tracking, tracking of
simple black squares, tracking of planar images, camera calibration, optical stereo calibration, optical head-mounted
display support (Glover, 2018). All of these four SDKs are working with Unity 3D software which is the most common
used software for the production of AR applications. Without considering the common use of some software and their
price, the focus is on their features.
This study consists of an introduction section includes the problem, hypothesis, aim and scope; a literature review in
the fields of architectural and urban design, construction and digital fabrication; a methodology focused on the
technological properties and related tools of the SDKs (such as hardware, software, programming languages), the
opportunities they provide and shortcomings of the SDKs; and a results and conclusions section (Figure 1).
Figure 1: General framework of the study.
2. Literature Review
In the literature review, only those who developed AR application and used it in their study were examined. The articles
published in the fields of architecture, engineering and construction that contain information about the SDKs of mobile
augmented reality applications constitute the data source of this research.
184 research papers published between 2010-2021 are reviewed on ScienceDirect, International Symposium on
Augmented Reality, Google Scholar and Cumincad data base. Cumincad data base is taken from annual conferences
of the Association for Computer Aided Design In Architecture (ACADIA) and its sibling organizations in Europe (ECAADE
and CAAD Futures), Asia (CAADRIA), the Middle East (ASCAAD), South America (SIGRADI) and International Journal of
Automation and Computing (IJAC). Studies in these indexes include the keywords such as augmented reality
applications in architecture, engineering and construction (AEC), mobile augmented reality applications, augmented
reality SDKs (Figure 2).

A Decision Support System Proposal on the Usage of Extended Reality SDKs in AEC Disciplines | Karacif, Esranur; Gürer, Ethem
5
MSTAS 2022
The SDKs ARCore, Vuforia, ARToolKit, ARKit, Augment, Gamma AR, Kudan, Tango, Firebase, Estimote Indoor Location
SDK, Fologram, Metaio Creator and xBIMToolkit are studied in the articles related to this specific field. The author
eliminates Metaio Creator SDK as it’s bought by Apple in 2015 and is no longer available. The other eliminated software
are Tango, Firebase, Estimote Indoor Location SDK because of their availability.
The articles are divided into four categories: BIM/Construction/Building Physics, Architectural Design, Urban Design,
Digital Production. 28 articles about Architectural Design, Urban Design, BIM/Construction/Building Physics and Digital
Fabrication are examined with the aspects of utilized technological properties of the SDKs, the opportunities they
provide and the shortcomings of the software used in the process (Figure 3).
Figure 2: Literature review (filtered).
Figure 3: Outputs of the SDKs.
3. Methodology
The methodology of the study focuses on the technological properties and related tools of the SDKs (such as hardware,
software, programming languages), the opportunities they provide and shortcomings of the SDKs. Comparative
diagrams help easily comprehending the relationships between SDKs and AEC Fields, Software, Hardware,
Programming Languages and technological properties.
3.1. Comparative diagrams of SDKs
In the field of digital manufacturing the most common used SDK is fologram, besides ARCore, ARKit, ARToolKit, Vuforia
are also utilized. In the literature of BIM/Construction and Building Physics fields, divergent SDKs such as Gamma AR,

A Decision Support System Proposal on the Usage of Extended Reality SDKs in AEC Disciplines | Karacif, Esranur; Gürer, Ethem
6
MSTAS 2022
Kudan, xBIMToolKit, Vuforia, ARToolkit are in use. While Vuforia, Fologram and ARKit are used in architectural design
literature, Vuforia and ARKit are applied in urban design field (Figure 4 & 5).
Figure 4: Relationships between the SDKs and AEC Fields.
Figure 5: Relationships between the SDKs and AEC Fields.
While Fologram is mostly driven with Rhinoceros/Grasshopper, Vuforia is driven with Unity 3D, Gamma AR and
xBIMTools are used with Autodesk Revit together (Figure 6).
Figure 6: Relationships between the SDKs and software.
Figure 7: Relationships between the SDKs and hardware.
While smart phones and tablets are the main hardware runned with SDKs, there are other tools, such as camera,
multicopters, UAV camera, bluetooth and wifi enabled microcontroller, RGB Led, 3D printer, 3D scan, paper, robotic
arm, smart gloves, multi touch table, head mounted display, kinect (Figure 7).
The relationship between the SDKs and the programming languages like C#, Processing, C++, xBIM scripting language
is weak, as they are used mostly linear (Figure 8).

A Decision Support System Proposal on the Usage of Extended Reality SDKs in AEC Disciplines | Karacif, Esranur; Gürer, Ethem
7
MSTAS 2022
Figure 8: Relationships between the SDKs and programming languages.
Figure 9: Relationships between the SDKs and technological properties.
Technological properties have a more complex relationship, because mobile devices do not have similar qualities. For
example, ARKit uses TrueDepth Camera function and is only used in IPhones (Figure 9).
4. Results and Discussions
Although in the design field mostly both Vuforia and Unity are used, ARCore, ARToolKit and Tango SDKs are also utilized
in some cases. In the field of digital fabrication, most common used SDK and plugin is Fologram, because of its easy
integration with RhinoCeros and Grasshopper. Besides that, Vuforia, ARKit, ARCore are also applied.
The state-of-art of mobile AR applications orient the studies in terms of their potentials an limitations. The studies
show that the use of more mobile application SDKs enhances the researches in terms of their different qualities.
As a result of the research, it has been observed that software development kits allow different uses in various fields,
and the researches show that by the use of these kits, working time is shortened, the spent budget is reduced, and
more qualified results are obtained.
In the next stage of the study, the production of applications including architectural and urban design,
BIM/construction and digital fabrication over these SDKs is important as it will also enable the testing of SDKs. In the
future researches, the sound integrated to the SDKs could also be used as in navigation systems to assist the lay-
person.
This research is limited with the SDKs in 184 research articles in the field of AEC industry. Some of the SDKs such as
DeepAR, Wikitude, Easy AR, D’Fusion, ARMedia are not included in these articles. A more enhanced research is
required to compare with other SDKs.
The SDKs need to be enhanced by artificial intelligence (deep learning) and computer vision.
The technological potentials and limitations of SDKs affect the design of research. This is why it is important to select
the SDKs according to the focus area. It is also important to create the relationalty-diagrams about which SDK should
be used and how it will result during the research phase.
Acknowledgement
We would like to thank Prof. Leman Figen Gül and Assoc. Prof. Sema Alaçam (Istanbul Technical University) for their
valuable contribution to this study with their guidance, lectures and constant supervision.

A Decision Support System Proposal on the Usage of Extended Reality SDKs in AEC Disciplines | Karacif, Esranur; Gürer, Ethem
8
MSTAS 2022
REFERENCES
Abe, U., Hotta, K., Hotta, A., Takami, Y., Ikeda, H., & Ikeda, Y. (2017). DIGITAL CONSTRUCTION: Demonstration of Interactive
Assembly Using Smart Discrete Papers with RFID and AR codes. CAADRIA, 75-85.
Ashour, Z., & Yan, W. (2020). BIM-Powered Augmented Reality for Advancing Human-Building Interaction. eCAADe, 169-178.
Betti, G., Aziz, S., & Ron, G. (2019). Pop Up Factory: Collaborative Design in Mixed Reality Interactive live installation for the
makeCity festival, 2018 Berlin. eCAADe 37 / SIGraDi 23, (s. 115-124).
Carozza, L., Valero, E., Bosché, F., Banfill, G., Mall, R., & Nguyen, M. (2017). Urbanplanar: BIM Mobile Visualisation in Urban
Environments with Occlusion-Aware Augmented Reality. Proceedings of the Joint Conference in Computing in
Construction, (s. 229-236). Heraklion.
Chaltiel, S., Bravo, M., & Chronis, A. (2017). Digital fabrication with Virtual and Augmented Reality for Monolithic Shells. eCAADe
35, (s. 211-218).
Fazel, A., & Izadi, A. (2018). An interactive augmented reality tool for constructing free-form modular surfaces. Automation in
Construction, 135-145.
Glover, J. (2018). Unity 2018 Augmented Reality Projects: Build four immersive and fun AR applications using ARKit, ARCore, and
Vuforia. Birmingham: Packt Publishing.
Goepel, G. (2019). Augmented Construction: Impact and opportunity of Mixed Reality integration in Architectural Design
Implementation. ACADIA, (s. 430-437).
Goepel, G., & Crolla, K. (2020). AUGMENTED REALITY-BASED COLLABORATION: ARgan, a bamboo art installation case study.
CAADRIA, (s. 313-322). Hong Kong.
Grasser, A. (2019). Towards an Architecture of Collaborative Objects. eCAADe 37 / SIGraDi 23, (s. 325-332).
Gül, L. F. (2017). Studying Architectural Massing Strategies in Co-design Mobile Augmented Reality Tool versus 3D Virtual World.
eCAADe 35, (s. 703-710).
Hashimoto, J., & Park, H.-J. (2020). Dance with Shadows: Capturing tacit knowledge with smart device augmented reality (SDAR).
eCAADe 38, (s. 165-172).
Jahn, G., Newnham, C., Berg, N. v., & Beanland, M. (2018). Making in Mixed Reality: Holographic Design, Fabrication, Assembly,
and Analysis of Woven Steel Structures. ACADIA, (s. 88-97).
Koch, V., Ritterbusch, S., Kopmann, A., Müller, M., Habel, T., & Both, P. v. (2011). Flying Augmented Reality: Supporting planning
and simulation analysis by combining mixed reality methods using multicopter and pattern recognition. eCAADe 29, (s.
843-849).
Kontovourkis, O., Georgiou, C., Stroumpoulis, A., Kounnis, C., Dionyses, C., & Bagdati, S. (2019). Implementing Augmented Reality
for the Holographic Assembly of a Modular Shading Device. eCAADe 37 / SIGraDi 23, (s. 149-158).
Liang Yan, T. F. (2019). An interactive augmented reality tool for constructing free-form modular surfaces. CAADRIA, 424-432.
Mirshokraei, M., Gaetani, C. I., & Migliaccio, F. (2019). A Web-Based BIM–AR Quality Management System for Structural Elements.
MDPI, Applied Sciences.
Narazani, M., Eghtebas, C., Klinker, G., Jenney, S. L., Mühlhaus, M., & Petzold, F. (2019). Extending AR Interaction through 3D
Printed Tangible Interfaces in an Urban Planning Context. UIST'19, (s. 116-118). New Orleans.
Silcock, D., Schnabel, M. A., Moleta, T., & Brown, A. (2021). PARTICIPATORY AR: A Parametric Design Instrument. CAADRIA, (s.
295-304). Hong Kong.
Song, Y. (2020). BloomShell: Augmented Reality for the assembly and real-time modification of complex curved structure. eCAADe,
(s. 345-354).
Song, Y., Koeck, R., & Luo, S. (2021). AR DIGI-COMPONENT: AR-assisted, real-time, immersive design and robotic fabrication
workflow for parametric architectural structures. CAADRIA, (s. 253-262).
Sun, M., Sun, P., Dong, Y., & Lopez, J. L. (2021). MASS PRODUCTION: Towards Mu ltidi mensi onal, Real-time Feedback in Early Stages
of Urban Design Processes. CAADRIA, (s. 649-658). Hong Kong.
Syahputra, M. F., Hardywantara, F., & Andayani, U. (2018). Augmented Reality Virtual House Model Using ARCore Technology
Based on Android. Journal of Physics: Conference Series.
Yan, L., Fukuda, T., & Yabuki, N. (2019). Intergrating UAV Development Technology With Augmented Reality Toward Landscape.
CAADRIA, (pp. 423-432). Hong Kong.
Yao, J., Lin, Y., Zhao, Y., Yan, C., Chang-lin, L., & Yuan, P. (2018). Augmented Reality Technology Based Wind Environment
Visualisation. CAADRIA, 369-377.
Yao, J., Lin, Y., Zhao, Y., Yan, C., Li, C., & Yuan, P. F. (2018). Augmented Reality in the Design Process: Using visual effects (VFX)
motion tracking techniques to conduct quantification research on the performance of augmented reality. eCAADe 36, (s.
761-70).
Zhang, Y., & Kwok, T.-H. (2018). Design and Interaction Interface using Augmented Reality for Smart Manufacturing. 46th SME
North American Manufacturing Research Conference, (s. 1278-1286). Texas.