No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Review and analysis of augmented reality (AR) literature for digital fabrication in architecture
Abstract
The use of Augmented Reality (AR) technologies has increased recently, due to the equipment update and the mature technology. For architectural design, especially in digital fabrication projects, more designers begin to integrate AR methods to achieve the visualization in the process. To help unskilled labors for holographic on-site previewing and instruction training, experimental and practice-based studies in AR for the architectural digital fabrication have emerged in recent years. Now, it is a great opportunity to discuss the topic of AR in architectural digital fabrication. By presenting a statistical review of AR technology in architecture projects, this literature review aims to review ongoing research and provide pathways for further research in architectural digital fabrication. This review article is based on information found in journal publications and conference papers in the fields of architecture, engineering, robotics, and digital fabrication, published to date (from 2010 to 2020). The review narrows the literature within these papers by filtering 84 articles through the keyword “Augmented Reality”, “Digital Fabrication” and “Assembly”. The selected articles can be categorized based on the most use of AR function in architectural digital fabrication into an order of the following three classifications with the most significant growth in the last years: (A) AR 3D holographic instruction, (B) AR data sharing, (C) AR for Human-Computer interaction. The information collected from these articles within their classifications is meant to give insight into the current state-of-the-art of AR in the architectural digital fabrication area, as well as to summarize how the topic has matured and developed over time in the research and industry literature. This article has not only analyzed the existing literature but also highlighted new emerging fields in AR research and the future trends of AR function in architectural digital fabrication.
... A década seguinte apresentou um amadurecimento no uso de AR e VR em arquitetura, especialmente nas áreas de fabricação digital, através de instruções holográficas para montagem, compartilhamento de dados e interação com os computadores (Song;Koeck;Luo, 2021), mas também auxiliando na adoção do BIM como ferramenta de suporte quando integrada a esse processo de trabalho (Sidani et al., 2021). Apesar do amadurecimento da tecnologia e um aumento de sua adoção na área de arquitetura, a revisão sistemática sobre técnicas e ferramentas recentes de Realidade Aumentada associadas ao BIM, realizada por Sidani et al. (2021), mostra que sua implementação ainda está distante do desejável, principalmente a limitações que dificultam o uso durante a fase de execução no canteiro de obras. ...
... A década seguinte apresentou um amadurecimento no uso de AR e VR em arquitetura, especialmente nas áreas de fabricação digital, através de instruções holográficas para montagem, compartilhamento de dados e interação com os computadores (Song;Koeck;Luo, 2021), mas também auxiliando na adoção do BIM como ferramenta de suporte quando integrada a esse processo de trabalho (Sidani et al., 2021). Apesar do amadurecimento da tecnologia e um aumento de sua adoção na área de arquitetura, a revisão sistemática sobre técnicas e ferramentas recentes de Realidade Aumentada associadas ao BIM, realizada por Sidani et al. (2021), mostra que sua implementação ainda está distante do desejável, principalmente a limitações que dificultam o uso durante a fase de execução no canteiro de obras. ...
... A década seguinte apresentou um amadurecimento no uso de AR e VR em arquitetura, especialmente nas áreas de fabricação digital, através de instruções holográficas para montagem, compartilhamento de dados e interação com os computadores (Song;Koeck;Luo, 2021), mas também auxiliando na adoção do BIM como ferramenta de suporte quando integrada a esse processo de trabalho (Sidani et al., 2021). Apesar do amadurecimento da tecnologia e um aumento de sua adoção na área de arquitetura, a revisão sistemática sobre técnicas e ferramentas recentes de Realidade Aumentada associadas ao BIM, realizada por Sidani et al. (2021), mostra que sua implementação ainda está distante do desejável, principalmente a limitações que dificultam o uso durante a fase de execução no canteiro de obras. ...
O presente ensaio aborda a experiência e os conceitos de Realidade Aumentada e Realidade Virtual para refletir sobre o projeto bioclimático como tema e conteúdo a ser explorado na docência em arquitetura. Problematiza o ensino de projeto bioclimático a partir da expectativa da adoção da Realidade Aumentada como recurso didático para o ensino superior na área, o que foi feito com vistas às estratégias adotadas por meio da experiência do estágio de docência junto à disciplina ARQ 326 – Conforto Térmico, ministrada para estudantes do terceiro período do curso de graduação em Arquitetura e Urbanismo da Universidade Federal de Viçosa (UFV). Ao abordar o tema da Bioclimatologia e do projeto baseado em desempenho, a disciplina representa o primeiro contato dos estudantes com esses temas, alguns dos quais, na percepção estagiário de docência, foram mais bem aproveitados a partir da adoção das ferramentas digitais se comparadas aos recursos analógicos em sala de aula. Além disso, o estágio de docência apresentado aqui, proporcionou a reflexão sobre a transposição didática dos conteúdos pautados no intuito de vislumbrar recursos para a mediação dos temas a serem explorados em AR pelo pesquisador junto aos graduandos.
... The use of AR is becoming more widespread and normalized, AR and digital tools are increasingly being combined in new ways. AR technology is already being used as an interface between digital information and the real world, facilitating human access to information and providing visual aids and manufacturing guidance for existing 3D modelling schemes (Song, 2021). In traditional building construction, forms of buildings are as 2D drawings are unable to convey the same degree of complexity as a 3D visualisation can, whereas AR technology can effectively visualise 3D designs (Dunston, 2008). ...
... In traditional building construction, forms of buildings are as 2D drawings are unable to convey the same degree of complexity as a 3D visualisation can, whereas AR technology can effectively visualise 3D designs (Dunston, 2008). The superimposition of computerised 3D model information and structural drawings onto real-time scenarios in construction facilitates human-computer interaction in digital construction through real-time shared information and assistance (Song, 2021). In digital fabrication projects for architectural design, many architects use AR to visualise the construction process (Song, 2021). ...
... The superimposition of computerised 3D model information and structural drawings onto real-time scenarios in construction facilitates human-computer interaction in digital construction through real-time shared information and assistance (Song, 2021). In digital fabrication projects for architectural design, many architects use AR to visualise the construction process (Song, 2021). In recent years, several experiments and studies on AR-assisted digital construction have emerged to guide construction workers to understand the site and construction process more quickly. ...
The prevailing issue of low housing supply is caused in part by a lack of skilled construction workers. If unskilled residents were adequately equipped, they could potentially address this issue, expanding and adapting their homes without professional builders and designers. This research proposes the use of discrete components to build on unused community lands using augmented reality (AR) assisted assembly. Enfield, a low-density community in London, was chosen as the case study location. Modular components, pre-assembled off-site by robots, were designed to form a set of interlocking structures making them easy to install and dismantle. AR-assisted construction integrates design and construction processes, helping to resolve sequencing and flow issues during assembly. A robotic simulator was used to visualise the robotic assembly process and an AR programming was used to guide self-builders with no building experience through assembly. An experiment was set up, where AR-assisted non-professionals use modular components to assemble a 1:2 scale pavilion model. This workflow could encourage individual and community-led building expansions and could be effective in reducing pressure on the construction industry and providing more housing.
... The use of digital tools and methods to deal with problems and challenges in several commercial and industrial sectors, including the architecture, engineering, and construction industry (AEC), has become common in the last decade [7,8] since they provide opportunities to obtain well-tailored parts and structures capable of being adapted to specific functions [9]. Tools such as additive manufacturing (AM) and robotic manufacturing, integrated with techniques such as artificial intelligence (AI), augmented reality, virtual data connectivity, machine learning (ML), and automation, among others, have created a landscape with product design possibilities that would not be possible with traditional manufacturing techniques [8,10]. ...
... The use of digital tools and methods to deal with problems and challenges in several commercial and industrial sectors, including the architecture, engineering, and construction industry (AEC), has become common in the last decade [7,8] since they provide opportunities to obtain well-tailored parts and structures capable of being adapted to specific functions [9]. Tools such as additive manufacturing (AM) and robotic manufacturing, integrated with techniques such as artificial intelligence (AI), augmented reality, virtual data connectivity, machine learning (ML), and automation, among others, have created a landscape with product design possibilities that would not be possible with traditional manufacturing techniques [8,10]. ...
Currently, cities face the challenges of achieving net-zero emissions, sustainable resource usage, and occupational safety. Sustainable manufacturing processes (SMP) in the architecture, engineering, and construction industry (AEC) could help to master such challenges if non-digitized or insufficiently networked processes did not repeatedly hinder it. The smart combination of additive manufacturing (AM) and nature-based design (NbD) could lead to an economic breakthrough in SMP. AM quickens process fulfillment and automation, offering potential to reduce expenditures in resources, costs, and associated risks while ensuring sustainability, and if early integrated into the design, it allows defining lightweight, sustainable, and material as objectives. NbD approaches in AM for AEC lead to complex structures with superior performance, minimizing material usage, and fostering regenerative, inclusive, and climate-adapted designs that shape contemporary architecture. Thus, this chapter comprehensively reviews NbD for AM projects, analyzing (i) geometry-focused design strategies, (ii) modeling approaches with performance criteria, and (iii) challenges of implementing NbD approaches in AEC.
... (1) What is the DT-CI concept, and what DT-CI representation exists? (2) How to create and implement DT-CI? (3) Which services can DT-CI provide for the project? ...
... This paper suggests choosing the DT-CI module to be implemented from the following two perspectives. The first is Fig. 13 Five-part structure of the DT-CI framework (2) to quantitatively evaluate the research status and development trends of DT-CI through a bibliometric review. (3) to systematically clarify the concept, creation, services, and future directions of DT through a framework-based review. ...
Currently, the engineering problems encountered in digital transformation of the construction industry are very complicated and need to besolved by integrating multiple techmologies. Consequently, the concept of digital twin (DT) was introduced and quickly applied throughout the building lifecycle. Despite this, many practitioners lack understanding of DT in the construction industry (DT-Cl) and its implementation. To overcome this issue, this paper presents a comprehensive and detailed review of DT-CI through a systematic literature review (SLR) that incorporates both quantitative and qualitative analysis. In this study, 222 DT-Cl studies were selected from a pool of 2619 publications across multiple databases, and 43 related researches were supplemented by the backward snowballing method based on co-citation analysis to generate the fimal bibliographic database. This paper quantitatively analyzes the curent state, hotspots, and development trends of DT-Cl research through a bibliometric review, and systematically clarifies the concept, creation, services, and future directions of DT-CI through aframework-based review. Finally, based on the SLR outcomes, this paper offers recommendations for future work and DT-Cl implementation. Contrary to other reviews within this field, this paper adheres to a rigorous SLR protocol to ensure the reproducibility of review results. Moreover, by comparing construction and non-construction DT concepts, we highlight the unique characteristics of DT-Cl namely its association with building information modeling (BlM) and emphasis on geometric reconstruction of building entities.
... In recent years, MR applications have been widely implemented for design, fabrication and assembly phase [6]. MR technology provides the reality-virtuality continuum, where digital BIM models and physical objects coexist and interact in real-time. ...
... The realscale of deck model has been automatically positioned on the physical girders based on three ArUco markers (as shown in Figure 6). Marker-based registration is most common method for AR-MR model registration in physical environment [6][7]. Marker-based tracking use a high-contrast image marker to provide a real-world point for of reference attached on the marker. ...
Geometric and dimensional deviations often lead to significant assembly errors during the implementation of design for manufacturing and assembly (DfMA) for prefabricated bridges. However, these errors are rarely detected until the actual structure is assembled on the construction site. This paper introduces a novel approach for implementing Mixed Reality (MR) technology using the Microsoft HoloLens 2 (HL2) device for a Preassembly Analysis System (PAS). The MR-based PAS includes indoor and on-site simulations for matching analysis, which aims to identify potential assembly errors caused by actual fabrication deviations. Matching analysis was conducted for the deck and girder to detect collision positions of shear connectors and pockets. Additionally, the length deviation of the lateral beam was automatically calculated to adjust the assembly plan using HL2. Ultimately, the results of the case study implementation demonstrate that the MR application significantly enhances the detection of assembly errors, leading to quicker and more accurate identification.
... • Client Interaction and Feedback: AR provides clients with interactive experiences of their future spaces by overlaying digital models onto physical site views. Clients can explore design options and provide feedback based on a more tangible representation of the project (Tonn et al., 2008;Song et al., 2021). AR enhances client engagement by offering an interactive and immersive experience, helping clients better visualize and understand design choices. ...
In “AI-Driven Architecture: Pioneering the Digital Frontier”, explore how artificial intelligence is reshaping architecture and urban design, pushing the limits of what’s possible. From AI-supported design processes and evolving transparency in algorithms to policy impacts on smart transportation and new paradigms in architectural education, this book delves into the intersection of technology and creativity. Discover how AI is integrated into design processes, drives innovations in conceptual design, and even re-imagines the concept of infinity in architecture. Featuring case studies, thought-provoking insights, and practical examples, this book offers a comprehensive guide to the transformative power of AI in shaping the built environment.
... Using AR devices, construction workers are equipped with a dynamic digital toolkit that enhances their decision-making, reduces errors, and streamlines complex assembly processes by avoiding the need for 2D plans. 7,8 By enhancing direct communication under the umbrella of a digital realm and boosting collaboration at scale, AR can serve as an ecological lever between robotic and human agents in the ecosystem of automated timber construction. 9 solely through the interlocking of the structural elements using 5-axis CNC routers or robotic arms to create integral connections. ...
This research introduces an innovative Augmented Reality (AR) workflow for Human-Robot Interaction (HRI) in timber construction. The approach leverages human dexterity and adaptability alongside the strength and precision of robotic arms to assemble timber structures connected by wood-wood connections. While research in the field of automated construction generally focuses on singular interactions, such as robot agents carrying components and human agents attaching them, this paper explores multiple degrees of interaction involving cooperation or collaboration between agents. A new algorithmic framework is developed to automate the generation of holographic instructions and allocate assembly tasks to human and robot agents according to their abilities. The application to a full-scale demonstrator reveals that certain elements necessitate collaboration for assembly, while others can exclusively be assembled manually or robotically. Ultimately, the research also highlights the benefits of AR in assisting manual assembly, simulating robot trajectories, and increasing safety during collaborative tasks.
... During the last decade, research using AR and MR workflows in the area of architectural fabrication have increased exponentially (5). Projects such as Woven Steel, Timber De-Standardized,, Code-Bothy, and many more have explored human interaction with digital instructions in MR through digital interfaces such as buttons and menus or fiducial markers such as QR codes and AruCo markers (6)(7)(8). ...
Mixed Reality (MR) platforms enable users to interact with three-dimensional holographic instructions during the assembly and fabrication of highly custom and parametric architectural constructions without the necessity of two-dimensional drawings. Previous MR fabrication projects have primarily relied on digital menus and custom buttons as the interface for user interaction with the MR environment. Despite this approach being widely adopted, it is limited in its ability to allow for direct human interaction with physical objects to modify fabrication instructions within the MR environment. This research integrates user interactions with physical objects through real-time gesture recognition as input to modify, update or generate new digital information enabling reciprocal stimuli between the physical and the virtual environment. Consequently, the digital environment is generative of the user's provided interaction with physical objects to allow seamless feedback in the fabrication process. This research investigates gesture recognition for feedback-based MR workflows for robotic fabrication, human assembly, and quality control in the construction of the UnLog Tower.
... W. McClung (1983) used the word "paradise", limiting himself to traditional architecture in England. The metaphor of paradise was also used by C. Daswatte (1997) design, parametric and generative shaping methods (Caetano et al., 2020) and, ultimately, the prospect of expanding the architectural experience with elements of virtual and mixed realities (Song et al., 2021). Consequently, along with the great intensity of the change in the "stylistic" and functional programme (Setti, 2021), competition between offers of various forms of presence increased, the main driving force of which was the motive of avoiding the mundane and striving for an attractive interaction between the individual and the architectural work (Heidari et al., 2018). ...
Due to the potential for creating immersive spaces in which the “function” of attracting and retaining attention replaces traditional calculations of climatic conditions, safety, gravity, etc., there is a need for terms for broader generalisation and evaluation. The purpose was to argue the expediency of using the term “perfect presence space”, as one that can collectively determine the value level of various design attempts to spatially express media and communicative realities, including comparing them with the experience of traditional architectural styles. The main method is a comparative analysis of specially selected architectural phenomena. On the basis of a brief historical retrospective, the tendency to gradually replace the functionalist “rational” substantiation of elements of space (associated with everyday life) with the attraction “irrational” (associated with non-ordinary life) is shown; the presence of experience in the materialisation of non-ordinary states in the architecture of different eras and purposes. Further, a review of several content frameworks that led architectural design to the interpretation of an exceptional or perfect presence is carried out, such as urban-social utopias of “ideal cities”, objects of religious pilgrimages and narrative stories, places of exceptional aesthetic qualities of the natural environment outlined by “paradise” metaphors, and representative and solemn squares of capital cities. The potential of the term “perfect presence space” is shown, which allows effectively operating with the motivational foundations of creating a modern space, which are based on attracting and retaining attention and stay; it provides a conceptual basis for evaluating offers of visual and spatial stay (non-virtual and virtual). The assumption is made about the mechanism of “idealisation” of presence, which is developed through the interaction of three components: person – attention – stay. The practical significance of the study is the opportunity, based on succinct, meaningfully clear and universal terms, to facilitate the perception of the architectural environment and improve the involvement of a wider range of stakeholders and participation groups in the study of design
... For instance, in Singapore, the use of an AR application called MUSE allowed students to create virtual galleries, enriching their learning experience through multimedia elements [16]. In the interior design sector, MAR is invaluable for enhancing collaboration among designers, clients, and fabricators, streamlining the design process from concept to construction [17,18,19]. It addresses challenges such as accurately capturing client preferences, which are crucial for design decisions regarding style and color schemes [20,21]. ...
The shift towards more adaptable learning approaches in design education, particularly remote and online learning, has been significant following the pandemic. This study explores the integration of mobile augmented reality (MAR) in ubiquitous learning settings with a focus on interior design education. We involved 163 undergraduate students in experiments to assess their use of MAR tools such as SketchUp and Fologram within Rhinoceros-Grasshopper. Their experiences were evaluated using the User Experience Questionnaire (UEQ), and a chi-squared test analyzed the relationship between these MAR tools and two educational models: the traditional classroom (Model-I) and the flipped classroom (Model-II). Results showed a preference for SketchUp across both models due to its user-friendly interface, while Fologram was preferred in the traditional classroom for its innovative features. The study underlines the importance of balancing advanced technology with effective teaching strategies to improve learning outcomes. Future research is recommended to enhance design education in distance-learning environments.
... Additionally demonstrating virtual and physical reality content in the same vision field can alleviate the problem of breaking the consistency of interactive experience when the human's field of view switches back and forth between the external screen and robots. According to research by Song et al. (2021) AR-based user interface for robotic control and AR-based programming by demonstration (PbD) method are the two main application scenarios in fabrication. The UI system can display functions of robot status and information applied for simplifying operations, improving safety and maintenance. ...
This paper presents a novel Digital Twin scenario for enhancing Human-Robot Collaboration (HRC) in the context of Industry 4.0. Traditional HRC environments, characterised by their repetitive and standardised tasks, are limited in adapting to dynamic and custom requirements. We propose a dynamically configurable Digital Twin (DCDT) system, enabling dynamic configuration of virtual HRC workspaces for creative design and prototyping, addressing the limitations of conventional HRC environments confined to repetitive tasks. By integrating AR, the system allows human operators to intuitively interact with robots, planning and simulating robotic trajectories with interactive holographic visualisations. We cooperate with The Design Modelling and Fabrication (DMaF) Lab at The University of Sydney and propose a workflow based on the DCDT system to assist students complete basic robot co-creative drawing programming under routine teaching scenarios. This research underscores the importance of AR in bridging the gap between physical and virtual workspaces, facilitating a flexible and efficient HRC setup. Future work will focus on refining the technology, addressing scalability, and exploring its adaptability to various creative and prototyping domains.
... The fundamental interactivity of augmented reality (AR) facilitates the easy incorporation of input and resulting modifications into the design process. The virtual model allows for real-time visibility of changes to all stakeholders, facilitating an adaptable and agile design development process [34]. The quick execution of design tasks not only accelerates the design process but also guarantees the appropriate incorporation of client preferences and needs into the design. ...
The explores into the significant impact of augmented reality (AR) within the domain of architectural. The research being conducted examines the transformative effect of augmented reality (AR) on the practice of architectural design, including the entire process from basic conceptualization through ultimate execution. The present investigation analyzes the utilization of augmented reality (AR) as an approach to enhance visualization, increase design accuracy, and ease client communication. This study discusses the advantages of augmented reality (AR) in facilitating the production of realistic and engaging design experiences. By using AR technology, designers are able to go past the restrictions presented by standard two-dimensional drawings and realistic models. The work addresses the challenges and obstacles linked to augmented reality (AR) technology, such equipment standards and technical knowledge. This article provides a comprehensive examination of current uses of augmented reality (AR) in architecture, utilizing a combination of mathematical modeling and practical case studies. Also, it analyzes the potential of AR to reshape the next phase of architectural design.
... Previous research using AR and MR workflows in the area of architectural fabrication have increased exponentially (Song et al., 2021b). Projects such as Woven Steel, Timber De-Standardized,, Code-Bothy, and many more have explored human interaction with digital instructions in MR through digital interfaces such as buttons, menus and/or fiducial markers such as QR codes and AruCo markers (Jahn et al., 2018a;Lee, 2022;Lok & Bae, 2022). ...
This research presents an innovative approach that integrated gesture recognition into a Mixed Reality (MR) interface for human–machine collaboration in the quality control, fabrication, and assembly of the Unlog Tower . MR platforms enable users to interact with three-dimensional holographic instructions during the assembly and fabrication of highly custom and parametric architectural constructions without the necessity of two-dimensional drawings. Previous MR fabrication projects have primarily relied on digital menus and custom buttons within the interface for user interaction between virtual and physical environments. Despite this approach being widely adopted, it is limited in its ability to allow for direct human interaction with physical objects to modify fabrication instructions within the virtual environment. The research integrates user interactions with physical objects through real-time gesture recognition as input to modify, update, or generate new digital information. This integration facilitates reciprocal stimuli between the physical and virtual environments, wherein the digital environment is generative of the user’s tactile interaction with physical objects. Thereby providing user with direct, seamless feedback during the fabrication process. Through this method, the research has developed and presents three distinct Gesture-Based Mixed Reality (GBMR) workflows: object localization, object identification, and object calibration. These workflows utilize gesture recognition to enhance the interaction between virtual and physical environments, allowing for precise localization of objects, intuitive identification processes, and accurate calibrations. The results of these methods are demonstrated through a comprehensive case study: the construction of the Unlog Tower , a 36’ tall robotically fabricated timber structure.
... AR technology allows a person to see a computer-generated virtual world that is simultaneously integrated with the real world. Thus, it can be said that AR tools can be used as an interface providing a link between digital information and the physical world [11]. Van Krevelan and Poelman went a step further in their definition, stating that AR "is an emerging technology with which a person can see more than others see, hear more than others hear, and perhaps even touch, smell and taste things that others cannot [12]". ...
(1) Background: A current trend observed in the logistics sector is the use of Industry 4.0 tools to improve and enhance the efficiency of cargo handling processes. One of the popular solutions is an augmented reality system that supports operators in everyday tasks. The article aims to present design assumptions for implementing an augmented reality system to support air cargo handling at the warehouse. (2) Methods: Research was carried out based on a five-stage analytical procedure, aiming to analyze the current state and identify the potential for implementing the AR system. The following methods were used to collect data: co-participant observations, process analysis, direct interviews, analysis of internal documentation, and applicable legal regulations. (3) Results: The conducted research allowed for identifying information flows accompanying cargo flows and developing a project to automate selected information flows. The obtained results made it possible to identify operations for which the AR system’s implementation will increase their effectiveness and efficiency. (4) Conclusions: The obtained results identified the need to develop a hybrid algorithm for arranging cargo in the warehouse and to build a system supporting self-verification of markings on air cargo.
... AR technology was recently applied in architectural construction fields to augment the current 2D-based construction files into three-dimensional geometries, by its unique visual characteristic of combining real and virtual objects in the aligned physical site (Chu et al., 2020). Besides its application on construction sites, AR technology, with its visual and interactive performance, could potentially augment the conventional architectural design and analysis process (Song et al., 2021). Although the architectural modelling method has fundamentally changed in its history, the AR tools currently integrated with the corresponding design methods are mainly limited to enhanced design draft visualization only. ...
This paper presents experimental research about developing a contextual design and analysis workflow utilizing augmented reality (AR) technology to enrich the current design process with an immersive experience. The limitation in the current design process is that due to the lack of spatial perception of the architectural draft on-site preview, designers sometimes find it difficult to fully evaluate their design proposals, which often causes unreasonable design outcomes and incomplete related analysis. To respond to the above problems, the HoloBrick research aims to create a unique augmented parametric algorithm for masonry façade designs to develop and validate the proposed contextual AR-assisted design and analysis workflow, which consists of two phases: a) AR contextual design, and b) AR contextual analysis and modification. The research findings highlight the advantages of using AR in the design and analysis process, such as providing immersive preview and interactive input methods, analyzing design outcomes within context, and enhancing draft modifications with better understanding, which are not offered in the current design process.
... In 2016, Süheyla Müge Halıcı et al. [45] researched the use of AR in collaborative design. In 2021, Shan Luob et al. [46] found that the use of augmented reality (AR) was significantly increasing in three areas: AR data exchange, AR human-computer interaction, and AR 3D whole-system training. In the same year, Fernando Moreu et al. studied the application of AR in civil infrastructure management and construction of buildings throughout their life cycle. ...
With the emergence of Architecture 4.0 and the occurrence of the COVID-19 pandemic, extended reality (XR) technology has been increasingly applied in architectural education. This study aims to systematically organize and analyze the applications and outcomes of XR technology in construction education over the past five years, provide a theoretical framework for its future widespread use, and highlight its drawbacks as well as future research directions. The paper employs content analysis to summarize and analyze the findings. The report reveals that more institutions are integrating XR technology into their architectural education programs and that it has a significant impact on teacher effectiveness, student motivation, reflection and improvement, and teacher–student communication. The study suggests that XR technology will increasingly replace conventional teaching techniques in classrooms.
... Nevertheless, limited precision across largescale environments, low display fidelity, and the diversity of tasks required in the AEC industry have continued to be barriers yet to be resolved (Shin and Dunston 2008). Recent improvements in usability and on-board computing capabilities of head-mounted displays (HMD), such as the Microsoft HoloLens 2, have enabled significant growth for AR applications in several areas of architectural digital fabrication: AR 3D holographic instruction, AR data sharing, and AR for Human-Computer interaction (Song et al. 2021). Examples of recent research integrating VR, AR, parametric design, and robotic fabrication include interactive dynamic toolpath generation and full-loop design-to-fabrication-to-assembly workflows for robotically-milled notched wooden struts (Buyruk and Çağdaş 2022;Morse et al. 2020). ...
The intent of this research is to study potential improvements and optimizations in the context of robotic fabrication paired with Augmented Reality (AR), leveraging the technology in the fabrication of the individual part, as well as guiding the larger assembly process. AR applications within the Architecture, Engineering, and Construction (AEC) industry have seen constant research and development as designers, fabricators, and contractors seek methods to reduce errors, minimize waste, and optimize efficiency to lower costs (Chi, Kang, and Wang 2013). Recent advancements have made the technology very accessible and feasible for use in the field, as demonstrated by seminal projects such as the Steampunk Pavilion in Tallinn, Estonia (Jahn, Newnham, and Berg 2022). These types of projects typically improve manual craft processes. They often provide projective guidelines , and make possible complex geometries that would otherwise be painstakingly slow to complete and require decades of artisanal experience (Jahn et al. 2019). Building upon a previously developed robotic tube bending workflow, our research implements a custom AR interface to streamline the bending process for multiple, large, complex parts with many bends, providing a pre-visualization of the expected fabrication process for safety and part-verification purposes. We demonstrate the utility of this AR overlay in the part fabrication setting and in an inadvertent, human-robot, collaborative process when parts push the fabrication method past its limits. The AR technology is also used to facilitate the assembly process of a spatial installation exploring a unique aesthetic with subtle bends, loops, knots, bundles, and weaves utilizing a rigid tube material.
... Over recent years, AR has seen substantial advancements, such as improved frame rates, resolution, and wearability, making it an increasingly suitable platform for scientific experimentation and practical application [1,2]. AR has been applied in a diverse array of areas, including medical education and surgery [3,4], repair and maintenance [5][6][7], architectural design [8,9], cultural heritage preservation [10,11], and remote collaboration [12,13], amongst others. With regard to information presentation, AR typically overlays context-specific information [14,15] or enables the integration of digital elements in a real environment [16,17]. ...
A spectrum of control methods in human-robot interaction was investigated, ranging from direct control to telepresence with a virtual representation of the robot arm. A total of 24 participants used a setup that included a Franka Emika Panda robot arm, Varjo XR-3 head-mounted display, and Leap Motion Controller. Participants performed a box-and-block task using the bare hand (A), and under five gesture-controlled robotic operation methods: direct sight (B), sight via video-feedthrough (C), in a 3D telepresence environment with (D) and without (E) virtual representation of the robot arm, and using a 2D video feed (F). The number of grabbing attempts did not differ significantly between conditions, but local operation (B & C) yielded more transferred blocks than tele-operation (D-F). Teleoperation using a 3D presentation was advantageous compared to teleoperation using a 2D video feed, as demonstrated by lower peak forces and smaller range in gripper heights in conditions D and E compared to condition F, a finding supported by analyses of the head movement activity. Finally, the bare hand yielded the best performance and subjective ratings. In summary, teleoperation using a 3D presentation provided a smoother interaction than teleoperation with a 2D video feed. However, direct human interaction remains a benchmark yet to surpass.
... The use of AR for fabrication and construction tasks has grown over time, with significant advances in the areas of 3D instruction, data sharing, and human-computer interaction [27]. While many lab studies have shown ideas for enhanced safety, instruction guidance, and task performance, there is limited literature on the use of these systems in the wild [20,29]. ...
Semi-automated timber fabrication tasks demand the expertise and dexterity of human workers in addition to the use of automated robotic systems. In this paper, we introduce a human-robot collaborative system based on Augmented Reality (AR).
To assess our approach, we conducted an exploratory user study on a head-mounted display (HMD) interface for task sharing between humans and an industrial robotic platform (N=16). Instead of screen-based interfaces, HMDs allowed users to receive information in-situ, regardless of their location in the workspace and the need to use their hands to handle tools or carry out tasks. We analyzed the resulting open-ended, qualitative user feedback as well as the quantitative user experience data. From the results, we derived challenges to tackle in future implementations and questions that need to be investigated to improve AR-based HRI in fabrication scenarios. The results also suggest that some aspects of human-robot interaction, like communication and trust, are more prominent when implementing a non-dyadic scenario and dealing with larger robots. The study is intended as a prequel to future work into AR-based collaboration between multiple humans and industrial robots.
... In this context, it is aimed to list two-dimensional and threedimensional drawings in a way that will be included in the product portfolio with the help of increased augmented reality so that digital manufacturing products can find a place in the industry. Data sharing, implementation of holographic instructions, listing and comparison of statistical analyzes between products can be counted among the 238 advantages of augmented reality (Song et al, 2021). Although architecture students' access to and application of augmented reality and artificial intelligence applications differs between institutions, statistics show that students tend to perceive artificial intelligence and augmented realitybased simulations and training models more easily. ...
The study focuses on how digital museum techniques can be used in the exhibition of cultural heritage items. The method of the study consists of the steps of examining the differences in the contemporary exhibition forms of cultural heritage items through literature review and the digital exhibition methods that can be used in this context and creating a model. Cultural heritage includes places, items and practices that a society considers ancient, important and worthy of protection; architecture, rituals, traditions, local knowledge, social interactions, arts and crafts, music, political and ideological beliefs, history, practices related to the natural environment, language, sports, habits and traditional clothing contains components. Exhibiting cultural heritage items with digital museology techniques not only helps visitors grasp the exhibited items through different interaction methods, but also conveys cultural heritage items to the future in a contemporary language. In this context, digital museum techniques that can be used to exhibit cultural heritage items include touch screens, digital audio and video systems, sensors, hologram technology, QR codes, simulations, virtual reality (VR), augmented reality (AR), projection mapping and mixed reality (MR) can be used. In the study, how the use of different digital museum techniques for the characteristic features of cultural heritage items mediates the comprehension of cultural heritage items by large masses is expressed with examples and a model is created with the data obtained from here. As a result, through this model, an exhibition scenario will be designed by considering the differing features of cultural heritage items and the methods which allow visitors’ experience of cultural heritage items to be diversified.
... The use of digital tools to deal with problems and challenges in several commercial and industrial sectors, including the construction sector, has become common in the last decade [1], [2] since they provide opportunities to obtain well-tailored parts and structures capable of being adapted to specific functions [3]. In several other sectors, such as automotive, aerospace, food industry and processing, aeronautics, and medical field, among many others, digital transformation, typical of industry 4.0, has been a great ally [4]- [8], which has also served as a pathway to achieve sustainability [9]- [11] and to face the many challenges that COVID-19 pandemic has left worldwide [12], [13]. ...
... During the last decade, research using AR and MR workflows in the area of architectural fabrication have increased exponentially (5). Projects such as Woven Steel, Timber De-Standardized,, Code-Bothy, and many more have explored human interaction with digital instructions in MR through digital interfaces such as buttons and menus or fiducial markers such as QR codes and AruCo markers (6)(7)(8). ...
Mixed Reality (MR) platforms enable users to interact with three-dimensional holographic instructions during the assembly and fabrication of highly custom and parametric architectural constructions without the necessity of two-dimensional drawings. Previous MR fabrication projects have primarily relied on digital menus and custom buttons as the interface for user interaction with the MR environment. Despite this approach being widely adopted, it is limited in its ability to allow for direct human interaction with physical objects to modify fabrication instructions within the MR environment. This research integrates user interactions with physical objects through real-time gesture recognition as input to modify, update or generate new digital information enabling reciprocal stimuli between the physical and the virtual environment. Consequently, the digital environment is generative of the user's provided interaction with physical objects to allow seamless feedback in the fabrication process. This research investigates gesture recognition for feedback-based MR workflows for robotic fabrication, human assembly, and quality control in the construction of the UnLog Tower.
... Recent developments in mixed reality (MR) technologies have enabled designers to reconsider the way nonstandard materials are used in construction. MR can be utilized as a fabrication and assembly tool to bridge the gap between digital design and manual construction (Song et al., 2021). Using MR, digital models are superimposed into real-world contexts to provide three-dimensional visualizations to guide onsite design and fabrication. ...
The research addresses the difficulties of designing with a non-standard and structurally dynamic material, such as bamboo, by using mixed reality (MR) as a virtual and collaborative design environment with integrated material physics simulations. This paper describes the development of a workflow that integrates (1) a reconfigurable modular bamboo system leveraging the active bending property of bamboo, (2) a custom physics-based MR environment facilitating on-site collaborative design, and (3) an MR user interface enabling users to customize material parameters unique to bamboo. The reconfigurable modular bamboo system is a kit of parts consisting of linear, triangular, and tetrahedral bundled bamboo modules that can be assembled on the ground and lofted into complex active bending structures. Through a Physics-Based Mixed Reality design framework, multiple users can reconfigure these modules virtually, dynamically, and collaboratively, iterating through several configurations on-site before deploying real-world resources. The result is a custom MR environment that enables non-expert users to participate in the design process through a user interface for managing module properties and connectivity.
... The need for connectivity and interactivity with digital information that has direct implications in the real environment is stronger than ever in the face of the increased use of cyber-physical systems, the use of digital twins, and artificial intelligence, among others [28]. A valuable particularity of MR for the purpose of the present project lies in its ability to deliver aggregated information in the workspace that is immediately linked to the elements on site, which allows help for unskilled workers to accelerate and strengthen their training [29]. It is possible to bet on a manufacturing model with digital twins that link and communicate in real time the implications of the process, in advance, for decision making and safety. ...
In the field of construction, human-robot collaboration and mixed reality (MR) open new possibilities. However, safety and reliability issues persist. The lack of flexibility and adaptability in current preprogrammed systems hampers real-time human-robot collaboration. A key gap in this area lies in the ability of the robot to interpret and accurately execute operations based on the real-time visual instructions and restrictions provided by the human collaborator and the working environment. This paper focuses on an MR-based human-robot collaboration method through visual feedback from a vision-based collaborative industrial robot system for use in wood stereotomy which we are developing. This method is applied to an alternating workflow in which a skilled carpenter lays out the joinery on the workpiece, and the robot cuts it. Cutting operations are instructed to the robot only through lines and conventional "carpenter's marks", which are drawn on the timbers by the carpenter. The robot system's accuracy in locating and interpreting marks as cutting operations is evaluated by automatically constructing a 3D model of the cut shape from the vision system data. A digital twin of the robot allows the carpenter to previsualize all motions that are required by the robot for task validation and to know when to enter the collaborative workspace. Our experimental results offer some insights into human-robot communication requirements for collaborative robot system applications in timber frame construction.
... Moreover, based on the above unique features, AR is an interfacing technology that has recently become popular in remote architectural practice and industrial robotic fabrication sectors (Chu et al., 2020). Intertwining AR technology with architectural practice opens up possibilities for a remote environment where humans can intuitively communicate with digital content, objects, machines, and spatial context (Song et al., 2021). This paper presents experimental research using AR technology as an interface for robotic remote programming in architectural practice. ...
... The authors found that the use of AR technology led to a significant increase in consumer engagement and purchase intention, indicating the potential of AR in marketing and advertising. In addition to these areas, AR technology has also been applied in various other fields, such as entertainment [23], tourism [24], and architecture [25]. In conclusion, the AR field has seen significant growth in recent years, with numerous studies exploring its potential applications and impact on various areas of research [26]. ...
This research studied people’s responses to requests that ask for accessing their personal information when using augmented reality (AR) technology. AR is a new technology that superimposes digital information onto the real world, creating a unique user experience. As such, AR is often associated with the collection and use of personal information, which may lead to significant privacy concerns. To investigate these potential concerns, we adopted an experimental approach and examined people’s actual responses to real-world requests for various types of personal information while using a designated AR application on their personal smartphones. Our results indicate that the majority (57%) of people are willing to share sensitive personal information with an unknown third party without any compensation other than using the application. Moreover, there is variability in the individuals’ willingness to allow access to various kinds of personal information. For example, while 75% of participants were open to granting access to their microphone, only 35% of participants agreed to allow access to their contacts. Lastly, monetary compensation is linked with an increased willingness to share personal information. When no compensation was offered, only 35% of the participants agreed to grant access to their contacts, but when a low compensation was offered, 57.5% of the participants agreed. These findings combine to suggest several practical implications for the development and distribution of AR technologies.
... Conversely, the systematic analysis serves as an additional base for discovering the main existing research directions, highlighting research gaps, and recommending future research directions (Aghimien et al., 2021). Therefore, this approach was adopted by many researchers in the construction domain, such as (Bedarf et al., 2021) in the applications of foam 3D printing for constructions and (Song et al., 2021) in implementing augmented reality in digital fabrication projects. ...
Offsite construction (OSC) is an innovative construction technique that transfers the site-based construction processes away from the jobsite into a more controlled environment. Prefabricated components, panelized elements, and fully prefabricated volumetric modules (modular construction) are some examples of OSC techniques. Numerous advantages are perceived by the adoption of OSC, such as minimizing construction waste, speedy construction, higher sustainability, and better quality. However, OSC techniques are characterized by a slow pace of adoption due to several barriers. These barriers are associated with complex supply chains, financial issues, cost uncertainty, lack of knowledge, etc. Accordingly, comprehensive literature has revealed that previous studies have overlooked the financial barriers of the OSC projects. These financial barriers are associated with the inconsistent cash flow of participants and unclear procurement approaches. Moreover, the adoption of advanced technologies, such as BIM and blockchain, in managing the cash flow of the OSC participants is yet to be addressed. Therefore, this research contributes to the body of knowledge by investigating the financial management and the appropriate contractual relationships (procurement approaches) in OSC projects. The study comprises four main objectives: 1) critically review the literature on construction cash flow management; 2) prioritize the criteria that govern the selection of procurement approaches in OSC projects; 3) Assess different procurement approaches according to the identified criteria; 4) develop an automated cash flow system that mitigates a number of OSC financial barriers.
A mixed review of literature on construction cash flow is adopted using scientometric and systematic review methods to fulfill the research's first objective. The scientometric analysis using science mapping has shown that more attention to the cash flow analysis has been gained in recent years and the United States and the United kingdom contributed most to cash flow research. The systematic analysis has resulted in creating five main categories, and integrating cash flow with schedule uncertainties was the dominant research theme. Consequently, five potential research frontiers are identified that motivated the current research. Two main models are developed to fulfill the rest of the research objectives. First, an MCDM model is developed to accurately assess the performance of various procurement approaches in OSC projects considering various criteria, i.e., project characteristics, cost, and schedule. The required data were collected through a questionnaire survey, while the analytic network process (ANP) and the evidential reasoning (ER) techniques were used to analyze the data. The model results indicate that the funding arrangements and the project quality are the prominent selection factors. On the other hand, the model reveals that the Integrated project delivery (IPD) and the construction management (CM) approaches have the highest utility score. The model results are verified by comparing them to similar studies.
Second, The result obtained from the MCDM model helped create the cash flow management model. The model integrates blockchain and BIM technologies to mitigate some identified OSC financial barriers considering the chosen procurement approach. 4D and 5D BIM models are developed to serve as information sources for the blockchain network. The network is established on the Hyperledger Fabric (HLF) platform using the Hyperledger Composer tool. In addition, a smart contract is developed to execute different transactions among the OSC participants. The developed system enables parties the make transaction securely, with no third-party involvement. The system also mitigates possible disputes by efficient documentation of the submitted claims with respect to a predefined endorsement policy. The developed model is validated by introducing a modular cable-stayed bridge case study. In addition, several scenarios are defined in the case study to test and validate the developed smart contract.
... Over recent years, AR has seen substantial advancements, such as improved frame rates, resolution, and wearability, making it an increasingly suitable platform for scientific experimentation and practical application [1,2]. AR has been applied in a diverse array of areas, including medical education and surgery [3,4], repair and maintenance [5][6][7], architectural design [8,9], cultural heritage preservation [10,11], and remote collaboration [12,13], amongst others. With regard to information presentation, AR typically overlays context-specific information [14,15] or enables the integration of digital elements in a real environment [16,17]. ...
The current study investigated a spectrum of control methods in human-robot interaction, ranging from direct control to telepresence with a virtual representation of the robot arm. Twenty-four participants used a setup comprising a Franka Emika Panda robot arm, Varjo XR-3 head-mounted display, and Leap Motion Controller. Participants performed a box-and-block task with the bare hand (A), and under five gesture-controlled robotic operation methods: direct sight (B), sight via video-feedthrough (C), in a 3D telepresence environment with (D) and without (E) virtual representation of the robot arm, and using a 2D video feed (F). The number of grabbing attempts did not differ significantly between conditions, but local operation (B & C) yielded more transferred blocks than teleoperation (D–F). Teleoperation using a 3D presentation was advantageous compared to using a 2D video feed, as demonstrated by lower peak forces and smaller range in gripper heights compared to the 2D video feed, findings supported by analyses of head movement activity. Finally, the bare hand yielded the best performance and subjective ratings. In summary, teleoperation using a 3D presentation provided a smoother interaction than a 2D video feed. However, direct human interaction remains a benchmark yet to surpass.
... (2) Construction site roaming Before the municipal infrastructure model of BIM is transformed into VR model, it can be displayed on PC directly, and help municipal infrastructure construction managers quickly understand the municipal infrastructure construction site (Song et al. 2021); After transforming the site layout model of municipal infrastructure construction in BIM database to VR, a roaming model of municipal infrastructure construction under virtual reality scene is formed. ...
In this paper, the digital management method of municipal infrastructure engineering construction is proposed, to improve the management level of municipal infrastructure engineering construction. After determining the project construction cycle, BIM technology is used to build the municipal infrastructure engineering model, screen the errors in the model through collision and roaming inspection, and use the DELMIA module of BIM software to realize the visual management of municipal infrastructure construction progress. BIM-VR conversion software is selected to convert the three-dimensional model of municipal infrastructure into a VR model, and immersively view the construction site of municipal infrastructure engineering through VR equipment, so as to realize the dynamic digital management of municipal infrastructure engineering construction. The experimental results show that this method can realize the digital management of municipal infrastructure engineering construction, accurately predict the project cycle of municipal infrastructure engineering construction, and ensure the smooth progress of municipal infrastructure engineering construction.
Purpose
This study explores practitioners’ perspectives on the perceived gap between university and practice beyond the hard and soft skill paradigm. Utilising Tomlinson’s graduate capital model of employability (2017), we explored human, social, cultural, and psychological capitals to enrich the understanding of this issue and employability. It provided a new perspective, useful for implementing curriculum renewal.
Design/methodology/approach
This study utilised a two-stage mixed methods design. Using Tomlinson’s (2017) Graduate capital model as a framework, the first stage involved distributing an online survey to qualified architects in hiring positions practising in Australia. This served as the foundation for generating qualitative and quantitative data. The second stage involved a two-hour practitioner workshop where the survey results were discussed and expanded upon.
Findings
Our results found that the practitioner’s perspective on the perceived skills gap is more complex than the hard/soft skill paradigm commonly discussed. Practitioners expressed a need for students/graduates to possess identity and cultural capital to contextualise industry norms and expectations. This knowledge lets students know where and how hard/soft skills are used. Our results also suggest practitioners are concerned with the prevailing individualistic approach to the higher education system and traditional architectural teaching methods, instead suggesting a more industry-aligned collaborative disposition.
Originality/value
By expanding the employability discourse beyond hard/soft skills, the results of this research provide an opportunity for architectural curriculum renewal in line with industry expectations.
Abstrak: Dalam perancangan arsitektural, kertas masih menjadi media utama untuk visualisasi desain, terutama melalui gambar teknis dan presentasi proyek. Namun, penggunaan kertas secara berlebihan berdampak negatif pada lingkungan akibat penebangan hutan dan limbah kertas, yang bertentangan dengan upaya global menuju keberlanjutan di sektor konstruksi. Penelitian ini bertujuan untuk mengeksplorasi potensi Augmented Reality (AR) sebagai media visualisasi yang lebih ramah lingkungan. Metodologi penelitian yang di impelementasikan menggabungkan pendekatan kualitatif dan eksperimen, pengabungan dua pendekatan ini akan menghasilkan hasil yang lebih maksimal. Hasil analisis dan eksperimen menunjukkan bahwa penggunaan AR tidak hanya efektif dalam mengurangi limbah kertas, tetapi juga meningkatkan efisiensi proses desain dan komunikasi. Teknologi AR memungkinkan pengguna menelusuri desain bangunan dengan lebih rinci, mengidentifikasi potensi masalah lebih awal, dan melakukan penyesuaian desain secara cepat. Di samping dampak positif terhadap lingkungan, penggunaan AR juga mempercepat proses pengambilan keputusan dan kolaborasi antar-stakeholder melalui pengalaman visual yang realistis dan interaktif. Hasil penelitian ini diharapkan dapat mendorong penerapan teknologi yang mendukung keberlanjutan dalam industri konstruksi, serta pelaku industri untuk mengadopsi metode visualisasi modern yang lebih efisien dan ramah lingkungan.
Materialization practices in the architecture and building industry have evolved with the advancement of manufacturing and information technologies. This evolution is evident across various design and production phases, with a pronounced impact on prototyping. Advances in design and fabrication tools have empowered prototypes, integral in any production cycle, to furnish a growing array of information and feedback for designers and manufacturers. In this context, prototypes have transformed from merely showcasing data-driven building solutions to presenting socio-environmentally conscious systems. Innovation in prototyping connects the initial design and construction stages to the operational phase, creating a seamless transition throughout the project lifecycle. This chapter provides a range of definitions and prototypical case studies for smart prototyping by identifying practiced approaches in integrated design to production workflows. This chapter introduces three paradigms for smart prototyping: Digital prototyping focuses on data-driven design for mass customization, phygital prototyping involves mixed-reality-enabled design and assembly, and thirdly collaborative prototyping explores human-machine hybrid intelligence and co-production in architectural and urban contexts. The chosen case studies in this chapter and how they are categorized aim to provide a comprehensive overview of smart prototyping, covering projects conducted in both research and practice. This chapter concludes with potential future trends and the role of emerging and evolving mediums of prototyping for smart design and construction.
Space planning, a crucial phase of architectural design, has significantly benefited from recent performance-based generative methods, enabling early discovery of innovative layouts among a wide range of alternatives. The application of generative design in open-office layout, where flexibility to support different work styles is essential, should become more practical. This paper fills gaps in automated space planning for office buildings, presenting a generative algorithm for robust circulation in multi-story modular geometries. By inputting a modular 3D model, the algorithm generates multiple interior configurations using a library of predefined modules adaptable to cube dimensions and client preferences. This facilitates extensive layout options, streamlining decision-making for efficient workstation and corridor designs. The use of a user-friendly visualization platform allows standard drawing modifications by architects with no programming knowledge. The algorithm's effectiveness is proven through application on various 3D geometries, showcasing its potential to improve practical office design.
The concept of Industry 4.0 in its initial phase referred primarily to revolutionary technological changes in the production environment. However, its huge success related to the increase in the efficiency and productivity of manufacturing processes has meant that this concept is more and more often transferred to production-related processes and supporting the basic activities of production and service enterprises. For this reason, the purpose of the chapter is to present the concept of Industry 4.0 and its impact on the changes in recent years in internal logistics and supply chain management. The chapter characterises the most important tools and technologies used in complex cyber-physical systems, outlining the assumptions for their application and the benefits and risks involved. The principles of implementing Industry 4.0 solutions and their impact on modifying internal processes and business models implemented in companies were also presented. The author also described the concept of Logistics 4.0 and the Industry 4.0 solutions that most often support internal logistics processes.
Unsafe behaviors and hazardous conditions in the workplace could pose serious risks resulting in major accidents. Safety training is critical for enhancing personal safety awareness and reducing accident severity and numbers. Augmented Reality (AR) is an emerging technology that superimposes digital information onto the real-world environment, which can offer an active, immersive, and interactive learning environment, making it a promising tool for safety training. However, to date, there is limited research that systematically evaluates the application and effectiveness of AR safety training.
This study aims to comprehensively explore the application and effectiveness of AR safety training solutions in various industries, excluding the military and medicine. To achieve this, a systematic analysis of 37 papers published between 2014 and February 2024 was conducted. Our analysis addresses eight research questions. The results indicate that existing AR applications cover thirteen domains, with the construction, manufacturing, and transportation industries being the most common domains. The results also indicate a limited focus on measuring knowledge retention in the selected studies. Additionally, there is a lack of comparison between different AR hardware and display technologies. Two meta-analyses were carried out and they reveal that AR technology has a significant positive impact on safety training. However, no significant difference in knowledge acquisition was observed between AR training and traditional training. These findings have implications for future research and applications of AR safety training.
Even though computational design has paved the way for the design of complex forms, construction is still difficult, inefficient, and costly. This paper aims to compile the most recent fabrication techniques for shell structures to create an overview of fabrication methods. This paper reviews existing, recently developed, and previously unexplored techniques for building and fabricating shell structures. The objective of this review is to find trends, patterns, or gaps in the literature that can direct future research. As a result, each fabrication technique is explained and demonstrated with relevant built projects. Moreover, each method's advantages, disadvantages, and limitations are highlighted. These techniques are grouped into four main classes and evaluated. In conclusion, categorizing fabrication techniques reveals the state-of-the-art in fabrication and highlights potential research gaps. Moreover, this research might encourage researchers to explore novel methods and solutions to overcome the current method's limitations.
The use of technology is spreading throughout the AEC (Architecture, Engineering and Construction) sector, involving all aspects of the construction process, from design and maintenance to the very way of teaching in the sector. The BIM (Building Information Modeling) methodology is aiding this transition, implementing the digital twin of the artifact as an always up-to-date information model. The optimization of the process and the reinforcement of interaction is the main objective of digital reconstruction, highlighting the importance of managing the integration of different disciplinary sub-models, considering both architectural, structural, and MEP (Mechanical, Electrical and Plumbing). Among the latest technologies that are being incorporated into the AEC sector, Augmented Reality offers great potential for interaction between the virtual and real world and 3D printing is generating a change in how we build our reality. This contribution aims to experiment the integration of BIM with AR and 3D printing applications for educational purposes specifically as a prototype to evaluate interoperability, data loss within the flow as well as the feasibility of using the result in university courses. The case study chosen for the application is an abandoned hospital structure in Roccaromana (Italy). A mixed survey (laser and photogrammetry) of the existing structure was carried out, constructing a project proposal exploiting the potential of the BIM methodology for the integration of different disciplines. Afterwards, to test the complete methodology, the focus was on the area of the toilets, with the aim of helping the students in the process of learning the plumbing systems through AR and 3D printing. The experiment had a positive impact on the students, who could better understand this discipline thanks both to the materic model, phisically observing spatial interference among pipes and architectural elements, and to the simulated flow, which allows the correct positioning of connections to be tested.
The primary aim of the study is to develop an augmented reality (AR) acceptance scale within the framework of the unified theory of acceptance and use of technology (UTAUT) model to measure individuals’ acceptance and use of AR technology. The study was performed with a total of 546 university students with three participant groups in the 2022–2023 academic year. Reliability and validity studies of the scale were also carried out. The face validity and content validity of the scale were investigated by receiving constructive feedback from various field experts. To scrutinize the construct validity, exploratory factor analysis (EFA) was applied to the first participant group (n = 315) while confirmatory factor analysis (CFA) was conducted with the second participant group (n = 182). The EFA results confirmed the four-factor structure with 21 items accounting for 71.86% of the total variance, while the CFA results showed that the four-factor model had a sufficient fit. The reliability of the UTUAT-based AR acceptance measurement tool was investigated using Cronbach’s alpha and test-retest methods. The Cronbach’s alpha coefficient of the scale was 0.95 whereas the test-retest reliability of the tool was 0.97. To investigate the discriminative power of the items, corrected item-total correlations and 27% upper and lower comparison of the participants were made. In conclusion, the UTUAT-based AR acceptance measurement tool is valid and reliable in measuring users’ AR acceptance and use levels.
This study aims to examine the relationship between mobile augmented reality, promotion-mix elements, destination
awareness, and intention to visit halal destinations. By using explanatory research, questionnaires were distributed online
(Google form) and offline to tourists visiting Karimunjawa, Indonesia. A total of 110 tourists participated in this study. Data were
analyzed using the Partial Least Square-structural equation model. This study found that mobile augmented reality had no
positive and significant effect on destination awareness and intention to visit halal destinations. Thus, Augmented reality has not
been able to become an important factor in destination awareness and intention to visit halal destinations. Conversely, destination
awareness has a positive and significant effect on the intention to visit halal destinations. Then, promotion-mix elements (PME)
also have a positive and significant influence on destination awareness and intention to visit halal destinations. This study also
found that destination awareness can mediate the relationship between promotion-mix elements and intention to visit halal
destinations. Theoretical and empirical implications are also highlighted in this study.
The study discusses the outcomes of the Atelier Design and Modeling course in the third-year Bachelor of Studies in Architecture at the Polytechnic of Turin. The course, a collaboration between Drawing and Architectural and Urban Composition disciplines, focuses on designing for an area in the Cenisia district in Turin. Emphasizing the connection between conception and representation, students are introduced to architectural and urban composition through physical and digital modeling, integrating AR technologies. The paper examines the hybridization between models, referencing similar educational experiences from other universities. Then delves into the relationship between physical and digital models, highlighting the interdisciplinary educational approach and the potential of different modeling techniques.KeywordsBachelor in architectureDigital modelPhysical modelAugmented realityRapid prototyping
Augmented reality (AR) and mixed reality (MR) technologies have gained significant interest throughout the past two decades in the Architecture, Engineering, and Construction (AEC) Industry. However, despite the rapid growth of these technologies, their effective implementation in the AEC industry is still in its infancy. Therefore, a comprehensive investigation of the state-of-the-art applications and categories of AR/MR in the construction industry can guide researchers and industry experts to choose the most suitable AR/MR solution for research and implementation. This paper provides a comprehensive overview of 103 AR/MR articles published in credible journals in the field of the AEC industry within the years 2013–2021. Typically, review-type papers assess articles primarily based on their application areas. However, this classification approach overlooks some other critical dimensions, such as the article’s technology type, the maturity level of technology used in the research, and the project phase in which technology is implemented. Accordingly, this paper classifies articles based on ten dimensions and their relevant categories: research methodology, improvement focus, industry sector, target audience, project phase, stage of technology maturity, application area, comparison role, technology type, and location. The results reveal that AR/MR literature has increasingly focused on simulation/visualization applications during construction and maintenance/operation phases of the project, emphasizing improving the performance of workers/technicians. Additionally, the increasing trend in AR/MR articles was identified as using self-contained headsets (e.g., Microsoft HoloLens). Markerless tracking systems show a significant trend among the articles. Moreover, the target location of implementing AR/MR primarily found to be in on-site and in outdoor spaces. The trend indicates an increase in immersive and mobile AR/MR applications in outdoor job sites such as construction sites to aid workers/technicians in assembly works during the construction phase.KeywordsAugmented and mixed technologiesConstruction
This article presents a survey on extended reality (XR) application to architecture, engineering, and construction (AEC). A systematic literature review (SLR) of studies found in the Scopus digital library is carried out for that purpose. The SLR comprises 983 relevant papers published between 2011 and 2022. It frames and aggregates research by the AEC field and by the usage of XR technologies. With that aim, specific taxonomies are developed and applied. Research typologies, including used approaches and contributions, are also examined. Furthermore, the limitations and challenges cited in the analyzed studies are addressed. This SLR is primarily addressed to researchers for decision-making in identifying and scoping future research activities. It is also directed to AEC practitioners as a source to determine the conditions in which XR can be used and what its implications are.
This paper presents an experimental performative design workflow for masonry structures, incorporating augmented reality (AR) technology and real-time stability simulation. We aim to resolve the lack of structural stability simulation in currently existing immersive design workflows. Our workflow consists of two phases: a) an AR design phase, in which interactive inputs, such as gestures, screen-based inputs, or marker recognitions, are translated to a design algorithm for the masonry structure generation and preview as holograms; b) an AR-assisted structural stability simulation phase, in which the designed structure is simulated on its structural performance attributes (without mortar or other adhesives) and shown as an AR holographic result animation on-site with the structurally optimised feedback. Our research findings highlight the advantages of the proposed workflow, which allows users modification suggestions and design implementation control for the assembly of structurally stable masonry structures, as well as the challenges arising regarding design diversity.
KeywordsAugmented Reality (AR)Immersive DesignStructural Stability Simulation
Augmented Reality (AR) as a new technical tool has developed rapidly in the last few years and has now the potential of bridging the gap between holographic drawings and the real world. This paper addresses whether AR can guide unskilled labour on complex structure assembly and fabrication process. It contains three experiments developed with AR. The research aims to prove that with intuitive holographic instructions, AR helps to reduce the time spent in comparing 2D drawings to the real site during the assembly process, and therefore offers possibilities to improve the construction efficiency significantly. The research also paves the way for shell structures, considering the latest technology such as AR and AI, and gives emphasis on the communication between computer and human during the fabrication process through the physical model. It is an exploration of how people might change their mind or decisions can be changed in a real-time manner harmoniously using AI through AR.
The paper presents a mixed reality-based approach for interactive control of robotic manipulators and mobile platforms. In particular, we designed an interactive and understandable interface for human-robot interaction. The interface provides tools for robot path programming and visualizes it. Path visualization helps workers understand robot behavior, it is important for safety human-robot interaction. The paper presents an architecture of that system and the implementation for an industrial robot KUKA iiwa and mobile robot platform Plato. The main issue of a multi-platform system is related to the synchronization of coordinate frames for all elements. To deal with this problem we realized 3 setting options: manually, by a camera with markers, point clouds processing. We implemented our interface on Microsoft HoloLens and evaluated it on users.
This paper discusses the integration of Mixed Reality in the design and implementation of non-standard architecture. It deliberates a method that does not require conventional 2D drawings, and the need for skilled labor, by using the aid of holographic instructions. Augmented Construction allow builders to execute complex tasks and to understand structural relations intuitively by overlaying digital design information onto their field of view on the building site. This gives the implementation system authors different levels of control. As a proof of concept, a group of non-professionals reconstructed the south wall of Corbusier’s Ronchamp chapel, the Notre-Dame du Haut, at scale 1:5 using no architectural 2D drawings but only custom-built Augmented Reality apps for HoloLens and mobile devices. This project focused on the assembly of non-standard prefabricated elements, based on an optimized parametric structure that enables designers to integrate imprecision within the construction phases into the design through a constant feedback-loop between the real and the digital. The setup was designed in a non-linear process that allows the integration of new information during the Augmented Construction phases. The paper evaluates applied Augmented Construction for further improvements and research and concludes by discussing the impact potential of Augmented Construction on architectural design, socio-cultural, and economical levels.
Irregular materials such as unprocessed rocks and wooden branches are currently rarely used in contemporary architectural construction due to the high cost of skilled labor required for preparation. This project investigates methods of graphic statics in combination with technologies such as photogrammetry and 3D printing to build structures from natural reusable building materials. We developed a method for dry-stacking unprocessed rock. We implemented an algorithm to produce 3D printed parts that connect digitized stones in the plane normal to the thrust line. A manually designed input curve is recalculated according to the weights of the objects to generate a compression-only arch. We employed evolutionary optimization to calculate optimal orientation of digitized stones along the thrust-line. Different stone arrangements and 3D printed connectors were tested. The manual assembly is done using a mixed reality setup. Unprocessed or rubble rock is a massive resource of cheap building material with very low embodied energy that might see a revival through digital technologies.
This paper presents a three-part investigation in the use of off-loom weaving and augmented reality (AR) in the construction of architectural building components and assemblies. Tim Ingold’s model of morphogenesis, a conception of form emerging from the interplay of people, tools, and materials, guides the ambitions of the investigation. The goal is to develop architectural spaces not from pre-determined drawings, but through an emergent process of collaborative exchange. AR is explored as an instrument to facilitate this objective.
The investigation draws on material science studies on cementitious composite technologies, computational design and construction methods of off-loom weaving, and methods of construction using AR headsets. From these it develops an expanded application for cement composites to make free-standing structures, a collaborative building method using off-loom weaving for architecture, and a method using AR headsets to assist off-loom weaving. These methods and applications facilitate the development of form through expert and non-expert exchange, building through gesture, and interacting with materials. Further advancement of the methodology requires applying a technique to fluidly input geometric coordinates of physical form into a virtual environment.
Under the circumstances of a patient's upper limb disability, aided by a robotic arm with faulty controls, assistance is needed, using augmented reality as an auxiliary. Our system, with a headset, using an internet connection and an augmented reality device, placed on the assistant's head, can ensure communication between the two, for both remote supervision and control. The assistant can enhance the control over the robotic arm, while having a head up display on the augmented reality glasses, based on what the patient sees. The communication is established through PC or mobile devices, connected to the internet. Having the patient's view, and enhanced control over the robotic arm, the assistant can interact with nearby smart objects.
This paper examines a novel, integrated and collaborative approach to design and fabrication, enabled through Mixed Reality. In a bespoke fabrication process, the design is controlled and altered by users in holographic space, through a custom, multi-modal interface. Users input is live-streamed and channeled to 3D modelling environment,on-demand robotic fabrication and AR-guided assembly. The Holographic Interface is aimed at promoting man-machine collaboration. A bespoke pipeline translates hand gestures and audio into CAD and numeric fabrication. This enables non-professional participants engage with a plethora of novel technology. The feasibility of Mixed Reality for architectural workflow was tested through an interactive installation for the makeCity Berlin 2018 festival. Participants experienced with on-demand design, fabrication an AR-guided assembly. This article will discuss the technical measures taken as well as the potential in using Holographic Interfaces for collaborative design and on-site fabrication.Please write your abstract here by clicking this paragraph. INTRODUCTION In this paper we argue that AR can improve man-machine collaboration in architecture making. Holo-graphic, multimodal interfaces simplify access to cutting-edge technology for the all-user. Human operators can edit digital information with using intuitive interfaces, for CAD/CAM. Leveraging human capabilities to communicate vie speech and hand-gestures allows non-professionals easy access to 3D modelling and robotic fabrication. We examined AR representation and multimodal interface potential in the inclusion of non-professional in architectural design and making, human-machine collaboration and multi-participant design. This approach was tested through a fully integrated cycle of holographic 3D modelling, on-demand robotic machining and AR-guided assembly that resulted in a collaborative installation for the MakeCity Festival 2018 in Berlin.
Digital fabrication technologies, such as 3D printers, are receiving more and more attention, not only from professionals but also from hobbyists. However, even though people have easier access to these devices, 3D printers remain a black box for many users. To support the appropriation of 3D printers, this demonstration presents a system which extends a physical printer to include virtual components using augmented reality (AR). With these components, we try to explain how the printer works and allow the user to operate the printer through an AR application. We extend existing software with a custom solution to create a unique user interface and user experience. Our user interface provides a new way of inspecting models in AR before they are printed.
In this paper, we present a set of enabling technologies developed for the KUKA Innovation Award to facilitate Human Robot collaboration targeted for the architecture, engineering and construction (AEC) sector. Critically, little progress has been made in the usability of user interfaces for industrial robots [1]. We targeted our investigation explicitly towards human-robot collaboration (HRC) in wood based prefabrication production. Although wood is a sustainable material with abundant processing possibilities, it is also a material system where process knowledge remains critically important, making highly automated workflows unfeasible and inefficient. We propose an interactive fabrication process where a user such as a construction worker could wear an augmented reality head mounted display (ARHMD) as an interface to plan robotic trajectories, influence production sequencing, and view superimposed diagnostic feedback. We describe necessary system components including a robotic workcell consisting of a KUKA LBR iiwa, flexFELLOW mobile platform, a Robotiq 2-finger gripper and a custom platform and material feeding station. In addition, we describe a communication framework and set of protocols connecting a CAD digital design environment, a user interface (UI) for Microsoft HoloLens, a ROS server for backend path planning and coordination, and a 3D graphical web interface for downstream visualization of construction status. We conclude with an outlook on enabling technologies for human-robot collaboration in construction, and the importance of increasing digital integration and accessibility in characteristic production workflows through accessible and intuitive digital interfaces.
Nowadays digital tools support architects, engineers and constructors in many specific tasks in the construction industry. While these tools are covering almost all aspects of design and manufacturing, the planning and design for the assembly of buildings remain an unexplored area. This research aims to lay the foundations of a new framework for the design for assembly in architectural applications entitled Assembly Information Modeling. In practice, it is a central digital model containing the structure architectural design, construction details, three dimensional representations, assembly sequences, issue management and others. This framework forms the base for a multitude of novel applications for assembly design, planning and execution, such as assembly simulation and strategies communication, problem detections in the early design phases and interdisciplinary coordination. This paper describes the specifications of the digital assembly model and illustrate two use cases: collaborative assembly design using AEC cloud-based platforms and Augmented Assembly using Augmented reality devices.
The digital revolution has instigated numerous changes to the architectural design process, which have distanced physical and intuitive material exploration from the standard procedures and protocol of the discipline. By combining augmented reality technologies with real-time computer simulation, sensory feedback and robotic fabrication tools, new workflows enable the architect to design spontaneously and intuitively with seemingly stochastic material processes while managing the complex performance criteria associated with ‘highly informed’ design. This paper presents a prototypical design process to these ends and discusses this approach and its implications in relation to alternative workflows as practised in architectural design and fabrication.
In recent years, the field of virtual reality (VR) has shown tremendous advancements and is utilized in entertainment, scientific research, social networks, artistic creation, as well as numerous approaches to employ VR for psychotherapy. While the use of VR in psychotherapy has been widely discussed, little attention has been given to the potential of this new medium for art therapy. Artistic expression in VR is a novel medium which offers unique possibilities, extending beyond classical expressive art mediums. Creation in VR includes options such as three-dimensional painting, an immersive creative experience, dynamic scaling, and embodied expression. In this perspective paper, we present the potentials and challenges of VR for art therapy and outline basic principles for its implementation. We focus on the novel qualities offered by this creative medium (the virtual environment, virtual materials, and unreal characteristics) and on the core aspects of VR (such as presence, immersivity, point of view, and perspective) for the practice of art therapy.
The demands on companies caused by the markets are becoming more and more fast-moving and complex. Reasons are the increasing number of variants of products as well as reduced product life cycle times. This is particularly relevant for assembly tasks, since a very high flexibility and adaptability to varying ambient condition must be ensured in this area. Therefore, a lot of steps are currently being carried out manually. However, if an automation is attempted in the field of assembly, companies are often facing a trade-off between a high degree of automation and flexibility of the production system. A concept to increase the automation rate in assembly tasks can be seen in the use of collaborative robots, so that the benefits of both, humans and robots, can be combined to accomplish the task. In doing so, one key issue is to simplify and thus to accelerate the programming process so that the necessary programming skills of employees can be reduced. Although some of today’s collaborative robots already offer good programming approaches like kinesthetic teaching, this article introduces a new and more intuitive programming method which is based on Augmented Reality. For this purpose, components of an assembly group are virtually linked with CAD models by using optical markers. The operator can then virtually assemble the components according to the assembly sequence. Results of first tests indicate, that once the assembly process is recorded, the robot can accomplish the assembly in reality. Finally, possibilities for future developments are presented.
We present the Robotic Modeling Assistant (RoMA), an interactive fabrication system providing a fast, precise, hands-on and in-situ modeling experience. As a designer creates a new model using RoMA AR CAD editor, features are constructed concurrently by a 3D printing robotic arm sharing the same design volume. The partially printed physical model then serves as a tangible reference for the designer as she adds new elements to her design. RoMA's proxemics-inspired handshake mechanism between the designer and the 3D printing robotic arm allows the designer to quickly interrupt printing to access a printed area or to indicate that the robot can take full control of the model to finish printing. RoMA lets users integrate real-world constraints into a design rapidly, allowing them to create well-proportioned tangible artifacts or to extend existing objects. We conclude by presenting the strengths and limitations of our current design.
This paper describes the implementation of a discrete in situ construction process using a location-aware mobile robot. An undulating dry brick wall is semi-autonomously fabricated in a laboratory environment set up to mimic a construction site. On the basis of this experiment, the following generic functionalities of the mobile robot and its developed software for mobile in situ robotic construction are presented: (1) its localization capabilities using solely on-board sensor equipment and computing, (2) its capability to assemble building components accurately in space, including the ability to align the structure with existing components on site, and (3) the adaptability of computational models to dimensional tolerances as well as to process-related uncertainties during construction. As such, this research advances additive non-standard fabrication technology and fosters new forms of flexible, adaptable and robust building strategies for the final assembly of building components directly on construction sites. While this paper highlights the challenges of the current state of research and experimentation, it also provides an outlook to the implications for future robotic construction and the new possibilities the proposed approaches open up: the high-accuracy fabrication of large-scale building structures outside of structured factory settings, which could radically expand the application space of automated building construction in architecture.
Wearable computers and Generation-5 Digital Eye Glass easily recognize a user's own gestures, forming the basis for shared AR (Augmediated Reality). This Studio-workshop presents the latest in wearable AR, plus an historical perspective with new insights. Participants will sculpt 3D objects using hand gestures and create Unity 3D art+game objects using computational lightpainting.
Participants will also learn how to use 3D gesture-based AR to visualize and understand real-world phenomena, including being able to see sound waves, see radio waves, and see sight itself, through abakographic user-interfaces that interact with "sightfields" (time-reversed lightfields). Participants will also surveilluminescent devices that change color when watched by a camera. Long exposure photographs made with such devices generate "sightpaintings" that show what a camera can "see".
Recently building information models have substantially improved the explicit semantic content of design information. Information models are used to integrate the initial phases of project development. On the construction site, however, the designs are still mostly represented as line-based paper drawings or projections on portable displays. A generic technology that can integrate information and situate it in time, place and context is augmented reality. The specific research issues addressed are (1) does augmented reality have a potential use in civil engineering, (2) how big – in comparison to other technologies - is this potential and (3) what are the main barriers to its adoption. The generic research issue was to develop a methodology for evaluation of potentials of technology. A prototype was built. It was tested on a real construction site to evaluate the potential of its use using the action-research method. A set of structured interviews with potential users was then conducted to compare the prototype to conventional presentation methods. Using this methodology it has been found out that augmented reality is expected to be as big a step as the transition from 2D line drawings to photorealistic 3D projections. The main barrier to the adoption is immature core virtual reality technology, conservative nature of construction businesses and size of building information models.
Robot controller user interface is one of the most technical challenging issues for robot manufacturers because of human presence as an actor in this interaction. Many developments made for having intuitive and easy interaction between user and robot controller while complexity of the robot systems is continuously increasing.
This 1992 USAF report introduces the concept of Virtual Fixtures (i.e., Mixed Reality overlays) to enhance human performance in real world dexterous tasks. Tools and fixtures in the physical world (e.g., a ruler guiding a pencil) enhance human performance by guiding manual operations, providing localizing references, reducing mental workload, and increasing precision. Virtual fixtures are computer-generated objects overlaid on top of a real workspace that are shown to provide similar benefits. Because these perceptual overlays are virtual construction they can be diverse in modality, abstract in form, and custom tailored to individual situations. This study investigates the potential of virtual fixtures as perceptual aids in a standardized Fitts Law peg-insertion task. Subjects viewed the remote environment through a vision system while wearing an upper-body exoskeleton. The Eight subjects used a telerobotic system to perform standard Fitts' law peg-insertion tasks with and without the aid of a variety of virtual fixtures. Simple virtual objects overlaid on a real workspace where found powerful perceptual aids, increasing performance up to 70%; a workstation environment is proposed in the report to allow users to design and implement their own virtual fixtures interactively to enhance real-world tasks. (Keywords: Virtual Fixtures, Mixed Reality, Haptics, Telepresence, Telerobotics, Virtual Reality, Augmented Reality, Spatial Audio, AR, VR, MR, HCI ) Technical Report AL-TR-0089, USAF 1992.
Augmented Reality is a technology that integrates pictures of virtual objects into images of the real world. Besides the technical problems still to be solved, for industry to be interested in this tech-nology the additional amount of work needed to use the technology in relation to the benefit has to be visible. Furthermore the question how this integrates into the information technology infras-tructure of the company is important. This paper describes an Augmented Reality demonstrator for the task of doorlock assembly into a car door that was developed trying to create a practical, realis-tic application that can transport the concepts be-hind Augmented Reality to a casual observer. To reach that goal a new fast and robust optical track-ing algorithm was developed and integrated into a three-dimensional animation and rendering system, creating a real-time fully three-dimensional HMD-based training application showing how to assemble the doorlock into the door. The system was demon-strated to the general public at the Hannover In-dustrial Fair 1998 and this demonstration of Aug-mented Reality for one of the first times to a large non-expert audience created a lot of interest into this new area.
First construction robots had been designed in the beginning seventies in order to increase the quality in prefabrication
of modular homes in Japan and the late 70ies planning started for use of robots in construction sites. In the 80ies the first
construction robots appeared on sites and in the 90ies integrated automated building construction sites had been developped
and implemented about 20 times. Furthermore maintenance robots for cleaning and inspection of buildings, infrastructure and
real estate and safety robots guarding buildings had been developped. In the first decade of this century humanoid construction
robots had been tested. In the future service robots will be a big market in the built envrironment.
Robot PbD started about 30 years ago, growing importantly during the past decade. The rationale for moving from purely preprogrammed robots to very flexible user-based interfaces for training the robot to perform a task is three-fold. First and foremost, PbD, also referred to as {\em imitation learning} is a powerful mechanism for reducing the complexity of search spaces for learning. When observing either good or bad examples, one can reduce the search for a possible solution, by either starting the search from the observed good solution (local optima), or conversely, by eliminating from the search space what is known as a bad solution. Imitation learning is, thus, a powerful tool for enhancing and accelerating learning in both animals and artifacts. Second, imitation learning offers an implicit means of training a machine, such that explicit and tedious programming of a task by a human user can be minimized or eliminated (Figure \ref{fig:what-how}). Imitation learning is thus a ``natural'' means of interacting with a machine that would be accessible to lay people. And third, studying and modeling the coupling of perception and action, which is at the core of imitation learning, helps us to understand the mechanisms by which the self-organization of perception and action could arise during development. The reciprocal interaction of perception and action could explain how competence in motor control can be grounded in rich structure of perceptual variables, and vice versa, how the processes of perception can develop as means to create successful actions. PbD promises were thus multiple. On the one hand, one hoped that it would make the learning faster, in contrast to tedious reinforcement learning methods or trials-and-error learning. On the other hand, one expected that the methods, being user-friendly, would enhance the application of robots in human daily environments. Recent progresses in the field, which we review in this chapter, show that the field has make a leap forward the past decade toward these goals and that these promises may be fulfilled very soon.
The recently introduced robotic surgical systems were developed to overcome the limitations of conventional minimally invasive surgery. We analyse the impact of the da Vinci robotic system on general surgery.
The da Vinci operating robot is a telemanipulation system consisting of a surgical arm cart, a master console and a conventional monitor cart. Since its purchase in June 2001, 128 patients have undergone surgery using the da Vinci robot in our department. The mean age of the 78 female and 50 male patients was 52 (range 18-78) years.
The procedures included 29 cholecystectomies, 16 partial fundoplications, 16 extended thymectomies, 14 colonic interventions, 10 splenectomies, 10 bariatric procedures, 7 hernioplasties, 6 oesophageal interventions, 5 adrenalectomies, 5 lower lobectomies, 4 neurinomectomies and 6 others. 122 of 128 procedures (95%) were completed successfully with the da Vinci robot. Open conversion proved necessary in 4 patients due to surgical problems, and two other procedures were completed by conventional laparoscopy due to robot system technical errors. 30-day mortality was 0%, one redo-operation was necessary and two lower complications not requiring surgical re-intervention occurred. The resection margins of all tumour specimens were histologically tumour free.
Various general surgical procedures have proved feasible and safe when performed with the da Vinci robot. The advantage of the system is best seen in tiny areas difficult of access and when dissecting delicate, vulnerable anatomical structures. However, in view of longer operating times, higher costs and the lack of adequate instruments, robotic surgery does not at the moment represent a general alternative to conventional minimally invasive surgery.
In this paper, we present the user interface and the system architecture of an Internet-based telelaboratory, which allows researchers and students to remotely control and program two educational online robots. In fact, the challenge has been to demonstrate that remote programming combined with an advanced multimedia user interface for remote control is very suitable, flexible, and profitable for the design of a telelaboratory. The user interface has been designed by using techniques based on augmented reality and nonimmersive virtual reality, which enhance the way operators get/put information from/to the robotic scenario. Moreover, the user interface provides the possibility of letting the operator manipulate the remote environment by using multiple ways of interaction (i.e., from the simplification of the natural language to low-level remote programming). In fact, the paper focuses on the lowest level of interaction between the operator and the robot, which is remote programming. As explained in the paper, the system architecture permits any external program (i.e., remote experiment, speech-recognition module, etc.) to have access to almost every feature of the telelaboratory (e.g., cameras, object recognition, robot control, etc.). The system validation was performed by letting 40 Ph.D. students within the "European Robotics Research Network Summer School on Internet and Online Robots for Telemanipulation" workshop (Benica`ssim, Spain, 2003) program several telemanipulation experiments with the telelaboratory. Some of these experiments are shown and explained in detail. Finally, the paper focuses on the analysis of the network performance for the proposed architecture (i.e., time delay). In fact, several configurations are tested through various networking protocols (i.e., Remote Method Invocation, Transmission Control Protocol/IP, User Datagram Protocol/IP). Results show the real possibilities offered by these remote-programming techniques, in order to design experiments intended to be performed from both home and the campus.
Wearable computers and Generation-5 Digital Eye Glass easily recognize a user's own gestures, forming the basis for shared AR (Augmediated Reality). This Studio-workshop presents the latest in wearable AR, plus an historical perspective with new insights. Participants will sculpt 3D objects using hand gestures and create Unity 3D art+ game objects using computational lightpainting
Although modern software has paved the way for architects to design complex forms, such as free-forms, construction remains challenging, costly, and time-consuming which requires skilled workers. Advanced digital fabrication technologies can offer new ways to fill the gap between design and construction. Augmented Reality (AR) technology is one such technology that has many potentials in various fields, however, its capabilities are not sufficiently explored yet, especially in the field of digital fabrication. This study presents a new affordable interactive multi-marker augmented reality tool for constructing free-form modular surfaces implemented by integrating common accessible devices. The proposed tool consists of two digital cameras, a head-mounted display, a processor, and two markers that enable the user to virtually see the accurate location of any proposed object in the real world. A controlling subsystem was also designed to enhance the accuracy of construction. Method efficiency was studied in five full-scale prototypes. The results showed that the majority of errors (91%) were less than 6 mm, and 2° for lateral placements and orientation errors.
Construction is a high hazard industry which involves many factors that are potentially dangerous to workers. Safety has always been advocated by many construction companies, and they have been working hard to make sure their employees are protected from fatalities and injuries. With the advent of Virtual and Augmented Reality (VR/AR), there has been a witnessed trend of capitalizing on sophisticated immersive VR/AR applications to create forgiving environments for visualizing complex workplace situations, building up risk-preventive knowledge and undergoing training. To better understand the state-of-the-art of VR/AR applications in construction safety (VR/AR-CS) and from which to uncover the related issues and propose possible improvements, this paper starts with a review and synthesis of research evidence for several VR/AR prototypes, products and the related training and evaluation paradigms. Predicated upon a wide range of well-acknowledged scholarly journals, this paper comes up with a generic taxonomy consisting of VR/AR technology characteristics, application domains, safety scenarios and evaluation methods. According to this taxonomy, a number of technical features and types that could be implemented in the context of construction safety enhancement are derived and further elaborated, while significant application domains and trends regarding the VR/AR-CS research are generalized, i.e., hazards recognition and identification, safety training and education, safety instruction and inspection, and so on. Last but not least, this study sets forth a list of gaps derived from the in-depth review and comes up with the prospective research works. It is envisioned that the outcomes of this paper could assist both researchers and industrial practitioners with appreciating the research and practice frontier of VR/AR-CS and soliciting the latest VR/AR applications.
In recent years, extensive research in the HCI literature has explored interactive techniques for digital fabrication. However, little attention in this body of work has examined how to involve and guide human workers in fabricating larger-scale structures. We propose a novel model of crowdsourced fabrication, in which a large number of workers and volunteers are guided through the process of building a pre-designed structure. The process is facilitated by an intelligent construction space capable of guiding individual workers and coordinating the overall build process. More specifically, we explore the use of smartwatches, indoor location sensing, and instrumented construction materials to provide real-time guidance to workers, coordinated by a foreman engine that manages the overall build process. We report on a three day deployment of our system to construct a 12-tall bamboo pavilion with assistance from more than one hundred volunteer workers, and reflect on observations and feedback collected during the exhibit.
As robots enter the human environment and come into contact with inexperienced users, they need to be able to interact with users in a multimodal fashion—keyboard and mouse are no longer acceptable as the only input modalities. In this paper we introduce a novel approach for programming robots interactively through a multimodal interface. The key characteristic of this approach is that the user can provide feedback interactively at any time—during both the programming and the execution phase. The framework takes a three-step approach to the problem: multimodal recognition, intention interpretation, and prioritized task execution. The multimodal recognition module translates hand gestures and spontaneous speech into a structured symbolic data stream without abstracting away the user’s intent. The intention interpretation module selects the appropriate primitives to generate a task based on the user’s input, the system’s current state, and robot sensor data. Finally, the prioritized task execution module selects and executes skill primitives based on the system’s current state, sensor inputs, and prior tasks. The framework is demonstrated by interactively controlling and programming a vacuum-cleaning robot. The demonstrations are used to exemplify the interactive programming and the plan recognition aspect of the research.
Research on augmented reality has mostly explored technical options; thus, currently, there is a limited understanding of human experiences with digitally augmented environments in mobile devices. Through the reviews of related works, this paper proposes a novel concept of context immersion, with the key features emphasizing the potential of augmented reality embodied in smart phones. The framework of the context immersion is structured around a combination of three dimensions, time and location-based context, object-based context and user-based context, where users can experience immersion in augmented environments, but still be connected to a real context in the physical world. A customized questionnaire was developed based on the factors of the context immersion, and the experience of creating a digital exhibition was presented as a case study to illustrate the utility of context immersion as an analytical tool for mobile augmented reality applications. Through understanding the user experience in mobile digital exhibitions, crucial components of context immersion were identified for the design and implementation of mobile augmented reality. Finally, implications of the results and the direction for the development of mobile augmented reality are presented.
Human–robot interaction in industrial robotics has largely been confined to finding better ways to reconfigure or program the robots. In this paper, an Augmented Reality based (RPAR-II) system is proposed to facilitate robot programming and trajectory planning considering the dynamic constraints of the robots. Through the various simulation capabilities provided in the proposed AR environment, the users are able to preview the simulated motion, perceive any possible overshoot, and resolve discrepancies between the planned and simulated paths prior to the execution of a task. By performing the simulation, the performance of the trajectory planning and the fitness of the selection of the robot controller model/parameters in the robot programming process can be visually evaluated. Practical issues concerning the system implementation are also discussed.
The NASA Johnson Space Center is developing a series of prototype flight test vehicles leading to a functional Crew Return Vehicle (CRV). The development of these prototype vehicles, designated as the X-38 program, will demonstrate which technologies are needed to build an inexpensive, safe, and reliable spacecraft that can rapidly return astronauts from onboard the International Space Station (ISS) to earth. These vehicles are being built using an incremental approach and where appropriate, are taking advantage of advanced technologies that may help improve safety, decrease development costs, reduce development time, as well as outperform traditional technologies. This paper discusses the creation of real-time 3-D displays for flight guidance and situation awareness for the X-38 program. These displays feature the incorporation of real-time GPS position data, three-dimensional terrain models, heads-up display (HUD), and landing zone designations. The X-38 crew return vehicle is unique in several ways including that it does not afford the pilot a forward view through a wind screen, and utilizes a parafoil in the final flight phase. As a result, on-board displays to enhance situation awareness face challenges. While real-time flight visualization systems limited to running on high-end workstations have been created, only flight-rated Windows are available as platforms for the X-38 3-D displays. The system has been developed to meet this constraint, as well as those of cost, ease-of-use, reliability and extensibility. Because the X-38 is unpowered, and might be required to enter its landing phase from anywhere on orbit, the display must show, in real-time, and in 3 dimensions, the terrain, ideal and actual glide path, recommended landing areas, as well as typical heads-up information. Maps, such as aeronautical charts, and satellite imagery are optionally overlaid on the 3-D terrain model to provide additional situation awareness. We will present a component-based toolkit for building these displays for use with the Windows operating systems.
A multimodal teaching advisor (MTA) has been developed for sensor-enhanced robotic systems used in manufacturing. The MTA utilizes the work-site operator’s know-how and robotic-systems information, including that from sensors, in a complementary manner. This system integrates information from various sources to carry out robotic tasks and presents this information through user interfaces that are easily understood by the operator. That is, the system integrates task specifications including tolerances, system constraints, sensory information, and operator- and robot-related information. The synthesized information is then presented in realtime to the operator at the work-site through a mobile multimodal interface. Experimental results show that this system can significantly improve total robotic system performance by ensuring a high-quality teaching task.