Figure 7 - uploaded by Giovanni Betti
Content may be subject to copyright.
Source publication
This paper examines a novel, integrated and collaborative approach to design and fabrication, enabled through Mixed Reality. In a bespoke fabrication process, the design is controlled and altered by users in holographic space, through a custom, multi-modal interface. Users input is live-streamed and channeled to 3D modelling environment,on-demand r...
Similar publications
Microdisplay is a key technology for realizing augmented reality (AR) and mixed reality (MR) devices, which have attracted much attention of late. Even though the operating voltage in the tandem structure is higher than that in the single structure, 2-stack tandem OLED exhibited 20,000 cd/m² at 9 V, which is compatible with CMOS circuit driving. Du...
Materials that obstruct the path of acoustic waveforms in free-field to the human ear, may introduce distortions that can modify the natural Head-Related Transfer Functions. In this paper, the effect of wearing commercially available Head-Mounted Displays for Mixed and Augmented Reality has been measured via a dummy head mannequin. Such spectral di...
Citations
... With the installation Pop-Up Factory, realised for the festival Make. City Berlin in 2018 [12], we aimed at creating a richer design and assembly experience. To allow for more complex and nuanced interactions between NEROs and robots, we decided to explore the use of Augmented Reality (AR) devices such as the Microsoft HoloLens. ...
The introduction of robotic construction methods in the building industry holds great promise to increase the stagnating productivity of the construction industry and reduce its current carbon footprint, in considerable part caused by waste and error in construction. A promising aspect is the implementation of integrated CAD/CAM processes where the design intent -expressed in CAD- is directly translated in machine instructions (CAM) without loss of information or need for intermediate translation and refactoring. As the introduction of robotics will hardly spell the disappearance of human workers, a key challenge will be orchestrating human-machine collaboration in and around the construction site or fabrication plant. Our contribution presents explorations in this field. Key to our approach is the investigation of the above-mentioned questions in conjunction with Mixed Reality (MR) interfaces to give access to both human and machine workers to the same dynamic CAD model and assembly instructions. This paper describes our work in this field through two artistic installations and ongoing research on additive manufacturing enabled construction. We probe principles for precise, customised, efficient, almost waste-free and just-in-time productions in the construction sector.
... There are also studies where mixed-reality tools and industrial robots were used together in robotic-fabrication applications. In a robotic wire-cutting-application study, the Styrofoam pieces were produced using an industrial robot and they were knitted using the mixed-reality device [14]. In a study of knitting wooden sticks, an industrial robot was used to notch the joints of wooden sticks, and the mixed-reality device was used during the knitting of wooden sticks [15]. ...
In this study, a method, in which parametric design and robotic fabrication are combined into one unified framework, and integrated within a mixed reality environment, where designers can interact with design and fabrication alternatives, and manage this process in collaboration with other designers, is proposed. To achieve this goal, the digital twin of both design and robotic fabrication steps was created within a mixed-reality environment. The proposed method was tested on a design product, which was defined with the shape-grammar method using parametric-modeling tools. In this framework, designers can interact with both design and robotic-fabrication parameters, and subsequent steps are generated instantly. Robotic fabrication can continue uninterrupted with human–robot collaboration. This study contributes to improving design and fabrication possibilities such as mass-customization, and shortens the process from design to production. The user experience and augmented spatial feedback provided by mixed reality are richer than the interaction with the computer screen. Since the whole process from parametric design to robotic fabrication can be controlled by parameters with hand gestures, the perception of reality is richer. The digital twin of parametric design and robotic fabrication is superimposed as holographic content by adding it on top of real-world images. Designers can interact with both design and fabrication processes both physically and virtually and can collaborate with other designers.
... This SDK is used in the literature because of its simplicity of use, which is another important factor [14]. It enables navigation between the various CAD/CAM phases and acts as a reference during the assembly step [15]. The Fologram SDK recognizes user gestures, screen taps, device position, it has a customizable interface on mobile phones and HoloLens, and the Fologram SDK offers the opportunity to interact with Grasshopper in the AR environment [11]. ...
... These holographic instructions make it feasible to create anything precisely and quickly. It assists unskilled laborers in the assembly of complex structures [13][14][15]. Collaborative and creative production is possible [14]. ...
Technology is employed in the fields of architecture, engineering, and construction (AEC) for characteristics like producing visual representations and offering assistance during the building phase. Both users and creators of these tools are able to immediately take advantage of the technology's potential as well as create a variety of workarounds for its drawbacks. Both viewpoints will be looked at in this study with regard to mobile extended reality SDKs (software development kit). By excluding the articles that did not provide the relevant information, this research concentrates solely on the papers that discuss the technological aspects of the SDK that were used, the opportunities the SDK offers, and/or the flaws of the SDK. The study's main objective is to compare the technological contributions made by the SDKs employed in the scope of the examined literature to the AEC disciplines and to the contexts in which such contributions are made. Through applications in literature research, the study aims to highlight the contributions of mobile extended reality SDKs to the fields of architecture, engineering, and construction. An entry-level developer can use the SDKs in accordance with his work by using the comparison diagrams, produced in this study, to see the relationships and comparisons between them, as well as to build a framework for what uses should be made in which domains. The technological capabilities and constraints of SDKs have an impact on how research is designed. Making relationality diagrams on the SDK to use and the effects it will have throughout the research phase is also crucial. As a result of the research, SDKs permit flexible uses in a variety of sectors, and their use also financially and logistically supports literature studies.
... ere are many kinds of applications where many formats of files have been produced, and these files are scattered in the minds of project personnel, which requires a compatible platform. e files of the whole project are stored in the project directory according to certain rules, which can be accessed by project personnel with project authority, thus improving the For files about co-design of cave dwellings on the platform, fuzzy search can be performed by file name, or advanced search can be performed by file attribute [17] which is convenient for users to find the required files quickly when they are unfamiliar with the project folder structure. ...
Collaborative design is an important link in building construction at present, while the communication and cooperation among disciplines need further regulation. In order to maintain the cost-effectiveness and design quality of building engineering in the competitive environment and obtain higher economic benefits, the design team must ensure the collaborative design quality of building engineering. Based on the analysis of collaboration theory and collaboration mechanism, a collaborative design platform for caves in Henan province is constructed under the current cloud environment. In addition, the requirements and nonfunctional requirements of the platform are analyzed, and the overall structure with each part is designed specifically, hoping to improve the quality of collaborative design effectively.
... alter design shapes in real-time by the control of points through an AR interface (Betti, Aziz and Ron 2019), to influence digital simulation through gesture recognition and hand gestures (Hahm et al. 2019), and to adjust to dynamic structural performance simulations by tracking the movement of a structure's control points (Forren et al 2019). Mitterberger et al. applied an "eye-in-hand" system (Schmalstieg and Höllerer 2016) with an object trackingand registration system that estimates the deviations between the as-planned and the as-built model in realtime for the assembly of a fair-faced brickwork facade (Mitterberger et al. 2020). ...
Augmented Reality (AR) has the potential to create a paradigm shift in the production of architecture. This paper discusses the assembly and evaluation of a bamboo prototype installation aided by holographic instructions. The case study is situated within the framework of AR-driven computational design implementation methods that incorporate feedback loops between the as-built and the digital model.
The prototype construction aims to contribute to the ongoing international debate on architectural applications of digital technology and computational design tools and on the impact these have on craftsmanship and architecture fabrication. The case study uses AR-aided construction techniques to augment existing bamboo craftsmanship in order to expand its practically feasible design solution space. Participating laypersons were challenged to work at the interface of technology and material culture and engage with both latest AR systems and century-old bamboo craft.
This paper reflects on how AR tracking can be used to create a constant feedback loop between as-built installations and digitally designed source models and how this allows for the real-time assessment of design fidelity and deviations. The case study illustrates that this is especially advantageous when working with naturally varying materials, like bamboo, whose properties and behaviour cannot straightforwardly be accurately simulated digitally.
... Similarly, Betti developed a system with which the users can input digital parameters and change digital data by using some default gestures in the real world. The system recognizes and transforms gestures into digital commends for 3D modeling, and displays the holographic on-site model in real-time through the AR immersive design environment [15]. ...
... Betti et al. show in their research that AR technology can play in promoting collaborative design. The authors designed an AR representation and multimodal interface, which shows the potential in the inclusion of non-professional in architectural design and fabrication process, especially in on-site multi-participant immersive design [15]. The structure shape will generate in real-time according to the control points located in the AR interface (Fig. 10) [15]. ...
... The authors designed an AR representation and multimodal interface, which shows the potential in the inclusion of non-professional in architectural design and fabrication process, especially in on-site multi-participant immersive design [15]. The structure shape will generate in real-time according to the control points located in the AR interface (Fig. 10) [15]. The use of a holographic interface confirmed that AR has the possibility of in-suit immersive design, fabrication, and assembly [15]. ...
The use of Augmented Reality (AR) technologies has increased recently, due to the equipment update and the mature technology. For architectural design, especially in digital fabrication projects, more designers begin to integrate AR methods to achieve the visualization in the process. To help unskilled labors for holographic on-site previewing and instruction training, experimental and practice-based studies in AR for the architectural digital fabrication have emerged in recent years. Now, it is a great opportunity to discuss the topic of AR in architectural digital fabrication. By presenting a statistical review of AR technology in architecture projects, this literature review aims to review ongoing research and provide pathways for further research in architectural digital fabrication. This review article is based on information found in journal publications and conference papers in the fields of architecture, engineering, robotics, and digital fabrication, published to date (from 2010 to 2020). The review narrows the literature within these papers by filtering 84 articles through the keyword “Augmented Reality”, “Digital Fabrication” and “Assembly”. The selected articles can be categorized based on the most use of AR function in architectural digital fabrication into an order of the following three classifications with the most significant growth in the last years: (A) AR 3D holographic instruction, (B) AR data sharing, (C) AR for Human-Computer interaction. The information collected from these articles within their classifications is meant to give insight into the current state-of-the-art of AR in the architectural digital fabrication area, as well as to summarize how the topic has matured and developed over time in the research and industry literature. This article has not only analyzed the existing literature but also highlighted new emerging fields in AR research and the future trends of AR function in architectural digital fabrication.
... They enable evidencing contextual information about the product and the associated processes. Thus, they could provide an immersive environment, helping human designers to perform their operations in all stages of design (Betti et al. 2019). ...
The process of verifying the compliance of design solutions to regulations is critical in healthcare design due to the complexity of the requirements present in healthcare settings. The majority of previews research concentrates on "mistake-searching", assuming that design will not be compliant and, as such, necessitates an approach focussed on finding inconsistencies, rather than avoiding mistakes during the design process. This paper reports findings of an ongoing research that follows the Design Science Research approach, with the aim of exploring how existing technologies can support incorporating mistakeproofing (poka yoke) into healthcare design, framed within the regulations compliance process. A set of technological strategies was assessed according to mistakeproofing principles. The analysis evidenced their characteristics, benefits, limitations and examples of application. These technologies rely on the use of hybrid approaches, providing assistance to designers across all design stages, which in turn supports better decision-making and contributes towards improving value generation. A theoretical framework was proposed based on the synergy between design support systems, requirements subjectivity and jidoka. It highlights the importance of better understanding and enhancing the relationship between human designers and different technologies through automation.
This research presents an innovative approach that integrated gesture recognition into a Mixed Reality (MR) interface for human–machine collaboration in the quality control, fabrication, and assembly of the Unlog Tower . MR platforms enable users to interact with three-dimensional holographic instructions during the assembly and fabrication of highly custom and parametric architectural constructions without the necessity of two-dimensional drawings. Previous MR fabrication projects have primarily relied on digital menus and custom buttons within the interface for user interaction between virtual and physical environments. Despite this approach being widely adopted, it is limited in its ability to allow for direct human interaction with physical objects to modify fabrication instructions within the virtual environment. The research integrates user interactions with physical objects through real-time gesture recognition as input to modify, update, or generate new digital information. This integration facilitates reciprocal stimuli between the physical and virtual environments, wherein the digital environment is generative of the user’s tactile interaction with physical objects. Thereby providing user with direct, seamless feedback during the fabrication process. Through this method, the research has developed and presents three distinct Gesture-Based Mixed Reality (GBMR) workflows: object localization, object identification, and object calibration. These workflows utilize gesture recognition to enhance the interaction between virtual and physical environments, allowing for precise localization of objects, intuitive identification processes, and accurate calibrations. The results of these methods are demonstrated through a comprehensive case study: the construction of the Unlog Tower , a 36’ tall robotically fabricated timber structure.
This research posits design methods and implications for architectural fabrication in mixed reality. The notion of spatialising fabrication instructions in mixed reality to guide processes of formation by hand is a new and novel contribution to the discourse on digital fabrication and craft. The benefits of mixed reality environments for visualisation of unbuilt designs, or assembly of designs agnostic to methods of construction has been well established by the literature. However the impact that these new affordances will have on architectural design thinking and production represents a gap in disciplinary knowledge that this research seeks to address. By defining the capacities, limitations and affordances of subjective interpretation of digital fabrication instructions in mixed reality, this research proposes a framework for thinking about the new design conventions and opportunities for designing for mixed reality fabrication. Several case study projects provide practical evidence of this framework by demonstrating the impact of mixed reality on enabling creative exploration and application of traditional craft skills within digital design-to-production processes, as well as improving performance in conventional fabrication processes such as non-linear bricklaying. A reflection on these projects speculates on the broader implications of adoption of mixed reality fabrication by manufacturing and construction industries, the extent to which traditional craft practices may be reinterpreted in mixed reality and the possibilities of an expanded design space for architecture and art.
With increased prefabrication in the construction industry, fabrication workers are tasked to assemble more complicated assemblies with tighter tolerances. However, the existing measurement tools and processes have not changed to accommodate this shift. Lack of advanced measurement tools and existing processes results in increased risk of late detection of geometric errors. To reduce these risks, three-dimensional (3D) quality control systems leveraging scan-vs-BIM methods can be adopted as part of the fabrication process. However, these systems have not been widely adopted yet by fabrication shops, because: (1) fabrication shops often do not have 3D models corresponding to shop drawings; and (2) the cost of integrating accurate 3D scanning equipment into fabrication workflows is assumed to be too high. To remove the first barrier, in this article, a framework for developing 3D digital templates is developed for inspecting received parts. The framework is used for developing a library of 600 3D-models of piping parts. The library is leveraged to deploy a 3D quality control system that was then tested in an industrial scale case study. The results of the case study are used to develop a discrete event simulation model. The simulation results from the model and subsequent cost benefit analysis show that investment in integrating the scan-vs-3D-model quality control systems can have significant cost savings and provide a payback period of less than two years.
