Conference PaperPDF Available

Holographic Construction

Authors:
Conference Paper

Holographic Construction

Abstract and Figures

We present a method for generating holographic construction information from parametric models. Holographic models replace 2D drawings and templates with unambiguous, contextual, shared and interactive design information. We show that our method enabled a team of expert bricklayers to complete a section of wall in a fraction of expected construction time and within typical tolerances, measured through comparative analysis of digital models to 3D point cloud scans of as built conditions.
Content may be subject to copyright.
Holographic Construction
Gwyllim Jahn1, Cameron Newnham1, Nick van den Berg1, Melissa Iraheta1 and Jack-
son Wells2
1 Fologram, Collingwood, Victoria, AUSTRALIA
2 University of Tasmania, Launceston, Tasmania, AUSTRALIA
Abstract. We present a method for generating holographic construction infor-
mation from parametric models. Holographic models replace 2D drawings and
templates with unambiguous, contextual, shared and interactive design infor-
mation. We show that our method enabled a team of expert bricklayers to com-
plete a section of wall in a fraction of expected construction time and within typ-
ical tolerances, measured through comparative analysis of digital models to 3D
point cloud scans of as built conditions.
Keywords: Mixed Reality, Holographic Construction, Digital Craft, Digital
Fabrication.
1 Background
Computer aided design and manufacturing has afforded architects more control over
fabrication processes and brought about the possibility of generating construction in-
formation directly from design information (Kolarevic, 2003). However, the limited
ability to interfere with digital fabrication processes during their operation has inhibited
adaptable, flexible, intuitive and reactive approaches to materialising buildings and ex-
acerbated the distinction between the practices of designing and making. In response to
these concerns, researchers have focused on improving the capacity of industrial robots
to sense and learn from their material environments (Menges, 2015), though the com-
puter vision systems required for industrial robots to negotiate typical building sites and
work with human construction teams present significant challenges (Dörfler et al,
2016).
Take the particularly prevalent fascination with variable brick structures and the au-
tomation of their construction within the discourse of the digital design community.
2
While the construction of structures from brick by robotic arms can be automated
within laboratory environments, these same robots perform poorly on construction sites
where mortar viscosity is unpredictable, detritus turns up in unexpected locations and
construction workers tend to get in the way. Bricklayers attempting to construct varia-
ble brick structures require templates that are expensive, wasteful and labour intensive
(Yuan et al, 2013). Mixed Reality (Milgram & Kishino, 1994) environments lend them-
selves well to non-standard bricklaying as taking physical measurements to determine
brick locations can be replaced with a single set out of the entire holographic model
relative to the physical construction site. (Fazel & Izadi, 2018).
2 Aims
We present a mixed reality platform for carrying out construction tasks by following
interactive holographic instruction sets generated directly from design models. Holo-
graphic instructions effectively replace conventional 2D drawings and physical tem-
plates with unambiguous, contextual, shared and interactive design information. By
augmenting the physical workspace with accurate digital information, we aim to extend
the demonstrable skill of construction workers by reducing construction time and cre-
ating new opportunities for training, collaboration and skill development. By working
from complex design models and simple instructions, we further aim to demonstrate
how precise digital models can be used as guides that describe complex and explicit
end states while still enabling fabricators to deploy advanced skills and material exper-
tise in their realization.
This research attempts to establish the degree to which mixed reality applications
can be integrated into and impact upon typical brick construction workflows. Extensive
research (Shin & Dunston, 2008) has identified common challenges within the con-
struction industry that mixed reality systems are well positioned to solve, including
reducing the manual complexity of setout tasks without known reference points or
checking as-built conditions against design intent (Cote et al, 2014). We aim to demon-
strate the degree to which current generation consumer mixed reality hardware can ad-
dress these challenges and evaluate the effectiveness of our method and platform
through the construction of a brick wall consisting of 585 uniquely positioned bricks,
completed by a team of two bricklayers in Hobart, Tasmania.
3 Method
3.1 Mixed reality in Rhino and Grasshopper
Mixed reality experiences are created by blending physical space with digital content,
through projections, spatial sound, video compositing on 2D screens or with transparent
displays. The Microsoft HoloLens is a commercially available head-mounted display
that creates the illusion of digital content appearing fixed within a wearers physical
environment by tracking the position of the device using onboard cameras and
3
rendering the relative position of holograms to a transparent, stereoscopic display. The
HoloLens facilitates hands-free mixed reality experiences at the scale of a small build-
ing site without the need for external sensors.
3.2 Dynamically Displaying Geometry
Fologram, a software application for mobile phones, the HoloLens and McNeel and
Associates’ Rhino and Grasshopper 3d, was used to create an interactive holographic
instruction set from parametric models describing the design. Fologram synchronizes
geometry within a Rhino or Grasshopper document with geometry rendered on mixed
reality devices connected over a local WiFi network. Whenever a user made changes
to a model in Rhino or Grasshopper, these changes were detected and streamed to all
connected mixed reality devices. This allowed users to experience digital models in a
physical context and at scale while making changes to these models using familiar and
powerful desktop CAD software.
Fig. 1. Bricklaying Interface
3.3 Event based parametric modelling
Changes could also be made to models in the mixed reality environment by parametri-
cally associating detected gesture events with geometry rendered on the device. Buttons
were created and placed in physical space adjacent to the construction site that allowed
bricklayers in the mixed reality experience to increase or decrease the current course of
bricks rendered on the Hololens without needing to use the Rhino or Grasshopper in-
terface (see Fig. 1). An additional button was created that toggled the display of all
geometry in the entire finished wall.
4
3.4 Shared Experiences
When multiple devices were connected to Fologram, the coordinate systems of these
devices were automatically synchronized to display holographic content to within
20mm of the same physical location using Microsoft’s proprietary spatial anchor sys-
tem. Any connected device could interact with buttons in the mixed reality experience
and have the changes reflected on all other connected devices. This enabled bricklayers
in the experience to discuss current and future construction tasks (by changing the dis-
play of the current course of bricks), develop collaborative workflows for identifying
and addressing causes of error (by comparing the relative position of previously laid
bricks to their holographic representation and then re-placing the model datum if they
no longer aligned), or for reviewing progress (by displaying all bricks in the wall).
Fig. 2. Wall Desig n
3.5 Parametric Wall Geometry
A parametric model was developed to explore variable brick spacings along courses
derived by creating contours through arbitrary mesh geometry. When double-curved
mesh geometry was used as an input to the model, the generated brick courses described
curves both in plan and in section, prohibiting their construction using conventional
tools of string lines and plumb bobs. Further variation was introduced by rotating each
brick around its center point such that patterns of shadows and lines could be read across
the surface of the structure (see Fig. 2). These design decisions were made to demon-
strate the degree to which masonry systems can express variation in architectural form,
pattern, texture and permeability with relatively low cost and simple construction tech-
nology.
5
Fig. 3. Holographic instructions overlaid on the construction site
3.6 2D instructions from 3D design models
The bricks in the design model were grouped into courses to enable bricklayers to view
each course individually as required and prevent unbuilt parts of the design creating
visual distractions during construction. Because the design model only contained planar
courses of bricks and all bricks were of the same size and type, the 3D model of each
brick contained more information than was required during construction. Instead an
outline of the top face of each brick was all that was required to accurately locate each
brick in its correct 2D position along the course (see Fig. 3).
3.7 Holographic Setout
Building directly from holographic models required accurate placement of these mod-
els within the physical environment of the construction site. This was achieved by lo-
cating the origin of the holographic model at an arbitrary physical location adjacent to
the construction site indicated by a fiducial marker. Fine adjustments were then made
to the position of the digital model relative to this origin point in Rhino. By visually
observing the position of the hologram through the HoloLens while moving the model
in Rhino, the model was accurately aligned to two columns that served as the edges of
the brick wall without the need to take any additional site measurements or perform any
additional set out. While performing this check, the observed holographic model ex-
ceeded the width between the two existing columns by approximately 15mm. The input
surface of the parametric model was adjusted to accommodate this change in width,
and visually checked again to ensure accurate alignment and fit with the physical con-
ditions on site.
6
After initial placement, the fiducial marker was used to precisely reposition the hol-
ographic model after restarting a session and for performing visual checks on holo-
graphic drift by comparing a set of holographic cross hairs representing the 0,0,0 point
of the holographic model with a corresponding graphic printed on the marker. If these
two representations did not align the bricklayers could reposition the origin point on
the marker using the Fologram interface on the HoloLens. The marker was only used
for hologram placement and did not need to be visible to the HoloLens at any other
time during construction.
3.8 Maintaining alignment over time
Because the position of the holographic model within the physical construction site is
entirely reliant on the ability of the HoloLens to track the relative position of feature
pixels identified by the device’s cameras, changes in the environment would introduce
drift in the position of the holographic model. This error was caused by movement of
people, scaffolding or material on site, construction of the brick structure that occluded
existing feature points and variation in lighting conditions introduced by changing
weather. Vertical drift was detected by comparing the position of a holographic course
of bricks with the existing courses on the neighbouring columns. Horizontal drift was
identified by displaying both the current course of bricks being constructed as well as
the previous course of bricks. By visually checking if the previous course no longer
aligned to the physically placed bricks, the bricklayers could identify that the model
had drifted and replace the origin using the fiducial marker. The bricklayers anecdotally
reported the need to re-place the hologram twice during construction.
Fig. 4. Correcting brick placement
7
3.9 Intuitive collaboration workflows
While the relative positions of each brick were correct for all connected devices, the
origin point of this model would be inconsistent between devices by up to 20mm due
to the limitations of working from shared spatial anchors. For this reason, one HoloLens
was used to place bricks approximately in the correct position and another would then
correct these positions after a course had been completed to maintain consistency (see
Fig. 4). During construction the bricklayers discussed and made minor changes to the
graphic representation of the instructions, changing the hologram of the brick footprint
from white to green in order to create contrast with the physical environment and mov-
ing the brick footprint from describing the base of the brick to the top of the brick in
order to use this information as a digital string line indicating level.
Fig. 5. Completed Brick Wall
4 Results
Working in parallel, two bricklayers were able to lay all 585 bricks in the structure in
six and a half hours of bricklaying (see Fig. 5) We observed that the depth perception
provided by the stereoscopic display of the HoloLens was sufficient to accurately locate
bricks by eye, with the bricklayers easily able to position a brick within its wireframe
representation to within the tolerances of the display resolution. The most common de-
viation from the digital model occurred from differences in the dimensions of the recy-
cled bricks rather than from inaccurate placement, as the roles established by the
8
bricklayers for placement and correction was an effective method for identifying and
correcting for mistakes resulting from drift in the holographic model.
Although the bricklaying team reported that the rate of 300 bricks per bricklayer per
day was slower than the 500-600 they would expect to achieve on a straight section of
wall, they estimated that working from conventional methods and tools the structure
would have taken 14 days to complete by a single skilled bricklayer. Replacing time
consuming manual tasks of measuring and setting out brick locations or templates from
a known datum with a single holographic model effectively reduced construction time.
A further reduction in time was achieved by enabling the structure to be constructed in
parallel by multiple bricklayers in the same shared holographic experience. This is
made uniquely possible working from shared mixed reality experiences because the
bricklayers have access to a single, constant, shared description of the finished result,
minimizing the need to consult a project lead to conduct quality control and verify
setout tasks.
The bricklaying team have extensive experience constructing high quality structures
from brick. They actively seek out projects that challenge this capability, enable them
to improve and take on more complex projects. The bricklaying team reported that con-
veying construction information in a ready-to-hand way simplified the task of construc-
tion by enabling bricklayers to focus on common and familiar workflows with which
they already possessed significant expertise and apply this expertise to the new and
challenging construction task. We observed construction teams make and review trade-
offs between construction time and construction accuracy through the common medium
of the holographic model. By enabling designers to also participate in this shared mixed
reality experience, discussions could be had with bricklayers as to the degree to which
construction information could be simplified, reducing the need to develop documen-
tation sets without compromising on formal complexity or variation in the structure.
9
Fig. 6. 3D point cl oud scan of constructed brick wall
4.1 Accuracy analysis
3D point cloud scan results (see Fig. 6) were compared to the digital model to conduct
an empirical analysis of the accuracy of our method. We observed an average deviation
in plan of between 21mm for courses 1-14 (see Fig. 7) and 20-30 (see Fig. 9) and 5mm
for courses 15-20 (see Fig. 8). This deviation can be attributed to variation in the shape
of reclaimed bricks and discrepancies in the wall faces of columns adjacent to the site
that lean up to 12mm from vertical. In elevation we observed that courses 8-18 deviated
up to 18mm from relative digital models, though this deviation allowed for alignment
of these courses to existing brickwork. All other courses were found to be accurately
levelled relative to the ground plane and with themselves. Dimensional deviation of the
completed structure did not exceed tolerances expected when using traditional methods
for brickwork laying.
Fig. 7. Course 1 plan view of digital model and scan composite
10
Fig. 8. Course 15 plan view of digital model and scan composite
Fig. 9. Courses 30 plan view of digital model and scan composite
Fig. 10. Elevation of brick wall scan showing levelling relative to ground plane.
5 Discussion
Unlike drawings, holographic instructions serve as contextual, scaled and unambiguous
descriptions of goal states for tasks. By providing both experienced bricklayers and
trainees with the same constant reference to a desired end state for a given task or set
of tasks, mistakes were easily identified and suggestions for improvement could be un-
ambiguously communicated and agreed upon. Methods for negotiating between holo-
graphic models and existing physical conditions, adjusting for level and rapid alignment
of bricks to their holographic footprints were all established and shared between the
team. These intuitive approaches to following holographic instructions optimized both
construction time and construction precision within acceptable tolerances. Because
11
experienced bricklayers could immediately verify the quality of the bricklaying by per-
forming visual checks on the alignment of physical bricks to the holographic model,
the construction team was able to minimize the difficulty of the construction task to the
point where it could be used as an effective training job for new apprentices.
5.1 Causes of error and inefficiency
Error in brick placement resulted from placing a brick using a holographic guide that
had drifted from the correct physical location on site, incorrectly judging the depth of
the hologram due to poor device calibration or placing the brick incorrectly despite
correct holographic information. In each case these errors were avoided by checking
the holographic model relative to other bricklayers participating in the shared experi-
ence and by performing visual comparisons of the holographic origin with the printed
physical marker. These errors were further minimised by assigning specific roles to the
bricklayers making them responsible for placing or checking brick locations.
In the approach outlined in this paper, the mixed reality experience is ‘always on’
providing accurate information describing the position of each brick in the wall. Rela-
tive to constructing a straight wall with conventional techniques, holographic instruc-
tions introduce inefficiencies due to bricklayers relying on holographic information that
with current generation hardware must be repeatedly checked for drift before it can be
reliably followed. Inefficiencies were introduced by the need to view each course of
bricks clearly in plan in order to accurately place a brick within its holographic foot-
print. The scaffolding set up and adjustment to facilitate placement at waist height is
more time consuming than required with conventional techniques that enable bricklay-
ers to place bricks up to eye height. In future work we intend to address these causes of
error, inefficiencies and limitations to implementing our system on large scale construc-
tion sites.
5.2 Future work
By parametrically generating construction information directly from design infor-
mation and enabling bricklayers to view this information in the physical context of the
instruction environment in real time, we have developed a design to production process
that effectively eliminates the need for drawn documentation or rationalization of de-
sign models into geometry that can be described through 2D projection. However, our
current system requires bricklayers to have access to a laptop running Grasshopper in
order to facilitate real time exchange of geometry, a constraint that is not practical on
most construction sites. Future work aims to address this issue by allowing designers
to export interactive holographic instructions developed in Grasshopper to run as
standalone applications on mixed reality headsets.
There are several quality control challenges in masonry construction that are difficult
to address with digital information assessed by eye. These include checking for plumb
and level, where analogue techniques are well suited to straight sections of wall but are
12
inefficient when used on curved or textured structures. We imagine several alternative
methods that utilize ‘in and out’ or ‘just in time’ mixed reality experiences that resolve
or assist with specific and difficult construction tasks when needed rather than requiring
those tasks to be entirely augmented. Providing clearer feedback to bricklayers on task
performance and construction accuracy may be achieved by combining our system with
third party depth scanners or photogrammetry tools and in future work we aim to con-
tinue to improve on what we have established can be achieved by eye alone.
The small structure described in this paper reflects the current limitations of mixed
reality hardware with respect to managing the drift of holographic instructions relative
to construction sites. The construction of larger structures will require improvements in
mixed reality hardware, a combination of on-board SLAM tracking with external posi-
tioning devices, software solutions for automatically aligning digital and physical mod-
els or a hybrid system that utilizes multiple fiducial markers in known locations on site
to automatically detect and correct the position of holographic models. Researchers
have, for instance, demonstrated the use of computer vision systems to provide feed-
back to a bricklayer for accurate brick positioning, especially when working with
screen-based mixed reality systems (Sandy & Buchli, 2018). Future work will explore
the utility of these possible improvements as we continue to apply our system to the
challenges of larger structures.
References
1. Côté S., Beauvais M., Girard-Vallée A., Snyder R.: A live Augmented Reality Tool for Fa-
cilitating Interpretation of 2D Construction Drawings. In: De Paolis L., Mongelli A (eds.)
Augmented and Virtual Reality. AVR 2014. Lecture Notes in Computer Science. vol 8853.
Springer, Cham (2014).
2. Dörfler, K., Sandy, T., Giftthaler, M., Gramazio, F., Kohler, M., & Buchli, J.: Mobile Ro-
botic Brickwork. In: Reinhardt, D, Saunders, R, Burry, J (eds.) Rob|Arch 2016 Robots in
Architecture, Art and Design pp. 204–217 (2016).
3. Fazel, A., & Izadi, A.: An interactive augm ented reality tool for constructing free-form mod-
ular surfaces. In: Automation in Construction, vol 85, pp. 135–145 (2018).
4. Kolarevic, B.: Architecture in the Digital Age. Spon Press, New York (2003).
5. Menges, A. The new cyber-physical making in architecture: Computational construction. In:
Architectural Design, vol 85(5), pp. 28–33 (2015).
6. Milgram, Paul & Kishino,: Fumio. A Taxonom y of Mixed Reality Visual Displays. In:
IEICE Trans. Information Systems. vol. E77-D, no.12. pp. 1321-1329 (2015).
7. Sandy, T., & Buchli, J.: Object-Based Visual-Inertial Tracking for Additive Fabrication. In:
IEEE Robotics and Automation Letters, vol 1, pp. (99), (2018).
8. Shin, D. H., & Dunston, P. S.: Identification of application areas for Augmented Reality in
industrial construction based on technology suitability. In: Autom ation in Construction, vol
17(7), pp. 882–894 (2008).
9. Yuan, P. F., Zhang, M., & Han, L.: Low-tech digital fabrication: Traditional brick as mate-
rial in digital practice. In: Communications in Computer and Informat ion Science, 369
CCIS, pp. 139–148. (2013).
... Due to some limiting factors such as low payload, limited robustness, and high energy consumption, aerial robots are less developed in the field of construction. Ground-based robots, on the other hand, have more capabilities for in-situ fabrication; however, they also have some limitations in terms of cost, complexity of control, maintenance, etc. [20,21]. Alongside these recent developments to fully automated systems that require specific controlled conditions (like industrial factories), a stream of research has recently emerged that follows another approach called human augmentation. ...
... For instance, Fazel and Izadi proposed a marker-based AR tool for constructing free-form modular surfaces in buildings [25] and Lopez et al. introduced a marker-based handheld AR tool for foam printing on the building site [26]. A custom-built marker-less AR tool was proposed by Mitterberger et al. for constructing free-form brick walls [27] and an AR assistive tool was introduced by the Fologram team for design, share and create in mix reality that runs on Microsoft Hol-oLens or smartphones [20]. Marker-based AR systems have some beneficial points such as utilizing low-cost tools; however, they have some limitations concerning camera view angle and environmental lighting conditions [25,28]. ...
... Comparing advantages and disadvantages of some existing in-situ digital fabrication technologies in architecture. [20,27] In addition, a controlling subsystem was also designed to enhance the accuracy of the system by reducing the risk of human errors. Eventually, the accuracy of the proposed system was evaluated by constructing four full-scale prototypes. ...
Article
The level of automation in the construction industry is currently low, and there is a growing need for new fabrication techniques that can bring more flexibility. This paper aims to introduce an optical tool which can bring the digital fabrication data on the building site in a real-time procedure to facilitate construction of complex brick structures. This tool consists of three custom-made two-axis laser pointer devices, a micro-controller, and a processor that enable the user to identify the exact location of the objects in the real world. Additionally, a controlling sub-system is considered to reduce human errors. The efficacy of the proposed system was studied by four full-scale prototypes. The results showed that the average lateral error was 2.5 mm, the average orientation error was 1°, and the average construction time for each module was 27 s. This tool can provide more flexibility for constructing complex brick structures on the building site.
... In addition, in order to approach the traditional construction methods, in which the heavy formworks are replaced by the intelligently positioned guideworks, a "virtual formwork" will be explored. This "virtual tool" is based on the new applications in construction, offered by reality-augmented technology [3]. It is supposed to improve the construction process by minimizing the need to the cumbersome formworks and by virtually assisting the traditional masons. ...
... Later the DFAB house [68] became the showcase for how different non-standardized components in a building can be shaped through digital design and robotic fabrication process [69][70][71]. In the context of masonry construction, the development of specialized digital design software [72,73] and augmented reality frameworks [74][75][76] have allowed for more geometrically complex geometries to be materialized on site. Industrial robotic arms have also been been applied in complex setups, for example on gantry cranes [77] and as mobile platforms [78,79]. ...
Article
Geometrically complex masonry structures (e.g., arches, domes, vaults) are traditionally built with scaffolding or falsework to provide stability during construction. The process of building such structures can potentially be improved through the use of multiple robots working together in a cooperative assembly framework. Here a robot is envisioned as both a placement and external support agent during fabrication-the unfinished structure is supported in such a way that scaffolding is not required. The goal of this paper is to present and validate the efficacy of three cooperative fabrication approaches using two or three robots, for the scaffold-free construction of a stable masonry arch from which a medium-span vault is built. A simplified numerical method to represent a masonry structure is first presented and validated to analyze systems composed of discrete volumetric elements. This method is then used to evaluate the effect of the three cooperative robotic fabrication strategies on the stability performance of the central arch. The sequential method and cantilever method, which utilize two robotic arms, are shown to be viable methods, but have challenges related to scalability and robustness. By adding a third robotic agent, it becomes possible to determine a structurally optimal fabrication sequence through a multi-objective optimization process. The optimized three robot method is shown to significantly improve the structural behavior over all fabrication steps. The modeling approaches presented in this paper are broadly formulated and widely applicable for the analysis of cooperative robotic fabrication sequences for the construction of masonry structures across scales.
... Later the DFAB house [68] became the showcase for how different non-standardized components in a building can be shaped through digital design and robotic fabrication process [69][70][71]. In the context of masonry construction, the development of specialized digital design software [72,73] and augmented reality [74][75][76] have allowed for ever more geometrically complex geometries to be materialized on site. Industrial robotic arms have also been been applied in complex setups, for example on gantry cranes [77] and as mobile platforms [78,79]. ...
Preprint
Full-text available
Geometrically complex masonry structures (e.g., arches, domes, vaults) are traditionally built with expensive scaffolding or falsework to provide stability during construction. The process of building such structures can potentially be improved through the use of multiple robots working together in a cooperative assembly framework. Here a robot is envisioned as both a placement and external support agent during fabrication - the unfinished structure is supported in such a way that scaffolding is not required. The goal of this paper is to present and validate the efficacy of three cooperative fabrication approaches using two or three robots, for the scaffold-free construction of a stable masonry arch from which a medium-span vault is built. A simplified numerical method to represent a masonry structure is first presented and validated to analyse systems composed of discrete volumetric elements. This method is then used to evaluate the effect of the three cooperative robotic fabrication strategies on the stability performance of the central arch. The sequential method and cantilever method, which utilize two robotic arms, are shown to be viable methods, but have challenges related to scalability and robustness. By adding a third robotic agent, it becomes possible to determine a structurally optimal fabrication sequence through a multi-objective optimization process. The optimized three robot method is shown to significantly improve the structural behavior over all fabrication steps. The modeling approaches presented in this paper are broadly formulated and widely applicable for the analysis of cooperative robotic fabrication sequences for the construction of discrete element structures across scales and materials.
Chapter
Research on mobile Augmented Reality (AR) technologies has proven many potentials and benefits for assisting craftspeople in various building applications within the AEC domain. However, little research has been done on the use of multi-user mobile AR systems coordinating several people at the same time. This paper examines the potentials of a collective construction process enabled by AR technology that distributes and guides manual assembly tasks for multi-user participation. For this purpose, a custom mobile AR app is developed that uses cloud services and allows multiple people to participate in the construction and be coordinated with each other at the same time. In the proposed setup, digital building instructions for the stepwise assembly of a physical building structure can be retrieved by multiple users via the app. The app positions these building instructions in 3D space, visually superimposed on the building site, where the building structure is being assembled. Methods are proposed for synchronizing the construction progress via user-specific AR content visualization over the app’s user interface (UI). Based on the principle of topologically interlocking structures, the material system developed for this research offers several form-fitting connections with one modular wooden component without the need for mechanical fasteners for their assembly. This principle enables the manual implementation of various complex building structures at full architectural scale, which are reconfigurable and fully disassemblable. The proposed methods were experimentally validated in a 1:1 scale demonstrator. A pavilion was assembled collectively by students and researchers over two days, and the UI was evaluated through a qualitative user study. As an outlook, the paper discusses the potential of such AR systems to make digitally-driven construction processes more tangible and accessible to laypersons and unskilled people and thus encourage community participation.KeywordsAugmented realityMulti-user mobile applicationTopological interlockingCloud dataParticipatory digital fabrication
Conference Paper
Full-text available
Augmented Reality (AR) has the potential to create a paradigm shift in the production of architecture. This paper discusses the assembly and evaluation of a bamboo prototype installation aided by holographic instructions. The case study is situated within the framework of AR-driven computational design implementation methods that incorporate feedback loops between the as-built and the digital model. The prototype construction aims to contribute to the ongoing international debate on architectural applications of digital technology and computational design tools and on the impact these have on craftsmanship and architecture fabrication. The case study uses AR-aided construction techniques to augment existing bamboo craftsmanship in order to expand its practically feasible design solution space. Participating laypersons were challenged to work at the interface of technology and material culture and engage with both latest AR systems and century-old bamboo craft. This paper reflects on how AR tracking can be used to create a constant feedback loop between as-built installations and digitally designed source models and how this allows for the real-time assessment of design fidelity and deviations. The case study illustrates that this is especially advantageous when working with naturally varying materials, like bamboo, whose properties and behaviour cannot straightforwardly be accurately simulated digitally.
Conference Paper
Full-text available
This paper focusses on teaching the integration of Augmented (AR) and Mixed Reality (MR), combined referred to as Extended-Reality (XR), and photogrammetry technology into handicraft using an online-taught digital fabrication workshop as an educational case study. Set up in response to restrictions from Covid-19, workshop "Secret Whispers & Transmogrifications" had students and instructors around the world participate in a course that challenged our understanding of educating craft and technology without the necessity of physical presence. The integration of AR into craftsmanship enhances architectural design and fabrication processes as it overlays computation-driven information onto the hands of the end user. These computer-numerically-controlled workflows incorporate and rely on manual actions as an integral part of a process that is typified by inevitable, unpredictable, human error. In doing so, the workshop questions common infatuation with precision in digital fabrication and construction by striving for alternative approaches that embrace the inaccuracies and imprecisions innate to technologically-augmented human craftsmanship. Participants took part in a hands-on clay modelling "secret whispers" experiment that was designed to introduce theoretical concepts and applications of XR technology into the production workflows. This paper concludes by highlighting that the accessibility of today's technology enables AR-enhanced craftsmanship to be successfully taught remotely and online.
Article
Full-text available
Augmented bricklaying explores the manual construction of intricate brickwork through visual augmentation, and applies and validates the concept in a real-scale building project—a fair-faced brickwork facade for a winery in Greece. As shown in previous research, robotic systems have proven to be very suitable to achieve various differentiated brickwork designs with high efficiency but show certain limitations, for example, in regard to spatial freedom or the usage of mortar on site. Hence, this research aims to show that through the use of a craft-specific augmented reality system, the same geometric complexity and precision seen in robotic fabrication can be achieved with an augmented manual process. Towards this aim, a custom-built augmented reality system for in situ construction was established. This process allows bricklayers to not depend on physical templates, and it enables enhanced spatial freedom, preserving and capitalizing on the bricklayer’s craft of mortar handling. In extension to conventional holographic representations seen in current augmented reality fabrication processes that have limited context-awareness and insufficient geometric feedback capabilities, this system is based on an object-based visual-inertial tracking method to achieve dynamic optical guidance for bricklayers with real-time tracking and highly precise 3D registration features in on-site conditions. By integrating findings from the field of human-computer interfaces and human-machine communication, this research establishes, explores, and validates a human-computer interactive fabrication system, in which explicit machine operations and implicit craftsmanship knowledge are combined. In addition to the overall concept, the method of implementation, and the description of the project application, this paper also quantifies process parameters of the applied augmented reality assembly method concerning building accuracy and assembly speed. In the outlook, this paper aims to outline future directions and potential application areas of object-aware augmented reality systems and their implications for architecture and digital fabrication.
Chapter
Full-text available
This paper describes the implementation of a discrete in situ construction process using a location-aware mobile robot. An undulating dry brick wall is semi-autonomously fabricated in a laboratory environment set up to mimic a construction site. On the basis of this experiment, the following generic functionalities of the mobile robot and its developed software for mobile in situ robotic construction are presented: (1) its localization capabilities using solely on-board sensor equipment and computing, (2) its capability to assemble building components accurately in space, including the ability to align the structure with existing components on site, and (3) the adaptability of computational models to dimensional tolerances as well as to process-related uncertainties during construction. As such, this research advances additive non-standard fabrication technology and fosters new forms of flexible, adaptable and robust building strategies for the final assembly of building components directly on construction sites. While this paper highlights the challenges of the current state of research and experimentation, it also provides an outlook to the implications for future robotic construction and the new possibilities the proposed approaches open up: the high-accuracy fabrication of large-scale building structures outside of structured factory settings, which could radically expand the application space of automated building construction in architecture.
Article
Full-text available
Mixed Reality (MR) visual displays, a particular subset of Virtual Reality (VR) related technologies, involve the merging of real and virtual worlds somewhere along the 'virtuality continuum' which connects completely real environments to completely virtual ones. Augmented Reality (AR), probably the best known of these, refers to all cases in which the display of an otherwise real environment is augmented by means of virtual (computer graphic) objects. The converse case on the virtuality continuum is therefore Augmented Virtuality (AV). Six classes of hybrid MR display environments are identified. However quite different groupings are possible and this demonstrates the need for an efficient taxonomy, or classification framework, according to which essential differences can be identified. An approximately three-dimensional taxonomy is proposed comprising the following dimensions: extent of world knowledge, reproduction fidelity, and extent of presence metaphor.
Article
In this paper, a system is presented to track the motion of a sensor-head relative to multiple objects of known geometry. Measurements from a monocular camera and an inertial measurement unit are probabilistically fused in a moving horizon estimator to obtain high accuracy estimates. Methods for detecting tracking loss and automatically resuming tracking are presented. The performance of the system is shown through experiments tracking various objects and ground truth measurements demonstrate the system's ability to provide accurate real-time motion estimates. As an initial application of the system, a 100 brick structure with complex geometry was built by hand using the tracking system and an augmented reality visualizer to guide construction.
Article
Although modern software has paved the way for architects to design complex forms, such as free-forms, construction remains challenging, costly, and time-consuming which requires skilled workers. Advanced digital fabrication technologies can offer new ways to fill the gap between design and construction. Augmented Reality (AR) technology is one such technology that has many potentials in various fields, however, its capabilities are not sufficiently explored yet, especially in the field of digital fabrication. This study presents a new affordable interactive multi-marker augmented reality tool for constructing free-form modular surfaces implemented by integrating common accessible devices. The proposed tool consists of two digital cameras, a head-mounted display, a processor, and two markers that enable the user to virtually see the accurate location of any proposed object in the real world. A controlling subsystem was also designed to enhance the accuracy of construction. Method efficiency was studied in five full-scale prototypes. The results showed that the majority of errors (91%) were less than 6 mm, and 2° for lateral placements and orientation errors.
Article
In architectural history, the advent of new fabrication and construction technologies has always been a catalyst for design innovation, and the latent next paradigm shift facilitated by the introduction of cyber-physical production systems will be no exception. What in other domains is often referred to as the ‘Fourth Industrial Revolution’ will also have a major impact on architecture, making possible a move away from instruction-based making towards that based on behaviour. In this article, Guest-Editor Achim Menges, Director of the Institute for Computational Design (ICD) at the University of Stuttgart, argues that these emerging technologies not only challenge our understanding of how buildings are made, but more importantly how we think about the genesis of form, tectonics and space.
Article
Research studies in the application of Augmented Reality (AR) in the Architecture, Engineering, and Construction (AEC) industry have suggested its feasibility. However, realization of the use of AR in AEC requires not only demonstration of feasibility but also validation of its suitability. This paper comprehensively identifies AR application areas in industrial construction based on suitability of AR technologies. In order to successfully explore suitability of AR, this paper assesses work tasks from the viewpoint of human factors regarding visual information requirements to find rationale for the benefits of AR in work tasks. Based on the assessment of work tasks, this paper presents a comprehensive map that identifies AR application areas in industrial construction. The comprehensive map reveals that eight work tasks (layout, excavation, positioning, inspection, coordination, supervision, commenting, and strategizing) out of 17 classified work tasks may potentially benefit from AR support.
A live Augmented Reality Tool for Facilitating Interpretation of 2D Construction Drawings
  • S Côté
  • M Beauvais
  • A Girard-Vallée
  • R Snyder
Côté S., Beauvais M., Girard-Vallée A., Snyder R.: A live Augmented Reality Tool for Facilitating Interpretation of 2D Construction Drawings. In: De Paolis L., Mongelli A (eds.) Augmented and Virtual Reality. AVR 2014. Lecture Notes in Computer Science. vol 8853. Springer, Cham (2014).
Architecture in the Digital Age
  • B Kolarevic
Kolarevic, B.: Architecture in the Digital Age. Spon Press, New York (2003).
Low-tech digital fabrication: Traditional brick as material in digital practice
  • P F Yuan
  • M Zhang
  • L Han
Yuan, P. F., Zhang, M., & Han, L.: Low-tech digital fabrication: Traditional brick as material in digital practice. In: Communications in Computer and Information Science, 369 CCIS, pp. 139-148. (2013).