Conference PaperPDF Available

Augmenting Craft with Mixed Reality: A Case Study Project of AR-driven Analogue Clay Modelling

Authors:

Abstract

This paper discusses novel methods for and advantages of integrating augmented reality (AR) and photogrammetry in hand clay-sculpting workflows. These techniques permit nontrained users to achieve higher precision during the sculpting process by holographically overlaying instructions from digital 3D source geometry on top of the sculpting material. By employing alternative notational systems in design implementation methods, the research positions itself in a postdigital context aimed at humanizing digital technologies. Throughout history, devices have been developed to increase production, such as Henry Dexter’s 1842 “Apparatus for Sculptors” for marble sculpting. Extrapolating from this, the workflow presented in this paper uses AR to overlay extracted information from 3D models directly onto the sculptor’s field of vision. This information can then become an AR-driven guidance system that assists the sculptor. Using the Microsoft HoloLens, holographic instructions are introduced in the production sequence, connecting the analog sculpture fabrication directly with a digital environment, thus augmenting the craftspeople’s agency. A series of AR-aided sculpting methods were developed and tested in a demonstrator case study project that created a small-scale clay copy of Henry Moore’s Sheep Piece (1971–1972). This paper demonstrates how user-friendly software and hardware tools have lowered the threshold for end users to develop new methods that straightforwardly facilitate and improve their crafts’ effectiveness and agency. This shows that the fusion of computational design technology and AR visualization technology can innovate a specific craft’s design and production workflow, opening the door for further application developments in more architecture-specific fabrication contexts.
436
Augmenting Craft
with Mixed Reality
1 Postdigital “Apparatus for
Sculptors,” using augmented
reality to overlay 3D digital
modeling information onto the
analog material.
Jacky Chun Hin Fong
The Chinese University of Hong
Kong, School of Architecture
Adabelle Long Wun Poon
The Chinese University of Hong
Kong, School of Architecture
Wing Sze Ngan
The Chinese University of Hong
Kong, School of Architecture
Chung Hei Ho
The Chinese University of Hong
Kong, School of Architecture
Garvin Goepel
The Chinese University of Hong
Kong, School of Architecture
Kristof Crolla
The University of Hong Kong,
Departments of Architecture &
Civil Engineering
A Case Study Project of AR-Driven
Analog Clay Modeling
1
ABSTRACT
This paper discusses novel methods for and advantages of integrating augmented reality
(AR) and photogrammetry in hand clay-sculpting workows. These techniques permit
nontrained users to achieve higher precision during the sculpting process by holograph-
ically overlaying instructions from digital 3D source geometry on top of the sculpting
material. By employing alternative notational systems in design implementation methods,
the research positions itself in a postdigital context aimed at humanizing digital technol-
ogies. Throughout history, devices have been developed to increase production, such as
Henry Dexter’s 1842 “Apparatus for Sculptors” for marble sculpting. Extrapolating from
this, the workow presented in this paper uses AR to overlay extracted information from
3D models directly onto the sculptor’s eld of vision. This information can then become
an AR-driven guidance system that assists the sculptor. Using the Microsoft HoloLens,
holographic instructions are introduced in the production sequence, connecting the analog
sculpture fabrication directly with a digital environment, thus augmenting the crafts-
people’s agency. A series of AR-aided sculpting methods were developed and tested in a
demonstrator case study project that created a small-scale clay copy of Henry Moore’s
Sheep Piece (1971–1972). This paper demonstrates how user-friendly software and
hardware tools have lowered the threshold for end users to develop new methods that
straightforwardly facilitate and improve their crafts’ effectiveness and agency. This shows
that the fusion of computational design technology and AR visualization technology can
innovate a specic craft’s design and production workow, opening the door for further
application developments in more architecture-specic fabrication contexts.
CULTURE AND ACCESS 437
2
INTRODUCTION
“Every block of stone has a statue inside it and it is the task
of the sculptor to discover it. I saw the angel in the marble
and carved until I set him free.” —Michelangelo
The Italian Renaissance architect, sculptor, and painter
Michelangelo claimed that he could see “David“ inside
the uncarved marble before he created the masterpiece
(Perkin 2018) (Fig. 2). Very few gifted people presumably
have the ability to imagine a completed product within an
uncarved object. However, with the advancement of AR
technology, the overlay of a virtual object onto a mate-
rial reality has become possible. With a digitally built 3D
model, and through apps for Smart Glasses, such as the
Microsoft HoloLens, or Smart Mobile Devices, one can
visualize a nished product inside uncarved material as
if one possessed the talent described in Michelangelo’s
quote (Fig. 1).
This study aims to make carving and sculpting more
approachable to laypeople through the aid of holographic
instructions. The workow presented in this paper uses AR
to overlay extracted information from 3D models directly
onto the sculptor’s eld of vision. This information can
then become an AR-driven guidance system that assists
the sculptor in their actions. Using the Microsoft HoloLens,
holographic instructions are introduced in the production
sequence, connecting the analog sculpture fabrication
directly with a digital environment, thus augmenting crafts-
people’s agency. A series of AR-aided sculpting methods
were developed and tested in a demonstrator case study
project that created a small-scale clay copy of Henry
Moore’s Sheep Piece (1971–1972).
Contextualizing this in the world of modern architecture
construction, the possibility is proposed that a wider
scope of workers can participate in complex crafting work,
without the need of extensive prior craft-specic skills.
This study explores the augmentation of craft by combining
art, technology, and production.
BACKGROUND
Sculpting
This paper challenges the historical reliance of sculpting
on skilled crafting. Since the start of human civilization,
sculptures have evolved from representing mythical
gods, ancient rulers, and religious characters to the
worshipping of socially and politically respected individ-
uals. Transcendentally, pieces of craft appear in the form
of artistic expressions, realistic human gures, as well as
architectural elements (Palagia 2006). Basic traditional 2 David (Michelangelo, 1504).
438
3
sculpting techniques include carving and modeling through
addition. They can be applied to stone, metal, ceramics,
wood, and other materials. The difference in materiality
leads to the use of different tools. Metal sculptures, espe-
cially bronze, are usually duplicated from an existing model
using a lost-wax casting method. For clay and other similar
softer mediums, human hands are the tools to shape
matter at the stage of modeling when malleable parts are
added and smudged to desired forms. In addition, small
wooden tools can be helpful, some of which commonly have
strong wire curves at the extremities (Lanteri 2018). For
harder materials like stone and marble, the techniques
and procedures will be different: chisels of various sizes
and end shapes are used, and a mallet is essential to strike
the other tools against the stone. Due to the irreversibility
of sculpting on these materials, drills and bouchardes are
used in critical carvings as tools suitable to carve at right
angles (Palagia 2006).
Although the tools and detailed techniques vary for
different sculptures, their processes share some common-
alities. In general, the uncarved material is rst xed onto
a working base as preparation, often holes are drilled, or
a penetration through the material is needed to join it to a
base (Lanteri 2018). Next, wide planar areas are removed
using coarser tools to achieve a rough form. Finer
modeling is then done by smaller tools with fewer claws or
curves. After this modeling stage is completed, tool marks
are removed using any means suitable to the material
(Palagia 2006).
This raises the question of whether people must receive
extensive artistic training before being able to use these
tools to create sculptures. A similar question was central
to the 2013 documentary lm Tim’s Vermeer, in which Tim
Jenison investigated a possible methodology the Dutch
painter Johannes Vermeer (1632–1675) could have used
to create his paintings, which are typied by the hyperac-
curate and realistic depiction of light and color, including
chromatic aberrations and depth of eld normally impos-
sible to perceive with the naked eye alone (Fig. 3). Jenison
discovered that using a rather ingenious small device
with a particularly placed mirror allowed him to recreate
quasi-identical realism, despite not having any back-
ground in painting (Kenny 2014). In doing so, an argument
is made for clever tinkering and creative engineering as
being an essential and integral part of art production—an
argument this paper takes into today’s context where
augmented reality (AR) and mixed reality (MR) technology
are becoming part of readily available and everyday means.
3D Referencing and Photogrammetry
The methodology presented in this paper relies extensively
on 3D referencing. The concept of 3D referencing already
appeared during the Renaissance, where a form of it was
implemented by architect Leon Battista Alberti (1404–
1472). Alberti proposed a revolutionary 3D fabrication and
design method based on digital data that excluded conven-
tional drawings. He invented a measuring device that was
to be positioned on top of the body to be measured (Fig. 4).
Then, sculptors were to mark down the spatial coordinates
of as many different points of the body as possible and
4 5
Augmenting Craft With Mixed Reality Fong, Poon, Ngan, Ho, Goepel, Crolla
3 Tim Jenison duplicating The Music Lesson by Johannes Vermeer.
4 Leon Battista Alberti’s device for 3D referencing by measuring spatial coordinates (15th century).
5 Henry Dexter’s “Apparatus for Sculptors“ to duplicate sculpture (1842).
439CULTURE AND ACCESS
use this numeric data to create copies (Carpo 2017). With
this method, Alberti in principle enabled multiple studios
in different places to reproduce different parts of the
same statue, which could then be reassembled perfectly
(Grayson 1972).
This method, however, was never widely implemented, as
Alberti allegedly failed to explain clearly how the spatial
coordinates recorded could have been used to execute a
sculpture (Wrobel 1987). Nonetheless, Alberti’s proposal
provided the necessary insights for following generations
to develop a machine to replicate “perfect copies of any
model.” In 1842, American sculptor Henry Dexter (1806–
1876) developed the “Apparatus for Sculptors,” which
guided the sculpting of uncarved marble by measuring the
same distances from a nished marble product (Wrobel
1987) (Fig. 5). Such appliances increased the control of
less-specialized craftspeople over their work. Yet mass
production was still only possible after a rst master
sculpture was completed, requiring an expert sculptor.
This study uses photogrammetry as a 3D referencing
tool. By scanning the gradually transforming sculpture in
between carving sequences and comparing it to the digital
target sculpture, instructions can be updated at the begin-
ning of each iteration. This allows the sculptor to carve
repeatedly through the iterations until the discrepancies
between the physical object’s scan and the digital base
model are acceptable.
Augmented Reality (AR)
The third main component of this study addresses the
increasing ability for nonspecialist users to integrate
augmented reality into creative design and production.
AR technology applications are developing exponentially
6 Left: Clay scale model copy (authors, 2019). Right: Sheep Piece (Henry Moore, 1971–1972).
6
as more and more software and hardware tools and
techniques are becoming available in the market. Today,
architectural and engineering applications generally facili-
tate information extraction from design information models
to improve the efciency and effectiveness of workers’
tasks (Chi, Kang, and Wang 2013; Chu, Matthews, and Love
2018). These include onsite applications where AR can be
seen implemented in smart helmets and tablets, primarily
to help engineers make more accurate and rapid judg-
ments for construction review tasks (Ren, Liu, and Ruan
2017). In industrial settings, case studies of AR systems’
user experiences have demonstrated their potential to
reduce errors in assembly and improve the quality of
maintenance work (Aromaa et al. 2018). A wide range of
AR applications can be found in medicine, art, marketing,
communication, etc.
This study focuses on the implementation of commonly
available AR and photogrammetry techniques in sculpture
production. It uses them to develop a method that permits
nontrained users to achieve higher precision during the
sculpting process. For this, it holographically overlays
instructions from digital 3D source geometry on top of the
sculpting material using smart glasses and mobile devices
like smartphones and tablets, and uses these same devices
for iterative 3D referencing through photogrammetry.
OBJECTIVE
The objective of this study is to iteratively develop and
test clay modeling methods that incorporate augmented
reality to increase a layperson’s craft, agency, and
opportunities, and to validate the method’s effective-
ness through a demonstrator. For this, the study utilizes
digital sculpture models and holographic projections to
create a novel AR-driven guidance system, building on
440
fabrication concepts initiated by Alberti and Dexter. With
Microsoft’s HoloLens as a central tool, mixed-reality
holographic instructions are introduced to augment and
connect analog sculpture fabrication directly with a digital
environment.
One limitation of Henry Dexter’s “Apparatus for Sculptors”
lies in the difculty of producing the initial reference model.
However, with the ability to replace this with a virtual
model, either newly digitally modeled or 3D-scanned by the
user, or obtained from online resources, the procedure to
carve or replicate a sculpture can be dramatically simpli-
ed. In doing so, the proposed postdigital “Apparatus for
Sculptors” allows unskilled workers to “see” the virtual
model together with the analog sculpting material and
create a precise replica accordingly.
METHOD
A series of AR-aided sculpting techniques were developed
and tested, using as a case study the creation of a small-
scale clay copy of Henry Moore’s Sheep Piece (1971–1972)
(Fig. 7). For this, we used the software platform Fologram
(Fologram 2020), an add-on to McNeel’s Rhinoceros
procedural modeler Grasshopper that allows for real-
time streaming of digital data to the Microsoft HoloLens.
Currently, the built-in capacity of the HoloLens to automat-
ically scan its surrounding is still too rough and imprecise
to permit its direct use in our workow. Therefore, open-
source photogrammetry software Meshroom (AliceVision
2021) was used in combination with high-denition photo
cameras from handheld smartphone devices to carry out
3D scanning of the sculpture and derive a digital mesh
from the physical sculpted matter.
The overall method contained two major processes, with
rst an iterative “rough cutting” process to get a relatively
precise overall shape approximation in the clay block, and
then a “ne cutting” process in which wooden sticks were
used to guide further carving (Fig. 7). Photogrammetry
was used in both to iteratively measure and visualize the
differences between actual modeled clay geometry and
the target digital model le, thus providing a feedback loop
between the physical model and the digital source le.
The rst step of rough cutting visualizes the distance
between the 3D-scanned actual clay block and the virtual
model by overlaying a predened color spectrum onto the
clay block. This holographic instruction helps determine
where large areas of the clay block are to be trimmed
(Fig. 8). The setup starts with the acquisition or creation
of a digital mesh model of the desired sculpture—for our
demonstrator, an open-source model of Sheep Piece. This
model is scaled down and placed to t within a bounding
box with the dimensions of the to-be-sculpted material
block—here, a block of pottery clay measuring roughly
400 mm x 300 mm x 300mm. This block is 3D-scanned
and brought into the digital working environment. There,
a sphere is digitally centered on the sculpture model
geometry and clay block scan with a radius big enough
to encompass both. On this sphere, numerous points are
7 Workow diagram.
7
Augmenting Craft With Mixed Reality Fong, Poon, Ngan, Ho, Goepel, Crolla
441
8
9
10
8 Round 1—rough cutting: model differences are visualized by overlaying color information onto the clay block.
9 Overcarved areas are highlighted in red.
10 Round 2—ne cutting: wooden sticks are inserted according to the model differences.
evenly populated and connected to the center through a
line. This line is trimmed at both the clay bounding box and
the sculpture’s model geometry. The remainders’ lengths
dene the required carving depth. A mesh of the clay
bounding box is then colored at intersection points with
these remainders according to their lengths, with red to
green zones indicating greater to smaller proximity to the
sculpture (Fig. 8). Using the HoloLens, this information is
then holographically overlaid on top of the physical clay
block, indicating areas where material removal is needed.
The edited clay block is then rescanned and its geometry
replaces the previous digital model of the clay block, with
the script instantly updating its color-coded mesh for
streaming to the HoloLens. Upon updating the data, the
sculptor can repeat the carving process guided by a new
spectrum of color codes again. A built-in warning system
automatically informs the sculptor of overcarving, as those
areas will be indicated in red (Fig. 9).
Once the overall shape is sufciently approximated, a
second tool guides the ner nishing touches. In detailed
areas with only minimal distance between the physical clay
surfaces and the digital targeted sculpture, the previ-
ously derived trimmed lines with distance information are
isolated and displayed on the HoloLens. These lines now
precisely and directly operate as guides for the insertion of
wooden sticks into the clay mass to identify how deep the
remaining carving must go. Once all distances and angles
of the physical sticks match the digital measurements,
the sculptor removes the sticks, and the holes left behind
provide the sculptor the reference needed to carve until
their imprint can no longer be seen on the clay surface (Fig.
10).
OBSERVATIONS
Efficiency and Accuracy
The production of our demonstrator combined the effort
of four laypeople. Once the digital workow was speci-
ed, the fabrication of the sculpture copy, involving three
rounds of rough carving and one of ne carving, took the
team around 12 hours. The delity of the analog replica-
tion of the digital model can be evaluated by comparing the
3D scan of the clay model with the digital model. Our nal
sculpture was estimated by the system to be largely within
the 5 mm accuracy range.
Alternative Measuring Methods to Increase Accuracy
In the process of visualizing the measured distances
between scanned object and target object, two coloration
methods were tested: relative or absolute value visual-
ization. Absolute distance visualizations used the actual
distance between the two objects over the color spectrum
CULTURE AND ACCESS
442
and proved to be a great tool early on in round 1 of the
process to gauge when larger parts of the massing need
to be removed. Relative distance visualization provides
color information in relation to the observed range of
lengths, and is useful to more clearly identify and high-
light the extremities and the ner discrepancy distribution
throughout the sculpture (Fig 11).
In round 2, when wooden sticks are inserted in the
clay, two ways of measuring distances were also tested.
Lines could either be placed between the surfaces of the
scanned object and the target object according to the
normal vectors of the scanned object surface, or radially
from the centroid of the target object. The latter allowed
the lines to look more organized and be installed slightly
easier, but didn’t allow for the capturing of more intricate
and detailed surface orientations.
Alternative Photogrammetry Tools
Several 3D-scanning tools and techniques were tested
in this research. In addition to using 3D cameras, many
open-source 3D scanning software platforms were avail-
able online for our use. In this research, we tested the
limitations and advantages of three selected 3D-scanning
platforms: Meshroom, 3DF Zephyr, and Colmap. All of these
work by using snapshots taken from a camera circulating
around the material to reconstruct a solid shape through
computational triangulation. Meshroom provided the
most control options for the nal product and gave the
most precise results. Comparatively, however, it required
relatively long generation times. 3DF Zephyr has fewer
options to customize and edit the outcome but was the
quickest to produce 3D-scanned objects of acceptable
resolution. Therefore, if a high-quality 3D scan of the
object is required, like during the nal stages of sculpting,
Meshroom is recommended. During earlier, more rough
carving processes, 3DF Zephyr is recommended as it takes
less time and allows for faster iterations.
(Ir)reversibilit y
We opted to use clay for our demonstrator project, which
as a material is highly controllable. Its sculpting method
using carving wires, scrapers, and spatulas is relatively
exible, and mistakes are reversible, making it a very suit-
able rst study material. Other materials and techniques,
like stone or wood carving, are not as forgiving, and
further testing of the workow in these contexts is needed.
For hard or brittle sculpting materials, chiseling tech-
niques should be employed, which can be incorporated in
an adaptation of the workow now that the data feedback
system between the virtual and physical model is in place.
DISCUSSION
The study shows how a workow augmented by AR
technology can facilitate and improve end user’s effective-
ness and agency and allow laypeople to engage with an
otherwise prohibitively challenging trade. In doing so, this
pragmatic evolution of a manual craft might help increase
its competitiveness when pressured by mechanical
automation.
As demonstrated by work from peers, applications can
easily be diversied to other trades, such as bricklaying
(Franco 2019), plywood construction (Jahn, Wit, and Pazzi
2019), steel artwork production (Jahn et al. 2018), bamboo
construction (Goepel and Crolla 2020), and many more,
indicating that a paradigm shift in manual production has
been set in motion. Rather than surrendering human skill
to automation in manufacturing, AR enhances human
capacities to participate in complex processes through
simplied instructions (Goepel 2019).
Bottlenecks for this shift are largely hardware-based. In
our example here, the integration in the workow of accu-
rate real-time environment scanning could remove the
necessary tedious step of photogrammetry. These chal-
lenges can be presumed to be solved soon, as research by,
11 Absolute (left) versus relative (right) distance value visualization.
11
Augmenting Craft With Mixed Reality Fong, Poon, Ngan, Ho, Goepel, Crolla
443
for example, Microsoft’s Mixed Reality Capture Studios is
rapidly advancing (Roettgers 2018). Future generations of
mixed reality headsets will likely improve their 3D-scanning
functionality, allowing processed scanned data to be fed
back to the craftspeople synchronously with their manual
actions, allowing for even smoother real-time sculpting.
CONCLUSION
This study demonstrates how the fusion of computational
design technology and AR visualization technology can
bring innovative solutions to a specic analog craft’s
design and production workows. User-friendly software
and hardware tools have lowered the threshold to a point
where end users can practically facilitate and improve
their effectiveness and agency. Here, through the aid of
holographic instructions, amateur sculptors succeeded
in replicating complex sculpture geometry with sufcient
accuracy without prior training. These and similar tools
facilitate democratizing computational design and fabrica-
tion techniques beyond the typical applications in CNC or
robotic fabrication. This paradigm shift in manual crafting
has the potential to profoundly impact the future of manual
trades, often under pressure from automation, suggesting
opportunities for substantially widening their practically
feasible implementation solution space.
ACKNOWLEDGMENTS
This project was developed by Jacky Chun Hin Fong, Adabelle
Long Wun Poon, Wing Sze Ngan, and Chung Hei Ho under the
supervision of Garvin Goepel and Dr. Kristof Crolla as part of a
computational design course at the Chinese University of Hong
Kong (CUHK) School of Architecture.
REFERENCES
AliceVision. 2021. Meshroom. V.2021.1.0. AliceVision. PC.
Aromaa, S., A. Väätänen, E. Kaasinen, M. Uimonen, and S. Siltanen.
2018. Human Factors and Ergonomics Evaluation of a Tablet Based
Augmented Reality System in Maintenance Work.” In Mindtrek
'18 [Proceedings of the 22nd International Academic Mindtrek
Conference], Tampere, Finland, 10–11 October 2018, 118–125.
Association for Computing Machinery.
Carpo, Mario. 2017. The Second Digital Turn: Design beyond
Intelligence. Cambridge, MA: MIT Press.
Chi, H.-L., S.-C. Kang, and X. Wang. 2013. “Research Trends and
Opportunities of Augmented Reality Applications in Architecture,
Engineering, and Construction.Automation in Construction 33:
116–122.
Chu, M., J. Matthews, and P. E. D. Love. 2018. “Integrating Mobile
Building Information Modelling and Augmented Reality Systems: An
Experimental Study.Automation in Construction 85: 305–316.
Crolla, Kristof. 2018. “Building Simplexity: The ‘more or less’ of
post-digital architecture practice.” PhD diss., RMIT University.
Fologram. 2020. Fologram. V.2020.3.23. Fologram Pty Ltd. PC.
Franco, José Tomás. 2019. “This Is How a Complex
Brick Wall Is Built Using Augmented Reality.ArchDaily,
January 25. https://www.archdaily.com/908618/
this-is-how-a-complex-brick-wall-is-built-using-augmented-reality.
Goepel, Garvin. 2019. “Augmented Construction: Impact and
Opportunity of Mixed Reality Integration in Architectural
Design Implementation.” In ACADIA 19: Ubiquity and Autonomy
[Proceedings of the 39th Annual Conference of the Association for
Computer Aided Design in Architecture (ACADIA)], Austin, TX, 21–26
October 2019, edited by K. Bieg, D. Briscoe, and C. Odom, 430–437.
CUMINCAD.
Goepel, Garvin, and Kristof Crolla. 2020. “Augmented Reality-Based
Collaboration—ARgan, a bamboo art installation case study.” In
Anthropocene [Proceedings of the 25th International Conference of
the Association for Computer-Aided Architectural Design Research
in Asia (CAADRIA)], Bangkok, Thailand, 5–6 August 2020, 313–322.
CAADRIA.
Grayson, Cecil. 1972. Leon Battista Alberti: On Painting and On
Sculpture. London: Phaidon Press Limited.
Jahn, Gwyllim, Cameron Newnham, Nicholas van den Berg, and
Matthew Beanland. 2018. “Making in Mixed Reality: Holographic
Design, Fabrication, Assembly and Analysis Of Woven Steel
Structures.” In ACADIA 2018: Recalibration: On Imprecision and
Indelity [Proceedings of the 38th Annual Conference of the
Association for Computer Aided Design in Architecture (ACADIA)],
Mexico City, Mexico, 18–20 October 2018, edited by P. Anzalone, M.
del Signore, and A. J. Wit, 88–97. CUMINCAD.
Jahn, Gwyllim, Andrew Wit, and James Pazzi. 2019. “[BENT]
Holographic Handcraft in Large-Scale Steam-Bent Timber
Structures.” In ACADIA 19: Ubiquity and Autonomy [Proceedings of
the 39th Annual Conference of the Association for Computer Aided
Design in Architecture (ACADIA)], Austin, TX, 21–26 October 2019,
edited by K. Bieg, D. Briscoe, and C. Odom, 438–447. CUMINCAD.
Kenny, Glenn. 2014. “Tim’s Vermeer.” Roger Ebert.com, January 31.
https://www.rogerebert.com/reviews/tims-vermeer-2013.
CULTURE AND ACCESS
444
Lanteri, Edward. 2018. Modelling: A Guide for Teachers and
Students. California: Palala Press.
Palagia, Olga. 2006. Greek Sculpture: Function, Materials, and
Techniques in the Archaic and Classical Periods. Cambridge:
Cambridge University Press.
Perkin, Neil. 2018. “The Angel in the Marble.” Only Dead Fish
(blog), August 24. https://www.onlydeadsh.co.uk/only_dead_
sh/2018/08/the-angel-in-the-marble.html.
Ren, J., Y. Liu, and Z. Ruan. 2016. “Architecture in an Age of
Augmented Reality: Applications and Practices for Mobile
Intelligence BIM-based AR in the Entire Lifecycle.” In International
Conference on Electronic Information Technology and
Intellectualization 2016, Guangzhou, China, 18–19 June 2016,
664–665. DEStech Publications.
Roettgers, Janko. 2018. “106 Cameras, Holograms and
Sticky Tape: Inside Microsoft’s Mixed Reality Capture Studios.
Variety, April 24. https://variety.com/2018/digital/features/
microsoft-mixed-reality-capture-behind-the-scenes-1202784950/.
Wrobel, Arthur. 1987. Pseudo-Science and Society in 19th-Century
America. Lexington: University Press of Kentucky.
IMAGE CREDITS
Figure 2: © Jörg Bittner Unna, 2008.
Figure 3: © The Washington Post, 2014.
Figures 4–5: © The Second Digital Turn: Design Beyond
Intelligence, 2017.
Figure 6 (right): © The Nelson-Atkins Museum of Art.
All other drawings and images by the authors.
Chun Hin Jacky Fong is a Master of Architecture graduate from
The Chinese University of Hong Kong (CUHK) School of Architecture
(SoA). He is the recent president of the Graduation Show 2020
Committee. His master’s thesis was nominated to participate in
the ARCASIA Thesis of the Year Awards 2020 and explored the
relationship between the Cantonese language and architecture
through bamboo theaters as traditional spaces for performing arts
in Hong Kong. His research in AR technology received the CUHK
CAADRIA Student Award 2020—commendation.
Long Wun Adabelle Poon is a CUHK SoA Master of Architecture
student. She worked previously on East Kowloon Cultural Centre
and its theater spaces as a year-out in a local rm. Prior to her
postgraduate study, her collaborative project, Experiential Food
Arcadia, was shortlisted in a competition by Unfuse. Her master’s
thesis explores methods to deal with negative emotions through
sensory experiences and body architecture in Hong Kong’s urban
setting. Her research in AR technology received the CUHK CAADRIA
Student Award 2020—commendation.
Wing Sze Ngan is a CUHK SoA Master of Architecture grad-
uate who was nominated by the social science faculty to be the
recipient of Lion Dr. Francis K. Pan Scholarship 2019/20. Her
master’s thesis, selected by the school as one of the best works
of the year, explores an open-source design framework that
preserves the bottom-up construction technique and spirits of Dai
Pai Dong, an important cultural asset of Hong Kong. Her research
in AR technology received the CUHK CAADRIA Student Award
2020—commendation.
Chung Hei Ho is a CUHK SoA undergraduate who works in a local
Hong Kong architecture rm on projects combining Modular
Integrated Construction (MIC) and BIM modeling. His research
in AR technology received the CUHK CAADRIA Student Award
2020—commendation.
Garvin Goepel is a PhD researcher at CUHK specializing in
augmented reality (AR) implementation in fabrication and design
processes. He received his Master of Architecture degree with
distinction from die Angewandte Studio Greg Lynn, with his thesis
“AUGSTRUCTION.” He has taught AR and design workshops at
multiple international institutions and gained academic experience
by researching at CUHK, with Kristof Crolla, focusing on bending
active bamboo grid shells and collaborative holographic fabrication
techniques. From spring to summer 2020, he joined the ETH’s Block
Research Group.
Kristof Crolla PhD is an architect who combines his architectural
practice, Laboratory for Explorative Architecture & Design Ltd.
(LEAD), with an Associate Professorship at the University of Hong
Kong’s Departments of Architecture & Civil Engineering. His work
on the strategic integration of latest technologies in the architec-
tural design and implementation process has received numerous
design, research, and teaching awards and accolades, including
the RMIT Vice-Chancellor’s Prize for Research Impact—Higher
Degree by Research. He is best known for the projects “Golden
Moon“ and “ZCB Bamboo Pavilion,“ for which he received the World
Architecture Festival Small Project of the Year 2016 award.
Augmenting Craft With Mixed Reality Fong, Poon, Ngan, Ho, Goepel, Crolla
Conference Paper
Full-text available
Augmented reality environments have been demonstrated to assist with architectural fabrication tasks by displaying construction information at full scale and in context. However, this information typically needs to be sparse in order to prevent virtual models occluding a fabricators view of the physical environment, and this limits the application of augmented reality to tasks such as surface forming. To address this issue, we propose a method for guided fabrication in augmented reality using real time comparisons between depth scans of as built conditions and target conditions defined by design models. Through the design and fabrication of a small proof of concept prototype from paper strips, we demonstrate that guided fabrication is adequate for high speed, approximate and ad-hoc fabrication of complex surface geometries without the need for extensive rationalization for fabrication constraints or explicit documentation of parts. We further show how this method generalizes to other processes such as additive fabrication or part placement and speculate on the implications of accessible real time depth data from the HoloLens within Grasshopper.
Conference Paper
Full-text available
ARgan is a geometrically complex bamboo sculpture that relied on Mixed Reality (MR) for its joint creation by multiple sculptors and used latest Augmented Reality (AR) technology to guide manual fabrication actions. It was built at the Chinese University of Hong Kong in the fall of 2019 by thirty participants of a design-and-build workshop on the integration of AR in construction. As part of its construction workflow, holographic setups were created on multiple devices, including a series of Microsoft HoloLenses and several handheld Smartphones, all linked simultaneously to a single digital base model to interactively guide the manufacturing process. This paper critically evaluates the experience of extending recent AR and MR tool developments towards applications that centre on creative collaborative production. Using ARgan as a demonstrator project, its developed workflow is assessed on its ability to transform a geometrically complex digitally drafted design to its final physically built form, highlighting the necessary strategic integration of variability as an opportunity to relax notions on design precision and exact control. The paper concludes with a plea for digital technology’s ability to stimulate dialogue and collaboration in creative production and augment craftsmanship, thus providing greater agency and more diverse design output.
Conference Paper
Full-text available
This paper discusses the integration of Mixed Reality in the design and implementation of non-standard architecture. It deliberates a method that does not require conventional 2D drawings, and the need for skilled labor, by using the aid of holographic instructions. Augmented Construction allow builders to execute complex tasks and to understand structural relations intuitively by overlaying digital design information onto their field of view on the building site. This gives the implementation system authors different levels of control. As a proof of concept, a group of non-professionals reconstructed the south wall of Corbusier’s Ronchamp chapel, the Notre-Dame du Haut, at scale 1:5 using no architectural 2D drawings but only custom-built Augmented Reality apps for HoloLens and mobile devices. This project focused on the assembly of non-standard prefabricated elements, based on an optimized parametric structure that enables designers to integrate imprecision within the construction phases into the design through a constant feedback-loop between the real and the digital. The setup was designed in a non-linear process that allows the integration of new information during the Augmented Construction phases. The paper evaluates applied Augmented Construction for further improvements and research and concludes by discussing the impact potential of Augmented Construction on architectural design, socio-cultural, and economical levels.
Conference Paper
Augmented reality (AR) technologies start to be mature enough to be used in industrial work settings. However, human factors and ergonomics (HFE) and safety issues have not been considered thoroughly yet. The purpose of this study was to identify what kind of postures users adopt when using a tablet based AR system during a maintenance task. In addition, safety, user experience and user acceptance were studied. Results indicate that the participants adopted varying kind of working postures with the AR system, but none of the postures were severe for the well-being. User experience was positive and user acceptance on a good level. The participants saw some safety concerns related to using the AR system but were mainly concerned if the tablet could be used in the harsh maintenance environments. The findings of this study can be used to improve HFE and safety of AR systems in industrial settings.
Building Simplexity: The 'more or less' of post-digital architecture practice
  • Kristof Crolla
Crolla, Kristof. 2018. "Building Simplexity: The 'more or less' of post-digital architecture practice." PhD diss., RMIT University.
This Is How a Complex Brick Wall Is Built Using Augmented Reality
  • José Franco
  • Tomás
Franco, José Tomás. 2019. "This Is How a Complex Brick Wall Is Built Using Augmented Reality." ArchDaily, January 25. https://www.archdaily.com/908618/ this-is-how-a-complex-brick-wall-is-built-using-augmented-reality.
Leon Battista Alberti: On Painting and On Sculpture
  • Cecil Grayson
Grayson, Cecil. 1972. Leon Battista Alberti: On Painting and On Sculpture. London: Phaidon Press Limited.