Storytelling in extended reality (XR) for spatial experiences. What are the best practices and relevant strategies for using augmented reality (AR) in exhibitions?

Abstract

Stories shape us as humans. They are one of the fundamental factors in literature, radio, film, and other media. For content creators, storytelling has dramatically evolved since the popularization of mass media, the personal computer (PC), and the smartphone. Over time, extended reality (XR) technologies will likely transform our current media landscape. Unlike other digital media, XR allows us to fundamentally shift the story perspective from third-person to first-person narrative, redefining the presence. How is XR related to augmented reality (AR)? How can experience designers use XR effectively to tell a story? New possibilities are emerging every year in XR, and content creators are continuously searching for new ways to create experiences. How will XR storytelling affect experiences in a given space? This study analyzes existing theories and empirical data in XR, AR, storytelling, and spatial design to explore different narrative approaches for spatial experiences in AR. Research questions: What are the defining media attributes of XR/AR? What new possibilities does AR offer for spatial experience? What are the key narrative aspects of AR experiences in built environments? What are the essential factors for integrating AR in built spaces and exhibitions? What progress can be expected in the coming years?

Full text

Storytelling in extended reality (XR)
for spatial experiences
What are the best practices and relevant strategies
for using augmented reality (AR) in exhibitions?
Bosen Zhou
bosezhou@gmail.com
M.A. Erlebniskommunikation | 181000129
Supervisors: Prof. Sebastian Waschulewski, Pablo Dornhege
Berlin, November 2020

Masterarbeit | Bosen Zhou
Abstract
Stories shape us as humans. They are one of the fundamental factors in literature,
radio, film, and other media. For content creators, storytelling has dramatically
evolved since the popularization of mass media, the personal computer (PC), and
the smartphone. Over time, extended reality (XR) technologies will likely transform
our current media landscape. Unlike other digital media, XR allows us to
fundamentally shift the story perspective from third-person to first-person
narrative, redefining the presence. How is XR related to augmented reality (AR)?
How can experience designers use XR effectively to tell a story? New possibilities
are emerging every year in XR, and content creators are continuously searching for
new ways to create experiences. How will XR storytelling affect experiences in a
given space? This study analyzes existing theories and empirical data in XR, AR,
storytelling, and spatial design to explore different narrative approaches for spatial
experiences in AR.
Research questions:
What are the defining media attributes of XR/AR?
What new possibilities does AR offer for spatial experience?
What are the key narrative aspects of AR experiences in built environments?
What are the essential factors for integrating AR in built spaces and exhibitions?
What progress can be expected in the coming years?
Keywords:
Extended reality (XR), Augmented reality (AR), Storytelling, XR design, Spatial
experience, Exhibition design, Immersive storytelling, Interactive storytelling.
II

Masterarbeit | Bosen Zhou
Contents
Abstract II .......................................................................................................................
Contents III ....................................................................................................................
Abbreviations V ..............................................................................................................
1. Introduction 1 .............................................................................................................
1.1 Research question 2 ...............................................................................................
1.2 Chapter outline 4 ...................................................................................................
1.3 Methodology 4 .......................................................................................................
2. Context 8 .....................................................................................................................
2.1 Extended reality 8 .................................................................................................
2.1.1 Augmented reality 10 .....................................................................................
2.1.2 Mobile augmented reality 11 ..........................................................................
2.1.3 Spatial augmented reality 12 .........................................................................
2.1.4 Digital eyewear 13 ..........................................................................................
2.2 Storytelling 15 .......................................................................................................
2.2.1 Media and storytelling 15 ..............................................................................
2.2.2 The nature of stories 18 .................................................................................
2.3 Spatial experiences 22 ..........................................................................................
2.3.1 Dimensions of experience 23 ........................................................................
2.3.2 Narrative environments 25 ...........................................................................
2.3.3 Storytelling in exhibitions 28 ........................................................................
3. Analysis 30 ..................................................................................................................
3.1 XR storytelling 30 .................................................................................................
3.1.1 A new narrative media 31 ...............................................................................
III

Masterarbeit | Bosen Zhou
3.1.2 AR media attributes 34 ..................................................................................
3.2 AR in exhibition 36 ...............................................................................................
3.2.1 Challenges 38 .................................................................................................
3.2.2 Opportunities 40 ...........................................................................................
3.3 AR storytelling 42 .................................................................................................
3.3.1 People and interaction 43 ..............................................................................
3.3.2 Space and perspective 46 ..............................................................................
3.3.3 Narrative and emotion 47 .............................................................................
3.4 AR integration 50 .................................................................................................
3.4.1 Interdisciplinarity 50 .....................................................................................
3.4.2 Design process 53 ..........................................................................................
3.4.3 Best practices 55 ............................................................................................
3.5 Future scenarios 57 ...............................................................................................
4. Conclusion 60 ............................................................................................................
4.1 A new path for storytelling 61 ...............................................................................
4.2 AR for spatial experiences 62 ...............................................................................
4.3 AR narrative fundamentals 63 .............................................................................
4.4 Building stories in AR 64 ......................................................................................
4.5 Limitations and future research 65 ......................................................................
5. References 67 ..............................................................................................................
6. Appendices 71 .............................................................................................................
6.1 Appendix A: Axial codes 71 ...................................................................................
6.2 Appendix B: Topic guide 74.................................................................................
IV

Masterarbeit | Bosen Zhou
Abbreviations
6DoF Six Degrees of Freedom
AI Artificial Intelligence
ANT Actor-Network Theory
API Application Programming Interface
AR Augmented Reality
AV Augmented Virtuality
CPU Computer Processing Unit
FOV Field of View
GPS Global Positioning System
GPU Graphics Processing Unit
HCI Human-Computer Interaction
HMD Head-Mounted Display
IoT Internet of Things
IPS Indoor Positioning System
LiDAR Light Detection and Ranging
MR Mixed Reality
PC Personal Computer
SAR Spatial Augmented Reality
SDK Software Development Kit
SLAM Simultaneous Localization and Mapping
UI User Interface
UX User Experience
VR Virtual Reality
XR Extended Reality
V

Masterarbeit | Bosen Zhou
1. Introduction
Stories shape us as humans, they define our cultures and values. Storytelling
techniques have significantly evolved together with the development of mass
media, the internet, and the smartphone, particularly in art, film, music, game,
design, and other creative industries. It is a direct result of technological
advancements that have been accelerating since the 20th century. Today, we can
obtain new devices every month for a lower price with faster computer processing
units (CPUs), more data storage, and better connectivity. These advancements
influence innovations such as Artificial Intelligence (AI), Blockchain, and the
Internet of Things (IoT). Among which, Extended Reality (XR) technology is one of
the fastest growing sectors, including Augmented Reality (AR), Virtual Reality
(VR), and Mixed Reality (MR). Unlike other digital media, XR allows us to
fundamentally shift the perspective of the story from third-person to first-person
narrative and redefine the presence for the users.
In time, XR will likely transform our current media landscape, yet how to get there
remains an open question. XR is currently a very fast-evolving area, with both
software and hardware seemingly changing by the week. Content creators are
continually searching for new approaches to create experiences while keeping pace
with technological advancements. Zuckerberg (2015) divided the XR ecosystem
into three major parts: apps/experiences, platform services, and hardware/
systems; these are listed in order of importance. Experience and content will be a
significant driving force to push the XR industry forward. How can content creators
use XR/AR effectively to tell a story? How will AR storytelling affect experiences in
a given space? This study aims to connect existing theories of XR, storytelling, and
spatial experience, together with qualitative research methods, to explore narrative
techniques and best practices for AR in built environments.
1

Masterarbeit | Bosen Zhou
1.1 Research question
The primary purpose of this study is to explore the narrative aspect of AR in built
spaces. A qualitative method was applied using in-depth interviews and content
analysis to gather qualitative data, and five primary factors are examined in this
paper. First, it outlines the media attributes of XR, specifically of AR, and the
influence of conventional media narrative techniques. Second, it evaluates the
approach of AR in exhibitions and spatial contexts. Third, it examines various
design processes for AR experience, including story development, interdisciplinary
collaboration, and user interaction. Fourth, it identifies some of the essential
narrative aspects of AR content creation. Last, it discusses the future of AR in
exhibitions and its relation to the XR industry. The initial research question has
been formulated as:
Storytelling in extended reality (XR) for spatial experiences. What
are the best practices and relevant strategies for using augmented
reality (AR) in exhibitions?
This research question is structured and followed by five subquestions:
SQ1: What are the defining media attributes of XR and AR? A broad scope
of literature about XR and AR was analyzed to overview the technical aspects of XR
media to address this question. Furthermore, pertinent theories in narratology and
storytelling were briefly introduced to comprehend the commonality and
differences between XR and traditional media. Moreover, existing frameworks
about spatial narratives and experiences were reviewed to address the spatial
aspects of AR.
2

Masterarbeit | Bosen Zhou
SQ2: What new possibilities does AR offer for spatial experience? This
subquestion addresses both the opportunities and challenges to the approach of AR
in exhibitions. Three different types of AR were introduced relating to spatial
experiences. Some best practice examples in the industry are also presented in this
section.
SQ3: What are the key narrative aspects of AR experiences in built
environments? This subquestion focuses on some of the most important
storytelling factors that emerged from theoretical and empirical knowledge. The
aim is to create a theoretical framework for AR storytelling in built spaces.
SQ4: What are the essential factors for integrating AR in built spaces
and exhibitions? This question explores the key factors in AR spatial experience
based on data from in-depth interviews, including design processes,
interdisciplinary work, and some best practices for using AR in exhibitions.
SQ5: What progress can be expected in the coming years? The last
subquestion gives an outlook on the future of location-based AR and addresses the
limitations and challenges of future research.
This study first highlights the narrative aspects of AR experience and summarizes
some best practices from content creators. Furthermore, it focuses on the
importance of built environments, particularly exhibitions, to deeper understand
AR’s nature as an emerging media. It also encourages interdisciplinary work and
explores some methods of the design process. The aim is to promote AR content
innovation in order to keep pace with technological advancements.
3

Masterarbeit | Bosen Zhou
1.2 Chapter outline
After the introductory chapter, chapter two will examine several related theories
and circumstances for AR storytelling in exhibitions. First, the concepts and
relations of XR and AR will be outlined, followed by the categorization of three
distinct AR technologies and the current state of development in AR. Second, the
evolution of narrative techniques in media and the nature of stories will be
described, focusing on digital and interactive storytelling. Last, selected theories
relating to AR spatial experience will be presented, including the dimensions of
experience, narrative environments, and narration in exhibitions.
Chapter three will present the analysis and results of this study based on the
research questions. First, storytelling in XR and AR will be discussed, along with
the fundamental AR media attributes. Second, the current challenges and
opportunities will be presented for implementing AR in exhibitions and built
environments. Third, a narrative framework will be proposed for using AR in built
spaces in the context of existing theories. Fourth, various essential factors and best
practices for implementing AR will be presented. Last, the chapter will introduce
future scenarios and prospects of AR technologies. The concluding chapter will
summarize the findings of this study by answering the research questions.
1.3 Methodology
This exploratory study is most suitable for qualitative research because XR/AR and
the narrative exploration of AR as media are still relatively new. The
interdisciplinary nature of AR increases the complexity of understanding this
media. Hence, the first step was to find relevant disciplines for location-based AR
4

Masterarbeit | Bosen Zhou
experience design in distinct academic fields, including computer science,
psychology, and ergonomics studies for AR and Human-Computer Interaction
(HCI), literary and media studies for storytelling, and spatial theory studies for
architecture and media installations. By connecting relevant theories, the results
can deepen the perception of the complexity and scope of AR. Such an approach
can challenge readers’ common-sense assumptions and practices (Tracy, 2010).
The second step was to conduct in-depth interviews to conceptualize ideas based
on past experiences and practices. Ten semi-structured interviews with industry
experts were conducted as the primary method of this study. Each interview lasted
around one hour, the sample size was determined by theoretical saturation, the
point at which no new concepts emerged from data. Strauss and Corbin (1998)
advise that at least ten interviews or observations with detailed coding are
necessary for building a grounded theory. The selected experts have worked in
content creation for AR and spatial experience, with over ten years of professional
experience on average. They are from different creative industry backgrounds,
including interaction design, communication design, exhibition, theater, film,
animation, and game development. They also have different cultural backgrounds.
The interviews were conducted in four languages: German, English, Chinese, and
Spanish. The interdisciplinary and intercultural approach of interviews can
generate new knowledge by looking at the same topic from different perspectives.
The aim is to determine the most relevant storytelling elements through worldwide
professional experience and collect the best practices in AR content creation.
The interview questions were semi-structured and open-ended to encourage in-
depth insight and opinion, focusing primarily on the topic of AR content creation,
while maintaining flexibility for new emerging concepts. First, the introductory
conversation provides a general overview of the experts’ background and creates a
relaxed atmosphere. Second, interviewees were asked about the approach of AR in
5

Masterarbeit | Bosen Zhou
exhibitions, best practice examples, and interdisciplinary work in different
industries. Following this, the interviewees were asked in-depth questions about
the fundamental storytelling elements in AR, providing insights from their varied
professional experience for later sampling. The interview questions were iterated in
different ways to ensure a complete overview of AR storytelling. After that, the
conversations addressed interaction as one of the most complex AR topics. Last,
interviewees were asked about the future development of AR to explore the
prospects of AR technologies in exhibitions.
After the interviews, the interview data was manually transcribed and coded
through two coding cycles, using the qualitative content analysis software
MAXQDA. The first cycle involved four methods: structural coding, holistic coding,
initial coding, and magnitude coding. Structural coding applies a content-based or
conceptual phrase representing a topic of inquiry to a segment of data that relates
to a specific research question used to frame the interview. Structural coding
generally results in the identification of large segments of text on broad topics;
these segments can then form the basis for an in-depth analysis within or across
topics (MacQueen et al., 2008, p. 125). Holistic coding is applicable when the
researcher already has a general idea of what to investigate in the data, or “to
‘chunk’ the text into broad topic areas, as a first step to seeing what is
there” (Bazeley, 2007, p. 67). Initial coding is breaking down qualitative data into
discrete parts, closely examining them, and comparing them for similarities and
differences (Strauss & Corbin, 1998, p.102). Magnitude coding is appropriate for
descriptive qualitative studies that include basic statistical information such as
frequencies or percentages, and qualitative studies in social science and health care
disciplines that also support quantitative measures as evidence of outcomes
(Saldaña, 2013, p. 73). In this study, magnitude coding was managed for some
specific questions to determine the approach of AR in exhibitions.
6

Masterarbeit | Bosen Zhou
The second cycle applied mixed coding methods, focused coding, and axial coding.
Focused coding searches for the most frequent or significant codes to develop “the
most salient categories” in the data corpus and “requires decisions about which
initial codes make the most analytic sense” (Charmaz, 2006, p. 46). Axial coding
extends the analytical work from initial coding and, to some extent, focused coding.
The goal is to strategically reassemble data that was “split” or “fractured” during
the initial coding process (Strauss & Corbin, 1998, p. 124). The “axis” of axial
coding is a category discerned from first cycle coding. This method “relates
categories to subcategories and specifies the properties and dimensions of a
category” (Charmaz, 2006, p.60). Grouping similarly coded data reduces the
number of initial codes developed while sorting and relabeling them into
conceptual categories. During this cycle, “the code is sharpened to achieve its best
fit” (Glaser, 1978, p.62).
After the second cycle, the results were organized and analyzed into different
categories. As the focus of this study, narrative elements were categorized into
three themes, extending upon the framework of narrative environments developed
by Tricia Austin. Additionally, some of the most relevant aspects of the AR
implementation and future scenarios were conceptualized to answer the research
questions.
7

Masterarbeit | Bosen Zhou
2. Context
2.1 Extended reality
In 1994, Paul Milgram introduced the term reality-virtuality continuum, classifying
all technologies related to AR and VR into one spectrum (Figure 1). The left side of
the continuum represents reality, and the right side represents solely the virtual
world. Within this framework, he defined the mixed reality (MR) environment as a
combination of the real-world and the virtual world. MR includes augmented
reality (AR), which refers to overlaying digital content on top of the real world, and
augmented virtuality (AV), which refers to merging real-world objects into virtual
worlds.
Figure 1: Simplified representation of Reality-Virtuality
Continuum, Paul Milgram, 1994.
Virtual reality (VR) has evolved in many different sectors, and several definitions of
VR have been used over time. Originally the majority of technological research in
the field of VR came from the flight simulation and military training industry,
however more recently the use of VR has expanded within consumer industries,
principally in entertainment and education. The definition of VR provided by
Unity, the most popular framework for the creation of VR content, is “computer-
generated stereo visuals which entirely surround the user, entirely replacing the
8

Masterarbeit | Bosen Zhou
real world environment around them” (Unity, 2020). Today, there are a variety of
VR headsets for consumer use on the market, such as Oculus, Playstation VR, and
HTC Vive.
By comparison, the development of AR has been relatively recent. Two current
leading AR hardware devices are Microsoft Hololens and Magic Leap. However,
Microsoft has been pushing the terminology mixed reality (MR) to the consumer
market in recent years. They defined mixed reality as a spectrum: “Mixed Reality is
a blend of physical and digital worlds… The combination of all three - computer
processing, human input, and environmental input - sets the stage for creating true
Mixed Reality experiences” (Microsoft, 2019). The use of the term MR has caused
some confusion relating to VR and AR. However, Qualcomm (2017) projects that
over time all these technologies will emerge into a more personalized user
experience, which will happen through a convergence of technologies under the
term extended reality (XR).
Extended reality or X-reality (XR) is an umbrella term to group different immersive
technologies into one. Unity defines it as “Technology-mediated experiences that
combine virtual and real-world environments and realities” (Unity, 2020). Here, X
represents a variable for any current or future spatial computing technologies,
encapsulating augmented reality (AR), mixed reality (MR), virtual reality (VR),
cinematic reality, and all other realities. According to this definition, the vision
behind AR and VR hardware will eventually converge into a new type of device
capable of supporting both.
Today, AR and VR are still distinct experiences; however, they share many of the
same underlying technologies. In 2019, the tech consortium Khronos Group
launched an initiative called OpenXR to create a standard application
programming interface (API) for both AR and VR, enabling developers to build
9

Masterarbeit | Bosen Zhou
applications that will not be strictly identified in either category. XR devices do not
yet exist, and it remains undetermined if and when this XR convergence will
happen. However, for the purposes of this study, the term XR is used to refer to
storytelling in all digitally extended environments.
2.1.1 Augmented reality
The term augmented reality (AR) was coined by former researcher Thomas Caudell
in 1990. Working in computer research at Boeing Caudell developed a heads-up
display to superimpose a computer generated diagram on components during
aircraft construction. Augment comes from the Latin word ‘augere’, meaning to
increase or to add, and AR’s main purpose is to enhance the real-world
environment, adding digital data to our existing reality.
Microsoft’s efforts to promote MR as their core terminology for Hololens have
caused a discrepancy in understanding between MR and AR. In the software
development world, the distinction between AR and MR is hierarchical. AR refers
to overlaying virtual objects on the real-world environment, primarily through the
use of smartphones and tablets, whilst MR overlays and anchors virtual objects to
the real world with digital glasses. In this sense, MR represents a more advanced
version of AR. However, in research and development, AR is still widely used to
refer to a range of technologies that allow users to interact with digital objects in
the physical world.
Current AR technologies include digital eyeglasses, optical projection systems, 3D
displays, head-up displays, spatial projection, contact lenses and mobile devices.
Most of these technologies are not yet in the consumer market or are still under
development. In this study, AR technologies are arranged into three categories:
mobile AR, spatial AR, and digital eyewear. This categorization focuses on the
10

Masterarbeit | Bosen Zhou
technologies relevant to spatial experiences and exhibitions, which are the focus of
this study. As AR technology develops, further categorizations may become
applicable, but as of 2020, these are the three most used AR applications in the
creative industry.
2.1.2 Mobile augmented reality
Mobile AR is currently the quickest growth area for AR applications, and applies to
those applications that you can take with you wherever you go (Craig, 2013),
primarily using widely available consumer hardware such as smartphones or
tablets. One of the most prominent examples is the mobile game Pokémon Go,
released in July 2016. Pokémon Go uses mobile devices to locate, capture, and
battle with virtual creatures, overlaid in real world environments. However, there
are still many technical limitations for AR applications within the game, such as
IPS (Indoor Positioning System) accuracy. Current mobile devices have different
GPUs, gyroscopes, and camera capacities that directly affect latency, position
accuracy, and plane detection, crucial factors for overlaying digital objects in
physical space. While the AR experience in Pokémon Go may not be perfect,
despite these technical issues, it represents a meaningful landmark in AR content
development.
Social media face filters are another success story in mobile AR. In 2017, Snapchat
introduced AR Lens Studio, using facial tracking technology for real-time video, to
allow end-users to alter their images with filters. In 2018, not long after the release
of this feature, Snapchat became the most popular AR-based app, according to one
case study (Clack, 2018). It did not take long for other companies to follow this
trend. By 2019, apps with AR face filters were commonplace: Facebook, Instagram,
TikTok, amongst others, now provide face filters as a core feature. Meanwhile,
companies such as Warner Brothers, Netflix, and Coca-Cola have used AR face
11

Masterarbeit | Bosen Zhou
filters to promote their products and services via social media apps, and a new
trend, AR marketing, started to appear in the marketing industry (Orgagui, 2019).
Around the same time, in 2017, Apple and Google introduced the ARcore and
ARKit SDKs for Android and iOS developers respectively. New software
possibilities are constantly emerging in mobile AR, making mobile AR experiences
more accessible than ever. Businesses such as IKEA use mobile AR to allow users to
visualize furniture in their personal space, prior to purchase. Meanwhile,
architecture, education, manufacturing, the visual arts, and other sectors are also
exploring the possibilities provided by mobile AR. With on-going advances in the
performance of smartphones, in particular the network performance brought by 5G
networks, the potential for mobile AR continues to grow. As a result many
businesses are exploring mobile AR applications as means to level up their user
experience.
2.1.3 Spatial augmented reality
Spatial augmented reality (SAR) refers to the augmentation of the real world
without using displays such as mobile, monitors, or glasses. Comparing to other
types of AR, spatial AR offers many advantages and solves several problems related
to visual quality, technical issues, and human factors. However, they are limited to
non-mobile applications (Bimber & Raskar, 2005). The most common example of
spatial AR is projection mapping, using projection to turn different shaped objects
into display surfaces. This technique is widely used in the art and exhibition
industry. Spatial AR also includes many other technologies that can enhance space
digitally, including 3D display and holographic projection.
In recent years, projection mappings have been favored to digitally animate
complex objects such as buildings, indoor objects, and theatrical stages. This
12

Masterarbeit | Bosen Zhou
technique can add extra dimensions and optical illusions to static objects. It is also
one of the most affordable options compared to other displays. Designers
commonly combine spatial AR with audio-visual narratives for cultural events.
Holographic technologies are another area of spatial AR that has been a shared
vision for future scenarios. In the Star Wars movies, Princess Leia pleads for help
using a holographic projection. The term hologram was established by electrical
engineer and physicist, Dennis Gabor, who later invented the holographic method,
enabling a light field to be recorded and later reconstructed without the original
objects (Gabor, 1968). Currently, there are several holographic techniques. The
most common ones are the “Pepper’s Ghost” effect, adapted in many concerts
worldwide to bring absent artists to live on stage; holographic nets and
smokescreens, widely used in museums and exhibitions; as well as fan-type
holographs and see-through displays, popular in retail locations. However, these
techniques are ultimately 2D images that give the illusion of 3D holographs, and
fail to provide real 3D content as the user moves around the object from the
intended viewpoint. Other techniques include holographic tables and 3D displays
such as the Hololamp and the Looking Glass display. Looking at all technologies in
Spatial AR, it is also a fast-developing area where we can see new technologies
emerging every year.
2.1.4 Digital eyewear
Digital eyewear, also referred to as a head-mounted display (HMD) or XR headset,
is a display device that the user wears on their head, allowing them to see digital
objects or virtual worlds. The research of XR technology has been around since the
1960s, when the US military revealed the first HMD for simulation and training
purposes. Since then, there have been many studies on XR headsets in various
industries such as aviation, engineering, medicine, education, and entertainment.
13

Masterarbeit | Bosen Zhou
In 2012, Google announced the Google Glass Explorer program, a prototype that
resembled eyeglasses with a head-up display. Their glass allows users to use many
applications without a smartphone, directly merging digital content into the user’s
field of vision (Kress & Starner, 2013). Google discontinued development of this
product for the consumer market in 2015 after a series of criticisms, including
privacy and safety concerns, and focused instead on the enterprise market in
manufacturing and logistics. Magic Leap followed a similar path in 2019, and
whilst initially intended for the consumer market, shifted to industrial use, due to
the high cost of production. One of the main reasons for the low adaption rate of
digital eyewear is the current lack of compelling content. In other words, AR
content does not provide added value or solve any real problems for the user.
Despite disillusionment regarding digital eyewear and the consumer market, it is
still the domain with the most potential in the following years compared to mobile
AR and spatial AR. Digital eyewear belongs to the Internet of Things (IoT), and the
introduction of 5G network infrastructure provides new opportunities for high-
bandwidth AR content. Many companies are starting to produce affordable digital
eyewear and explore their use in the experience economy. Nreal released digital
glasses in 2020 that look like regular sunglasses, with the ability to connect to
smartphones and display running Android apps overlaid in designated physical
spaces. Further, Apple, Facebook, Google, and Amazon have all registered patents
for AR glasses, with the expectation of devices coming to market in the next few
years.
14

Masterarbeit | Bosen Zhou
2.2 Storytelling
We learn and collaborate through language and stories, which are indispensable to
the existence of our culture. Without storytelling, we would not be able to
communicate complex information or solve problems. Stories help us share
knowledge using our pattern-recognizing brain, which is more memorable than
simple facts and statements. Today, stories are everywhere in our lives; we spend
our days listening to podcasts, reading news, watching TV and talking to people.
Different storytelling techniques are developed over time for each form of media, to
guide content creators. There are many narrative theories with different
perspectives, such as dramaturgy, classical narratology, psychological approaches,
digital storytelling, and interactive storytelling. This section aims to outline some of
the aspects that are most relevant to XR/AR and spatial experiences, by examining
the relationship between media and narrative, and then describes the essential
factors of storytelling.
2.2.1 Media and storytelling
Literary theorist Roland Barthes writes that “narrative is first and foremost a
prodigious variety of genres, themselves distributed amongst different substances,
as though any material were fit to receive man’s stories” (Barthes, 1977, p.79).
Narrative is present in language, literary, theatrical, cinematic, visual, sound, and
spatial forms. Considering the peculiarities of XR media, it will eventually develop
distinct narrative methods compared to other media, which could derive from
different areas of study, namely, media narrative, interactive narrative, and spatial
narrative.
Media theorist Marshall McLuhan asserts “the medium is the message. This is
merely to say that the personal and social consequences of any medium - that is, of
15

Masterarbeit | Bosen Zhou
any extension of ourselves - result from the new scale that is introduced into our
affairs by each extension of ourselves, or by any new technology (McLuhan, 1964,
p.7)”. Although published long before the invention of personal computers and the
internet, McLuhan believed that the medium fundamentally changes our behavior
and the message itself by the specific form it carries. For instance, the difference
between radio and television is that television offers visual information, allowing
storytellers to work with the message in two modalities, presenting information
that is impossible to convey in audio form. This fact essentially makes TV and radio
incapable of delivering the same story with the same impact. The popularization of
the internet and PC introduced new possibilities such as interactivity and
connectivity. As a result, services like YouTube and Netflix are changing the way we
use media, hence, changing the content and the story we consume. Algorithms
typically recommend content limited to the range of our interest and that of our
network.
Narrative theories usually focus on specific media. According to Aristotle’s classic
Poetics, there are six components in a good story: plot, character, theme, dialogue
or language, melody or chorus, and spectacle. These components were based on the
early forms of media, such as poetry and theater. Even though most parts are
relevant to all forms of storytelling today, it does not account for mass distribution
or digital editing. In the early twentieth century, Russian scholar Vladimir Propp
published his work, The Morphology of the Folktale. He analyzed 100 Russian
folktales and the common elements of these, which he called universal functions.
The goal was to create a grammar that could represent all folktales and combine
them into new ones (Propp, 1968). Propp’s study was based on oral culture stories
and print-based media, and faces similar limitations as the Poetics from Aristotle.
Another well-known example is the hero’s journey developed by American scholar
Joseph Campbell. Campbell writes that “a hero ventures forth from the world of
16

Masterarbeit | Bosen Zhou
common day into a region of supernatural wonder: fabulous forces are there
encountered and a decisive victory is won: the hero comes back from this
mysterious adventure with the power to bestow boons on his fellow
man” (Campbell, 1949, p23). This narrative pattern is widely used in the literary
and film industry until today. More media theories emerged after the age of mass
media; however, the idea of structuring stories was later rejected by the post-
structuralism movement. Post-structuralism stressed that one should not conclude
specific structures to any form of narrative. In this study, it is important to
acknowledge narrative structures cannot be applied in all cases, especially to a new
and developing medium that is not yet fully understand. However, structures can
be helpful in establishing norms before breaking them.
Media theorist Bryan Alexander argues that when a new medium appears, existing
narrative practices are first copied into the new medium. Alexander names this
Platform Affordances, explaining that “every digital story can take advantage of the
unique affordances of each digital platform it uses… Each segment of a story can
push the unique nature of its digital housing to accentuate the story’s
power” (Alexander, 2011, p43). Over time, content creators will explore and
develop a unique way to tell stories for XR media.
The internet and technologies have radically changed how we tell stories, the same
way mass media such as printing, radio, and television. Digital storytelling begins
to appear after the popularization of digital media, which initially refers to
storytelling in a digitized form. Alexander (2011) describes it as any story that is
“born digital” and published in a digital format, including blogs, web video,
computer games, and mobile apps. Over the years, digital storytelling has further
developed with various focuses, such as Digital Brand Storytelling, Transmedia
Storytelling, and more. Media theorist Dieter Herbst proposed four main
characteristics for digital storytelling, the “Big Four”: integration, accessibility,
17

Masterarbeit | Bosen Zhou
connectivity, and interactivity. Integration means the integration of different media
devices, services, objects, and forms. Accessibility refers to the content which is
accessible at any time and anywhere. Connectivity relates to the internet and
networks in the digital space. Interactivity applies to the interaction between
human, machine, and digital content (Herbst, 2013).
Another area of research that can be related to XR media is interactive storytelling,
a variety of narrative practices mostly used in the game industry, where the
storyline is not predetermined. Game designer Chris Crawford introduced several
techniques in his book Interactive Storytelling, where every user would experience
a unique story based on their interactions. The user takes an active role in
conducting personalized plots and co-creating unique storylines. It involves
algorithms to manage interactions with the user - the drama manager - essentially
creating an open story world (Crawford, 2004). Interactive storytelling focuses on
first-person narrative, similar to XR media.
The development of media generates advancements in narrative techniques,
providing us new ways to receive information and new opportunities to tell stories,
and XR is predicted to be the next new form of media.
2.2.2 The nature of stories
While different forms of media may use distinct narrative techniques, there are also
commonalities between them. Our brains receive stories the same way as our
ancestors did by using languages thousands of years ago. There are fundamental
features of storytelling that do not change with the media and identifying those
commonalities provides a first step towards understanding XR storytelling.
18

Masterarbeit | Bosen Zhou
A story is a connected system of information that activates multiple parts of
knowledge together in our brain. They work by connecting our previous
experiences with new information and creating room for memories and ideas.
Shane Snow and Joe Lazauskas stated in their book!The Storytelling Edge that
“when we get information through stories, we engage more neurons. As a result,
the story is wired into our memory much more reliably… When we hear a story, the
neural activity increases fivefold, like a switchboard has suddenly illuminated the
city of our mind (Snow & Lazauskas, 2018, p11)”.
Crawford refers to these systems of information as mental modules. He suggests
that “stories are complete patterns that communicate a special kind of knowledge
to our pattern-recognizing mental modules.” He presented four of the most
commonly recognized mental modules (Figure 2). “Visual-spatial: Handles visual
perception and spatial imagination; based on pattern recognition. Social: Handles
relationships with others; also based on pattern recognition. Natural history:
Storage of facts about the environment and logical analysis of those facts; some
sequential processing. Language: Permits communication and ties together all the
other mental modules; sequential processing” (Crawford, 2004, p. 36).
Figure 2: Interactions between the mental modules. Crawford, 2004.
19

Masterarbeit | Bosen Zhou
These four mental modules interact together to reinforce our knowledge and
memories, similar to a schema in psychology. Together, they form cognitive
networks of information, which guide us through our daily behaviors. Among the
four most common mental modules, language lies at the center of our
communication network. We obtain our writing abilities by connecting language
with the visual-spatial module; we study science by linking language with the
natural history module. Storytelling is a result of connections between language
and social relationships.
Through storytelling, our brains cast us as the heroes of the narrative that we
create. French philosopher Paul Ricoeur examined the relationship between time
and narrative and argued that we do not only experience time as a line of
succession. Ricouer asserts that we also use narrative to create a coherent story of
our lives, and thus we construct our identity using stories (Ricoeur, 1986).
According to Alistair Macintyre “Man is in his actions and practice, as well as in his
fictions, essentially a storytelling animal” (Macintyre, 1981. p. 216). Our brains
create a linear plot on our lives, order memories into sequences, and use narratives
to tie together our experiences, memories, and identity.
A story usually has a strong structure that includes protagonists, conflict, struggle,
and resolution. When listening to a story, we expect a protagonist engaged in a
form of conflict, with which the protagonist will struggle, and the resolution of
which will bring the story to an end. It is the simplest structure of a story that we
have in our minds. In spatial narratives, we see ourselves as the protagonists. The
conflict is the new experience we did not expect in that space. The struggle would
be our attempts to understand the meaning of that new experience, the resolution
is the answer and the memory we create with that experience.
20

Masterarbeit | Bosen Zhou
Stories are about people and relatability, and we will often find a reflection of
ourselves in the stories we observe. In popular books or films, such as Star Wars or
Harry Potter, the protagonists are presented in a disadvantaged situation and have
to undergo a series of struggles and changes to achieve a goal. The characters are
commonly presented in a way that we can relate to, so that our brains can simulate
the situation and engage with the story.
Conflict or tension is another essential element of a story and all stories typically
have some sort of conflict. For example, in the Shakespearean tragedy, Romeo and
Juliet, everything in the world of the protagonists works against them, each step
forward reveals new obstacles. The tensions in the story keep the audience actively
engaged with the drama.
Another component can be novelty or spectacle. Novel experience itself is a direct
form of entertainment, including exotic imagery, lighting effects, or exiting new
sounds. In exhibitions, people expect a new experience, from viewing the Sistine
Chapel ceiling to a new audiovisual installation. XR technology is still a novel
element by itself and people will wait in line to experience XR. However, this will
not last with the popularization of VR headsets and games; eventually, compelling
XR content will be required to attract the audience. Interestingly, Aristotle ranked
spectacle as the least important element in a story.
Stories are usually in linear form, which is referred to as plot-line, they are not
isolated facts. The audience is not able to understand the story’s content until the
entire plot has been fully revealed. This is what makes a story different from simple
information. However, this also raises a question about interactivity in storytelling:
if the plot-line is not predetermined, how can we ensure that a message can be
delivered through user’s choices?
21

Masterarbeit | Bosen Zhou
This question relates to the research of interactive storytelling. It is the
fundamental conflict between plot and interactivity. To understand this dilemma, it
is necessary first to define interactivity. Crowford defined interactivity as “a cyclic
process between two or more active agents in which each agent alternately listens,
thinks, and speaks (Crowford, 2004, p. 78)”. He explains that one agent is the
author, and the other agent is the audience. In interactive storytelling, the audience
becomes the co-author. If we have to take the audience’s opinion into consideration
of the story plot, which can vary for each individual, it would be challenging to
deliver the author’s intention. The game industry has been trying to solve this for
decades. On one side, there are open-world games, where players have the total
freedom to do what they want; however, there is no plot in these types of games.
On the other side, some games offer a plot where players can follow, yet players
have very limited choices in decision-making.
Today, the most advanced computer games use a mixture of methods to implement
interactivity. Players can have some degree of freedom and a story plot in which the
player can follow. Crowford argues that interactivity depends on the choices
available to the user (Crowford, 2004). Interactivity is fundamental for media such
as computers and smartphones, and this is also inherent in XR media. Thus,
research into interactive storytelling can be very beneficial for innovation in the
creation of XR content. In AR, nevertheless, we have to consider another essential
element: the real-world environment.
2.3 Spatial experiences
AR is fundamentally a mixture of physical and digital spaces. Therefore,
understanding physical space is essential when designing for AR experiences. This
section looks at relevant spatial theories that can apply to the magnitude of AR
22

Masterarbeit | Bosen Zhou
experience design, especially approaches that combine multiple disciplines.
Dimensions of experience explore the intersection of user experience design and
spatial design. Narrative environments examine the relation between space and
narrative. Lastly, the section will address the current situation of storytelling in
exhibitions.
2.3.1 Dimensions of experience
Space can trigger different emotions such as feeling welcomed, comfortable and
safe, or contrasting negative emotions. Researchers try to answer this architectural
problem from different perspectives, such as behavioral, social, cultural, and
technical. However, experience design as a newly recognized approach has never
integrated design as a part of the solution to architectural problems (Shedroff,
2009). Studies in experience design have not been engaged with spatial aspects.
This means that currently, there is a disconnection between space and experience
design.
Architectural theorist Bernard Tschumi sees spatial experience as a blend of space,
event, and activity. He believes that “there is no architecture without event, and
spatial experience is not simply about space, form, and function, but also about
event and activity (Tschumi, 2012, p. 176)”. An event is an unexpected quality of an
experience. The essential component is the activity and the circumstances that
structure the setting for that activity. Therefore, spatial experience is an experience
related to space that involves people in a certain activity within a context (Rahimi
et al., 2018). People, context, and activity are considered as the essential
components of spatial experiences in the built environment.
Experience designer Nathan Shedroff developed a model for experience design in
his book!Experience Design 1.1. He concluded six essential components that
23

Masterarbeit | Bosen Zhou
contributed to experience and called them dimensions of experience (Figure 3).
The dimensions are duration, trigger, breadth, significance, intensity, and
interactivity (Shedroff, 2009). The first dimension is the duration with four phases:
initiation, immersion, conclusion, and continuation. The second dimension is the
trigger; it means the stimulus response of an experience such as sense, emotion,
concepts, and symbols. The third dimension is breadth; it represents the broadness
and various ways the experience can be conveyed. It links to service, channel,
security/safety, and accessibility. For example, AR can be considered as a channel
that broadens the dimension of an experience. The fourth dimension is
significance; it refers to meaning, identity, values, and function in a built
environment. The meaning of a space is usually personal and subjective. The fifth
dimension is interaction, which includes four types, status, active, passive, and
interactive. An exhibition can be both active and interactive. Shedroff describes
that interactive spatial experiences occur when people are in mutual engagement
with context and activities throughout the experience. The last dimension is
intensity; it is a measurement of user and experience connections, including
engagement, reflex, and habit (Shedroff, 2009).
Based on the six dimensions of experience, the University of Calgary conducted a
study to find the variables for enhancing spatial experience in a built environment.
The conceptual model has three stages in the following sequence: encouraging,
enabling, and enclosing. “The model asserts that in every enhanced spatial
experience, the audience gets encouraged at the outset by a variety of strategies
such as persuasion, designing for meanings, and including concepts in design. The
audience must be then enabled by special means, such as immersive and
interactive capabilities of the environment along with its security and safety
attributes, to get involved with the spatial experience. Consequently, the experience
shifts towards a cognitive level at the enclosing stage, focusing on emotion and
engagement” (Rahimi et al., 2018).
24

Masterarbeit | Bosen Zhou
Figure 3: Essential components, dimensions, and elements of spatial
experience. Rahimi et al., 2018, Shedroff, 2009.
2.3.2 Narrative environments
Narrative environments is a relatively new field that ties together narrative theory,
spatial theory, and design theory. Similar to spatial experiences, it explores the
possibilities of storytelling in built spaces such as exhibitions, brand experiences,
and other socially engaged participatory projects. It argues that environment-based
narratives are very different from media-based narratives. While watching a movie,
playing a game, or reading a book can often be deeply, emotionally and
intellectually immersed in a story, narrative environments have another layer that
makes the audience physically immersed in the narrative.
25

Masterarbeit | Bosen Zhou
The term is established by Tricia Austin from the University of the Arts London.
Austin argues that the key that differentiates media narratives from narrative
environments is that the visitor moves through the narrative space and plays a
more physically active role in the story: “An exhibition visitor, for example, is more
engaged through full-body, sensory stimuli and the author of the exhibition loses
the power to order strictly the sequence of events”. She further explained that
“Literary and film narratives have heroes, sometimes called protagonists, with
whom we identify. In a conventional narrative, we empathize with protagonists, we
empathize with the protagonist’s struggle, pain, and triumph. This positioning
changes in a narrative environment. As we follow the spatial narrative, we not only
empathize with a protagonist, but we literally became a parallel embodied
protagonist in our own story of discovery and identity building” (Austin, 2020).
Austin developed a model for narrative environments with three interrelated
components: People, Narrative, and Environment. This model emphasizes that
people cannot exist without place, nor can they exist without narrative. Each
component is interwoven with the other two. There is a two-way dynamic in each
interaction between the components. Here, people equals to the subjective, which
is mediated by story and place (Figure 4). Adopted from the Actor-Network Theory
(ANT), all three components act simultaneously, creating a flow and dynamics of
narrative environments.
This theory proposes that everyone experiences conflict between their subjective
world and their complex relations with the conceived space. Dramatic conflict is
essential for the narrative. It is the persistent tension and the driving force that
produces the content of the stories. According to Robert McKee, nothing moves
forward in a story except through conflict (McKee, 1999). Based on this idea,
Austin further elaborated on the dynamics of stories using narrative structure
theory from Algirdas Julien Greimas (1983). Greimas modeled three contraries:
26

Masterarbeit | Bosen Zhou
sender vs. receiver; subject vs. object; and helper vs. opponent. He argues that this
generates three types of relations, which can act as narrative axes: knowledge,
desire, and power. Knowledge is the communication between the sender and the
receiver, desire is felt by the subject for the object, and power is realized through
the struggle experienced by the subject to achieve the object of desire (Greimas,
1983). This structure is relevant for creating engaging narrative environments
using the dynamics of knowledge, desire, and power. It enables us to analyze
narrative environments when combining digital and physical spaces while
understanding the emotions of the narratives.
Figure 4: Tripartite network model of narrative environments. Austin, 2020.
Phenomenological philosopher Maurice Merleau-Ponty suggested that our bodies
are inseparable from the space around us. Our immediate environment is a part of
our body schema that extends through our senses of our surroundings (Merleau-
Ponty, 2005). Immersive technologies can overwrite our senses, with digital
eyewear and holography for vision, 3D sound for auditory, and haptic technology
for tactile senses. AR industry is continually searching for the right combination of
technologies. Hence, the blurring boundary between physical and digital allows
narrative environments theory to be a valuable tool for creating AR narrative
experiences.
27

Masterarbeit | Bosen Zhou
2.3.3 Storytelling in exhibitions
Exhibition design is the process of developing an exhibition from idea to
implementation. It is a collaborative process that requires disciplines such as
architecture, graphic design, content specialist, audiovisual engineering, interior
design, multimedia design, and more. Exhibitions use various media to convey the
message through object, text, image, video, audio, and architecture. The goal is to
build communicative environments to deliver information. Over the last few
decades, more tools are available for exhibitions, especially digital media.
The core of every cultural institution is the transmission of knowledge and the
study, conservation, and exhibition of its works (Falco & Vassos, 2017). Nowadays,
this mission takes on a new meaning, connecting the facilities where knowledge
and artifacts are kept with the theories of experience economics (Vannini, 2012).
Digital media now plays a significant role in offering experiences in exhibitions and
other public spaces. Over time, the role of the exhibition has begun to move away
from merely displaying items and has started to offer novel experiences, acting as a
laboratory to experiment with the field of art and technology. Exhibitions,
museums, fairs, galleries, theaters, interior design, or architecture start to converge
by implementing technologies such as spatial mapping, LED screens, and other
digital media. Design studios begin to broaden their disciplinary focus on designing
experience because digital technology is needed everywhere. Therefore, Spatial
Experience is a more suitable term to represent activities such as exhibition design.
Narrative approaches in exhibitions are now commonly accepted. Leslie Bedford
wrote: “Stories are the most fundamental way we learn. They have a beginning, a
middle, and an end. They teach without preaching, encouraging both personal
reflection and public discussion. Stories inspire, wonder, and awe; they allow a
listener to imagine another time and place, to find the universal in the particular,
28

Masterarbeit | Bosen Zhou
and to feel empathy for others. They preserve individual and collective memory and
speak to both the adult and the child... It makes sense that storytelling is
appropriate to the work of a museum for museums are storytellers. They exist
because once upon a time some person or group believed there was a story worth
telling, over and over, for generations to come” (Bedford, 2001).
Mediatecture is a term for spatial experiences that focuses on media use as a part of
the architecture. Andreas Rostásy and Tobias Siever described in their book
Handbuch Mediatektur that there are two types of narration forms in
mediatecture: synchronous or asynchronous. In synchronous narration, designers
can define the space and temporal rhythm; viewers follow the narrator in which the
space can behave like a stage. In asynchronous narration, viewers take a more
active role and discover the story themselves; they can freely decide the sequence,
time, and engagement. Here, a modular narration approach with many entry points
is required (Rostásy & Siever, 2018).
These two types of narration are similar to linear and non-linear narration in
interactive storytelling. In spatial experience, they are mediated by the design of
spaces and a ground plan with visiting directions. The visiting route of an
exhibition can be linear, non-linear, or mixed. How spaces are designed has direct
consequences on visitors’ experience. It is important to note that our relationship
with space is never neutral; every design decision made, including form, material,
light, sound, and smell, can provoke emotional responses (Locker, 2011). In
exhibitions, the space is mostly temporary, which triggers different audience
engagements than other space designs.
Bauhaus designer Herbert Bayer suggested: “The organism of a successfully
designed exhibition unit includes in itself unity, mobility, aesthetic pleasure,
forcefulness, and economy” (Bayer, 1939). Digital media allows a holistic
29

Masterarbeit | Bosen Zhou
connection between elements. They also enable interactivity, which offers a
dialogue with exhibition contents, objects, and artifacts, compared to traditional
media such as text, video, and audio. Through interaction, visitors can experience
the contents in a self-determined way. They can either superficially or intensively
delve into a topic (Schwarz, et al., 2012). By implementing AR technologies, we
have the opportunity to compose all elements into one unified experience.
Especially the introduction of digital eyewear will considerably enhance the
development of spatial experiences. Consequently, pushing us to rethink
storytelling for spatial experiences.
3. Analysis
3.1 XR storytelling
XR, as a future medium, can be the ultimate version of our imagination, a seamless
blend between reality and the digital world. However, we are still technologically
far away from achieving the XR convergence. As of today, we mainly separate the
technology into two categories, VR and AR. On the one hand, there are many
similarities between these two media. In terms of HCI studies and user experience
(UX)/user interface (UI) design, both VR and AR headsets can offer multimodal
interaction using six degrees of freedom (6DoF), spatial sound, and haptic
technology via a controller or other devices. On the other hand, the differences
between them are also enormous. VR takes the user to a completely virtual world.
In contrast, in AR, we have to consider the interaction between digital objects and
the physical environment, including natural lighting, plane detection, navigation,
sound, and more. Therefore, VR and AR should be discussed both jointly and
separately. In this study, the discussion starts with the commonality of VR and AR.
30

Masterarbeit | Bosen Zhou
Furthermore, it explains the relation of XR and media narratives, subsequently
shifting the focus to spatial narratives and AR storytelling.
3.1.1 A new narrative media
XR is fundamentally different from other media. One of the reasons is that the
narrative perspective shifted from third-person to first-person. For instance, the
film industry has evolved significantly since the late 19th century. We enjoy
watching films through different media; however, the form of this medium
essentially has not changed in the last century. It is a group of rectangles which are
played in a sequence. The rectangles are like windows, and we are used to
experience stories without greatly getting involved with the scenes. We know that
the story happened in other space and time, which are being delivered to us
through the window. There is no sense of presence involved by watching TV or
smartphones. Even today’s most advanced 3D films are just 2D illusions that trick
our eyes with no content or interaction outside the rectangular frame. Everything is
happening inside the frame, which we know is unrelated to us.
Film director Steven Spielberg said: “I think we are moving into a dangerous
medium with virtual reality, the only reason I say it is dangerous because it gives
the viewer a lot of latitudes not to take direction from the storytellings but make
their own choices of where to look (Spielberg, 2016)”. One difference between XR
and current digital media is the sense of presence. Defined in a study from The US
Army Research Institute as “the subjective experience of being in one place or
environment, even when one is physically situated in another” (Witmer & Singer,
1998). Unity has a similar definition: “The sense of being somewhere, be it in
reality or virtual reality. In reality, a present person may be particularly aware and
socially interactive. In VR, the term applies to the experience of believing you do
occupy the virtual world” (Unity, 2020). The degree of presence one can feel
31

Masterarbeit | Bosen Zhou
depends on various factors such as degree of control, sensory modality, distraction,
and realism factors. These aspects can be simulated with immersive technology.
Presence is equally essential in VR and AR experiences. Presence in AR means a
seamless integration of virtual content with the physical environment. The virtual
content has to align with the user’s expectations. In an ideal AR immersive
scenario, the user cannot distinguish virtual objects from real objects. Nevertheless,
our current technology is not yet ready to overwrite reality or seamlessly blend the
virtual world and the physical world into one continuity. Consequently, the use of
AR in physically built spaces will play an increasingly important role in the near
future.
The sense of presence can create an immersive experience. According to Staffan
Björk and Jussi Holopainen, in!Patterns In Game Design,!there are four different
types of immersion: “Spatial immersion, emotional immersion, cognitive
immersion, and sensory-motoric immersion. Spatial immersion results from
extensive maneuvering in the game world in real-time games and can sometimes be
felt in movies. Emotional immersion is obtained by responding to the events that
characters are part of during the unfolding of a narrative structure and is similar to
the immersion that books, theater, or movies provide. Cognitive immersion is
based upon the focus on abstract reasoning and is usually achieved by complex
problem-solving. Sensory-motoric immersion results from feedback loops between
repetitious movements players make to perform actions in the game and the
game’s sensory output” (Björk & Holopainen, 2005). XR creates a different kind of
immersion, one that is similar to spatial immersion by manipulating the presence.
However, the XR environment itself does not offer other immersion types unless
implemented with narrative techniques or game mechanics.!!
As mentioned in the previous chapter, when a new medium appears, people will
firstly copy the existing narrative practices into the medium. When film technology
32

Masterarbeit | Bosen Zhou
first appeared in the late 19th century, some artists were copying the entire theater
play into the film format. The result was not optimum, because theatrical
narratives were not made for films. Not until after years of attempts by content
creators, film techniques were then developed. Today, Cannes Film Festival and
Sundance Film Festival have introduced XR as a film category. Artists and content
creators are trying to create stories in VR in many different ways. Meanwhile, the
game industry has also rolled out VR headsets and games to the consumer market.
With both the film and game industry on board, the content and narrative
techniques in VR media will gradually improve in the following years. However,
these developments are mostly targeted to the use of VR. Storytelling in AR is more
complicated compared to VR because the real physical environment is more
challenging to control than digital environments.
Many believe that the future use of AR should be spatial computing. By placing our
daily digital content in the physical space, AR can potentially replace all screens in
our life. Our current digital content can be applied to the medium, but AR
storytelling can be much more than that. AR opens a new content channel that we
have not seen before. Our daily displays will be changed and replaced, so will the
digital content eventually. XR designer Bernard Bettenhäuser describes that XR is
like the industrial revolution; it is an undiscovered land. In films, most storytelling
techniques have already been used. AR brings a massive upheaval not only in
storytelling but also in how we work and communicate (Interview 2).
XR media should go beyond technical or gamified immersion to enter the mass
market. Emotional immersion in VR can evolve from traditional media narratives.
Yet, AR storytelling involves both media and spatial narratives; therefore, finding
the commonalities in these two areas can be the first step toward understanding
AR storytelling. Tom Duncan and Noel McCauley argue that both film and
architecture are temporal storytelling media. They communicate stories by
33

Masterarbeit | Bosen Zhou
movement through a sequence of spatial images; viewers move through the spaces
of a film in the same manner that visitors move through physical space. (Duncan &
McCauley, 2012). By understanding the techniques in both disciplines, content
creators can explore new forms of narrative content for AR experiences based on
common elements.
3.1.2 AR media attributes
AR, as a new media born in the digital age, theoretically inherited all characteristics
of digital media: integration, accessibility, connectivity, and interactivity (Herbst,
2014). AR content can be shown on different media devices, using different
technologies. They are technically accessible at any time and space, and are
intrinsically connected to the internet. Interactivity is another essential attribute
that AR media has inherited from digital media. AR artist Mate Steinforth points
out that AR only makes sense if it has interactions or game elements: “AR would be
the next step after mass media and the internet, you don’t want to consume films of
other content that works well with other media in AR” (Interview 5). AR
interactivity is related to HCI by using hand tracking, joysticks, or mobile
interaction and spatial interactivity using 6DoF and location-based technologies
such as GPS or IPS. For this reason, AR is more like a tool, and interactivity is the
key to AR experience.
There are also a few AR-specific attributes that distinguish AR from conventional
digital media. First and foremost is AR’s extended visuality, a feature that unites
spatial AR, mobile AR, and digital eyewear. Traditional digital media content is
usually fixed in a rectangular frame, while AR allows us to digitally enhance our
physical environment without being limited to a particular frame. Multimodality is
another attribute of AR media. It adds auditory and haptic information on top of
visuality to the AR experience. With these attributes, AR can create a kind of
34

Masterarbeit | Bosen Zhou
immersion that overlays and enhances the physical reality we perceive. This gives
AR some specific advantages compared to VR, especially for spatial computing and
other areas where people need to stay in the physical world.
AR technologies have different social and individual perception levels. If we see it
as a spectrum from social to individual, spatial AR has the highest social
perceptibility (Figure 5). The content of spatial AR can be collectively perceived by
the mass, depending on the project’s scale. Mobile AR is in the middle of this
spectrum. It is mostly personal; however, it can be shared with one or two people,
especially using a tablet. Digital eyewear is on the very personal end of this
spectrum. AR glasses are designed for one individual, similar to headphones.
However, it is possible to program and display the same content on various devices.
Jeremy Vanhoozer, the creative director of Magic Leap suggests that having
personal content is a unique advantage of the AR glasses:
“There is something very personal about an AR presentation, I am looking at
a detailed piece of content, and suddenly it springs to life, it’s in my space,
maybe I have interaction using hand gesture, it feels very one to one. For
me, maybe the biggest opportunity of AR is the ability to make history,
entertainment or storytelling feel more personal, you are a participant, it is a
quiet moment that you share with the digital character versus standing with
a thousand people looking at a projection, that is still very cool, but it’s a
very different feeling that you have (Interview 8).”
Figure 5: AR perceptibility scale.
35

Masterarbeit | Bosen Zhou
Many technologies can be implemented in AR, such as eye-tracking, simultaneous
localization and mapping (SLAM), light detection and ranging (LiDAR), AI, and
deep learning; these implementations are constantly redefining the characteristics
of AR. However, the main challenge is to synergize various attributes that bring
value to the user. How many of these attributes can be fully utilized is still an open
question.
3.2 AR in exhibition
While spatial AR, such as projection mapping and Pepper’s ghost, is common in
exhibitions today, mobile AR and digital eyewear are not yet widely adopted. Some
of the best examples of AR in exhibitions are commonly combinations of various
technologies, including spatial AR, mobile AR, VR augmented with physical
objects, augmented virtuality, and scenographic installation.
The Dial was an AR project directed by Peter Flaherty, in collaboration with
NightLight Labs, premiered at Sundance Film Festival 2018 in the USA. It
combined projection mapping, sculptural elements, interactive lighting, and mobile
AR to create a compelling story. Users can control the narrative by walking around
the sculpture with their mobile phones in the designed space. In July 2020,
TeamLab launched its latest immersive installation, Catching and Collecting
Forest, in Fukuoka, Japan. They combined spatial AR and mobile AR, where people
can explore the space and using a smartphone to capture and collect animals in a
constructed digital forest. STRP festival in the Netherlands has also been exploring
XR technologies for many years now. One of their latest projects, Atlas, was
premiered in 2019, which mixes AR, VR, and a scenographic exhibition. It uses
physical models with interactive installations, allowing users to explore and create
cities.
36

Masterarbeit | Bosen Zhou
The findings suggest that the AR approach in exhibitions is mostly positive;
however, it depends on the context of the exhibition (Figure 6). According to
exhibition designer Felix Hardmood Beck: “AR/VR is like any other medium. If it
fits the context, it is great. If you overlay an artifact with information that you
cannot add in a better way, then AR is perfect” (Interview 4).
Figure 6: Magnitude codes, how positive or negative
would you evaluate the approach of AR in exhibitions?
The use of AR in exhibitions is not only limited to content but also technology and
infrastructure. Interaction designer Josue Ibañez believes that with enough
resources, it can be very positive. However, if the infrastructure is not there, with a
slow internet connection, or incompatible devices, then using AR can segment the
audience's experience (Interview 6). Using mobile AR and digital eyewear in the
exhibition space can easily turn the experience into a gimmick show. Multimedia
artist Jesse Garrison (Interview 9) shares the experience at Sundance Festival: “I
feel like it is still finding its footing. At Sundance, where The Dial premiered two
years ago, a lot of the stuff we saw felt like tech demos. People didn’t necessarily
take advantage of the form, there were a few amazing pieces, but a lot of it was
mostly interested in the technology itself as opposed to using it as a tool to explore
something else”.
37

Masterarbeit | Bosen Zhou
Despite all current technological limitations, interestingly, exhibitions can be the
one area where AR can have a significant presence today. The founder of
ARinChina Michael Zhang describes: “from the beginning of my career, AR has
mostly been used in the exhibition area, showcasing technologies. I believe AR
technologies are not yet widely used to increase productivity, and it has not yet
revolutionized any industry. Until now, most AR usages are experimental units in
different industries (Interview 7)”.
An exhibition is where content creators can explore all possibilities in AR before
adopting other use cases. It serves as a bridge to connect AR to its next stage,
ubiquitous spatial computing. Interactive artist Peter Flaherty argues that the need
for the exhibition experience will diminish because the at-home experience will get
better. However, the exhibition experience will still be really exciting because you
can craft different social awareness (Interview 10). This study aims to address
several challenges and opportunities for AR in the exhibition space.
3.2.1 Challenges
There are currently many hardware and software challenges in terms of using AR
in exhibitions. For digital eyewear, the most prominent challenges are hardware
accessibility and technical hurdles. AR glasses currently on the market are quite
costly for mass adoption in exhibitions. The FoV (field of view) and processing
power are not ideal yet, even though it is improving every year. Beck describes that
not everyone has the hardware; not every museum can give everyone hardware
(Interview 4). Bettenhäuser shares the same view. He argues that technical hurdles
are currently very high, too much is still done for a show effect.
In terms of hybrid solutions with mobile AR and spatial AR, technological
standardization is another major obstacle, including hardware and software. On
38

Masterarbeit | Bosen Zhou
the one hand, current smartphones use different technologies to capture the space
and have different capacities to render content. Network speed also plays a crucial
role in the experience. Steinforth points out that at the moment, using the mobile
device is not very intuitive for the visitors (Interview 5). When overlaying different
complex technologies together in an installation, the challenge can be more
difficult. Flaherty suggests that AR technology wants everything else in the world to
remain static. It creates an image of the world using its photo capability and creates
a geometric space (Interview 10). Lighting, user movements, and other external
factors can significantly affect space tracking technology. On the other hand, there
is not yet a common UX standard for AR solutions. Mobile AR occupies at least one
hand of the user while hand tracking in digital eyewear is still in its infancy.
Exhibition designer Tobias Sievers describes that we should make it intuitively
accessible to people. You need different approaches, understand, and support
certain things for the visitor to develop the use of technology. Every museum would
like to offer something new, but if the visitors are not so tech-savvy, then it could be
difficult. Unfortunately, as of now, this technology is moving quite slowly
(Interview 3).
Having compelling and relevant AR content is another barrier to using AR in
exhibitions. Understanding each AR technology’s advantages compared to other
media will be crucial for creating a good narrative in a built space. Vanhoozer
(Interview8) argues that persistent content is one of the biggest challenges in AR:
“One of the biggest challenges is the idea of persistent content that can stay
or lock into a place, no matter where people go. One of the most convincing
things about AR is that content feels very rooted in real life, having
persistent content that lasts overtime or a state of something affected by the
visitors that stays very persistent in a part of a room. I think it is important
to convey believability, so the technology, the backend services, and servers
39

Masterarbeit | Bosen Zhou
working together to keep everything pined is very important for the
believability of the instance. There are a lot more technical challenges. I
think believable persistent content is going to be one of the biggest keys”.
With the pandemic outbreak in 2020, logistics and safety are critical issues to
consider for exhibitions. Events and exhibitions are among the most affected
industries this year. It is currently unimaginable to share AR glasses among
different people. Garrison describes that using AR glasses for something long-term
can be a logistical nightmare (Interview 9).
Zhang suggests that “the main challenge is the difference between our expectations
and technological reality” (Interview 7). He stressed that many questions still need
to be solved before any mass adoption of AR, so exhibitions can be an environment
to test the technology. The expectation-reality discrepancy has been a critical
problem when implementing AR technology in built spaces.
3.2.2 Opportunities
If we overcome some of the challenges, then AR glasses can offer some
groundbreaking opportunities. Firstly, AR offers new perceptions. For exhibitions
with cultural or historical artifacts, AR can provide a new way of understanding,
potentially replacing traditional 2D textual-imagery presentation. It allows content
creators to build 3D spatial and sensorial scenes around the objects and use AR to
connect the items with temporal aspects using superimposition, showing different
states in different periods. Flaherty attributes this as crafting different physical and
sensorial awareness (Interview 10). Sievers describes the current circumstances in
museums: “We have century-old objects that display in the vitrine, they look nice
for visitors, but to understand the context and the history would be more
complicated, and if AR can help people to understand better these objects, then it
40

Masterarbeit | Bosen Zhou
would be a great story” (Interview 3). XR and exhibition designer Pablo Dornherge
(Interview 1) suggests that this can enable new mediation strategies and make
complex issues understandable through interaction and spatial access.
Currently, AR in exhibitions can also overcome a major AR challenge: space
mapping. Due to current technological limitations, AR digital content cannot be
applied perfectly in any given space. Different rooms have different structures,
lighting, and materials. In exhibitions, designers can build that space to make sure
AR works properly according to the specific technology. Vanhoozer explains that
“one of the biggest challenges in AR, especially in storytelling, is that you don’t
know what the stage is… If we have an opportunity to set the stage, we can create a
lot of sync areas and clean presentation space, for the audience, the opportunities
are limitless” (Interview 8).
Secondly, digital eyewear can be the next display in exhibitions, especially for
multimedia installations. Spatial computing is one of the biggest promises of digital
eyewear, with the idea that it replaces the current displays in our life and has digital
content in any physical space. This can potentially reduce the cost of buying and
installing traditional large displays in exhibitions, allowing us to visualize data,
display information, and contextualize artifacts in an entirely new approach. Zhang
mentions that currently, smartphones are the most important displays in our life,
AR glasses will become the next display. When the next display comes, our life,
study, communication, and production will be upgraded to that display (Interview
7). Bettenhäuser shares a similar view: “I see it in spatial computing, sitting in front
of the display working with a mouse should not be the end form. I think it (AR
glasses) could come more from that direction (Interview 2)”.
Thirdly, AR glasses can enable new accessibilities and create more personalized
experiences. Sievers suggests that virtualized narration can be very different for
41

Masterarbeit | Bosen Zhou
children, teenagers, or people with different degrees of interest. This can be not
easy in a physical exhibition, for example, to have the text suitable for everyone to
read (Interview 3). Having personalized content in exhibitions can vastly improve
the participation and engagement of the audience. It also increases the scale of the
target audience. Ibañez describes that this can deepen the experiences of
individuals and offer many more different layers of experience (Interview 6). In
mobile AR, users can use their device to take a piece of the digital objects with them
as a souvenir of the experience. Having the right combination of technologies and
AR content in exhibitions can open up many exciting new opportunities.
3.3 AR storytelling
Storytelling in AR, specifically AR storytelling in built environments, is difficult to
define by using structural frameworks, if not impossible. Different spaces, topics,
and situations require different narrative strategies and priorities, especially in the
realm of art space. The existence of a structure limits artistic freedom by its nature.
However, the intention of having a structure in this study is not to restrict
explorations in AR, but to provide a guideline that helps to connect different
perspectives and rationality in the complex world of AR. To create such a
framework, abstraction is necessary to provide maximum applicability.
Three new categories were built based on the tripartite network model of narrative
environments by Tricia Austin: people, environment, and narratives derived from
the ANT. She argues that without this triangulation, ‘narrative’, ‘environment’ and
‘people’ would remain three distinct, unrelated spheres, with narrative studied
through narratology, under the aegis of literary studies; environment studied
through spatial theory, under the aegis of architecture and the natural and
42

Masterarbeit | Bosen Zhou
ecological sciences; and people studied through the social sciences, under the aegis
of economics, psychology, sociology and anthropology (Austin, 2020).
The findings of this study show that content creation in AR is relevant to three
major distinct areas. The element of people directly generates interaction; the
environment creates space and perspective; and narrative triggers emotion (Figure
7). All three components interconnect multiple disciplines and act simultaneously.
Figure 7: AR storytelling.
3.3.1 People and interaction
Stories are about people. For AR experience, people interact with both physical
space and digital content. This dynamic brings interactivity to the forefront of AR
storytelling. Interaction is also the most salient feature that differentiates AR from
traditional media storytelling, such as films or books. Steinforth believes
interactivity is an essential feature when using AR because non-interactive content
already works very well with other media. Zhang shares the same view and
describes that interaction is crucial when mixing the digital with reality
(Interview!7).
43

Masterarbeit | Bosen Zhou
For AR storytelling, there can be many different forms of interaction. It is not
limited to HCI or the interaction between the user and digital content. By
physically moving around creates an interaction between the user and the
environment; social interaction is another factor to consider designing for spatial
experience. Sievers asserts: “we associate interaction with our hands, but actually,
we interact with our minds, that is where the story comes from” (Interview 3).
The first type of interaction is HCI. Mobile AR can generally apply with the mobile
UX design system. However, there is not yet a common practice for spatial AR and
digital eyewear. Different technologies, such as joystick, hand tracking, eye
tracking, and voice recognition, are being applied in AR. Zhang points out: “There
are some good examples, but in general, there is no perfect solution yet” (Interview
7). Many agree that it is still a new area for explorations, and AR interactions
should be simple, intuitive, or even avoid using them in exhibitions as of now. Over
time, as XR usages become more common, designers can add more interactions.
Ibañez shares his experience in spatial AR: “our learning is that the worst
interactions are the ones that were not approved by people from outside of the
project. If there are more than three instructions, then it could almost be
impossible to continue the experience. One instruction is already a lot” (Interview
6). Garrison argues that interaction should always be meaningful, and there should
be a certain degree of flexibility for different types of users: “ideally, it should be
complex and intuitive in general. In an exhibition setting, you have to go toward
simple because you have people there for little time. You want something that
teaches you how to use it as you use it. I prefer something complex or something
you can keep learning as much as you want. For people who want to spend five
minutes there and wave, that’s fine, but if you want to spend half an hour there and
invest your time, you should be rewarded for that” (Interview 9).
44

Masterarbeit | Bosen Zhou
The second interaction is between the user and the environment. This kind of
interaction seems more complex and uncontrollable, but content creators can use
AR to introduce it neatly into the narrative. Beck suggests that reactivity is can be
optimal in an exhibition context: “I realized over time that reactivity often is
enough to transport the story, interactivity is much bigger. To interact with an AR
object, you need time to be mentally ready to interact. In exhibition contexts, the
majority of people do not have that mental component. Reactivity works most of
the time ” (Interview 4).
Flaherty brings another interesting idea that can fit into this context, which is
imperceptible interaction, through digital content reacting to user movements in an
environment. He refers to this as participatory storytelling:
“The best interactivity is the interactivity that the audience does not even
know is happening. If you’re really aware of the interactive event, I think
that makes it very difficult to be immersed in a story experience
simultaneously… I use the word participatory because it changes the concept
of what the audience’s interaction with the story is. I think the participation
component makes it something softer, more emotionally open, something
that is that makes the audience more capable of experiencing a wider range
of emotional and physical responses. It’s very small interactive gestures. The
way you participate is actually on a small scale because if the audience is
thinking every time they need to make a choice, that pulls them out of the
story… With participation, I try to create situations where the audience feels
free to flow through the experience. So they are making choices, but they’re
flowing. I think it’s a positive sign when the interaction becomes
invisible” (Interview 10).
45

Masterarbeit | Bosen Zhou
3.3.2 Space and perspective
People cannot exist without space. Space is essential to our identity and narrative.
Brian Lawson (2001) describes space and architecture as a vital language central to
human communication. Austin suggests that in exhibitions, objects are specifically
used to bring other worlds to mind. They seek to change your perception of being in
a present experiential moment to being part of human history, which may then
open up to the diversity of distinct cultural geographies (Austin, 2020). The use of
AR has the potential to multiply content, temporal, and geographical connections
in built environments, which is why multi-perspectivity is one of the key aspects of
AR storytelling.
Sievers envisions museums to use AR to reach multi-perspectivity: “I try to bring
someone into the topic and at the same time to open their mind to understand how
many different perspectives are playing a role. If we look at one object, a daily
object we use, it is also an object with an origin story, and it has a cultural context.
There are perspectives of people relating to that cultural context. Maybe a
perspective that is foreign to us, there is a variety of perspectives where we want
people to confront. Not only on the material or aesthetics, but take people on a
journey to discover the meaning of the collection, and open their minds a little.
That would be the task in the context” (Interview 3). The use case is similar in the
art space, AR offers a solution to extend the artworks in a new dimension to
transcend its physical state and provides new interpretations to the visitors.
The first step is to understand the physical space. Connecting spatial design with
digital design, it means to have curators and installation designers understand the
digital specificity and vice versa. Bettenhäuser argues: “you always have to take
something from the physical space. To do that, you have to take back some physical
46

Masterarbeit | Bosen Zhou
pieces, to present less the physical part” (Interview 2). Content creators should
achieve a balance between physical and digital offering AR experiences.
Having that balance can create new spatial accessibility and encourage exploration,
which offers more flexibility and widens the audience range. The exploration can
happen individually or socially, depending on the narrative. Flaherty suggests: “I
think the most exciting thing about AR storytelling is the social aspect, the ability to
transport you into a world but take you there with other people, it creates the
ability for a shared experience. Great stories are that you’re going down the rabbit
hole, and at each step, you discover something new about this character or the plot
or the story itself” (Interview 10).
Another spatial aspect of AR is that it can offer location-based experiences, which
can extend the experience of the given space. Users can take a digital piece of the
experience and manifest it in other spaces as a continuation of that experience.
MauAR is a mobile AR application developed to celebrate the 30th anniversary of
the Berlin Wall fall. People can visit historical sites throughout the city with a
unique AR experience at each location. Steinforth defines it as an augmented
journey (Interview 5).
3.3.3 Narrative and emotion
AR narrative begins with digital media narrative at its core, but it is also involved
with the movement of bodies through space. One common aspect of media and
spatial narratives is conflict. Many theorists suggest that all stories have some sort
of conflict, they are not about the daily mundane, and there is no drama without
conflict. In AR storytelling, content creators need to consider conflict and tension
as a base component in both media and spatial levels. Conflict can then transform
characters through events and evoke emotions.
47

Masterarbeit | Bosen Zhou
However, in media narratives, the users are mostly spectators. They empathize
with the protagonist and their conflicts. In spatial narratives, the audience can be
the protagonists themselves. They can directly experience conflicts. AR narrative
offers the possibility of both perspectives depending on the context. Sometimes the
two perspectives can be combined.
In media narratives, where the users are spectators, stories follow a pattern and
usually have a linear structure. They rise to a peak of tension or climax and then go
towards a resolution. Beck describes: “There are different meta-levels that go into a
good story, on the one hand, the story has to be good, the narration following a
proper curve of excitement with an intro, main body, an end, and a climax
somewhere in between” (Interview 4). Vanhoozer shares a similar notion: “The first
thing to know is that you start with the same challenges… you still need a very
compelling story, it needs to read well to the audience, you need compelling
characters that people care about” (Interview 8).
In spatial narratives, where the users are protagonists, dramatic arcs are
manifested in a different way. Austin argues that they have a spatio-temporal start
and end. The triple movement of the body, firstly, over time, secondly, through
different spaces and atmospheres, and thirdly, through different representations of
content is crucial to shifting visitors emotionally, intellectually, and normatively
away from the expectations of the everyday and into the world of the story (Austin,
2020). Emotion can play a crucial role in spatial narratives, especially in AR
applications. One way to achieve that is through the suspension of disbelieve, the
idea of feeling in a different space, avoiding logic, and critical thinking to be able to
experience something surreal. Ibañez suggested: “we are in the entertainment
sector; suspension of disbelieve is the goal of our work” (Interview 6).
48

Masterarbeit | Bosen Zhou
Non-linear storytelling is another important narrative aspect. In spatial design,
designers can plan visitor flows to present a linear narrative, but generally, it is
hard to control where people want to move in space, especially in AR. Many believe
that the use of AR in a built environment should apply non-linear stories. Beck
suggests that linear storytelling is problematic in exhibitions: “The problem of
storytelling is often a linear one, there are a clear start and a clear end. The
audience has to enter at the beginning to get the full experience, and they are not
allowed to leave before the end. It is problematic for exhibition and museum
context, often people come in the middle, or they want to leave earlier, then they
get only a certain portion of the narration. This is a huge challenge of the linear
narration” (Interview 4).
One way to tackle this problem is to break the story into smaller parts and
fragments of events, similar to interactive storytelling. In the interactive drama
Façade developed by Michael Mateas and Andrew Stern, they refer to these events
as story beats. Each individual beat contains several joint dialog behaviors, they are
the atomic unit of narrative. The more beats there are for the drama manager to
work with, the more possible orderings that emerge, and the more global agency
(plot control) the player will experience (Mateas and Stern, 2004). Crawford refers
to them as sub-stories put together in a story world. Story beats or sub-stories can
form different tension arcs depending on the path users want to take. In AR spatial
narrative, designers can arrange sub-stories according to spatial, timely, and
thematic settings. There are several ways to arrange the sub-stories, which will be
discussed in detail in the next section.
49

Masterarbeit | Bosen Zhou
3.4 AR integration
Integrating AR can be a complicated process using different technological
approaches. This chapter aims to provide some of the key aspects of integrating AR
into exhibitions and spatial experiences. The use of AR in exhibitions can have a lot
of potentials. However, it cannot yet technically replace other media. In media
installations, every media has different specificities, and sometimes, the best
solution to offer a great story experience is often a combination of different
technologies.
Dornherge concludes that AR should be equal to any other exponent in exhibitions:
“AR should not be essential to understand the story, but it offers the possibility to
contextualize information and connect different objects with stories” (Interview 1).
Beck shares the same notion: “AR is a medium you place to support a certain
context, the beauty of AR is virtual, it is like a ghost of the past, it has a certain
aesthetic, every medium has a certain aesthetic and a certain metaphor that it
transports. If those meta-components fit what I’d like to explain with the artifacts,
then it is a perfect match… I would never hype AR more than display or printed
graphics; everything has a role to play” (Interview 4). There are a few important
elements to consider in AR integration: interdisciplinary work, design process, and
best practices.
3.4.1 Interdisciplinarity
Interdisciplinary collaboration and mindsets are indispensable for integrating AR
in exhibitions and spatial experience projects; such projects draw skills from
several fields. From interaction design to spatial design, game design to
scenography, every discipline has some distinct advantages in the world of AR
(Figure 8).
50

Masterarbeit | Bosen Zhou
A collaborative work environment is critical to ensure a successful project. The
main discrepancies of different mindsets are between science and art, physical and
digital. Dornhege points out: “The programmer should have notions and affinity
about design and exhibitions. There are often different languages between different
disciplines. AR projects are usually a lot more complex than people assume. Often
new technologies are not properly understood neither in their function,
development, application, and complexity. Moreover, the technologies often
polarize museum employees: some see them as a problem and enemy of the
classical exhibition; others see them as the solution to all problems” (Interview 1).
Figure 8: Interdisciplinary network for AR in built environments.
On the other hand, exhibition designers and curators should also have basic
knowledge about software technologies to work with programmers. Ibañez argues:
“You cannot have a one-man studio nowadays, including the greatest artists; they
have their interdisciplinary team. As an art director, it is important to know and
51

Masterarbeit | Bosen Zhou
understand the technologies to speak the same language as developers and
technicians. They have to know the limitations of the technology” (Interview 6).
An integrated team would be ideal for AR integration in exhibitions, but depending
on the project, sometimes external teams are necessary. Sievers emphasizes that
most people in the market are either in art and experimental work or only in the
corporate world. It is hard to find people who can think beyond (Interview 3).
Dornhege suggests that joint workshops are essential in such situations, in which
all stakeholders, including museum, design, curation, coding, and technology,
develop a common language (Interview 1).
Finding the right people in the team is fundamental. Interestingly, Bettenhäuser
suggests that the best teamwork is working with those who have trained themselves
in the field. He explains: “I always find it difficult working with Unity developers,
who are mostly game designers, they can have very particular mindsets. The best
case is to work with people with an interdisciplinary experience because they
always think that way themselves. They always look at things with distance.
Interdisciplinary working is now a buzzword, everyone knows it, but not many
people can do it. Our Unity developers and programmers are all self-learned;
interdisciplinary is already there by definition” (Interview 2). Zhang believes that
having programming artists or creative programmers in the team can be a great
advantage (Interview 7).
The designer’s role in an AR spatial experience project can be more than just
designing the experience. Beck suggests: “The designer has to start very early in the
process… they are in the center between hardware and software, then you have the
curator and archeologist, they also have a certain kind of expectation. The designer
is like a translator of the content that comes from the curator and goes to the
visitors. The essence of what has to be transported should be visible; you have to
52

Masterarbeit | Bosen Zhou
negotiate all those components. I see the designer as the one holding everything
together” (Interview 4). Therefore, designers should be involved from the very
beginning of the project.
A game developing team would have the most advantages in developing AR. There
are many similarities between AR and game development, such as real-time
rendering, low-poly animation, and the Unity game engine. A game development
team structure is usually very interdisciplinary. However, Vanhoozer argues that
there are also many differences in those two fields: “In games, you still deal with a
2D screen, you still cheat by hiding content in the corner. Understanding the fact
that the person who wears the device is the camera and has all kinds of access to
your content, depending on where they are and where the content is, you are
thinking about the space differently. From a design perspective, you have to change
the lenses; you have to think differently because the user will experience your
content differently” (Interview 8).
There is currently no perfect recipe for building AR projects in exhibitions because
we have not found the right technological combination and narrative solution. The
only way to ensure a successful AR implementation is through collaboration and
openness.
3.4.2 Design process
The design process for AR applications varies depending on the project and the
team. For exhibition design, Pam Locker (2011) suggests six stages in the design
process for exhibitions: analysis, idea, development, proposal, detail, and
installation. At each stage of the project, there will be critical feedback, which at the
end of the project takes the form of reflection on the whole in order to inform
future projects. Ibañez suggested a five-step model, reunite, define, explore,
53

Masterarbeit | Bosen Zhou
filtrate, and activate (Interview 6). Design processes combined with UX design can
be adapted to the design model proposed by Hansso Plattner Institute of Design at
Stanford: empathize, define, ideate, prototype, test, and iterate (Plattner et al.,
2018). These phases are not always sequential and do not have to follow any
specific order. Empathize means understanding the mediation goal and the target
group of the experience and understanding the physical space. It requires close
communication with all stakeholders to understand the purposes of the project.
Define refers to defining the problem that is to be solved. In the exhibition context,
it also means determining the limits and scope of the project. Ideation is to explore
all possible solutions and narrow down to the best ones. Prototype and test imply
to start creating solutions and conduct user tests for further iterations. Iteration
can happen in any phase, and it refers to fixing the problems encountered in the
previous steps. For using AR in built environments, testing and iteration are
essential to ensure a smooth experience. Sometimes, designers can also develop the
narrative strategy based on a novel technology because a structured process may
undermine AR’s current innovative nature.
Many experts agree that understanding the physical space should be the first step
to develop the mediation and narrative strategy for spatial experiences. Duncan
McCauley is an exhibition design agency from Berlin. They have developed three
spatial narrative strategies for exhibitions: linear, radial hub-and-spokes, and
dispersed, multidirectional spaces (Austin, 2020). As previously mentioned in AR
narratives, the non-linear narration is potentially more suitable for AR technology.
The way to design non-linear experiences is to break them into sub-stories or
smaller bits of event according to the physical space or content. In the arrangement
of sub-stories, Sievers suggests giving users a piece of information to trigger the
initial interest and then leave an open space for visitors to find information
(Interview 3). Beck uses a similar strategy: “With a rough structure model, I try to
break it down into a non-linear model where I can jump in different places. I also
54

Masterarbeit | Bosen Zhou
place linear modules like film snippets or animation sequences inside those non-
linear interactive applications” (Interview 4).
Flaherty suggests arranging the sub-stories into a hierarchy for content-focused
narration and strategically placing them into the experience. The ranking of them
can be “crucial”, “important”, and “good to have” for the story. Crucial pieces of
information can repeat deliberately in the experience (Interview 10). For example,
The Dial installation is a linear story but presented in a non-linear way. Each user
can use their body movements to control the timeline, and they can receive sub-
stories in different orders depending on their movements.
After having the mediation strategy accordingly to the given space or content, time
should be the next important factor. Currently, spatial AR does not have a time
limitation. However, the experience of mobile AR and AR glasses should not be too
long because it could cause tiredness; ideally, each continuous experience should
be under 10 minutes. Beck argues that it needs to be up and down of excitement
walking through the exhibition (Interview 4). A smart combination of different
media technologies can help to maximize user engagement in the experience.
3.4.3 Best practices
In the previous sections, two of the most relevant topics for AR integration have
been examined. This section explores three best practices relating to AR content
strategy: appropriateness and fluency, focus and limits, and physical-digital
balance.
The appropriateness of media in built environments is crucial for the experiences,
whether using spatial AR, mobile AR, digital eyewear, or traditional media forms.
Each media has different advantages and aesthetics to deliver certain messages.
55

Masterarbeit | Bosen Zhou
Choosing the right tools is essential for designers to create a good story world.
Ibañez describes: “Ultimately it is about the tool we have in our hands. It is also
about what the message is, with or without AR, the message has to make sense
using AR” (Interview 6). Vanhoozer shares a similar view: “You should only use a
medium if it’s the best way or the only way of telling a story, or if that medium can
maximize the story. Not everything is right for AR and VR. I strongly believe you
should fit the medium to the story, or the other way around” (Interview 9).
The media appropriateness can ensure the fluency of the story world. It means
connecting seamlessly different media to the narrative and guiding the audience’s
attention to the story, not the technology itself. If an AR interaction experience is
not intuitive or displaying something that works better in other media, it can pull
the audience out of the momentum. If we look at literature, the reading level of
Ernest Hemingway’s work is the same as an average of ten years old, similar to J.K.
Rowling’s Harry Potter series. These authors value fluency above complexity. Snow
and Lazauskas found that, on average, the most highly regarded authors on any
given subject wrote at a lower level than their peers (Snow and Lazauskas, 2018).
The same principle applies to other media. When the content is fluently connected
and easily understandable, the audience would not be intimidated by the
complexity of the media and focus more on characters, dramatic tension, and the
story itself. Ultimately, fluency makes the information more accessible for the user
to absorb.
Another significant factor in creating an AR spatial experience is defining the focus
and limits of the project. Focus means defining the message for the audience to
take away. Ibañez suggests that great storytelling is about having one central and
focused message: “We have learned from our experience, one of the problems of
the exhibition is that they want to tell you a lot of things.” He further explains that
having one single message can significantly reduce the cost and time of
56

Masterarbeit | Bosen Zhou
development and increase the enjoyment and engagement of the user’s experience.
Defining the limits of a project is equally important: “If you define the limits of the
story, there can be a lot more space for creativity” (Interview 6).
Finally, having a physical-digital balance can be one of the most essential and
distinguishing factors for creating content using AR technologies. Bettenhäuser
states: “I think an exhibition will not work where a physical exponent looks the
same as it did before if it engages with digital content. You have to rethink about
the analog pieces, probably reduce them if you want to expand digitally, it must be
a combination of these two worlds” (Interview 2). As mentioned in previous
chapters, every pixel in the AR digital space replaces a pixel in the physical space.
Vanhoozer shares his experience of creating AR content in Magic Leap Studio: “The
thing that we learned with AR is to be very thoughtful with your pixels, don’t try to
fill the screen…. Sometimes only a few pixels on the screen, a very tiny character, or
a single leaf fall in a room. The room is a character, don’t occlude it, don’t try to
overshadow it with lots of effects and noises” (Interview 8). The balance is not only
related to visual content but also to auditory and multimodal content. Designers
can thoughtfully use lighting and sound to guide the audience’s attention through
the narrative moments.
3.5 Future scenarios
According to the Gartner hype cycle representing the maturity, adoption, and social
application of specific technologies, the peak of inflated expectations of AR was
around 2011. In 2018, AR was at the trough of disillusionment, which is the lowest
cycle point. In 2019, Gartner removed AR from the hype cycle; the team describes
that AR has matured so rapidly that it is no longer considered an emerging
technology (Gartner, 2020). AR is slowly reaching the plateau of productivity.
57

Masterarbeit | Bosen Zhou
However, this is mainly referring to mobile AR. Spatial AR has already reached its
maturity many years ago, while digital eyewear development and adaptation is still
a few years away.
Vanhoozer describes his experience at Magic Leap: “The hardware is getting better
even in a year or two. This means not only will we have a wider FoV to work with,
for example, human-scale characters, but also the processing power is going
straight up. The ability to have more characters on screen, integrate deeper AI, and
try different rendering techniques that were not possible before” (Interview 8).
Similar rapid developments and innovations are happening with the tech giants in
the USA and China; this will accelerate AR’s technical standardization.
The joint efforts among the tech giants are to bring digital closer to reality.
Steinforth suggests that as the technology develops, the digital world is becoming
more prominent and closer to the real world, “Mobile AR is a kind of gimmick at
the moment, but it is an intermediate step for people to understand what AR can
be” (Interview 5). Many believe that AR in exhibitions will be a testing ground for
innovations in AR technologies before mass adoption, especially when combining
different technologies.
Location-based AR will bring a new channel of content and possibilities that do not
exist previously in the realm of spatial experience. Among which, the improvement
of content interaction could be one of the keys to open that channel. Digital
character interaction can be a substantial area of development. Interacting with
people is one of the most natural interactions we have as humans. Combing AI,
speech recognition, drama manager, spatial design, and digital eyewear
technologies, we can create remarkably personalized experiences. An example is
HotStepper, a Google Maps-based AR app where you follow a virtual character to
navigate the street. Vanhoozer describes his viewpoint: “For me, the next
58

Masterarbeit | Bosen Zhou
generation of storytelling in AR is character interaction… The concept of building a
digital character and then having the character remembering you… Those are
things we dreamed about as kids, and now we have the technologies to start putting
these together” (Interview 8).
The Covid-19 pandemic has changed the outlook of exhibitions in 2020; events and
festivals are among the most affected industries. The pandemic situation can
diminish location-based experiences and accelerate the development of ubiquitous
AR applications. Still, it will not stop us from moving toward a more connected and
integrated future. Flaherty states: “I think that experience centers for VR and AR
will continue to move forward so that people will go and experience these curated
exhibition versions” (Interview 10).
Experimentation is the driver of finding innovative content in AR. Flaherty believes
that AR in exhibitions is about finding the synthesis of corporate pressure and
artistic exploration. Currently, most investments in AR seek mass adoption and
monetization. However, this can potentially discourage creative explorations. He
argues that it is too early to segment the AR industry; for many digital artists, one
of the biggest challenges is having an income while keeping experimenting with
new things. “I think it’s important to try to figure out how to create a synthesis and
to try to create a connection between those two worlds and those two impulses so
that you can find balance in the ability to both be open and creative and be
rewarded for your work” (Interview 10).
Furthermore, another noteworthy aspect is to find a synthesis between ethics and
profit in the AR industry. Designers should consider the ramifications of having
exciting experiences with relevant content that can impact society. Flaherty further
explains: “I think it’s also about finding the synthesis between stories that are going
to help society and stories that are going to sell units… we need to be pushing
59

Masterarbeit | Bosen Zhou
forward the realm of XR that moves exciting stories, but moving away from the
supposition that first-person shooters are the only way to make income because
ultimately we need to create a community where openness, equity, inclusion, and
mental health are considered as part of what we’re creating. We need to create a
world where this technology will make humans and society better” (Interview 10).
4. Conclusion
This study explores the narrative possibilities of using AR in built environments by
mapping out relevant strategies and best practices. The context chapter outlines AR
technologies’ current context and categorizes three different categories of AR:
spatial AR, mobile AR, and digital eyewear. Moreover, it looks into the
development and characteristics of storytelling, particularly in media, digital, and
interactive narrative techniques. Finally, it inserts relevant theories in spatial
experience and narrative environments, connecting various disciplines into using
AR in the exhibition context.
The analysis chapter first examines the relevance and links between XR and
storytelling, AR, and spatial narratives. It also concludes various AR media
attributes, including the common characteristics with digital media, visualness,
multimodality, and social-individual perception levels. Moreover, it outlines the
current position, challenges, and opportunities for using AR in exhibitions.
Subsequently, it illustrates AR narrativity by constructing a triangular framework:
people and interaction, space and perspective, narrative and emotion. Together,
they form a preliminary guideline for adopting storytelling in AR. Followed by
examining some fundamental factors integrating AR in spatial experience,
including interdisciplinarity, design process, and best practices: appropriateness
60

Masterarbeit | Bosen Zhou
and fluency, focus and limits, and physical-digital balance. Lastly, the chapter
depicts future scenarios of AR development, new content channels, pandemic
impacts, creative constraints, and ethical concerns.
4.1 A new path for storytelling
The analysis chapter first outlines XR media attributes that define the common
elements of AR and VR. A fundamental difference contrasted with other media is
shifting the narrative perspective from third-person to first-person. The shift of
perspective generates a sense of presence, it means the sense of being in a
particular virtual or real environment. Secondly, four different immersion types
were introduced: spatial immersion, emotional immersion, cognitive immersion,
and sensory-motoric immersion (Björk & Holopainen, 2005). The sense of
presence creates an immersive experience similar but superior to spatial
immersion; concurrently, XR media itself does not offer other immersion types. A
narrative structure is necessary to generate emotional immersion for the user. VR
industry can develop narrative techniques based on the current film and gaming
industry, while AR narrative innovation should be based not only on media and
interactive narratives, but also on spatial narratives.
The discussion is followed by listing the current AR media attributes. AR media
technically inherited all characteristics of digital media: integration, accessibility,
connectivity, and interactivity (Herbst, 2014). Among which, interactivity should be
the essential starting point of AR because it includes HCI and spatial interactivity.
Other characteristics of AR include visual perceptibility and multimodality. AR has
some specific advantages compared to VR by combining the above attributes and
overlaying digital content on top of the physical reality. Different AR technologies
also have various social perceptibility levels. From individual to social, where
61

Masterarbeit | Bosen Zhou
digital eyewear is the most individual and spatial AR is the most social, mobile AR
is between the two. We still don’t know what attribute is most relevant in AR
because new technologies are continually being implemented into AR; therefore,
finding the right content techniques and technology combination is the key to
advancement in the AR industry.
4.2 AR for spatial experiences
Different AR technologies have different maturity levels in exhibitions; spatial AR
such as projection mapping and pepper’s ghost techniques are common practices
today, while mobile AR is mostly gimmicky. Some of the best solutions using AR in
exhibitions are usually a combination of different technologies, and the findings
suggest that most experts agree that the approach of AR in exhibition is positive.
However, such projects’ success is highly dependent on the infrastructure, and
most projects are mostly experimental. Simultaneously, exhibitions and location-
based AR experiences provide a good testing ground to explore the possibilities of
the technologies.
There are many challenges in terms of using AR in exhibitions and spatial
experiences. On the one hand, digital eyewear is currently quite expensive, and the
technology is still evolving. For AR hybrid solutions, technological standardization
is another challenge; most notably, the incompatibilities in both software and
hardware for mobile AR. For content creation, persistent digital content can help
improve the plausibility of AR experiences. Content creators have to consider the
discrepancies between expectation and reality when designing for AR in spatial
experience. On the other hand, the pandemic outbreak has heavily affected the
event and experience industry; it has also created many concerns surrounding the
logistics and safety issues when using digital eyewear in exhibitions.
62

Masterarbeit | Bosen Zhou
Nevertheless, the opportunities are just as numerous as the challenges. Firstly, AR
offers a new content channel and new perceptions for spatial experiences,
especially for contextualizing cultural and historical objects and making complex
information more accessible. Moreover, having AR experiences in a built space can
also help overcome the current challenge of ubiquitous AR space mapping, making
it easier to design content. Furthermore, digital eyewear can potentially replace all
current displays, opening new approaches to visualize content. Lastly, AR enables
more accessibility and personalized content for exhibitions and spatial experiences.
4.3 AR narrative fundamentals
This study proposes a framework of three fundamental pillars for AR storytelling
in built environments: people and interaction, space and perspective, narrative and
emotion, derived from the tripartite network of narrative environments, and active
network theory. The three areas act together as a dynamic network, mapping out
the essential components of AR narratives.
People and interaction emphasize people as the core of all narratives, creating
different types of interactions. The first interaction is between people and content,
or HCI; the second interaction is between people and environment, or spatial
interaction. Designers should consider balancing the two and incorporate social
interaction factors based on context, narrative, and media constraints. Content
interaction can often pull the audience out of the narrative mediacy. Sometimes
simple reactivity can work better than interactivity; however, the most natural
interaction is imperceptible interaction, of which the users are not aware of the
interaction.
63

Masterarbeit | Bosen Zhou
Space and perspective act as the second ingredient of the network, essential to
people and narrative. AR provides the opportunity to extend temporal and
geographical connections within a given space, offering multiple perspectives on
the topic. Designers should ultimately achieve an equilibrium between the physical
and digital space, maximize yet not overwhelm the user with content through
narrative. Fair use of space and perspective can enable new accessibility, encourage
exploration, and increase user engagement.
Narrative and emotion form the third pillar of the network. AR storytelling involves
both media and spatial narrative. The core aspect is conflict and tension, which
exist in all stories, a common component in both digital and spatial levels. Conflict
can carry out through events and evoke empathy and emotion in the audience; it
plays a crucial role in the AR narrative. The user can be both the spectator or the
protagonist in AR narratives, depending on the context. Non-linear story structures
appear to be the optimal way to engage the user in AR. Designers can manage it by
breaking the story into sub-stories or small events, allowing the audience to create
the storyline.
4.4 Building stories in AR
This study summarizes some of the essential elements when integrating AR in
exhibitions. Every media in exhibition has certain specificities and aesthetics. AR
solutions can offer some groundbreaking features but cannot yet replace other
media. As of now, hybrid solutions are commonly the most practical approaches for
using AR in spatial experiences.
Developing a project in AR involves many different disciplines, including
interaction design, spatial design, game design, scenography, and more.
64

Masterarbeit | Bosen Zhou
Collaborative work is critical to ensure a successful AR project. It is also one of the
biggest challenges in AR innovation. Apart from joint workshops to develop a
common language and perspective, designers can play a crucial role in mediating
between stakeholders during the design process.
Having a suitable design process is another area of exploration in developing AR
projects. Various design process models can be based depending on the project,
space, and team structure. However, for AR in built environments, testing and
iteration are essential to ensure a successful project. In the narrative development
process, content creators can establish a non-linear structure by categorizing the
sub-stories into a hierarchy and implement them according to spatial and timely
factors.
This study also presented some best practices in developing location-based AR
projects. Most importantly is to ensure media appropriateness and narrative
fluency. Furthermore, creators should define the focus and limits of the
experiences, particularly the message and the takeaway. Lastly, physical-digital
balance is essential to enrich the experience without overwhelming the audience.
4.5 Limitations and future research
This study’s qualitative research method provides an overview of the complexity of
the emerging AR industry today. The combination of multidisciplinary theoretical
approaches and expert interviews seeks to offer new perspectives for content
creators and promote AR content innovation. It reimagines the role of content
creators by adopting a collaborative, open, multidisciplinary approach for AR
content creation and creates valuable insights for research means, design process,
and strategy creation of AR projects. Alternatively, this study encourages
65

Masterarbeit | Bosen Zhou
innovations in the exhibition industry, as exhibitions rarely have holistic narratives
that establish connections between objects, content, space, and people. AR
technologies can potentially serve to unify these factors. This study also suggests
that the convergence of space usages and XR hybrid approaches enable the
narrative framework, not only relevant for AR in exhibitions, but also XR and
spatial experiences.
However, many limitations need to be addressed in future studies. AR is a rapidly
evolving field; each new interdisciplinary combination creates new possibilities
that can potentially change the industry’s course. The result of best practices in this
study might not be relevant in the future as new options emerge. Additionally, this
study used methods based on past experiences and individual perceptions. Further
research and experiments with quantitative and mixed methods should be
conducted to validate the outcome. Furthermore, each of the conceptualized
narrative aspects forms a new path for further and more in-depth AR content
research from diverse viewpoints. Finally, this study focuses on AR in built
environments as the testing ground and stepping stone for AR innovation;
however, with technological advancement, future research should also emphasize
ubiquitous AR.
66

Masterarbeit | Bosen Zhou
5. References
Alexander, B. (2011). The New Digital Storytelling. Creating Narratives with New
Media. Praeger.
Aristotle (1996). Poetics. Penguin.
Austin, T. (2020). Narrative Environments and Experience Design: Space as a
Medium of Communication. Routledge.
Austin, T. (2018). Some Distinctive Features of Narrative Environments. Central
Saint Martins University of the Arts London.
Barthes, R. (1977). Introduction to the Structural Analysis of Narratives, Fontana.
Bayer, H. (1939). Fundamentals of Exhibition Design. The New York Public
Library
Bazeley, P. (2007). Qualitative data analysis with NVivo. Sage.
Bedford, L. (2001). Storytelling: The Real Work of Museums. Curator The
Museum Journal, p27-34.
Bimber, O., & Raskar R. (2005), Spatial Augmented Reality: Merging Real and
Virtual Worlds, A K Peters.
Björk, S., & Holopainen, J. (2005). Patterns In Game Design. Charles River Media.
Campbell, J. (1949). The Hero with a Thousand Faces. Pantheon Books
Charmaz, K. (2006). Constructing grounded theory: A practical guide through
qualitative analysis. Sage.
Child, B. (2016). Steven Spielberg warns VR technology could be 'dangerous' for
film-making, https://www.theguardian.com/film/2016/may/19/steven-spielberg-
warns-vr-technology-dangerous-for-film-making
67

Masterarbeit | Bosen Zhou
Clack, E. (2018). How Location Tracking Affects Mobile Apps: Pokémon Go, A
Case Study, https://themanifest.com/app-development/how-location-tracking-
affects-mobile-apps-pokemon-go-game
Craig, A. B. (2013), Understanding Augmented Reality, Morgan Kaufmann.
Crawford, C. (2004), On Interactive Storytelling. New Riders
Duncan, T., & McCauley, N. (2012). A Narrative Journey: Creating storytelling
environments with architecture and digital media. Museum Making. Routledge.
Falco, F. D. & Vassos, S. (2017). Museum Experience Design: A Modern
Storytelling Methodology. The Design Journal
Gabor, D. & Stroke, G. W. (1968). The theory of deep holograms. Royal Society.
Glaser, B. G. (1978). Theoretical sensitivity. Sociology Press.
Greimas, A. J. (1983). Structural Semantics: An Attempt at a Method. University
of Nebraska Press.
Guest, G., & MacQueen, K. M. (2008). Handbook for team-based qualitative
research. AltaMira Press.
Herbst, D. G. (2013), Digital Brand Storytelling. Geschichten in digitalen Medien.
Academia.
Herbst, D. G. (2014), Storytelling. UVK Verlagsgesellschaft.
Herbst, D. G., & Musiolik, T. H. (2016). Digital Storytelling. Spannende
Geschichten für interne Kommunikation, Werbung und PR. UVK
Verlagsgesellschaft.
Herdina, M. (2020). Augmented Reality Disappeared From Gartner’s Hype Cycle.
https://arpost.co/2020/09/25/augmented-reality-gartners-hype-cycle/.
Kress, B., & Starner, T. (2013). A review of head-mounted displays (HMD)
technologies and applications for consumer electronics. Google Inc.
Lawson, B. (2001). The language of space. Architectural Press.
68

Masterarbeit | Bosen Zhou
Locker, P. (2011). Basics Interior Design Exhibition Design. AVA Publishing.
Macintyre, A. (1981). After Virtue: A Study in Moral Theory. University of Notre
Dame.
Mateas, M., & Stern, A. (2004). Façade: An Experiment in Building a Fully-
Realized Interactive Drama, Georgia Tech.
McKee, R. (1999). Story: substance, structure, style and the principles of
screenwriting. Methuen.
McLuhan M. (1964). Understanding Media: The Extensions of Man, McGraw-Hill.
Merleau-Ponty, M. (2005). Phenomenology of perception. Routledge.
Microsoft (2019). What is Mixed Reality? https://docs.microsoft.com/en-us/
windows/mixed-reality/discover/mixed-reality
Oragui, D. (2019). Mobile Augmented Reality in 2019. The Manifest, https://
themanifest.com/mobile-apps/mobile-augmented-reality-2019
Plattner, H., Meinel, C., & Leifer, L. (2018). Design Thinking Research, Making
Distinctions: Collaboration versus Cooperation. Springer.
Propp, V. (1968). The Morphology of the Folktale. Second Edition. University of
Texas Press
Qualcomm (2017). Extended reality: A convergence is coming. https://
www.qualcomm.com/news/onq/2017/05/31/extended-reality-convergence
Rahimi, F. B., Levy, R. M., Boyd, J. E., & Dadkhahfard, S. (2018). Human
behaviour and cognition of spatial experience; a model for enhancing the quality
of spatial experiences in the built environment, International Journal of Industrial
Ergonomics. University of Calgary.
Ricoeur, P. (1986). Life: A story in search of a narrator. Dordrecht: Martinus
Nijhoff.
Rostásy A., Sievers T. (2018). Handbuch Mediatektur: Medien, Raum und
Interaktion als Einheit gestalten. Methoden und Instrumente. Transcript Verlag.
69

Masterarbeit | Bosen Zhou
Saldaña, J. (2013). The coding manual for qualitative researchers. Sage.
Schwarz, U., Bertron, A., & Frey, C. (2012). Designing Exhibitions, Ausstellungen
Entwerfen. Birkhäuser
Shedroff, N. (2009). Experience Design 1.1: a Manifesto for the Design of
Experiences. New Riders
Snow, S., & Lazauskas, J. (2018), The Storytelling Edge. Wiley.
Strauss, A., & Corbin, J. (1998). Basics of qualitative research: Techniques and
procedures for developing grounded theory. Sage.
Strauss, A., & Corbin, J. (1990). Basics of qualitative research. Sage publications.
Tracy, S. J. (2010). Qualitative Quality: Eight “Big-Tent” Criteria for Excellent
Qualitative Research. Arizona State University.
Tschumi, B. (2012). “Events:The Turning Point” in Architecture Concepts: Red is
Not a Color. Rizzoli.
Unity Glossary (2020). What is AR, VR, MR, XR, 360? https://unity3d.com/what-
is-xr-glossary
Vannini, M .C. (2012). The search for authenticity in the museum visit process.
Tafter Journal experiences and tools for culture and territory. http://
www.tafterjournal.it/2012/02/01/la-ricerca-di-autenticita-nel-processo-di-visita-
museale/
Witmer, B. G., & Singer M. J. (1998). Measuring Presence in Virtual
Environments: A Presence Questionnaire. U.S. Army Research Institute for the
Behavioral and Social Sciences.
Zuckerberg, M. (2015). VR/AR strategy and One, https://www.scribd.com/
document/399594551/2015-06-22-MARK-S-VISION#
70

Masterarbeit | Bosen Zhou
6. Appendices
6.1 Appendix A: Axial codes
71

Masterarbeit | Bosen Zhou
72

Masterarbeit | Bosen Zhou
73

Masterarbeit | Bosen Zhou
6.2 Appendix B: Topic guide
Introductory conversation
What is your original field of work?!
Are you more practice-oriented or academically oriented?!
What got you interested in XR?!
How strong would you rate your insight in the field of AR and VR from 1 to 10?!
What are your experiences in AR?
AR in exhibitions
How would you evaluate the approach of AR in exhibitions?
(Negative; More Negative; More Positive; Positive)
What is your experience of using AR in exhibitions?!
In your opinion, what are the opportunities and challenges for implementing AR in
exhibitions?!
What would be some best practice examples for the use of AR in exhibitions for
you?
AR development is complex and involves museum, design, curation, coding,
engineering, and more. What are your experiences in interdisciplinary
collaboration?!
Is your development team usually internal or external?!
What would be optimal for multidisciplinary work?!Is it worth it?
74

Masterarbeit | Bosen Zhou
Storytelling in AR and exhibitions
What are the most significant aspects of storytelling in exhibitions?!
What is your design process for storytelling in exhibitions?
How do you develop a story?
How important is it to integrate into your idea with AR?!
(Not important at all; Can function as an additional element; Equal to other
exponents; It is essential)
What are the most important aspects of integrating AR into Storyline for you?!
What are your processes for combining AR in storytelling?
How important is the interaction in an exhibition with AR?
What are your experiences regarding the complexity of the interaction of AR?
Conclusion
What is the future of AR in exhibitions for you?!
What would you like to try out?
75