ChapterPDF Available

Worldmaking: Designing for Audience Participation, Immersion and Interaction in Virtual and Real Spaces

Authors:

Abstract and Figures

Artists often try to open up new experiences for people, challenging them to extend horizons and perception. This becomes particularly relevant when thinking about experiencing built environments: Here, technologies like Cave Automatic Virtual Environments (CAVE) or Head-Mounted Displays (HMD) can be used as a tool to offer richer experiences to the audience in both art installations and exhibitions. We have been developing several exhibitions tackling the challenges that come with exhibiting in (semi -) public spaces: how do we engage visitors in our exhibitions, what role do bystanders play and how can this be considered in the development and design process? The exhibitions were built in a chronological order (2015–2018) and increasing degree of immersion and interaction. For exhibition one (“step-in/Ideal Spaces”), we built a CAVE-like “tryptic” projection showing linear pre-rendered videos of seven different built environments. In exhibition two (“fly-over/Super Nubibus”) we build a replica of a hot-air-balloon and let people experience architecture from birds eye view using a HMD. Exhibition three (“cruise/Biketopia”) is also an immersive VR using a HMD, but from a very different angle. Here we use a bike to let people actively explore a space by regulating speed and direction of the bike. By using the discreet method of observation, we ensured that the visitors were not disturbed in their experience, which in turn would falsify our findings. So we are able to compare and discuss these three approaches in regards to the above mentioned criteria within this paper.
Content may be subject to copyright.
Worldmaking: Designing for Audience
Participation, Immersion and Interaction
in Virtual and Real Spaces
Andreas Siess1(B
), Daniel Hepperle1, Matthias W¨olfel1,
and Michael Johansson2
1Karlsruhe University of Applied Sciences, Moltkestr. 30, 76133 Karlsruhe, Germany
{andreas.siess,daniel.hepperle,matthias.wolfel}@hs-karlsruhe.de
2Kristianstad University, Elmetorpsv¨agen 15, 291 88 Kristianstad, Sweden
michael.johansson@hkr.se
Abstract. Artists often try to open up new experiences for people, chal-
lenging them to extend horizons and perception. This becomes partic-
ularly relevant when thinking about experiencing built environments:
Here, technologies like Cave Automatic Virtual Environments (CAVE)
or Head-Mounted Displays (HMD) can be used as a tool to offer richer
experiences to the audience in both art installations and exhibitions.
We have been developing several exhibitions tackling the challenges that
come with exhibiting in (semi -) public spaces: how do we engage visi-
tors in our exhibitions, what role do bystanders play and how can this
be considered in the development and design process? The exhibitions
were built in a chronological order (2015–2018) and increasing degree of
immersion and interaction. For exhibition one (“step-in/Ideal Spaces”),
we built a CAVE-like “tryptic” projection showing linear pre-rendered
videos of seven different built environments. In exhibition two (“fly-
over/Super Nubibus”) we build a replica of a hot-air-balloon and let
people experience architecture from birds eye view using a HMD. Exhi-
bition three (“cruise/Biketopia”) is also an immersive VR using a HMD,
but from a very different angle. Here we use a bike to let people actively
explore a space by regulating speed and direction of the bike. By using
the discreet method of observation, we ensured that the visitors were not
disturbed in their experience, which in turn would falsify our findings.
So we are able to compare and discuss these three approaches in regards
to the above mentioned criteria within this paper.
Keywords: Virtual reality ·Spatial perception ·Exhibition
(Semi-) public space ·Architecture ·Museum
1 Introduction
Prototyping and exploring worlds and environments for creating exhibitions with
and through technology we sometimes use consumer technology that shortens
c
ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2019
Published by Springer Nature Switzerland AG 2019. All Rights Reserved
A. L. Brooks et al. (Eds.): ArtsIT 2018/DLI 2018, LNICST 265, pp. 58–68, 2019.
https://doi.org/10.1007/978-3-030-06134-0_7
Worldmaking: Designing for Audience Participation 59
and speeds up our work process, and we can start sketching in hard- and software
already early in the conceptual phase of a project. Even consumer products today
come with a developing kit and widely open APIs which make them suitable
also for experiments and research. One risk although here is that one as an
artist, designer and/or researcher gets trapped inside of the presumed future
use of this technology, with all limitations that comes with that [9]. Relying
on “produced” technology we clearly see a need for a set of tactics to bring
our work in our own direction. In our recent practice based work and research,
that focuses on exhibition work that spans from 2015 and onwards, we have
returned to the idea that if we want to have an audience or a visitor experience
3D spaces of different sorts in an exhibition format, we need to find ways for the
visitor to enter, interact and experience these virtual worlds, without too much
former knowledge how to enter, navigate and perceive them in order to have
the visitor feel both present and immersed by them. Because there is always a
risk with new, not seen and experienced technologies that we have the visitor
exploring, the technology itself instead rather than the topic of our attention,
e.g. spatial constructions. Through our work we gained knowledge that many
forms of interaction takes hours of practice to learn and even longer time to
master, and therefore are not fit for use directly in an exhibition, but valuable
tools for us as artists, designers and technicians to help us create things that have
not existed before. In this line of work, we have learned that configuring space
via user participation and interaction is not easy, but crucial for an immersed
experience to take place. Throughout our work, developing, methods, tools and
processes we try to emphasize the importance of a multiperspective view of
space and its entities based on the idea to transcend merely scientific or artistic
approaches into a more comprehensive and immediate approach and working
practice, in which we try to use different forms of interaction and telepresence to
create a state of immersion. It is about symbolic objects and entireties (the issue
of “Gestalt”, not about mere construction and functions). To have the created
worlds to stand out as something believable (the issue of “representation”) and
how through thoughtfully designed user interaction can create an immersive
experience for the visitor.
2 Our Three Use Cases: Step in, Fly over and Cruise
The origins of the environments for this article: The used spaces are all derived
from the exhibition at the Biennale of Architecture in Venice 2016. Starting from
this basis, selected places (i.e. the shown worlds) were made accessible with new
media devices in order to explore how the spatial impression is changing as a
result of this new approach.
60 A. Siess et al.
Fig. 1. User’s perspective: (a) “step in” (b) “fly over” (c) “cruise” as WiP.
Fig. 2. Setup: (a) “step in” (b) “fly over” (c) “cruise”
2.1 Step in: The CAVE Installation
The installation1was shown from May to November as part of the Architectural
Biennale 2016 in Venice. The setup consisted of three projectors displaying a
seamless image onto a tryptic screen (total size: 3.60 ×6.0 m) (See: Fig. 2(a)).
The screen itself consisted of three canvases, each at an angle of 150to each
other. This CAVE-like installation—where also the user’s peripheral viewport
was covered—allowed each visitor to “dive” into the shown environments and
therefore enabled an immersive experience, and to create a bodily notion of that
featured space. On this projection screen, pre-rendered movies were shown. Due
to the fact that the camera movement was not bound to any (simulated) physical
or time constraints free movement in all dimensions was possible: The environ-
ments were shown from different perspectives, but always with a constant focal
length (35 mm). This approach was chosen in order to offer visitors a realistic
size estimation and a feeling of “being there” (dweller/pedestrian).
Findings: A: Size matters, being almost inside of a projection in a cave projec-
tion creates a presence of the spaces shown. B: Since every visitor individually
can approach the cave projection and also share the experience directly with
others. C: No technical barrier for the users to overcome. D: Immediateness: No
latency/lag and low-threshold for a majority of users.
1Created by ideal spaces working group 2016 (Matthias W¨olfel, Michael Johansson,
Daniel Hepperle, Andreas Siess, Ulrich Gehmann et al.).
Worldmaking: Designing for Audience Participation 61
2.2 Fly over: The Balloon Installation
For the ZKM—Center for Arts and Media in Karlsruhe, Germany we wanted to
set up an installation that is connected to the place where it is exhibited. Because
of its special kind of city planning (“fan city”) we already presented a version
of Karlsruhe in our CAVE Installation and therefore it was of interest for us to
make it accessible (here: Karlsruhe from 1834) from another perspective by rid-
ing over it via a balloon. In general spaced VR installations have some inherent
disadvantages for each individual visitor: The action space available to the user is
very limited and any danger of collision must be prevented in advance, e.g. by dis-
playing warnings (“chaperones”). How useful these interventions might be, they
strongly interrupt the immersive experience. Therefore, a concept is required that
plays creatively with the limitations of VR and simultaneously offers firm sup-
port in case of need. Hence we developed an installation that naturally and cred-
ibly shapes the playing space without being perceived as a limitation. We chose
to build a replica of a balloon-basket that is also mapped 1:1 in the virtual envi-
ronment (See: Figs. 1(b) and 2(b)). With this setup our installation features: 1.
birds-eye perspective for experiencing the special kind of Karlsruhe’s architecture
2. slow movements to reduce motion sickness 3. When wanting to ride/fly over a
city there currently is a rise in hardware that require users to strap themselves
onto it, which might be not perfectly suitable for (semi-) public space and also
for physically handicapped or introverted people (see [21]or[17]) and therefore
we thought a balloon might suit better. In addition, clinging to the balloons rail-
ing also reduces motion sickness and increases immersion [3]. To start the ride,
visitors had to enter the nacelle, which is also represented within the virtual envi-
ronment and pull down the rope (represented in physical and virtual space) to
start the ride/virtual burner (See: Fig. 2(b)). The ride took place on a pre-given
path with the possibility to change height by pulling down the rope to fuel the
burner. In the real environment, the burner was represented by spotlights which
radiated enough heat and was augmented with vibrations of the nacelle to foster
the illusion. Another trick to further improve the illusion of flying, was to add an
airstream using a fan. This, of course, is not correct from a physical point of view,
but fits well to most of the visitors (most of them who have not yet experienced a
real balloon ride).
Findings: A: One can increase immersion using elements that might not be
physically correct (wind in a balloon, vibration when pulling the trigger). B:
One can regulate viewing direction by implementing spatially located audio files.
C: Intentionally regulating the given space by using a physical restriction also
displayed in the virtual world (nacelle) helps people to orientate oneself within
the virtual reality. D: Although HMDs are becoming increasingly popular and
we added clear instructions next to the exhibition, museum guides still had to
attend the exhibition to help out the visitors.
62 A. Siess et al.
2.3 Cruise: The Bike Installation
The bike installation was shown to a broader public in an art/design per-
formance/exhibition in two cities (Kristianstad and Copenhagen). In this
art/design project with the bike we returned to Jeffrey Shaw’s original idea
from 1986, “Legible City” [18]. The Bike installation was one of five stations in
which the visitors could explore different ideas about urban development, here
in this installation we had the visitors to visit 15th century Milano to experience
a conceptual city space never realized. “As a space, [it] is constructed in such
a way that we always have the impression to see only a fragment, a more or
less small section of it—because at every meters x, the next structural element
can appear, ad infinitum” [5]. For this project we use a VirZoom bike with its
speed and direction sensors to explore two of the worlds from the 2016 exhibi-
tion. Here we want to see what happens when you take some worlds developed
for animation/film and translate that world into real time graphics (Unity3d).
What specific qualities get lost and which transfers well?
Findings: A: Direct immersion for the visitor when entering a virtual world
with a familiar navigational device such as the bike. They know how to direct
themselves almost immediately and there is a low threshold to learn how to nav-
igate oneself. B: Biking is not walking, therefore a space needs some adjustment:
this was a bit tricky since this world needs to be changed to fit navigation by
bicycle rather than by walking, for example have the bicycle go from one floor
to the next, using stairs could have been an option, but does not relate very
well to cruising—it will be a bumpy ride. We also wanted to keep the original
plan of the city and not introducing “modern” or “alien” elements into it. So to
keep the flow in constantly biking through Milano, we used ramps as discreet as
possible integrated in the original environment to move the visitor on it’s bicycle
in between the different levels of the city, and at the same time point the biking
experience for the visitor into new directions. C: The bike soon became an impor-
tant tool developing the worlds—iterating between modeling/texturing/lighting
and the experience of the changes made by bicycling in that area of attention.
D: It made us aware of the difference between seeing something on the screen
(editing) and experience it in full VR [19]. E: The bike itself is as we found out
also a camera rig and can be used for classical camera work to get camera data
out to other 3d programs but also to produce animation paths for Unity3d itself.
3 Thoughts on Audience Participation
One downside that comes with head mounted devices (HMD) in (semi-) public
spaces is, that they can only be used by one person at the time. In addition, while
experiencing the virtual world, the person who wears the HMD is almost com-
pletely isolated from his surrounding although other visitors can easily observe
him. This “voyeurism” behavior is quite similar to the one described as one of
the first parts of the audience funnel framework by Michelis and M¨uller 2011
[15]. Their framework is based on observations made in regards to interactions
Worldmaking: Designing for Audience Participation 63
with public displays, but while they argue that visitors either pass by the display
or view and react, we would like to add another category: The voyeur implicitly
does not want to experience the VR exhibition, but wants to passively observe
and find out what is happening. While in general, the term voyeurism has a
rather unpleasant notion, in this setting, this behaviour can lead to a honey-pot
effect, which then will bring other people to observe and interact with the current
user/voyeur or to try it by oneself. There are several possibilities one can come
up with to include bystanders into the installation (see Fig.3): 1. Directly dis-
play the virtual world onto a 2D display placed so that bystanders and voyeurs
can see it. 2. Include the actual user into the 3d world using a greenscreen as
described by Intel in 2017 [22]. 3. (Floor) projections [7,23]. 4. Project facial
expression on the front of the VR headset for bystanders to see it [11,13]. There
is also a lot to decide about on how to add auditory cues, but it would extend
the scope of this discussion2. In our current state, we are considering several
possibilities in engaging visitors to interact with the current user. For example:
should the visitor be able to see the virtual content on a 2D screen befo re he got
fully immersed into the virtual environment via HMD or should one try to only
make the content visible for participants after they went into the virtual reality
itself?
Fig. 3. Audience “funnel” for “Biketopia” VR exhibition.
4 Thoughts on Immersion
By using familiar or at least conventional devices, such as the CAVE, the balloon
and the bike, the users’ direct physical presence in the virtual environment rein-
2Kuutti 2014 discusses some of the different possibilities in (semi-) public spaces [10].
64 A. Siess et al.
forced the sense of physical presence of the virtual world and the interactivity
of the respective space or device used in our case studies enhance their sense of
immersion. We tried to lay the “focus on the activity, not the technology” and
made “the control mechanism obvious” [14] when designing the necessary tech-
nical part of our installations. Conversely, this means that immersion cannot be
designed directly, but that we have ensured that as many interfering factors as
possible are eliminated. As an artist/designer one should always be aware that
one never designs the actual immersed experience itself, but only the frame-
work wherein that experience can take place. Since the amount of immersion
is not easily measurable, we classify our three works by using the parameters
Slater and Wilbur came up with [20]. As one can see in Table 1the range of
sensor modalities (Extensiveness) used, varies. Also the idea on how visitors can
interact (Interactability) with the different installations is altered between the
exhibitions.
Table 1. Immersion in dependence on to the parameters defined in [20]
Cave Balloon Bike
Extensiveness Linear audio, video 3D spatial audio,
interactive visuals
(HMD)
3D spatial audio
interactive visuals
(HMD)
Matching None Vive tracking sensors Oculus rift sensors
Surroundness
(horizontal)
120110110
Vividness 30 fps
3600 ×1920 px
60 fps
2160 ×1200 px
30 fps
2160 ×1200 px
Interactability None Start, change height Start, change
direction, position,
velocity
Experience Multi person Single user
(observation possible)
Single user
(observation possible)
5 Thoughts on Interaction
On interaction we played safe and in each of our examples we deliberately used
well familiar forms of interaction by having the visitor, “step into”, “fly over” or
“cruise” our different environments. We wanted established forms that helped
the visitors more directly make the transition from the physical world on a device
they already used or at least seen before. Therefore interactivity and navigation
used in both the balloon and the bike was directly added to the experience
by a tight coupling between display, the movements of the user’s body (when
maneuvering the device) on which they made the transition from one world to
Worldmaking: Designing for Audience Participation 65
another. In the case of the CAVE example there were no interaction through any
technology, here we instead used the design of the physical space itself to have
the visitor just step into the worlds showed. Fundamentally, we can state that the
concept of affordance [6] from interaction design can also play to its strengths in
our context: Although it is ultimately just a sophisticated controller for a virtual
environment, the bicycle is clearly recognizable to every visitor. This simplicity
and accessibility effectively prevents fears of contact and takes little time to get
used to. The balloon can show these strengths even more effectively: The basket
restricts the possible degrees of freedom very effectively without actually being
perceived as a plain restriction or overruling. We believe that the museal context
demands this mixture of strict constraints and clear affordances to be successful.
As a designer it is our job to find concepts that integrate these principles into
a harmonious environment that plays creatively with these limitations and, at
best, uses them as framework for conceptual design.
6 Developing Visual Style and Aesthetics
Developing through iteration is common practice in many art/design disciplines
and also for our type of work in which we strive to facilitate an openness towards
what happens in-between the design cycles. Similar to the OODA-Loop [2]in
which one Observes, Orients, Decides and takes Action, in our case to challenge
the machine and software logic and our own limits and conventions, to produce
something that is both unexpected and valuable, and in the end will point out
possible new directions. So using this loop switching between the state of editing
(tweaking the parameters) and experience (being there) help us develop and put
forward qualities (that sometimes is produced by errors or wrongdoings) that
are hidden from the concept’s point of view and by doing so—have the concept
redeveloped itself through the results and experiences we achieve by iteration.
We try to explore what is being shadowed by the concept itself. Therefore we
do not formulate any detailed specification in relation to the concept before-
hand, or rather loosely “not photo real” or “do not use textures”. But at the
same time it is important for us to be able to control or at least understand
the in- and outputs of the development environment itself—from bits to spaces.
The shortcomings, errors and quirks are vital resources that suddenly can reveal
themselves as major feature with specific new qualities. When trying to follow the
design intention or concept, the materials/methods and prototypes themselves
and when we are “bending” [4] them, can produce qualities not known before-
hand. In our line of practice-based work for this to happen, the production needs
to reach a certain state of complexity, to be able for all of the possible parame-
ters to be explored thoroughly and have an impact on the details as well as the
whole. Therefore a production environment itself is crucial to facilitate practice
based research in the area of art and design, where one can game the rules of
play [9], otherwise there is a risk that art/design projects are just illustrations
of technology, and technology driven projects unreflected use conventions and
qualities from art. In the three projects we specifically address this in the visual
66 A. Siess et al.
styles we tried to develop. At present, one of the most important parameters to
benchmark an experience in VR is “realism”. But realism even technologically
advanced is still a cultural construction [12]. And realism in relation to digital
simulated ones, is not realism, it is reality seen by the camera lens, as Allen points
out “The intention of all technological systems developed since the beginning of
the 1950s has been towards reducing the spectators sense of their real world,
and replacing it with a fully believable artificial one” [1]. And even if realism
would be possible in a perfect replica, it is still a blunt showcase of technological
frameworks and not an independent aesthetic quality. If one really wanted to
depict physical realism in its original form, then a theoretically infinitely level
of detail would be necessary, which not only seems technically impossible, but
in particular also not desirable from an artistic perspective. So when designing
our environments in the first we have deliberately decided to work in favor of
this non realistic approach, to find a level that is believable rather then realistic.
For example we used the same light and shaders to develop a common style that
connected the seven disparate worlds from our first project with each other, to
find a reasonable level to avoid the uncanny valley—a mismatch or break in
presence where believability does not clash with falsehood [16]. And it is also
important to avoid that, “the virtual environment becomes less immersive since
they lose the interest and engagement of the user envision the reconciliation of
immersion and interactivity” [15]. During our iterative design process, a visual
aesthetic gradually emerged that portrayed the spatial situation in a very mini-
malistic, reduced yet consistent way. In the context of the “hunt for realism”, we
constructed just the opposite: a world that deviates completely from this ideal
and only shows details where they are indispensable. And that’s why we assume
these environments worked so well in our use case: because we found that con-
sistent and credible design is not tied to photorealism–although this hypothesis
needs empirical foundation in our future user tests.
7 Future Work
Currently we are actively working on Motopia—an environment designed by
Geoffrey Jellicoe in the 1960ies [8]—to enable this space for the VR-bike as well
as for a VR-car simulation. Since this conceptual environment was once meant to
be the city of motor-mobility we are interested if the 3D bike is able to tap these
potentials. For the Biketopia installation we are currently thinking about let-
ting the user take pictures from within the virtual environment and only present
those pictures to the audience (bystanders, voyeurs etc.). This could improve
conversation between the user and the bystanders in such a way, that they want
to ask more questions and maybe want to experience the world itself. In addi-
tion, the pictures might help the authors to get an idea of what the actual users
think might be the most interesting parts of their journey. We then can evaluate
this and improve our work based on this feedback. For the balloon flight we
implemented an Immersit Shaker System, which simulated the vibrations of a
balloon’s basket and ensured direct haptic feedback for pulling the drawstring.
Worldmaking: Designing for Audience Participation 67
We expect that the repetitive vibration that occurs when the (virtual) expan-
sion joints of the roadway are passed once again clearly emphasizes the spatial
impression as the underlying concept of Motopia as a modern era functional
city. In regards to our observations on audience participation, the next step will
be to monitor visitor-behaviour via top-down camera to generate heat-maps in
regards to where they are while they are in the balloon’s “sphere of activity” We
will continue to further investigate how space in different media is represented
via media technology and how it is used.
References
1. Allen, M.: Technology in Contemporary Hollywood Cinema. Taylor & Francis, New
York (2013)
2. Boyd, J.: Organic design for command and control (2005). https://www.
ausairpower.net/JRB/organic design.pdf. Accessed 15 July 2018
3. Carrozzino, M., Bergamasco, M.: Beyond virtual museums: experiencing immersive
virtual reality in real museums. J. Cult. Herit. 11, 452–458 (2010)
4. Eagleman, D., Brandt, A.: The Runaway Species: How Human Creativity Remakes
the World. Catapult, London (2017)
5. Gehmann, U.: Exhibition Venice 2016–ideal spaces (2016). https://www.
idealspaces.org/exhibition-venice-2016/#leonardodivinci. Accessed 29 Sept 2018
6. Hartson, R.: Cognitive, physical, sensory, and functional affordances in interaction
design. Behav. Inf. Technol. 22(5), 315–338 (2003)
7. Ishii, A., et al.: ReverseCAVE. In: ACM SIGGRAPH 2017. ACM Press (2017)
8. Jellicoe, G.: Motopia: A Study in the Evolution of Urban Landscape (1961)
9. Kajo, M., Johansson, M.: Common playground. In: Proceedings of Cast01 (2001)
10. Kuutti, J., Leiwo, J., Sepponen, R.E.: Local control of audio environment: a review
of methods and applications. Technologies 2(1), 31–53 (2014)
11. Kwatra, V., Frueh, C., Sud, A.: Headset “removal” for virtual and
mixed reality. https://ai.googleblog.com/2017/02/headset-removal-for-virtual-
and-mixed.html. Accessed 15 July 2018
12. Lister, M., Giddings, S., Dovey, J., Grant, I., Kelly, K.: New Media: A Critical
Introduction. Routledge, Abingdon (2010)
13. Mai, C., Rambold, L., Khamis, M.: TransparentHMD: revealing the HMD user’s
face to bystanders. In: Proceedings of the 16th International Conference on Mobile
and Ubiquitous Multimedia. MUM 2017, pp. 515–520. ACM (2017)
14. Maynes-Aminzade, D., Pausch, R., Seitz, S.: Techniques for interactive audience
participation. In: Proceedings of the 4th IEEE International Conference on Multi-
modal Interfaces, p. 15. IEEE Computer Society (2002)
15. Michelis, D., M¨uller, J.: The audience funnel: observations of gesture based inter-
action with multiple large displays in a city center. Int. J. HCI 27(6), 562–579
(2011)
16. Mori, M.: Bukimi no tani [the uncanny valley]. Energy 7, 33–35 (1970)
17. Revresh: Para parachute! http://revresh.com/paraparachute/. Accessed 15 July
2018
18. Shaw, J., Groeneveld, D.: Legible city. https://www.jeffreyshawcompendium.com/
portfolio/legible-city/. Accessed 15 Sept 2018
19. Sieß, A., H¨affner, N., W¨olfel, M.: Color preference differences between head
mounted displays and PC screens. IEEE (2018)
68 A. Siess et al.
20. Slater, M., Wilbur, S.: A framework for immersive virtual environments (FIVE):
speculations on the role of presence in virtual environments. Presence-Teleop. Virt.
Environ. 6(6), 603–616 (1997)
21. Somniacs: Birdly - the ultimate dream of flying. http://www.somniacs.co/.
Accessed 15 July 2018
22. Tyrrell, J., Bancroft, J., Gerald, M.: Sharing VR through green screen mixed reality
video (2017). https://software.intel.com/en-us/articles/sharing-vr-through-green-
screen-mixed-reality-video. Accessed 12 June 2018
23. Zenner, A., Kosmalla, F., Speicher, M., Daiber, F., Kr¨uger, A.: A projection-based
interface to involve semi-immersed users in substitutional realities. IEEE (2018)
... They replaced "subtle interactions" by "get in touch with the hardware" to emphasize that the bystanders need to familiarize themselves with the hardware by inspecting and even touching it. Siess et al. [16], on the other hand, introduced an adapted version of the audience funnel to fit to immersive VR installations in (semi-)public places. ...
... The general idea of our installation "Super Nubibus" 4 is to present a ride in a hot air balloon over the town of Karlsruhe in the year 1834. A first version, without floor projection, had already been presented at the ZKM Karlsruhe [16]. Users can experience the ride using a physical replica of a balloon basket and an HTC Vive Pro VR headset. ...
... It is worthwhile to take a look at the other aspects that also motivated passersby to try out the installation. As already stated in the previous paper [16], the honeypot effect is also a good motivator to convince passers-by. Watching another person try it out seems to be an excellent way to break down the barriers to entry. ...
Conference Paper
Full-text available
As virtual reality becomes more popular to be used in semi-public spaces such as museums and other exhibition venues, the question on how to optimally stage such an experience arises. To foster interaction between participants and bystanders, to lower the primary threshold in regards to participation and to moderate the transition between real and virtual worlds we propose to augment a virtual hot-air balloon ride by a large scale floor projection in addition to a physical basket and other extras. Exhibited at a venue in Stuttgart, Germany a total of 140 participants evaluated our approach. We could confirm that adding a floor projection helped to attract additional users, to increased the overall motivation on using the installation, and to established a connection between the real and the virtual worlds.
... In addition, theories regarding the interaction with large displays and HMD have been introduced: To describe the interaction of immersive VR installations in public places, the audience funnel [26], which has been introduced in relation to public displays, needs to be adapted. While Mai and Khamis [21] proposed to replace "subtle interactions" (which we refer to implicitly interacting) by "get in touch with the hardware" to emphasize that the bystanders need to familiarize themselves with the hardware by inspecting and even touching it, we think [30] that both steps need to be considered in such cases where the HMD is accompanied by large screens. ...
... The installation Super Nubibus 5 presents a VR ride in a hot air balloon over the town of Karlsruhe in the year 1834. It has been presented on various occasions including the ZKM Karlsruhe [30]. Users can experience the ride using a physical replica of a balloon basket and an HMD. ...
Article
Full-text available
Location-based virtual reality (LBVR) is promising to offer the full potential of virtual reality (VR) becauseit is not completely virtual, as in space independent VR, which usually relies on a head-worn display andhand-held controllers only. Therefore, LBVR can include and arrange the physical surrounding according to aparticular application or include parameters, such as objects or light conditions, of the physical surroundinginto the virtual environment. While LBVR is drawing a lot of attention in the creative industry and a lot ofLBVR-installations are already entertaining millions of users, not much research has been employed in thisdirection. In this work, we offer an overview containing relevant challenges and accompanying solutions thatneed to be considered in an LBVR-installation.
... For them, extraction was performed, and the snowballing method was applied, identifying 30 new noteworthy items. After a similar process 6 additional publications [13][14][15][16][17][18] were selected for analysis. ...
... is paper, we want to show how 3D objects, movement and light sources can facilitate new forms of image and map-making, through a series of transformational steps mediated as a shadow world and captured on a white surface. From previous projects, we highlight the importance of making use of the quirks, errors and shortcomings that constantly appear (Siess et. al 2019) and that-if used smartly-can become major features of the design with specific new qualities. "We must integrate the element of the unknown into the design process as a constitutive, productive factor for design-not simply as a lack of data, but as a driver of design development." (Folkmann 2014) The paper presents a model of how our pr ...
Conference Paper
Full-text available
This paper explores the initial phase of a series of prototype-based design investigations in the field of visual and interactive computing from an artistic and design-oriented perspective. We propose a novel paradigm for interacting with prototypes, particularly suited for the contexts of design and art. Accordingly we demonstrate how this interaction, referred to as "probing", differs from the traditional approach of prototyping (i.e. experimenting). These findings are exemplified and illustrated by an actual prototype that is presented alongside. By introducing this prototype, which can be understood as an artistic framework, we derive a model that systematises the creative work with and on prototypes into an epistemological typology. Through this "probing" we come to realise the importance of embracing and utilising the quirks, flaws and limitations that arise, which can become prominent features of the design with unique qualities. Finally, we provide insights and a model how these concepts can be applied to prototype-based design and development in general. Keywords: Art & Design; Probing; Art; Playful Interaction; Transformational Stepping.
Article
Full-text available
Art installations that engage in a dialogical relationship with their surrounding environment, transcending the confines of an isolated existence, demand a nuanced articulation of the dynamic interplay amongthe artwork, the spatial context, and the observer. The following reportendeavors to delineate and investigate the central elements of reception and the aesthetic of production pivotal to the media art installation ‘Skopéin’, exhibited at the Evangelische Stadtkirche Karlsruhe during the late summer of 2022, through the lens of ethnographic introspection (‘autoethnography’). Given that the authors of this discourse are concurrently the creators of the aforementioned installation, the following text serves as an exploratory analysis into the fabrication process of amedia art installation, employing anthropological methods.
Chapter
As virtual reality becomes more popular to be used in semi-public spaces such as museums and other exhibition venues, the question on how to optimally stage such an experience arises. To foster interaction between participants and bystanders, to lower the primary threshold in regards to participation and to moderate the transition between real and virtual worlds we propose to augment a virtual hot-air balloon ride by a large scale floor projection in addition to a physical basket and other extras. Exhibited at a venue in Stuttgart, Germany a total of 140 participants evaluated our approach. We could confirm that adding a floor projection helped to attract additional users, to increased the overall motivation on using the installation, and to established a connection between the real and the virtual worlds.
Conference Paper
Full-text available
Recently virtual reality (VR) applications are shifting from professional use cases to more entertainment-centered approaches. Therefore aesthetic aspects in virtual environments gain in relevance. This paper examines the influence of different color determining parameters on user perception habits between head mounted displays (HMD) and computer screens. We conducted an empirical study with 50 persons that were asked to adjust the color temperature, saturation and contrast according to their personal preferences using a HMD as well as a computer screen, respectively. For cross validation we tested a second user group of 36 persons that were asked to adjust the color temperature exclusively. By using a set of five different panorama images-each of them representing an exemplary scenario-we have found that color perception differs significantly. This depends on the used output device as well as gender: i.e. females preferred a significantly colder color scheme in VR compared to their preferences on the computer screen. Furthermore they also chose a significant colder color scheme on the HMD compared to their male counterparts. Our findings demonstrate that content created for conventional screens can not simply be transferred to immersive virtual environments but for optimal results needs reevaluation of its visual aesthetics.
Conference Paper
Full-text available
While the eyes are very important in human communication , once a user puts on a head mounted display (HMD), the face is obscured from the outside world's perspective. This leads to communication problems when bystanders approach or collaborate with an HMD user. We introduce TransparentHMD, which employs a head-coupled perspective technique to produce an illusion of a transparent HMD to bystanders. We created a self contained system, based on a mobile device mounted on the HMD with the screen facing bystanders. By tracking the relative position of the bystander using the smartphone's camera, we render an adapting perspective view in realtime that creates the illusion of a transparent HMD. By revealing the user's face to bystanders, our easy to implement system allows for opportunities to investigate a plethora of research questions particularly related to collaborative VR systems.
Article
Full-text available
The concept of a local audio environment is to have sound playback locally restricted such that, ideally, adjacent regions of an indoor or outdoor space could exhibit their own individual audio content without interfering with each other. This would enable people to listen to their content of choice without disturbing others next to them, yet, without any headphones to block conversation. In practice, perfect sound containment in free air cannot be attained, but a local audio environment can still be satisfactorily approximated using directional speakers. Directional speakers may be based on regular audible frequencies or they may employ modulated ultrasound. Planar, parabolic, and array form factors are commonly used. The directivity of a speaker improves as its surface area and sound frequency increases, making these the main design factors for directional audio systems. Even directional speakers radiate some sound outside the main beam, and sound can also reflect from objects. Therefore, directional speaker systems perform best when there is enough ambient noise to mask the leaking sound. Possible areas of application for local audio include information and advertisement audio feed in commercial facilities, guiding and narration in museums and exhibitions, office space personalization, control room messaging, rehabilitation environments, and entertainment audio systems.
Conference Paper
Virtual Reality (VR) has advanced significantly in recent years and allows users to explore novel environments (both real and imaginary), play games, and engage with media in a way that is unprecedentedly immersive. However, compared to physical reality, sharing these experiences is difficult because the user's virtual environment is not easily observable from the outside and the user's face is partly occluded by the VR headset. Mixed Reality (MR) is a medium that alleviates some of this disconnect by sharing the virtual context of a VR user in a flat video format that can be consumed by an audience to get a feel for the user's experience. Even though MR allows audiences to connect actions of the VR user with their virtual environment, empathizing with them is difficult because their face is hidden by the headset. We present a solution to address this problem by virtually removing the headset and revealing the face underneath it using a combination of 3D vision, machine learning and graphics techniques. We have integrated our headset removal approach with Mixed Reality, and demonstrate results on several VR games and experiences.
Conference Paper
Virtual reality (VR) with HMD is closed experience among those who are experiencing the VR, and can only be individually experienced by the specific person. We call this "perspective gap." These perspective gaps exist in many situations. To address these problems, we present "ReverseCAVE", a system for sharing the experiences of people in VR with others (observer). As another application, it is possible to visually recognize the actual appearance of the person performing the act at the motion capture studio and the superimposed character at the same time. ReverseCAVE has four translucent screens surrounding the player. VR environment that the player is experiencing is projected onto the screens. By this, the observer can see both the physical player and the VR environment experienced by the player simultaneously. Also, in the motion capture system, when viewing the actor from the observer outside of ReverseCAVE, the character is superimposed to the actor. This makes it look as if the actor is the actual character from the observer. ReverseCAVE enhances the observers' experience.
Article
First description of the uncanny valley theory
Article
Contemporary museums are much more than places devoted to the placement and the exhibition of collections and artworks; indeed, they are nowadays considered as a privileged means for communication and play a central role in making culture accessible to the mass audience. One of the keys to approach the general public is the use of new technologies and novel interaction paradigms. These means, which bring with them an undeniable appeal, allow curators to modulate the cultural proposal by structuring different courses for different user profiles. Immersive Virtual reality (VR) is probably one of the most appealing and potentially effective technologies to serve this purpose; nevertheless, it is still quite uncommon to find immersive installations in museums. Starting from our 10 years' experience in this topic, and following an in-depth survey about these technologies and their use in cultural contexts, we propose a classification of VR installations, specifically oriented to cultural heritage applications, based on their features in terms of interaction and immersion. On the basis of this classification, aiming to provide a tool for framing VR systems which would hopefully suggest indications related to costs, usability and quality of the sensorial experience, we analyze a series of live examples of which we point out strengths and weak points. We then summarize the current state and the very next future, identifying the major issues that prevent these technologies from being actually widespread, and outline proposals for a more pervasive and effective use of Immersive VR for cultural purposes.
Article
Data are presented from observations of Magical Mirrors, a set of four large public displays with gesture-based interaction installed in downtown Berlin, Germany. The displays show a mirror image of the environment in front of them and react with optical effects to the gestures of the audience. Observations of audience behavior revealed recurring behavioral patterns, like glancing at a first display while passing it, moving the arms to cause some effects, then directly approaching one of the following displays and positioning oneself in the center of the display. This was often followed by positioning oneself in the center of the other displays to explore the possibilities of the different effects, and sometimes by taking photographs or videos. From these observations a framework of interaction with gesture-based public display systems was deduced. It describes the phases of passing by a display, viewing & reacting, subtle interaction, direct interaction, multiple interactions, and follow-up actions. Quantitative data of these behavioral phases was collected by observing 660 passers-by on 2 weekend evenings. This article shows how many passers-by pass the thresholds between these phases. This “Audience Funnel” should provide a framework to encourage systematic investigation of public display systems and enable comparability between different studies.
Article
This paper reviews the concepts of immersion and presence in virtual environments (VEs). We propose that the degree of immersion can be objectively assessed as the characteristics of a technology, and has dimensions such as the extent to which a display system can deliver an inclusive, extensive, surrounding, and vivid illusion of virtual environment to a participant. Other dimensions of immersion are concerned with the extent of body matching, and the extent to which there is a self-contained plot in which the participant can act and in which there is an autonomous response. Presence is a state of consciousness that may be concomitant with immersion, and is related to a sense of being in a place. Presence governs aspects of autonomic responses and higher-level behaviors of a participant in a VE. The paper considers single and multi-participant shared environments, and draws on the experience of Computer-Supported Cooperative Working (CSCW) research as a guide to understanding presence in shared environments, The paper finally outlines the aims of the FIVE Working Group, and the 1995 FIVE Conference in London, UK.