An Immaterial, Dual-sided Display System with 3D Interaction

Abstract

We present an interactive wall-sized immaterial display that introduces a number of interesting possibilities for advanced interface design. The immaterial nature of a thin sheet of fog allows users to penetrate and even walk through the screen, while its dual-sided nature allows for new possibilities in multi-user faceto- face collaboration and pseudo-3D visualization.

Full text

Page 1

An Immaterial, Dual-sided Display System with 3D Interaction
Alex Olwal1,2 Stephen DiVerdi1 Nicola Candussi1 Ismo Rakkolainen1,3 Tobias Höllerer1
1University of California at Santa Barbara
Santa Barbara, CA 93106
USA
alx@kth.se, sdiverdi@cs.ucsb.edu, nicola@cs.ucsb.edu, ismo.rakkolainen@tut.fi, holl@cs.ucsb.edu
2Royal Institute of Technology (KTH)
100 44 Stockholm
Sweden
3FogScreen Inc
Kampusranta 9 C, 60320 Seinäjoki
Finland
ABSTRACT
We present an interactive wall-sized immaterial display that in-
troduces a number of interesting possibilities for advanced inter-
face design. The immaterial nature of a thin sheet of fog allows
users to penetrate and even walk through the screen, while its
dual-sided nature allows for new possibilities in multi-user face-
to-face collaboration and pseudo-3D visualization.
CR Categories and Subject Descriptors: H.5.1 [Multimedia
Information Systems]: Artificial, augmented, and virtual realities;
H.5.2 [User Interfaces]: Graphical user interfaces; I.3.6 [Method-
ology and Techniques]: Interaction Techniques.
Additional Keywords: display, fogscreen, immaterial, walk-
through, reach-through, two-sided, transparent, interactive
1INTRODUCTION
Interactive screens, objects and user interfaces that seem to float
in mid-air can be generated in a variety of ways. Traditional AR
and VR often use head-worn displays where 6DOF head tracking
is used to provide the correct perspective. Artifacts such as mis-
registration and lag are commonly experienced problems that
detract from the sense of presence in the VR or AR scene.
The interest in non-encumbering technologies has led to several
related systems, such as volumetric displays , primitive immaterial
displays based on water, smoke or fog [8, 3, 12, 1], and large
translucent interactive systems [6, 13] using transparent materials
(such as [4] and [7]). The Heliodisplay [5] is a medium-sized (22-
42 inches) immaterial display based on the same principle as the
patented FogScreen. Single-sidedness, smaller format and table-
top setup, makes it less suitable as a walk-through multi-user in-
teractive system.
Our contribution is a novel interactive, dual-sided, wall-sized
system that allows multiple unencumbered users to naturally ma-
nipulate objects floating in midair, as shown in Figure 1. Two
individual, but coordinated, images are projected onto opposite
sides of a thin film of dry fog and an integrated 3D tracking sys-
tem allows users on both sides to interact with the content, while
the non-intrusive and immaterial display makes it possible to
freely pass physical objects between users or move through the
shared workspace. Our system opens up possibilities for a wide
range of collaborative applications where face-to-face interaction
and maximum use of screen estate is desirable, as well as the
maintenance of individual views for different users.
2THEIMMATERIALFOGSCREEN
The FogScreen is an optimal, patented method for forming a
physically penetrable particle display [9, 11] that allows projec-
tion of interactive content to appear floating in free space. The
basic principle is the use of a large non-turbulent airflow to pro-
tect a flow of particles, such as dry fog, inside it from turbulence.
The resulting thin, stable sheet of fog enables projections on a
screen that is dry and feels like moderately cool air. The light
from a standard LCD projector is scattered through this sheet of
fog, creating a rear-projection image.
2.1 Dual-sided Projection and Interaction
The FogScreen’s opacity depends on many parameters such as
fog density, projector and image brightness, and background of
the viewing area. Very little of the projected light actually reflects
from the fog layer back towards the projecting source. This makes
it possible to simultaneously display different images on the two
sides of the FogScreen using two opposing projectors.
Dual-sided systems present many new opportunities in applica-
tion interfaces. Our elastic surface deformer application renders
both sides of a 3D object on the two sides of the FogScreen, creat-
Figure 1. Our immaterial dual-sided display system in action, running several demo applications. Left to right: a) Consigalo. A multiplayer
game with responsive soundtrack and touchscreen-style interaction that exploits the dual-sided aspects of our system. b) Rigid body simula-
tor. An easy-to-use mechanical dynamics simulator that allows you to interact with objects in a physically intuitive manner. c) Virtual forest
tour. A portal into a virtual world with thousands of dynamically rendered trees, where the user can walk and look around.
285
Please see supplementary material on conference DVD.
IEEE Virtual Reality 2006
March 25 - 29, Alexandria, Virginia, USA
1-4244-0223-9/06/$20.00 ©2006 IEEE
Proceedings of the IEEE Virtual Reality Conference (VR’06)
1087-8270/06 $20.00 © 2006 IEEE

Page 2

ing a more convincing sense of presence of the virtual object in
the physical environment, as shown in Figure 2.
Collaboration among multiple people within a single applica-
tion can be greatly enhanced by a dual-sided display. With our
dual-sided system, the same interface and layout can be presented
on both sides of the screen, but text and images can be properly
oriented so all users can actively participate. Collaboration is also
encouraged through its inherent support for face-to-face interac-
tion and eye contact.
2.2Walk-through and Reach-through Interaction
The immaterial nature of the display is important for enabling
new interaction possibilities. While two traditional displays
mounted back-to-back could present a similar pseudo-3D display
to multiple users, the opaque displays would prohibit users from
effectively collaborating across the two displays by obscuring
users’ views of each other, distorting speech, and making it diffi-
cult to pass physical objects across the display. This permeability
of the FogScreen is important not only for collaboration but also
for a variety of new interfaces.
The ability for a user to walk completely through the FogScreen
makes it appropriate as a portal in virtual or mixed reality envi-
ronments. A CAVE with a FogScreen as a wall, for instance,
would allow the user to easily enter the virtual environment from
the outside. Interfaces could also initiate actions when certain
objects are moved through the display. Our system could for in-
stance support the Magic Lens metaphor [2] – e.g., objects that are
moved through the screen could get augmented with graphics.
This would create an intuitive way to add virtual annotation and
augmentation to physical objects without requiring the user to
wear any equipment or learn an interface to operate.
3 3DINTERACTION
We developed several applications for our system that we pre-
sented at SIGGRAPH 2005, Emerging Technologies (See Figure
1 and 2). A vision-based 3D tracking system (WorldViz PPT) was
used with custom-built wireless controllers to enable 3D interac-
tion. The goal with each application was to explore a different
interaction mechanism using the unique capabilities of the
FogScreen and our input devices.
The Rigid Body Simulator lets users intuitively interact with
realistically behaving virtual objects. Our application simulates a
low-gravity environment using standard mechanical dynamics,
where virtual rigid bodies bounce around as they collide with
user-controlled paddles.
The Virtual Forest Tour explores the concept of a portal to a
virtual environment with a first-person point of view using our
3DOF wireless controller for interaction. The user navigates in an
aesthetic landscape where thousands of plants are realistically
rendered and animated with the Planta SDK [10].
Consigalo is an engaging multiplayer game where the user
touches the screen to grab and drag objects at a location, or moves
away from the screen to release objects. Collaborative face-to-face
3D interaction allows two-sided play – one can switch sides by
moving through the screen while interacting with the content.
The Elastic Surface Deformer allows the user to stretch and
sculpt, as well as to move and rotate, an elastic 3D model of a
human head. The front and back views are projected on opposite
sides, such that the user can walk through the screen to see what
the object looks like from the other side, in a pseudo-3D fashion.
4CONCLUSIONS AND FUTUREWORK
The FogScreen has shown itself to be a captivating display
technology that immediately generates interest and excitement in
the audience. The feedback from SIGGRAPH 2005, Emerging
Technologies was unanimously enthusiastic about the display
system as well as the potential for the interface technology. Ongo-
ing work includes improved image quality, stereoscopic render-
ing, and haptic feedback. In the interest in developing a fully
volumetric 3D display, we are currently investigating the use of
multiple FogScreens to allow images to occupy a tangible physi-
cal volume surrounding the user. This would overcome a funda-
mental limitation of existing volumetric displays, allowing the
user to become fully immersed in the 3D visualization, while ena-
bling direct manipulation of the scene.
ACKNOWLEDGEMENTS
Thanks to Karri Palovuori at Tampere University of Tech-
nology, Andy Beall and Matthias Pusch at Worldviz LLC, and
Rogerio Feris, Jason Wither and Alberto Candussi at UCSB.
This research was in part funded by grants from NSF IGERT in
Interactive Digital Multimedia, the Sweden-America Foundation,
Academy of Finland, TTY tukisäätiö, Emil Aaltonen Foundation,
Tekes, Alfred Kordelin Foundation, Jenny and Antti Wihuri Foun-
dation, and Finnish Cultural Foundation and its Pirkanmaa
Branch.
REFERERNCES
[1] Aquatique. Aquatique Show International,
http://www.aquatique-show.com/. Dec 2005.
[2]Bier, E., Stone, M., Pier, K., Buxton, W. and DeRose, T. Toolglass
and Magic Lenses: The Seethrough Interface, Proc. ACM SIG-
GRAPH, 1993, pp. 73-80.
[3] Desert Rain project, http://www.crg.cs.nott.ac.uk/events/rain/. De-
cember 2005.
[4] dnp Holo Screen. DNP, http://www.dnp.dk/. Dec 2005.
[5] Heliodisplay. http://www.io2technology.com/. Dec 2005.
[6] Hirakawa, M. and Koike, S. A Collaborative Augmented Reality
System using Transparent Display. Proc. ISMSE'04, 2004, pp. 410-
416.
[7]HoloClear. HoloDisplays, http://www.holodisplays.com/. Dec 2005.
[8] Just, P. C. Ornamental fountain. U.S. patent 620,592. March 7, 1899.
[9] Palovuori,. K. and Rakkolainen., I. FogScreen. U.S. patent 6,819,487
B2. November 16, 2004.
[10] Planta SDK. Fluid Interactive,
http://www.fluidinteractive.com/. Dec 2005.
[11] Rakkolainen, I., Palovuori, K., A Walk-thru Screen. Proc. SPIE Vol.
4657, Projection Displays VIII, 2002, pp. 17-22.
[12] Sugihara, Y., Tachi, S., Water Dome - An Augmented Environment.
Proc. Information Visualization Conf. ’00, 2000, pp. 548-553.
[13] Wilson, A. D. 2004. TouchLight: an imaging touch screen and dis-
play for gesture-based interaction. Proc. ICMI '04, 2004, pp. 69-76.
Figure 2. The support for dual-sided projection allows two independ-
ent images to be projected on each side, such that a 3D application
can render two interactive views for a pseudo-3D effect.
Left: Front view. Right: Rear view.
286
Proceedings of the IEEE Virtual Reality Conference (VR’06)
1087-8270/06 $20.00 © 2006 IEEE