An adaptive color marker for Spatial Augmented Reality environments and visual feedback.
ABSTRACT This demonstration presents an adaptive visual marker optimised to improve tracking performance in Spatial Augmented Reality environments. The adaptive marker uses a color light sensor to capture the projected light color from a SAR system. The color information is used to select the optimal tracking color that is displayed on a diffused Red, Green, Blue Light Emitting Diode marker attached to a user’s finger. We have selected to use the visible light spectrum for the marker since it can be leveraged to present visual feedback to support user interface interactions in addition to the tracking system operation. Our initial results have shown a performance improvement compared to a fixed color passive marker. Index Terms: H.5.2 [Information interfaces and Presentation]: Graphical User interfaces—Input Devices and Strategies; I.3.6 [Computer Graphics]: Methodology and Techniques—Interaction Techniques
- SourceAvailable from: Ross T. Smith
Conference Paper: Digital foam interaction techniques for 3D modeling.[Show abstract] [Hide abstract]
ABSTRACT: Digital Foam is a new input sensor developed to support clay like sculpting and modeling operations. We present techniques facilitat- ing navigation and manipulation operations performed using Spher- ical Digital Foam as a sole input device. Our free-form sculpting technique allows manipulation of new and existing 3D models us- ing accumulated sculpting like motions. Digital Foam's multi-point pressure sensitive surface captures the separate locations of a user's fingertips allowing controlled manipulation of multiple model ver- tices simultaneously. Additionally, we developed a technique that allows the camera view and zoom to be controlled by applying vary- ing pressure to the Digital Foam surface. Furthermore, we have designed a menu system tailored for operation using Spherical Dig- ital Foam as a sole input device using both the internal orientation sensor and the pressure sensitive surface. A new higher resolution Spherical Digital Foam input device with 162 unique pressure sensors is presented. This is a significant im- provement in comparison to the previous Spherical Digital Foam version with only 21 sensors. We discuss the design issues and how an increased resolution affects the operation and design of the algorithms used. We propose a new dynamic button allocation tech- nique made possible using the new high resolution Spherical Digital Foam. Finally, we performed a trial study using the new 162 sen- sor Spherical Digital Foam input device to evaluate elements of the menu system.Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST 2008, Bordeaux, France, October 27-29, 2008; 01/2008
Conference Paper: Validating Spatial Augmented Reality for interactive rapid prototyping.[Show abstract] [Hide abstract]
ABSTRACT: This paper investigates the use of Spatial Augmented Reality in the prototyping of new human-machine interfaces, such as control panels or car dashboards. The prototyping system uses projectors to present the visual appearance of controls onto a mock-up of a product. Finger tracking is employed to allow real-time interactions with the controls. This technology can be used to quickly and inexpensively create and evaluate interface prototypes for devices. In the past, evaluating a prototype involved constructing a physical model of the device with working components such as buttons. We have conducted a user study to compare these two methods of prototyping and to validate the use of spatial augmented reality for rapid iterative interface prototyping. Participants of the study were required to press pairs of buttons in sequence and interaction times were measured. The results indicate that while slower, users can interact naturally with projected control panels.9th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2010, Seoul, Korea, 13-16 October 2010; 01/2010
An Adaptive Color Marker for Spatial Augmented Reality Environments
and Visual Feedback
Ross T. Smith∗
Michael R. Marner†
Bruce H. Thomas‡
University of South Australia
Wearable Computer Laboratory
This demonstration presents an adaptive visual marker optimised to
improve tracking performance in Spatial Augmented Reality envi-
ronments. The adaptive marker uses a color light sensor to capture
the projected light color from a SAR system. The color information
is used to select the optimal tracking color that is displayed on a dif-
fused Red, Green, Blue Light Emitting Diode marker attached to a
user’s finger. We have selected to use the visible light spectrum for
the marker since it can be leveraged to present visual feedback to
support user interface interactions in addition to the tracking system
operation. Our initial results have shown a performance improve-
ment compared to a fixed color passive marker.
H.5.2 [Information interfaces and Presentation]:
Graphical User interfaces—Input Devices and Strategies; I.3.6
[Computer Graphics]: Methodology and Techniques—Interaction
Spatial Augmented Reality (SAR) uses projected light to present
perspectively correct computer generated graphics onto physical
world objects. Current research has been exploring how SAR can
be leveraged to support industrial designers during mock-up pro-
totype development . The appearance of simple white physical
compelling graphics for design. Additionally, user interface func-
tionality is incorporated into the designs without the need to install
electronic switches, dials or displays. This is achieved by using the
projected light to present a visual interface and a tracking system to
capture the finger gestures allowing interactive controls to be pro-
vided in software.
One approach to capturing the finger locations is to use passive
colored markers attached to the users fingers. Porter et al. em-
ployed orange thimbles worn on the users index finger  to allow
the 3D position to be captured using stereo cameras. Although this
technique is successful in many conditions, when the color of the
marker the reliability of the tracking is decreased. This problem is
demonstrated in Figure 1(a) where the orange passive marker is op-
erating correctly under green projected light and Figure 1(b) shows
the tracking system failing when operating under orange projected
INTRODUCTION AND MOTIVATION
We have been exploring how the performance of markers under the
projected light of a SAR system can be improved by using an ac-
tive marker. We have constructed an adaptive marker that can de-
ADAPTIVE FINGER MARKER DEMONSTRATION
Figure 1: (a) Orange passive marker performing well in green pro-
jected light. (b) Orange passive marker failing in orange projected
light. (c) Adaptive marker detecting the red projected light and se-
lecting blue as the opposite color to optimise tracking.
tect the environmental lighting conditions and change its own color
and be tracked in real-time. The adaptive marker uses an Avago
ADJD-S371-Q999 RGB color sensor with a dome lens to capture
the color of the environmental light. The captured RGB color is
then converted into the Hue Saturation Value (HSV) color space
which places colors on a virtual color wheel. We select the opti-
mal marker color by moving 180° from the environmental color
to find the opposite color. The tracking software is also configured
to search for the new color which is now easily identified since the
contrast between the environmental light and the marker is max-
The active marker uses a diffused Red, Green, Blue Light Emit-
ting Diode (RGB LED) to identify the marker in a video stream.
The visible spectrum was selected rather than IR since it can also
be leveraged for visual feedback. For example, when operating vir-
tual SAR buttons the state of the button can be displayed on the
finger worn marker. This overcomes occlusions of projected light
and can still be utilised for tracking during operation.
In this demonstration participants will be given the opportunity
to try the adaptive marker in a small scale projected SAR environ-
ment. Users will wear the marker on their index finger and move
it through a variety of projected scenes. The marker will detect
the projected light color and change its appearance to optimise the
tracking performance in real-time.
IEEE Virtual Reality 2011
19 - 23 March, Singapore
978-1-4577-0038-5/11/$26.00 ©2011 IEEE
The Wearable Computer Laboratory (WCL) was founded by Pro-
fessor Bruce H. Thomas in 1998. The WCL is located in Adelaide,
Australia at the University of South Australia. Since its inception,
the laboratory has been exploring Augmented Reality with a fo-
cus on user interfaces, input devices and visualisation techniques.
Currently there are three primary research focuses; Outdoor Mobile
Augmented Reality, Spatial Augmented Reality and Input Devices.
WEARABLE COMPUTER LABORATORY
The Tinmith system is a well established wearable computer
system developed at the WCL. Tinmith provides users with the
ability to visualize and interact with computer generated worlds
in outdoor environments. The current Tinmith wearable computer
uses a belt worn system equipped with a PC, GPS, orientation
sensor, head mounted display, head worn camera and custom made
input gloves. The research has explored how users can interact
with the virtual worlds without using traditional keyboard, mouse
or touch screen input devices. Figure 2(a) shows how each of the
users fingers is mapped to a menu item using custom pinch gloves
to capture pinching gestures. The Tinmith system has been used to
explore mining visualisations, military training scenarios, virtual
weather simulators and sensor management visualisations.
Mobile Augmented Reality
Figure 2(b) is a computer generated vision picture of a custom
designed spatial augmented reality visualisation environment. This
laboratory has recently been constructed and is now known as the
Mawson Institute Visualisation Laboratory (shown in Figure 2(c).
It consists of forty ceiling mounted projectors, ceiling mounted
scaffolding, white projections surfaces on all walls, ten dedicated
server computers with a matrix switch allowing each server to
address any projector and two large doors allowing full size cars to
enter the room. In this large scale projection area, the WCL team
is exploring how projected light can be used to augment large scale
items to assist designers during prototype development. We have
been developing a variety of interactive physical-virtual tools that
are designed to integrate traditional prototype mock-up techniques
with modern SAR technologies.
Spatial Augmented Reality
The Digital Foam sensor was developed to explore the use of
deformable surfaces for computer interactions. Figure 2(d) is an
example of a user sculpting using the Digital Foam deformable
input device . As the user deforms the surface of the device the
deformations are captured using a custom array of foam sensors
allowing the physical sculpting gestures to be applied directly to
the virtual model. We are currently exploring how the Digital Foam
technology can be incorporated into medical training devices,
robotic sensors and interactive sculpting applications.
Deformable Input Devices
 S. R. Porter, M. R. Marner, R. T. Smith, J. E. Zucco, and B. H. Thomas.
Validating the use of spatial augmented reality for interactive rapid pro-
totyping. In IEEE International Symposium on Mixed and Augmented
 R. T. Smith, B. H. Thomas, and W. Piekarski. Digital foam interaction
techniques for 3D modeling. In VRST ’08: Proceedings of the 2008
ACM symposium on Virtual reality software and technology, pages 61–
68, Bordeaux, France, 2008.
(a) Tinmith glove based input description.
(b) Vision picture of a large scale spatial augmented reality laboratory.
(c) Spatial augmented reality laboratory with projected control panels.
(d) Digital Foam deformable input device being used to sculpt a 3D model.
Figure 2: Wearable computer laboratory projects