An Adaptive Color Marker for Spatial Augmented Reality Environments
and Visual Feedback
Ross T. Smith∗
Michael R. Marner†
Bruce H. Thomas‡
University of South Australia
Wearable Computer Laboratory
This demonstration presents an adaptive visual marker optimised to
improve tracking performance in Spatial Augmented Reality envi-
ronments. The adaptive marker uses a color light sensor to capture
the projected light color from a SAR system. The color information
is used to select the optimal tracking color that is displayed on a dif-
fused Red, Green, Blue Light Emitting Diode marker attached to a
user’s finger. We have selected to use the visible light spectrum for
the marker since it can be leveraged to present visual feedback to
support user interface interactions in addition to the tracking system
operation. Our initial results have shown a performance improve-
ment compared to a fixed color passive marker.
H.5.2 [Information interfaces and Presentation]:
Graphical User interfaces—Input Devices and Strategies; I.3.6
[Computer Graphics]: Methodology and Techniques—Interaction
Spatial Augmented Reality (SAR) uses projected light to present
perspectively correct computer generated graphics onto physical
world objects. Current research has been exploring how SAR can
be leveraged to support industrial designers during mock-up pro-
totype development . The appearance of simple white physical
compelling graphics for design. Additionally, user interface func-
tionality is incorporated into the designs without the need to install
electronic switches, dials or displays. This is achieved by using the
projected light to present a visual interface and a tracking system to
capture the finger gestures allowing interactive controls to be pro-
vided in software.
One approach to capturing the finger locations is to use passive
colored markers attached to the users fingers. Porter et al. em-
ployed orange thimbles worn on the users index finger  to allow
the 3D position to be captured using stereo cameras. Although this
technique is successful in many conditions, when the color of the
marker the reliability of the tracking is decreased. This problem is
demonstrated in Figure 1(a) where the orange passive marker is op-
erating correctly under green projected light and Figure 1(b) shows
the tracking system failing when operating under orange projected
INTRODUCTION AND MOTIVATION
We have been exploring how the performance of markers under the
projected light of a SAR system can be improved by using an ac-
tive marker. We have constructed an adaptive marker that can de-
ADAPTIVE FINGER MARKER DEMONSTRATION
Figure 1: (a) Orange passive marker performing well in green pro-
jected light. (b) Orange passive marker failing in orange projected
light. (c) Adaptive marker detecting the red projected light and se-
lecting blue as the opposite color to optimise tracking.
tect the environmental lighting conditions and change its own color
and be tracked in real-time. The adaptive marker uses an Avago
ADJD-S371-Q999 RGB color sensor with a dome lens to capture
the color of the environmental light. The captured RGB color is
then converted into the Hue Saturation Value (HSV) color space
which places colors on a virtual color wheel. We select the opti-
mal marker color by moving 180° from the environmental color
to find the opposite color. The tracking software is also configured
to search for the new color which is now easily identified since the
contrast between the environmental light and the marker is max-
The active marker uses a diffused Red, Green, Blue Light Emit-
ting Diode (RGB LED) to identify the marker in a video stream.
The visible spectrum was selected rather than IR since it can also
be leveraged for visual feedback. For example, when operating vir-
tual SAR buttons the state of the button can be displayed on the
finger worn marker. This overcomes occlusions of projected light
and can still be utilised for tracking during operation.
In this demonstration participants will be given the opportunity
to try the adaptive marker in a small scale projected SAR environ-
ment. Users will wear the marker on their index finger and move
it through a variety of projected scenes. The marker will detect
the projected light color and change its appearance to optimise the
tracking performance in real-time.
IEEE Virtual Reality 2011
19 - 23 March, Singapore
978-1-4577-0038-5/11/$26.00 ©2011 IEEE
3 Download full-text
The Wearable Computer Laboratory (WCL) was founded by Pro-
fessor Bruce H. Thomas in 1998. The WCL is located in Adelaide,
Australia at the University of South Australia. Since its inception,
the laboratory has been exploring Augmented Reality with a fo-
cus on user interfaces, input devices and visualisation techniques.
Currently there are three primary research focuses; Outdoor Mobile
Augmented Reality, Spatial Augmented Reality and Input Devices.
WEARABLE COMPUTER LABORATORY
The Tinmith system is a well established wearable computer
system developed at the WCL. Tinmith provides users with the
ability to visualize and interact with computer generated worlds
in outdoor environments. The current Tinmith wearable computer
uses a belt worn system equipped with a PC, GPS, orientation
sensor, head mounted display, head worn camera and custom made
input gloves. The research has explored how users can interact
with the virtual worlds without using traditional keyboard, mouse
or touch screen input devices. Figure 2(a) shows how each of the
users fingers is mapped to a menu item using custom pinch gloves
to capture pinching gestures. The Tinmith system has been used to
explore mining visualisations, military training scenarios, virtual
weather simulators and sensor management visualisations.
Mobile Augmented Reality
Figure 2(b) is a computer generated vision picture of a custom
designed spatial augmented reality visualisation environment. This
laboratory has recently been constructed and is now known as the
Mawson Institute Visualisation Laboratory (shown in Figure 2(c).
It consists of forty ceiling mounted projectors, ceiling mounted
scaffolding, white projections surfaces on all walls, ten dedicated
server computers with a matrix switch allowing each server to
address any projector and two large doors allowing full size cars to
enter the room. In this large scale projection area, the WCL team
is exploring how projected light can be used to augment large scale
items to assist designers during prototype development. We have
been developing a variety of interactive physical-virtual tools that
are designed to integrate traditional prototype mock-up techniques
with modern SAR technologies.
Spatial Augmented Reality
The Digital Foam sensor was developed to explore the use of
deformable surfaces for computer interactions. Figure 2(d) is an
example of a user sculpting using the Digital Foam deformable
input device . As the user deforms the surface of the device the
deformations are captured using a custom array of foam sensors
allowing the physical sculpting gestures to be applied directly to
the virtual model. We are currently exploring how the Digital Foam
technology can be incorporated into medical training devices,
robotic sensors and interactive sculpting applications.
Deformable Input Devices
 S. R. Porter, M. R. Marner, R. T. Smith, J. E. Zucco, and B. H. Thomas.
Validating the use of spatial augmented reality for interactive rapid pro-
totyping. In IEEE International Symposium on Mixed and Augmented
 R. T. Smith, B. H. Thomas, and W. Piekarski. Digital foam interaction
techniques for 3D modeling. In VRST ’08: Proceedings of the 2008
ACM symposium on Virtual reality software and technology, pages 61–
68, Bordeaux, France, 2008.
(a) Tinmith glove based input description.
(b) Vision picture of a large scale spatial augmented reality laboratory.
(c) Spatial augmented reality laboratory with projected control panels.
(d) Digital Foam deformable input device being used to sculpt a 3D model.
Figure 2: Wearable computer laboratory projects