A.K. Dey et al. (Eds.): UbiComp 2003, LNCS 2864, pp. 200–207, 2003.
© Springer-Verlag Berlin Heidelberg 2003
A 2-Way Laser-Assisted Selection Scheme
for Handhelds in a Physical Environment
Shwetak N. Patel and Gregory D. Abowd
College of Computing & GVU Center
Georgia Institute of Technology
801 Atlantic Drive, Atlanta, GA 30332-0280, USA
Abstract. We present a 2-way selection method to select objects in a physical
environment with a novel feedback and transfer of control mechanism. A
modulated laser pointer signal sent from a handheld device triggers a photosen-
sitive tag placed in the environment. The tag responds via a standard wireless
channel directly to the handheld with information regarding an object it repre-
sents. We describe a prototype implementation for a Motorola iDEN i95cl cell
phone, discuss the interaction challenges and application possibilities for this
physical world selection that extends a common handheld device. We also
compare this solution to related attempts in the literature.
It is becoming increasingly common to carry a handheld device (a PDA or a mobile
phone), with nearly always-on network connectivity and significant computational
capabilities. What is not so common is the ability for these devices to facilitate inter-
actions with the physical world. Take, for example, the development of universal
remote controls for use at home. Several commercial and research projects have con-
sidered the challenge of reducing control of a large variety of home devices down to
one universal platform (see, for example the recent work on the Personal Universal
Controller , or the Gesture Pendant ). But in an environment with many con-
trollable devices, it is not clear the best way to select which device should be con-
trolled. Pointing from a distance, naturally supported by a laser pointer, seems like a
good solution to this problem. We are motivated, therefore, to explore direct and
natural interactions with devices in the physical world mediated through a handheld
A number of researchers have explored ways to tag the physical world, using
printed barcodes or 2-dimensional glyphs, RF ID, or active beacons, in order to con-
nect the physical and electronic worlds. We are particularly interested in applications
of lasers because they provide a simple means of visual feedback as well as at-a-
distance interaction. Furthermore, unlike any other physical world selection tech-
nique in the literature, we see the advantages of creating 2-way communication be-
tween the object in the physical world and the laser-augmented handheld. The chal-
lenge we faced was to create a practical, 2-way selection technique using a laser
mounted on a conventional handheld device that provides a two-way interaction. The
A 2-Way Laser-Assisted Selection Scheme for Handhelds in a Physical Environment 207
1. Cooperstock, J., Fels, S., Buxton, W. and Smith, K.: Reactive Environments: Throwing
Away Your Keyboard and Mouse. Communications of the ACM, 40(7), pp. 65-73, Septem-
2. GKDesign Engineering: RS-232 Laser Transceiver. Electronics Australia, pp. 56-61, Octo-
3. HP Labs: The CoolTown Project. http://www.cooltown.com/research.
4. Kirstein, C and Müller, H.: Interaction with a Projection Screen using a Camera-tracked La-
ser Pointer. Proceedings of the International Conference on Multimedia Modeling. IEEE
Computer Society Press, pp. 191-192, 1998.
5. Ljungstrand, P., Redstrm, J. and Holmquist, L.E.: WebStickers: Using Physical Tokens to
Access, Manage and Share Bookmarks to the Web. In Proceeding of Designing Augmented
Reality Environments (DARE 2000), pp. 23-31, ACM Press, 2000
6. Ma , H. and Paradiso, Joseph A.: The FindIT Flashlight: Responsive Tagging Based on Op-
tically Triggered Microprocessor Wakeup. Proceedings of Ubicomp 2002, Springer-Verlag
Lecture Notes in Computer Science, Volume 2498, pp. 160-167, 2002.
7. Myers, B.A., Bhatnagar, R., Nichols, J. Peck, C.H., Kong, D., Miller, R. and Long, A.C.:
Interacting at a Distance: Measuring the Performance of Laser Pointers and Other Devices.
Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems
(CHI 2002). ACM Press, pp. 33-40, 2001, Minneapolis, Minnesota.
8. Nichols, J., Myers, B.A., Higgins, M., Hughes, J., Harris, T.K., Rosenfeld, R. and Pignol,
M.: Generating remote control interfaces for complex appliances. Proceedings of the 15th
annual ACM symposium on User Interface Software and Technology (UIST 2002), ACM
Press, pp. 161-170, Paris, France.
9. Olsen, D.R., Jr., Nielsen, T.: Laser pointer interaction. Proceedings of the ACM SIGCHI
Conference on Human Factors in Computing Systems (CHI 2001). ACM Press, pp. 17-22,
March 2001, Seattle, Washington.
10. Rekimoto J. and Ayatsuka Y.: CyberCode: Designing Augmented Reality Environments
with Visual Tags, Proceedings of DARE 2000, 2000.
11. Rekimoto, J and Katashi, N.: The World through the Computer: Computer Augmented In-
teraction with Real World Environments, Proceedings of the ACM Symposium on User In-
terface Software and Technology (UIST ’95), ACM Press, pp.29-36, Pittsburgh, PA.
12. Ringwald, M.: Spontaneous Interaction with Everyday Devices Using a PDA. Presented at
the Supporting Spontaneous Interaction in Ubiquitous Computing Settings Workshop
(UBICOMP 2002). http://www.inf.ethz.ch/~mringwal/publ/ringwald-interaction.pdf. Göte-
borg, Sweden, September 2002.
13. Starner, T., Auxier, J., Ashbrook, D. and Gandy. M.: The Gesture Pendant A Self-
illuminating, Wearable, Infrared Computer Vision System for Home Automation Control
and Medical Monitoring. The IEEE International Symposium on Wearable Computing
(ISWC 2000), IEEE Computer Society Press, pp. 87-94, Atlanta, GA.
14. Wilson, A. and Shafer, S.: XWand: UI for intelligent spaces. Proceedings of the ACM
SIGCHI Conference on Human Factors in Computing Systems (CHI 2003). ACM Press, pp.
545-552, 2003, Ft. Lauderdale, Florida.
15. Wilson, A.: Pointing in Intelligent Environments with the WorldCursor. Proceedings of In-
teract 2003, the Ninth IFIP International Conference on Human-Computer Interaction. Zu-
rich, Switzerland. To appear, September 2003.