Scrollpad: tangible scrolling with mobile devices

Abstract

In this paper we approach the problem of managing large visual sets of data on small mobile devices. While current approaches either focuses on 1) scrolling on the mobile device, or 2) reducing the content in various ways (e.g. zooming, automatic redesign depending on the screen size of a mobile device, etc) our approach is to scroll with the mobile device itself (i.e. object in the world scrolling) over a large virtual area. We present the background for this project and working prototype called Scrollpad developed to illustrate this concept. We then present an initial user study conducted and relate this project to similar efforts made before concluding the paper.

Full text

ScrollPad: Tangible Scrolling With Mobile Devices
Daniel Fällmana, Andreas Lundb, Mikael Wibergb
aInteractive Institute, Tools for Creativity
Studio, Tvistev. 47, SE-90719, Umeå, Sweden
bInteraction Theory Lab, Department of Informatics, Umeå University
901 87 Umeå, Sweden
{dfallman, alund, mwiberg}@Informatik.umu.se
Abstract
In this paper we approach the problem of
managing large visual sets of data on small mobile
devices. While current approaches either focuses on 1)
scrolling on the mobile device, or 2) reducing the
content in various ways (e.g. zooming, automatic
redesign depending on the screen size of a mobile
device, etc) our approach is to scroll with the mobile
device itself (i.e. object in the world scrolling) over a
large virtual area. We present the background for this
project and working prototype called ScrollPad
developed to illustrate this concept. We then present
an initial user study conducted and relate this project
to similar efforts made before concluding the paper.
1. Introduction
A problem with mobile devices is that they have
limited screen real estate. Further on, many sets of
visual data are larger then a normal PDA, or mo -bile
phone screen (e.g. regular web pages or street maps)
thus making it hard for the user to get an overview of
the content. This problem has so far been approached
from two angles. Either 1) traditional desktop computer
scrolling functionality has been implemented on the
mobile device, or 2) content has been reduce in various
ways (e.g. zooming, automatic redesign of screen
layout depending on the screen size of a mobile device,
[6] etc). However, these two approaches each have
their drawbacks. The first approach (i.e. traditional
graphical scrolling functionality) brings an additional
graphical element onto an al-ready tiny screen thus
reducing the area for the content to be displayed on
even further. The second approach (i.e. reducing or
reconfiguring the content for "optimal" fit according to
a specific device) has several drawbacks in that several
versions of the content has to be developed (or at least
several algorithms for automatic redesign has to be
developed), and the user will not be presented to the
same look ´n feel as in front of an ordinary computer
which might introduce additional navigational problems
for the user since he/she might not recognize e.g. a
familiar web page although it's design and navigation
might be well known on a desktop PC.
In our research we have explored an alternative
approach to these two alternatives outlined above. Our
approach builds on an idea to scroll with the mobile
device itself over a large virtual area.
The rest of the paper is structured as follows: In
section 2 the background and fundamental idea behind
our project is outlined followed by section 3 where we
describe the design of ScrollPad, a working prototype
that enables its user to, in a very tangible way, scroll
large virtual areas by just grabbing the PDA in one
hand and then move the mobile device itself around
over a flat surface. In section 4 we present an initial
user study of the ScrollPad followed by a summary of
the results from that study in section 5. We then
discuss the design of ScrollPad (section 6) before
relating it to similar efforts made (section 7) and
concluding the paper in section 8.
2. Background
In this section we outline the basic idea behind the
design of ScrollPad. Basically, the difference between

scrolling the content on the device vs. scrolling with
the device itself can be formulated accordingly:
A PDA can on the one hand be understood as a
self-contained world of objects (e.g. programs, files,
etc). On the other hand, the mobile device itself is an
object in the world. In our project we can thus
distinguish between on the one hand: 1) traditional
stylus scrolling on a mobile device as an act of
manipulating a world of objects inside a PDA without
considering that the device itself is actually a movable
(read "mobile") object in the world outside the digital
world (often referred to as the "real" world in contrast
to the virtual world), i.e. world of object scrolling and
on the other hand 2) scrolling with the device itself as
an act of object in the world scrolling.
We believe that our approach (i.e. the object in the
world scrolling approach) have great potential, not at
least due to the current trend towards embodied
interaction [1], and TUIs (Tangible User Interfaces)
where the user interact with the computers in a more
physical and direct way.
3. Prototype design
In order to illustrate how these two high-level
conceptualizations (i.e. world of object scrolling vs.
object in the world scrolling) can inform design of
mobile technology and to make empirical evaluation
possible, two prototypes were designed and
implemented. In the following, each of these designs is
presented in relation to their respective high-level
conceptualization.
3.1 Scrolling a world of objects
The first prototype (i.e. the StylusScroller) was
designed to reflect a typical way of designing scrolling
interfaces, where the PDA takes on the role of a world
of objects that can be manipulated by the user. The
only functionality it supports is the possibility to scroll
a bitmap image by means of direct manipulation of the
image. In order to display a new section of the image,
the user grabs hold of the image and drags it in any
direction. Given this style of interaction, the document
subject to scrolling can be considered to be in motion
relative to a fixed PDA.
3.2 Introducing ScrollPad: Scrolling the world
with an object
The second prototype was designed to manifest the
view of PDAs as objects in the world. Compared to the
first prototype, the PDA is not only a container of
interactive objects but also very much an interactive
object in itself. In order to scroll a document, the user
slides the PDA on a flat surface to gradually reveal new
areas of the document. The interactive qualities of the
prototype emerge in the blend of the characteristics of
the PDA and the flat surface. Figure 1 (top) illustrates
this idea:
Fig. 1. By physically moving the PDA itself around
like an ordinary computer mouse the virtual display
"underneat" the device is displayed on the mobile

device (top). Schematic figure of a PDA mounted onto
an optical mouse to enable scrolling over a flat surface
by moving the device around (middle), and final
implementation of the working ScrollPad (bottom).
The design of ScrollPad was realized through of a
combination of the sensing circuitry of an off-the-shelf
optical mouse and a custom made circuit. The main
function of the custom made circuit is to read
movement data as reported by the sensing circuitry and
communicate the data to the PDA. The implementation
makes use of the PDA’s capacity for infrared data
communication, which allows for an untethered, serial
link between the PDA and the sensing device.
As shown in figure 1 (bottom), the circuitry is fitted
inside a casing designed to allow for smooth sliding
over flat surfaces.
As the user slides the PDA, the application software
in the PDA reads the motion data from the infrared port
of the PDA and updates the display accordingly. This
solution has proved to be a simple, yet effective way of
prototyping the notion of the PDA as an object in the
world.
4. User study
In order to substantiate the conceptual distinction
between PDAs as objects in the world versus worlds of
objects with some empirical grounding, we have
conducted a preliminary user study. This study was
part of a larger research project (i.e. [5] on visualization
of time and events. In this paper, we report only on
those aspects that are relevant in relation to the
conceptual distinction.
4.1 Design of user study and test procedure
A total of 16 participants took part in the evaluation,
with an equal gender balance. Each participant used
both the ScrollPad and the StylusScroller alternative
(i.e. traditional scrolling on the mobile device).
Two different datasets and associated tasks were
devised to avoid situations where a participant’s self-
reported experience is influenced by the information
content of the prototypes rather than the design. Thus,
each participant was confronted with two datasets, one
for each prototype. To address potential effects that a
combination of a prototype design and a dataset might
have, measures were taken alter the combinations of
dataset and design. Also, to compensate for any effects
that the order in which the different design are used,
the order was altered as well.
Given the two prototypes and two datasets, the
following orderings are possible:
1. First ScrollPad with dataset 1, then StylusScroller
with dataset 2
2. First ScrollPad with dataset 2, then StylusScroller
with dataset 1
3. First StylusScroller with dataset 1, then SlideScroller
with dataset 2
4. First StylusScroller with dataset 2, then ScrollPad
with dataset 1.
This set of combinations was repeated two times for
each gender.
A dataset in this context refers to a temporal
landscape with events with a start and an end. A
portion of a landscape is shown in figure 2 and a larger
part of it is visible in figure 3. The datasets implies a
fictional scenario of university events (lectures,
seminars, etc) that takes place in time at specified
locations, with names persons responsible for each
event. This kind of scenario was chosen because it is a
familiar kind of context for the participants.
Associated with each dataset was a set of tasks that
the participants were given one at a time after an initial
briefing about the evaluation and about a minute of
getting familiar with the prototype. Examples of such
tasks are:
• What event or events overlap the course introduction
in political led by A Wigren?
• Assume that you should schedule a session in the
timetable that must not start before 11.00 and no later
than 20.00. What alternatives do you have at your
disposal?
• I would like you to locate the event that concerns
Preparations that K Lavander and others from
informatics are involved in?
The evaluation was not concerned with
performance measures such as time to completion, error
rates and other quantitative aspects of performance.
The tasks were used as a means to get the participants
to express themselves and, hence, they were asked to
think aloud while attending to the tasks.
The participant’s interactions and expressions
thereof were recorded with a digital video camera.

Figure 2 below shows a user scrolling on the device
with an ordinary stylus (i.e. the StylusScroller). This
figure also shows what is actually visible to the user on
both the StylusScroller prototype and the ScrollPad.
Figure 3 on the other hand shows the ScrollPad in use
where the user physically moves the whole mobile
device around to uncover the information that is
virtually “underneath” the device. However, the virtual
surface outside the screen of the PDA is not visible to
the user during actual use. Here we have made a
photomontage where we have added that virtual
surface underneath the device just to illustrate how the
device shows different parts of a larger material when it
is physically dragged over the flat surface.
Fig. 2. Stylus scrolling on the PDA.
Fig. 3. The ScrollPad in use and here presented as a
photomontage that helps illustrating the idea of a
virtual surface “underneath” the device.
Finally, figure 4 shows a user during the user study as
he is holding the device in his left hand and physically
drags it up and down across the A3 paper on the desk.
Figure 4. Picture taken of the ScrollPad in use during
this initial user study.
5. Results
In this section we present some observations made
during this initial user study. The observations
concerns both 1) interactional aspects of the ScrollPad
in use as well as it covers 2) some observations made
that is more related to the technical aspects of the
current implementation of the ScrollPad prototype.
Below we present these findings in more detail.
5.1 Interactional aspects of ScrollPad in use
As seen in figure 4, a sheet of A3 paper was
fastened to the desktop. The reason for doing that was
twofold. First, it proved to be a better surface for the
ScrollPad’s optical sensor. Second, and more
interestingly, it was meant to serve as a point of
reference for the interaction with the prototypes.
When asked what appeared to be a somewhat
surprising question concerning the extension in
centimeters of the temporal landscape, several
participants answered by referring gesturally to the A3
paper as if it were a yardstick. This happened only
when the question was asked in relation to the
ScrollPad prototype. This seems to resonate well with
the aim of realizing the notion of a PDA as an object in
the world.
What we take to be an inclination to refer to the
environment outside of the PDA is not solely based on
gestures that the participants made. Also, there were
some articulated expressions such as the one in the
following quotation:

-“Well, I see that this.., you scroll upwards, along the
wall here or the table here to get as far up as
possible” (Respondent 4).
We find this particular kind of expression interesting
because it suggests that the participant’s experience of
interacting with the PDA, in the case the ScrollPad, is
not confined to the device us such. Rather, the
interaction experience seems explicitly influenced by
the physical context where it was used.
5.2 Technical aspects of ScrollPad in use
The sensing technology used for the ScrollPad is
inherently relative. Position is inferred from motion,
rather than tracked absolute according to a specific
geographic position. This design aspect was evident in
the user study. Some participants used the ScrollPad
much the same way as a mouse is used, i.e. by
repeatedly sliding, lifting, retracting, sliding, etc. An
absolute position tracking, would not allow for that
kind of interaction. Plausibly, a more forcing, absolute
tracking of the PDA geographic position would
emphasize the notion of the PDA as an object in the
world. The device position and current interface view
would be more tightly coupled. However, the choice
between relative and absolute position tracking is not
without trade-offs, as will be argued in the discussion
below.
The design of the ScrollPad requires the device to be
used in conjunction with a flat surface, preferably a
tabletop. On the hand, this requirement can be seen as
a limitation, as done by [4]. On the other hand, it can
also be seen as a quality of the design that ensures a
certain distance between the device and its users that
allows for collaborative use of a single device. Further
on, while observing the participants using the two
prototypes, it seems as if the use of the ScrollPad is
associated with problems related to lack of visibility
due to glare and a less than optimal viewing angle. This
was not the case with the StylusScroller. This can be
explained by the fact that PDAs are typically designed
to be used vertically by one user that is holding the
device upright in his or her hand. This is very different
from the use scenario we are envisioning where several
persons together and simultaneously uses a one PDA
horizontally on a flat surface. The advantage here
when it comes to our implementation of this by relying
on a relative position tracking technique is that it will
not only allow for passing the device around, but also
allow for the whole virtual workspace “underneath” the
device to be moved around from person to person as
the device is passed on to another user.
6. Discussion
Stylus interaction – as embodied by the first
prototype (i.e. the StylusScroller) – promotes a certain
kind of use that is largely private and individual. In
order to manipulate the interface successfully, the user
needs to handle the PDA in such a way that makes it
difficult for other people than the one holding the
device to take part in the interaction. In that regard,
PDAs are similar to interaction with physical notepads
in that shared use typically requires one user to hand
the device over to some else for that person to take part
in the interaction. Just as stylus interaction promotes a
certain kind of use, the sliding kind of interaction
promotes a different kind of interaction that allows for a
more social use, where several people can take part in
the interaction.
The aim of the evaluation was not about arriving at a
conclusion concerning which of the two prototypes is
the best one. Rather, the results seem to indicate that
PDAs can be designed to embody quite fundamentally
different high-level conceptualisations. We have
investigated PDAs as objects in the world and
compared with what we take to be the more common
conceptualisation, PDAs as worlds of objects. In doing
so, we believe that we have uncovered a largely
unexplored parts of the design space of PDAs and
mobile technology. Having that said, we could however
observe that several subjects had various kinds of
problems with the StylusScroller whereas the ScrollPad
seemed to be a very intuitive tool to the users. This
might be because all of the subjects in this study where
used to scrolling with an ordinary computer mouse.
Finally, the view of PDAs as worlds of objects is
associated to a kind of mobility that emphasizes the
mobility of the user. We suggest that viewing PDAs as
such emphasizes the mobility of the artifact in
interaction. This corresponds to the distinction
between local and micro-mobility made by [4].
7. Related work
Of particular relevance to the work presented in this
paper is the work of Fitzmaurice et al on spatially aware
plamtop computers (see for instance, [2] and [3]). More
recently, Yee [7] has presented work on so-called

Peephole Displays. However, while Yee [7] focuses on
how to create an exact mapping between a certain
absolute position in the real world and a similar position
in the virtual world we focus our attention on relative
positioning solutions. We believe that relative
positioning between the physical and virtual world has
its advantages for three different reasons:
1) First, people are used to traditional mouse
interaction, which also relies on relative positions (e.g.
the mouse pointer might not be at the very right on the
screen just because the mouse is place close to the
right edge of the desk).
2) Second, relative positioning allows for mobile use
of the device since the virtual content can follows the
user wherever he/she goes (compared to an absolute
positioning solution where the virtual world is in a fix
location as a layer on the physical world).
3) Finally, the relative positioning solution does not
require heavy additional equipment (e.g. a ultrasonic
tracking system mounted in the sealing) to make it run.
Instead it can be run everywhere (both indoors and
outdoors) as long as it can rest on a flat surface such as
a desk or a floor.
8. Conclusions and future work
In this paper we have presented the concept of
object in the world scrolling to approach the problem
of viewing large sets of data on small mobile devices.
Our approach contributes to the two current
approaches to this problem (i.e. on display scrolling
and automatic content reconfiguration) by enabling a
mobile device itself to function as a combined scrolling
and viewing device. Our work also contributes to
similar efforts made by focusing on relative positioning
to enable also the virtual surface to become mobile,
thus enable its user to use the device wherever he/she
wants to go, thus supporting mobility rather then exact
mapping between the geographic position in the real
world against a similar position in the virtual world. In a
sense, ScrollPAD realizes relative context awareness on
a micro-mobility [2] level of analysis.
We believe that our approach to the scrolling
problem on small displays might be fruitfully applied for
application areas like web navigation and street map
viewing where the user needs to get an overview of a
large graphical area. We also think that this interaction
technique can open up new dimensions for computer
games on mobile devices.
The designs presented in this paper open up
possibilities for different kinds of future research.
Amongst other things, we find it important to generate
knowledge concerning the design and use of mobile
technology conceived of as worlds of objects versus
objects in the world. In our studies we will also focus
on the use of the prototypes in collaborative settings. It
is our hypothesis that the different prototypes will give
rise to quite different interaction patterns. We
anticipate the ScrollPad to support a relatively more
concurrent, collaborative interaction, whereas the
stylus scroller will foster a turn-taking oriented
collaboration where each participant might look at the
device in short individual turns compared to
collectively moving the ScrollPad device around and
make suggestions to each other about where to go next
in the collective interaction with the device.
9. Acknowledgements
We thank Anders Hasselqvist and Björn Yttergren
for their work on electronics construction and firmware
programming.
10. References
[1] Dourish, P. (2001). Where the Action Is: The
Foundations of Embodied Interaction. Cambridge: MIT
Press.
[2] Fitzmaurice, G. W. (1993). Situated information spaces
and spatially aware palmtop computers. Communications of
the ACM, 36(7), 39-49.
[3] Fitzmaurice, G. W., Zhai, S., & Chignell, M. H. (1993).
Virtual Reality for Palmtop Computers. ACM Transactions
on Information Systems, 11(3), 197-218.
[4] Luff, P., & Heath, C. (1998). Mobility in Collaboration,
Proceedings of ACM CSCW'98 Conference on Computer-
Supported Cooperative Work (pp. 305-314).
[5] Lund, A. (2003) Massification of the Intangible: An
investigation into embodied meaning and information
visualization, Phd-thesis, Department of Informatics, Umea
University, Sweden.
[6] Schilit, B., Trevor, J., Hilbert, D., & Koh, T. (2002) Web
interaction using very small internet devices, IEEE Computer,
October 2002, p. 37-45.
[7] Yee, K.-P. (2003). Peephole Displays. Handheld
Computers as Virtual Windows, Proceedings of ACM CHI
2003 Conference on Human Factors in Computing.