ArticlePDF Available

More than dotting the i's - Foundations for crossing-based interfaces

Authors:

Abstract and Figures

Today's graphical interactive systems largely depend upon pointing actions, i.e. entering an object and selecting it. In this paper we explore whether an alternate paradigm-- crossing boundaries -- may substitute or complement pointing as another fundamental interaction method. We describe an experiment in which we systematically evaluate two targetpointing tasks and four goal-crossing tasks, which differ by the direction of the movement variability constraint (collinear vs. orthogonal) and by the nature of the action (pointing vs. crossing, discrete vs. continuous). We found that participants ' temporal performance in each of the six tasks was dependent on the index of difficulty formulated in the same way as in Fitts' law, but that the parameters differ by task. We also found that goal crossing completion time was shorter or no longer than pointing performance under the same index of difficulty. These regularities, as well as qualitative characterizations of crossing actions and their application in HCI, lay the foundation for designing crossing-based user interfaces.
Content may be subject to copyright.
More than dotting the i’s — Foundations
for crossing-based interfaces
Johnny Accot Shumin Zhai
IBM Almaden Research Center
650 Harry Road, San Jose CA 95120, USA
{accot, zhai}@almaden.ibm.com
ABSTRACT
Today’s graphical interactive systems largely depend upon
pointing actions, i.e. entering an object and selecting it. In
this paper we explorewhetheran alternate paradigm cross-
ing boundaries may substitute or complement pointing
as another fundamental interaction method. We describe an
experiment in which we systematically evaluate two target-
pointing tasks and four goal-crossing tasks, which differ by
the direction of the movementvariability constraint (collinear
vs. orthogonal) and by the nature of the action (pointing vs.
crossing, discrete vs. continuous). We found that partici-
pants’ temporal performance in each of the six tasks was de-
pendent on the index of difficulty formulated in the same way
as in Fitts’ law, butthat the parameters differ by task. We also
found that goal crossing completion time was shorter or no
longer than pointing performance under the same index of
difficulty. These regularities, as well as qualitative character-
izations of crossing actions and their application in HCI, lay
the foundation for designing crossing-based user interfaces.
Keywords
Graphical user interfaces, interaction techniques, goal cross-
ing, goal passing, pointing, Fitts’ law, widgets, events, input,
input performance
INTRODUCTION
Modern human computer interfaces are based almost exclu-
sively on one type of motor actions pointing. Loosely
speaking, pointing in human computer interaction consists of
moving a cursor into a graphical object with an input device
and clicking a button (Figure 1.a). This simple sequence of
action constitutes the basic interaction scheme for designing
most traditional interactive widgets, such as buttons, menus,
scrollbars, and folders. Attempts have been made to create
alternative interaction techniques (e.g. ray/cone casting [5],
bounding box or target inclusion [8, 15], or gestures), but
pointing undoubtedly remains the most universal interaction
paradigm across diverse application domains and contexts.
However, pointing has a number of inherent drawbacks: for
instance, it may be time-consuming if the object to be pointed
is small; pointing-driven widgets can use a significant screen
real-estate; and some users find double click difficult to han-
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies
are not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
CHI 2002, April 20-25, 2002, Minneapolis, Minnesota, USA.
Copyright 2002 ACM 1-58113-453-3/02/0004...$5.00.
dle because it requiresrapid successive clicking actions with-
out accidentally moving the cursor position. Finding an inter-
action paradigm that improves performance or circumvents
the drawbacks of pointing thus remains a worthy challenge.
In this paper, we explore the possibility of using “crossing"
as such a paradigm.
Crossing events, which signal that the cursor intersected the
boundary of a graphical object (Figure 1.b), are already used
in modern interfaces, e.g. for detecting Enter/Leave eventson
graphical objects. For another example, Ren and Moriya [12]
studied entering and leaving a button as alternative strategies
to clicking, and further pointed out the need of a theoretical
model for studying these strategies. We call the process of
moving a cursor beyond the boundary of a targeted graphical
object a goal-crossing task [1, 2].
(a) Pointing a target (b) Crossing a goal
Figure 1: Two different paradigms for triggering ac-
tions in a graphical user interface
Fitts’ law [3], a quantitative human performance model, has
provided a scientific foundation for studying and designing
pointing-based user interfaces. A similar scientific founda-
tion is needed if crossing is to be considered as another major
paradigm for interaction, particularly if we want to compare
the two paradigms systematically.
As a stepping stone towards devising the steering law, Ac-
cot and Zhai [2] found that goal-crossing indeed follows a
quantitative relationship among movement time, movement
distance, and the constraint of the goal width. In fact, such a
relationship takes the same form as in Fitts’ law. More pre-
cisely, both the time needed to go and click on a target of
width
that lies a distance away (Figure 2.a), and the
time needed to cross a goal of width which lies a distance
away (Figure 2.b) are given by:
Index of difficulty ( )
(1)
where
and are experimentally-determined psychological
constants. The logarithm factor in Equation 1 is called the
index of difficulty
of the pointing or crossing task.
minneapolis, minnesota, usa • 20-25 april 2002 Paper: Smooth Moves
Volume No. 4, Issue No. 1 73
(a) Pointing: variability is al-
lowed in the direction collinear
to movement
(b) Crossing: variability is al-
lowed in the direction orthogo-
nal to movement
Figure 2: Pointing vs. crossing: the experimental
paradigms differ in the direction of the variability al-
lowed in the termination of the movement.
The above model discovered in [2] on goal-crossing consti-
tutes a necessary, but not a sufficient foundation for study-
ing and designing crossing-based interfaces. First, it is not
known if the coefficients in the above equation are identical
for pointing and crossing tasks. If not, using
and
to denote the pointing and crossing performance re-
spectively, how do these coefficients compare? Is it easier
to point a target than cross an equivalent goal? Does the
comparison depend on the index of difficulty? If crossing
provides a superior performance, how would it be used in
practice for the design of graphical interactive systems?
Upon examining the two tasks in Figure 2, we realized that
they vary in two dimensions. One of the differences is point-
ing vs. crossing. The other difference lies in the direction of
task constraint, either orthogonal or collinear to movement.
Because the two directions of constraint could exist in both
two tasks, we need a
factorial comparison.
One of these
combinations is pointing with orthogo-
nal target constraint. Although pointing is typically studied
with collinear constraint, pointing with orthogonal constraint
does exist. For example (Figure 3), as noted by Tognazz-
ini [13], “Apple’s menu bar has, in essence, infinite height,
since the mouse pointer pins once it arrives. The width of
these menu bars, however, is finite and hence become the
main constraint, albeit in the orthogonal direction to cursor
movement when the cursor is moved upwards to the menu
bars. Tognazzini further remarked that “the worst ‘feature’
of the Windows UI is their having the menu bar on line 2,
instead of at the top of the display. This makes menu bar
acquisition perhaps five to ten times slower than on the Mac-
intosh. A model for pointing with orthogonal constraint,
particularly if made comparable to the traditional Fitts point-
ing task with collinear constraint, would put the comparison
between the menu bars in Macintoshand Windows on a more
scientific ground.
Pragmatically, crossing can be done in two ways: either dis-
cretely or continuously. When there is nothing between the
targeted goals, one can continuously stroke through these
goals (continuous crossing). On the other hand, when there
are non-target objects (distractors) between the individual
goals, one has to land the stylus (or finger) before an intended
goal, cross it, and then lift up (discrete crossing). We there-
fore investigated both discrete and continuous crossing in a
systematic study presented in the next section.
(a) The Apple
R
Macintosh
R
menu bar
(b) The Microsoft
R
Windows
R
task bar
Figure 3: Accessing menu bar and task bar items on
the Macintosh and Windows
EXPERIMENT
Participants
Twelve people, three female and nine male, all right-handed,
participated in the experiment. They ranged in age from 21
to 51.
Apparatus
The experimentwasconductedonanIBMPCrunning Linux,
equipped with a Wacom
R
Intuos
TM
graphics tablet (model
GD-0608-U,
active area, resolu-
tion) and a 19” IBM CRT monitor (model P76,
visual area, resolution). The tablet active area was
mapped onto the display visual area, in absolute mode; the
control gain was close to
. The experiment was done in
full-screen mode, with a black background color. The com-
puter was running in single-user mode, with only a few active
system processes. It was disconnected from the network.
Procedure and design
Participantsperformed six differenttasks,includingtwopoint-
ing tasks and four goal-crossing tasks. The two pointing
tasks differed in the direction of movement variability con-
straint — one collinear and the other orthogonal to the main
movement direction. The four goal-crossing tasks differed
by both the constraint direction (collinear/orthogonal) and
the nature of the action (discrete/continuous). The details
of each condition are as follows:
CP
Pointing with collinear constraint (Figure 4.a): This is
the traditional Fitts’ tapping task [3]. Participants click
alternately on two vertical rectangles with width
and
“infinite” height. The two target centers is separated by
distance
. We call this task collinear pointing because
the variability constraint ( ) imposed on the termina-
tion of the movement is in the direction collinear to the
hand movement.
OP
Pointing with orthogonal constraint (Figure 4.b): This
is a variant of Fitts’ original tapping task. Participants
click alternately on two horizontal rectangles of height
and “infinite” width (by one side), separated by dis-
tance
measured by the gap between the two targets.
The variability constraint imposed on this task is in the
direction orthogonal to the movement. To our knowl-
edge, no performance model has been previously estab-
lished for this task (or for the next three tasks).
Paper: Smooth Moves
CHI
changing the world, changing ourselves
74 Volume No. 4, Issue No. 1
D/CC
Discrete collinear crossing (Figure 4.c): Participants
alternately cross, by a stroke, two horizontal goals of
width
and distance . For consistency, they are
asked to perform the stroke downward for both goals.
This crossing task is discrete since the stylus tip touches
the tablet surface only when crossing the goal; the rest
of the time the stylus is lifted from the tablet surface.
An obstacle line, causing a beep when stroked through,
was drawn between the two goals to remind the partic-
ipants to use discrete strokes for crossing the goals.
D/OC
Discrete orthogonal crossing (Figure 4.d): Partici-
pants alternately stroke through two vertical goals with
height
and distance . They are asked to cross the
goals from left to right for consistency. As in the previ-
ous condition, an obstacle, i.e. a distractor, was drawn
between the two goals to remind participants to lift up
the stylus when traveling from one goal to the other.
C/CC
Continuous collinear crossing (Figure 4.e): Partici-
pants alternately move the cursor through two horizon-
tal goals of width
over distance . The crossing
task is called continuous as participants have to con-
stantly slide the stylus tip on the tablet surface. If the
stylus is lifted during a block of trials, the system will
beep until stylus-tablet contact is resumed.
C/OC
Continuous orthogonal crossing (Figure 4.f): Partici-
pants move the cursor reciprocally through two vertical
goals of height
over distance . Same as in con-
dition
C/CC
, the system will beep when the stylus is
lifted during a block of trials until stylus-tablet contact
is resumed. This task was introduced in [2] and found
to follow Fitts’ law when performed non-reciprocally.
In all six tasks, participants were asked to perform as fast
and as accurately as possible. When a target (or goal) was
missed, a beep was played to remind the participant to im-
prove accuracy. In case of a miss, participants continued the
trial until they hit the target. When hit, the target changed
color from green to orange. The time at which participants
successfully clicked on the target or crossed the goal was
recorded.
A within-subject full factorial design with repeated measures
wasused. The independent variables were the task type
CP
,
OP
,
D/CC
,
D/OC
,
C/CC
,
C/OC
, the distance be-
tween targets or goals
and the tar-
get/goal width
. For each
task, participants performed three consecutive sets of 10
-
combinations, the first set being a practice session and the
later two, data collection sessions. The ten
- combina-
tions were presented in a random order within each session.
With each
- combination, participants performed a block
of 9 trials. The order of testing of the six different tasks was
balanced between six groups of participants according to a
Latin square.
Results and Analyses
Learning, Time and Error
Figure 5 shows the average trial
time over the three experimental sessions. The average trial
completion time in the practice session was longer than the
(a)
CP
Pointing with
collinear variability constraint
(b)
OP
— Pointing with orthog-
onal variability constraint
(c)
D/CC
Discrete collinear
goal-crossing
(d)
D/OC
Discrete orthogo-
nal goal-crossing task
(e)
C/CC
Continuous
collinear goal-crossing
(f)
C/OC
Continuous orthog-
onal goal-crossing
Figure 4: The six tested conditions. All tasks were
reciprocal.
time in the two data-collection sessions, due to participants’
inexperience and occasional experimentation with the tablet-
stylus device and task strategy. The performance difference
between the two data-collection sessionswasrelativelysmall,
and hence both were used in the following data analyses.
Variance analysis showed that mean trial completion times
were significantly different across the six tasks
. Task
C/CC
was the slowest (see Figure 5).
Tasks
OP
and
C/OC
were the fastest, at least 10% faster than
other tasks. The rest of the tasks, including the traditional
Fitts’ tapping task, fall in the middle range of performance.
1200
11001000
900
800
Practice
First block
Second block
CP
OP
D/CC
D/OC
C/CC
C/OC
Time ( )
Figure 5: Learning effect on average completion time
As illustrated in Figure 6.a, movement distance significantly
changed mean trial completion
.
Across all tasks, the greater the distance between targets, the
longer time duration of the trial. As illustrated in Figure 6.b,
target/goal width also significantly changed mean trial com-
pletion
. For all tasks, the greater
the width of the target, the shorter the duration of the trial.
minneapolis, minnesota, usa • 20-25 april 2002 Paper: Smooth Moves
Volume No. 4, Issue No. 1 75
(a)
600 700 800 900 1100
500
256
1024
CP
OP
D/CC
D/OC
C/CC
C/OC
Distance between targets/goals (pixels)
Time ( )
(b)
400 600 800 1400
1000
8 16 32 64 128
CP
OP
D/CC
D/OC
C/CC
C/OC
Target/goal width (pixels)
Time ( )
Figure 6: Effect of distance and width on task com-
pletion time
The error rate, measured by the percentage of trials that took
more than one click or crossing to hit the target, varied signif-
icantly with task
, target distance
and target width
. As expected, smaller and more distant targets
tended to cause more errors. As shown in Figure 7, except for
Task
C/CC
, all new tasks studied in this experiment
had error rates close to and lower than that of Fitts’ tapping
task
CP
, .
D/OC
has the lowest error rate with .
8
6
4
2
0
CP OP
D/CC
D/OC C/CC
C/OC
Average number of errors (%)
Figure 7: Error rates for each task
When participants made an error in a trial (missing the tar-
get), they were asked to continue the trial until they hit the
target, with multiple clicks or multiple crossing attempts.
The completion time of these trials does not reflect the same
perceptual motor mechanism as successful trials without er-
ror; hence we did not include them in the time analysis in
this section. As an extra caution for the robustness of the
conclusions, we also repeated all time-related analyses with
the error trial completion times included, but found no impor-
tant or qualitative differences from the conclusions reported
in this section.
Lawful Regularities
Most interestingly, the movement time
in each and every of the six tasks could be largely accounted
for by the target-distance to target-width ratio. More pre-
cisely, the difficulty in each task can be qualified by the fol-
lowing common index:
(2)
and the movement time can be determined by:
(3)
where a and b are empirically determined in each task. Specif-
ically, linear regression of the experimental data resulted in
the following equations (all time values in this paper are in
milliseconds):
CP
: (4)
OP
: (5)
D/CC
: (6)
D/OC
: (7)
C/CC
: (8)
C/OC
: (9)
In other words, there was a lawful regularity between move-
ment time and
ratio in each of the six tasks. Further-
more, all six laws take the same form of logarithmic transfor-
mation of
ratio as in Fitts’ law, with very high fitness
values (
ranging from to ).
Figure 8 displays the regression lines of completion time as
a function of
. As we can see, despite the very different
constraints and varying action patterns across the six tasks,
the laws of these tasks all fall into a band that is even nar-
rower than the range of data based on the same Fitts’ tapping
task reported by different authors in the literature (see [9]).
1500
1000500
0
2
3
4
56
7
CP
OP
D/CC
D/OC
C/CC
C/OC
Time ( )
(bits)
Figure 8: The fit to the model for the six studied tasks
Paper: Smooth Moves
CHI
changing the world, changing ourselves
76 Volume No. 4, Issue No. 1
Task Comparison
Although relatively small, there were im-
portant differences between the six tasks studied in this ex-
periment. Due to space limitations, we report only the anal-
yses of a few comparisons most relevant to human-computer
interaction tasks (illustrated by Figure 9).
First, Task
OP
, i.e. pointing with orthogonal constraint, had
similar performance to Task
CP
, i.e. pointing with constraint
collinear to the movementdirection (the traditional Fitts’ tap-
ping task). On average,
OP
was 10% faster than
CP
(Fig-
ure 5). As displayed separately in Figure 9.a, starting from
essentially zero, the difference between
OP
and
CP
increased
with Index of Difficulty. It appeared that variability con-
straints orthogonal to the pointing movementdirection is “eas-
ier” to deal with than constraints in the same direction as the
movement. This is plausible because the former can be dealt
with in the entire course of movement, while the latter can
only be controlled at the very end of the movement. The
greater the
(either smaller width or greater distance), the
more pronounced this effect (see
CP
and
OP
lines in Fig-
ure 6.a).
Second, the two discrete crossing tasks, with collinear and
orthogonal constraint respectively, followed similar regres-
sion lines to the standard Fitts’ tapping task, as separately
shown in Figure 9.b, suggesting that it is possible to substi-
tute pointing tasks with crossing tasks with essentially the
same time but lower error rate (see Figure 7). Between the
two discrete crossing tasks, there was a trade-off swung by
.
D/OC
tended to be faster than
D/CC
when was
greater than
, and the reverse was true when was
less than
. The fact that
D/OC
was slower than
D/CC
in low can be partially explained by the “obstacle” line
positioned between the two goals (see Figures 4.c and 4.d).
As the distance between the two goals reduces, the constraint
of landing the stylus between the obstacle and the right goal
increases. Implicitly there is a traditional Fitts’ tapping task
(
CP
) involved here with increasing difficulty. This is not true
to the
D/CC
task because the stylus always lands above the
horizontal goals. This effect is partly an experimental ma-
nipulation artifact, and partly a reflection of realistic situa-
tions in computer interfaces where the target object can be
surrounded by objects that are not targets at the moment.
Another large difference between the two discrete crossing
tasks was that much smaller number of errors were made
with the
D/OC
task than with the
D/CC
task.
Third, there was a trade-off between
C/OC
and
D/OC
tasks
(Figure 9.c).
C/OC
was much faster than
D/OC
when was
much lower than , and
C/OC
was longer than
D/OC
when was greater than . This again was partly due
to the obstacle effect in the
D/OC
task. There was no obstacle
in the
C/OC
task. When the two goals were very close to each
other, it waspossible to cross the two in one stroke. This is an
advantage that should be taken in interface design whenever
possible.
Fourth, the
C/CC
task was uniformly slower than the
C/OC
task (Figure 9.d). It also has the highest error rate (Figure 7).
This suggests that, wheneverpossible, the goals to be crossed
should be positioned orthogonal to the movement direction.
(a)
15001000500
0
2
3
4
56
7
CP
OP
(bits)
Time ( )
(b)
15001000500
0
2
3
4
56
7
CP
D/CC
D/OC
(bits)
Time ( )
(c)
15001000500
0
2
3
4
56
7
OP
D/OC
C/OC
(bits)
Time ( )
(d)
15001000500
0
2
3
4
56
7
C/CC
C/OC
(bits)
Time ( )
Figure 9: Tasks comparisons
minneapolis, minnesota, usa • 20-25 april 2002 Paper: Smooth Moves
Volume No. 4, Issue No. 1 77
Note on the
OP
task
The
OP
task can be parameterized in two ways. The one we
used, which defines the movement amplitude as the distance
between the two edges of the horizontal targets (Figure 4.b),
has the advantage of being closer to the orthogonal goal-
crossing tasks. But it also has a drawback: when perform-
ing the task, the actual movement amplitude is a little higher
than the one controlled in the experiment, possibly making
the pointing task more difficult than predicted. A way to
cope with this issue is to introduce the concept of effective
amplitude, similar to the idea of effective width (see [9]),
and use the average actual movement amplitude to compute
the difficulty of the task. The drawback of this method is that
the difficulty is computed a posteriori, which goes against
the idea of a priori performance modeling and prediction. A
second way is to change the definition of distance between
targets. A definition that appears particularly suited is to add
half a target width on both sides of the movement amplitude
in Figure 4.b, as illustrated by Figure 10. This new definition
has an increased compatibility with the
CP
task (Figure 4.a):
when
is equal to , the two targets are adjacent and the
task consists in merely jumping over a single line from one
target to the other; when
is equal to , the two targets
are partially overlapping, and the task can be performed by
clicking repeatedly without even moving. This definition of
the
OP
task may then be more adequate for comparing per-
formance within the pointing task class. With this definition
of movement amplitude, the linear regression between time
and index of difficulty is:
OP
: (10)
The fitness is even greater than the one of Equation 5, sug-
gesting that the variant definition may be a more appropriate
model. If we use this model, conclusions regarding
OP
task
in relation to others in this paper are qualitatively the same
but quantitatively stronger.
Figure 10: A variant of the
OP
task, with a different
measure of movement amplitude
APPLICATIONS AND DISCUSSION
As with other laws of natural phenomena, the regularities of
human movement revealed in this study can be applied in
ways and scope beyond expectations at the time of discovery.
In this section we explore only a few obvious implications
of these movement laws to the design of human computer
interfaces.
The
OP
task and the primacy of border locations
As we found in the experiment, an
OP
task takes the same
or less time than a
CP
task for the same amount of constraint
(Figure 4.a, 4.b, Figure 9). This can be very informative
to interface designers. When choosing the orientations and
locations of interaction objects (widgets), they can use this
empirical relationship to estimate the impact of their choices.
Consider for instance the two pointing tasks shown in Fig-
ure 3 and discussed in the Introduction section. For the Mac-
intosh menu bar (Figure 3.a), the menus are virtually in-
finitely extended beyond the edge of the screen where the
mouse cursor is stopped. The primary pointing variability
constraint being orthogonal to the movementdirection, point-
ing at these menus is an
OP
task, modeled by Fitts’ law with
OP
and
OP
(Equations 1 and 10). On the
other hand, since the buttons of the Windows task bar are not
pushed onto the very edge of the screen, the pointing task
has both orthogonal and collinear constraints (see Figure 3).
The collinear variability allowed here is much smaller than
the orthogonal one, such that pointing a button is mainly a
CP
task (see also [6, 10, 11, 14]) and modeled by Fitts’ law
with
CP
and
CP
(Equations 1 and 4). Now let
us assume we want to select an item of width
and
height
1
. Then, for a movement distance of , the
time to select a Macintosh menu item is:
(11)
while the time to select a Windows task button is:
(12)
These two time models, plotted in Figure 11, clearly show
the advantage of the Macintosh menu bar over the Windows
task bar, the former being about twice as fast as the latter for
most amplitudes.
1000
1000
800
800
600
600
400
400
200
200
0
0
Macintosh menu bar (Equation 11)
Windows task bar (Equation 12)
Distance
Time
Figure 11: Comparison of the access time for the
Macintosh menu bar and the Windows task bar
This simple exercise shows the primacy of the border loca-
tions of the computer screen real estate. Widgets on the bor-
der locations should take the maximum advantage of extend-
ing beyond where the cursor stops. To take another example,
scrollbars on the very edge of screen should also extend be-
yond the screen pixels.
Note that fundamentally all 2D widgets are constrained in
two dimension and pointing at these targets is ultimately “bi-
variate”. The above analysis focuses on only the primary
1
As a reference, a standard task button in Microsoft
R
Windows NT
R
has a width of and a height of .
Paper: Smooth Moves
CHI
changing the world, changing ourselves
78 Volume No. 4, Issue No. 1
constraint, which is valid only when the objects are elon-
gated. For objects that are close square, more empirical work
is needed (see [10] for a previous investigation).
Crossing Interfaces
Perhaps the most important HCI implications of the current
study is a paradigm of interaction based on goal crossing in-
terfaces where the basic interactive widget is a goal (1D bar)
instead of target (2D area). Just as Fitts’ law in the Fitts’ tap-
ping task has served as a foundation for analyzing pointing
based interface, the laws of goal-crossing tasks can be the
basis for designing and analyzing interfaces based on goal
crossing (Figure 12.a).
(a) To trigger an action: on
the left we push the button;
on the right we cross the
goal.
(b) Unlike a traditional check
box, agoal can “store” two visual
states depending on the crossing
direction.
Figure 12: Using goal-crossing in graphical widgets
Quantitatively, as the results of the experiment show, there
are situations where user performance in goal-crossing tasks
is superior to that of target-pointing tasks. Depending on the
idiosyncrasies of the particular cases, user interface designs
can be grounded on the relative pros and cons of different
types of pointing or crossing tasks. In some cases pointing
is preferred; in other cases, a particular type of crossing (dis-
crete or continuous) is preferred.
Qualitatively, crossing a continuousactionoccurring more
often in the natural world than discrete pointing and click-
ing affords some unique characteristics. First, crossing
can be bi-directional (Figure 12.b). This means that richer
semantics can be assigned to crossing than pointing. Fur-
ther, it is also possible to cross a goal back and forth once
(double crossing) or even more than once to represent a va-
riety of commands. Indeed, in some sense crossing is a step
from pointing towards free gesturing, which is even richer
but harder to quantify and disambiguate than crossing.
Another characteristic of crossing is that multiple goals can
be crossed in a cascaded fashion. Such an application can
in fact be found in existing software products. For example,
in Lotus Notes
R
, multiple messages can be selected by one
continuous cursor drag, crossing a series of implicit goals
(Figure 13). To select many messages that are not consecu-
tive, one can first select a whole set of continuous messages,
and then deselect the ones that are not wanted. Such mixed
pointing and crossing performance can be quantified by a
combination of the laws discussed in the previous section. In
comparison, using pointing alone to select multiple messages
would require very precise and repetitive pointing actions. It
is possible to use a bounding box technique to select multiple
targets and then use pointing to activate all of them, but this
is less fluid than one continuous crossing stroke.
Figure 13: Selection of multiple messages by a con-
tinuous goal-crossing action in Lotus Notes
While it is possible to have both orthogonal and collinear
crossing, our experiment clearly showed the advantage of or-
thogonal crossing. Figure 14 illustrates the possibility of dy-
namically orienting the goals towards the cursor to maximize
this advantage (a “sunflower” interface, so to speak).
(a) As the cursor moves, ... (b) ... the goal rotates to offer max-
imum width
Figure 14: Dynamically orienting goals may mini-
mize crossing time for any cursor position
It should be noted that pointing and crossing do not have to
be mutually exclusive. It is in fact possible to have a “dou-
ble representation” interface where both actions are enabled.
To select individual objects
, ,or in Figure 15, the user
can either point and click on the circles or cross the links,
depending on subjective preference. To select a group of ob-
jects, the user may cross multiple links in one stroke, taking
advantage of the cascading property of crossing.
(a) Pointing at targets (b) Crossing the arcs
Figure 15: Input polymorphism: choosing the input
method that is best suited to the context
Special application domains of crossing interfaces
While goal-crossing based interaction techniques can be used
in any graphical user interface, there are several applications
to which they are particularly suited.
One follows the idea of double representation for univer-
sal accessibility of interfaces. Some users have gotten used
to pointing and will continue their habitual mode of action.
Others, including some elderly users and users with certain
motor disability may have difficulty with clicking without
moving the cursor position, or worse, double clicking. For
these users crossing may become the main interaction mode.
The second domain of special application is interfaces for
mobile or handheld devices. These devices usually come
minneapolis, minnesota, usa • 20-25 april 2002 Paper: Smooth Moves
Volume No. 4, Issue No. 1 79
with a stylus (pen), which is more suited for crossing ac-
tions than a mouse. Furthermore, double-click with a stylus
is often very difficult for any user. The more compact rep-
resentation of bars or links in a goal crossing interface also
requires less of the small screens in these devices.
Third, the text of hyperlinks in web pages are especially dif-
ficult for pointing due to the their narrow height (one line of
text) but easy for crossing due to the much greater width of
(one word long and often much longer). It is also possible to
cross a list of links in a cascade (Figure 16). This in fact has
been demonstrated in the “Elastic window” prototype [7].
Figure 16: Crossing multiple city names to get their
detailed weather forecast
Finally, traditional GUI widgets are very difficult to be in-
tegrated in virtual reality type of 3D interfaces, partly be-
cause point and click is necessarily dependent on a solid 2D
surface. In contrast, 1D goals (bars) can be easily crossed
(“chopped”) without having to be on a 2D surface. Similarly,
actions may be triggered when crossing a 2D surface, like a
door or portal (e.g. [4]).
CONCLUSION
In this paper, we studied human performance in both point-
ing and crossing tasks, with systematic variations in terms
of task difficulty, the direction of movement constraint, and
the nature of the task (discrete vs. continuous). We found a
robust regularity among trial completion time, distance be-
tween targets, and the target width in all six tasks studied. If
we refer to natural laws by their mathematical appearance,
we can say all of the six tasks follow Fitts’ law. If we refer
to natural laws by the task (or phenomenon) they describe,
we can say there is a law to each of the six tasks. Many
subjective impressions, such as the superior usability of the
menus on the screen border, can be in fact quantified by some
of these laws. Most significantly, together with the qualita-
tive characterizations of the crossing action, these laws lay
the foundation for designing crossing-based user interfaces
in which crossing, instead of or in addition to pointing, is the
fundamental interaction technique.
ACKNOWLEDGMENTS
We would like to thank Barton A. Smith, Paul Mauglio and
Teenie Matlock for their helpful comments on this work.
REFERENCES
1. J. Accot. Les Tâches Trajectorielles en Interaction
Homme-Machine Cas des tâches de navigation. PhD
thesis, Université de Toulouse 1, France, January 2001.
2. J. Accot and S. Zhai. Beyond Fitts’ law: models for
trajectory-based HCI tasks. In Proceedings of ACM
CHI’97 Conference on Human Factors in Computing
Systems, pages 295–302, 1997.
3. P. M. Fitts. The information capacity of the human
motor system in controlling the amplitude of move-
ment. Journal of Experimental Psychology, 47:381–
391, 1954.
4. M. Haisenko and P. Musgrave. The 3Dsia project.
http://threedsia.sourceforge.net,2001.
5. K. Hinckley, R. Pausch, J. C. Goble, and N. F. Kassell.
A survey of design issues in spatial input. In Proceed-
ings of the ACM Symposium on UserInterface Software
and Technology, pages 213–222, 1994.
6. E. R. Hoffmann and I. H. Sheikh. Effect of varying
target height in a Fitts’ movement task. Ergonomics,
37(6):1071–1088, 1994.
7. E. Kandogan and B. Shneiderman. Elastic windows: A
hierarchical multi-window world-wide web browser. In
Proceedings of the ACM Symposium on User Interface
Software and Technology, pages 169–177, 1997.
8. A. Leganchuk, S. Zhai, and W. Buxton. Manual and
cognitive benefits of two-handed input: An experimen-
tal study. ACM Transactions on Computer-Human In-
teraction, 5(4):326–359, Dec. 1998.
9. I. S. MacKenzie. Fitts’ law as a research and design
tool in human-computer interaction. Human-Computer
Interaction, 7:91–139, 1992.
10. I. S. MacKenzie and W. Buxton. Extending Fitts’lawto
two-dimensional tasks. In Proceedings of ACM CHI’92
Conference on Human Factors in Computing Systems,
pages 219–226, 1992.
11. A. Murata. Extending effective target width in Fitts’
law to a two-dimensional pointing task. International
journal of human-computer interaction, 11(2):137–
152, 1999.
12. X. Ren and S. Moriya. Improving selection perfor-
mance on pen-based systems: a study of pen-based
interaction for selection tasks. ACM Transactions on
Computer-Human Interaction, 7(3):384–416, 2000.
13. B. Tognazzini. Club Wired — Bruce Tognazzini Tran-
script. http://hotwired.lycos.com/talk/
club/special/transcripts/95-06-14.
tog.html, June 1995.
14. N. Walker and J. B. Smelcer. A comparison of selection
times from walking and pull-down menus. In Proceed-
ings of ACM CHI’90 Conference on Human Factors in
Computing Systems, pages 221–225, 1990.
15. S. Zhai, W. Buxton, and P. Milgram. The “silk cur-
sor”: Investigating transparency for 3D target acquisi-
tion. In Proceedings of ACM CHI’94 Conference on
Human Factors in Computing Systems, pages 459–464,
1994.
Paper: Smooth Moves
CHI
changing the world, changing ourselves
80 Volume No. 4, Issue No. 1
... This result suggested that a technique with smaller MT and ER performs worse, which is intuitively inappropriate. Such a result of low TP for Bubble Cursor comes from the fact that the 1 These plots seem to include only successful clicks. However, Chapuis et al. reported that the s including error clicks were always greater for Bubble Cursor than for the baseline, thus decreasing the TP for Bubble Cursor [15]. ...
... Defining the click-point used for computing TP as the first-touch point or the release point would be possible, but this needs further justifications. This is also true for mouse-based selection using a stroke or movement direction, including Marking Menu [37] and its variations for menu selection [5], techniques using the goal-crossing method [1] such as Enhanced Area Cursors [22], Attribute Gates [47], CrossY [3], and Don't Click, Paint! [8]. ...
Conference Paper
Full-text available
Target-selection tasks have been frequently conducted to evaluate novel input devices and pointing-facilitation techniques. Recently, Sharif et al. showed that a unified metric to evaluate the pointing performance, called throughput TP, was not stable across two sessions performed by the same participant group, which indicates poor test-retest reliability. Because there are cases in which using TP is inappropriate depending on the research topic, we extend their finding to two other metrics: movement time MT and error rate ER. We demonstrated that, even for the participants who kept their TPs across two sessions stable, they would exhibit unstable MT s and ERs. Thus, if time allows, researchers should design their experiments to run multiple sessions for obtaining the central tendency of user performance, which increases the validity of their user studies.
... For instance, ray-based methods were adapted [16] to select the objects that intersect with the ray one-by-one. On the other hand, parallel techniques select objects that are within a defined volume (usually a bounding box) [17][18][19][20][21]. Lucas et al. [22] showed that parallel techniques can be more effective than serial techniques. ...
Article
Full-text available
Some daily tasks involve grasping multiple objects in one hand and releasing them in a determined order, for example laying out a surgical table or distributing items on shelves. For training these tasks in Virtual Reality (VR), there is no technique for allowing users to grasp multiple objects in one hand in a realistic way, and it is not known if such a technique would benefit user experience. Here, we design a multi-object grasp technique that enables users to grasp multiple objects in one hand and release them in a controlled way. We tested an object placement task under three conditions: real life, VR with single-object grasp and VR with multi-object grasp. Task completion time, distance travelled by the hands and subjective experience were measured in three scenarios: sitting in front of a desktop table, standing up in front of shelves and a room-size scenario where walking was required. Results show that the performance in a real environment is better than in Virtual Reality, both for single-object and multi-object grasping. The single-object technique performs better than the multi-object, except for the room scenario, where multi-object leads to less distance travelled and reported physical demand. For use cases where the distances are small (i.e., desktop scenario), single-object grasp is simpler and easier to understand. For larger scenarios, the multi-object grasp technique represents a good option that can be considered by other application designers.
Article
Empirical research on single target selection in Virtual Reality/Augmented Reality (VR/AR) environments has been studied extensively. However, there is a lack of methods and research on improving the coherence when successively selecting multiple targets. We propose Sewing, a successive target selection method with handheld controllers in VR environments. We leverage real-life sewing as a design metaphor for the selection of multiple targets with one single spatial movement penetrating targets with handheld controllers. We conducted an empirical study to validate the efficiency of selecting multiple targets with Sewing compared with the conventional selection method, which is based on the button pressing metaphor. Results showed that Sewing can provide promising performance and coherence for rapid and accurate target selection. Based on the results, we implemented SewTyping, a Sewing-based VR text entry method to evaluate the applicability of Sewing, and conducted an empirical study to compare the typing performance with two other techniques (Controller Pointing and Drum-like Keyboard). Results showed that 1) SewTyping achieved 26.57 words per minute with a total error rate of 3.68% and 2) users adapted to the sewing-like movement easily and achieved steady performance with essential practice.
Article
Virtual reality (VR) user interfaces contain numerous dynamic interactive tasks, among which acquiring moving targets is a common basic one. Previous studies have investigated user performance in moving target acquisition in desktop and touchscreen settings. However, these findings are not directly transferable to VR where targets and user input have complete freedom in three dimensions. This paper concentrates on motion-in-depth, that is, where a target predominantly exhibits approaching or receding movement as opposed to lateral motion across the user’s field of view. We report on two studies investigating how various factors including texture, shadow, alignment, moving speed and moving direction affect: 1) perception accuracy of 3D targets with motion-in-depth, and 2) user performance, which we define as the combination of movement time (MT) and error rate (ER), in a target acquisition task involving motion-in-depth. Our data reveal a number of empirical results that are distinct from the depth perception of static targets and the user performance of 1D/2D target acquisition. We found that MT and ER when acquiring targets with motion-in-depth have strong regularities as the data showed good fits with Jagacinski’s model for movement time estimation and a Ternary-Gaussian model for error rate prediction. We conclude with implications derived from this study for future designs.
Conference Paper
Full-text available
Touch point distribution models are important tools for designing touchscreen interfaces. In this paper, we investigate how the finger movement direction affects the touch point distribution, and how to account for it in modeling. We propose the Rotational Dual Gaussian model, a refinement and generalization of the Dual Gaussian model, to account for the finger movement direction in predicting touch point distribution. In this model, the major axis of the prediction ellipse of the touch point distribution is along the finger movement direction, and the minor axis is perpendicular to the finger movement direction. We also propose using projected target width and height, in lieu of nominal target width and height to model touch point distribution. Evaluation on three empirical datasets shows that the new model reflects the observation that the touch point distribution is elongated along the finger movement direction, and outperforms the original Dual Gaussian Model in all prediction tests. Compared with the original Dual Gaussian model, the Rotational Dual Gaussian model reduces the RMSE of touch error rate prediction from 8.49% to 4.95%, and more accurately predicts the touch point distribution in target acquisition. Using the Rotational Dual Gaussian model can also improve the soft keyboard decoding accuracy on smartwatches.
Chapter
Full-text available
Soft skills training is considered important for employees to be successful at work. Several companies are offering immersive virtual soft skills training with head-mounted displays. The main contribution of this paper is to provide an overview of the research literature within the field of using immersive virtual soft skills learning and training of employees. The results of this preliminary scoping review show that there is a lack of research literature and empirical studies within this topic.
Book
Full-text available
The 7th International Conference of the Immersive Learning Research Network (iLRN 2021) is an innovative and interactive virtual gathering for a strengthening global network of researchers and practitioners collaborating to develop the scientific, technical, and applied potential of immersive learning. It is the premier scholarly event focusing on advances in the use of virtual reality (VR), augmented reality (AR), mixed reality (MR), and other extended reality (XR) technologies to support learners across the full span of learning—from K-12 through higher-education to workbased, informal, and lifelong learning contexts. Following the success of iLRN 2020, our first fully online and in-VR conference, this year’s conference was once again based on the iLRN Virtual Campus, powered by ©Virbela, but with a range of activities taking place on various other XR simulation, gaming, and other platforms. Scholars and professionals working from informal and formal education settings as well as those representing diverse industry sectors are invited to participate in the conference, where they may share their research findings, experiences, and insights; network and establish partnerships to envision and shape the future of XR and immersive technologies for learning; and contribute to the emerging scholarly knowledge base on how these technologies can be used to create experiences that educate, engage, and excite learners.
Article
Full-text available
Experiments are reported of the effect on movement times of varying target height in a Fitts' type of experiment. Data show that there are three well defined regions of control. The first of these, when the index of difficulty based on the target widdi in the direction of motion is larger than that based on the vertical height of the target, is dominated by the constraint in the direction of motion. The second is a transitional region where both vertical and horizontal constraints are effective. The third region is one where the vertical constraint dominates the times for the movement. Fitts' law is found to be applicable in the first and third regions. The same pattern of movement times, as a function of die two IDs, is obtained for discrete and reciprocal movements. As well, die time on target for die reciprocal movements demonstrates this same pattern.
Conference Paper
Full-text available
As a developing discipline, research results in the field of human computer interaction (HCI) tends to be "soft". Many workers in the field have argued that the advancement of HCI lies in "hardening" the field with quantitative and robust models. In reality, few theoretical, quantitative tools are available in user interface research and development. A rare exception to this is Fitts' law. Extending information theory to human perceptual-motor system, Paul Fitts (1954) found a logarithmic relationship that models speed accuracy tradeoffs in aimed movements. A great number of studies have verified and / or applied Fitts' law to HCI problems, such as pointing performance on a screen, making Fitts' law one of the most intensively studied topic in the HCI literature.
Article
Full-text available
One of the recent trends in computer input is to utilize users' natural bimanual motor skills. This article further explores the potential benefits of such two-handed input. We have observed that bimanual manipulation may bring two types of advantages to human-computer interaction: manual and cognitive. Manual benefits come from increased time-motion efficiency, due to the twice as many degrees of freedom simultaneously available to the user. Cognitive benefits arise as a result of reducing the load of mentally composing and visualizing the task at an unnaturally low level which is imposed by traditional unimanual techniques. Area sweeping was selected as our experimental task. It is representative of what one encounters, for example, when sweeping out the bounding box surrounding a set of objects in a graphics program. Such tasks cannot be modeled by Fitts' Law alone and have not been previously studied in the literature. In our experiments, two bimanual techniques were compared with the conventional one-handed GUI approach. Both bimanual techniques employed the two-handed “stretchy” technique first demonstrated by Krueger in 1983. We also incorporated the “Toolglass” technique introduced by Bier et al. in 1993. Overall, the bimanual techniques resulted in significantly faster performance than the status quo one-handed technique, and these benefits increased with the difficulty of mentally visualizing the task, supporting our bimanual cognitive advantage hypothesis. There was no significant difference between the two bimanual techniques. This study makes two types of contributions to the literature. First, practically we studied yet another class of transaction where significant benefits can be realized by applying bimanual techniques. Furthermore, we have done so using easily available commercial hardware in the context to our understanding of why bimanual interaction techniques have an advantage over unimanual techniques. A literature review on two-handed computer input and some of the relevant bimanual human mototr control studies is also included.
Article
Full-text available
Two experiments were conducted to compare pen-based selection strategies and their charac-teristics. Two state transition models were also formulated which provide new vocabulary that will help in investigating interactions related to target selection issues. Six strategies, which can be described by the state transition models, were used in the experiments. We determined the best strategy of the six to be the "Slide Touch" strategy, where the target is selected at the moment the pen-tip touches the target for the first time after landing on the screen surface. The six strategies were also classified into strategy groups according to their characteristics. We determined the best strategy group to be the "In-Out" strategy group, where the target is selected by contact either inside or outside the target. Analyses show that differences between strategies are influenced by variations in target size; however, the differences between strategies are not affected by the distance to the target (i.e., pen-movement-distance) or the direction of pen movement (i.e., pen-movement-direction). We also found "the smallest maximum size" of five pixels, i.e., the boundary value for the target size below which there are significant differences, and above which there are no significant differences between the strategies in error rate. Relationships between interaction states, routes, and strategy efficiency were also investigated.
Conference Paper
This paper reports on an experiment that investigated factors which effect selection time from walking menus and bar or pull-down menus. The primary focus was on the use of impenetrable borders and on expanding target areas on the two menus types. The results show that both factors can be used to facilitate menu selection, with the use of borders being most beneficial. In addition, the results suggest that even on large monitors, the time required to access items from a bar menu is less than that required for the best walking menu.
Conference Paper
Fitts' law, a one-dimensional model of human movement, is commonly applied to two-dimensional target acquisition tasks on interactive computing systems. For rectangular targets, such as words, it is demonstrated that the model can break down and yield unrealistically low (even negative!) ratings for a task's index of difficulty (ID). The Shannon formulation is shown to partially correct this problem, since ID is always ≥ 0 bits. As well, two alternative interpretations “target width” are introduced that accommodate the two-dimensional nature of tasks. Results of an experiment are presented that show a significant improvement in the model's performance using the suggested changes.
Conference Paper
We present a survey of design issues for developing effective free-space three-dimensional (3D) user interfaces. Our survey is based upon previous work in 3D interaction, our experience in developing free-space interfaces, and our informal observations of test users. We illustrate our design issues using examples drawn from instances of 3D interfaces. For example, our first issue suggests that users have difficulty understanding three-dimensional space. We offer a set of strategies which may help users to better perceive a 3D virtual environment, including the use of spatial references, relative gesture, two-handed interaction, multisensory feedback, physical constraints, and head tracking. We describe interfaces which employ these strategies. Our major contribution is the synthesis of many scattered results, observations, and examples into a common framework. This framework should serve as a guide to researchers or systems builders who may not be familiar with design issues in spatial input. Where appropriate, we also try to identify areas in free-space D interaction which we see as likely candidates for additional research.
Article
This research was designed to develop the definition of effective target width in a 2-dimensional pointing task. The idea of effective target width in a 1-dimensional pointing task was extended to effective target width in a 2-dimensional pointing task using the 2-dimensional joint probability density function. The validity of this theoretical definition was empirically verified. In the experiment, the moving direction of the mouse was from lower left to upper right. The approach angle was fixed to 45°. The fit of conventional and new models to the experimental data was compared by means of contribution of the regression line that showed the relation between the index of difficulty and the mean pointing time. As a result, we could obtain higher values of contribution for the modeling that introduced effective target width than that for the conventional modeling without it. In conclusion, the proposed 2-dimensional definition of effective target width may be promising for predicting and modeling pointing time in human-computer interactions.