Enhancing physicality in touch interaction with programmable friction.

Full text

Page 1

Enhancing Physicality in Touch Interaction
with Programmable Friction
Vincent Lévesque1, Louise Oram1, Karon MacLean1, Andy Cockburn2,
Nicholas D. Marchuk3, Dan Johnson3, J. Edward Colgate3, Michael A. Peshkin3
1 University of British Columbia
Vancouver, BC Canada
{vlev,olouise,maclean}@cs.ubc.ca andy@cosc.canterbury.ac.nz
2 University of Canterbury
Christchurch, New Zealand
3 Northwestern University
Evanston, IL USA
{colgate,peshkin}@northwestern.edu

ABSTRACT
Touch interactions have refreshed some of the ‘glowing
enthusiasm’ of thirty years ago for direct manipulation
interfaces. However, today’s touch technologies, whose
interactions are supported by graphics, sounds or crude
clicks, have a tactile sameness and gaps in usability. We use
a Large Area Tactile Pattern Display (LATPaD) to examine
design possibilities and outcomes when touch interactions
are enhanced with variable surface friction. In a series of
four studies, we first confirm that variable friction gives
significant performance advantages in low-level targeting
activities. We then explore the design space of variable
friction interface controls and assess user reactions. Most
importantly, we demonstrate that variable friction can have
a positive impact on the enjoyment, engagement and sense
of realism experienced by users of touch interfaces.
Author Keywords
Haptics, tactile feedback, touch screen.
ACM Classification Keywords
H5.2. [User Interfaces]: Interaction Styles, Haptic I/O.
General Terms
Design, Experimentation, Human Factors, Performance.
INTRODUCTION
As recognition of the impact of emotion on design grows
[19], designers seek natural, realistic and organic [26]
means of interaction. In 1983, Shneiderman [29] observed
the ‘glowing enthusiasm’ resulting from graphical user
interfaces that allowed users to directly manipulate objects.
The iPhone’s success suggests a similar role of engagement
and delight, presumably through the directness and realism
of touch interfaces. This is driving renewed research
interest in interaction metaphors using touch (e.g. [4,12]).
However, touch interactions with most current devices are
‘flat’ – all interface objects still feel like the same plastic or
glass, so any physical realism must be communicated
through visual and auditory illusions. Tactile effects are
generally limited to clicks and buzzes that can convey a
great deal of information [16] but lack realism.
This paper examines design possibilities and outcomes
when touch interactions are enhanced with variable surface
friction. We use a Large Area Tactile Pattern Display
(LATPaD) [17,32] which creates a friction-reducing
‘squeeze film’ of air on a touch sensitive display’s surface
through imperceptible high-frequency vibrations (Figure 1).
The prototype consists of a glass plate with bonded
piezoelectric actuators atop an LCD screen. A 57x76 mm
touchscreen is created with laser-based measurement of
finger position. Friction effects are produced by varying
vibration amplitude in response to finger movements.
(a)

(b)
Figure 1. (a) Picture and (b) illustration of components of the
Large Area Tactile Pattern Display (LATPaD).
The ability to vary friction raises interaction possibilities
which are interesting from both performance and emotional
standpoints. Activities such as pointing and dragging may
become more efficient: high friction objects might ‘grab’
the finger, reducing overshoot and errors, while low friction
surfaces should ease sliding movements and reduce finger
judder. Emotionally, variable friction may increase
perceived realism and subjective satisfaction. In this paper,
we report experiments measuring both effects. The specific
contributions of our work are as follows:
1. Performance data showing that variable friction can
improve performance in touch interactions;
2. Qualitative and quantitative evidence that friction
enhanced widgets can positively impact users’
emotional response to touch interactions;
3. An exploration of friction-enhanced interface design.

Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise,
or republish, to post on servers or to redistribute to lists, requires prior
specific permission and/or a fee.
CHI 2011, May 7–12, 2011, Vancouver, BC, Canada.
Copyright 2011 ACM 978-1-4503-0267-8/11/05....$10.00.

Page 2

After providing background, we describe two studies (S1-2)
examining the impact of variable friction on target selection
performance, without and with surrounding distracters; S3
characterizes the effect of friction on biomechanical
control. We then present design concepts exploiting
variable friction and examine users’ subjective responses
(S4). Finally, we discuss our findings and conclude.
BACKGROUND
Issues grounding our approach range from past work to our
own hypotheses on the value of variable friction.
Touch Interaction Without Tactile Feedback
Buxton et al.’s 1985 analysis of touchscreen limitations still
holds: signaling while pointing requires pressure and virtual
widgets need haptics [5]. Recent systems have exploited
non-feedback touch to address issues like ‘fat finger’
occlusion, accuracy and the need to feel edges. Roudaut et
al. identified these concerns for target acquisition on small
touchscreens and proposed zoom techniques for thumb-
based selection [27]. Physical metaphors have inspired new,
more fluid gestures [25]. Others have elaborated strategies
for particular control actions: Potter et al. attributed high
accuracy of ‘lift-off’ selection to its continuous nature [22].
Increasing the tactile feedback available during this contact
stream could be even more beneficial.
Tactility in Mobile Devices and Touchscreens
There are now many examples of interaction design for
tactile feedback in touch-surface devices, e.g. [15,23], as
well as research and commercial instantiations based on
technologies such as piezo- or solenoid-actuated screens.
Nearly all rely on vibration. An interesting alternative is
electrovibration, which produces sensations of friction or
vibration using periodic electrostatic forces [3]. Vibrations
can also be applied with passive [24] or actuated pens [10].
While most efforts have layered tactile feedback atop a
normal GUI (e.g. tactile overlays on soft keyboards that
indicate key proximity and presses [13]), it is arguable that
greater benefits are possible for interactions designed
around taction. Pokespace relies on forces for its gesture
set, and found reduced visual demand in augmented
widgets tested in a similar manner to our Study 4 [30].
Generally, speed and accuracy have improved when haptics
is included in pointing tasks [1,6,7]. This result is nuanced,
however: in absence of knowledge of user’s destination,
feedback may also be encountered for non-target elements,
which can introduce obstructions and slowdowns [6,14,20].
Furthermore, user preference does not always follow
targeting utility [31].
Theoretical Arguments for Variable Friction
Illusions can be exploited to improve the performance or
immediacy of a passive touch interaction. Synchronous
sound and graphics can suggest absent tactile feedback.
Users “feel” auditory clicks [8]; Apple’s iPod took this
illusion mainstream. But it fails when the earbuds are out,
and lacks the useful physical constraint of a real click.
Likewise, visuo-haptic effects such as ‘sticky widgets’, a
manipulation of mouse control-display gain, can improve
selection performance by curtailing overshoot in the closed-
loop phase of motion and enlarging the motor space [2].
Variable friction may further improve performance by
making the finger actually stick to the target.
Touch interfaces are also subject to the biomechanics of
finger sliding on glass, which produces asymmetric stick-
slip [18]. ‘Judder’ is greatest in the distal direction (‘north’)
where friction acts to bend the finger, opposed by extensor
muscles. Bending reduces contact angle, increasing the
force required to maintain movement (‘stick’), then the
finger springs forward (‘slip’). Proximal or sideways
dragging is resisted skeletally with a relatively constant
contact angle. Lowering friction should reduce judder for
even distal movements.
Variable Friction Devices: the LATPaD
The LATPaD varies the friction felt by the fingertip at the
touch surface. Its operating principle, a squeeze film of air
produced by 26 kHz piezo-actuated vibrations, lowers the
friction coefficient of a glass surface from ~1.0 to ~0.15.
Unpublished experiments indicate that the just-noticeable-
difference in friction is about 30-40%; thus the LATPaD’s
dynamic range provides several distinguishable friction
levels. Other models demonstrate this effect on larger and
smaller plates of arbitrary shape and a range of materials.
Still in early development, our prototype has several
limitations. The piezo actuation is compact, but the optical
position sensing uses a larger housing. The piezos produce
audible noise when active. The vibrational mode used
produces nodes where friction reduction is weaker (here, 2
narrow strips parallel to the longer screen axis).
Development continues, focusing on these issues. Within 2-
3 years, programmable friction is expected to be deployable
in a form factor similar to current touchscreens with
uniform feedback and no audible noise.
A Design Space for Variable-Friction Touch
These and other works (e.g. [28]) demonstrate that while
entrancing, current touch technology leaves usability gaps:
it is hard to accurately point, select and drag, to select and
enter text, and to achieve drag & drop functions. The
illusion of physicality, with both its utility and aesthetic,
disappears with the withdrawal of image or sound.
In ongoing work, we are defining the design space where
variable friction offers value by filling these gaps. This
space is structured around dimensions of (1) rendered
effects (e.g. impact, edges, stiffness) and (2) communicated
information (e.g. selection support and confirmation,
functional availability, spatial navigation). It has guided the
design of Study 4, by indicating both the extent of the space
to be sampled and opportunities within it.

Page 3

STUDIES 1-3: FRICTION PSYCHOPHYSICS
S1-S3 were conducted in one session and studied the effect
of variable friction on target selection and finger motion.
Study 1: Target Selection Without Distracters
S1 concerns the speed and accuracy of target selection with
and without variable friction. Based on the theoretical
ability of variable friction to halt the finger on target, we
hypothesize:
H1. Variable friction across the surface, with high friction
over the target, will improve selection speed and accuracy.
A secondary consideration is the overall level of friction
during targeting. Any benefit of varying friction could be
explained solely by faster inter-target movement caused by
the more slippery surface, rather than by differential friction
at the target. The study controls for this effect, leading to
the second hypothesis:
H2. There will be no significant difference between
selection speed and accuracy when using a constant low
level and a constant high level of friction.
Selection modality: lift-off. Friction only matters during
sliding surface contact. Thus traditional Fitts’ Law [9]
‘tapping tasks’, where most or all movement occurs above
the surface, are unlikely to be influenced by friction effects;
we therefore analyzed drag-based selections that are issued
when the finger lifts off the surface. This is a common
modality on touch devices, particularly when targets are
small (e.g., sliding text entry on the iPhone). Furthermore, a
lift-off selection modality has been seen to be more
accurate than others for touch input in some contexts [22].
Direction. We controlled for movement direction (north,
south, east and west), with the aim of revealing movement
dynamics rather than testing a specific hypothesis.
Procedure
Participants were given written instructions on how to
interact with the LATPaD. They then tried the device for
one minute by rubbing a finger across a checkerboard with
high and low friction. They were given the exact procedure
for each of trials, which consists of the following steps:
1. Initial state. A thin blue ‘control’ line and a red ‘target’
line appear on the display.
2. Acquire control line by touching and remaining
stationary on it. For any movement off the control line
or off the surface, the trial is repeated.
3. Free control line and begin. After holding for 0.2
second, an audible beep is heard and the control line is
freed to move. The clock starts on movement.
4. Drag control line over target. The target turns green to
confirm the over-target state, and in some conditions
friction changes over the target.
5. Lift-off to select by raising the finger off the target. The
target turns back to red briefly and the control line
disappears until the next trial.
Participants completed 30 training selections with a 36
pixel (5.62 mm) wide target (data discarded). Ten were
completed with each interface condition with the same
order of exposure as for experimental trials (below). To
reduce any possible influence of the LATPaD’s audible
sound, participants listened to white noise through Direct
Sound Extreme Isolation EX-29 headphones throughout.
Each participant then completed 336 experimental trials (96
discarded) covering three factors: interface, direction, and
target width. The three levels of interface were constant
high friction (HF), constant low friction (LF) and variable
friction (VF). In HF and LF, LATPaD oscillations were
always turned off or maximally on, respectively. In VF,
friction was high (LATPaD off) over the target, but low
(maximally on) everywhere else. We did not test inverse
variable friction (low over the target and high elsewhere) as
it does not offer the psychophysical advantages of ‘finger
trapping’ promised by VF. The four levels of direction were
north (n), south (s), east (e), and west (w); and the four
levels of width were 6, 12, 24, and 48 pixels (0.94, 1.87,
3.74 and 7.49 mm). The device was physically rotated when
changing direction axis (n/s, e/w) so that movement
remained within an optimal friction region. Movement
amplitude was always 225 pixels (35.1 mm). The control
and target lines were shifted slightly towards the n or w side
of the screen to avoid interference from the raised screen
rim. Figure 2 shows the interface for north-direction
selections for the four widths.

(a)

(b)

(c)

(d)
Figure 2. Target acquisition task for S1 in the North direction:
(a) entire screen for first interface with width of 6 pixels, and
partial screen with widths of (b) 12, (c) 24 and (d) 48 pixels.
The
administered as blocks of 14 trials, each block sharing a
direction axis (n/s, e/w), interface level (LF, HF, VF) and
target size (6, 12, 24, 48). The first 4 trials of each block
were discarded to allow for strategy adaptation. Initial
direction was randomized for each block, then alternated on
the direction axis. Blocks sharing a direction axis were
administered consecutively to minimize physical device
rotation; then grouped by interface level to allow
questionnaire assessment. Block sets were counterbalanced
such that all combinations of 2!=2 direction axis orderings
and 3!=6 interface level orderings were used for one
participant. Finally, block target size was randomized
within same-interface sets. A total of 24 blocks (2 direction
axis ! 3 interface levels ! 4 target sizes) were administered.
experimental trials (target acquisitions) were

Page 4

(a)

(b)

(c)
Figure 4. (a) Approach time, (b) dwell time, and (c) number of
target entries as a function of the interface condition (S1).

After each of the 6 sets, participants were asked to
comment on the interface used and respond to 5-point
Likert-scale questions (strongly disagree to strongly agree):
“I performed well / needed to concentrate to accomplish the
task / felt confident in my ability to hit the target / felt
frustrated / enjoyed interacting with the touchscreen.”
After completing all trials, participants were asked to rank
the interfaces (‘ties’ permitted) and for final comments.
Interface was referred to by order of appearance, reinforced
with a numerical label on the side of the display during use.
Participants
Twelve participants (6 female) were recruited from a local
university: aged 19-48 (mean 29.4), all right-handed.
Design and Analysis
Dependent measures are analyzed using a 3!4!4 repeated
measures analysis of variance for the factors interface !
{HF, LF, VF}, direction ! {n, s, e, w}, and target width !
{6, 12, 24, 48 pixels}. The dependent measures are
selection time, number of errors, time between entering the
target and lifting off it, and number of overshoots. We also
analyze the goodness of fit to Fitts’ Law models
(coefficient of determination) and subjective responses.
Results
In summary: variable friction (VF) improved targeting
performance over HF without compromising accuracy, thus
we accept H1. The constant low friction conditions (LF)
produced similar results as HF, so we also accept H2.
Acquisition time. There was a significant effect of interface
(F2,22=6.89, p<.01), with VF fastest (mean 921 ms, s.d.
324), then HF (mean 990, s.d. 344) and LF (mean 1002, s.d.
341); see Figure 3. Posthoc comparison using Bonferroni
correction confirms differences between VF and both HF
and LF (p<.05), but not between HF and LF.
As anticipated, there was a significant main effect of width
(F3,33=82.2, p<.001); but there was also a significant
interface!width interaction (F6,66=3.85, p<.01). Figure 3
suggests that VF performance deteriorated less rapidly
across increasing Index of Difficulty than the other
conditions. This explanation is supported by the Fitts’ Law
analysis, which showed strong models for all conditions
(R2>0.98). The lower slope for VF corresponds to an Index
of Performance (reciprocal of the slope) of 7.26 bits/s,
which is higher than either HF (5.07) or LF (5.74). There
was no significant main effect of direction (F3,33=1.8,
p=.17), with means of 943, 960, 996 and 984 ms for n, s, e
and w movement respectively.
Accuracy. Analysis of count of trials per block containing
an error shows no significant effect of interface (F2,22=0.74,
p=0.49), with similar means of 0.82 errors with VF, and 0.7
and 0.81 for HF and LF respectively. The relatively high
error rate is due to the use of small targets, and as expected,
there is a significant effect of width (F3,33=53.9, p<0.001),
with errors increasing from 0.14 errors per block with 48-
pixel targets to 1.8 with 6-pixel targets. There was a
significant effect of direction (F3,33=10.0, p<.01) with the s
movement being the most error prone (1.2 errors per block)
and n being the least (0.4). Importantly, however, there was
no interface!width (F6,66=1.13,
interface!direction interaction (F6,66=1.16, p=0.34).
p=0.36) or
Source of VF performance advantage. There are several
possible explanations for the performance advantage with
variable friction: users may move more quickly, resulting in
a shorter target approach; they may respond more quickly
to the over-target state, resulting in a shorter dwell time
over the target; or variable friction may ‘trap’ the finger on
the target, reducing overshoot. To understand which of
these are at play, we conducted three more one-way
ANOVAs (Figure 4) with dependent variables of approach
time (from initial movement to last target border entry),
dwell time (from last target border entry to lift-off), and
entry count (number of times the target border was entered).
This revealed a significant effect for approach time
(F2,22=5.69, p<0.05) with VF faster (mean 634 ms, s.d. 268)
than either HF (690, 283) or LF (706, 276). Neither dwell
time (F2,22=1.0, p=0.38) nor entry count (F2,22=0.87, p=0.43)
varied significantly, although VF had the lowest mean in
both. Consequently, it seems that the largest effect of target
acquisition with VF is that it increases users’ confidence in
moving towards the target, allowing them to approach more
quickly without compromising ability to stop abruptly on
the target and select it accurately [14].
Subjective results. Participants were asked to rank each
interface condition after both e/w and n/s trial blocks. VF
was ranked 1st 58.3% of the time, 2nd 29.2%, and 3rd 12.5%,
producing a significant difference (Friedman "2=9.5,
p<.01); mean rankings were 1.5, 2.0 and 2.1 for VF, LF,

Figure 3. Results and Fitts’ law models for interfaces (S1).

Page 5

and HF. Questionnaire responses (Table 1) show that mean
ratings for VF were most appreciative in all five questions,
but only significantly so for enjoyment.
Study 2: Target Selection With Distracters
Previous work has demonstrated that tactile feedback can
negatively influence performance in the presence of
distracter targets [6]. This is a critical limitation, as most
practical deployments will involve distracters. Therefore we
tested the hypothesis that:
H3. Variable friction will not adversely affect targeting
performance in the presence of distracter targets.
Procedure, Apparatus and Participants
The 12 participants from S1 proceeded immediately to S2.
The task was identical to S1 but the space from the control
line to the target and beyond was populated with distracters
of the target’s width. All distracters produced the same
visual and tactile effects as the target, i.e. highlighted green
to indicate over-target state, and presence of friction effects.
Distracters were otherwise black.
Distracter density is an important variable for H3, so three
levels of distracter separation were used: 5, 20, and 40
pixels (Figure 5). Targets and distracters were 24 pixels in
width. The number of distracters placed before the target
varied from 1 to 3; one distracter was always behind the
target. We tested only one direction axis, and selected n/s
for a greater range of biomechanical effects.
A block held twenty-two target selections (trials); three
blocks were performed per interface condition. The
direction (n, s) was initially randomized for each block and
then alternated. The number of distracters and their spacing
were selected randomly for the first four trials (discarded).
To prevent memorization of the tactile pattern leading to
the target and its use as an aid, the remaining 18 trials
cycled randomly through all 3!3 combinations of distracter
number and spacing in each direction (n, s). Interface order
and subjective responses were controlled as for S1.
Design
The same dependent measures as S1 are analyzed in a
3!2!3!3 repeated measures analysis of variance for
interface ! {HF, LF, VF}, direction ! {n, s}, distracter
spacing ! {5, 20, 40 pixels} and number ! {1, 2, 3}.
Results
In summary: the results show no effects of interface (main
or interactions) for dependent measures of time or errors,
thus we accept H3.
Performance. A four-way ANOVA showed a significant
effect of distracter spacing on acquisition time (F2,22=13.3,
p<.01), with mean of 908, 892 and 874 ms for 5, 20 and 40
pixel spacing respectively. There were no significant main
effects or interactions involving interface for either task
time or errors. Mean times were similar (900, 892, 882 ms
for VF, HF, LF; F2,22=0.26, p=0.78), as were per block error
rates (0.16, 0.22, 0.19; F2,22=0.71, p=0.5). S1 trials (zero
distracters) with n/s directions and 24-pixel targets were
also compared to S2 (1, 2 or 3 distracters). A three-way
ANOVA for factors interface, direction and number of
distracters revealed no significant effects.
Subjective results. The participant’s interface rankings were
similar to S1: a significant preference for VF, but non-
significant responses to other questions.
Study 3: Constant Velocity Dragging
To characterize physical effects occurring at friction
borders, we conducted a third shorter study with the same
participants immediately after S2. Participants tried to
achieve a target drag velocity in repeated bidirectional
strokes across the display by following an audible
metronomic tempo. We tested two speeds (50 / 100 mm/s,
or 320.5 / 641.0 pixels/s), and both orientations (n/s, e/w).
Friction patterns (not shown visually) were produced in the
middle of the strokes, including constant levels, step
changes, and single or sequential pulses with the same
extent, number and separation as S1 and S2’s targets and
distracters. Each participant performed 168 trials in six
blocks. The first six trials of each block were discarded to
allow speed adjustment. The order of patterns was
randomized within blocks and each pattern shown once.
Results
Finger position and velocity trajectories were plotted
against friction state and inspected visually for all 1584
trials. An effect of variable friction is clear in many
trajectories. Figure 6 shows the best examples, where a step
increase or decrease in friction leads to a temporary
deceleration or acceleration of the finger. This suggests that
finger velocity is affected at least under certain conditions.
A sticky target with variable friction may therefore be truly
sticky. This effect may in other cases have been reduced by

(a)

(b)

(c)

(d)

(e)
Figure 5. Target acquisition task for S2 in the n direction: (a)
entire and (b-e) partial screen for first interface with 1 to 3
pre-target distracters and a separation of 5, 20 and 40 pixels.


Performance
Concentration
Confidence
Frustration
Enjoyment
Table 1. Mean (st. dev.) questionnaire responses, with 1=
strongly disagree, and 5 = strongly agree (S1).
VF LF HF
3.5 (0.7)
3.5 (1.0)
3.7 (0.6)
2.2 (0.8)
3.6 (0.8)
2
r ! Sig
2.3 =0.31
3.4 =0.18
2.8 =0.25
2.3 =0.31
5.8 =0.05
3.9 (0.7)
3.2 (1.0)
4.0 (0.7)
1.8 (0.7)
4.2 (0.7)
3.7 (0.8)
3.7 (1.0)
3.5 (0.8)
2.0 (0.8)
3.8 (0.8)

Page 6

(a)

(b)

(c)


(a)

(b)

(c)

(d)

(e)

(d)
Figure 7. Alarm Clock: (a) hour and minute wheels, (b)
AM/PM wheel, (c) sound combo box and friction patterns
while selecting (d) hour and (e) sound. The finger color
changes from light blue to dark red as friction increases.
Figure 8. File Manager: (a) initial screen, moving a file into (b) a
folder or (c) recycle bin, and (d) friction patterns while over a
folder or bin. The finger color changes from light blue to dark
red as friction increases.
stick-slip of the fingerpad or finger pressure, or masked by
the low spatio-temporal data. A quantitative analysis is
beyond the scope of this paper, but currently underway.

STUDY 4: DESIGN EXAMPLES AND USER EXPERIENCE
We explored interface design for variable friction interfaces
in an iterative process, beginning with glass etched
prototypes, then exemplar designs, which finally led to a
study to establish their emotional and subjective impact.
Demonstration Applications and Study Tasks
Four exemplar widgets were designed to provide good
coverage of tactile sensations and of communication
functions that variable friction might support.
Alarm Clock. Users set the alarm time using wheel widgets
and the alarm sound using a combo box (Figure 7a-c). The
wheels produce strong ticks by abruptly increasing friction
as items near their center (Figure 7d). The combo box
produces similar ticks but with friction peaks between
targets (Figure 7e). The Alarm Clock study task involved
setting the time and sound to a value displayed at the
bottom of the screen.
File Manager. File, folder and recycle bin icons are
arranged in a 3!4 array (Figure 8a); target icons enlarge
20% when a file hovers over them (Figure 8b-c). Initially
low, friction increases abruptly over folders and oscillates
at 37.5 Hz over the recycle bin, producing a bump and
buzzing respectively (Figure 8d). The File Manager study
task involved moving eight files labeled 1 to 3 into folders.
Game. The game consists of bouncing a ball against a
round cursor around the finger and breaking bricks (Figure
9a-c). Some require multiple hits to be broken; others
produce special effects - releasing a second ball or making
the ball bounce erratically. The ball is launched by
compressing a spring, with gradual increase in friction to
simulate resistance (Figure 9a, d). For ball impact (Figure
9b, e), friction abruptly increases as the ball nears the
cursor. Erratic bouncing (Figure 9c) produces the friction
oscillation used on the File Manager recycle bin. The Game
study task was to play the game, with ten difficulty levels.
Text Editor. Words are selected by dwelling with a cursor
extending above the finger, avoiding occlusions. While
dragging a word, collisions with adjacent words result in
visual compression up to 30%, after which words remain
fixed (Figure 10a). A word swaps with its neighbor when
the cursor reaches a position where it can be relocated.
Swiping left or right flips pages (Figure 10b). When
dragging within a line, friction increases with compression,
and drops abruptly after a swap, creating a popping
sensation (Figure 10c). When dragging between lines,
friction effects fade in and out across lines, with a brief
friction pulse in-between. Page swaps trigger a tick via an
abrupt increase in friction. The Text Editor study task
consisted of ‘fixing’ sentences by reordering words within
pages – e.g. “the store grocery sells yellow tomatoes green
bananas red lettuce and eggplants purple.”

(a)

(b)
Figure 6. Selected S3 results: (a) deceleration after increase in
friction (P2, e), (b) acceleration after step decrease (P4, n).

Page 7

Experimental Procedure
Participants interacted with each of the four applications
twice, with and without variable friction. Each interaction
was limited to 2 minutes to provide exposure without
boredom. Each application was presented first for # of the
participants, and the order of the other applications was
randomized. Half of the participants experienced all
applications first with variable friction, the other half
without. Participants were instructed to focus on experience
rather than performance.
A User Engagement Scale (UE1-10) was used after each
condition (twice per application; with and without friction).
A tactile feedback questionnaire (TF1-7) was completed
after each interaction with variable friction (once per
application feature). Once both friction conditions were
completed with an application, a comparison questionnaire
(C1-5) was administered, followed by a short interview (I1-
3). This procedure was repeated for all four applications,
and followed by a final questionnaire (F1-2). The User
Engagement Scale used a 7-point Likert scale and all others
a 5-point scale (strongly disagree to strongly agree).
The User Engagement Scale (UE1-10) was adapted from a
validated 31-question questionnaire developed to assess six
aspects of engagement: Focused Attention, Perceived
Usability, Aesthetics, Endurability, Novelty and Felt
Involvement [21]. Ten of the 31 questions were adapted,
spanning all aspects:
UE1. I was absorbed in my interaction task.
UE2. I felt in control of my interactive experience.
UE3. I found this application confusing to use.
UE4. I liked the visual and tactile effects used in this application.
UE5. This application appealed to my visual and tactile senses.
UE6. I would recommend this application to my friends and family.
UE7. I would have continued to interact with this app. out of curiosity.
UE8. I felt interested in my interaction task.
UE9. This interactive experience was fun.
UE10. I felt involved in this interaction task.
The tactile feedback questionnaire (TF1-7) was completed
for the main tactile features of each application:
hour/minute wheel, AM/PM wheel and sound combo box
for Alarm Clock; folders and recycle bin for File Manager;
launcher, normal and erratic bounce for Game; movement
within or between lines and page swapping for Text Editor.
Participants were asked if they noticed the feature, and if so
rated whether the tactile feedback was (TF1) weak, (TF2)
natural, (TF3) informative, (TF4) annoying, (TF5) matched
the visuals, (TF6) felt good, and (TF7) was preferred.
The comparison questionnaire (C1-5) asked if tactile
feedback (C1) was preferred, (C2) made the task easier to
perform, (C3) the application more enjoyable, (C4) the
interface more realistic and (C5) made them more confident.
Interview questions (I1-3) asked participants (I1) to describe
the sensations, (I2) what they liked and didn’t like about the
tactile feedback, and (I3) how they would improve it. The
final questionnaire (F1-2) asked (F1) if participants would
turn this type of feedback off on their phone, and (F2) if
tactile feedback improved their experience.
Participants
The data for eight participants were rejected due to protocol
irregularities and hardware or software complications that
may have affected subjective responses. The remaining
twelve participants (6 females) were aged from 19 to 38
(mean 24.3). Eleven were students, only four from
engineering and computer science. Ten used touchscreen
phones or music players once a week or more. None
participated in S1-3.
Results
The participants’ comments and questionnaire responses
demonstrate that friction can improve the subjective
experience of touch interactions. They also provide insights
into potential negative effects, which need to be addressed
by design. We begin with interview responses and
comments, and then report questionnaire results.

(a)

(b)

(c)
Figure 10. Text Editor: (a) word movement, (b) page swap and
(c) friction patterns while moving a word. The finger color
changes from light blue to dark red as friction increases.


(a)

(b)

(c)

(d)

(e)
Figure 9. Game: (a) launch, (b) normal and (c) erratic bounce,
and friction patterns during (d) launch and (e) bounce. Low
friction shown as a blue finger, high friction red.

Page 8

Interview Responses
Several comments show that variable friction enhanced the
participant’s sense of realism: “When I was moving the
words against something, I could feel something squeeze
back.” (P3, Text Editor); “I knew I was actually touching
it.” (P2, File Manager); “It feels […] as if turning the
wheels.” (P11, Alarm Clock).
Comments also show that variable friction increased
awareness of the system state, some suggesting a reduced
dependence on vision: “I think it gives me accuracy, […] if
I closed my eyes I would be able to predict the amount of
scrolling that I do.” (P5, Alarm Clock); “Feel more
informed ... when I am moving on the line I can feel each
word.” (P11, Text Editor); “For the garbage bin it’s like
oh ah don’t do it.” (P3, File Manager).
Importantly, several comments showed that participants
liked the friction effects: “This is nice... it makes things a
lot more interesting.” (P3, Game); “I liked the sensation
while I am rolling” (P8, Alarm Clock).
Nine of the twelve participants were predominantly positive
in their comments about variable friction for one (1), two
(3), three (4) or four (1) of the applications. The remaining
three were predominantly negative or neutral. Negative
words used to describe variable friction included
“unpleasant”, “weird”, “creepy”, “annoying” and “itchy”.
The tactile feedback was often described more neutrally
using physics-related terms such as “resistance”,
“friction”, “slippery” and “sticky”. Negative comments
were often aimed at the limitations of the applications but
also suggested potential pitfalls of variable friction such as
“[getting] in the way of trying to move” (P8, File Manager)
or inducing fatigue through overuse (P3, Alarm Clock).
Interestingly, there was little cross-participant consistency
in assessing which applications and effects were positive
additions. Similarly, participants
assessment of feedback strength, with two stating that
stimuli were too weak and one too strong. This suggests a
need for friction effects to be very carefully designed and
customizable by end users. A majority of participants also
spontaneously discussed variable friction’s integration in
commercial devices and
impressions for 5-15 minutes after experiment completion.
differed in their
voluntarily shared their
Questionnaire Responses
The questionnaire responses tend to amplify the overall
positive response to variable friction effects.
User Engagement Scale (Table 2). Responses for variable
friction were positive or neutral, except for control (UE2) in
Alarm Clock, confusion (UE3) in Alarm Clock and Game,
and liking (UE4) in Game; none statistically significant. Of
forty comparisons (4 applications ! 10 questions), variable
friction received better scores in 30: "2=9.0, p<.005.
Tactile Feedback. Marked differences were found in the
noticeability of friction effects, with only 25% noticing the
rapid pulse on page swap, but all noticing wheel effects.
Weak noticeability is explained by rare use (recycle bin,
page swap) or subtlety (launcher). Means for 67/77
contribution assessments of friction effects made by each
participant reflected positive opinions ("2=40.7, p<.0001).
Negative assessments were most common in Game.
Comparison. Table 3 shows that all but one of the 20 direct
comparisons favors variable friction: "2=14.4, p<.001. None
of the participants expressed a strong preference for
constant high friction in any of the questions.

Final Questionnaire. Participant responses suggest that
friction effects would not be turned off (mean 2.4, s.d. 1.0)
and that they improved the experience (mean 4.1, s.d. 0.5).
DISCUSSION AND NEXT STEPS
We first investigated performance for variable friction
effects and found measurable benefits without harm. S1
shows that variable friction (high on target, low elsewhere)
significantly improves targeting performance. S2 verifies
that variable friction is no worse than normal friction when
distracters are present, crucial since pseudohaptic or haptic
targets can decrease targeting performance in the presence
of distracters [6,14,20]. S3 suggests that friction variations
cause actual, not only perceived, velocity changes: unlike
pseudohaptic and vibrotactile aids, friction slows the

Table 3. Mean ratings [1-5] and distribution of answers
(1 bottom - 5 top) for comparison questionnaire.



UE1.
Absorbed.
UE2.
Control.
UE3.
Confusion.
UE4.
Liked.
UE5.
Appeal.
UE6.
Recommend.
UE7.
Curious.
UE8.
Interested.
UE9.
Fun.
UE10.
Involved.
Table 2. Mean (and s.d.) for the User Engagement Scale [1-7].
Rating pairs favorable to VF are bold.

Clock
HF
4.7
(1.2)
5.3
(0.8)
2.0
(1.2)
4.4
(1.3)
4.2
(1.3)
4.2
(1.4)
4.0
(1.9)
4.3
(1.7)
4.2
(1.6)
4.6
(1.6)
Files
HF
4.8
(1.2)
5.6
(0.7)
1.7
(0.8)
4.9
(0.9)
4.1
(1.5)
4.1
(1.4)
3.6
(1.7)
4.6
(1.5)
4.2
(1.6)
4.6
(1.4)
Game
HF
5.6
(0.9)
4.9
(1.2)
1.8
(0.9)
4.7
(1.7)
5.1
(1.1)
5.0
(1.3)
5.6
(1.5)
5.8
(1.0)
5.6
(0.8)
5.2
(1.2)
Text
HF
5.1
(0.8)
4.0
(1.7)
2.5
(1.4)
4.5
(1.7)
3.8
(1.5)
4.7
(1.7)
4.2
(1.9)
4.7
(1.5)
4.4
(1.8)
4.3
(1.8)
VF
4.9
(0.9)
5.1
(1.3)
2.3
(1.3)
5.0
(1.3)
5.3
(1.1)
5.0
(1.4)
4.9
(1.5)
5.2
(1.5)
5.0
(1.7)
5.3
(0.8)
VF
5.4
(1.2)
5.6
(0.9)
1.7
(0.9)
5.4
(1.4)
5.5
(1.0)
5.0
(1.6)
4.6
(1.9)
5.0
(1.5)
4.7
(2.0)
5.7
(1.0)
VF
5.8
(0.6)
5.3
(0.9)
2.2
(1.2)
4.5
(1.6)
5.7
(0.8)
5.0
(1.5)
5.6
(1.6)
5.7
(1.0)
5.7
(1.0)
5.7
(0.7)
VF
5.1
(1.2)
4.8
(1.9)
2.3
(1.6)
5.1
(1.5)
5.2
(1.3)
4.9
(1.8)
4.9
(1.8)
5.4
(1.6)
5.0
(1.5)
5.5
(0.8)

Page 9

fingertip. Yet movement time does not rise, even with
distracters, because approach stages are faster (S1-2).
We then focused on user experience, not performance [11]
– engagement, enjoyment, directness, perceived utility –
and found through our S4 application samples that variable
friction can enhance the emotional aspect of using a touch
interface. This approach freed us to explore many design
concepts and highlights what may ultimately be the most
crucial factor in improving upon passive touch interfaces.
In the following we reflect upon generalizing these results
to real-world use and understanding their value.
Hardware Factors
The LATPaD is currently a bulky prototype. However, the
critical components for producing variable friction are the
small, thin piezoelectric actuators visible in Figure 1a and
there are no major barriers to miniaturizing the technology
to the scale of current mobile devices. Rendering non-
uniformities are also expected to be resolved in the near
future, and were successfully avoided in our S4 designs.
Performance in real-world target acquisition
The LATPaD’s variable friction effects are only felt when
sliding against the display surface; S1-2 therefore used
dragging tasks to maximally expose participants to the
effects of interest. The lift-off selection used here is
common on mobile devices, including the iPhone’s text
entry keypad. However, when approach occurs in the air,
friction effects would only be felt during the final
acquisition. This has been shown to reduce targeting
performance with pen-based vibrotactile feedback [24]. We
need to better understand how variable friction can benefit
other selection modalities, but feel that first analyzing the
most effective interaction for friction was appropriate.
Variable Friction Versus Vibrotactile Feedback
Vibrotactile (VT) feedback
performance improvements similar to variable friction, at
least in single target conditions [24]. Variable friction and
VT actuators, however, produce sensations that are very
different. Friction tends to feel more natural and provides
continuous feedback during sliding; vibrations are ideal for
discrete clicks and textures, including tapping confirmation
[23,24]. Variable friction can physically alter finger
velocity, whereas VT can communicate a larger range of
informative sensations [16], even without sliding. These
two tactile modalities are complementary and in theory can
be produced with the same actuators.
can provide selection
Theoretical Design Space for Variable Friction
The exemplar applications and widgets examined in S4
were developed through an iterative design process, and
were successful in generating subjective responses. We
took guidance both from the beginnings of our design space
and from our intuition and iterations. We will refine this
beginning with a taxonomy of variable friction sensations
and their mapping to interactive widgets. This includes
examining generalizations of our current set of widgets to
other uses. For example, the Alarm Clock wheel was among
the most popular, and we will examine how it can be
deployed in support of related uses, such as scrolling. We
will also investigate interaction techniques that minimize
the use of low friction and hence actuator activation and
power consumption, such as reduced friction on targets.
Towards Variable Friction Design Heuristics
In its infancy, the design of interactions with tactile
feedback is prone to naive uses and excesses. We hope to
launch a discussion of best practices for variable friction,
extending a design space with heuristics such as these:
Sliding not tapping. To be effective, friction-augmented
interfaces must work around the notion of sliding, with
appropriate metaphors and visual representations. The Text
Editor, for example, associated friction with word
compression to give meaningful feedback during dragging.
Shaping friction to increase expressiveness. To compensate
for the limited human sensitivity to friction variations and
the current display range, expressiveness can be enhanced
by varying friction ‘attack’, modulating it to create textures
and patterns, and tying sensations to visual representations.
Stop only for a purpose. Some users felt that friction
variations seemed occasionally to slow them down. Strong
feedback should have an equally strong purpose.
Nice not strong. We tend to maximize tactile signal strength
to ensure feedback is felt and performance improvements
are measurable; but this can lead to unpleasant sensations.
CONCLUSION
Programmable friction displays vary the friction felt while
sliding against a touch sensitive display. Through a series of
studies and design explorations, we have demonstrated the
strong potential of programmable friction interfaces. Most
importantly, participants preferred our exemplar designs to
traditional touch interactions and reported a variety of
positive effects, including increased engagement, a sense of
realism and reduced dependence on vision. In addition, our
examination of programmable friction showed significant
performance advantages for drag-based selections and no
adverse effects when distracter targets are present.
This is the first analysis we are aware of for interaction with
variable friction displays. These quantitative and qualitative
results show exciting possibilities; the technology is on a
development path 2-3 years from commercial realizability.
There is great potential for more investigation: further
performance analysis, design exploration and then
deployment in mobile handhelds and laptop touchpads.
REFERENCES
1. Akamatsu, M. and MacKenzie, I.S. Movement
characteristics using a mouse with tactile and force
feedback. Int. J. of Human-Computer Studies 45, 4
(1996), 483-494.

Page 10

2. Balakrishnan, R. "Beating" Fitts’ law: virtual
enhancements for pointing facilitation. Int. J. of Human-
Computer Studies 61, 6 (2004), 857-874.
3. Bau, O., Poupyrev, I., Israr, A., and Harrison, C.
TeslaTouch: Electrovibration for Touch Surfaces. In
Proc. UIST 2010, ACM (2010).
4. Benko, H., Wilson, A.D., and Balakrishnan, R. Sphere:
Multi-Touch Interactions on a Spherical Display. In
Proc. UIST 2008, ACM (2008), 77-86.
5. Buxton, W., Hill, R., and Rowley, P. Issues and
techniques in touch-sensitive tablet input. In Proc.
SIGGRAPH 1985, ACM (1985), 215-224.
6. Cockburn, A. and Brewster, S. Multimodal feedback for
the acquisition of small targets. Ergonomics 48, 9
(2005), 1129-1150.
7. Dennerlein, J.T., Martin, D.B., and Hasser, C. Force-
feedback improves performance for steering and
combined steering-targeting tasks. In Proc. CHI 2000,
ACM (2000), 423-429.
8. DiFranco, D.E., Beauregard, G.L., and Srinivasan, M.
The effect of auditory cues on the haptic perception of
stiffness in virtual environments. In Proc. Haptics
Symposium 1997, ASME (1997), 17-22.
9. Fitts, P.M. The information capacity of the human motor
system in controlling the amplitude of movement. J. of
Experimental Psychology 47, 6 (1954), 381-391.
10. Forlines, C. and Balakrishnan, R. Evaluating tactile
feedback and direct vs. indirect stylus input in pointing
and crossing selection tasks. In Proc. CHI 2008, ACM
(2008), 1563-1572.
11. Greenberg, S. and Buxton, B. Usability evaluation
considered harmful (some of the time). In Proc. CHI
2008, ACM (2008), 111-120.
12. Hancock, M., Carpendale, S., and Cockburn, A.
Shallow-depth 3d interaction: design and evaluation of
one-, two- and three-touch techniques. In Proc. CHI
2007, ACM (2007), 1147-1156.
13. Hoggan, E., Kaaresoja, T., Laitinen, P., and Brewster, S.
Crossmodal congruence: the look, feel and sound of
touchscreen widgets. In Proc. IMCI 2008, ACM (2008),
157-164.
14. Keyson, D.V. Dynamic cursor gain and tactual feedback
in the capture of cursor movements. Ergonomics 40, 12
(1997), 1287-1298.
15. Li, K.A., Baudisch, P., Griswold, W.G., and Hollan,
J.D. Tapping and rubbing: exploring new dimensions of
tactile feedback with voice coil motors. In Proc. UIST
2008, ACM (2008), 181-190.
16. MacLean, K.E. Foundations of Transparency in Tactile
Information Design. IEEE Transactions on Haptics 1, 2
(2008), 84-95.
17. Marchuk, N.D., Colgate, J.E., and Peshkin, M.A.
Friction measurements on a Large Area TPaD. In Proc.
Haptics Symposium 2010, IEEE (2010), 317-320.
18. Moscovich, T. Contact area interaction with sliding
widgets. In Proc. UIST 2009, ACM (2009), 13-22.
19. Norman, D. Emotional Design: Why We Love (or Hate)
Everyday Things. Basic Books, NY, NY, USA, 2004.
20. Oakley, I., Brewster, S., and Gray, P. Solving multi-
target haptic problems in menu interaction. In Proc. CHI
2001, ACM (2001), 357-358.
21. O’Brien, H.L. and Toms, E.G. The development and
evaluation of a survey to measure user engagement.
Journal of the American Society for Information Science
and Technology 61, 1 (2009), 50-69.
22. Potter, R.L., Weldon, L.J., and Shneiderman, B.
Improving the accuracy of touch screens: an
experimental evaluation of three strategies. In Proc. CHI
1988, ACM (1988), 27-32.
23. Poupyrev, I. and Maruyama, S. Tactile interfaces for
small touch screens. In Proc. UIST 2003, ACM (2003),
217-220.
24. Poupyrev, I., Okabe, M., and Maruyama, S. Haptic
feedback for pen computing: directions and strategies.
In Proc. CHI 2004, ACM (2004), 1309-1312.
25. Ramos, G. and Balakrishnan, R. Fluid interaction
techniques for the control and annotation of digital
video. In Proc. UIST 2003, ACM (2003), 105-114.
26. Rekimoto, J. Organic interaction technologies.
Communications of the ACM 51, 6 (2008), 38-44.
27. Roudaut, A., Huot, S., and Lecolinet, E. TapTap and
MagStick: improving one-handed target acquisition on
small touch-screens. In Proc. AVI 2008, ACM (2008),
146-153.
28. Schiphorst, T., Motamedi, N., and Jaffe, N. Applying an
Aesthetic Framework of Touch for Table-Top
Interactions. In Proc. TABLETOP 2007, IEEE (2007),
71-74.
29. Shneiderman, B. Direct Manipulation: A Step Beyond
Programming Languages. Computer 16, 8 (1983), 57-
69.
30. Smyth, T.N. and Kirkpatrick, A.E. A new approach to
haptic augmentation of the GUI. In Proc. ICMI 2006,
ACM (2006), 372-379.
31. Swindells, C., MacLean, K.E., Booth, K.S., and
Meitner, M.J. Exploring affective design for physical
controls. In Proc. CHI 2007, ACM (2007), 933-942.
32. Winfield, L., Glassmire, J., Colgate, J.E., and Peshkin,
M. T-PaD: Tactile Pattern Display through Variable
Friction Reduction. In Proc. World Haptics 2007, IEEE
(2007), 421-426.