Conference PaperPDF Available

Impact of the Size of Modules on Target Acquisition and Pursuit for Future Modular Shape-changing Physical User Interfaces

Authors:

Abstract and Figures

Shape-changing User Interfaces (UIs) explore the ability of a UI to change its physical shape to support multiple interaction modalities for users’ input and/or system’s output. An approach currently studied to implement such interfaces at a high resolution is based on mm-sized, round, and self-actuated modules. The problem we tackle in this paper is to find the range of usable sizes of such modules, to better inform the trade-off between usability and technological feasibility. We assessed four sliders in a controlled user study: a standard slider and three sliders made of mock-up rounded modules of ø1 mm, ø2.5 mm, and ø5 mm. Experimental results show that (1) ø5 mm modules significantly impair performance for the pursuit task and subjective perception for both tasks, (2) performance increases when the size of modules decreases, but (3) users reportedly enjoyed the haptic feedback provided by ø1 mm to ø2.5 mm modules. These results provide deeper understanding on the impact of the size of modules on performance and subjective perception to inform current technological development of physical user interfaces made of small robotic modules.
Content may be subject to copyright.
HAL Id: hal-03325220
https://hal.archives-ouvertes.fr/hal-03325220
Submitted on 30 Aug 2021
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entic research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diusion de documents
scientiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
Impact of the Size of Modules on Target Acquisition and
Pursuit for Future Modular Shape-changing Physical
User Interfaces
Laura Pruszko, Yann Laurillau, Benoît Piranda, Julien Bourgeois, Céline
Coutrix
To cite this version:
Laura Pruszko, Yann Laurillau, Benoît Piranda, Julien Bourgeois, Céline Coutrix. Impact of the
Size of Modules on Target Acquisition and Pursuit for Future Modular Shape-changing Physical User
Interfaces. Proceedings of the 2021 International Conference on Multimodal Interaction (ICMI ’21),
Oct 2021, Montréal, Canada. �10.1145/3462244.3479936�. �hal-03325220�
Impact of the Size of Modules on Target Acquisition and Pursuit
for Future Modular Shape-changing Physical User Interfaces
Laura Pruszko
Yann Laurillau
rst.last@univ-grenoble-alpes.fr
Université Grenoble Alpes
Benoît Piranda
Julien Bourgeois
rst.last@femto-st.fr
Univ. Bourgogne Franche-Comté,
Institut FEMTO-ST, CNRS
Céline Coutrix
celine.coutrix@imag.fr
CNRS, Université Grenoble Alpes
Figure 1: The four sliders used to investigate the impact of the size of future modules constituting the PUI: (from left to right)
a current smooth slider and three sliders made of mock-up modules of 1 mm,2.5 mm and 5 mm.
ABSTRACT
Shape-changing User Interfaces (UIs) explore the ability of a UI to
change its physical shape to support multiple interaction modali-
ties for users’ input and/or system’s output. An approach currently
studied to implement such interfaces at a high resolution is based
on mm-sized, round, and self-actuated modules. The problem we
tackle in this paper is to nd the range of usable sizes of such
modules, to better inform the trade-o between usability and tech-
nological feasibility. We assessed four sliders in a controlled user
study: a standard slider and three sliders made of mock-up rounded
modules of
1 mm
,
2.5 mm
, and
5 mm
. Experimental results
show that (1)
5 mm
modules signicantly impair performance
for the pursuit task and subjective perception for both tasks, (2)
performance increases when the size of modules decreases, but (3)
users reportedly enjoyed the haptic feedback provided by
1 mm
to
2.5 mm
modules. These results provide deeper understanding
on the impact of the size of modules on performance and subjective
perception to inform current technological development of physical
user interfaces made of small robotic modules.
CCS CONCEPTS
Human-centered computing Empirical studies in HCI.
KEYWORDS
Physical interaction, Slider, Target acquisition, Target pursuit, Mod-
ular Robotics, Shape-changing Interfaces, Modularity
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specic permission and/or a
fee. Request permissions from permissions@acm.org.
ICMI ’21, October 18–22, 2021, Montréal, QC, Canada
©2021 Association for Computing Machinery.
ACM ISBN 978-1-4503-8481-0/21/10. . . $15.00
https://doi.org/10.1145/3462244.3479936
ACM Reference Format:
Laura Pruszko, Yann Laurillau, Benoît Piranda, Julien Bourgeois, and Céline
Coutrix. 2021. Impact of the Size of Modules on Target Acquisition and
Pursuit for Future Modular Shape-changing Physical User Interfaces. In
Proceedings of the 2021 International Conference on Multimodal Interaction
(ICMI ’21), October 18–22, 2021, Montréal, QC, Canada. ACM, New York, NY,
USA, 11 pages. https://doi.org/10.1145/3462244.3479936
1 INTRODUCTION
Recent work on shape-changing User Interfaces (UIs) explored the
ability of a UI to change its physical shape to support multiple
interaction modalities [
11
] for users’ input and system’s output [
2
].
Shape-changing UIs oer the benets of physicality and software
dynamics, and enable, e.g., the unique support of adaptative aor-
dance [
2
]. Examples include a slider of variable length to compro-
mise between performance and footprint [19] (Figure 2a).
One possible approach to implement shape-changing UIs is based
on modules. Modules are small building block elements, which can
rearrange or be rearranged to form the UI. Such modules can be
robotic, i.e., microelectromechanical modules embedding compu-
tational capabilities (e.g., Catoms [
94
], MBlocks [
80
]). We do not
consider in this paper modular approaches that do not allow for
3D reconguration (e.g., Lumen [
76
], InFORM [
26
]) or non-self-
actuated ones (e.g., GaussBricks [
58
]). Modules self-actuated in
3D allow for a larger range of shapes and for system’s shape out-
put [
41
,
86
]. In addition, while non-self-actuated modules can reach
small sizes (e.g., 2mm [
34
]), it is dicult for users to accurately ma-
nipulate very small modules. Previous work explored applications
enabled by such modules, e.g., computer-assisted design on the
physical prototype itself [
91
], animation directly with a physical
prop [
64
,
83
], adapting the interface to the users’ needs [
82
]. This
modular approach promises to allow in the future for a large range
of shape-change, while enabling high resolution and scalability [
2
].
The problem we tackle in this paper is to nd the range of usable
sizes of modules for modular shape-changing physical UIs (PUIs) (i.e.
a physical object that the user can manipulate, as opposed to graph-
ical UIs. Previous work stress the importance of this problem [
54
].
A rst reason is that the smallest possible modules are desirable,
ICMI ’21, October 18–22, 2021, Montréal, QC, Canada Laura Pruszko, et al.
as they allow for instance for the highest possible resolution for
the system’s shape output modality [
54
,
77
]. A second reason for
this importance is that the largest possible modules is desirable to
ease their fabrication. In particular, the size of modules was found
to have the most impact on other technological design parameters,
such as the computational and interaction capabilities embedded
in each module [
77
]. As a consequence, the size of modules suf-
fers from a trade-o between usability and technological feasibility.
This paper aims at providing a very rst measure of usability to
inform future technological development about this trade-o.
A diculty –and motivation– of this problem lies in the fact that
current technology is not yet ready for very small self-actuated
modules. Figure 2 shows the modular shape-changing systems
that can recongure in 3D and that reached a size smaller than
20cm [
77
]. For instance, the smallest modules studied in HCI are the
9mm DynaBlocks [
91
] (Figure 2). However, Dynablocks are not self-
actuated, preventing the system to actuate their 3D display unless
they lie on the dedicated table. Major technological advances are
done in the autonomous robotics community, where the currently
most advanced robotic modules being assembled are the
3.6mm
Catoms [
94
] (Figure 2). Figure 2 shows that it is not possible yet to
prototype high-resolution modular shape-changing UIs to conduct
user studies. It is therefore dicult for the research community
to know how far technological research should go to decrease the
size of modules in order to provide usable interaction, as we do not
know how much the size of the modules impact the interaction.
To solve this diculty and begin to study the impact of the size
of modules on interaction, our approach in this paper is to hypoth-
esize that this technology is close to enable shape-changing PUIs.
Based on this hypothesis, we study modular non-shape-changing
UIs with non-actuated mock-ups of modules based on the current
design of this technology. This allows us to start evaluating the
impact of the size of modules between shape-changing phases. As
HCI researchers, we believe user-centered design should drive tech-
nological development as early as possible [42].
Another diculty –and grand challenge [
2
]– lies in the fact that
the shape of the modules can also impact user experience with mod-
ular UIs [
2
,
77
]. We chose to start our research with quasi-spherical
modules, as this shape (1) is the most promising for miniaturization
of actuated robotic modules [
74
] and (2) is expected to have the
most impact on interaction due to the friction caused at the surface
of the UI by the gaps between modules.
First, as shown in Figure 2, while the research community mostly
explores cubic modules, the robotics community found that quasi-
spherical shape is a promising approach to decrease the size of
modules [
74
]. Indeed, the technologies used to actuate cubic imple-
mentations are dicult to miniaturize (e.g., electro-magnets [
104
],
mechanical hinges [
64
] or ywheels [
80
]), whereas the rounded
edges provided by quasi-spherical modules lower the energy as nec-
essary for miniature actuation [
74
]. Quasi-spherical shape is there-
fore a promising research direction for modular shape-changing
UIs. For this reason, the impact of this shape should be assessed.
Second, we expect a high dierence in user experience between
small and large rounded modules, as the size and number of gaps
between rounded modules are correlated with the size of the mod-
ules. In contrast, we expect a lower dierence in user experience
between small and large cubic modules, as only the number of gaps
is correlated with the size of the modules. While we expect quasi-
spherical modules to hinder performance of common tasks such as
target acquisition and pursuit, we do not know to which extent their
size will hinder performance. This paper addresses this important
problem to re-center technological development on users.
We present the results of two studies assessing four sizes of
modules constituting a PUI (namely a slider): a plain, smooth slider
(modules of size ~
0 mm
) and three sliders made of
1 mm
,
2.5 mm
and
5 mm
mock-up modules. We nd that (1) the largest mod-
ules (
5 mm
) signicantly impairs performance for a pursuit task
and subjective perception for pursuit and target acquisition tasks,
(2) performance increases when the size of modules decreases but
(3) users reportedly enjoyed the haptic feedback of
1 mm
and
2.5 mm
modules (e.g., to brake, change speed and change direc-
tion). These results provide deeper understanding on how users
interact with a device made of small, rounded modules, to inform
future technological goals for shape-changing PUIs.
2 RELATED WORK
The study of the size of rounded modules constituting a PUI builds
on previous work in PUIs, haptics and psychophysics.
Haptic perception of PUIs. The haptic feedback of an interface
involves two kinds of perception: First, the kinesthetic perception
is the perception of forces and positions by the muscles and joints
[
18
]. Second, the tactile perception is the perception of the physical
properties of an object when in contact with the user’s skin [
67
].
When interacting with a PUI, both kinesthetic and tactile percep-
tions come into play. For example, users will perceive the shape
and roughness of a slider (tactile perception) and the force required
to move its cursor (kinesthetic perception).
Previous works distinguish two dimensions of texture: micro (or
ne) and macro [
67
]. Despite dierences in perceptual mechanisms,
micro- and macro-roughness are hard for users to separate [
67
].
Prior work studied roughness with groove widths of 1mm up to
8mm [
50
], which is in the order of magnitude of the size of the gaps
between the modules constituting future recongurable PUIs (e.g.,
[
26
,
64
,
91
,
101
]). On the kinesthetic level, surfaces with groove
width of 5mm or 8mm give a kinesthetic feeling of detents –i.e. the
feeling that the cursor is locked at a position, often spring-loaded to
make the cursor move toward this position [
8
]– but are also on the
roughness spectrum [
50
]. Surfaces with smaller groove width, and
even micro-levels of roughness, may not give the same feeling of de-
tents but still involve kinesthetic perception through proprioception
[
102
]. Although we expect the size of modules to fall into the macro-
roughness category (i.e., textures with spatial periods >
200 µm
) [
53
],
the surface of the modules may present micro-roughness character-
istics (i.e., textures with spatial periods <
200 µm
). We conducted a
pilot comparing a smooth manufactured slider and the smooth 3D
printed one of Figure 1 showing micro-roughness to evaluate how it
might impact the results of our experiment. We found no dierence.
For this reason, this paper focuses on the macro-roughness caused
by modules.
Tactile discrimination of textures. The haptics literature recog-
nizes three fundamental dimensions of tactile perception: warmness
(e.g., wood vs. metal), hardness (e.g., metal vs. foam) and roughness
Impact of the Size of Modules on Target Acquisition and Pursuit for Future Modular Shape-changing PUIs ICMI ’21, October 18–22, 2021, Montréal, QC, Canada
Shape
DynaBlock [91]
PolyBot [100]
CONRO [16] Stochastic 3D [98]
M-TRAN [52]
ATRON [68]
SuperBot [84]
Miche [30]
Robot Pebbles [29]
Blinky Blocks [49]
Molecube [56] lineFORM [65]
M− Blocks [80]
BitDrones [31]
Cubimorph [82]
GridDrones [13]
SMORES [46]
Sambot [92]
Catom [94]
0
50
100
150
2000 2005 2010 2015 2020
Size of modules (mm)
Round
Community
Robotics
Other
Rectangular HCI
a
b
Figure 2: Left: Evolution over time of the size and shape of modules used for modular 3D shape-changing UIs <20mm (data
source and references [77] found at http://molecularhci.imag.fr). Right (a): In the future, modular PUIs could be able to recon-
gure their shape. For instance here, a slider could become larger [19]. Right (b): The simulated robotic movement of small,
interchangeable modules currently developed that could allow such changes in shape [73].
(e.g., sand paper is rough vs. plain paper is smooth) [
67
]. We particu-
larly review prior work on roughness, as small modules impact the
roughness of the interface. When users manipulate PUIs such as a
slider, their ngers can (1) stick to the surface of the object, such as
when ngers grasp the cursor of the slider. Their ngers can also (2)
move tangentially to the surface of the object, such as when ngers
swipe a surface. Their ngers can also (3) move a proxy (also called
a probe) tangentially to the surface of the object, such as when
manipulating an object on a surface or the cursor on its support
surface. We thus report results on tactile perception through direct
touch with (1) static and (2) moving ngers (e.g., [
37
,
63
,
72
,
89
]),
and through moving probes (3) (e.g., [50, 53, 102, 103]).
The perception of roughness is primarily driven by changes in
groove width [
50
,
55
]. Human tactile perception of textures with
moving ngers heavily depends on the size of the gaps between
bumps [
14
], rather than the size of the bumps. The deeper the
skin of the nger can penetrate into the groove between tactile
modules, the rougher the user perceives the surface. Specically,
the perception of roughness (1) increases with the distance between
the raised tactile modules, (2) increases with the height of the raised
modules and, (3) modestly decreases when the size of the raised
modules increases [89].
Minimal size of gaps between modules. Users better perceive
roughness when they perform tangential movements [
61
,
89
], such
as when they perform target acquisition or pursuit with a PUI.
The tactile discrimination of two points is more accurate when the
ngers move across the surface of the object: the nger’s static two-
points discrimination normal ability is <
6 mm
and its moving two-
points discrimination normal ability is <2 mm [39, pp.146-148][22,
pp.118-121]. We will not study gaps >
6 mm
because humans are
able to discriminate two points >
6 mm
away from each other, even
when their nger is static; Also, robots are already research at sizes
<
6 mm
. Additionally, increasing the surface roughness impacts the
kinesthetic perception of friction [
67
,
89
]. When the gaps and bumps
of a rough surface are repeated in a 2D texture, such as with abrasive
paper [
38
], the thresholds are even lower. Through static contact
of the nger, users perceive textures if the size of their modules
is >
0.1 mm
[
38
]. With moving contact, human tactile perception is
possible for modules of sizes <
0.1 mm
[
38
], extending to amplitudes
as small as
10 nm
for wrinkled surfaces [
88
]. Based on this we will
study modules as small as allowed by standard 3D printers available
to the general public. Overall, prior work primarily focus on the
perception of roughness, rather than on its impact on another,
primary task, such as target acquisition and pursuit, as we do in
this paper.
Tactile perception through a probe. The literature on haptics and
psychophysics studied the perception of roughness through a probe,
i.e. the tip of a stylus (e.g., [
50
,
53
,
102
,
103
]). E.g., with a tip of size
3 mm
, humans are able to perceive gaps of size as small as
0.125 mm
.
However, this work does not study task performance on a textured
surface through a probe, nor through a textured probe or through
a probe with a dierent and larger shape. Manipulating two rough
objects on each others, e.g. a rough slider cursor on a rough railing,
results in multiple contact points covering a larger contact surface.
We do not know yet how these previous results would apply to
larger objects and how they relate to common tasks such as target
acquisition and pursuit. This is however important to assess to help
the design of future modular shape-changing PUIs. Prior work also
proposed tools to capture and generate textures through an active
haptic probe and tested the correct perception of the captured then
regenerated textures [
20
]. In contrast, our work aims at assessing
the impact on task performance of the texture resulting from small
modules constituting the PUI.
Tactile perception of roughness and PUIs. Previous work explored
the change of tactile perception of the roughness of the UI. For
instance, prior work explores vibrotactile feedback for physical
dials [
10
]. Such feedback can easily be perceived even with a static
contact of the nger. It is therefore dierent from the feedback
provided by rough surfaces of recongurable PUIs made of small
modules. Rough devices with prominent textures are an incentive
to touch [
63
] and can be used in the future design of perceived
aordances [
66
] for recongurable PUIs. To our knowledge, the
literature in PUI and tangible UIs did not explore roughness be-
yond its subjective perception and abstract qualities [
40
]. Tactile
perception to perform grasp and lift manipulation has been studied
[32]: the shape of the object in the region of contact with the skin
has a strong eect on grip and lift tasks. However, we do not know
yet how the roughness of the surface of the recongurable PUI
ICMI ’21, October 18–22, 2021, Montréal, QC, Canada Laura Pruszko, et al.
–caused by its small modules– aects the performance of common
interactive tasks such as target acquisition and pursuit.
Kinesthetic perception and PUIs. Modular implementation of PUIs
will cause friction, as the size of modules is correlated with fric-
tion [
89
]. Previous work explored the change of the kinesthetic
perception through programmable friction and movement resis-
tance. Programmable friction improves overall movement time and
the time to stop the cursor after entering the target with a tactile
graphical slider [
57
]. Variable movement resistance on a physi-
cal slider such as with the FireFader [
7
] improves performance of
eye-free interaction [
59
] or the kinesthetic visualization and ma-
nipulation of sound [
93
]. In prior work, programmable friction and
movement resistance were applied in the area of the target. On the
contrary, in this paper we are interested in studying the impact of
the roughness, and its resulting friction, on the whole surface of
the PUI. Previous work also explored the impact of the resolution
of pin-based display on a target pursuit task [
79
]. However, this
work did not explore target acquisition tasks. Another limitation is
the single, rather large (
15 mm
) size of pins compared to current,
smooth and plain PUIs. In addition, the proposed implementation
of a slider does not build on top of current widespread sliders. This
makes dicult the generalization of this result to widespread sliders
made of smaller round modules arranged in 3D such as the ones of
Figure 1 and 2ab.
On the one hand, while prior work inform on the human tactile
perception according to the size of the gaps between the modules
constituting the recongurable PUI, it primarily focuses on direct
touch. When interaction through probes was studied, these probes
are far from widespread PUIs such as dials and sliders, and the
tasks performed by participants are far from the tasks performed
with PUIs. On the other hand, prior work studied the impact of
friction and movement resistance on target acquisition tasks that are
widespread in PUIs. However, the friction and movement resistance
are applied in the area of the targets only, on the contrary to future
recongurable PUIs made of small modules. We could not nd
previous work assessing how the size of modules might impact the
performance or the user experience for such tasks.
3 EXPERIMENTS
We conduct two user experiments assessing the impact of the size
of rounded modules constituting a PUI when using a physical slider.
One experiment uses a target acquisition task and the other one
a pursuit task. For each task, we assess the performance, and the
perceived task diculty and workload.
3.1 Method
3.1.1 Apparatus. We chose to study a plain, smooth slider like
current sliders as a control condition, and sliders made of modules
of three dierent sizes. First,
1 mm
is the smallest size we achieved
with current mainstream 3D printers. Second, we chose to study
5 mm
spheres, as higher sizes importantly impair the global shape
of the PUI. Finally, we chose an intermediate size of
2.5 mm
to
uniformly study performance in the 0-5mm range.
We 3D printed the sliders. The STL models, printers and materials
used for the prototypes are provided in the supplementary material
for replicability. Our cursor is based on a manufactured, widespread
one (Figure 1) [
23
]. In order to provide a passive haptic feedback
comparable to current sliders, the arrangement of the spheres pro-
vides a hole for the rail, in which the cursor moves (Figure 1). To
allow for a comfortable manipulation of the slider, we provided a
support surface around the mock-up (Figure 1). To log the location
of the cursor, we mounted the prototypes of modular cursor and
casing of modular sliders on a slide potentiometer, as shown in
the supplementary video. For accuracy, we chose a
100 mm
-long
Bourns PTB0143-2010BPB103 [
1
] over other sizes and manufac-
turers, as, for instance, PaletteGear
60 mm
-long sliders could not
be as accurate as
100 mm
-long ones [
19
]. We connected the poten-
tiometers to a computer through a Teensy 3.6 board running the
Firmata rmware. The experimental software runs at 60 fps. We
choose the Teensy 3.6 board for its high performance and its ability
to handle 12 bits of resolution when reading the potentiometer. The
supplementary video shows how the prototypes work and how
participants used the prototypes during the experiment.
3.1.2 Participants. We recruited 24 participants from the local
campus. Participants only took part in one of the experiments.
Twelve participants performed the target acquisition task (4 female,
8 males, M = 31.33 y.o., SD = 5.69) and the other twelve participants
performed the pursuit task (4 female, 8 males, M = 30.8 y.o., SD =
7.6). All were occasional users of physical sliders.
3.1.3 Experimental design. Target acquisition and pursuit tasks are
common in the literature for physical interaction [
25
,
44
,
79
,
95
]. The
within-subject independent variable was the size of the modules:
Smooth, i.e. no modules (plain 3D printed), spheres of
1 mm
,
spheres of
2.5 mm
, and spheres of
5 mm
. The dierent levels of
sizes were presented in permuted order (latin square).
3.1.4 Procedure. The experimenter veries the tactile sensitivity of
the dominant hand’s thumb, index and middle ngers through two-
points discrimination test [
22
,
39
] and the Semmes-Weinstein mono-
lament aesthesiometer [
6
]. There was no sensitivity impairment
susceptible to skew the results. The participant then sits in front of a
laptop running the experimental software and is presented with the
rst prototype, on the side of their dominant hand. The participant
adjusts the location of the box to comfortably reach the highest and
lowest values of the slider. The experimenter secures the location of
the box with blu tack. The participant then performs the task. For
both tasks, the user controls a
1 mm
-thick (4.4px) on-screen cursor
with their dominant hand, along an
180 mm
-long (800px) vertical
axis displayed on-screen (Figure 3). As the potentiometer measures
100 mm
, the on-screen cursor moves faster than the physical one
with a CD gain of 1.8. The transfer function is a linear mapping.
The participants do not wear earplugs to cancel the noise caused
by the friction: We focus in this very rst study on the measure of
the impact of the size of the modules. As suggested in [
69
], we leave
for future work the identication of the factors causing this impact,
such as, e.g., the friction, or the noise caused by this friction.
Target acquisition task.
A blue rectangle acting as a target is
shown on the vertical axis (Figure 3). The widths (W) and distances
(D) of the targets are randomized (W=8px or W=35px, D=90px or
D=400px) and result in three rounded Fitts’ Index of Diculty (ID):
1.8, 3.6 and 5.7. The target becomes green when the cursor is within
the target. The participant validates by pressing the space key of a
Impact of the Size of Modules on Target Acquisition and Pursuit for Future Modular Shape-changing PUIs ICMI ’21, October 18–22, 2021, Montréal, QC, Canada
Experimenter
Participant
Camera 1
Camera 2
Prototype
Target acquisition Target pursuit
static target
participant’s
cursor
moving target
Figure 3: Close-ups of the UI of the experimental software
for the target acquisition (left) and pursuit tasks (right).
laptop with their non-dominant hand. If they validate outside the
target, we log an error. The task ends when the target is successfully
acquired. After completing a training with 5 targets, the participant
is required to successfully validate 5 blocks of 26 targets. As the dis-
tance (D) is the distance between two consecutive targets, the rst
and last targets are dummies to re-position the cursor at the center
of the screen. Thus, the rst and last targets are discarded during
analysis. Participants can take breaks between blocks. Instructions
require the participant to be as quick and accurate as possible. To
account for speed-accuracy trade-o, the experimental software
compute the error rate and instruct the participant to speed up
(error rate < 4 %) or slow down (> 4 %) between blocks.
Target pursuit task.
A
1 mm
-thick (4.4px) green target is mov-
ing up and down the vertical axis (Figure 3). The target moves at
constant speed in a linear motion and darts o at pseudo-random
intervals (as in [
44
,
79
]). Instructions require the white cursor, con-
trolled by the user, to be as close as possible to the green target.
Participants complete two trials of
90 s
, as in [
25
,
44
,
79
]. They can
take a break between trials.
After completion of the task, the participant lls a questionnaire.
The procedure is repeated with each of the four sizes of modules.
At the end of the experiment, the participant lls a nal question-
naire. We conducted interviews after the task using the explicitation
interview method [96]. The experiment lasts around 30 minutes.
3.1.5 Data collection and analysis. For the target acquisition task,
we collect movement time, error rate and distance from target
center to the cursor. For the target pursuit task, we collect the
pursuit error for every frame during the experiment. For both,
we gather subjective feedback: First, we gather overall perceived
diculty with a Single Ease Question [
97
] (Overall, how dicult or
easy did you nd this task?). The participant answers on a seven
points Likert scale. Second, we gather perceived workload with
a NASA Task Load Index (TLX) questionnaire which assesses six
items: mental, physical and temporal demand, performance, eort,
and frustration [
36
]. We use the unweighted version which is as
reliable but faster to complete [
35
,
62
]. We also record synchronized
videos of the participants’ arms, hands and eyes. The experimenter
took note of used limbs and user behaviour.
We chose to compare the impact of the size of the modules on our
chosen measures as in previous work [
21
,
51
]. Other widespread
experimental procedures (e.g., two-alternative forced choice [
9
],
free-magnitude estimation [
38
], or just-noticeable dierence [
43
])
require to many sizes of spheres or only assess the impact on a
subjective measure.
Movement Time (ms) Error rate (%) SEQ
Smooth 1441.29±623.23 2.28±14.95 6.01±1.16
1 mm 1471.04±660.18 3.59±18.61 5.25±1.36
2.5 mm 1506.15±594.66 3.12±17.39 5.34±1.15
5 mm 1508.95±737.33 1.94±13.81 4.92±1.5
Table 1: Mean movement time (ms), error rate (%) and score
of the Single Ease Question for target acquisition (±stdev).
3.2 Results - Target acquisition
In this section, we report our results on the impact of the size
of modules constituting the slider on the performance of users
performing a target acquisition task. We particularly report in table
1 our results on movement time, error rate and SEQ score.
3.2.1 Performance. While our participants were asked to aim at a
4% error rate, we wanted to account for any residual bias in speed-
accuracy trade-o. A common approach is to use the eective
width [
90
]. However, in order to measure actual movement times,
we chose in our experimental procedure to measure the movement
time until successful target acquisition as in [
27
]. In this case, we can
directly analyze movement times and error rates separately [17].
Movement Time.
Figure 4 shows the geometric mean of the
movement time [
85
] for each size of modules, with its
95 %
con-
dence interval. Since we obtain a p-value of 0.021 with Shapiro-
Wilk’s test on our data, we cannot assume the normality of our data.
We therefore perform an Aligned Rank Transform (ART) [
99
]. We
conducted a one-way analysis of variance (ANOVA) on the aligned
data and showed no signicant dierence between the movement
times of the dierent sizes of modules (F(3,33)=1.81, p>.05).
Error Rate.
Since we obtain a p-value of 0.0003 with Shapiro-
Wilk’s test on our data, we cannot assume the normality of our data.
We therefore align our data with the ART method. We conducted
a one-way analysis of variance (ANOVA) on the aligned data and
showed no signicant dierence between the error rate of the
dierent sizes of modules (F(3,33)=1.67, p>.05).
3.2.2 estionnaires. As questionnaires provide Likert scale data,
we used the ART method to align their data, e.g., the mean SEQ
scores. We conducted a one-way ANOVA to compare the impact
of the size of modules on the aligned SEQ score. As there was sig-
nicance (F(3,33)=4.0637, p<.05), we carried out post-hoc pairwise
comparisons based on Kenward-Roger approximation. Participants
perceived the task as signicantly more dicult with the slider
made of 5 mm modules compared to the smooth slider (p<.05).
We conducted a one-way ANOVA to compare the impact of
the size of modules on each aligned TLX item score and found no
signicant dierences between the TLX items of the dierent sizes
of modules. The raw scores for the SEQ and TLX questionnaires
can be found in the supplementary material.
3.3 Results - Target pursuit
In this section, we report our results on the impact of the size of
modules on the performance of users performing a target pursuit
task. We particularly report in table 2 our results on pursuit error
and SEQ score.
ICMI ’21, October 18–22, 2021, Montréal, QC, Canada Laura Pruszko, et al.
Figure 4: Left: Movement time (in ms) for each size of modules. Points show geometric means and error bars show the 95 %
condence interval. Right: Error rate (%) for each size of modules. Points show means and error bars show the 95 % condence
interval. All conditions had the same slider range on the device (8 cm) and on the screen (18 cm).
Pursuit error (%) Pursuit error (mm) SEQ
Smooth 1.27±10.31 2.29±18.56 5.42±1.08
1 mm 1.32±10.21 2.38±18.38 4.33±1.37
2.5 mm 1.42±10.06 2.56±18.11 4.5±1.24
5 mm 1.68±10.39 3.02±18.70 3.83±1.80
Table 2: Median pursuit error (% and mm) and mean score of
the Single Ease Question for pursuit (±stdev).
3.3.1 Performance. Figure 5 shows the median pursuit error for
each size of modules. The median of the pursuit error is a better
indicator of the data central tendency [
19
]. Since we obtain a p-
value < 0.001 with Barlett’s test on our data, we cannot assume the
normality of our data. We aligned our data using ART [
99
]. We
then conducted a one-way analysis of variance (ANOVA) on the
aligned data that showed a signicance dierence (F(3,33)=17.879,
p<.01). We carried out post-hoc pairwise comparisons based on
Kenward-Roger approximation [
47
] and showed two signicant
increase in pursuit error: First, the slider made of
5 mm
modules
lead to signicantly more error compared to all other sizes (smooth
(p<.0001),
1 mm
modules (p<.0001) and
2.5 mm
modules (p<.01)).
Second, the slider made of
2.5 mm
modules lead to signicantly
more error compared to the smooth slider (p<.05).
3.3.2 estionnaires. We followed the same statistical approach
as for the questionnaire data of the target acquisition task. We com-
pare the impact of the size of modules on the aligned SEQ score. As
there was signicance (F(3,33)=2.978, p<.05), we carried out post-
hoc pairwise comparisons. The task was perceived as signicantly
more dicult with the slider made of
5 mm
modules compared
to the smooth slider (p<.05). We compare the impact of the size
of modules on each aligned TLX item score. There was a signi-
cant impact of the size of modules on the perceived performance
(F(3, 33)=3.9714, p<.05). Post-hoc pairwise comparisons show that
the slider made of
5 mm
modules signicantly decreased the per-
ceived performance compared to the smooth slider (p<.01). Results
from the nal questionnaire comparing the size showed that all
users ranked the slider made of
5 mm
modules last for the SEQ
as well as for the physical demand (“The task was physically de-
manding. ) and frustration items (“I felt discouraged/irritated while
performing the task. ) of the TLX. The raw scores for the SEQ and
TLX questionnaires are available in the supplementary material.
4 DISCUSSION
4.1 Size of modules & target acquisition
Experimental results showed that, even though increasing the size
of the modules did not signicantly impact performance, the task
was perceived as signicantly harder with the
5 mm
modules. This
is consistent with participant feedback from interviews. Only two
participants expressed liking the slider made of
5 mm
modules
(P14 "the large notches helped me to be accurate", P17 "I like the [slider
made of
5 mm
modules ] because I think I didn’t make any error
with it"). The other participants expressed disliking the slider made
of
5 mm
modules, as they reportedly perceived it as frustrating:
it is too loud (e.g., P21 “the noise is annoying") and is physically
uncomfortable to operate (e.g., P20 “it was annoying to move it" and
P24 “it’s just to the touch, without moving it. I don’t like big spheres.").
Overall, participants agreed that all sliders were similar but had
distinctive qualities. On the one hand, the smooth slider allows for
faster movements and accurate adjustments. However, participants
complained that it led them to overshoot more. On the other hand,
the sliders made of
1 mm
to
2.5 mm
modules provide haptic
feedback which helped to prevent overshoots and accurately stop
on the target. Even though participants reportedly enjoyed the
sliders made of
1 mm
to
2.5 mm
modules, most still rated the
smooth slider as easiest to complete the task with. Our hypothesis is
that even though the smooth slider causes overshoots, its increased
speed of movement to initially reach the target compensates for the
adjustment time required by the overshoot. Participants expressed
that, despite their preferences, they did not think any size of mod-
ules signicantly impacted their performance. For example, P20
said that with the slider made of
5 mm
modules, "completing the
task was no problem, I just didn’t like it".
Previous work found that programmable friction improves the
time to stop the cursor after entering the target and improves over-
all movement time for non-physical graphical sliders controlled
through direct touch [
57
]. This is consistent with our results with
physical sliders: in prior work [
57
], as friction was only applied
on the target, users benet from (1) the faster movements enabled
by low friction to initially reach the target and (2) the higher fric-
tion on the target to reduce overshoots. Our results are important
as the HCI community takes a growing interest in new shapes,
materials and textures [
2
,
40
,
75
], in particular for recongurable
PUIs made of small modules (e.g., [
26
,
31
,
54
,
58
,
64
,
91
,
101
]). Fu-
ture PUIs can leverage materials that can change roughness or
size of modules, such as wrinkling materials (e.g., [
28
,
71
]) with
changing zero-crossing physical feature [
81
]. It is important that
future materials are able to change roughness only in dedicated,
programmable locations corresponding to potential targets, e.g. to
prevent overshoots while ensuring a comfortable interaction, as
shown in our target acquisition experiment.
Impact of the Size of Modules on Target Acquisition and Pursuit for Future Modular Shape-changing PUIs ICMI ’21, October 18–22, 2021, Montréal, QC, Canada
Figure 5: Pursuit error (% of the slider range). Points show medians and error bars show the 95 % condence interval. All
conditions had the same slider range on the device (8 cm) and on the screen (18 cm).
4.2 Size of modules & target pursuit
The size of modules impacts target pursuit more than acquisition.
Participants were less accurate at pursuit as the size increased. As
the size of modules is correlated with friction [
89
], we explained this
result with the varying friction between the cursor and the rail of
the slider. During interviews, participants commented on the rela-
tionship between friction, size of modules and performance. On the
one hand participants mentioned preferring the plain slider for con-
tinuously pursuing the target, whereas it caused overshoots when
the target darted o. On the other hand, participants mentioned
that larger modules provided haptic feedback. They preferred the
sliders made of
1 mm
to
2.5 mm
modules when the target darted
o, and said these rendered braking, changing speed and changing
direction easier, thus preventing overshoots. This is consistent with
the results and user feedback of the target acquisition task.
Even though participants liked the haptic feedback provided by
modules, the performance suered from the large “bumps” made
by
5 mm
modules. When the cursor is in a gap between bumps, it
is dicult to move because of static friction [
78
] and because of the
deep and steep slope for the cursor to climb the next module. Our
results thus support previous work advocating for lower friction [
3
,
4
,
12
,
78
], probably because the friction resulting from modules is
high compared to prior work that found no eect [15].
4.3 Grasps observed
Physical sliders allow for several grasps. In particular, when the
nger is on the central concave curve of the cursor (“curve” grasp,
Figure 6), users can simultaneously control several sliders [
5
, p.140].
Thus, supporting this “curve” grasp is important. To explore the
suitability of a modular implementation for this grasp, we report
how participants grasp the cursor in the video recordings of both
target acquisition and pursuit tasks. We observed between and
within subject variation in grasps. Participants rst tried multiple
grasps for a same slider. We only report here the grasp they settle on
after the rst target acquisition block or pursuit trial. We categorize
the grasps we found in the three following categories of common
grasps for physical sliders [23] (Figure 6, left):
Curve
is the grasp of the cursor with a single nger placed in the
cursor’s central concave curve;
Pinch
is the grasp of the cursor between two ngers, often the
index and the thumb;
Curve+Pinch
is the grasp combining a pinch between one or two
nger(s), often the middle and the thumb, and another nger, often
the index, in the cursor’s concave curve.
The smooth slider and the slider with
1 mm
modules present
close performance for all three grasps (Figure 6). The slider with
2.5 mm
modules caused a higher error with the Curve grasp for
the pursuit task (yellow in Figure 6). The slider with
5 mm
mod-
ules (orange in Figure 6) do not allow for the Curve grasp at all,
while the Curve+Pinch grasp caused high error. The Pinch grasp
showed the best performance, although its accuracy with
5 mm
modules is still far from the other sizes for the pursuit task.
We hypothesize that this is due to the interaction between the
size of the modules and the contact forces applied with the Curve
grasp. E.g., P5 said about the smooth slider and the slider with
1 mm
modules, that they are a lot smoother so I have to be aware, I
have to change the force that I’m applying. P9 said I think that it’s
the force too, that I’m not pressing on it the same way so it grips less.
With the smooth slider, increasing the contact force between the
cursor and the railing does not impair the performance since they
are both smooth. With
1 mm
modules, it seems that the bumps
and gaps are small enough to accommodate the contact forces of
aCurve grasp without impairing the performance. With 2.5 mm
modules, the Curve grasp impairs the performance for the pursuit
task. Applying rather tangential forces (either with the Curve+Pinch
or Pinch grasps) results in better performance. With
5 mm
mod-
ules, applying contact forces renders the movement of the cursor
challenging. E.g., P5 performs the Curve grasp: "[on the smooth
slider] this is easy with one nger. [switches to
1 mm
modules]
easy [switches to
2.5 mm
modules] still easy [switches to
5 mm
modules, the cursor gets stuck] no, not easy". The Curve+Pinch grasp
shows little improvement for
5 mm
modules. For this reason, most
participants used the Pinch grasp: 6/12 for target acquisition and
10/12 for pursuit. Large size of modules (
5 mm
) prevents rich
grasps opportunities, and causes lower performance and lower
satisfaction.
4.4 Limitations of the studies
These studies are limited to the impact of the macro-texture of quasi-
spherical modules. Depending on the future actual implementation,
their micro-texture, the strength of their bond, the friction at their
surface, their shape, their arrangements, etc. might also have an
impact on the results. In particular, we left the micro-texture apart
for these studies, as a pilot experiment comparing a standard slider
and a 3D printed one did not show any dierence in performance.
However, it is possible that the micro-texture further impairs the
interaction when applied at the surface of small modules. We plan
to assess this in a future separate experiment when the rst im-
plementations are available. The studies presented here are a rst
and necessary step towards informing the design of small robotic
modules for user interaction with modular PUIs. To our knowledge,
these are the rst studies assessing the design parameters of small
modules constituting PUIs.
ICMI ’21, October 18–22, 2021, Montréal, QC, Canada Laura Pruszko, et al.
Smooth
1mm
2.5mm
5mm Neverused Neverused
Smooth
1mm
2.5mm
5mm
Smooth
1mm
2.5mm
5mm
1.0 1.5 2.0 2.5
PURSUITERROR(% OF THERANGE)
Targetpursuit Targetacquisition
1200 1400 1600 1800 2000
MOVEMENT TIME (MS)
Curve
Curve+
Pinch
Pinch
GRASP SIZE OF
MODULES
Figure 6: Target pursuit error and target acquisition time for each grasp, across sizes of modules. Boxplots show the median
through the bold central line, the rst and third quartile through the box and the extremums through the whiskers.
4.5 Implications
Modular interfaces will enable the change of the physical shape to
support multiple interaction modalities [
11
] for users’ input and/or
system’s output [
2
]. We evaluated the impact of the size of rounded
modules on target acquisition and pursuit with a slider. Our results
already inform the design of rounded modules constituting modular
interfaces when they take the shape of a slider.
First, an alternative for future modular recongurable PUIs in the
future are cubic modules, although such small and actuated cubic
modules were never proposed so far. The smallest cubic modules we
know to allow for 3D shapes are Dynablocks [
91
] (
9 mm
). However
these are not self-actuated. We hypothesize that cubic modules
would lead to higher target acquisition and pursuit performance
compared to rounded ones. Indeed, only the number of gaps is
correlated with the size of cubic modules. In contrast, the size and
number of gaps between rounded modules are correlated with the
size of the rounded modules. Our results can then be used as a worst
case scenario for the performance with cubic modules. Future work
should assess the precise impact of the size of cubic modules on
user experience.
Second, a modular recongurable PUI can change to other PUIs
besides sliders. Other widespread PUIs are push buttons and di-
als [
5
,
48
]. We expect the impact of the size of rounded modules
to be comparable for dials, as the rotation of the dial on the sup-
port surface will also cause friction. We hypothesize that the con-
tact area between a dial and its support surface (between 78 and
707 mm2
[
5
, p.136]) is comparable to the contact area between the
cursor of the slider and its support surface, as recommended [
23
]
and studied in the paper (
231 mm2
). Our results also oer a rst
measure for future modular tangible UIs oering object translation
on a table [
86
]: These objects could change shape in the future
thanks to a modular implementation, allowing to change their af-
fordance [
60
] or the shape of their support surface depending on
the scenario of use [
33
]. Such tangible objects are currently made
of large modules [
54
], but prior work called for decreasing the size
of these modules [
54
], in order to allow for larger shape resolu-
tion [
81
]. Future technological development can now consider user
performance to decide for a target size.
5 CONCLUSION
We explored the impact of the size of rounded modules for target
acquisition and pursuit with future modular PUIs. For target acqui-
sition, our experimental results showed that
1 mm
to
2.5 mm
modules do not signicantly impact performance. However, our
participants perceived the task as signicantly more dicult when
using a slider made of
5 mm
modules. Participants found the
5 mm
modules useful to reduce overshoots in eye-free target ac-
quisition. For target pursuit, pursuit accuracy decreases when the
size of modules increases, whereas users reportedly prefer sliders
with
1 mm
to
2.5 mm
modules to reach darting-o targets. We
found that the higher the size of the modules, the fewer the grasps
user can perform on the slider’s cursor. Impairing the rich grasping
possibilities of PUIs can prevent, e.g., their abilities to act as external
cognitive aids [
45
,
87
]. Our studies show that the research should
consider how the design of these emerging UIs impacts not only
user performance and subjective perception, but also the grasping
possibilities. Our studies show the importance of addressing the
grand challenge [2] of developing <5 mm modules.
Future work on recongurable PUIs made of modules <
5 mm
also include the study of their arrangements in order to change
tactile and kinesthetic feedback only in dedicated, programmable
locations corresponding to potential targets. Such a material would
entice users to touch the interface on particular locations, and
prevent overshoots while ensuring a comfortable interaction.
ACKNOWLEDGMENTS
This work was supported by the French National Research Agency
(ANR-15-CE23-0011) and the LabEx PERSYVAL-Lab (ANR-11-LABX-
0025-01) funded by the French program Investissement d’avenir.
We would like to thank Thomas Achard for providing the data
in Figure 2. We are grateful for the feedback we received from
colleagues and reviewers.
REFERENCES
[1]
2019. PTB Series - Low Prole Slide Potentiometer. https://www.bourns.com/
docs/product-datasheets/ptb.pdf ?sfvrsn=11289f1_7.
[2]
Jason Alexander, Anne Roudaut, Jürgen Steimle, Kasper Hornbæk, Miguel
Bruns Alonso, Sean Follmer, and Timothy Merritt. 2018. Grand Challenges
Impact of the Size of Modules on Target Acquisition and Pursuit for Future Modular Shape-changing PUIs ICMI ’21, October 18–22, 2021, Montréal, QC, Canada
in Shape-Changing Interface Research. In the CHI 2018 Conference. New York,
New York, USA, 1–14. https://doi.org/10.1145/3173574.3173873
[3]
Myroslav Bachynskyi, Gregorio Palmas, Antti Oulasvirta, Jürgen Steimle, and
Tino Weinkauf. 2015. Performance and Ergonomics of Touch Surfaces: A
Comparative Study Using Biomechanical Simulation. In Proceedings of the
33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul,
Republic of Korea) (CHI ’15). ACM, New York, NY, USA, 1817–1826. https:
//doi.org/10.1145/2702123.2702607
[4]
Ravin Balakrishnan and I. Scott MacKenzie. 1997. Performance Dierences in
the Fingers, Wrist, and Forearm in Computer Input Control. In Proceedings of
the ACM SIGCHI Conference on Human Factors in Computing Systems (Atlanta,
Georgia, USA) (CHI ’97). ACM, New York, NY, USA, 303–310. https://doi.org/
10.1145/258549.258764
[5]
Thomas B. Baumann, K. 2001. User Interface Design of Electronic Appliances.
Londo: CRC Press. https://doi.org/10.1201/b12625
[6] J. A. Bell. 1978. Sensibility evaluation. Rehabilitation of the hand (1978).
[7]
Edgar Berdahl and Alexandros Kontogeorgakopoulos. 2013. The Firefader:
Simple, Open-Source, and Recongurable Haptic Force Feedback for Musicians.
Comput. Music J. 37, 1 (March 2013), 23–34. https://doi.org/10.1162/COMJ_a_
00166
[8]
Edgar Berdahl, Julius O. Smith, Stefan Weinzierl, and Gunter Niemeyer. 2013.
Force-Sensitive Detents Improve User Performance for Linear Selection Tasks.
6, 2 (2013), 206–216. https://doi.org/10.1109/TOH.2012.55
[9]
Sanjiv K. Bhatia, Vasudevan Lakshminarayanan, Ashok Samal, and Grant V.
Welland. 1995. Human Face Perception in Degraded Images. J. Visual Commu-
nication and Image Representation 6 (1995), 280–295.
[10]
Andrea Bianchi, Ian Oakley, Jong Keun Lee, Dong Soo Kwon, and Vassilis
Kostakos. 2011. Haptics for tangible interaction: a vibro-tactile prototype. (2011),
2.
[11]
Jullien Bouchet, Laurence Nigay, and Thierry Ganille. 2004. ICARE Software
Components for Rapidly Developing Multimodal Interfaces. In Proceedings of
the 6th International Conference on Multimodal Interfaces (State College, PA,
USA) (ICMI ’04). Association for Computing Machinery, New York, NY, USA,
251–258. https://doi.org/10.1145/1027933.1027975
[12]
James V Bradley. 1969. Optimum Knob Diameter. Human Factors: The Journal
of the Human Factors and Ergonomics Society 11, 4 (Aug. 1969), 353–360. https:
//doi.org/10.1177/001872086901100406
[13]
Sean Braley, Calvin Rubens, Timothy Merritt, and Roel Vertegaal. 2018. Grid-
Drones: A Self-Levitating Physical Voxel Lattice for Interactive 3D Surface
Deformations. In Proceedings of the 31st Annual ACM Symposium on User Inter-
face Software and Technology (UIST ’18). Association for Computing Machinery,
New York, NY, USA, 87–98. https://doi.org/10.1145/3242587.3242658
[14]
C J Cascio and K Sathian. 2001. Temporal cues contribute to tactile perception
of roughness. The Journal of neuroscience : the ocial journal of the Society for
Neuroscience 21, 14 (July 2001), 5289–5296. http://eutils.ncbi.nlm.nih.gov/entrez/
eutils/elink.fcgi?dbfrom=pubmed&id=11438604&retmode=ref&cmd=prlinks
[15]
Géry Casiez, Nicolas Roussel, Romuald Vanbelleghem, and Frédéric Giraud.
2011. Surfpad: Riding Towards Targets on a Squeeze Film Eect. In Proceedings
of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver,
BC, Canada) (CHI ’11). ACM, New York, N Y, USA, 2491–2500. https://doi.org/
10.1145/1978942.1979307
[16]
Andres Castano, Wei-Min Shen, and Peter Will. 2000. CONRO: Towards Deploy-
able Robots with Inter-Robots Metamorphic Capabilities. Autonomous Robots 8,
3 (June 2000), 309–324. https://doi.org/10.1023/A:1008985810481
[17]
Olivier Chapuis and Pierre Dragicevic. 2011. Eects of motor scale, visual
scale, and quantization on small target acquisition diculty. 18, 3 (2011), 1–32.
https://doi.org/10.1145/1993060.1993063
[18]
Andy Cockburn and Stephen Brewster. 2005. Multimodal feedback for the
acquisition of small targets. 48, 9 (2005), 1129–1150. https://doi.org/10.1080/
00140130500197260
[19]
Céline Coutrix and Cédric Masclet. 2015. Shape-Change for Zoomable T UIs:
Opportunities and Limits of a Resizable Slider. In Human-Computer Interaction
– INTERACT 2015. Springer International Publishing, Cham, 349–366. https:
//doi.org/10.1007/978-3- 319-22701- 6_27
[20]
H. Culbertson, J. Unwin, and K. J. Kuchenbecker. 2014. Modeling and Rendering
Realistic Textures from Unconstrained Tool-Surface Interactions. 7, 3 (2014),
381–393. https://doi.org/10.1109/TOH.2014.2316797 Conference Name: IEEE
Transactions on Haptics.
[21]
Andrea De Cesarei and Maurizio Codispoti. 2010. Eects of Picture Size Reduc-
tion and Blurring on Emotional Engagement. PLOS ONE 5, 10 (10 2010), 1–7.
https://doi.org/10.1371/journal.pone.0013399
[22] A. Lee Dellon. 1997. Somato Sensory Testing and Rehabilitation.
[23]
European Committee for Standardization. 2008. Safety of machinery - Er-
gonomics requirements for the design of displays and control actuators - Part 3:
Control actuators.
[24]
Brian Feeny, Ardeshir Guran, Nikolaus Hinrichs, and Karl Popp. 1998. A
Historical Review on Dry Friction and Stick-Slip Phenomena. Applied
Mechanics Reviews 51, 5 (05 1998), 321–341. https://doi.org/10.1115/1.3099008
arXiv:https://asmedigitalcollection.asme.org/appliedmechanicsreviews/article-
pdf/51/5/321/4748671/321_1.pdf
[25]
George W Fitzmaurice and William Buxton. 1997. An empirical evaluation of
graspable user interfaces. In CHI. ACM Press, New York, New York, USA, 43–50.
https://doi.org/10.1145/258549.258578
[26]
Sean Follmer, Daniel Leithinger, Alex Olwal, Akimitsu Hogge, and Hiroshi Ishii.
2013. inFORM: dynamic physical aordances and constraints through shape
and object actuation. In Proceedings of the 26th annual ACM symposium on
User interface software and technology - UIST ’13 (St. Andrews, Scotland, United
Kingdom, 2013). ACM Press, 417–426. https://doi.org/10.1145/2501988.2502032
[27]
Clifton Forlines, Daniel Wigdor, Chia Shen, and Ravin Balakrishnan. 2007.
Direct-touch vs. mouse input for tabletop displays. In Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems - CHI ’07 (San Jose, Califor-
nia, USA, 2007). ACM Press, 647–656. https://doi.org/10.1145/1240624.1240726
[28]
Jack Forman, Taylor Tabb, Youngwook Do, Meng-Han Yeh, Adrian Galvin,
and Lining Yao. 2019. ModiFiber: Two-Way Morphing Soft Thread Actuators
for Tangible Interaction. In Proceedings of the 2019 CHI Conference on Human
Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association
for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/
3290605.3300890
[29]
Kyle Gilpin, Ara Knaian, and Daniela Rus. 2010. Robot pebbles: One centime-
ter modules for programmable matter through self-disassembly. In 2010 IEEE
International Conference on Robotics and Automation. IEEE, Anchorage, AK,
2485–2492. https://doi.org/10.1109/ROBOT.2010.5509817
[30]
Kyle Gilpin, Keith Kotay, Daniela Rus, and Iuliu Vasilescu. 2008. Miche: Modular
Shape Formation by Self-Disassembly. The International Journal of Robotics Re-
search 27, 3-4 (March 2008), 345–372. https://doi.org/10.1177/0278364907085557
[31]
Antonio Gomes, Calvin Rubens, Sean Braley, and Roel Vertegaal. 2016. Bit-
Drones: Towards Using 3D Nanocopter Displays As Interactive Self-Levitating
Programmable Matter. In Proceedings of the 2016 CHI Conference on Human
Factors in Computing Systems (San Jose, California, USA) (CHI ’16). ACM, New
York, NY, USA, 770–780. https://doi.org/10.1145/2858036.2858519
[32]
Antony W Goodwin and Heather E Wheat. 2004. Sensory signals in neural
populations underlying tactile perception and manipulation. Annual review of
neuroscience 27 (2004), 53–77. https://doi.org/10.1146/annurev.neuro.26.041002.
131032
[33]
Jens Emil Grønbæk, Majken Kirkegaard Rasmussen, Kim Halskov, and Mar-
ianne Graves Petersen. 2020. KirigamiTable: Designing for Proxemic Tran-
sitions with a Shape-Changing Tabletop. In Proceedings of the 2020 CHI Con-
ference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI
’20). Association for Computing Machinery, New York, NY, USA, 1–15. https:
//doi.org/10.1145/3313831.3376834
[34]
Hongri Gu, Quentin Boehler, Daniel Ahmed, and Bradley J. Nelson. 2019. Mag-
netic quadrupole assemblies with arbitrary shapes and magnetizations. Science
Robotics 4, 35 (Oct. 2019), eaax8977. https://doi.org/10.1126/scirobotics.aax8977
[35]
Sandra G Hart and Moett Field. 2006. NASA-TASK LOAD INDEX (NASA-TLX);
20 YEARS LATER. (2006), 5.
[36]
Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX
(Task Load Index): Results of Empirical and Theoretical Research. In Advances
in Psychology. Vol. 52. Elsevier, 139–183. https://doi.org/10.1016/S0166- 4115(08)
62386-9
[37]
Mark Holliins, Richard Faldowski, Suman Rao, and Forrest Young. 1993. Percep-
tual dimensions of tactile surface texture: A multidimensional scaling analysis.
54, 6 (1993), 697–705. https://doi.org/10.3758/BF03211795
[38]
M Hollins and S R Risner. 2000. Evidence for the duplex theory of tac-
tile texture perception. Perception & psychophysics 62, 4 (May 2000), 695–
705. http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&
id=10883578&retmode=ref&cmd=prlinks
[39]
James M. D. Hunter, Evelyn P. T. Mackin, and Anne M. S. Callahan. 1995. Reha-
bilitation of the Hand: Surgery and Therapy (4 ed.). Vol. 1.
[40]
Jörn Hurtienne and Daniel Reinhardt. 2017. Texture Metaphors and Tangible
Interaction: No Smooth Relationship?. In Proceedings of the Tenth International
Conference on Tangible, Embedded, and Embodied Interaction - TEI ’17 (Yokohama,
Japan, 2017). ACM Press, 79–87. https://doi.org/10.1145/3024969.3024986
[41]
Hiroshi Ishii, Dávid Lakatos, Leonardo Bonanni, and Jean-Baptiste Labrune.
2012. Radical atoms: beyond tangible bits, toward transformable materials.
Interactions 19, 1 (Jan. 2012), 38–51. https://doi.org/10.1145/2065327.2065337
[42]
ISO9241-210:2019 2019. Ergonomics of Human-System Interaction – Part 210:
Human-Centered Design for Interactive Systems. Standard. International Organi-
zation for Standardization, Geneva, CH.
[43]
Sungjune Jang, Lawrence H. Kim, Kesler Tanner, Hiroshi Ishii, and Sean Follmer.
2016. Haptic Edge Display for Mobile Tactile Interaction. In Proceedings of
the 2016 CHI Conference on Human Factors in Computing Systems (San Jose,
California, USA) (CHI ’16). ACM, New York, NY, USA, 3706–3716. https://doi.
org/10.1145/2858036.2858264
[44]
Yvonne Jansen, Pierre Dragicevic, and Jean-Daniel Fekete. 2012. Tangible remote
controllers for wall-size displays. In CHI ’12: Proceedings of the SIGCHI Conference
ICMI ’21, October 18–22, 2021, Montréal, QC, Canada Laura Pruszko, et al.
on Human Factors in Computing Systems. ACM, New York, New York, USA, 2865–
2874. https://doi.org/10.1145/2207676.2208691
[45]
Yvonne Jansen, Pierre Dragicevic, and Jean-Daniel Fekete. 2013. Evaluating the
Eciency of Physical Visualizations. In Proceedings of the SIGCHI Conference
on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association
for Computing Machinery, New York, NY, USA, 2593–2602. https://doi.org/10.
1145/2470654.2481359
[46]
Gangyuan Jing, Tarik Tosun, Mark Yim, and Hadas Kress-Gazit. 2018. Accom-
plishing high-level tasks with modular robots. Autonomous Robots 42, 7 (Oct.
2018), 1337–1354. https://doi.org/10.1007/s10514- 018-9738-1
[47]
Michael Kenward and James Roger. 2009. An improved approximation to the
precision of xed eects from restricted maximum likelihood. Computational
Statistics & Data Analysis 53 (05 2009), 2583–2595. https://doi.org/10.1016/j.
csda.2008.12.013
[48]
Hyunyoung Kim, Céline Coutrix, and Anne Roudaut. 2018. KnobSlider: Design
of a Shape-Changing UI for Parameter Control. In CHI ’18. https://dl.acm.org/
citation.cfm?id=3173913
[49]
Brian T. Kirby, Michael Ashley-Rollman, and Seth Copen Goldstein. 2011. Blinky
Blocks: A Physical Ensemble Programming Platform. In CHI ’11 Extended Ab-
stracts on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI
EA ’11). ACM, New York, NY, USA, 1111–1116. https://doi.org/10.1145/1979742.
1979712
[50]
Roberta L. Klatzky, Susan J. Lederman, Cheryl Hamilton, Molly Grindley, and
Robert H. Swendsen. 2003. Feeling textures through a probe: Eects of probe
and surface geometry and exploratory factors. 65, 4 (2003), 613–631. https:
//doi.org/10.3758/BF03194587
[51]
Hendrik Knoche, John D. McCarthy,and M. Angela Sasse. 2008. How lowcan you
go? The eect of low resolutions on shot types in mobile TV. Multimedia Tools
and Applications 36, 1 (01 Jan 2008), 145–166. https://doi.org/10.1007/s11042-
006-0076- 5
[52]
Haruhisa Kurokawa, Kohji Tomita, Akiya Kamimura, Satoshi Murata, Yuzuru
Terada, and Shigeru Kokaji. 2006. Distributed Metamorphosis Control of a
Modular Robotic System M-TRAN. In Distributed Autonomous Robotic Systems
7, Maria Gini and Richard Voyles (Eds.). Springer Japan, Tokyo, 115–124. https:
//doi.org/10.1007/4-431- 35881-1_12
[53]
Michael A Lawrence, Ryo Kitada, Roberta L Klatzky, and Susan J Lederman.
2007. Haptic Roughness Perception of Linear Gratings via Bare Finger or Rigid
Probe. 36, 4 (2007), 547–557. https://doi.org/10.1068/p5746
[54]
Mathieu Le Goc, Lawrence H Kim, Ali Parsaei, Jean-Daniel Fekete, Pierre Drag-
icevic, and Sean Follmer. 2016. Zooids: Building Blocks for Swarm User Inter-
faces. In UIST’16. 1–13. https://doi.org/10.1145/2984511.2984547
[55]
S. J. Lederman and R. L. Klatzky. 2009. Haptic perception: A tutorial. 71, 7 (2009),
1439–1459. https://doi.org/10.3758/APP.71.7.1439
[56]
Jong-Hyun Lee and Chang Wook Ahn. 2012. Evolutionary self-assembling
swarm robots using genetic programming. In 2012 Proceedings of SICE Annual
Conference (SICE). 807–811.
[57]
Vincent Levesque, Louise Oram, Karon MacLean, Andy Cockburn, Nicholas D.
Marchuk, Dan Johnson, J. Edward Colgate, and Michael A. Peshkin. 2011.
Enhancing physicality in touch interaction with programmable friction. In
Proceedings of the 2011 annual conference on Human factors in computing
systems - CHI ’11 (Vancouver, BC, Canada, 2011). ACM Press, 2481. https:
//doi.org/10.1145/1978942.1979306
[58]
Rong-Hao Liang, Liwei Chan, Hung-Yu Tseng, Han-Chih Kuo, Da-Yuan Huang,
De-Nian Yang, and Bing-Yu Chen. 2014. GaussBricks: Magnetic Building Blocks
for Constructive Tangible Interactions on Portable Displays. In Proceedings of the
SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario,
Canada) (CHI ’14). Association for Computing Machinery, New York, NY, USA,
3153–3162. https://doi.org/10.1145/2556288.2557105
[59]
Lars Lischke, Paweł W. Woźniak, Sven Mayer, Andreas Preikschat, and Morten
Fjeld. 2017. Using Variable Movement Resistance Sliders for Remote Discrete
Input. In Proceedings of the Interactive Surfaces and Spaces on ZZZ - ISS ’17
(Brighton, United Kingdom, 2017). ACM Press, 116–125. https://doi.org/10.
1145/3132272.3134135
[60]
Rafael Morales González, Caroline Appert, Gilles Bailly, and Emmanuel Pietriga.
2016. TouchTokens: Guiding Touch Patterns with Passive Tokens. In Proceedings
of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose,
California, USA) (CHI ’16). Association for Computing Machinery, New York,
NY, USA, 4189–4202. https://doi.org/10.1145/2858036.2858041
[61]
J.W. Morley, A.W. Goodwin, and I. Darian-Smith. 1983. Tactile discrimination of
gratings. 49, 2 (1983). https://doi.org/10.1007/BF00238588
[62]
W. F. Moroney, D. W. Biers, F. T. Eggemeier, and J. A. Mitchell. 1992. A compari-
son of two scoring procedures with the NASA task load index in a simulated
ight task. In Proceedings of the IEEE 1992 National Aerospace and Electronics
Conference. 734–740 vol.2. https://doi.org/10.1109/NAECON.1992.220513
[63]
Hikaru Nagano, Shogo Okamoto, and Yoji Yamada. 2014. Haptic Invitation of
Textures: Perceptually Prominent Properties of Materials Determine Human
Touch Motions. 7, 3 (2014), 345–355. https://doi.org/10.1109/TOH.2014.2321575
[64]
Ken Nakagaki, Artem Dementyev, Sean Follmer, Joseph A. Paradiso, and Hiroshi
Ishii. 2016. ChainFORM: A Linear Integrated Modular Hardware System for
Shape Changing Interfaces. In Proceedings of the 29th Annual Symposium on
User Interface Software and Technology (Tokyo, Japan, 2016-10-16) (UIST ’16).
Association for Computing Machinery, 87–96. https://doi.org/10.1145/2984511.
2984587
[65]
K Nakagaki, S Follmer, and H Ishii. 2015. LineFORM: Actuated Curve Interfaces
for Display, Interaction, and Constraint. In Proceedings of the 28th Annual ACM
Conference on Human Factors in Computing. https://doi.org/10.1145/2807442.
2807452
[66]
Donald A. Norman. 1999. Aordance, Conventions, and Design. Interactions 6,
3 (May 1999), 38–43. https://doi.org/10.1145/301153.301168
[67]
Shogo Okamoto, Hikaru Nagano, and Yoji Yamada. 2013. Psychophysical
Dimensions of Tactile Perception of Textures. 6, 1 (2013), 81–93. https:
//doi.org/10.1109/TOH.2012.32
[68]
Esben Hallundbæk Østergaard, Kristian Kassow, Richard Beck, and Henrik Hau-
top Lund. 2006. Design of the ATRON lattice-based self-recongurable robot.
Autonomous Robots 21, 2 (01 Sep 2006), 165–183. https://doi.org/10.1007/s10514-
006-8546- 1
[69]
Antti Oulasvirta and Kasper Hornbæk. 2016. HCI Research as Problem-Solving.
In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
(San Jose, California, USA) (CHI ’16). Association for Computing Machinery,
New York, NY, USA, 4956–4967. https://doi.org/10.1145/2858036.2858283
[70]
Irene Parada, Vera Sacristán, and Rodrigo I. Silveira. 2016. A new meta-
module for ecient reconguration of hinged-units modular robots. In 2016
IEEE International Conference on Robotics and Automation (ICRA). 5197–5202.
https://doi.org/10.1109/ICRA.2016.7487726
[71]
Sylvain Pauchet, Jean-Luc Vinot, Catherine Letondal, Alexandre Lemort, Claire
Lavenir, Timothée Lecomte, Stéphanie Rey, Valentin Becquet, and Guillaume
Crouzet. 2019. Multi-Plié: A Linear Foldable and Flattenable Interactive Display
to Support Eciency, Safety and Collaboration. In Proceedings of the 2019 CHI
Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk)
(CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13.
https://doi.org/10.1145/3290605.3300384
[72]
Delphine Picard, Catherine Dacremont, Dominique Valentin, and Agnès Gi-
boreau. 2003. Perceptual dimensions of tactile textures. 114, 2 (2003), 165–184.
https://doi.org/10.1016/j.actpsy.2003.08.001
[73]
Benoit Piranda and Julien Bourgeois. 2018. Designing a quasi-spherical module
for a huge modular robot to create programmable matter. Autonomous Robot
Journal, Special Issue: Distributed Robotics: From Fundamentals to Applications
42, 8 (2018), 1619–1633. https://doi.org/10.1007/s10514- 018-9710-0
[74]
Benoît Piranda and Julien Bourgeois. 2018. Geometrical Study of a Quasi-
spherical Module for Building Programmable Matter. In Distributed Autonomous
Robotic Systems, Roderich Groß, Andreas Kolling, Spring Berman, Emilio Fraz-
zoli, Alcherio Martinoli, Fumitoshi Matsuno, and Melvin Gauci (Eds.). Vol. 6.
Springer International Publishing, 387–400. https://doi.org/10.1007/978-3-319-
73008-0_27 Series Title: Springer Proceedings in Advanced Robotics.
[75]
Ingrid Maria Pohl and Lian Loke. 2013. Touch toolkit: a method to convey
touch-based design knowledge and skills. In Proceedings of the 8th International
Conference on Tangible, Embedded and Embodied Interaction - TEI ’14 (Munich,
Germany, 2013). ACM Press, 251–258. https://doi.org/10.1145/2540930.2540957
[76]
Ivan Poupyrev, Tatsushi Nashida, Shigeaki Maruyama, Jun Rekimoto, and Ya-
sufumi Yamaji. 2004. Lumen: interactive visual and shape display for calm
computing. In ACM SIGGRAPH 2004 Emerging technologies on - SIGGRAPH ’04.
ACM Press, Los Angeles, California, 17. https://doi.org/10.1145/1186155.1186173
[77]
Laura Pruszko, Céline Coutrix, Yann Laurillau, Benoît Piranda, and Julien Bour-
geois. 2021. Molecular HCI: Structuring the Cross-disciplinary Space of Modular
Shape-changing User Interfaces. Proceedings of the ACM on Human-Computer
Interaction (2021). https://doi.org/10.1145/3461733
[78]
Christopher Richard and Mark Cutkosky. 2000. The Eects of Real and Computer
Generated Friction on Human Performance in a Targeting Task. In Proceedings
of the ASME IMECE Symposium on Haptic Interfaces for Virtual Environments
and Teleoperator Systems. ASME, 1101–1108.
[79]
Simon Robinson, Céline Coutrix, Jennifer Pearson, Juan Rosso, Matheus Fer-
nandes Torquato, Laurence Nigay, and Matt Jones. 2016. Emergeables: De-
formable Displays for Continuous Eyes-Free Mobile Interaction. In CHI ’16:
Proceedings of the 2016 CHI Conference on Human Factors in Computing Sys-
tems. Swansea University, ACM, New York, New York, USA, 3793–3805. https:
//doi.org/10.1145/2858036.2858097
[80]
John W. Romanishin, Kyle Gilpin, Sebastian Claici, and Daniela Rus. 2015. 3D
M-Blocks: Self-reconguring robots capable of locomotion via pivoting in three
dimensions. In 2015 IEEE International Conference on Robotics and Automation
(ICRA). IEEE, Seattle, WA, USA, 1925–1932. https://doi.org/10.1109/ICRA.2015.
7139450
[81]
Anne Roudaut, Abhijit Karnik, Markus Löchtefeld, and Sriram Subramanian.
2013. Morphees: toward high "shape resolution" in self-actuated exible mobile
devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems - CHI ’13 (Paris, France, 2013). ACM Press, 593. https://doi.org/10.1145/
Impact of the Size of Modules on Target Acquisition and Pursuit for Future Modular Shape-changing PUIs ICMI ’21, October 18–22, 2021, Montréal, QC, Canada
2470654.2470738
[82]
A Roudaut, D Krusteva, M McCoy, A Karnik,K Ramani, and S Subramanian. 2016.
Cubimorph: Designing modular interactive devices. In 2016 IEEE International
Conference on Robotics and Automation (ICRA. IEEE, 3339–3345. https://doi.
org/10.1109/ICRA.2016.7487508
[83]
Calvin Rubens, Sean Braley, Julie Torpegaard, Nicklas Lind, Roel Vertegaal,
and Timothy Merritt. 2020. Flying LEGO Bricks: Observations of Children
Constructing and Playing with Programmable Matter. In Proceedings of the Four-
teenth International Conference on Tangible, Embedded, and Embodied Interaction
(TEI ’20). Association for Computing Machinery, New York, NY, USA, 193–205.
https://doi.org/10.1145/3374920.3374948
[84]
Behnam Salemi, Mark Moll, and Wei-min Shen. 2006. SUPERBOT: A Deployable,
Multi-Functional, and Modular Self-Recongurable Robotic System. In 2006
IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, Beijing,
China, 3636–3641. https://doi.org/10.1109/IROS.2006.281719
[85]
Je Sauro and James R. Lewis. 2010. Average Task Times in Usability Tests:
What to Report?. In Proceedings of the SIGCHI Conference on Human Factors
in Computing Systems (Atlanta, Georgia, USA) (CHI ’10). ACM, New York, NY,
USA, 2347–2350. https://doi.org/10.1145/1753326.1753679
[86]
Orit Shaer and Eva Hornecker. 2010. Tangible User Interfaces: Past, Present and
Future Directions. Vol.3. Now Publishers Inc. https://doi.org/10.1561/1100000026
[87]
Orit Shaer, Nancy Leland, Eduardo H Calvillo Gámez, and Robert J K Jacob. 2004.
The TAC paradigm: specifying tangible user interfaces. Personal and Ubiquitous
Computing 8, 5 (2004), 359–369. https://doi.org/10.1007/s00779-004-0298- 3
[88]
Lisa Skedung, Martin Arvidsson, Jun Young Chung, Christopher M. Staord,
Birgitta Berglund, and Mark W. Rutland. 2013. Feeling Small: Exploring the
Tactile Perception Limits. 3, 1 (2013), 2617. https://doi.org/10.1038/srep02617
[89]
Allan M. Smith, C. Elaine Chapman, Mélanie Deslandes, Jean-Sébastien Langlais,
and Marie-Pierre Thibodeau. 2002. Role of friction and tangential force variation
in the subjective scaling of tactile roughness. 144, 2 (2002), 211–223. https:
//doi.org/10.1007/s00221-002- 1015-y
[90]
R. William Soukore and I. Scott MacKenzie. 2004. Towards a standard for
pointing device evaluation, perspectives on 27 years of Fitts’ law research in
HCI. 61, 6 (2004), 751–789. https://doi.org/10.1016/j.ijhcs.2004.09.001
[91]
Ryo Suzuki, Junichi Yamaoka, Daniel Leithinger, Tom Yeh, Mark D. Gross, Yoshi-
hiro Kawahara, and Yasuaki Kakehi. 2008. Dynablock: Dynamic 3D Printing for
Instant and Reconstructable Shape Formation. In The 31st Annual ACM Sym-
posium on User Interface Software and Technology - UIST ’18 (Berlin, Germany,
2018). ACM Press, 99–111. https://doi.org/10.1145/3242587.3242659
[92]
Wenshuai Tan, Hongxing Wei, and Bo Yang. 2018. SambotII: A New Self-
Assembly Modular Robot Platform Based on Sambot. Applied Sciences 8, 10
(2018). https://doi.org/10.3390/app8101719
[93]
Atau Tanaka and Adam Parkinson. 2016. Haptic Wave: A Cross-Modal Interface
for Visually Impaired Audio Producers. Association for Computing Machinery,
New York, NY, USA, 2150–2161. https://doi.org/10.1145/2858036.2858304
[94]
Pierre Thalamy, Benoit Piranda, and Julien Bourgeois. 2020. 3D Coating Self-
Assembly for Modular Robotic Scaolds. In 2020 IEEE/RSJ International Con-
ference on Intelligent Robots and Systems (IROS). IEEE, Las Vegas, NV, USA,
11688–11695. https://doi.org/10.1109/IROS45743.2020.9341324
[95]
Philip Tuddenham, David Kirk, and Shahram Izadi. 2010. Graspables Revisited:
Multi-touch vs. Tangible Input for Tabletop Displays in Acquisition and Ma-
nipulation Tasks. In Proceedings of the SIGCHI Conference on Human Factors
in Computing Systems (Atlanta, Georgia, USA) (CHI ’10). ACM, New York, NY,
USA, 2223–2232. https://doi.org/10.1145/1753326.1753662
[96] Pierre Vermersch. 2018. The explicitation interview.
[97]
Werner Wetzlinger, Andreas Auinger, and Michael Doeringer. 2014. Comparing
Eectiveness, Eciency, Ease of Use, Usability and User Experience When
Using Tablets and Laptops. Lecture Notes in Computer Science 8517, 402–412.
https://doi.org/10.1007/978-3- 319-07668- 3_39
[98] Paul White, Victor Zykov, Josh Bongard, and Hod Lipson. 2005. Three Dimen-
sional Stochastic Reconguration of Modular Robots. In Robotics: Science and
Systems I. Robotics: Science and Systems Foundation. https://doi.org/10.15607/
RSS.2005.I.022
[99]
Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011.
The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only
Anova Procedures. In Proceedings of the SIGCHI Conference on Human Factors in
Computing Systems (Vancouver, BC, Canada) (CHI ’11). ACM, New York, NY,
USA, 143–146. https://doi.org/10.1145/1978942.1978963
[100]
M. Yim, D.G. Du, and K.D. Roufas. 2000. PolyBot: a modular recongurable
robot. In Proceedings 2000 ICRA. Millennium Conference. IEEE International Con-
ference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065),
Vol. 1. IEEE, San Francisco, CA, USA, 514–520. https://doi.org/10.1109/ROBOT.
2000.844106
[101]
Shigeo Yoshida, Yuqian Sun, and Hideaki Kuzuoka. 2020. PoCoPo: Handheld
Pin-based Shape Display for Haptic Rendering in Virtual Reality. (2020), 13.
[102]
Takashi Yoshioka, James C Craig, Graham C Beck, and Steven S Hsiao. 2011.
Perceptual Constancy of Texture Roughness in the Tactile System. (2011), 9.
[103]
Takashi Yoshioka and Julia Zhou. 2009. Factors Involved in Tactile Texture
Perception Through Probes. (2009), 22.
[104]
Victor Zykov, Efstathios Mytilinaios, Mark Desnoyer, and Hod Lipson. 2007.
Evolved and Designed Self-Reproducing Modular Robotics. IEEE Transactions on
Robotics 23, 2 (April 2007), 308–319. https://doi.org/10.1109/TRO.2007.894685
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Shape-changing User Interfaces attract growing interest in Human-Computer Interaction. Modular robotics offer a great opportunity for their implementation. However, the current theoretical and technical advances of modular robotics are fragmented and little centered on the user. To unify existing work and center future research on the user, we perform a systematic literature review enabling us to build a unifying space for the design of modular shape-changing user interfaces. Our aim is to bridge the gap between HCI and robotics. Towards this aim, we conduct a thorough cross-disciplinary survey to propose: 1) a set of design properties at the scale of the interface (macro-scale) and at the scale of the modules (micro-scale) and 2) the impact of these properties on each other. We relate properties of different domains and identify inconsistencies to structure the design space. This paper can be used to describe and compare existing modular shape-changing UIs and generate new design ideas by building upon knowledge from robotics and HCI.
Conference Paper
Full-text available
This paper addresses the self-reconfiguration problem in large-scale modular robots for the purpose of shape formation for object representation. It aims to show that this process can be accelerated without compromising on the visual aspect of the final object, by creating an internal skeleton of the shape using the previously introduced sandboxing and scaffolding techniques, and then coating this skeleton with a layer of modules for higher visual fidelity. We discuss the challenges of the coating problem, introduce a basic method for constructing the coating of a scaffold layer by layer, and show that even with a straightforward algorithm, our scaffolding and coating combo uses much fewer modules than dense shapes and offers attractive reconfiguration times. Finally, we show that it could be a strong alternative to the construction of dense shapes using traditional self-reconfiguration algorithms.
Preprint
Full-text available
A core challenge in tabletop research is to support transitions between individual activities and team work. Shape-changing tabletops open up new opportunities for addressing this challenge. However, interaction design for shape-changing furniture is in its early stages - so far, research has mainly focused on triggering shape-changes, and less on the actual interface transitions. We present KirigamiTable - a novel actuated shape-changing tabletop for supporting transitions in collaborative work. Our work builds on the concept of Proxemic Transitions, considering the dynamic interplay between social interactions, interactive technologies and furniture. With KirigamiTable, we demonstrate the potential of interactions for proxemic transitions that combine transformation of shape and digital contents. We highlight challenges for shape-changing tabletops: initiating shape and content transformations, cooperative control, and anticipating shape-change. To address these challenges, we propose a set of novel interaction techniques, including shape-first and content-first interaction, cooperative gestures, and physical and digital preview of shape-changes.
Conference Paper
Full-text available
We present the design concept of an accordion-fold interactive display to address the limits of touch-based interaction in airliner cockpits. Based on an analysis of pilot activity, tangible design principles for this design concept are identified. Two resulting functional prototypes are explored during participatory workshops with pilots, using activity scenarios. This exploration validated the design concept by revealing its ability to match pilot responsibilities in terms of safety, efficiency and collaboration. It provides an efficient visual perception of the system for real-time collaborative operations and tangible interaction to strengthen the perception of action and to manage safety through anticipation and awareness. The design work and insights enabled to specify further our needs regarding flexible screens. They also helped to better characterize the design concept as based on continuity of a developed surface, predictability of aligned folds and pleat face roles, embodied interactive properties, and flexibility through affordable reconfigurations.
Conference Paper
Full-text available
Despite thin-line actuators becoming widely adopted in different Human-Computer Interaction (HCI) contexts, including integration into fabrics, paper art, hinges, soft robotics, and human hair, accessible line-based actuators are very limited beyond shape memory alloy (SMA) wire and motor-driven passive tendons. In this paper, we introduce a novel, yet simple and accessible, line-based actuator. ModiFiber is a twisted-then-coiled nylon thread actuator with a silicone coating. This composite thread actuator exhibits unique two-way reversible shrinking or twisting behaviors triggered by heat or electrical current (i.e., Joule heating). ModiFiber is soft, flexible, safe to operate and easily woven or sewn, hence it has a great potential as an embedded line-based actuator for HCI purposes. In this paper, we explain the material mechanisms and manufacturing approaches, followed by some performance tests and application demonstrations.
Conference Paper
We present the results of haptic interaction with real and virtual Coulomb friction. Tests were conducted to explore the effects of various types of friction on human performance in a Fitts-style targeting task. The results indicate that haptically rendered friction affects subject performance in a manner quite similar to that of real physical friction. Furthermore, moderate low-stiction friction tends to improve subject performance both in terms of speed and accuracy. High-stiction friction, however, degrades subjects performance, especially in terms of speed.
Article
Magnetic dipole-dipole interactions govern the behavior of magnetic matter across scales from micrometer colloidal particles to centimeter magnetic soft robots. This pairwise long-range interaction creates rich emergent phenomena under both static and dynamic magnetic fields. However, magnetic dipole particles, from either ferromagnetic or paramagnetic materials, tend to form chain-like structures as low-energy configurations due to dipole symmetry. The repulsion force between two magnetic dipoles raises challenges for creating stable magnetic assemblies with complex two-dimensional (2D) shapes. In this work, we propose a magnetic quadrupole module that is able to form stable and frustration-free magnetic assemblies with arbitrary 2D shapes. The quadrupole structure changes the magnetic particle-particle interaction in terms of both symmetry and strength. Each module has a tunable dipole moment that allows the magnetization of overall assemblies to be programmed at the single module level. We provide a simple combinatorial design method to reach both arbitrary shapes and arbitrary magnetizations concurrently. Last, by combining modules with soft segments, we demonstrate programmable actuation of magnetic metamaterials that could be used in applications for soft robots and electromagnetic metasurfaces.