Content uploaded by Antonio Gomes
Author content
All content in this area was uploaded by Antonio Gomes on Jan 18, 2015
Content may be subject to copyright.
PaperFold: Evaluating Shape Changes for Viewport
Transformations in Foldable Thin-Film Display Devices
Antonio Gomes
Human Media Lab
Queen’s University
Kingston, ON, Canada, K7L3N6
gomes@cs.queensu.ca
Roel Vertegaal
Human Media Lab
Queen’s University
Kingston, ON, Canada, K7L3N6
roel@cs.queensu.ca
ABSTRACT
In this paper, we investigate the use of shape changes in a
multi-segmented mobile device for triggering viewport
transformations in its graphical interface. We study
PaperFold, a foldable device with reconfigurable thin-film
electrophoretic display tiles. PaperFold enables users to
attach, reorient and fold displays in a mobile form factor that
is thin and lightweight even when fully collapsed. We
discuss how our design was informed by a participatory
study that resulted in 14 preferred shape changes. In a
subsequent study, we asked users to rank the utility of shape
changes for triggering common view operations in map and
text editing applications. Results suggest participants were
able to attribute specific view operations as automated
responses to folding, attaching, reorienting or detaching
displays. Collated or full screen views were preferred when
users collocated two displays. When adding a third display,
alternative views such as toolbars or a list of apps were
suggested. Showing 3D views was strongly associated with
folding PaperFold segments into a three dimensional
structure.
ACM Classification Keywords
H.5.2. [Information interfaces and Presentation]: User
Interfaces - Interaction Styles, Evaluation/Methodology.
Author Keywords
Human Factors; Organic User Interfaces; Flexible Displays;
Foldable User Interfaces.
INTRODUCTION
For reasons of portability, mobile devices have displays that
are limited in size and constrained by weight. Despite
increases in resolution, such size restrictions have resulted in
sequential interaction paradigms for mobile platforms, in
which apps are opened and closed one by one. By contrast,
the display size of paper documents can be modulated easily
through folding, tearing or combining multiple page
elements. Such properties allow paper documents to be
navigated and organized more efficiently, allowing
concurrent access to multiple documents [19]. Paper is also
very thin, durable and lightweight, perceived advantages in
the mobile design space. As such, the development of
electronic paper computers that adopt certain qualities or
metaphors of interacting with paper documents has been an
enduring research goal [9,19]. While initial research was
aimed at mimicking form factors or interaction techniques of
paper documents using rigid display devices [1,8], in recent
years, progress has been made towards developing
computers made of flexible E-Ink displays as thin and
lightweight as paper [4,22].
In this paper, we investigate how thin-film, paper-like
electrophoretic mobile devices [4,5,12,22] might adopt
dynamic modulation of screen real estate through folding and
tearing techniques in a prototype device called PaperFold1
(Figure 1) [5]. A number of paper metaphors served as
inspiration to the design of our prototype, which, like paper,
allows for multiple display segments to be folded into
smaller form factors. Books use folding as both a
navigational and space saving technique. Paper maps use
display size modulation via folding techniques of increasing
shape resolution [17]. PaperFold combines the benefits of
context-aware multi-display devices with paper-like thinness
and familiar interaction metaphors.
Why Thinness and Light Weight are Relevant
We believe thin, flexible displays are essential when trying
to mimic the kind of tactile-kinesthetic navigation,
Figure 1. PaperFold prototype folded into a 3D Hull.
Permission to make digital or hard copies of all or part of this work fo
r
p
ersonal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. Copyrights fo
r
components of this work owned by others than ACM must be honored.
Abstracting with credit is permitted. To copy otherwise, or republish, to
p
ost on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from Permissions@acm.org.
TEI '15, January 16 - 19 2015, Stanford, CA, USA
Copyright 2015 ACM 978-1-4503-3305-4/15/01…$15.00
htt
p
://dx.doi.or
g
/10.1145/2677199.2680572
expandable screen real estate and lightweight multi-display
experiences that paper offers to users. The kind of folding
seen in today’s mobile devices [3,20] or previous
explorations [1,2,7,8] cannot be regarded as similar because
the thickness of individual display segments impacts the total
thickness of a folded device, while the weight of each display
segment impacts overall portability. Although none of the
known flexible display technologies are capable of
employing creases like paper, tessellation of flexible displays
allows for size modulations that approximate those of
foldable paper maps. While we aim to produce more
complex folds, we limited ourselves to what can be achieved
with current flexible technology.
Contribution
In this paper, we evaluate the effectiveness of shape changes
as viewport transformations. We present PaperFold, a
prototype device consisting of 3 flexible display segments
that can be combined in arbitrary configurations. We discuss
interaction techniques that utilize shape changes to perform
view operations, e.g. adding a display segment extends a
document’s view; folding segments such that they can be
viewed by others produces shared views. We report on a
participatory design study that resulted in 14 shapes that can
be produced with PaperFold, and a subsequent study in
which participants ranked view operations to changes
between shapes. Results suggest participants were able to
attribute specific view operations to specific shape changes.
We conclude with design recommendations for matching
shape changes to view operations in reconfigurable mobile
devices.
RELATED WORK
While there has been research into folding displays into
various form factors, most of the display technologies in
prior studies were either simulations with paper mockups
[14], projection [9,10,11], or thick and rigid display devices
that are not particularly suitable for folding [1,2,7,8,15,18].
Other explorations focused on interactions with single
displays that were stacked or placed in close proximity
[4,22]. PaperFold bridges the tradeoff between portability
and increased screen real estate by introducing a
reconfigurable mobile display device with thin-film screens
that stay within acceptable boundaries of weight and
thickness.
Slate Display Form Factors
There is a large body of work on book form factors involving
rigid dual-screen form factors that can orient around one
axis. Pocket Edge [3] combined a tablet computer and an
eBook reader in a dual-display interactive device. Sony
Tablet P [20] featured interoperating hinged displays. Chen
et al. [1] discussed an e-book reader featuring two displays
mounted on two separate slates that can be used in side-by-
side or detached configurations. Their design supported
embodied navigation techniques like folding, flipping, and
fanning. Their findings suggested that having additional
displays offers a better support for lightweight navigation.
Multi-display Form Factors
Chen et al. [2] evaluated a multi-slate reading system, aimed
at understanding how increased screen real estate across
multiple devices can enhance navigation and interaction
techniques in reading activities. Their results indicated that
multi-slate reading systems have the potential to recapture
many of the affordances associated with physical documents,
while providing functionality that surpasses that of paper.
Hinckley et al. [7] showed the use of tablet PCs for spanning
contiguous images across multiple display surfaces. The
system used simultaneous deceleration as a means for
detecting connections, and was capable of performing simple
view operations such as collating two displays, or spanning
an image across two displays upon connection. Hinckley et
al.’s [8] Codex, a dual-screen tablet computer, was an
exception to the above work in that its displays are
detachable and reconfigurable into various form factors.
Codex can be oriented in a variety of postures to support
individual work, ambient display, or collaboration with
another user. The authors demonstrate interaction techniques
supporting division of labor for tasks across displays. Their
findings identify several benefits for detachable displays
systems (i.e. Division of Tasks between Screens;
Collaboration between devices; Sidebars). Siftables [15]
demonstrated a series of networked compact devices with
embedded displays that could be physically manipulated as
a group to interact with digital information. Although it was
able to span visuals across multiple displays, Siftables lacked
connective hinges and featured thick screen objects. The
rigidity and thick form factor of the displays employed in the
above explorations posed real drawbacks in terms of
portability and physical effort.
Thin-film Display Form Factors
DisplayStacks [4] presented a system for physically
interacting with digital information via stacks of thin-film
displays. It introduced tools for navigating digital documents
using piles of physical display windows. DisplayStacks
combined affordances of paper documents with electronic
content. PaperTab [22] later featured an environment in
which multiple thin-film displays work together through
proximity sensing to mimic a physical paper desktop
experience.
Foldable Thin-film Designs
Lee et al. [13] explored image projection on foldable
materials to simulate flexible displays with variable form
factors. Their findings suggested that devices that can
incorporate large surfaces in a small collapsible volume offer
advantages for mobile contexts. Xpaaand [10] featured a
handheld device encapsulating a rollable display. The
authors explored dynamic resizing as a means of interacting
with digital contents. Their results indicated that having the
ability to physically manipulate screen real state may
effectively improve interaction with handheld devices.
FoldMe [11] simulated thin-film displays on the front and
back sides of a mockup device foldable along predefined
hinges. Their results suggested folding navigation techniques
could improve manipulation of information for portable
devices. Folds in this system were limited to a pre-defined
axis. Paddle [16] used projection to represent highly
deformable mobile devices that can be transformed into
various special-purpose controls. Their results suggested
benefits over touch interaction techniques commonly used
on mobile devices. We borrow from the aforementioned
explorations, in designing a multi-segmented reconfigurable
mobile device that can be attached and detached into
arbitrary configurations along multiple axis, noting that none
of the above systems used real flexible displays.
DESIGN RATIONALE
We believe that building a prototype using real flexible
displays is valuable due to their unique properties in terms of
thinness, lightweight and malleability. Current displays are
too thick and heavy to be able to achieve the kind of shape
transformations seen in our study without significant
physical effort by the user. Additionally, to our knowledge
there has not been a systematic exploration of three-way
folding in multi-display systems. To inform the design of a
multi-segmented reconfigurable mobile device, we used the
following design criteria:
Use of Folding to Modulate Screen Size
Our first goal was to allow users to modulate the size of
screen real estate of the device between folded conditions
and expanded shapes. While there have been examples of
rollable devices [10], we consider folding to be a more
natural metaphor as it allows the overall form factor to be
similar to that of current smartphones. We believed
unfolding would typically be used to expand the size of an
existing view.
Reconfigurability and Tile Geometry
Hinged displays can only be folded along one dimension. In
order to increase the number of form factors, we explored
attachment/detachment on multiple sides of the displays.
This allows users to go from, e.g. tablet form factors to paper
notebook while maintaining a simple folding interface.
Unlike [1,2,8], our work explores view-transformations
beyond 2 segments while preserving portability.
Detachability also allows users to increase or reduce the
number of displays dynamically, and implement concurrent
usage scenarios with multiple displays. In particular, our
design aimed to increase the subset of shapes demonstrated
in previous explorations.
Orientation
The ability to alter the orientation has been used in
smartphones to switch between landscape and portrait
viewports. In PaperFold, orientation takes on a different role
as re-orienting the device may dramatically alter the
affordances of a shape, as well as visibility of information on
the displays. We hypothesized that orientation of the device
would therefore affect the function of the device.
Number and Angle of Tiles
The number of display segments creates a tradeoff between
shape resolution and portability of the device. More display
segments result in complex cable management, since
currently available flexible displays are not wireless. As we
will discuss in our participatory study, folding a four segment
device into a one segment device alters the shape geometry
of the device along two concurrent dimensions, making it
more difficult for users to (un)fold the device. Shapes created
by 3 segments are primarily determined by their geometry
and angle of displays towards one another.
3D Shapes
Using a minimum of three displays allows users to simulate
certain known display shapes through simple morphological
mimicry. A flat configuration resembles a book or map form
factor. Bending the displays inwards simulates a concave
display that might mimic gaming or virtual reality scenarios.
Bending the displays outwards, users can simulate a segment
of a spherical display. A 60-degree angle between 3 displays
creates a closed 3D hull. A 90-degree arrangement resembles
a portion of a cube [21]. One of the questions we were
interested in investigating was whether users would associate
such shapes with 3D objects.
Sharing
Different orientations of display segments may cause them
to be visible to the user, other users, or hidden altogether.
According to Hinckley [8], configuration of tiles could be a
design parameter for determining whether a shape indicated
shared or private interactions.
Matching Shape Changes to Viewport Transformations
Segments of PaperFold form building blocks that can be
“shaped” into an application. We investigated the relative
transformation between shape states as an interaction
technique. If we generalize the above discussion, we note
that shape transitions in PaperFold typically appear to affect
its viewport. To further investigate this, we conducted a
participatory study.
USER STUDY 1: PARTICIPATORY STUDY
Our methodologies were inspired by Hinckley et al., who
evaluated the affordances of pen-operated dual-display
devices [8], Chen et al. [1] who compared a dual-display
reader with single-display devices in typical reading tasks,
and Khalilbeigi et al. [11], who studied functions that can be
assigned to mockup double-sided displays with pre-defined
hinges. To investigate what would be the most functional
shapes of a foldable device, we conducted a participatory
design session in which subjects were presented with 4
10x15x0.15cm 3D printed tiles without displays. Each tile
contained magnets that allowed participants to interconnect
as well as hold shape at right angles.
Task & Participants
To determine what configurations users would find
appropriate for particular tasks, they were asked to imagine
a device in which the tiles featured displays showing apps
similar to those on their mobile phones. Users were told that
apps could span multiple displays. They were then asked to
create what they thought of as the most useful
transformations of shape for a 2 display hinged/detachable
device. We then repeated the exercise imagining 3 displays
and 4 displays. For each shape, we asked participants to
propose an associated functionality. 15 participants
volunteered for this study (6 females, 9 males; mean 23
years). Subjects received $10 for 1 hour of participation.
Results
Results suggest that participants had difficulty relating to the
4 tile configurations. All participants preferred using a
maximum of 3 tiles. Most participants carried multiple
devices, such as smartphones, tablets and laptops on a
routine basis (Mean: 4.07, s.d.: 1.16). However, they
preferred to use a single device with different form factors,
if this was possible (Mean: 4.73, s.d.: 0.46). Figure 2 shows
the most common shape changes produced by participants
for 2 and 3 tiles configurations: Holding the display in
portrait (A,E,F,K,L), landscape (B,C,H,J) and 3D (M,N)
orientations; Attaching displays horizontally (A,I) and
vertically (B,F,J); Detaching displays (D); Bending Displays
Inwards (L) and Outwards (K); Folding the displays into
triangular (E,N), perpendicular (G,N) and cube-like
structures (M). When asked what functionality these shapes
or shape changes might have in a device, participants
comments almost exclusively referred to view operations
that are common in GUIs. We synthesized these comments
into a list of potential view operations, shown in Figure 3.
APPARATUS
PaperFold is a foldable thin-film device with 3 detachable
flexible display tiles. The design of our prototype was
informed by the results of our participatory design study.
Each tile consists of a flexible thin-film E Ink display, and a
flexible 3D printed substrate with embedded sensors that
allows the system to determine the orientation and
connections of individual tiles (Figure 4).
Flexible Displays
Each PaperFold segment features a flexible 6.5” E-Ink
display. Magnetic hinges allow a continuous data connection
between displays. Each display is coupled to a driver board
connected to a computer controlling the logic and interface
graphics.
3D Printed Tiles
PaperFold’s flexible 3D printed segments measure 15x10 cm
with a thickness of 1.25mm. Magnetic hinges allow
PaperFold tiles to be attached in a variety of configurations
(Figure 4). Note that the increased thickness at the
extremities of the tile is only a function of the required size
of the embedded magnets. Hall Effect sensors in the tiles
monitor the magnetic field in their vicinity, allowing our
system to detect distinct connections among various tiles. A
9DOF IMU calculates the absolute orientation of each tile.
Using this information, we are able to determine what view
operations should occur upon distinct shape configurations.
Sensing Touch
E-Ink displays pose challenges to sensing touch, as their
functioning requires pixels to be applied with positive or
negative charges to produce the desired graphics. Readily
available touch solutions have shown to interfere with the
screen refresh mechanism of e-ink displays. We designed
flexible printed circuit layers to sense touch input by
measuring changes in capacitance between direct touch and
the electrode pads.
USER STUDY 2: MATCHING SHAPE CHANGES TO VIEW
OPERATIONS
In a subsequent experiment, we evaluated the use of our
PaperFold prototype for effecting view operations via shape
changes. We asked participants to rate the match between
each shape change and the list of view operations suggested
in our participatory study. Due to the nature of the
participatory design session, we used a different set of
participants to avoid bias towards choosing view operations
our first subset of users had suggested.
Shape Changes
Figure 2 shows the 14 shape transformations used in our
second study. Note that some are applied to different
orientations but topologically identical. Another defining
factor was the orientation of the hinge location: vertical or
horizontal. To illustrate, a vertically hinged portrait display
looks like a notebook. A horizontally hinged portrait display
looks like a book.
View Operations
In our study, we asked participants to match the list of 12
view operations in Figure 3 to the most frequently performed
shape changes derived from our participatory design session.
Task
For each shape change, we asked participants to rank each of
the 12 view operations for appropriateness using
questionnaire items with a 5-point Likert scale (Strongly
Figure 2. The 14 participant defined shape configurations used in matching shape changes to view operations.
Agree-Strongly Disagree) that answered the following
question: “When I change the shape of this device in this
way, it [View Operation].”
Experiment Design and Participants
All participants experienced all shape changes in the same
order. To avoid bias, the list of view operations on the
questionnaire was presented in random order. 12 participants
took part in the experiment (6 Female; 6 Male; Mean Age:
21.3). All participants received $10 for 1 hour of
participation.
Procedure
During training, each of the 12 view operations were
demonstrated in a GUI desktop application to avoid biasing
participants with PaperFold examples. Increasing the size of
the window and revealing more of an image demonstrated
Collate/Extend View. Show Full Screen was demonstrated by
showing a full screen window containing an entire photo.
Show Thumbnails was demonstrated by showing thumbnails
of the photo library; Zoom In/Out was demonstrated by
zooming in and out of a photo without changing the window
size. Duplicate View was demonstrated by showing two
windows with the same image. Show 3D View was
demonstrated by showing a 3D model of the object on the
photo. Other view operations were demonstrated by
showing: a keyboard; a toolbar; a phone keypad; a photo in
one window and a notepad in another; and an empty window.
Participants were then introduced to the prototype, instructed
on the task and given the questionnaire. A display showed a
list of image transformations from which each view
operation could be constructed on PaperFold. This list was
similar to the images shown in Figure 3. The experimenter
then loaded the first image onto the first display, after which
participants were asked to make the first shape change in
Figure 2: Attaching a second display horizontally.
Participants were asked to go through the questionnaire and
pick a favorite operation to perform upon this shape change,
if there was one. They were then asked to select the images
on the display that would best show this operation on the
prototype. They then rated the view operation on the
questionnaire, and were asked whether there were any
operations in the list that they thought would match this
shape change. If so, they were given the option to show these
on the prototype and rate them.
Applications
All shapes were repeated for each of two applications: 1) A
text editor showing pages of a scientific paper 2) A Google
map application showing a map of London. We based our
choice on applications described in previous work: [1]
focused on reading activities on dual-display devices; [8]
presented a pen-operated note taking application; and [11]
demonstrated interaction techniques for browsing maps
using mockup foldable displays. Additionally, the results of
our participatory design study showed a strong user
preference for these application scenarios.
RESULTS
Figure 5 shows the means and standard deviations for the
scores matching view operations with shape changes, for
portrait, landscape and 3D conditions in both application
scenarios. A non-parametric analysis of variance (Friedman
Test evaluated at α<.05) showed significant differences
between view operation scores for each shape change in the
text application (Friedman’s 2(11)>21.11, p<.05), with the
exception of Fold 3D Triangular Hull (Friedman’s
2(11)=18.436, p=0.072). In the map application, results also
showed significant differences between view operation
scores for each shape change (Friedman’s 2(11)>23.9,
p<.05). Due to the exhaustive nature of possible
comparisons, and because we intended our results as design
heuristics, we did not perform further post-hoc tests. Instead,
we assigned the highest ranked viewport for any given shape
change as a best match, but only if it had a positive rating,
i.e., higher than 3.0. Note that by picking the top match, we
interpreted our data at the ordinal level of measurement.
Approximately half our top matches in the Text Editing
Scenario had a score above or equal to 4.0, suggesting strong
preferences for these particular view operations. Optional
alternative view operations, with scores between 3.0 and 4.0,
are also displayed. Finally, note that in the below section,
whenever we make a statement such as: “users preferred”,
we do not suggest such statement scientifically accurate, but
rather as a trend that informs the design process.
Figure 3. A list of view operations in GUIs suggested by
participatory design session.
Figure 4. Exploded view of a PaperFold segment.
Figure 5. Mean ratings and (standard deviations) for matching view operations to presented shape changes for both
applications
User Agreement
Since we expected multiple view operations to match any
one particular shape, we decided against calculating a
measure of agreement. Instead, because the Likert scale
deployed does approximate an interval level of
measurement, we examine the standard deviations as
providing some indication of user agreement. In the text
application, standard deviations, as shown in Figure 5, were
low in most cases, averaging 1.33. In the map application,
standard deviations averaged 1.3. This suggests users
generally agreed well on rankings for both applications.
Matching View Operations to Shape Changes
Text Application
Extending the display, in both orientations, was strongly
associated with the Collate operation. In portrait conditions,
participants preferred Show Thumbnails as an alternative. In
landscape conditions, Full Screen or Show Keyboard were
suggested. Neither bending 3 displays inwards or outwards
had a strong user association in the case of the text
application. Bending inwards was associated with Collate,
while bending outwards was associated with Zoom In. Users
had a weak association with Show 3D View in the landscape
condition. In portrait mode, there was an equally weak
association with folding the display into a perpendicular
shape as pertaining to Collate. However, in the landscape
condition, there was a strong preference for Show Keyboard.
In portrait conditions, Folding Displays into a triangular
shape was associated with Duplicate Screen. Finally, for
Detach Displays, Duplicate Screen was preferred.
Map Application
Extending the displays in the portrait condition was highly
ranked for the Collate operation, in both horizontal and
vertical conditions. For the third display, Show Thumbnails,
Show Keyboard, Show Different App and Duplicate Screen
were also ranked positively. In the landscape condition,
second display, participants split their top ratings between
Collate an d Show Keyboard. For the landscape, third display,
there was a clearer preference for Collate, with Show
Thumbnails, Show Keyboard and Show Different App ranked
positively. Bending 3 Displays Inwards, Outwards and Fold
3D Hull (Triangle) were all associated with Show 3D View.
In the landscape condition, there was a strong preference of
Show 3D View for Fold 3D Hull (Cube), with Duplicate
Screen considered an alternative. In both c, there was a clear
association between folding the display into a perpendicular
shape and Show Keyboard. In both portrait and landscape
conditions, Folding Displays into a shape was associated
with Duplicate Screen. Finally, for Detach Displays Show
Different App had the strongest rating, with Duplicate Screen
and Show Phone considered weaker alternatives.
User Comments and Observations
Throughout the experiment, participants were encouraged to
verbalize their thought processes, which were noted by the
experimenters. Overall, participants liked having the ability
to arrange PaperFold in various configurations, thus being
able to dynamically alter viewports. “I would like to be able
to customize the functions of different shapes.” [P2]. There
were benefits identified for using PaperFold as a
multitasking system. “I enjoyed having the ability to
distribute different applications across displays and access
them concurrently.” [P5, P8]. While browsing through maps,
users were particularly impressed with the 3D View features
of our system. “3D shapes with multiple displays lead
themselves to viewing 3D models.” [P11]. Additional
benefits identified include adding peripherals [P4,P10];
thumbnails/toolbars [P3,P11]; or using PaperFold as means
for distributing information, by physically detaching and
sharing panels [P12].
DISCUSSION
For both application scenarios, results suggest that users
associate shape changes of PaperFold’s screens with specific
view operations. Some shape changes are, however, more
ambiguous than others. For example, extending PaperFold
wi th a 3rd horizontal display rates above neutral on 4 different
view operations. But even in those cases, generally only one
or two view operations rank highly for each shape change.
We will now discuss differences in trends between the two
application scenarios, trying to generalize our results into
design recommendations for viewport behaviors in folding
multi-display user interfaces.
Horizontal Extensions
When extending the displays horizontally, the most highly
ranked view operation was consistently Collate. In the case
of the text editor, where a contextual overview of data objects
is applicable, participants ranked Show Thumbnails higher
than a Full Screen operation. In the map browser, which has
no clear delineation of specific data objects, participants
were more inclined to try and fit the data on two screens.
Observations of image selections during the experiment
strongly suggests that participants preferred to extend a
thumbnail display with a second display showing a full
screen view of the first thumbnail. Since participants chose
to extend a full screen display with a collated view they
overwhelmingly chose Collated View as the highest rated
answer for both cases. Only when a third display was
attached, did participants widen their options, with scores for
collated views ranking lower and thumbnails higher.
Vertical Extensions
For vertical portrait extensions, Collate generally appears the
preferred option. Collate is also the highest ranking option
for vertical landscape extensions, where keyboard is also a
strong option.
3D Shapes
Bending three displays inwards or outwards was much more
highly ranked for Show 3D View operations in the map
application, where data elements were not inherently limited
to 2D. Observations confirmed that in the text editor
scenarios, participants had difficulty relating any 3D
operations to the inherent 2D nature of the data element. In
general, folding the displays into a 3D hull was the shape
change most strongly associated with Show 3D View. This
suggests that users associate the shape of the device with
known objects of similar form factor.
Detach
Detaching a screen was less easily generalizable: In the map
application detaching the displays was most strongly
associated with Show Different App. In the text editor
application Duplicate Screen was pr efe rred. V iew ope rat ions
that were not ranked positively for any shape change in either
application scenario included: Zooming; Show Toolbar; and
Show Phone.
DESIGN RECOMMENDATIONS
We derived the following recommendations for the design of
segmented, foldable, multi-display devices:
1. Automated view operations appear to work well as
responses to folding, attaching or detaching displays.
2. Consider a Collate operation when users extend a first
display in Full Screen View with a second display.
3. Consider alternative views such as keyboards/toolbars or
different apps when extending a 2nd display with a 3rd
display.
4. Provide mechanisms for users to choose view operation
functions, or allow them to set preferences.
5. Consider showing a keyboard on the lowest display when
the device is folded into a perpendicular shape.
6. Consider mirroring the displays when the device is folded
into a triangular shape.
7. Consider showing a 3D view of the data when users bend
displays inwards or outwards, or into a 3D hull shape.
LIMITATIONS AND FUTURE DIRECTIONS
We note that the above recommendations are based on
heuristics and can be considered valid only within the
limitations of the presented device and user study. We do
believe that results are sufficiently clear to generalize to
other application scenarios and device morphologies. We
recognize that our PaperFold prototype has shortcomings
that did not allow us to evaluate use by multiple users outside
a laboratory environment. In particular, an important future
direction is to remove the dependency on cables and
extraneous equipment for what is supposed to be a mobile
device.
CONCLUSIONS
We evaluated PaperFold, a mobile device with flexible
display segments that can be folded into arbitrary
configurations. Our design was informed by a participatory
study that resulted in 14 preferred shape transformations. A
subsequent study investigated the effectiveness of shape
changes for viewport transformations. We evaluated how
participants ranked the utility of shape changes for triggering
common view operations in map and text editing
applications. Results indicate that participants were
generally able to select specific view operations as
automated responses to folding, attaching or detaching
displays. User feedback from our study indicates benefits
from having multiple detachable displays. Advantages
include better support for performing tasks that traditionally
require multiple devices, as well as physical manipulation
and sharing of views.
REFERENCES
1. Chen, N., et al. Navigation Techniques for Dual-Display
E-Book Readers. Proc. CHI ’08, 1779-1788.
2. Chen, N., Guimbretière, F., Sellen, A. Graduate student
use of a multi-slate reading system. Proc. CHI’13, 1799-
1808.
3. Entourage Edge (2013) - http://entourageedge.eu/
4. Girouard, A., Tarun, A., and Vertegaal, R.
DisplayStacks: Interaction Techniques for Stacks of
Flexible Thin-Film Displays. Proc CHI ’12, 2431-2440.
5. Gomes, A., Nesbitt, A., Vertegaal, R. MorePhone. Proc.
CHI ’13, 583-592.
6. Gomes, A., Vertegaal, R. PaperFold: A Shape Changing
Mobile Device with Multiple Reconfigurable
Electrophoretic Magnetic Display Tiles. Proc. CHI
EA’14, 535-538.
7. Hinckley, K. Synchronous gestures for multiple persons
and computers. Proc. UIST ‘03, 149-158.
8. Hinckley, K., Dixon, M., Sarin, R., Guimbretiere, F.,
and Balakrishnan, R. Codex. Proc. CHI ’99, 1933–1942.
9. Holman, D., et al. PaperWindows: interaction
techniques for digital paper. Proc. CHI '05, 591-599.
10. Khalilbeigi, M., Lissermann, R., Mühlhäuser, M., and
Steimle, J. Xpaaand. Proc. CHI ’11, 2729-2732.
11. Khalilbeigi, M., Lisserman, R., Kleine, W. FoldMe:
interacting with double-sided foldable displays. Proc.
TEI ’12, 33-40.
12. Lahey, B., et al. PaperPhone: Understanding the Use of
Bend Gestures in Mobile Devices with Flexible
Electronic Paper Displays. Proc. CHI ’11, 1303-1312.
13. Lee, J.C. Hudson, S.E., and Tse, E. Foldable interactive
displays. Proc. UIST ’08 (2008), 287-290.
14. Lee, S. et al. How users manipulate deformable displays
as input devices. Proc. CHI ’10, 1647-1656.
15. Merrill, D., Kalanithi, J., Maes, P. Siftables: towards
sensor network user interfaces. Proc. TEI ’07, 75-78.
16. Ramakers, R., Johannes, S. Luyten, K. Paddle. Proc.
CHI’14. 2569-2578.
17. Roudaut, A., et al. Morphees: toward high “shape
resolution” in self-actuated flexible mobile devices.
Proc. CHI ’13, 593-602.
18. Schwesig, C., Poupyrev, I., and Mori, E. Gummi: a
bendable computer. Proc. CHI '04, 263-270.
19. Sellen, A., Harper, R. The Myth of the Paperless Office,
MIT Press, Cambridge, MA, 2003.
20. Sony Corp - www.sony.co.uk/product/sony-tablet-p
21. Stavness, I., Lam, B., Fels, S. pCubee. Proc. CHI ’10,
1381-1390.
22. Tarun, A., et al. PaperTab: an electronic paper computer
with multiple large flexible electrophoretic displays. EA
CHI ’13, 3131-3134.
