No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Investigating user performance and preference for two magnitude levels of size and angle of bend on a smartphone-sized flexible device
ResearchGate has not been able to resolve any citations for this publication.
Display form factors such as size and shape have been conventionally determined in consideration of usability and portability. The recent trends requiring wearability and convergence of various smart devices demand innovations in display form factors to realize deformability and large screens. Expandable displays that are foldable, multi-foldable, slidable, or rollable have been commercialized or on the edge of product launches. Beyond such two-dimensional (2D) expansion of displays, efforts have been made to develop three dimensional (3D) free-form displays that can be stretched and crumpled for use in realistic tactile sensation, artificial skin for robots, and on-skin or implantable displays. This review article analyzes the current state of the 2D and 3D deformable displays and discusses the technological challenges to be achieved for industrial commercialization. This review article analyzes the current state of the 2D and 3D deformable displays and discusses the technological challenges to be achieved for practical uses and industrial commercialization.
Rigid sensors are a mature type of sensor, but their poor deformation and flexibility limit their application range. The appearance and development of flexible sensors provide an opportunity to solve this problem. In this paper, a resistive flexible sensor utilizes gallium−based liquid metal (eutectic gallium indium alloy, EGaIn) and poly(dimethylsiloxane) (PDMS) and is fabricated using an injecting thin−line patterning technique based on soft lithography. Combining the scalable fabrication process and unique wire−shaped liquid metal design enables sensitive multifunctional measurement under stretching and bending loads. Furthermore, the flexible sensor is combined with the glove to demonstrate the application of the wearable sensor glove in the detection of finger joint angle and gesture control, which offers the ability of integration and multifunctional sensing of all−soft wearable physical microsystems for human–machine interfaces. It shows its application potential in medical rehabilitation, intelligent control, and so on.
Piezoelectric energy harvesters are appealing for the improvement of wearable electronics, owing to their excellent mechanical and electrical properties. Herein, screen-printed piezoelectric nanogenerators (PENGs) are developed from triethoxy(octyl)silane-coated barium titanate/polyvinylidene fluoride (TOS-BTO/PVDF) nanocomposites with excellent performance based on the important link between material, structure, and performance. In order to minimize the effect of nanofiller agglomeration, TOS-coated BTO nanoparticles are anchored onto PVDF. Thus, composites with well-distributed TOS-BTO nanoparticles exhibit fewer defects, resulting in reduced charge annihilation during charge transfer from inorganic nanoparticles to the polymer. Consequently, the screen-printed TOS-BTO/PVDF PENG exhibits a significantly enhanced output voltage of 20 V, even after 7500 cycles, and a higher power density of 15.6 μW cm−2, which is 200 and 150% higher than those of pristine BTO/PVDF PENGs, respectively. The increased performance of TOS-BTO/PVDF PENGs is due to the enhanced compatibility between nanofillers and polymers and the resulting improvement in dielectric response. Furthermore, as-printed devices could actively adapt to human movements and displayed excellent detection capability. The screen-printed process offers excellent potential for developing flexible and high-performance piezoelectric devices in a cost-effective and sustainable way.
In human-computer interaction research, prototypes allow for communicating design ideas and conducting early user studies to understand user experience without developing the actual product. For investigating deformation-based interaction, functional prototyping becomes challenging due to the unavailability of commercial platforms and the marginal availability of flexible electronic components. During functional prototyping, incurred time and cost are essential factors that further depend on the ease of stiffness customization, reproduction, and upgrade. To offer these advantages, this work presents the fabrication workflow of Nāmya, a smartphone-sized flexible prototype that can detect bend gestures and touch-based inputs using off-the-shelf sensors and flexible materials. This do-it-yourself (DIY) approach to fabricating deformable prototypes focuses on addressing the challenges of selecting flexible material, type of sensor, and sensor positions. We also demonstrate that the proposed use of a flexible three-dimensional- (3D-) printed internal structure with sensor pockets and the one-part silicone cast allows the development of robust deformable prototypes. This fabrication process offers the opportunity to easily customize device stiffness, reproduce prototypes with similar physical properties, and upgrade existing prototypes.
Flexible electronics is an interdisciplinary research field. By employing the intrinsically soft materials, the resulting devices feature outstanding mechanical flexibility and conformability, and expected lightweight configuration, functional reconfigurability, large-area compatibility. Here we explore the current status, including materials, fabrication and applications, of flexible electronics from intrinsically soft materials. The future applications and challenges of this pervading technology are also considered. Flexible electronics is an interdisciplinary research field. By employing the intrinsically soft materials, the resulting devices feature outstanding mechanical flexibility and conformability, and expected lightweight configuration, functional reconfigurability, large-area compatibility. Here we explore the current status, including materials, fabrication and applications, of flexible electronics from intrinsically soft materials. The future applications and challenges of this pervading technology are also considered.
An objective analysis of the human movement can help both clinical assessment and sports performance. Kinovea is a free 2D motion analysis software that can be used to measure kinematic parameters. This low-cost technology has been used in sports sciences, as well as in the clinical and research fields. One interesting tool is that it can measure an object (or person) passing in front of the camera, taking into account the perspective between the camera and the recorded object. Although it has been validated as a tool to assess time-related variables, few studies assessed its validity compared to a Gold Standard; furthermore, its reliability in different perspectives has not been previously assessed. The main objective of this study is to determine the validity of the Kinovea software compared to AutoCAD, and its intra and inter-rater reliability in obtaining coordinates data; a second objective is to compare their results at 4 different perspectives (90°, 75°, 60° and 45°) and to assess the inter and intra rater reliability at each perspective. For this purpose, a wire structure figure in the shape of a human lower limb was designed and measured in AutoCAD; it was then recorded during a pendular motion with a video-camera placed at distance of 5 m and analyzed with Kinovea in the 4 perspectives (90°, 75°, 60° and 45°). Each frame was examined by three observers who made two attempts. A multiple approach was applied involving the analysis of the systematic error, with a two-way ANOVA 2x4; the relative reliability with Intraclass Correlation Coefficient (ICC) and the Coefficient of Variance (CV) (95% confidence interval); and the absolute reliability with the Standard Error (SE). The results indicate that the Kinovea software is a valid and reliable tool that is able to measure accurately at distances up to 5 m from the object and at an angle range of 90°–45°. Nevertheless, for optimum results an angle of 90° is suggested.
The landscape orientation of smartphone offers a better aspect ratio and extensive view for watching media and photography. However, it presents challenges of occlusion, reachability, and frequent re-gripping in one-handed interactions. To address these issues we took the opportunity of deformation gestures to interact with future flexible smartphones. A preliminary survey was conducted to understand one-handed landscape mode usage patterns. Then, the 1st study was conducted to identify 3 most preferred one-handed landscape mode grips. In the 2nd study, we gathered unique user-defined deformation gestures to identify the set of most natural and intuitive gestures corresponding to each grip. We also found 3 gestures that can be performed in more than one grip. Finally, we discuss the influence of the grips on performing gestures.
Soft electronic devices, particularly for healthcare applications, have been intensively studied over the past decade owing to their unique advantages over conventional rigid electronics. These advantages include conformal contacts on target tissues such as the skin, heart, and brain along with a high deformability that minimizes unwanted inflammatory responses. To achieve mechanically soft but multifunctional high performance electronics for wearable and implantable biomedical devices, several strategies have been employed including designed assembly of high quality nanomaterials, the combination of unconventional manufacturing processes with existing microprocessing technologies, new device designs with deformable structures, and disease-specific system-level integration of diverse soft electronics. In this focus review, we summarize recent advances in the soft electronic devices for healthcare applications. More specifically, we describe assembly methods for various nanomaterials, new device design and integration strategies, their applications to textile-based and skin-attached wearable electronics, and their incorporation in fully- and/or minimally-invasive medical devices. Finally, this review concludes with a brief description on the future direction of healthcare applications using nanomaterial-based soft bioelectronics.
ReFlex is a flexible smartphone with bend input and active haptic feedback. ReFlex's features allow the introduction of sensations such as friction or resistance. We report results from an experiment using ReFlex in a targeting task, as well as initial users' reactions to the prototype. We explore both absolute and relative tactile haptic feedback, paired with two types of bend input mappings: position-control and rate-control. We observed that position-controlled cursors paired well with relative bend feedback, while rate-controlled cursors paired well with absolute bend feedback to indicate targets. We also explored an eyes-free condition. Results suggest that while eyes-free, haptic feedback conditions were more error-prone than visual-only conditions, the size of the error was relatively small, and users were able to complete the task in all cases. We present two application scenarios that take advantage of the unique input and output modalities of ReFlex and discuss its potential for within document navigation.
Upcoming mobile devices will have flexible displays, allowing us to explore alternate forms of user authentication. On flexible displays, users can interact with the device by deforming the surface of the display through bending. In this paper, we present Bend Passwords, a new type of user authentication that uses bend gestures as its input modality. We ran three user studies to evaluate the usability and security of Bend Passwords and compared it to PINs on a mobile phone. Our first two studies evaluated the creation and memorability of user-chosen and system-assigned passwords. The third study looked at the security problem of shoulder-surfing passwords on mobile devices. Our results show that bend passwords are a promising authentication mechanism for flexible display devices. We provide eight design recommendations for implementing Bend Passwords on flexible display devices.
Interaction with large touch surfaces is still a relatively infant domain, particularly when looking at the accessibility solutions offered to blind users. Their smaller mobile counterparts are shipped with built-in accessibility features, enabling non-visual exploration of linearized screen content. However, it is unknown how well these solutions perform in large interactive surfaces that use more complex spatial content layouts. We report on a user study with 14 blind participants performing common touchscreen interactions using one and two-hand exploration. We investigate the exploration strategies applied by blind users when interacting with a tabletop. We identified six basic strategies that were commonly adopted and should be considered in future designs. We finish with implications for the design of accessible large touch interfaces.
Bend gestures have a large number of degrees of freedom and therefore offer a rich interaction language. We propose a classification scheme for bend gestures, and explore how users perform these bend gestures along four classification criterion: location, direction, size, and angle. We collected 36 unique bend gestures performed three times by each participant. The results suggest a strong agreement among participants for preferences of location and direction. Size and angle were difficult for users to differentiate. Finally, users performed and perceived two distinct levels of magnitude. We propose recommendations for designing bend gestures with flexible displays.
We present an introduction to the user centered research that we are conducting using functional deformable research prototypes. This work has recently crystallized in the demonstration of the Nokia Kinetic Device (figure 1). In the large design space that opens before us around deformable user interfaces, we have chosen to focus on mobile personal interfaces. We aim to investigate how human factors should influence the transition from rigid to deformable hardware. In this paper, we propose the topics that a research agenda should cover, and we discuss our research methodology. We also describe the deformable research prototypes (Kinetic DUI-RP) that allow us to conduct user research with functional prototypes. Finally, we present an initial set of design guidelines that future research will develop further.
Thin-film electronics in its myriad forms has underpinned much of the technological innovation in the fields of displays, sensors, and energy conversion over the past four decades. This technology also forms the basis of flexible electronics. Here we review the current status of flexible electronics and attempt to predict the future promise of these pervading technologies in healthcare, environmental monitoring, displays and human–machine interactivity, energy conversion, management and storage, and communication and wireless networks.
There has been little discussion on how the materials used to create deformable devices, and the subsequent interactions, might influence user performance and preference. In this paper we evaluated how the stiffness and required deformation extent (bending up and down bi-manually) of mobile phone-shaped deformable devices influenced how precisely participants were able to move to and maintain target extents of deformation (bend). Given the inherent haptic feedback available from deforming devices (over rigid devices), we also compared performance with, and without, external visual feedback. User perception and preference regarding the different devices were also elicited. Results show that, while device stiffness did not significantly affect task perfor-mance, user comfort and preferences were strongly in favour of softer materials (0.45 N·m/rad) and moderate amounts of defor-mation. Removing external visual feedback led to less precise user input, but inaccuracy remained low enough to suggest non-visual interaction with deformable devices is feasible.
This paper describes an affect-responsive interac-tive photo-frame application that offers its user a different experience with every use. It relies on visual analysis of activity levels and facial expressions of its users to select responses from a database of short video segments. This ever-growing database is automatically prepared by an of-fline analysis of user-uploaded videos. The resulting sys-tem matches its user's affect along dimensions of valence and arousal, and gradually adapts its response to each spe-cific user. In an extended mode, two such systems are cou-pled and feed each other with visual content. The strengths This paper is based on [9]. and weaknesses of the system are assessed through a us-ability study, where a Wizard-of-Oz response logic is con-trasted with the fully automatic system that uses affective and activity-based features, either alone, or in tandem.
Mobile devices provide people with disabilities new opportunities to act independently in the world. However, these empowering devices have their own accessibility challenges. We present a formative study that examines how people with visual and motor disabilities select, adapt, and use mobile devices in their daily lives. We interviewed 20 participants with visual and motor disabilities and asked about their current use of mobile devices, including how they select them, how they use them while away from home, and how they adapt to accessibility challenges when on the go. Following the interviews, 19 participants completed a diary study in which they recorded their experiences using mobile devices for one week. Our results show that people with visual and motor disabilities use a variety of strategies to adapt inaccessible mobile devices and successfully use them to perform everyday tasks and navigate independently. We provide guidelines for more accessible and empowering mobile device design.
We have developed an interface for browsing content that uses the metaphor of turning the pages of a book. Using our "Bookisheet" interface, which consists of two thin plastic sheets and bend sensors, a user can easily scroll digital content such as photos by bending one side of the sheet or the other. This action provides the tangible and pleasant sense of turning pages in a book. Bookisheet can be used not only as an interface for conventional information terminals but also as one for flexible displays.
We show interaction possibilities and a graphical user interface for deformable, mobile devices. WIMP (windows, icons, mouse, pointer) interfaces are not practical on mobile devices. Gummi explores an alternative interaction technique based on bending of a handheld device.
Based on the demand for flexible human-machine interaction devices, it is urgent to develop high-performance stretchable ionic conductive materials. However, most gel-based ionic conductive materials are composed of crosslinked polymer networks that contain liquids, and suffer from limitations of solvent volatilization and leakage, and the cross-linking restricts the movement and diffusion of polymer chains, making it difficult for them to achieve adhesion. Here, we introduce flexible and adhesive liquid-free ionic conductive elastomers (ICE) with salt using a non-crosslinked polymer strategy. The ICE show a transparency of 89.5%, Tg of -51.2 °C, negligible weight loss at 200 °C, a tensile fracture strain of 289.5%, and an initial modulus of 45.7 kPa, and is adhesive to various solid surfaces with an interfacial toughness of 11.4 to 41.4 J m-2. Moreover, the ICE exhibit stable electrical conductivity under ambient conditions. Triboelectric nanogenerators (TENGs) were assembled on an electrical shell surface with the adhesive ICE as an electrostatic induction layer and were displayed for use as human-machine interactive keyboards. This approach opens a route to making adhesive and stable polymer ionic conductors for human-machine interaction.
Self-powered devices and micro-sensors are in high demand for intelligent electronics and flexible wearables for applications in medical healthcare and human–computer interactive robotics. Flexible, stretchable, wearable and breathable high-sensitivity sensors that monitor signals from subtle changes in the environment provide solutions for personalized medical healthcare. In this article, we review the fundamental mechanisms, theoretical research, sensor fabrication methodologies, and applications of flexible piezoelectric materials. We focus on the flexibility and stretchability of inorganic, polymer, and bio-piezoelectric materials, explain their properties for physiological signal monitoring, motion detection, and force sensing. We provide an overview of the latest progress in piezoelectronics and piezoelectric photonics, the structures of devices, and self-powering technologies. We compared 10 types of polymers and composites with human skin in terms of elastic modulus and found that PLLA/PDMS, electro-spun PVDF, and ZnO/PVDF possess higher elastic modulus (50, 500, and 27900 MPa respectively) than human skin (18.8 MPa), thus being more suitable for wearable devices. In terms of electrical outputs of piezoelectric materials, polarized cast film has stronger piezoelectric property with output of 14–45.6 V than electro-spun filaments material (output 0.4–9 V). Future research and development should focus on increasing electrical performance while maintaining the required flexibility and durability.
Bend gesture-based input on a flexible device requires a distinct indication of gesture completion. Existing research commonly used threshold-based strategies for discrete input, whereas releasing the bend is used to confirm continuous input. However, the simplest form of these strategies does not allow the rejection of unwanted or unintended input and bi-directional manipulation of the output parameter before confirming continuous input. This work aims to identify user-defined gesture completion strategies with the potential to overcome these drawbacks. Ten blind or low vision (BLV) and fifteen sighted students participated in the study. We identified four and three unique strategies for discrete and continuous inputs, respectively. We believe the reported implications and recommendations with potential application scenarios will help researchers and interaction designers take advantage of these strategies.
Deformable interfaces are emerging in HCI and prototypes show potential for non-rigid interactions. Previous reviews looked at deformation as a material property of shape-changing interfaces and concentrated on output. As such, deformable input was under-discussed. We distinguish de-formable from shape-changing interfaces to concentrate on input. We survey 131 papers on deformable interfaces and review their key design elements (e.g., shape, material) based on how they support input. Our survey shows that deformable input was often used to augment or replace rigid input, particularly on elastic and flexible displays. However, when shapes and materials guide interactions, deformable input was used to explore new HCI paradigms, where gestures are potentially endless, and input becomes analogy to sculpting, metaphor to non-verbal communication, and expressive controls are enhanced. Our review provides designers and practitioners with a baseline for designing deformable interfaces and input methodically. We conclude by highlighting under-explored areas and identify research goals to tackle in future work with deformable interfaces.
Deformable interfaces are emerging in HCI and prototypes show potential for non-rigid interactions. Previous reviews looked at deformation as a material property of shape-changing interfaces and concentrated on output. As such,deformable input was under-discussed. We distinguish deformable from shape-changing interfaces to concentrate on input. We survey 131 papers on deformable interfaces and review their key design elements (e.g., shape, material) based on how they support input. Our survey shows that deformable input was often used to augment or replace rigid input, particularly on elastic and flexible displays. However, when shapes and materials guide interactions, deformable input was used to explore new HCI paradigms, where gestures are potentially endless, and input become analogy to sculpting, metaphor to non-verbal communication, and expressive controls are enhanced. Our review provides designers and practitioners with a baseline for designing deformable interfaces and input methodically. We conclude by highlighting under-explored areas and identify research goals to tackle in future work with deformable interfaces.
People with vision impairment are concerned about entering passwords in public as accessibility features (e.g. screen readers and screen magnifiers) make their passwords more vulnerable to attackers. This project aims to use bend passwords to solve this accessibility issue, as they are harder to observe than PINs. Bend passwords are a recently proposed method for user authentication that uses a combination of predefined bend and fold gestures performed on a flexible device. Our inexpensive prototype called BendyPass is made of silicone, with flex sensors able to capture and verify bend passwords, a vibration motor for gesture input haptic feedback, and a button to delete the last gesture or confirm the password. Bend passwords entered on BendyPass provide a tactile method for user authentication, designed to reduce the vulnerability to attackers and help people with vision impairment to better protect their personal information.
When compared to mainstream touch/button-centric devices, deformable devices enable a more organic and tangible way for human-computer interaction. This studio provides participants an opportunity to have hands-on experience in fabricating controllers that use various deformation inputs (e.g., bending, stretching), and promote novel interaction mechanisms using hand gestures. Participants will learn different types of deformation inputs and create their own sensors. They will work in groups to design deformable controllers using both the sensors they make in-session and/or pre-built ones, to afford novel hand gestural inputs beyond conventional touches and clicks. The main objectives of this studio are to facilitate exchange of experience in fabricating deformable sensors/materials, and to foster creativity in using such controllers as inputs with hand gestures.
Current smartphone technology presents many challenges for blind users. This article introduces the use of deformable inputs for blind users, which offers such users the ability to physically manipulate a device for system interaction. The authors describe the iterative design process of a deformable device prototype, Typhlex, with strategically placed grooves to elicit bend gestures. To understand its utility as a screen reader control, they conducted two exploratory studies with sighted participants (with the prototype hidden from view) and blind participants, focusing on comparing the usability of bend gestures to touch as primary forms of input. Their findings suggest that while easily learnable and enjoyed by both groups, the prototype had yet to improve blind users’ performances when compared to the commonly used touch input paradigm. They present lessons learned from their design process and studies, and they discuss the promise of deformable input devices in the area of accessibility for blind users. This article is part of a special issue on physical computing and shape-changing interfaces.
Bend gestures can be used as a form of Around Device Interaction to address usability issues in touchscreen mobile devices. Yet, it is unclear whether bend gestures can be easily learned and memorized as control schema for games. To answer this, we built a novel deformable smartphone case that detects bend gestures at its corners and sides, and created PaperNinja, a mobile game that uses bends as input. We conducted a study comparing the effect of three pre-game training levels on learnability and memorability: no training, training of the bend gestures only, and training of both the bend gestures and their in-game action mapping. We found that including gesture-mapping positively impacted the initial learning (faster completion time and fewer gestures performed), but had a similar outcome as no training on memorability, while the gestures-without-mapping led to a negative outcome. Our findings suggest that players can learn bend gestures by discovery and training is not essential.
We explore controller input mappings for games using a deformable prototype that combines deformation gestures with standard button input. In study one, we tested discrete gestures using three simple games. We categorized the control schemes as binary (button only), action, and navigation, the latter two named based on the game mechanics mapped to the gestures. We found that the binary scheme performed the best, but gesture-based control schemes are stimulating and appealing. Results also suggest that the deformation gestures are best mapped to simple and natural tasks. In study two, we tested continuous gestures in a 3D racing game using the same control scheme categorization. Results were mostly consistent with study one but showed an improvement in performance and preference for the action control scheme.
Recent work has explored how on-the-wrist devices can be augmented with displays, sensors, and aesthetics to produce a new genre of wearable, digital jewelry. Most often, these devices are augmented with hardware (e.g., touch-screens, gyroscopes) to receive wearer input. Breakthroughs in sensing materials allow novel inputs such as squeeze, bend, and stretch, which may be more suited to interaction with a discreet wearable.
We explore interactions with a flexible bracelet that can be manipulated through bend, stretch, and touch. We discuss the process involved in fabricating materials that convert these actions into measurable parameters. We particularly focus on eyes-free interactions with bracelets that take advantage of the tactile feedback from the act of deformation, and demonstrate some early, low-fidelity prototype concepts.
We explore the use of bend gestures in gaming with flexible devices and investigate the size differences between smartphones and tablets. We conducted a study asking users to select bend gestures for nine tasks derived from gaming, grouped by navigation, action, and deformation. Our results suggest pairing opposing tasks, such as navigating left and right, by gesture location. We saw low consensus for action tasks, and strong association between the location of the gestures and the location of the visual, relating to the Simon Effect. We suggest guidelines for the design of game controls for flexible devices. We implemented the proposed gestures into six games using an interactive flexible prototype in our second study. Our results are similar between sizes, yet with an overall user preference for the smaller prototype. We observed that hand positioning is an important usability issue to consider when designing flexible devices.
Smart phones have become the hub of people lives due to the overwhelming number and extensive range of apps available in app stores that are available to support their daily tasks. On average, smart phone users have around 100 apps installed on their devices and the number is ever growing. Thus, it becomes crucial to make sure they can quickly access these apps. In this paper, we present an exploratory study to understand users' memorability of their self-defined gestures for 15 frequently used mobile apps. The results show that although participants recalled their self-defined gestures most of the time, there are still certain factors that can influence their recall. The paper further analyses the underlining reasons and discusses how such issues could be addressed from a technical perspective.
This paper explores the use of haptic stimuli as non-visual affordances to assist in learnability of bend gestures. We tested 48 haptic Tactons with simulated blind participants to understand what haptic sensation could intuitively map to bend location and direction. We identify that a short, single motor Tacton indicates reliably a bend location, while participants agreed that the combination of two motors with varying intensities could indicate bend direction. This work is the first to explore the use of Tactons to communicate bend gesture location and direction, to eventually create a tactile interaction method for blind smartphone users
This paper explores the novel context of using bend gestures as a primary method of interaction for the blind. Our preliminary study assesses if this more tactile experience could enhance the usability and accessibility of technology for blind users, by comparing bend and touch interactions with simulated blind participants. Both input techniques showed similar results, indicating that bend gestures have potential in this context. We identify results that can help shape future research in this accessibility area, and potentially increase the overall interaction experience for screen reader based smartphones.
Smartphones are becoming larger, mainly because bigger screens offer a better experience for viewing content. One drawback of larger screens is that they make single-hand interactions difficult because of hard to reach touch targets and of the need to re-grip the device, both factors significantly reducing their usability. Flexible smartphones offer an opportunity for addressing this issue. We first set out to determine the use of common single-hand mobile interactions through an online survey. Then, we designed and evaluated one-handed deformable gestures that offer the potential for addressing the finger reach limitation on large smartphones. We identified that the top right up bend and the center squeeze up gestures are the fastest and preferred gestures. We found no hand preference, which indicates that the gestures could be implemented to fit the needs of a wider range of the population, instead of favoring right-handed users. Finally, we discuss the impact on deformable gestures on one-handed interactions issues.
Caret movement is difficult with touch input because of finger occlusion and the imprecision of interacting with a small display using fingers. We hypothesize that bend might provide a better text manipulation solution. We propose bend gestures to move a single caret or dual carets without requiring hand repositioning. We create a deformable prototype and implement the gestures. We present our bend interactions for caret manipulation and discuss our prototype.
We present a predictive model for the functional area of the thumb on a touchscreen surface: the area of the interface reachable by the thumb of the hand that is holding the device. We derive a quadratic formula by analyzing the kinematics of the gripping hand. Model fit is high for the thumb-motion trajectories of 20 participants. The model predicts the functional area for a given 1) surface size, 2) hand size, and 3) position of the index finger on the back of the device. Designers can use this model to ensure that a user interface is suitable for interaction with the thumb. The model can also be used inversely - that is, to infer the grips assumed by a given user interface layout.
Device deformation allows new types of gestures to be used in interaction. We identify that the gesture/use-case pairings proposed by interaction designers are often driven by factors relating improved tangibility, spatial directionality and strong metaphorical bonds. With this starting point, we argue that some of the designs may not make use of the full potential of deformation gestures as continuous, bipolar input techniques. In two user studies, we revisited the basics of deformation input by taking a new systematic look at the question of matching gestures with use cases. We observed comparable levels of UX when using bend input in different continuous bipolar interactions, irrespective of the choice of tangibility, directionality and metaphor. We concluded that device bend gestures use their full potential when used to control continuous bipolar parameters, and when quick reactions are needed. From our studies, we also identify relative strengths of absolute and relative mappings, and report a Fitts' law study for device bending input.
Prototypes for handheld, flexible devices are becoming popular in the research community. We explore opportunities in the domain of mobile gaming with flexible devices, by focusing on deformable inputs to control navigation in 3D virtual environments. We compare two sets of bend gestures to control a first person camera in a 3D maze, one inspired by console game controllers, and the other inspired by PC game controls (i.e. mouse and keyboard). Our results shows that users prefer the set inspired by the console controller: moving forward and backwards mapped to the top left corner, turning to the top right corner, and strafing to the bottom right corner. This results in lower wall collisions and an overall better user experience. We propose design recommendations to create deformable game controls in 3D spaces.
Deformable user interfaces leverage the physical actions we use intuitively to interact with real objects. It is therefore important to consider a prototype's physical characteristics when conducting research on deformable interactions. To create an authentic product experience, the authors set out to build a prototype that would mimic potential commercialized flexible devices and achieve the realism often lacking in low-fidelity prototypes. In this article, they outline the design and fabrication process to create Bendy, a prototype for studying deformable user interfaces. Their method creates prototypes quickly (one day) and inexpensively (approximately US $70) by using readily available materials. In addition, the process lets other researchers customize physical properties and interaction language to fit their specific purposes. The deformable prototype is composed of three main layers: a flexible plastic, an array of bend sensors connected to a flexible circuit, and a layer of silicone that encloses the sensors and circuit. The authors describe the fabrication process and demonstrate their method with two additional case studies. This article is part of a special issue on fabrication and printing. The Web extra shows the fabrication process, from the circuit design to printing and testing our flexible circuit, to creating the final Bendy prototype and playing Pac-Man using our deformable prototype. The web extra for this article can be found at http://youtu.be/PJ5ee5gAbm8.
Flexpad is an interactive system that combines a depth camera and a projector to transform sheets of plain paper or foam into flexible, highly deformable, and spatially aware handheld displays. We present a novel approach for tracking deformed surfaces from depth images in real time. It captures deformations in high detail, is very robust to occlusions created by the user's hands and fingers, and does not require any kind of markers or visible texture. As a result, the display is considerably more deformable than in previous work on flexible handheld displays, enabling novel applications that leverage the high expressiveness of detailed deformation. We illustrate these unique capabilities through three application examples: curved cross-cuts in volumetric images, deforming virtual paper characters, and slicing through time in videos. Results from two user studies show that our system is capable of detecting complex deformations and that users are able to perform them quickly and precisely.
Deformable user interfaces have received increasing attention in recent HCI research. However, the effect of device size on deformable user interfaces has not been studied yet. This study is aimed to investigate how the size of a deformable device affects users' interaction behavior and preferences. We observed users interacting with deformable mockup displays of two different sizes. Overall, 36 participants provided 769 user-defined gestures for 11 basic commands. We compared and discussed users' preferences toward two different sizes of deformable devices. We also covered user-defined gestures and patterns of use for each device size. As a preliminary study for understanding form factors for designing deformable user interfaces, this study clearly show that the device size is an important factor to consider when designing mobile devices which can be deformed.
We present FlexView, a set of interaction techniques for Z-axis navigation on touch-enabled flexible mobile devices. FlexView augments touch input with bend to navigate through depth-arranged content. To investigate Z-axis navigation with FlexView, we measured document paging efficiency using touch, against two forms of bend input: bending the side of the display (leafing) and squeezing the display (squeezing). In addition to moving through the Z-axis, the second experiment added X-Y navigation in a pan-and-zoom task. Pinch gestures were compared to squeezing and leafing for zoom operations, while panning was consistently performed using touch. Our experiments demonstrate that bend interaction is comparable to touch input for navigation through stacked content. Squeezing to zoom recorded the fastest times in the pan-and-zoom task. Overall, FlexView allows users to easily browse depth arranged information spaces without sacrificing traditional touch interactions.
Deformable User Interfaces (DUIs) are increasingly being proposed for new tangible and organic interaction metaphors and techniques. To design DUIs, it is necessary to understand how deforming different materials manually using different gestures affects performance and user experience. In the study reported in this paper, three DUIs made of deformable materials with different levels of stiffness were used in navigation tasks that required bending and twisting the interfaces. Discrete and continuous deformation gestures were used in each case. Results showed that the stiffness of the material and the type of gesture affected performance and user experience in complex ways, but with a pervading pattern: using discrete gestures in very short navigation distances and continuous gestures otherwise, plus using lower-stiffness materials in every case, was beneficial in terms of performance and user experience.