Ken Hinckley’s research while affiliated with Microsoft and other places

Reaching objects in a dynamic environment requires fast online corrections that compensate for sudden object shifts or postural changes. Previous studies revealed the key role of visually monitoring the hand-to-target distance throughout action execution. In the current study, we investigate how sensorimotor asymmetries associated with space perception, brain lateralization and biomechanical constraints, affect the efficiency of online corrections. Participants performed reaching actions in virtual reality, where the virtual hand was progressively displaced from the real hand to trigger online corrections, for which it was possible to control the total amount of the redirection and the region of space in which the action unfolded. The efficiency of online corrections and the degree of awareness of the ensuing motor corrections were taken as assessment variables. Results revealed more efficient visuo-motor corrections for actions redirected towards, rather than away from the body midline. The effect is independent on the reaching hand and the hemispace of action, making explanations associated with laterality effects and biomechanical constraints improbable. The result cannot either be accounted for by the visual processing advantage in the straight-ahead region. An explanation may be found in the finer sensorimotor representations characterizing the frontal space proximal to body, where a preference for visual processing has been documented, and where high-value functional actions, like fine manipulative skills, typically take place. This article is part of a discussion meeting issue ‘New approaches to 3D vision’.

p>Manipulation of hand-held objects in Virtual Reality (VR) requires input tracking with high freedom of movement, as well as haptic feedback of hand-object interactions. Through our prototypes we demonstrate a pragmatic approach to haptic feedback on controllers that render human scale forces. Our devices manifest haptic simulation of compliance, texture, surface normals, sizes, weights, and kinematic forces. These are brought to bear on hand-object interaction primitives such as palpation, manipulation, grasping, squeezing, cutaneous touch, stable grip, dexterity, and precision manipulation, which are collected as a taxonomy and represent a layer between the inherent haptic properties of the objects and the hand interaction of the operator. We implement prototypes that simulate the functional affordances of each of these aspects, and characterize their performance in human perception studies. Our work offers a model of hand-object interactions that goes beyond force rendering on a finger-by-finger basis (as typical of hand exoskeletons and gloves). </p

p>Manipulation of hand-held objects in Virtual Reality (VR) requires input tracking with high freedom of movement, as well as haptic feedback of hand-object interactions. Through our prototypes we demonstrate a pragmatic approach to haptic feedback on controllers that render human scale forces. Our devices manifest haptic simulation of compliance, texture, surface normals, sizes, weights, and kinematic forces. These are brought to bear on hand-object interaction primitives such as palpation, manipulation, grasping, squeezing, cutaneous touch, stable grip, dexterity, and precision manipulation, which are collected as a taxonomy and represent a layer between the inherent haptic properties of the objects and the hand interaction of the operator. We implement prototypes that simulate the functional affordances of each of these aspects, and characterize their performance in human perception studies. Our work offers a model of hand-object interactions that goes beyond force rendering on a finger-by-finger basis (as typical of hand exoskeletons and gloves). </p

Structured note-taking forms such as sketchnoting, self-tracking journals, and bullet journaling go beyond immediate capture of information scraps. Instead, hand-drawn pride-in-craftmanship increases perceived value for sharing and display. But hand-crafting lists, tables, and calendars is tedious and repetitive. To support these practices digitally, Style Blink ("Style-Blocks+Ink") explores handcrafted styling as a first-class object. Style-blocks encapsulate digital ink, enabling people to craft, modify, and reuse embellishments and decorations for larger structures, and apply custom layouts. For example, we provide interaction instruments that style ink for personal expression, inking palettes that afford creative experimentation, fillable pens that can be "loaded"with commands and actions to replace menu selections, techniques to customize inked structures post-creation by modifying the underlying handcrafted style-blocks and to re-layout the overall structure to match users' preferred template. In effect, any ink stroke, notation, or sketch can be encapsulated as a style-object and re-purposed as a tool. Feedback from 13 users show the potential of style adaptation and re-use in individual sketching practices.

To better ground technical (systems) investigation and interaction design of cross-device experiences, we contribute an in-depth survey of existing multi-device practices, including fragmented workflows across devices and the way people physically organize and configure their workspaces to support such activity. Further, this survey documents a historically significant moment of transition to a new future of remote work, an existing trend dramatically accelerated by the abrupt switch to work-from-home (and having to contend with the demands of home-at-work) during the COVID-19 pandemic. We surveyed 97 participants, and collected photographs of home setups and open-ended answers to 50 questions categorized in 5 themes. We characterize the wide range of multi-device physical configurations and identify five usage patterns, including: partitioning tasks, integrating multi-device usage, cloning tasks to other devices, expanding tasks and inputs to multiple devices, and migrating between devices. Our analysis also sheds light on the benefits and challenges people face when their workflow is fragmented across multiple devices. These insights have implications for the design of multi-device experiences that support people's fragmented workflows.