William R. Provancher’s research while affiliated with Santa Clara University and other places

During interaction with objects using a tool, we experience force and tactile feedback. One form of tactile feedback is local fingerpad skin deformation. In this paper, we provide haptic feedback to users of a teleoperation system through a skin deformation tactile feedback device. The device is able to provide tangential and normal skin deformation in a coupled manner, and is designed so that users can grasp it with a precision grip using multiple fingerpads. By applying skin deformation feedback on multiple fingerpads, the device is able to provide multi-degree-of-freedom interaction force direction and magnitude information to the user. To evaluate the effectiveness of this approach for the performance of teleoperated manipulation tasks, we performed a study in which 20 participants used a teleoperation system to perform one of two manipulation tasks (peg transfer and tube connection) using force feedback, skin deformation feedback, and the combination of both feedback. Results showed that participants are able to use all feedback to improve task performance compared to the case without haptic feedback, although the degree of improvement depended on the nature of the task. The feedback also improved situation awareness, felt consistent with prior experience, and did not affect concentration on the task, as reported by participants.

We investigate the accuracy with which the haptic sharpness perception of a virtual edge is matched to that of a real edge and the effect of the virtual surface stiffness on the match. The perceived sharpness of virtual edges was estimated in terms of the point of subjective equality (PSE) when participants matched the sharpness of virtual edges to that of real edges with a radius of 0.5, 2.5 and 12.5 mm over a virtual stiffness range of 0.6 to 3.0 N/mm. The perceived sharpness of a real and a virtual edge of the same radius was significantly different under all but one of the experimental conditions and there was a significant effect of virtual surface stiffness on the accuracy of the match. The results suggest that the latter is presumably due to a constant penetration force employed by the participants that influenced the penetration depth and perceived sharpness of virtual edges at different surface stiffness levels. Our findings provide quantitative relations for appropriately offsetting the radii of virtual edges in order to achieve the desired perceived sharpness of virtual edges.

An admittance-type robotic manipulator is a nonbackdrivable device whose motion is controlled to move in response to a user-applied force, typically with velocity proportional to force. This study characterizes the ability of ten human subjects to accurately and precisely control the velocity of such a device, using force applied by the index finger, as the user is provided visual feedback of device motion and a target velocity on a display. The admittance, the velocity, and the visualization scale factor are varied in a full factorial design, with parameter levels representative of microsurgery/micromanipulation tasks. The results indicate that: visual scaling has no effect, for the levels tested; low velocity at high admittance results in reduced precision and accuracy; high velocity at low admittance results in reduced accuracy; and an admittance-dependent velocity exists at which accuracy is maximized. The results suggest that gain scheduling will result in improved performance.

Skin stretch is a powerful haptic effect with a great potential as a feedback mechanism for digital gaming applications. For example, it has been shown to communicate directional information accurately to game players. However, the existing devices apply stretch to the tip of index finger except the Reactive Grip game controller by Tactical Haptics, which applies skin stretch to a user’s palm and finger pads. We have designed a compact hand-held haptic device that applies skin stretch to the palm via an actuated tactor. Compared to the fingertip, the palm is slightly less sensitive to skin stretch but affords larger stretch area. The stretch area of the palm enables us to control both tactor displacement and speeds for a broader range, resulting in richer haptic feedback. Using this device, we conduct experiments with 8 participants to investigate the effects of tactor displacement, speed, direction and hand orientation on perceived magnitude of skin stretch. The results of the study show that not only the tactor displacement but also the speed has a significant effect on the perceived intensity of skin stretch and the mapping function between them is nonlinear. Moreover, it appears that the tactile sensitivity of human palm to skin stretch is not homogeneous and stretch applied to the radial aspect of palm (towards the thumb) results in higher intensity than that of ulnar aspect.

This chapter provides an overview of force and tactile sensing, with the primary emphasis placed on tactile sensing. We begin by presenting some basic considerations in choosing a tactile sensor and then review a wide variety of sensor types, including proximity, kinematic, force, dynamic, contact, skin deflection, thermal, and pressure sensors. We also review various transduction methods, appropriate for each general sensor type. We consider the information that these various types of sensors provide in terms of whether they are most useful for manipulation, surface exploration or being responsive to contacts from external agents. Concerning the interpretation of tactile information, we describe the general problems and present two short illustrative examples. The first involves intrinsic tactile sensing, i. e., estimating contact locations and forces from force sensors. The second involves contact pressure sensing, i. e., estimating surface normal and shear stress distributions from an array of sensors in an elastic skin. We conclude with a brief discussion of the challenges that remain to be solved in packaging and manufacturing damage-tolerant tactile sensors.

When we use a tool to explore or manipulate an object, friction between the surface of the tool and the fingerpads generates skin stretch cues that are related to the interaction forces between the tool and the object. In this study, we emulate these naturally occurring skin stretch cues in order to convey force direction and magnitude information to users during teleoperation. We hypothesize that skin stretch feedback is a useful substitute for kinesthetic force feedback in force-sensitive teleoperated tasks. In this study, ten participants performed teleoperated palpation to determine the orientation of a stiff region in a surrounding artificial tissue using five feedback conditions: skin stretch, force, reduced gain force, graphic, and vibration. When participants received skin stretch feedback, they localized the stiff region as well as with force feedback, with no increase in task completion time. Additionally, participants receiving skin-stretch feedback localized the stiff region statistically significantly more accurately than those using vibration feedback. Although participants using skin stretch exhibited higher interaction forces than when using force, vibration, and graphical feedback, skin stretch statistically significantly decreased interaction forces compared with reduced gain force feedback. Thus, skin-stretch feedback is a compelling substitute for force feedback and may be useful in scenarios where force feedback is reduced or infeasible.

When a person interacts with an environment using a tool, he/she receives tactile information in the form of fingerpad skin deformation. Different interaction forces and torques on the tool cause different skin deformation patterns on the fingerpads. We designed a 6-degree-of-freedom tactile device that creates similar skin deformation patterns on the fingerpads. The device communicates force and torque information by translating and rotating skin deformation tactors relative to the fingerpads. An experiment was conducted to determine participants' ability to use skin deformation tactile cues to perform a peg-in-hole insertion task. Results show that participants can use the tactile cues to reduce interaction force and torque, and they use the tactile force cues to reduce interaction force more than they use the tactile torque cues to reduce interaction torque. Rendering force and torque cues simultaneously causes device saturation and degrades user performance. These results suggest that additional training may help participants use the skin deformation torque cues, and motivate a tactile device design that decouples force and torque skin deformation rendering to minimize device saturation. Fingerpad skin deformation is a promising form of tactile feedback to convey force and torque information in teleoperation systems such as robot-assisted surgery, where force feedback may be undesirable due to stability and safety concerns.

Teleoperated robots are used in a variety of applications. The immersive nature of the teleoperated experience is often limited by a lack of haptic information. However, in many applications there are difficulties conveying force information due to limitations in hardware fidelity and the inherent tradeoffs between stability and transparency. In situations where force feedback is limited, it is possible to use sensory substitution methods to convey this force information via other sensory modalities. We hypothesize that skin stretch feedback is a useful substitute for kinesthetic force feedback in force-sensitive teleoperated tasks. We created and tested a tactile device that emulates the natural skin deformation present during tool mediated manual interaction. With this device, experiment participants performed teleoperated palpation to determine the orientation of a stiff region in a surrounding artificial tissue using skin stretch, force, reduced gain force, graphic, or vibration feedback. Participants using skin stretch feedback were able to determine the orientation of the region as accurately as when using force feedback and significantly better than when using vibration feedback, but also exhibited higher interaction forces. Thus, skin stretch feedback may be useful in scenarios where force feedback is reduced or infeasible.

During multi-lateral collaborative teleoperation, where multiple human or autonomous agents share control of a teleoperation system, it is important to be able to convey individual user intent. One option for conveying the actions and intent of users or autonomous agents is to provide force guidance from one user to another. Under this paradigm, forces would be transmitted from one user to another in order to guide motions and actions. However, the use of force guidance to convey intent can mask environmental force feedback. In this paper we explore the possibility of using tactile feedback, in particular skin deformation feedback, skin deformation feedback to convey collaborative intent while preserving environmental force perception. An experiment was performed to test the ability of participants to use force guidance and skin deformation guidance to follow a path while interacting with a virtual environment. In addition, we tested the ability of participants to discriminate virtual environment stiffness when receiving either force guidance or skin deformation guidance. We found that skin deformation guidance resulted in a reduction of path-following accuracy, but increased the ability to discriminate environment stiffness when compared with force feedback guidance.

The Planar Active Handrest (PAHR) is a large workspace assistive device that improves user precision manipulation. The PAHR is restricted in its use, because it does not rotate in the horizontal plane, which limits the workspace size and reduces user comfort. This paper evaluates an improved device, the Enhanced Planar Active Handrest (E-PAHR), which allows lateral/medial rotation of the upper arm. Under the E-PAHR design, the desired hand position can be obtained with redundant arm and device configurations. As such, we gave consideration to controller input choices and resultant device motions that are available. Three experiments evaluated the controller designs to select the most effective method to control our device. We conclude that a rotational DOF (degrees of freedom) allows the E PAHR to better follow the kinematics of a user’s planar arm movements while allowing skill level equal to the PAHR, with reduced user force input and lower perceived exertion.