No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Bi-manual skin stretch feedback embedded within a game controller
Abstract
This paper presents the design and initial characterization of a video game controller prototype with tactile skin stretch displays embedded into its thumb joysticks. This work builds on previous work using lateral skin stretch at a single fingertip to provide direction cues. Our game controller allows a gamer to receive directional tactile feedback through the same interface he/she makes game inputs, and has the potential to enhance the gaming experience. Direction cues correspond to the direction of applied skin stretch. Modern game controllers typically have users angle their thumbs diagonally inwards to reach the joysticks, however, this may affect the cognition of skin stretch cues. Testing with a game controller prototype that allows for both forward and angled alignment of the user's thumbs shows no significant difference in accuracy or response time due to thumb orientation. Our results show no significant reduction in performance due to the required mental rotation of stimuli delivered in the angled thumb configuration. Furthermore, the angled thumb configuration appears to have ergonomic advantages. A more integrated game controller prototype is in development, which will permit further human factors testing to examine effects such as stimulus masking.
... The vibration create a sensations as if user are in the game themselves. Other types of controller even provide sensation like bumpy and rough surfaces and even a recoil of a gun although the track pad surface is smooth and flatten [3]. ...
... As parts of sensory substitution devices (SSDs), they can compensate for the loss of sensory function and augment sensory experiences of people with permanent (visually-or hearing impaired) or temporary deficiencies (e.g., rescue teams in difficult environments) (Bach-y-Rita and Kercel 2003;Cosgun et al. 2014;Hoffmann et al. 2018;Kristjánsson et al. 2016). Furthermore, vibrotactile arrays worn around the waist can assist people with balance impairments by providing vibratory feedback (Wall and Weinberg 2003) and can enhance immersive experiences in virtual environments for entertainment, or professional training (Faroque et al. 2015;Guinan et al. 2012). In the last decade, vibrotactile equipment has become more available, affordable, and less intrusive (Choi and Kuchenbecker 2013). ...
Vibrotactile displays can compensate for the loss of sensory function of people with permanent or temporary deficiencies in vision, hearing, or balance, and can augment the immersive experience in virtual environments for entertainment, or professional training. This wide range of potential applications highlights the need for research on the basic psychophysics of mechanisms underlying human vibrotactile perception. One key consideration when designing tactile displays is determining the minimal possible spacing between tactile motors (tactors), by empirically assessing the maximal throughput of the skin, or, in other words, vibrotactile spatial acuity. Notably, such estimates may vary by tactor type. We assessed vibrotactile spatial acuity in the lower thoracic region for three different tactor types, each mounted in a 4 × 4 array with center-to-center inter-tactor distances of 25 mm, 20 mm, and 10 mm. Seventeen participants performed a relative three-alternative forced-choice point localization task with successive tactor activation for both vertical and horizontal stimulus presentation. The results demonstrate that specific tactor characteristics (frequency, acceleration, contact area) significantly affect spatial acuity measurements, highlighting that the results of spatial acuity measurements may only apply to the specific tactors tested. Furthermore, our results reveal an anisotropy in vibrotactile perception, with higher spatial acuity for horizontal than for vertical stimulus presentation. The findings allow better understanding of vibrotactile spatial acuity and can be used for formulating guidelines for the design of tactile displays, such as regarding inter-tactor spacing, choice of tactor type, and direction of stimulus presentation.
... [1], [3]), and are very popular in scientific research of physical human-robot interaction, neuroscience, and medicine. However, their cost and size limits their applicability to entertainment and education, where compact and low-cost devices are needed [9], [12]. ...
If it becomes possible to represent the feelings of manipulating tools in virtual space, it would be applicable to the skill transfer and training. So, in recent years, a variety of tool-type haptic devices have been developed. Most existing tool-type haptic devices need to be grounded or attached to the user's body ([Sandor et al. 2007], [Hirose et al. 2001] et al), because the reaction force of displayed stimuli should be supported somewhere. However, grounded devices usually limit the workspace, and attached devices demand to wear the device. Recently, devices that only render cutaneous sensation were introduced. These devices stimulate the mechanoreceptors in the skin, and are not constrained by grounding [Guinan et al. 2013], [Minamizawa et al. 2008]. To the best of our knowledge, such cutaneous devices have never been used for a tool-mediated interaction with objects.
... [1], [3]), and are very popular in scientific research of physical human-robot interaction, neuroscience, and medicine. However, their cost and size limits their applicability to entertainment and education, where compact and low-cost devices are needed [9], [12]. ...
Force feedback in tool-mediated interactions with the environment is important for successful performance of complex tasks in our daily life as well as in specialized fields like medicine. Stylus-based haptic devices are studied and used extensively, and most of these devices require either grounding or attachment to the body of the user. Recently, non-grounded haptic devices are getting an increasing attention. In this paper, we propose a novel method to represent the vertical forces that are applied on the tip of a tool: a non-grounded rotation mechanism that mimics the cutaneous sensation that is caused by these tool-tip forces. To evaluate this method, we developed a novel ungrounded haptic device-HapSticks-that renders the sensation of manipulating objects using chopsticks. First, we present the novel mechanism, and test the pressure that it applies on the hand of the user when rendering a force at the tip of the tool in comparison to applying a real force at the tip of the tool. Next, we used the mechanism to build the HapSticks device as an example of an application of the proposed method, and present a psychophysical evaluation of this device in a virtual weight discrimination task.
... In related work, we are exploring the use of skin stretch feedback in video-game controllers and joysticks [22], [24], [25]. ...
In this paper, we show that a simple haptic device can accurately guide users through planar hand movements. The device guides the user through skin stretch feedback on the fingerpad of the user's index finger. In an angle matching test evaluating two types of stimuli, users are able to discriminate between eight stimulus directions and match the motion of their hand to the stimulus direction with 10 degree accuracy. In two motion guidance tests, haptic cues effectively guide users' arm motions through the full extent of their reachable workspace. Real-time corrective feedback greatly improves user performance, keeping average user hand motions within 12 mm of the prescribed path and within 4 degree of the indicated directions. Additionally, the paper shows that participants exhibit distorted haptic perceptual responses, finding that the distortion causes a response direction bias, but that appropriate haptic feedback can correct for the effect. Such motion guidance has applications in human-machine interaction, such as upper-extremity rehabilitation.
Force feedback in tool-mediated interactions with the environment is important for the successful performance of complex tasks in daily life as well as in specialized fields such as medicine. Most stylus-based haptic devices require either grounding or attachment to the user’s body. Recently, non-grounded haptic devices have attracted a growing interest. In this study, we propose a non-grounded rotation mechanism to represent the vertical forces applied on the tip of a tool by mimicking the cutaneous sensations that are caused by such forces. As an example of an application of our method, we developed a non-grounded haptic device called HapSticks, which mimicked the sensation of manipulating objects using chopsticks. First, using an adjustment paradigm, we directly compared a virtual weight rendered by our device and a real weight to investigate the relation of real weight and virtual weight. Next, we used a forced choice constant stimuli paradigm in a virtual and a real weight discrimination task. We conclude that our novel device renders a reliable illusion of sensed weight that leads to a discrimination ability that is typical of virtual-reality applications but worse than the discrimination between real weights.
Skin stretch is a powerful haptic effect with a great potential as a feedback mechanism for digital gaming applications. For example, it has been shown to communicate directional information accurately to game players. However, the existing devices apply stretch to the tip of index finger except the Reactive Grip game controller by Tactical Haptics, which applies skin stretch to a user’s palm and finger pads. We have designed a compact hand-held haptic device that applies skin stretch to the palm via an actuated tactor. Compared to the fingertip, the palm is slightly less sensitive to skin stretch but affords larger stretch area. The stretch area of the palm enables us to control both tactor displacement and speeds for a broader range, resulting in richer haptic feedback. Using this device, we conduct experiments with 8 participants to investigate the effects of tactor displacement, speed, direction and hand orientation on perceived magnitude of skin stretch. The results of the study show that not only the tactor displacement but also the speed has a significant effect on the perceived intensity of skin stretch and the mapping function between them is nonlinear. Moreover, it appears that the tactile sensitivity of human palm to skin stretch is not homogeneous and stretch applied to the radial aspect of palm (towards the thumb) results in higher intensity than that of ulnar aspect.
This paper presents the design and development of handheld gaming devices that provide a new form of touch feedback: skin stretch feedback. Our prior work showed 1-1.5 mm of skin stretch applied to the fingertips (or thumb tips) could provide direction cues with high accuracy. The direction of each cue corresponded to the direction of applied skin stretch. Haptic feedback via tactile skin stretch can be used to provide tactile gaming effects and directional information through the same interface used for game inputs. Tests conducted with devices that resemble modern game controllers indicated which grip style is preferred by users and how to present direction cues to achieve the highest recognition rates. The main contribution of this paper is to show an initial demonstration of the benefits of skin stretch feedback in a multimodal gaming scenario.
This paper presents the design of a game controller with integrated skin stretch feedback, a new form of touch feedback. This form of feedback can be used to provide directional information, as well as tactile gaming effects to a user through the input thumb joysticks. Prior testing has shown that a user can perceive and respond to single direction cues given through skin stretch feedback at the thumb joysticks. This paper presents further experiments to characterize user interaction and responses to feedback as a function of the relative timing between skin stretch cues given on both of the controller's thumb joysticks.
In this paper, we present three field-based evaluations of a tactile land navigation system. In Experiment 1, we transition from a laboratory setting to rugged terrain used to train US Army soldier land navigation. Navigation in this challenging terrain requires careful attention to one's surroundings. Participants navigated 3 waypoints along 600 meters through heavily wooded terrain, using 1) map and compass, 2) standard alpha-numeric handheld GPS device, and 3) the tactile GPS system, while also responding to radio requests for information. Experiment 2 used the same challenging terrain during night operations, where participants must also search for live and silhouette targets, using 1) handheld GPS device, 2) head-mounted map-based GPS, and 3) the tactile GPS system. In addition to navigating, participants searched for silhouette and live (human) targets. Experiment 3 had participants navigate with 1) a commercial GPS arrow display, 2) the tactile GPS system, and 3) both together. We conclude that tactile navigation displays can be used in strenuous outdoor environments and can outperform visual displays under conditions of high cognitive and visual workload.
This paper presents a tactile transducer device that is optimized from biomechanical data and has a compact, yet modular design. The tactile transducer comprises a 6 10 piezo bimorph actuator array with a spatial resolution of 1.8 1.2 millimeters and a wide tempo- ral bandwidth. The actuator mounting method was improved from a conventional cantilever method to a dual-pinned method, giving the actuator the capability of maximally deforming glabrous skin for laterotactile stimulation. The results were validated using a psy- chophysical method. The tactile display device is mod- ular, appeals to ordinary fabrication methods, and can be assembled and dismantled in a short time for debug- ging and maintenance. It weighs 60 g, it is self-contained in a 150 cm3 volume and may be interfaced to most com- puters, provided that two analog outputs and six digital io lines are available. Psychophysical experiments were carried out to assess its eectiveness in rendering virtual tactile features.
This article describes the design of a high-bandwidth, iron-less, recoil-based electromagnetic vibrotactile actuator. Its working principle, the theoretical analysis, the method used to determine its transfer function, its scaling properties and its design constraints are discussed along with its fabrication and possible improvements.
Multiple resource models suggest that there will be less interference between tasks that do not share the same processing resources or modalities. Most in-vehicle navigational systems rely on some type of auditory modality, which can be problematic in situations with high levels of background noise or when one is engaged in a secondary auditory task, such as talking on a cell phone. The haptic or tactile sensory modality may be an effective alternative for presenting important navigational information in a vehicle. The current study compared the effectiveness of auditory vs. tactile navigational instructions to the operator of a moving vehicle. Tactile shear cues were provided to the participants' index fingers as they gripped the steering wheel. These tactile cues provided directional information that coincide with the direction of the applied skin displacement. In single task driving, participants made accurate lane changes in response to both auditory and tactile instructions. In dual-task driving, where participants conversed on a cell phone, participants were less accurate with auditory instructions than for tactile instructions. This suggests that in some instances, presenting information via the tactile modality may bypass the bottleneck described by multiple resource theories when there is competition for the same resources. Copyright 2010 by Human Factors and Ergonomics Society, Inc. All rights reserved.
Several researchers have developed haptic devices capable of rendering directional stimuli. When these devices are integrated into mobile or handheld devices, it becomes possible for a user to hold the haptic device in any orientation and thereby receive directional stimuli that may be out of alignment with rest of the world. In such cases, it becomes necessary for the user to perform a mental transformation of the directional stimuli, so that the stimuli may be understood in a fixed or global reference frame. This paper addresses two questions: 1. Can users perform such transformations and successfully interpret stimuli, and 2. What cognitive processes are involved in these transformations? In our experiments, users performed timed identification of directional tactile stimuli with their hand in a variety of orientations around a single axis. The results show that: 1. Users can successfully identify directional stimuli both quickly and accurately, even when the stimuli are rendered in a rotated reference frame, and 2. These tasks involve the mental rotation of a spatial mental representation of the stimulus, and also show evidence of embodiment effects. Furthermore, small angles of rotation (up to ~40°) incur very little cognitive cost, suggesting that tactile direction stimuli delivered through a handheld device would be robust to variations in user hand orientation.
This paper reports on a series of user experiments evaluating the design of a multimodal test platform capable of rendering visual, audio, vibrotactile, and directional skin-stretch stimuli. The test platform is a handheld, wirelessly controlled device that will facilitate experiments with mobile users in realistic environments. Stimuli rendered by the device are fully characterized, and have little variance in stimulus onset timing. A series of user experiments utilizing navigational cues validates the function of the device and investigates the user response to all stimulus modes. Results show users are capable of interpreting all stimuli with high accuracy and can use the direction cues for mobile navigation. Tests included both stationary (seated) and mobile (walking a simple obstacle course) tasks. Accuracy and response time patterns are similar in both seated and mobile conditions. This device provides a means of designing and evaluating multimodal communication methods for handheld devices and will facilitate experiments investigating the effects of stimulus mode on device usability and situation awareness.
Tactile feedback could replace or augment visual and auditory communication in a range of important applications. This paper advances the field of tactile communication by presenting performance data on a variety of tactors and a finger restraint that is suitable for use in portable devices. Tactors, the contact elements between the device and the skin, and finger restraints were evaluated using a tangential skin displacement direction identification task. We tested tactors of three sizes and two different textures. Rough textured tactors improved communication accuracy compared to smooth tactors, but tactor size did not have a statistically significant effect. Aperture-based restraints of three sizes were evaluated on both the index finger and the thumb. The aperture-based restraint was effective when used on both the index finger and the thumb, with performances on par with our previously tested thimble-based restraint. Participants performed better with larger apertures than with smaller apertures, but there was no interaction between aperture size and finger size, meaning that the same aperture could be used with a range of finger sizes. Subjects' perceptual acuity varied with stimulus direction. We discuss the effects of contact force, finger size, and differences in perceptual acuity between the index finger and thumb.
Application of tangential skin displacement at the fingertip has been shown to be effective in communicating direction and has potential for several applications. We have developed a portable, fingertip-mounted tactile display capable of displacing and stretching the skin of the fingerpad, using a 7 mm hemispherical tactor. In vivo tests of fingerpad skin stiffness were performed to determine the forces required to effectively render stimuli. Other design parameters such as stimulus speed and displacement were derived from our earlier work. The tactile display is capable of rendering pm 1 mm of displacement at arbitrary orientations within a plane and with rates of approximately 5 mm/s. Compliance and backlash in the device's drive train were characterized using external measurements, and were compensated for in software to reduce the impact on device hysteresis.
A variety of tasks could benefit from the availability of direction cues that do not rely on vision or sound. The application of tangential skin displacement at the fingertip has been found to be a reliable means of communicating direction and has potential to be rendered by a compact device. Our lab has conducted experiments exploring the use of this type of tactile stimulus to communicate direction. Each subject pressed his/her right index fingertip against a 7 mm rounded rubber cylinder that moved at constant speed, applying shear force to deform the skin of the fingerpad. A range of displacements (0.05-1 mm) and speeds (0.5-4 mm/s) were tested. Subjects were asked to respond with the direction of the skin stretch, choosing from four directions, each separated by 90 degrees. Direction detection accuracy was found to depend upon both the speed and total displacement of the stimulus, with higher speeds and larger displacements resulting in greater accuracy. Accuracy rates greater than 95 percent were observed with as little as 0.2 mm of tangential displacement and at speeds as slow as 1 mm/s. Results were analyzed for direction dependence and temporal trends. Subjects responded most accurately to stimuli in the proximal and distal directions, and least accurately to stimuli in the ulnar direction. Subject performance decreased slightly with prolonged testing but there was no statistically significant learning trend. A second experiment was conducted to evaluate priming effects and the benefit of repeated stimuli. It was found that repeated stimuli do not improve direction communication, but subject responses were found to have a priming effect on future performance. This preliminary information will inform the design and use of a tactile display suitable for use in hand-held electronics.
The study of haptic interfaces focuses on the use of the sense of touch in human-machine interaction. This document presents a detailed investigation of lateral skin stretch at the fingertip as a means of direction communication. Such tactile communication has applications in a variety of situations where traditional audio and visual channels are inconvenient, unsafe, or already saturated. Examples include handheld consumer electronics, where tactile communication would allow a user to control a device without having to look at it, or in-car navigation systems, where the audio and visual directions provided by existing GPS devices can distract the driver’s attention away from the road. Lateral skin stretch, the displacement of the skin of the fingerpad in a plane tangent to the fingerpad, is a highly effective means of communicating directional information. Users are able to correctly identify the direction of skin stretch stimuli with skin displacements as small as 0.1 mm at rates as slow as 2 mm/s. Such stimuli can be rendered by a small, portable device suitable for integration into handheld devices. The design of the device-finger interface affects the ability of the user to perceive the stimuli accurately. A properly designed conical aperture effectively constrains the motion of the finger and provides an interface that is practical for use in handheld devices. When a handheld device renders directional tactile cues on the fingerpad, the user must often mentally rotate those cues from the reference frame of the finger to the world-centered reference frame where those cues are to be applied. Such mental rotation incurs a cognitive cost, requiring additional time to mentally process the stimuli. The magnitude of these cognitive costs is a function of the angle of rotation, and of the specific orientations of the arm, wrist and finger. Even with the difficulties imposed by required mental rotations, lateral skin stretch is a promising means of communicating information using the sense of touch with potential to substantially improve certain types of human-machine interaction.
The Ubi-Pen is a pen-like haptic interface incorporating a compact tactile display and a vibrating module. It can represent tactile patterns and provide feedback with the click of a button. It's also applicable to combined force- and tactile-feedback display.
This article examines why innovations such as the Sony EyeToy and Nintendo Wii have been so successful and discusses the research opportunities presented by the latest commercial push for spatial 3D interaction in games.
The Psychophysics Toolbox is a software package that supports visual psychophysics. Its routines provide an interface between a high-level interpreted language (MATLAB on the Macintosh) and the video display hardware. A set of example programs is included with the Toolbox distribution.
Haptics technology can simulate tactile and kinesthetic sensations so that computer interfaces not only look but also feel - in the literal sense of the word - like the interfaces we are accustomed to using in the real world. Although haptics in PC gaming began by focusing on kinesthetics, while haptics in console gaming began by focusing on vibrotactile effects, game players and market pressures are driving the wide adoption of both forms by both sectors.
Mobile Navigation using Haptic, Audio, and Visual Direction Cues with a Handheld Test PlatfonnThe Psychophysics Toolbox', Spatial Vision
- B T Gleeson
- J T Bever
- W R Provancher
- Ii
- D H Brainard
Gleeson, B.T.; de Bever, J.T.; Provancher, W.R.: 'Mobile Navigation using Haptic, Audio, and Visual Direction Cues with a Handheld Test Platfonn', IEEE Transactions on Haptics, (in press) [II] Brainard, D.H.: 'The Psychophysics Toolbox', Spatial Vision, 1997, 10, pp. 433-43