No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Altered touch: miniature haptic display with force, thermal and tactile feedback for augmented haptics
Abstract
In this paper, (1) we developed a fingertip haptic display with integrated force, tactile and thermal feedback in a miniature form-factor such that it can be worn easily and used with augmented reality applications without affecting the existing tracking technologies. (2) we propose the concept of "Altered Touch", where the integrated fingertip haptic display stated in (1) was used to alter the haptic properties of real objects by rendering projected visual and haptic feedback. The system consists of our own force display Gravity Grabber mechanism[Minamizawa et al. 2007] to render vertical, shearing forces, high frequency tactile vibrations, and a peltier module for thermal display. The integrated haptic display module weighs less than 50g, can be easily interfaced to a PC with just one micro USB cable, and works standalone from any other additional hardware. In this paper we use this wearable haptic actuator in several augmented reality applications to alter the softness/hardness and hot/cold sensation and several use cases have been discussed. Furthermore, the haptic display could be expanded to design a haptic glove that can interact with both virtual and augmented worlds.
... Others can render multimodal feedback by integrating tactile actuation inside the end-effector of a kinesthetic device [Choi et al., 2018;Wang et al., 2020a]. Further multimodal approaches integrate multiple tactile modalities such as skin-stretch, vibration and thermal feedback [Murakami et al., 2017]. In general, however, multimodal AHF interfaces are still rare [Wang et al., 2020a]. ...
... Lower center: ThermoVR HMD (extracted from [Peiris et al., 2017]; © 2017 ACM). Lower right: A multimodal device combining skin-stretching and thermal feedback (extracted from [Murakami et al., 2017]; © 2017 Murakami and co-authors). ...
... The device complements the vibrotactile feedback of commercial controllers with sensations of a rotating eccentric mass, wind, and temperature. Moreover, Murakami et al. [2017] combined temperature rendering with the belt mechanism of Gravity Grabber [Minamizawa et al., 2007] as shown in Figure 2.11. [Rietzler et al., 2017]; © 2017 Rietzler and co-authors). ...
This thesis advances haptic feedback for Virtual Reality (VR). Our work is guided by Sutherland's 1965 vision of the ultimate display, which calls for VR systems to control the existence of matter. To push towards this vision, we build upon proxy-based haptic feedback, a technique characterized by the use of passive tangible props. The goal of this thesis is to tackle the central drawback of this approach, namely, its inflexibility, which yet hinders it to fulfill the vision of the ultimate display. Guided by four research questions, we first showcase the applicability of proxy-based VR haptics by employing the technique for data exploration. We then extend the VR system's control over users' haptic impressions in three steps. First, we contribute the class of Dynamic Passive Haptic Feedback (DPHF) alongside two novel concepts for conveying kinesthetic properties, like virtual weight and shape, through weight-shifting and drag-changing proxies. Conceptually orthogonal to this, we study how visual-haptic illusions can be leveraged to unnoticeably redirect the user's hand when reaching towards props. Here, we contribute a novel perception-inspired algorithm for Body Warping-based Hand Redirection (HR), an open-source framework for HR, and psychophysical insights. The thesis concludes by proving that the combination of DPHF and HR can outperform the individual techniques in terms of the achievable flexibility of the proxy-based haptic feedback.
... The authors demonstrated the device's capabilities for a pickand-place task, obtaining overall lower interaction forces, completion time, and increased perceived effectiveness than in a visual-only condition. Other multi-cue devices featuring belt actuation have also been developed [73], [74]. However, these devices have yet to be evaluated within the context of weight rendering, particularly for those in which stimulation is relocated to the proximal phalanges. ...
... As discussed in Section II, numerous factors influence the perceived weight of an object, such as surface roughness, shape, or temperature. Some examples of haptic devices based on the proposed approaches integrating multimodal cues have already been presented [73], [74], [90], [112]. Combining existing actuation and sensing methods into a single device could help achieve this objective. ...
Haptic rendering of weight plays an essential role in naturalistic object interaction in virtual environments. While kinesthetic devices have traditionally been used for this aim by applying forces on the limbs, tactile interfaces acting on the skin have recently offered potential solutions to enhance or substitute kinesthetic ones. Here, we aim to provide an in-depth overview and comparison of existing tactile weight rendering approaches. We categorized these approaches based on their type of stimulation into asymmetric vibration and skin stretch, further divided according to the working mechanism of the devices. Then, we compared these approaches using various criteria, including physical, mechanical, and perceptual characteristics of the reported devices. We found that asymmetric vibration devices have the smallest form factor, while skin stretch devices relying on the motion of flat surfaces, belts, or tactors present numerous mechanical and perceptual advantages for scenarios requiring more accurate weight rendering. Finally, we discussed the selection of the proposed categorization of devices together with the limitations and opportunities for future research. We hope this study guides the development and use of tactile interfaces to achieve a more naturalistic object interaction and manipulation in virtual environments.
... Haptic devices and their possible effects and applications are presented in [34]. A more recent effort in this direction is presented in [124], where the authors present the concept of altered touch through an integrated fingertip haptic display that uses integrated force, tactile and thermal feedback, and the appropriate effects [35]. An extensive review of olfactory displays following different classifications (location, how the scent is delivered) is presented in [130]. ...
... Haptic devices and their possible effects and applications are presented in [34]. A more recent effort in this direction is presented in [124], where the authors present the concept of "altered touch" through an integrated fingertip haptic display that uses integrated force and tactile and thermal feedback and can be used with augmented reality applications. ...
Mulsemedia—multiple sensorial media—makes possible the inclusion of layered sensory stimulation and interaction through multiple sensory channels. The recent upsurge in technology and wearables provides mulsemedia researchers a vehicle for potentially boundless choice. However, in order to build systems that integrate various senses, there are still some issues that need to be addressed. This review deals with mulsemedia topics that remain insufficiently explored by previous work, with a focus on the multi-multi (multiple media-multiple senses) perspective, where multiple types of media engage multiple senses. Moreover, it addresses the evolution of previously identified challenges in this area and formulates new exploration directions.
... Tactile and thermal feedback have been combined in multisensory displays developed to create more realistic experiences in virtual or remote environments [72], [73], [74], [75]. For example, the addition of thermal cues to a display presenting vibrotactile feedback may assist in identifying both the surface texture and material composition of an object [76]. ...
This review focuses on the interactions between the cutaneous senses, and in particular touch and temperature, as these are the most relevant for developing skin-based display technologies for use in virtual reality (VR) and for designing multimodal haptic devices. A broad spectrum of research is reviewed ranging from studies that have examined the mechanisms involved in thermal intensification and tactile masking, to more applied work that has focused on implementing thermal-tactile illusions such as thermal referral and illusory wetness in VR environments. Research on these tactile-thermal illusions has identified the differences between the senses of cold and warmth in terms of their effects on the perception of object properties and the prevalence of the perceptual experiences elicited. They have also underscored the fundamental spatial and temporal differences between the tactile and thermal senses. The wide-ranging body of research on compound sensations such as wetness and stickiness has highlighted the mechanisms involved in sensing moisture and provided a framework for measuring these sensations in a variety of contexts. Although the interactions between the two senses are complex, it is clear that the addition of thermal inputs to a tactile display enhances both user experience and enables novel sensory experiences.
... There exists a wide range of wearable devices incorporating sensors and actuators that can provide multi-modal feedback in the domain of virtual reality (VR) and augmented reality (AR). However, thermal interaction in virtual reality (VR) is particularly emerging as studies have proven that providing thermal sensations can increase the sense of immersion and presence [24,30]. On the other hand, existing works have also extensively explored the characteristics and applications thermal feedback across various devices, including mobile devices [11,12,41,42], as well as smart wearable accessories like ear hooks [28], headbands [31,32], and finger rings [46] for mobile notifications, directional alerts and weather notifications, and VR headsets to increase immersion [33]. ...
The increasing interest in thermal haptic feedback devices, particularly for virtual reality (VR) applications, highlights the need for more immersive user experiences. However, replicating precise thermal sensations on the fingers remains challenging due to the complexity of finger joints and movements. In this paper, we introduce ThermoGrasp, a novel thermal display designed to enhance VR experiences by providing realistic thermal feedback during precision object grasping. ThermoGrasp is a modular wearable device that targets controlled thermal feedback on the distal phalanges. The implications of designing its VR application were assessed through two experimental studies. The first study focused on the device's ability to accurately convey thermal sensations across different fingers during various precision grasps. The second study investigated the overall haptic experience in VR, examining the impact of thermal feedback on user immersion and realism during interactions with objects of varying temperatures. Participants' subjective responses were analyzed based on factors such as autotelicity, expressiveness, immersion, realism, and harmony. The findings indicate that precise, localized thermal feedback significantly enhances the VR experience, offering a marked improvement over traditional haptic feedback methods.
... Wearable haptic interfaces that can display rich arrays of tactile cues play an essential role in bridging the gap between the physical world interactions and those in virtual environments. Most of these devices [1] [2] [3] embed a variety of sensors and actuators and are assembled in the form of a glove or ring to directly actuate fingertips, as these are the most sensitive skin regions and humans mostly use their fingertips to interact with physical objects. These devices can be programmed to display material sensations in virtual environments through controlled vibration, friction, pressure, or thermal stimuli on the fingertips. ...
Wearable devices that relocate tactile feedback from fingertips can enable users to interact with their physical world augmented by virtual effects. While studies have shown that relocating same-modality tactile stimuli can influence the one perceived at the fingertip, the interaction of cross-modal tactile stimuli remains unclear. Here, we investigate how thermal cues applied on the index finger's proximal phalanx affect vibrotactile sensitivity at the fingertip of the same finger when employed at varying contact pressures. We designed a novel wearable device that can deliver thermal stimuli at adjustable contact pressures on the proximal phalanx. Utilizing this device, we measured the detection thresholds of fifteen participants for 250 Hz sinusoidal vibration applied on the fingertip while concurrently applying constant cold (18 C°), neutral (32 C°), and warm (40 C°) stimuli at high (2 N) and low (0.5 N) contact pressures to the proximal phalanx. Our results revealed no significant differences in detection thresholds across conditions. These preliminary findings suggest that applying constant thermal stimuli to other skin locations does not affect fingertip vibrotactile sensitivity, possibly due to perceptual adaptation. However, the influence of dynamic multisensory tactile stimuli remains an open question for future research.
... This scientific finding has implications for the design of future sensory substitution and tactile communication systems. As tactile systems are surprisingly sensitive to a wide variety of features, state-of-the-art approaches in robotics and tactile communication systems have proposed ways to combine and present multiple features in order to make the most of their richness [49][50][51][52][53]. However, little attention has been paid to the limitations of feature binding. ...
For human sensory processing, cluttered real-world environments where signals from multiple objects or events overlap are challenging. A cognitive function useful in such situations is an attentional selection of one signal from others based on the difference in bound feature. For instance, one can visually select a specific orientation if it is uniquely colored. However, here we show that unlike vision, touch is very poor at feature-based signal selection. We presented two-orthogonal line segments with different vibration textures to a fingertip. Though observers were markedly sensitive to each feature, they were generally unable to identify the orientation bound with a specific texture when the segments were presented simultaneously or in rapid alternation. A similar failure was observed for a direction judgment task. These results demonstrate a general cognitive limitation of touch, highlighting its unique bias to integrate multiple signals into a global event rather than segment them into separate events.
... Work by Hirano et al. [17] and Punpongsanon et al. [18] investigated how virtual deforming can influence softness perception by visually changing how much it deforms. Research has looked at simulating various tactile properties without considering fully how the visual input affects the perception [10,[19][20][21]. ...
Underfoot haptics, a largely unexplored area, offers rich tactile information close to that of hand-based interactions. Haptic feedback gives a sense of physicality to virtual environments, making for a more realistic and immersive experience. Augmented Virtuality offers the ability to render virtual materials on a physical object, or haptic proxy, without the user being aware of the object’s physical appearance while seeing their own body. In this research, we investigate how the visual appearance of physical objects can be altered virtually to impact the tactile perception of the object. An Augmented Virtuality system was developed to explore this, and two tactile perception experiments, consisting of 18 participants, were conducted. Specifically, we explore whether changing the visual appearance of materials affects a person’s underfoot tactile perception and which tactile perception is most affected by the change through a within-subjects experiment. Additionally, the study examines whether people are aware of changes in visual appearance when focused on other tasks through a between-subjects experiment. The study showed that a change in visual appearance significantly impacts the tactile perception of roughness. Matching visual appearance to physical materials was found to increase awareness of tactile perception.
... Gabardi [11] designed a fingertip temperature rendering device that can move directionally around the fingertip, which can be used for transient thermal rendering in a virtual environment. In 2017, MurakamiT [12] developed a small fingertip tactile display integrating haptics and temperature feedback forces. The device is small in size and weighs less than 50 g. ...
Haptic interaction is a fundamental approach to our perception surrounding people; at this stage, haptic interaction is mainly used in virtual reality (VR) and remote medical procedures (teleoperation). The traditional large volume, complex operation seriously affects the function of immersive. This paper develops a multi-modal sensing interactive glove system for teleoperation and VR/AR. The device is small in size and convenient to wear; we integrate temperature sensing and tactile sensing into our glove system and deliver real-time and accurate environmental signals to the wearer through the acquisition and processing of information and program-controlled regulation to better realize the fusion of virtual and real. This paper first introduces the structure and working principle of the system and analyzes its performance. Then a sensory experiment based on human hand skin was designed, and volunteers were selected to experience it.
... The system uses basic tools such as a hot air blower, a fan, and a tool that can create mist as well as lighting to provide a fully immersive experience and mimic sensations such as desert heat or snow cold. Another idea is in [93] where the authors come up with a proposal dubbed as 'Altered touch' in which a small form factor fingertip-based haptic display is developed. The Altered touch can sense force, thermal, and tactile feedback and integrate with augmented reality. ...
The Metaverse offers a second world beyond reality, where boundaries are non-existent, and possibilities are endless through engagement and immersive experiences using the virtual reality (VR) technology. Many disciplines can benefit from the advancement of the Metaverse when accurately developed, including the fields of technology, gaming, education, art, and culture. Nevertheless, developing the Metaverse environment to its full potential is an ambiguous task that needs proper guidance and directions. Existing surveys on the Metaverse focus only on a specific aspect and discipline of the Metaverse and lack a holistic view of the entire process. To this end, a more holistic, multi-disciplinary, in-depth, and academic and industry-oriented review is required to provide a thorough study of the Metaverse development pipeline. To address these issues, we present in this survey a novel multi-layered pipeline ecosystem composed of (1) the Metaverse computing, networking, communications and hardware infrastructure, (2) environment digitization, and (3) user interactions. For every layer, we discuss the components that detail the steps of its development. Also, for each of these components, we examine the impact of a set of enabling technologies and empowering domains (e.g., Artificial Intelligence, Security & Privacy, Blockchain, Business, Ethics, and Social) on its advancement. In addition, we explain the importance of these technologies to support decentralization, interoperability, user experiences, interactions, and monetization. Our presented study highlights the existing challenges for each component, followed by research directions and potential solutions. To the best of our knowledge, this survey is the most comprehensive and allows users, scholars, and entrepreneurs to get an in-depth understanding of the Metaverse ecosystem to find their opportunities and potentials for contribution.
... Belt-based fingertip haptic devices have been proposed for rendering shear forces that integrate tactile, force and thermal feedback [24]. Tactor-displacement based haptic devices has also been proposed to render shear forces through the displacement of tactors with the fingertip [23]. ...
In-hand pivoting is one of the important manipulation skills that leverage robot grippers' extrinsic dexterity to perform repositioning tasks to compensate for environmental uncertainties and imprecise motion execution. Although many researchers have been trying to solve pivoting problems using mathematical modeling or learning-based approaches, the problems remain as open challenges. On the other hand, humans perform in-hand manipulation with remarkable precision and speed. Hence, the solution could be provided by making full use of this intrinsic human skill through dexterous teleoperation. For dexterous teleoperation to be successful, interfaces that enhance and complement haptic feedback are of great necessity. In this paper, we propose a cutaneous feedback interface that complements the somatosensory information humans rely on when performing dexterous skills. The interface is designed based on five-bar link mechanisms and provides two contact points in the index finger and thumb for cutaneous feedback. By integrating the interface with a commercially available haptic device, the system can display information such as grasping force, shear force, friction, and grasped object's pose. Passive pivoting tasks inside a numerical simulator Isaac Sim is conducted to evaluate the effect of the proposed cutaneous feedback interface.
... For example, Gravity Grabber [15] uses two motors with belts to present vibration and pressure stimuli. AlteredTouch [16] is an extension of Gravity Grabber [15], capable of presenting vibration, pressure, and thermal stimuli. Furthermore, haptic devices that focus on deforming the finger's skin and a haptic device [24] with two degrees of freedom (2-DOF) using link mechanisms [30] have been proposed. ...
... The early modes of supervisory control were limited to control based on audio, visual and force feedback signals of the robotic motion [36], [37]. However, with the advent of haptic based joysticks [38], [39], HGs [40]- [42], different mechanisms of tactile feedback [43]- [46] were also introduced. This allows the humans to get an approximate feel of the robot interacting with the environment. ...
The paper focuses on Haptic Glove (HG) based control of a Robotic Hand (RH) executing in-hand manipulation. A control algorithm is presented to allow the RH relocate the object held to a goal pose. The motion signals for both the HG and the RH are high dimensional. The RH kinematics is usually different from the HG kinematics. The variability of kinematics of the two devices, added with the incomplete information about the human hand kinematics result in difficulty in direct mapping of the high dimensional motion signal of the HG to the RH. Hence, a method is proposed to estimate the human intent from the high dimensional HG motion signal and reconstruct the signal at the RH to ensure object relocation. It is also shown that the lag in synthesis of the motion signal of the human hand added with the control latency of the RH leads to a requirement of the prediction of the human intent signal. Then, a recurrent neural network (RNN) is proposed to predict the human intent signal ahead of time.
... Although haptics enabled AR is in its infancy stage compared to that of the visual and olfactory technologies, extensive efforts have been made to develop mechanical haptics interfaces interfaces are demonstrated which provides different kinds of tactile sensations such as texture or shape, [16] vibration, [22] friction, [23] hardness and softness, [24] pressure, [23] and temperature. [25] One of the most crucial components to creating a realistic and interactive augmented environment is the thermal sensation. ...
In the past decade, remarkable progress has been made in the domain of augmented reality/virtual reality (AR/VR). The need for realistic and immersive augmentation has propelled the development of haptics interfaces‐enabled AR/VR. The haptics interfaces facilitate direct interaction and manipulation with both real and virtual objects, thus augmenting the perception and experiences of the users. The level of augmentation can be significantly improved by thermal stimulation or sensing, which facilitates a higher degree of object identification and discrimination. This review discusses the thermal technology‐enabled augmented reality and summarizes the recent progress in the development of different thermal technology such as thermal haptics including thermo‐resistive heater and Peltier devices, thermal sensors including resistive, pyroelectric, and thermoelectric sensors, which can be utilized to improve the realism of augmentation. The fundamental mechanism, design strategies, and the rational guidelines for the adoption of these technologies in AR/VR is explicitly discussed. The conclusion provides an outlook on the existing challenges and outlines the future roadmap for the realization of next‐generation thermo‐haptics enabled augmented reality.
... Murakami et al. developed a fingertip haptic display with integrated force, tactile and thermal feedback in a miniature form factor [25]. The apparatus can render vertical forces by pulling the belt, shearing forces by sliding the belt, textures by converting the textural audio signals into dc motor pulsewidth modulation commands and thermal sensations with the peltier module [26]. ...
Human haptic perception system is complex, involving both cutaneous and kinesthetic receptors. These receptors work together and enable human to perceive the external world. To simulate immersive interaction with virtual objects in virtual reality scenarios haptic devices are desired to reproduce multiproperties of virtual objects, support multigestures of human hands to perform fine manipulation, produce haptic stimuli for simultaneously stimulating multireceptors (including cutaneous and kinesthetic receptors) of human haptic channel, and thus invoke realistic compound haptic sensations. In recent years, such multimodal haptic devices have emerged. In this paper, we survey the latest progress on multimodal haptic devices, identify the gaps, and put forward future research directions on the topic.
... Thermal feedback could also contribute to the immersive user experience in VR/AR, to simulate virtual temperatures [29,35], season experience [30], wetness [24], and work as directional cues in VR [26]. Takaki et al. [22] presented a wearable haptic display that generates force and thermal feedback for augmented-reality applications. Han et al. developed HydroRing [11], a finger-worn device which could provide concurrent force, vibration and temperature feedback through a liquid medium. ...
... Thermal feedback could also contribute to the immersive user experience in VR/AR, to simulate virtual temperatures [29,35], season experience [30], wetness [24], and work as directional cues in VR [26]. Takaki et al. [22] presented a wearable haptic display that generates force and thermal feedback for augmented-reality applications. Han et al. developed HydroRing [11], a finger-worn device which could provide concurrent force, vibration and temperature feedback through a liquid medium. ...
... The wearable haptic device reduces the volume and mechanical complexity by moving the body-grounded base as close as possible to the point of application of the stimulus, and thus has the significant advantage of being small, easy to carry, comfortable and does not impair the wearer's motion [10]. Currently, various types of finger wearable devices have been developed to interact with virtual environments displayed in a computer, and provide haptic stimuli such as contact, normal force, lateral force, vibration, temperature, etc. for interaction [11], [12], [13]. However, with the exception of the literature [14], few studies have considered applying wearable devices to touch screen interactions. ...
The haptic interface plays an increasingly important role in enhancing the realism and immersion of the user's interaction with the touch screen. Inspired by the wearable haptic system, this paper proposes a finger wearable device called FW-Touch for touch screen interaction. The device provides normal force, lateral force and vibrotactile feedback for the interaction of the finger and the touch screen through three internally integrated actuators. By displaying the hardness, friction and roughness of a virtual surface, the device is capable of simulating the active exploration and sensing process of the finger on a real surface. This paper describes the design and specifications of the FW-Touch, and details the design process of a magnetorheological (MR) foam actuator that uses a Hall sensor to correct the output force. Through physical measurements and psychophysical experiments, we comprehensively evaluated the force feedback performance of the FW-Touch and its ability in displaying the stiffness and friction of the virtual surface. The results show that improving the accuracy of force feedback is necessary for virtual stiffness display, and the accuracy and effectiveness of the FW-Touch in displaying virtual surface features can be confirmed from the measured stiffness and friction Weber fractions.
... There is an increasing interest in developing tactile displays for Virtual Reality and Augmented Reality environments [10]. In [7] a wearable interface providing force and thermal feedback to subject's finger has been realized. The device uses PWM-driven DC motors to generate haptic stimuli and a Peltier module placed in contact with the fingernail. ...
Towards a more realistic feeling of interacting with virtual and remote objects, we developed a wearable cutaneous device for the proximal finger phalanx able to provide skin deformations and thermal cues on the user skin. A servomotor is used to move a belt applying a pressure on the user finger, while a Peltier cell renders thermal cues. In the proposed hands-on demonstration, a user wears such haptic ring and a VR headset, and interacts with a virtual environment. In the virtual scenario, objects with different temperatures are displayed and the user is asked to find the coldest or the hottest. During the interaction, the movements of the user hand are tracked by a Leap Motion sensor, while the haptic ring renders interaction forces and thermal cues, providing the user with the sensation of touching objects in the scene at different temperatures.
... Konyo et al. [18] utilize simple vibratory stimulations worn on the user finger to create the multiple tactile sensations such as roughness, pressure, and friction sensations. Altered Touch [19] utilizes peltier, motor and ribbon belt to alter the softness/hardness and thermal sensation. Wearable device also can enhance the VR experience. ...
In this paper, we present Haptic Around, a hybrid-haptic feedback system, which utilizes fan, hot air blower, mist creator and heat light to recreate multiple tactile sensations in virtual reality for enhancing the immersive environment and interaction. This system consists of a steerable haptic device rigged on the top of the user head and a handheld device also with haptics feedbacks to simultaneously provide tactile sensations to the users in a 2m x 2m space. The steerable haptic device can enhance the immersive environment for providing full body experience, such as heat in the desert or cold in the snow mountain. Additionally, the handheld device can enhance the immersive interaction for providing partial body experience, such as heating the iron or quenching the hot iron. With our system, the users can perceive visual, auditory and haptic when they are moving around in virtual space and interacting with virtual object. In our study, the result has shown the potential of the hybrid-haptic feedback system, which the participants rated the enjoyment, realism, quality, immersion higher than the other.
... Pyo et al. developed a tactile display using vibration and frictional force with two electro- static actuators [31]. Murakami et al. integrated a belt-type tactile display for pressure, shear force, and vibration using a Peltier element to create hybrid tactile stimulation [25]. Most of the previous hybrid tactile diplays aimed to stimulate only one fingertip owing to the bulk of electronic parts or circuits such as actuators and motors. ...
Humans can perceive tactile sensation through multimodal stimuli. To demonstrate realistic pseudo tactile sensation for the users, a tactile display is needed that can provide multiple tactile stimuli. In this paper, we have explicated a novel printed tactile display that can provide both the electrical stimulus and the electrostatic force. The circuit patterns for each stimulus were fabricated by employing the technique of double-sided conductive ink printing. Requirements for the fabrication process were analyzed and the durability of the tactile display was evaluated. Users' perceptions of a single tactile stimulus and multiple tactile stimuli were also investigated. The obtained experimental results indicate that the proposed tactile display is capable of exhibiting realistic tactile sensation and can be incorporated by various applications such as tactile sensation printing of pictorial illustrations and paintings. Furthermore, the proposed hybrid tactile display can contribute to accelerated prototyping and development of new tactile devices.
Cutaneous haptic feedback provides a sense of touch by displaying sensations — such as vibration, skin stretch, or normal force — directly to the skin. While the majority of these devices have been designed to display one main haptic sensation, recent work has begun to explore the creation of multi-modal haptic devices, capable of rendering a variety of cutaneous cues. In this work, we investigate using the tip and body of a continuum robot to directly render multiple cutaneous cues to the fingerpad. We present the design of a device that consists of two Haptic Continuum Robots (HCRs) that is capable of rendering four distinct haptic cues— skin stretch, skin slip, normal indentation, and vibration. We present and validate a model of the proposed HCR and characterize the device performance. Finally, we conduct a preliminary haptic sensation identification study, which showed that users were able to correctly identify the displayed haptic sensation with 90% accuracy.
Research and development of telerobotic systems supplemented by haptic feedback for future planetary exploration missions has gained significant importance in the past decade. Major space agencies endeavor to deploy such systems before sending humans to the surface of unknown or unexplored celestial bodies. Astronauts control these telerobotic systems from remote locations, such as an orbital space station. Haptic feedback for teleoperating the robots in outer space is extremely important, not only to improve user immersion and task performance, but also to improve our understanding of surface properties. At the same time, for spaceflight, making use of compact, light-weight and robust devices are preferred for precise tactile feedback from telemanipulation tasks. In this paper, we introduce “ViESTac”, a first attempt to develop a generic VR suite to be able to evaluate and compare fingertip-wearable tactile devices. Applications of such a suite include, but are not limited to allowing teleoperators to judiciously choose suitable tactile devices for a particular task. To account for the wide variety of existing fingertip-wearable tactile devices and their display capabilities, the suite contains a set of virtual scenarios to investigate different tactile properties of virtual objects. It also dedicates a virtual scenario to evaluate how tactile feedback may govern the accuracy of human positioning in standard tasks. This proposed suite is advocated by a pilot study with 13 participants and two distinct state-of-the-art tactile devices. Results of the study clearly indicate that the virtual suite can successfully cater to the need of evaluating and comparing fingertip-wearable tactile devices.
There are fundamental differences between the tactile and thermal sensory systems that must be accommodated when designing multisensory cutaneous displays for use in virtual or teleoperated robotic environments. In this review we highlight the marked temporal and spatial differences between the senses of cold and warmth as revealed in psychophysical experiments. Cold and warmth are distinct senses with marked differences in the time taken to respond to stimulation and in their temporal filtering processes. Such variations must be taken into account when time-varying profiles of thermal stimulation are delivered to the skin concurrent with tactile stimulation since the resulting sensations will not be perceived on the same time scale. Although it is often reported that the thermal senses are markedly inferior to the sense of touch with respect to their spatial acuity, it is also clear that there is considerable variability across the body in the accuracy with which thermal stimuli can be localized. The distal to proximal gradient in thermal acuity suggests that locations other than the palmar surface of the hand are better suited for displaying thermal cues, in contrast to the situation for tactile inputs. As was noted for temporal processes, there are differences between localizing warmth and cold stimuli, with localization being superior for cold. These properties provide benchmarks that can be used in designing thermal and multisensory displays.KeywordsCutaneous sensingMultisensoryThermal displays
In recent years, the multi-dimensional, multi-faceted, and multi-level applications of augmented reality/mixed reality have attracted more and more attention from researchers in the field of smart manufacturing. However, the disadvantage is that AR interaction cues supporting user cognition have not been fully analyzed and summarized in manual assembly, training and repair. This article reviews the current research status, projects and technical characteristics of manual operation instructions in the past 30 years, and extensively discusses these latest works. It is worth mentioning that this article provides comprehensive academic information for the development of AR-based assembly instructions, thereby providing unique insights for researchers in related fields. In short, it will help researchers designing AR instructions.
Touch between people is an integral part of human life. Touch is used to convey information, emotions, and other social cues. Still, everyday remote communication remains mainly auditive or audio-visual. The theme of this article, interpersonal haptic communication, refers to any communication system that supports mediation of touch between two or more persons. We first present a scoping review of the state of the art in interpersonal haptic communication, including physiological and psychological basis of touch, affective and social touch, and mediated social touch. We then discuss emerging research themes that shape the future of interpersonal haptic communication, identify research gaps and propose key research directions for each theme. Finally, societal impact and ethical aspects are discussed.
Most augmented reality (AR) interactions are facilitated by controllers and hand gestures. These tools simplify or replace actions with other actions, making an AR experience feel different from the actual experience. Although research on pseudo feedback has been conducted using sound and vibration, we believe that force-haptic feedback from manipulation is important in AR interactions. In other words, adding force-like tactile sensations to AR interactions has the potential to enhance the user’s given sense of reality. This paper examines our stamp-type device, which can present force tactile sensations, and the results of experiments on subjects’ interactions with the input method. In Experiment 1, we evaluated the usability of three input methods for AR interactions: two popular methods (“touch” and “pointer”) and the proposed method. In Experiment 2, we conducted the same experiment as in Experiment 1 and analyzed the effect of the presence or absence of a pressing sensation on the user.
A complete set of diverse haptic feedbacks is essential for a highly realistic and immersive virtual environment. In this sense, a multi-mode haptic interface that simultaneously generates multiple kinds of haptic signals is highly desirable. In this paper, we propose a new silicone-made pneumatically controlled fingertip actuator to render pressure and vibrotactile feedback concurrently to offer a realistic and effective haptic sensation. A new silicone-based stacked dual-layer air chamber was designed. The volume of the chambers is controlled by pneumatic valves with compressed air tanks. The top/upper air chamber renders vibration feedback, whereas the bottom/lower air chamber renders pressure feedback. The proposed silicone-made fingertip actuator is designed so that it can be easily worn at the fingertips. To demonstrate the potential of the system, a virtual environment for rendering three different types of haptic textures was implemented. Extensive performance evaluation and user studies were carried out to demonstrate the proposed actuator’s effectiveness compared to an actuator with single vibrotactile feedback.
In recent years, there has been a great deal of research on teleoperation of robots, and many end-effector-oriented control systems have been proposed, but these systems have difficulties in performing manipulation tasks with physical contacts between the target object, the robot, and the environment, such as dual-arm manipulation, because these systems can only express the end-effectors’ pose and force commands. In this study, we propose Target-object-oriented User Interface (TOOUI) that focuses on the target object rather than the end effectors of the robot, and develop a tangible device called Miniature Tangible Cube as a TOOUI. Through this device, the pilot can express the target orientation and contact state of the target object, and the robot can be teleoperated to do dual-arm manipulation tasks with geometric and physical constraints. For dual-arm manipulation tasks, we also implement a dual-arm telemanipulation system with the Miniature Tangible Cube using the motion planner based on state machine. In the experiments, we evaluated our system with the Miniature Tangible Cube by conducting dual-arm manipulation tasks with the dual-arm robot PR2 in the real world.
Community detection inside networks represents an area which has gained a paramount importance in the last decade. With the massive spreading of online communications several attempts have been performed to investigate correlations between the “Real world” and social networks, both in terms number of connections between people and consistency of relationship. This research delves into the relations occurred on Facebook and Twitter which can be used as a proxy for the relations in offline social networks, such as networks of human face-to-face (F2F) proximity. The final aim is focused on discovering the most meaningful patterns that rule the hierarchy of bonds by means of the comparison of several algorithms widely recognized as effective for different applications, such a routing, link prediction, crimes and recently containment of virus outbreak.
Studying the complex network of needs of a human being in space, it emerges that one of the problems facing astronauts is the lack of tactile sensitivity and therefore of connection with the external world, through the most extensive of our senses. This can cause emotional imbalance and depersonalization. In addition to this there is the need to protect the body with a very high protection index, constantly monitor the health of the subject and be able to intervene in real time. This study proposes potential options to overcome this phenomenon through interactive devices. The proposed device is a system that tries to solve all these problems simultaneously, using smart materials and innovative technology.
This paper presents a tactile device for displaying surface texture of flexible virtual objects. The texture is simulated using electrovibration effect produced by a flexible film. The deformation of the film is controlled by a DC motor to generate desired softness according to applied finger force. A prototype of the device is developed and integrated with a virtual environment. We show the application of the display using an online shopping scenario.
Humans have specific sensory organs and they can feel tactile sensation on the whole body. However, many haptic devices have limitations due to the location of the body part and might not provide natural haptic feedback. Thus, we propose a novel interface, TherModule, which is a wearable and modular thermal feedback system for embodied interactions based on a wireless platform. TherModule can be worn on multiple body parts such as the wrist, forearm, ankle, and neck. In this paper, we describe the system concept, module implementation, and applications. To demonstrate and explore the embodied interaction with thermal feedback, we implemented prototype applications, such as movie experiences, projector-based augmented reality, navigation, and notification based on a wireless platform, with TherModule on multiple parts of the body. The result of an experiment on movie experience showed that participants felt more interactions between temperature and visual stimulus.
Current haptic devices are often bulky and rigid, making them unsuitable for ubiquitous interaction and scenarios where the user must also interact with the real world. To address this gap, we propose HydroRing, an unobtrusive, finger-worn device that can provide the tactile sensations of pressure, vibration, and temperature on the fingertip, enabling mixed-reality haptic interactions. Different from previous explorations, HydroRing in active mode delivers sensations using liquid travelling through a thin, flexible latex tube worn across the fingerpad, and has minimal impact on a user's dexterity and their perception of stimuli in passive mode. Two studies evaluated participants' ability to perceive and recognize sensations generated by the device, as well as their ability to perceive physical stimuli while wearing the device. We conclude by exploring several applications leveraging this mixed-reality haptics approach.
Telexistence is fundamentally a concept named for the general technology that enables a human being to have a real-time sensation of being at a place other than where he actually exist, and to interact with the remote environment [Tachi et al. 1990]. We have achieved human-like neck movements to visually interact with a remote object in dimensional space through previous versions of TELESAR [Watanabe et al. 2007]. We introduce "TELESAR V" which maps a user's spinal neck, head and arm movements into a dexterous slave robot and allowing the operator to feel the robot's body as his own body through visual, auditory, kinesthetic and fingertip tactile [Sato et al. 2007] sensation. With TELESAR V, operator can perform teleoperations confidently with no prior practise.
Novel wearable tactile interfaces offer the possibility to simulate tactile interactions with virtual environments directly on our skin. But, unlike kinesthetic interfaces, for which haptic rendering is a well explored problem, they pose new questions about the formulation of the rendering problem. In this work, we propose a formulation of tactile rendering as an optimization problem, which is general for a large family of tactile interfaces. Based on an accurate simulation of contact between a finger model and the virtual environment, we pose tactile rendering as the optimization of the device configuration, such that the contact surface between the device and the actual finger matches as close as possible the contact surface in the virtual environment. We describe the optimization formulation in general terms, and we also demonstrate its implementation on a thimble-like wearable device. We validate the tactile rendering formulation by analyzing its force error, and we show that it outperforms other approaches.
We propose a wearable haptic display to present the weight sensation of a virtual object, which is based on our novel insight that the deformation on fingerpads makes a reliable weight sensation even when the proprioceptive sensation is absent. This device will provide a new form of ubiquitous haptic interaction.
- Takuji Narumi
- Takashi Kajinami
- Tomohiro Tanikawa
- Michitaka Hirose
- Takuji Narumi
- Takashi Kajinami
- Tomohiro Tanikawa
- Michitaka Hirose
- Acm Siggraph
Vibrotactile and pseudo force presentation using motor rotational acceleration
- Ryuta Vibol Yem
- Hiroyuki Okazaki
- Kajimoto