Conference PaperPDF Available

Stretchable transducers for kinesthetic interactions in virtual reality

Authors:

Abstract

Commercial virtual reality devices such as the Oculus Rift and HTC Vive enable experience designers to emulate a number of human sensory inputs with computer simulations. Commercial experiences have demonstrated plausibly realistic audiovisual sensory input, but somatosensory feedback has been more limited in scope. Most successful attempts in providing feedback to the human Kinesthetic system were considered power-demanding, expensive and potentially harmful to users, therefore somatosensory input has largely been addressed by vibration-based devices, like Linear Resonant Actuators and Eccentric Rotating Mass actuators, aimed at stimulating receptors near the surface of the skin. This method is widely accepted as a proxy in lieu of resisting to muscle tension, but it is not considered a path to realistic input.
Stretchable Transducers for Kinesthetic Interactions in Virtual
Reality
Extended Abstract
Robert Shepherd
Cornell University
Bryan Peele
Cornell University
Benjamin Mac Murray
Cornell University
Jose Barreiros
Cornell University
Omer Shapira
NVIDIA Corporation
Josef Spjut
NVIDIA Corporation
David Luebke
NVIDIA Corporation
Figure 1: The tools of soft robotics enable immersive kinesthetic experiences in augmented and virtual reality while remaining
safe to use. Left: Using uidic elastomer actuators (FEAs), we demonstrate a soft skin that can provide force feedback to a
motion tracked controller integrated with the VR Funhouse application. Middle and Right: A soft controller uses the variable
compliance of soft structures to simulate dierent textures and materials.
CCS CONCEPTS
Human-centered computing Haptic devices
;
Virtual re-
ality;
KEYWORDS
Human Computer Interaction, Haptics, Virtual Reality
ACM Reference format:
Robert Shepherd, Bryan Peele, Benjamin Mac Murray, Jose Barreiros, Omer
Shapira, Josef Spjut, and David Luebke. 2017. Stretchable Transducers for
Kinesthetic Interactions in Virtual Reality. In Proceedings of SIGGRAPH
’17 Emerging Technologies, Los Angeles, CA, USA, July 30 - August 03, 2017,
2 pages.
https://doi.org/10.1145/3084822.3091103
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
SIGGRAPH ’17 Emerging Technologies, July 30 - August 03, 2017, Los Angeles, CA, USA
©2017 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-5012-9/17/07.
https://doi.org/10.1145/3084822.3091103
OVERVIEW
Commercial virtual reality devices such as the Oculus Rift and HTC
Vive enable experience designers to emulate a number of human
sensory inputs with computer simulations. Commercial experiences
have demonstrated plausibly realistic audiovisual sensory input,
but somatosensory feedback has been more limited in scope. Most
successful attempts in providing feedback to the human Kinesthetic
system were considered power-demanding, expensive and poten-
tially harmful to users, therefore somatosensory input has largely
been addressed by vibration-based devices, like Linear Resonant
Actuators and Eccentric Rotating Mass actuators, aimed at stimu-
lating receptors near the surface of the skin. This method is widely
accepted as a proxy in lieu of resisting to muscle tension, but it is
not considered a path to realistic input.
Stemming from advances in materials science, the eld of soft
robotics constructs stretchable actuators, sensors and displays [Li
et al
.
2016; Mac Murray et al
.
2015; Zhao et al
.
2016] using structures
and materials that can be deformed at forces exerted by human
muscles. To date, most demonstrations of soft robotics have focused
on mobile robots, compliant grippers, and biomedical applications.
Here we present the use of these technologies for kinesthetic feed-
back in virtual reality. The materials used to build these devices
SIGGRAPH ’17 Emerging Technologies, July 30 - August 03, 2017, Los Angeles, CA, USA R. Shepherd et al.
(e.g silicon) have mechanical properties similar to that of human
tissue and are well-suited for applications that require direct and
prolonged contact with human skin. Additionally, these systems
are well-suited for commercial applications due to the low material
cost and compatibility with scalable manufacturing techniques.
We provide two key demonstrations to highlight the use of uidic
elastomer actuators to provide haptic feedback. These demos allow
users to progress through a series of brief experiences where the
hand-held controller adjusts its form and behavior to match that of
the virtual object used in each demo. The objects held in the demo
include a goo gun, pistol and mallet.
PNEUMATIC SKIN
First, we have developed a thin ( 3mm) pneumatic skin to cover
an HTC Vive controller. This pneumatic skin has 12 individually
inatable chambers placed in contact points similar to those of
objects used in the game play of NVIDIA’s VR Funhouse [NVIDIA
2016]. By sending impulses to the user based on actions in the
game, the ow of gas into the chambers enhases the gameplay of
VR Funhouse. For instance, pneumatic chambers along the bottom
of the controller inate to simulate the recoil of a revolver or the
chambers undulate to simulate uid moving through a water gun.
FOAM CONTROLLER
Our second demonstration is a rubber foam controller that changes
stiness and texture to represent a variety of objects from NVIDIA’s
VR Funhouse. The controller is composed of co-mingled blocks of
foam that we selectively pressurize to provide force feedback upon
gripping by the user. The combinations of stiening dierent foam
modules can represent a large number of Funhouse objects. For
instance, the absence of pressurized blocks can simulate the palm
area of a boxing glove, while a highly pressurized controller (all
blocks pressurized) can mimic the sti wooden handle of a mallet.
REFERENCES
Shuo Li, Bryan N Peele, Chris M Larson, Huichan Zhao, and Robert F Shepherd. 2016.
A Stretchable Multicolor Display and Touch Interface Using Photopatterning and
Transfer Printing. Advanced Materials 28, 44 (2016), 9770–9775.
Benjamin C Mac Murray, Xintong An, Sanlin S Robinson, Ilse M van Meerbeek, Kevin W
O’Brien, Huichan Zhao, and Robert F Shepherd. 2015. Poroelastic foams for simple
fabrication of complex soft robots. Advanced Materials 27, 41 (2015), 6334–6340.
NVIDIA. 2016. NVIDIA VR Funhouse. http://store.steampowered.com/app/468700/.
(2016). Accessed: 2017-03-20.
Huichan Zhao, Kevin O’Brien, Shuo Li, and Robert F Shepherd. 2016. Optoelectronically
innervated soft prosthetic hand via stretchable optical waveguides. Science Robotics
(2016).
... We can realize it by a variety of contact surfaces or motion sensors, combining real intangible cultural heritage with digital virtual intangible cultural heritage. In the process of constructing the 3D entity, users can use the position of fingers to calculate the digital information of the object such as the color and curvature of the virtual incomplete [14]. For this kind of interaction, it can be a natural human-computer one, because the user uses it just like using tools naturally in reality. ...
Article
Full-text available
The current predicament of intangible cultural heritage is its vulnerability and rarity, and an expression that is difficult to construct in the space-time dimension. It is difficult for visitors to really understand and touch these local intangible cultural heritages. For this reason, the need to develop new methodologies for cultural heritage learning and art appreciation is always present, and this is increasingly supported by digital technologies. The research in this paper is to apply the self-issuance method to the folk intangible cultural heritage center. It is different from the traditional AR method of presenting crafts directly through HMD. Instead, the use of low-cost technology, Augmented Reality technology and unity 3D technology provide a new way of interaction. By using HMD, smartphone, Leap motion and software created in UNITY 3D, the application of the augmented reality program is achieved. It allows users to interact with the intangible cultural heritage easily and fully it in a virtual physical environment. Based on the principle of human natural interaction, it can be achieved by structured light target image scanning and the design and experiment of photographic measurement. Then a local intangible cultural heritage model database can be established to provide virtual prototype files. The results of the experimental show that the data transmission is effective and the three-dimensional interaction between people and exhibits can be realized. The research results can be applied to the interactive mode practice of the intangible cultural heritage center, and provide an effective interactive mode for the intangible cultural heritage in the form of augmented reality interaction. In this way, visitors can have a deep impression on the intangible cultural heritage and the culture can be spreaded widely all over the world.
Article
Full-text available
Because of their continuous and natural motion, fluidically powered soft actuators have shown potential in a range of robotic applications, including prosthetics and orthotics. Despite these advantages, robots using these actuators require stretchable sensors that can be embedded in their bodies for sophisticated functions. Presently, stretchable sensors usually rely on the electrical properties of materials and composites for measuring a signal; many of these sensors suffer from hysteresis, fabrication complexity, chemical safety and environmental instability, and material incompatibility with soft actuators. Many of these issues are solved if the optical properties of materials are used for signal transduction. We report the use of stretchable optical waveguides for strain sensing in a prosthetic hand. These optoelectronic strain sensors are easy to fabricate, are chemically inert, and demonstrate low hysteresis and high precision in their output signals. As a demonstration of their potential, the photonic strain sensors were used as curvature, elongation, and force sensors integrated into a fiber-reinforced soft prosthetic hand. The optoelectronically innervated prosthetic hand was used to conduct various active sensation experiments inspired by the capabilities of a real hand. Our final demonstration used the prosthesis to feel the shape and softness of three tomatoes and select the ripe one.
Article
Full-text available
An intrinsically soft and stretchable multicolor display and touch interface is reported. Red, green, and blue pixels are formed separately by photopatterning transition-metal-doped ZnS embedded in silicone gels and transfer printing onto an elastomeric dielectric sheet. The device shows stable illumination while being stretched up to 200% area strain or under different deformation modalities. It also introduces capabilities for dynamic colorations and multipoint capacitive touch sensing.
Article
Open-celled, elastomeric foams allow the simple design of fully 3D pneumatic soft machines using common forming techniques. This is demonstrated through fabrication of simple actuators and an entirely soft, functional fluid pump formed in the shape of the human heart. The device pumps at physiologically relevant frequencies and pressures and attains a flow rate higher than all previously reported soft pumps.
Poroelastic foams for simple fabrication of complex soft robots
  • Benjamin C Mac Murray
  • Xintong An
  • S Sanlin
  • Ilse M Robinson
  • Kevin W Van Meerbeek
  • Huichan O'brien
  • Robert F Zhao
  • Shepherd
Benjamin C Mac Murray, Xintong An, Sanlin S Robinson, Ilse M van Meerbeek, Kevin W O'Brien, Huichan Zhao, and Robert F Shepherd. 2015. Poroelastic foams for simple fabrication of complex soft robots. Advanced Materials 27, 41 (2015), 6334-6340.