Touchable Video and Tactile Audio
DOI: 10.1109/ISM.2009.79 Conference: ISM 2009, 11th IEEE International Symposium on Multimedia, San Diego, California, USA, December 14-16, 2009
We propose a Haptic Audio Visual System (HAVS) which consists of a touchable video and a tactile audio. Touchable video generates tactile feedback according to the user's real-time interactions on the video. Tactile audio provides the tactile experiences through the automatic generation of vibro-tactile stimuli using a sound detection algorithm. The video screen is divided into a grid, and each cell has customizable tactile information which corresponds to the visual content. At the same time, the sound stream is also analyzed and customizable vibration responses are synchronized to specific sound effects or audio "signatures". One of the important advantages of HAVS is the ability to augment and enhance immersiveness and the interactivity of users in 3D virtual environments. The application of HAVS to commercial, military, educational and medical contexts including: home shopping, online gaming, interactive broadcasting, teaching/learning and teleoperator settings will be discussed.
- [Show abstract] [Hide abstract]
ABSTRACT: We propose a haptic rendering technology for touchable video. Our touchable video technique allows users for feeling the sense of touch while probing directly on 2D objects in video scenes or manipulating 3D objects brought out from video scenes using haptic devices. In our technique, a server sends video and haptic data as well as the information of 3D model objects. The clients receive video and haptic data from the server and render 3D models. A video scene is divided into small grids, and each cell has its tactile information which corresponds to a specific combination of four attributes: stiffness, damping, static friction, and dynamic friction. Users can feel the sense of touch when they touch directly cells of a scene using a haptic device. Users can also examine objects by touching or manipulating them after bringing out the corresponding 3D objects from the screen. Our touchable video technique proposed in this paper can lead us to feel maximum satisfaction the haptic-audio-vidual effects directly on the video scenes of movies or home-shopping video contents.
- [Show abstract] [Hide abstract]
ABSTRACT: Haptic interaction is commonly used with 3D objects defined by their geometric and solid models. Extension of the haptic interaction to 3D Cyber worlds is a challenging task due to the Internet bandwidth constraints and often prohibitive sizes of the models. We study how to replace visual and haptic rendering of shared 3D objects with 2D image visualization and 3D haptic rendering of the forces reconstructed from the images or augmenting them, which will eventually simulate realistic haptic interaction with 3D objects. This approach allows us to redistribute the computing power so that it can concentrate mainly on the tasks of haptic interaction and rendering. We propose how to implement such interaction with small function descriptions of the haptic information augmenting images and video. We illustrate the proposed ideas with the function-based extension of VRML and X3D.
Data provided are for informational purposes only. Although carefully collected, accuracy cannot be guaranteed. The impact factor represents a rough estimation of the journal's impact factor and does not reflect the actual current impact factor. Publisher conditions are provided by RoMEO. Differing provisions from the publisher's actual policy or licence agreement may be applicable.