Jun Rekimoto

• Ph.D
• The University of Tokyo

Juiciness is visual pizzazz used to improve player experience and engagement in games. Most research has focused on juicy particle effects. However, text effects are also commonly used in games, albeit not always juiced up. One type is onomatopoeia, a well-defined element of human language that has been translated to visual media, such as comic boo...

Whispering is a common privacy-preserving technique in voice-based interactions, but its effectiveness is limited in noisy environments. In conventional hardware- and software-based noise reduction approaches, isolating whispered speech from ambient noise and other speech sounds remains a challenge. We thus propose WhisperMask, a mask-type micropho...

Figure 1: This research leverages tunable (liquid) lenses to optically simulate two of presbyopia's primary visual representations: increased near-point and blurred vision within the near-point range. To achieve this, we modulate the diopter of the wearer by dynamically adjusting two tunable lenses of our eyewear. ABSTRACT Presbyopia is currently a...

This study explores the value of using virtual humans (VHs) to mask real entities within augmented reality (AR) interactive environments. We believe that one ultimate solution to bridge the gap between virtual and real worlds is to enable virtual entities to interact with the real world seamlessly and physically. However, unlike major approaches su...

The availability of digital devices operated by voice is expanding rapidly. However, the applications of voice interfaces are still restricted. For example, speaking in public places becomes an annoyance to the surrounding people, and secret information should not be uttered. Environmental noise may reduce the accuracy of speech recognition. To add...

Recognizing whispered speech and converting it to normal speech creates many possibilities for speech interaction. Because the sound pressure of whispered speech is significantly lower than that of normal speech, it can be used as a semi-silent speech interaction in public places without being audible to others. Converting whispers to normal speech...

Silent speech interface is a promising technology that enables private communications in natural language. However, previous approaches only support a small and inflexible vocabulary, which leads to limited expressiveness. We leverage contrastive learning to learn efficient lipreading representations, enabling few-shot command customization with mi...

Interactions based on automatic speech recognition (ASR) have become widely used, with speech input being increasingly utilized to create documents. However, as there is no easy way to distinguish between commands being issued and text required to be input in speech, misrecognitions are difficult to identify and correct, meaning that documents need...

視線入力は手や音声による操作を必要としないといった利点で特徴付けられており，それに基づいた様々なインターフェースが提案されている．しかし，視線計測器の性能制限や固視微動現象の影響などにより，視線入力の精度はまだ不十分である．一方で，音声入力は同様にハンズフリーな操作方式であるが，公共の場では情報漏洩や周囲に迷惑をかける他，音声認識の精度に依存するなどの制限が挙げられる．そこで本研究は，視線選択・無声発話操作を併用した手や音声によるそうさを必要としない入力方式Gaze+Lipを提案する．従来の視線のみの入力方式との比較実験を通して，以下の三つの利点があることを検証した：第一に，視線入力の曖昧さを解消し，誤操作が生じにくい入力を可能にした；第二に，視線選択した上で無声発話で操作することによって，...

Galvanic taste stimulation (GTS) is a non-invasive electrical stimulation of sensory nerves that induces, inhibits, and enhances taste sensation. It has been shown that the cathodal GTS taste enhancement effect occurs when only cathodal electrodes are attached in or near the mouth, while anodal GTS, whose anodal electrodes are attached in or near t...

Gastroenterological endoscopic surgery needs complex surgical skills such as a sensation of body movement and manipulation of the endoscope that is hard to be explained verbally. Prior research reported that endoscopic surgery is one of the most challenging surgery to teach. Thus, surgeons need long-term practice to master the skills. To support le...

Conversational agents are widely used in many situations, especially for speech tutoring. However, their contents and functions are often pre-defined and not customizable for people without technical backgrounds, thus significantly limiting their flexibility and usability. Besides, conventional agents often cannot provide feedback in the middle of...

This paper illustrates design work carried out to develop an interactive theater performance. HCI has started to address the challenges of designing interactive performances, as both audience and performers' experiences are considered and a variety of professional expertise involved. Nevertheless, research has overlooked how such design unfolds in...

Providing an experience that includes high-speed objects, such as tennis balls, with a virtual reality (VR) training environment might provide efficient training for trainers but is challenging to achieve. Because of the drawing performance of the display, high-speed objects are perceived as poor visual information more than in reality, such as ima...

We present Extramission, a method to a large scale interactive virtual environment. It consists of dual head mounted pico projectors and retro-reflective materials. With high-accuracy retro-reflective materials, laser beams scanned on user’s retina makes clear and free-focus vision. In this retinal scanning configuration, even if the luminance of t...

Digital musical instruments are essential technologies in modern musical composition and performance. However, the interface of the synthesizer is not intuitive enough and require extra knowledge because of the parameters. To address this problem, we propose pseudo-intention learning: a novel data collection method for supervised learning in musica...

We propose a potential fluid-measurement technology aimed at supporting biomechanics research of water sports using fluid simulation and motion analysis. Cellulose nanofibers introduced into the water as tracer particles to visualize the movement of water. An optical property of nanofibers, called flow birefringence, makes water flows brighter than...

Immersion is an important factor in video experiences. Therefore, various methods and video viewing systems have been proposed. Head-mounted displays (HMDs) are home-friendly pervasive devices, which can provide an immersive video experience owing to their wide field-of-view (FoV) and separation of users from the outside environment. They are often...

Understanding water flow around a swimmer is key to reducing the water resistance when swimming and improving propulsive force, since swimmers who produce less turbulent flows around their bodies can, for example, swim faster. However, existing water flow measurement technologies are not suitable for measuring human swimmers because they can only m...

" We present Bubble, a pneumatically actuated wearable device that enables people with hand disabilities to use their own hands to grasp objects without fully bending their fingers. Bubble offers a novel approach to grasping, where slim, ultra-lightweight silicone actuators are attached to the fingers. When the user wishes to grasp an object, the s...

The availability of digital devices operated by voice is expanding rapidly. However, the applications of voice interfaces are still restricted. For example, speaking in public places becomes an annoyance to the surrounding people, and secret information should not be uttered. Environmental noise may reduce the accuracy of speech recognition. To add...

Resistive force (e.g., due to object elasticity) and impact (e.g., due to recoil) are common effects in our daily life. However, resistive force continuously changes due to users' movements while impact instantly occurs when an event triggers it. These feedback are still not realistically provided by current VR haptic methods. In this paper, a wear...

Bouldering is an urban form of rock climbing that requires precise and complex movement. Similarly to other sports, the simplest way to learn bouldering skill is to mimic professional's motion. However, ordinary beginner boulders cannot learn to coaches, so that they learn by themselves or tutorial videos. Even if they managed, bouldering has a com...

The development of personal fabrication technologies has enabled end users to model and prototype desired objects. 3D printing technologies have eased our access to solid models, however, it is still a challenge to develop thin fibers rapidly at personal levels that may help enriching textures of models. We propose a system and method inspired by c...

Understanding the correct self-posture is known to improve the performance of motor skills in sports, dance, ballet, walking, and running. However, it is difficult to understand the accurate self-posture and move one's body as intended for imitation or learning, especially in-situation. Based on previous physiological research, we herein propose th...

Water flow is strongly related to swimming speed; thus, technologies for measuring the three-dimensional movement of water are highly desired in the water sports industry. However, existing fluid measurement methods are not suitable for use with humans because they introduce tiny plastic particles (known as tracer particles) that contain fluorescen...

Owing to the improvement in accuracy of eye tracking devices, eye gaze movements occurring while conducting tasks are now a part of physical activities that can be monitored just like other life-logging data. Analyzing eye gaze movement data to predict reading comprehension has been widely explored and researchers have proven the potential of utili...

Contributions of recent deep-neural-network (DNN) based techniques have been playing a significant role in human-computer interaction (HCI) and user interface (UI) domains. One of the commonly used DNNs is human pose estimation. This kind of technique is widely used for motion capturing of humans, and to generate or modify virtual avatars. However,...

There has been a long history in electrical muscle stimulation (EMS), which has been used for medical and interaction purposes. Human-computer interaction (HCI) researchers are now working on various applications, including virtual reality (VR), notification, and learning. For the electric signals applied to the human body, various types of wavefor...

This paper introduces a telepresence system called OmniGaze, which displays a remote user's presence using a spherical display on an omnidirectional camera. In a telepresence system, gaze has a variety of important functions such as presenting the direction of attention and making simple eye gestures. An omnidirectional camera is an effective tool...

Researchers have attempted to increase the realism of virtual reality (VR) applications in many ways. Combinations of the visual, auditory and haptic feedback have successfully simulated experiences in VR, however, multimedia contents may also stimulate emotions. In this paper, we especially paid attention to negative emotions that may be perceived...

Immersion is an important factor in video experiences. Therefore, various methods and video viewing systems have been proposed so far. Although head-mounted displays (HMDs) are home-friendly and more available among these devices, they can provide an immersive video experience owing to their wide field-of-view (FoV) and separation of users from the...

This paper introduces Augmented Jump, a backpack multirotor system for jumping ability augmentation. Augmented Jump hovers and supports users' weight by a constant upward power of thrust. Users can jump higher and stay in the air for a longer time than usual with Augmented Jump. We designed and developed our first proof-of-concept prototype that ca...

Occurrences of unknown words in a conversation can be challenging and often prevent people from engaging in fluent communication with each other. Even worse, currently very little is known about possible bodily responses when a listener comes across unknown words, especially when context information is not available in the conversation to facilitat...

The human body has unique electrical characteristics. These characteristics have been investigated in various studies in human-computer interaction (HCI) and related research fields. Such studies include applications for using the body as a conductive lead for transmission or electric field sensing and activating human muscles or organs. However, e...

A head-mounted display (HMD), which is common in virtual reality (VR) systems, normally hides the user's face. This feature prohibits to realize a face-to-face communication, in which two or more users share the same virtual space, or show a participant's face on a surrogate-robot's face when the user remotely connects to the robot through an HMD f...

We propose a system, called ExtVision, to augment visual experiences by generating and projecting context-images onto the periphery of the television or computer screen. A peripheral projection of the context-image is one of the most effective techniques to enhance visual experiences. However, the projection is not commonly used at present, because...

We propose a light-weight, wearable device ElasticVR to provide various force feedback in multi-level in virtual reality (VR) for more immersive and realistic VR experiences. Force feedback is generally categorized into passive and active force feedback. Passive force is produced passively and continuously against the body part movement, e.g., elas...

Low-Fab (low-fidelity fabrication) has allowed designers to speed up the initial prototyping of 3D objects. However, the method is used to quickly verify the key aspects of the model, and are not the final desired object. Therefore, the object generated by Low-Fab are temporal objects in which the disposability and/or reusability of the material sh...

We propose a system that detects the scene, where a specific speaker is speaking in the video, and displays the site as a heat map in the video's timeline. This system enables users to skip to the timeline they want to hear by detecting scenes in a drama, talk show, or discussion TV program, where a specific speaker is speaking. To detect a specifi...

Previous studies have revealed that understanding the 3D movement of water contributes to improving propulsion in the water when swimming. Fluid measurements are made by scattering tracer particles into a liquid, and cameras track the movement of these particles to measure the fluid's flow. Strong lasers illuminate tracer particles to make them vis...

The engagement of students in a classroom is an important factor in education. Teachers are not only expected to possess strong teaching skills but also be simultaneously attentive to their students. Researchers have found that the most effective teaching happens when students in the room are in sync with one another.[1] However, assessing the clas...

Electrical Muscle Stimulation (EMS) has attracted many users and researchers to apply the technique for various usages. However, EMS products for research purpose are limited and open source hardware still has limitations. We present a multi-channel EMS toolkit for researchers and designers to develop their original ideas. The toolkit was designed...

All innovative ideas are said to be pairings of existing ideas. In collaborative ideation, various methods are used to suggest words for stimulating ideation based on the theory that suggesting strongly related words to the ideation topic will activate and enhance the ideation. However, the relationship between the words suggested by the system and...

We propose an augmented aquarium that dynamically alters the color of translucent aquatic animals without injury. This technology produces colored aquatic animals that glow in the dark in a standard fish tank at home. For regulating the color, a monitor displaying colors is installed behind the fish tank. The fish tank becomes dark owing to light-s...

We present JackIn Airsoft, a system for generating maps for multi-player sport activities and sharing their first-person views (FPVs). In first-person shooting (FPS) games, maps are generated for the visualization of the player's location; these maps are used for strategic play and discussions. FPS games are a virtual experience of shooting or of m...