Joe Paradiso’s research while affiliated with Massachusetts Institute of Technology and other places

Haptics are a promising approach for improving situation awareness when visual and auditory channels are saturated. This study explores using head-mounted haptic feedback to improve pilot recovery from unusual attitudes. Seven participants were tested in Digital Combat Simulator (DCS) under two test conditions (haptic-visual, visual-only) in two environmental conditions (daytime, nighttime).

We introduce Purrfect Pitch, a system consisting of a wearable haptic device and a custom-designed learning interface for musical ear training. We focus on the ability to identify musical intervals (sequences of two musical notes), which is a perceptually ambiguous task that usually requires strenuous rote training. With our system, the user would hear a sequence of two tones while simultaneously receiving two corresponding vibrotactile stimuli on the back. Providing haptic feedback along the back makes the auditory distance between the two tones more salient, and the back-worn design is comfortable and unobtrusive. During training, the user receives multi-sensory feedback from our system and inputs their guessed interval value on our web-based learning interface. They see a green (otherwise red) screen for a correct guess with the correct interval value. Our study with 18 participants shows that our system enables novice learners to identify intervals more accurately and consistently than those who only received audio feedback, even after the haptic feedback is removed. We also share further insights on how to design a multisensory learning system.

We are now entering the new space age! In 2021, for the first time in history that there is civilian crew in space, demonstrating the next frontier of human space exploration that will not be restricted to highly trained astronauts but will be open to a more general public. However, keeping a human healthy, happy and productive in space is one of the most challenging aspects of current space programs [11]. Thus, there is an emerging opportunity for researchers in HCI to design and research new types of interactive systems and computer interfaces that can support humans living and working in space and elsewhere in the solar system. Last year, SpaceCHI workshop (https://spacechi.media.mit.edu/) at CHI 2021 welcomed over 130 participants from 20 countries around the world to present new ideas and discuss future possibilities for human-computer interaction for space exploration. The SpaceCHI 1.0, for the first time, brought together crossdisciplinary researchers from HCI, aerospace engineering, robotics, biological science, design, art, architecture to envision the future of human space exploration leading the workshop participants and organizers to form a new global community focused on HCI research for space applications. With success from the previous SpaceCHI, we are exploring the exciting opportunity for researchers in HCI to contribute to the great endeavor of space exploration by participating in our workshop.

Hand-to-Face transmission has been estimated to be a minority, yet non-negligible, vector of COVID-19 transmission and a major vector for multiple other pathogens. At the same time, as it cannot be effectively addressed with mainstream protection measures, such as wearing masks or tracing contacts, it remains largely untackled. To help address this issue, we have developed Saving Face - an app that alerts users when they are about to touch their faces, by analyzing the distortion patterns in the ultrasound signal emitted by their earphones. The system only relies on pre-existing hardware (a smartphone with generic earphones), which allows it to be rapidly scalable to billions of smartphone users worldwide. This paper describes the design, implementation and evaluation of the system, as well as the results of a user study testing the solution's accuracy, robustness, and user experience during various day-to-day activities (93.7% Sensitivity and 91.5% Precision, N=10). While this paper focuses on the system's application to detecting hand-to-face gestures, the technique can also be applicable to other types of gestures and gesture-based applications.

Can tattoos reveal changes in human physiology? The Dermal Abyss (d-abyss) presents an approach to biointerfaces in which the body surface is rendered an interactive display. Traditional tattoo inks are replaced with biosensors whose colours or intensity change in response to variations of biomarkers in the interstitial fluid. The Dermal Abyss is designed to use the aesthetics, permanence, and visible nature of tattoos to encode diagnostic information. Here, tattoo biosensors were designed to report on the concentration of sodium ions, glucose, and pH in the skin. We report the preliminary quantitative evaluation of these biosensors in an ex vivo skin model by assessing their visibility of color changes from the dermis. This work provides a proof of concept of a platform in which the skin reveals information inside the body, tattoos form wearable displays within the skin, and the body's metabolism works as an input for the d-abyss biosensors.

The Dermal Abyss (d-abyss) presents an approach to biointerfaces in which the body surface is rendered as an interactive display by patterning biosensors into the skin to produce color changes in response to biomarker variations in the interstitial fluid. It combines advances in biotechnology with traditional methods in tattoo artistry. d-abyss is designed to use the aesthetics, permanence, and visible nature of tattoos to encode information. In the present work, we replace traditional inks with colorimetric and fluorescent biosensors that can report on the concentration of sodium, glucose, and pH in the interstitial fluid of the skin. We report the preliminary evaluation of these biosensors in an ex vivo skin model, assessing their visibility from the dermis. We describe different applications of d-abyss in the medical, lifestyle, and security domains. This work is a proof of concept of a platform in which the skin reveals information inside the body, tattoos form wearable displays within the skin, and the body's metabolism works as an input for the d-abyss biosensors.