Myounghoon Jeon

Myounghoon Jeon
Virginia Polytechnic Institute and State University | VT · Grado Department of Industrial and Systems Engineering; Department of Computer Science (by courtesy)

PhD
"Philart": Mobile and Creative Computing Faculty at Virginia Tech ISE and CS. Co-hosting AutoUI2022: auto-ui.org.

About

286
Publications
31,590
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
1,905
Citations
Introduction
My Mind Music Machine Lab at Virginia Tech ISE and CS is conducting research on auditory displays, affective computing, assistive technologies, automotive interfaces, and aesthetic computing. The current projects include artistic performance and dancer-sonification in the immersive virtual environment, visual art sonification, musical robot design for children with autism, child-robot theater program as STEAM Education, novel interaction design for Levels 3, 4, & 5 automated vehicles (e.g., gesture interaction, driving-sonification, driver-agent/robot communication, takeover request displays), emotion detection and mitigation for drivers, in-vehicle multimodal warning design for intersection and rail crossings, etc.
Additional affiliations
August 2012 - July 2018
Michigan Technological University
Position
  • Professor
August 2008 - July 2012
Georgia Institute of Technology
Position
  • Research Assistant
Education
August 2008 - July 2012
Georgia Institute of Technology
Field of study
  • Engineering Psychology and Human-Computer Interaction

Publications

Publications (286)
Article
Intelligent agents are expected to become major computing systems in the near future. In-vehicle agents (IVAs) can be widely adopted and utilized in a driving environment. Despite their potential applicability, a holistic understanding of the functional objectives and design features of IVAs remains limited, which hinders the design of better IVAs....
Conference Paper
Empathic vehicles are expected to improve user experience in automated vehicles and to help increase user acceptance of technology. However, little is known about potential real-world implementations and designs using empathic interfaces in vehicles with higher levels of automation. Given advances in affect detection and emotion mitigation, we cond...
Book
This book is dedicated to user experience design for automated driving to address humane aspects of automated driving, e.g., workload, safety, trust, ethics, and acceptance. Automated driving has experienced a major development boost in recent years. However, most of the research and implementation has been technology-driven, rather than human-cent...
Chapter
There is little doubt that driving generates emotional responses, whether that’s the joy of freedom, the boredom of stop-and-go traffic or anger towards unsafe maneuvers. In this chapter we provide an overview of emotion research applied to the automotive context and highlight the impact of emotional states in varying levels of driving automation....
Chapter
In an automated vehicle, drivers or occupants may receive a takeover request (TOR) from the automated system, which would require them to respond in a safe and timely manner to take control of the vehicle. Because drivers are likely to engage in non-driving activities in an automated vehicle, it can be challenging for them to respond safely to a TO...
Article
Full-text available
Affect is a well-known motivating and guiding force in our daily lives. With technological advancement, there has been a growing interest to include affect in the design of complex socio-technical systems (Jeon, 2017), resulting in a new wave of applications following the embodied interaction paradigm (Marshall et al., 2013). Expressing one's own a...
Article
Highway-rail grade crossings (HRGCs) present multiple collision risks for motorists, suggesting the need for additional countermeasures to increase driver compliance. The use of in-vehicle auditory alerts (IVAAs) at HRGCs has been increasing, but there are limited standards or guidelines on how such alerts should be implemented. In the current stud...
Article
This paper takes a practitioner’s perspective on advancing bi-directional transparency in human-AI-robot teams (HARTs). Bi-directional transparency is important for HARTs because the better that people and artificially intelligent agents can understand one another’s capabilities, limits, inputs, outputs and contexts in a given task environment; the...
Article
Highly automated driving systems are expected to require the design of new user-vehicle interactions. Sonification can be used to provide contextualized alarms and cues that can increase situation awareness and user experience. In this study, we examined user perceptions of potential use cases for level 4 automated vehicles in online focus group in...
Article
Full-text available
The advancement of technology requires a new teaming dynamic between humans and AI/robots. As emotions and affect are critical in forming human-human relationships, they are also expected to have a significant impact on human-AI/robot team dynamics. However, little research has been conducted on the effects of emotions on this relationship and team...
Article
Construction sites experience low productivity due to particular characteristics such as unique designs in each project, sporadic arrival of projects, and complexity of the process. Another contributing factor to low productivity is poor communication among workers, supervisors, and a site’s centralized knowledge hub. Research shows that introducin...
Chapter
To investigate the full potential of non-speech sounds, this study explored the effects of different multimodal takeover request displays in semi-automated vehicles. It used a mixed design - the visual and auditory notification lead time was within-subjects, whereas the auditory notification spatiality was between-subjects. The study was conducted...
Article
In semi-automated vehicles, non-speech sounds have been prevalently used as auditory displays for control transitions since these sounds convey urgency well. However, there are no standards of specifications for warning sounds so that diverse non-speech sounds are being employed. To shed light on this, the effects of different non-speech auditory w...
Article
Full-text available
The introduction of fully automated vehicles (FAVs) will change user experiences (UX) in personal transportation. In order for FAVs to become a life enhancing technology, it is required to design vehicular applications and user interfaces based on users' expectations. To this end, we investigated user needs and design requirements. First, we elicit...
Conference Paper
Despite rail industry advances in reducing accidents at Highway Rail Grade Crossings (HRGCs), train-vehicle collisions continue to happen. The use of auditory displays has been suggested as a countermeasure to improve driver behavior at HRGCs, with prior research recommending the use of hybrid sound alerts consisting of earcons and speech messages....
Conference Paper
For more than two decades, a myriad of design and research methods have been proposed in the ICAD community. Neurological methods have been presented since the inception of ICAD, and psychological human-subjects research has become as a legitimate approach to auditory display design and evaluation. However, little research has been conducted on mod...
Article
Full-text available
Driving agents can provide an effective solution to improve drivers’ trust in and to manage interactions with autonomous vehicles. Research has focused on voice-agents, while few have explored robot-agents or the comparison between the two. The present study tested two variables - voice gender and agent embodiment, using conversational scripts. Twe...
Article
Full-text available
While STEAM (Science, Technology, Engineering, Arts and Mathematics) education programs have shown promise in increasing students’ interest in STEM and arts & design fields, the effects of demographic and other contextual factors have not been thoroughly investigated yet. While conducting robot-theater summer youth sessions with forty participants...
Article
The Interactions website (interactions.acm.org) hosts a stable of bloggers who share insights and observations on HCI, often challenging current practices. Each issue we'll publish selected posts from some of the leading and emerging voices in the field.
Chapter
In human-robot interaction, natural and intuitive communication between robot and human is one of the most important research topics. Emotion plays a crucial role to make natural and social interactions. Research has focused more on robots’ appearances and facial emotional expressions, but little research has investigated robots’ voices and their m...
Conference Paper
Full-text available
A worldwide pandemic has brought many challenges in numerous areas of everyone's life. The AutomotiveUI 2020 has also been moved to a virtual conference. Although the situation seems to be improving in some parts of the world, the impacts that the pandemic has brought to the research and academia may last long even after the pandemic is over. In th...
Conference Paper
Full-text available
Driver performance and behavior can be partially predicated based on one's emotional state. Through ascertaining the emotional state of passengers and employing various mitigation strategies, empathic cars can show potential in improving user experience and driving performance. Challenges remain in the implementation of such strategies, as individu...
Article
Using robots in therapy for children on the autism spectrum is a promising avenue for child-robot interaction, and one that has garnered significant interest from the research community. After preliminary interviews with stakeholders and evaluating music selections, twelve typically developing (TD) children and three children with Autism Spectrum D...
Article
Traditionally, driver distraction has been categorized into four types: visual, biomechanical, auditory, and cognitive. However, the place of emotion in driving research is largely undefined. The present study investigates the specific influences of anger – representative emotion arisen while driving, on driving performance, compared to those of tr...
Article
For almost 3 decades, research on auditory displays and sonification has been well advanced. Now, the auditory display community has arrived at the stage of sonic information design with a more systematic, refined necessity, going beyond random mappings between the referents and sounds. Due to its innate transdisciplinary nature of auditory display...
Conference Paper
Full-text available
An automated driving system is expected to pave the way for a new area of user experience in a vehicle. However, few studies have been conducted on the understanding what people want to do and how the vehicle can support user needs, specifically, in level 5, fully automated vehicles (FAVs). Therefore, the present study aimed at exploring user needs...
Article
Sonification has the potential to communicate a variety of data types to listeners including not just cognitive information, but also emotions and aesthetics. The goal of our dancer sonification project is to “sonify emotions as well as motions” of a dance performance via musical sonification. To this end, we developed and evaluated sonification st...
Article
One of the options to make science, technology, engineering, and mathematics (STEM) more accessible, especially for children, is to integrate STEM content into more attractive materials and familiar formats. In this line, by integrating STEM with arts and design, we have created an afterschool program, “Child-Robot Theater” for children in a rural...
Article
The diagnosis of Autism Spectrum Disorder (ASD) in children is commonly accompanied by a diagnosis of sensory processing disorders. Abnormalities are usually reported in multiple sensory processing domains, showing a higher prevalence of unusual responses, particularly to tactile, auditory, and visual stimuli. This article discusses a novel robot-b...
Article
While design theories in visual displays have been well developed and further refined, relatively little research has been conducted on design theories and models in auditory displays. The existing discussions mainly account for functional mappings between sounds and referents, but these do not fully address design aspects of auditory displays. To...
Conference Paper
Full-text available
AutoUI conference is the premier forum for user interface research in the automotive domain, annually bringing together over 200 researchers and practitioners interested in both the technical and the human aspects of in-vehicle user interfaces and applications. However, over 80% of its published papers come from only five countries from western Eur...
Conference Paper
In semi-autonomous vehicles (SAE level 3) that require driver's engagement in critical situations, it is important to secure reliable control transitions. There have been many studies on investigating appropriate auditory displays for takeover request (TOR) but most of them were empirical experiments. In the present study, we established two comput...
Conference Paper
Full-text available
A driving agent can be an effective interface to interact with drivers to increase trust towards the autonomous driving vehicle. While driving research on agent has mostly focused on the voice-agent, little empirical findings on the robot-agent were reported. In the present study, we compared three different agents (informative voice-agent, informa...
Chapter
This paper presents a human-agent interaction framework for emotional regulation through emotional interaction with a virtual agent that uses a consensus-based algorithm. This model-based framework provides a parameterized and dynamic approach for emotional interaction with an agent that can be applied to social robot-based therapies and virtual ag...
Chapter
Child facial expression recognition plays an important role in child-machine interactions. Recognition methods that were tuned for adults have been used for children in many studies without evaluating the applicability of these methods on children. This paper investigates this problem using a Support Vector Machine classification-based recognition...
Article
Full-text available
Recent advancements in automated vehicle technologies pose numerous opportunities and challenges to support the diverse facets of user needs [...]
Article
The connection between art and technology is much tighter than is commonly recognized. The emergence of aesthetic computing in the early 2000s has brought renewed focus on this relationship. In this article, we articulate how art and Human–Computer Interaction (HCI) are compatible with each other and actually essential to advance each other in this...
Conference Paper
Sonification and data processing algorithms have advanced over the years to reach practical applications in our everyday life. Similarly, image processing techniques have improved over time. While a number of image sonification methods have already been developed, few have delved into potential synergies through the combined use of multiple data an...
Conference Paper
Since non-speech sounds can convey urgency well, they have been used as alerts in the vehicle context, including control transitions (handover and takeover) in automated vehicles. However, their potential has not been fully investigated to make use in international standards. To contribute to making authentic standards, the present paper investigat...
Conference Paper
Take-over is one of the most crucial user interactions in semi-automated vehicles. To make better communication between driver and vehicle, research has been conducted on various take-over request displays, yet the potential has not been fully investigated. The present paper explored the effects of adding auditory displays to visual text. Earcon an...
Conference Paper
STEAM education fuses arts with traditional STEM fields so that the diverse disciplines can broaden and inform each other. Our eight-week STEAM after school program exposed elementary school children to social robotics and musical theater. Approximately 25 children grades K-5 participated over the course of the program with an average of 12 childre...
Article
: Recently, cutting-edge technology has led to the development of automated vehicles, but the limitations of the related technology may lead to hazardous situations. This resulted in the remarkable significance of the interaction between automated vehicles and drivers. In particular, the transition between the driver and the automated vehicle in ac...
Article
Using touchscreens while driving introduces competition for visual attention that increases crash risk. To resolve this issue, we have developed an auditory-supported air gesture system. We conducted two experiments using the driving simulator to investigate the influence of this system on driving performance, eye glance behavior, secondary task pe...
Article
Emotions influence the way drivers process and react to internal or environmental factors, but relatively little research has focused on drivers’ emotions. Of many emotional states, anger is considered the most serious threat on the road. Therefore, having an affective intelligent system in the car that can estimate drivers’ anger and respond to it...
Article
Train-vehicle collisions at highway-rail grade crossings (RR crossings) continue to be a major issue in the US and across the world. To prevent several decades of safety improvements from plateauing, experts are turning towards novel warning devices that can be applied to all crossings with minimal cost. One of the potential approaches is in-vehicl...
Preprint
The diagnosis of Autism Spectrum Disorder (ASD) in children is commonly accompanied by a diagnosis of sensory processing disorders as well. Abnormalities are usually reported in multiple sensory processing domains, showing a higher prevalence of unusual responses, particularly to tactile, auditory and visual stimuli. This paper discusses a novel ro...
Article
This is the second part of the special issue on sonic information design. It continues the theme of presenting information via intentionally designed auditory displays. The term auditory display was coined in the 1950s by Pollack and Ficks (1954), who were experimenting with auditory versions of the visual cockpit displays to improve pilot performa...
Article
Full-text available
The purpose of this study was to evaluate the functionality, accuracy, and usability of a novel smart exercise application (SEA). The functionality such as counting lunges, providing task-related auditory feedback, and testing short-term memory was examined while thirteen young adults (six men, age 25.4 ± 8.3 years) performed the lunge exercise wit...
Conference Paper
Full-text available
In-car emotion detection and regulation have become an emerging and important branch of research within the automotive domain. Different emotional states can greatly influence human driving performance and user experience both in manual and automated driving conditions. The monitoring and regulation of relevant emotional states is therefore importa...
Conference Paper
Auditory displays have widely been used in vehicle contexts because they do not require drivers' visual attention. Examples include collision warning systems, parking assistance systems, and personal navigation devices to name a few. To explore the full potential of sonification applications in the automotive context, we have developed an in-vehicl...
Conference Paper
Automated driving systems (ADS), especially in higher levels of automation, seem to be the new focus of innovation regarding future mobility. Technological achievements of traveling automation open up new challenges for road traffic. Existing automotive research focuses on problem solving and observational approaches including users and their imagi...
Article
To understand and apply Human Factors’ core knowledge more effectively, it is essential to provide a cohesive learning experience to students based on real-world applications. Researchers have developed ample ways to bring real-world examples to the classroom, but it is not always possible to obtain and demonstrate all the new technologies in the c...
Conference Paper
Socially assistive robots have been used extensively in designing therapy interventions for children on the autism spectrum. One of the major challenges facing this field is the difficulty in sustaining the child's engagement in a designed activity for longitudinal studies once the novelty of the robot wears off. Another major challenge is the desi...
Conference Paper
Full-text available
Auditory displays are a useful platform to convey information to users for a variety of reasons. The present study sought to examine the use of different types of sounds that can be used in auditory displays—music, earcons, spearcons, and lyricons—to determine which sounds have the highest learnability when presented in sequences. Participants were...
Conference Paper
Performing independent physical exercise is critical to maintain one's good health, but it is specifically hard for people with visual impairments. To address this problem, we have developed a Musical Exercise platform for people with visual impairments so that they can perform exercise in a good form consistently. We designed six different conditi...
Article
Performing independent physical exercise is critical to maintain one's good health, but it is specifically hard for people with visual impairments. To address this problem, we have developed a Musical Exercise platform for people with visual impairments so that they can perform exercise in a good form consistently. We designed six different conditi...
Article
Full-text available
Auditory displays are a useful platform to convey information to users for a variety of reasons. The present study sought to examine the use of different types of sounds that can be used in auditory displays—music, earcons, spearcons, and lyricons—to determine which sounds have the highest learnability when presented in sequences. Participants were...
Chapter
As development of automated vehicles and adoption of electric vehicles continue to grow, there is an increasing interest in the public opinions on these technologies. We conducted an international online survey to gather information about people’s hopes and concerns for automated and electric vehicles from a total of 866 people from four countries...
Conference Paper
Although accidents at Highway-Rail Grade Crossings (HRGCs) have been greatly reduced over the past decades, they continue to be a major problem for the rail industry, causing injuries, loss of life, and loss of revenue. Recently, the Strategic Highway Research Program sponsored a Naturalistic Driving Study, the SHRP2 NDS, which produced a unique op...