Mohammed E. Hoque

Massachusetts Institute of Technology, Cambridge, Massachusetts, United States

Are you Mohammed E. Hoque?

Claim your profile

Publications (11)0 Total impact

  • M. Hoque, R.W. Picard
    [Show abstract] [Hide abstract]
    ABSTRACT: We present a real-time system including a 3D character that can converse, capture, analyze and interpret subtle and multidimensional human nonverbal behaviors for possible applications such as job interviews, public speaking, or even automated speech therapy. The system works in a personal computer and senses nonverbal data from video (i.e., facial expressions) and audio (i.e., speech recognition and prosody analysis) using a standard web cam. We contextualized the development and evaluation of our system as a training scenario for job interviews. Using user-centered design and iterations, we determine how the nonverbal data could be presented to the user in an intuitive and educational manner. We tested efficacy of the system in context of job interviews with 90 MIT undergraduate students. Our results suggest that the participants who used our system to improve their interview skills were perceived to be better candidates by human judges. Participants reported that the most useful feature was being given feedback on their speaking rate, and overall they reported strong agreement that would consider using this system again for self-reflection.
    Affective Computing and Intelligent Interaction (ACII), 2013 Humaine Association Conference on; 01/2013
  • Javier Hernandez, Mohammed E. Hoque, Rosalind W. Picard
    [Show abstract] [Hide abstract]
    ABSTRACT: Have you ever wondered whether it's possible to quantitatively measure how friendly or welcoming a community is? Or imagined which parts of the community are happier than others? In this work, we introduce a new technology that begins to address these questions.
    ACM SIGGRAPH 2012 Emerging Technologies; 08/2012
  • Source
    M. Hoque, R.W. Picard
    [Show abstract] [Hide abstract]
    ABSTRACT: This work is part of research to build a system to combine facial and prosodic information to recognize commonly occurring user states such as delight and frustration. We create two experimental situations to elicit two emotional states: the first involves recalling situations while expressing either delight or frustration; the second experiment tries to elicit these states directly through a frustrating experience and through a delightful video. We find two significant differences in the nature of the acted vs. natural occurrences of expressions. First, the acted ones are much easier for the computer to recognize. Second, in 90% of the acted cases, participants did not smile when frustrated, whereas in 90% of the natural cases, participants smiled during the frustrating interaction, despite self-reporting significant frustration with the experience. This paper begins to explore the differences in the patterns of smiling that are seen under natural frustration and delight conditions, to see if there might be something measurably different about the smiles in these two cases, which could ultimately improve the performance of classifiers applied to natural expressions.
    Automatic Face & Gesture Recognition and Workshops (FG 2011), 2011 IEEE International Conference on; 04/2011
  • Source
    Affective Computing and Intelligent Interaction - 4th International Conference, ACII 2011, Memphis, TN, USA, October 9-12, 2011, Proceedings, Part I; 01/2011
  • [Show abstract] [Hide abstract]
    ABSTRACT: Affective computing (AC) is a unique discipline which includes modeling affect using one or multiple modalities by drawing on techniques from many different fields. AC often deals with problems that are known to be very complex and multi-dimensional, involving different kinds of data (numeric, symbolic, visual etc.). However, with the advancement of machine learning techniques, a lot of those problems are now becoming more tractable. The purpose of this workshop was to engage the machine learning and affective computing communities towards solving problems related to understanding and modeling social affective behaviors. We welcomed participation of researchers from diverse fields, including signal processing and pattern recognition, statistical machine learning, human-computer interaction, human-robot interaction, robotics, conversational agents, experimental psychology, and decision making. There is a need for a set of high standards for recognizing and understanding affect. At the same time, these standards need to take into account that the expectations and validations in this area may be different than in traditional research on machine learning. This should be reflected in the design of machine learning techniques used to tackle these problems. For example, affective data sets are known to be noisy, high dimensional, and incomplete. Classes may overlap. Affective behaviors are often person specific and require temporal modeling with real-time performance. This first edition of the ACII Workshop on Machine Learning for Affective Computing will be a proper venue to invoke such discussions and engage the community towards design and validation of learning techniques for affective computing.
    Affective Computing and Intelligent Interaction - Fourth International Conference, ACII 2011, Memphis, TN, USA, October 9-12, 2011, Proceedings, Part II; 01/2011
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Individuals on the autism spectrum often have difficulties producing intelligible speech with either high or low speech rate, and atypical pitch and/or amplitude affect. In this study, we present a novel intervention towards customizing speech enabled games to help them produce intelligible speech. In this approach, we clinically and computationally identify the areas of speech production difficulties of our participants. We provide an interactive and customized interface for the participants to meaningfully manipulate the prosodic aspects of their speech. Over the course of 12 months, we have conducted several pilots to set up the experimental design, developed a suite of games and audio processing algorithms for prosodic analysis of speech. Preliminary results demonstrate our intervention being engaging and effective for our participants.
    INTERSPEECH 2009, 10th Annual Conference of the International Speech Communication Association, Brighton, United Kingdom, September 6-10, 2009; 01/2009
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Participatory user interface design with adolescent users on the autism spectrum presents a number of unique challenges and opportunities. Through our work developing a system to help autistic adolescents learn to recognize facial expressions, we have learned valuable lessons about software and hardware design issues for this population. These lessons may also be helpful in assimilating iterative user input to customize technology for other populations with special needs.
    Proceedings of the 27th International Conference on Human Factors in Computing Systems, CHI 2009, Extended Abstracts Volume, Boston, MA, USA, April 4-9, 2009; 01/2009
  • Source
    Mohammed E. Hoque, Rana El Kaliouby, Rosalind W. Picard
    [Show abstract] [Hide abstract]
    ABSTRACT: This paper describes the challenges of getting gro und truth affective labels for spontaneous video, and presents implicat ions for systems such as virtual agents that have automated facial analysis capabilities. We first present a dataset from an intelligent tutoring application an d describe the most prevalent approach to labeling such data. We then present an alternative labeling approach, which closely models how the majority of automated facial analysis systems are designed. We show that while participan ts, peers and trained judges report high inter-rater agreement on expressions of delight, confusion, flow, frustration, boredom, surprise, and neutral when sh own the entire 30 minutes of video for each participant, inter-rater agreement d rops below chance when human coders are asked to watch and label short 8 s econd clips for the same set of labels. We also perform discriminative analysis for facial action units for each affective state represented in the clips. The results emphasize that human coders heavily rely on factors such as familiarity of the person and context of the interaction to correctly infer a person's affec tive state; without this information, the reliability of humans as well as m achines attributing affective labels to spontaneous facial-head movements drops s ignificantly.
    Intelligent Virtual Agents, 9th International Conference, IVA 2009, Amsterdam, The Netherlands, September 14-16, 2009, Proceedings; 01/2009
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Social communication in autism is significantly hindered by difficulties processing affective cues in realtime face-to-face interaction. The interactive Social-Emotional Toolkit (iSET) allows its users to record and annotate video with emotion labels in real time, then review and edit the labels later to bolster understanding of affective information present in interpersonal interactions. The iSET demo will let the ACII audience experience the augmentation of interpersonal interactions by using the iSET system.
    01/2009;
  • Source
    Philipp Robbel, Mohammed E Hoque, Cynthia Breazeal
    [Show abstract] [Hide abstract]
    ABSTRACT: This paper describes an integrated approach to recognizing and generating affect on a humanoid robot as it interacts with a human user. We describe a method for detecting basic affect signals in the user's speech input and generate appropriately chosen responses on our robot platform. Re-sponses are selected both in terms of content and emotional quality of the voice. Additionally, we synthesize gestures and facial expressions on the robot that magnify the effect of the conveyed emotional state of the robot. The guiding principle of our work is that adding the ability to detect and display emotion to physical agents allows their effective use in novel application areas such as child and elderly care, healthcare, education, and beyond.
    01/2009;
  • Source
    Mohammed E. Hoque
    [Show abstract] [Hide abstract]
    ABSTRACT: Many individuals diagnosed with autism and Down syndrome have difficulties producing intelligible speech. Sy stematic analysis of their voice parameters could lead to be tter understanding of the specific challenges they face in achieving proper speech production. In this study, 100 minute s of speech data from natural conversations between neurotypica ls and individuals diagnosed with autism/Down-syndrome was used. Analyzing their voice parameters indicated new find ings across a variety of speech parameters. An immediate extens ion of this work would be to customize this technology allowing participants to visualize and control their speech parameters in real time and get live feedback.
    Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 2008, Halifax, Nova Scotia, Canada, October 13-15, 2008; 01/2008