Chapter

Research in the era of sensing technologies and wearables - Bookchapter in The Cambridge Handbook of Technology and Employee Behavior

Chapter

Research in the era of sensing technologies and wearables - Bookchapter in The Cambridge Handbook of Technology and Employee Behavior

If you want to read the PDF, try requesting it from the authors.

Abstract

The Cambridge Handbook of Technology and Employee Behavior - edited by Richard N. Landers February 2019

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Researchers have called for studies investigating automated interview approaches because their effects on interviewees remain unknown (Blacksmith, Willford, & Behrend, 2016;Langer et al., 2017). In a first attempt to examine participant reactions to highlyautomated interviews, we therefore mimicked the general idea behind these approaches (see Chamorro-Premuzic, Akhtar, Winsborough, & Sherman, 2017;HireVue, 2018;Langer, Schmid Mast, Meyer, Maass, & König, 2019;Precire, 2017). Specifically, we introduced ACCEPTANCE OF HIGHLY-AUTOMATED INTERVIEWS 3 participants of the current study to a highly-automated interview tool within a virtual set-up (i.e., with a virtual agent as the interviewer). ...
... In the case of job interviews, acquiring information means to have an automated way to collect and extract data. At a low level of automation this could mean that video cameras and microphones automatically record video and audio data (Langer et al., 2019). On higher levels of automation, it might then be possible to use the recorded data to automatically transcribe the interviews, which saves time compared to taking notes during interviews (Middendorf & Macan, 2002). ...
... In practice, similar tools are already used. For instance, similar virtual training tools are already used to support human trainers in negotiation training (Langer et al., 2019), and in the field of personnel selection, Schmid Mast and colleagues (2017) used highly-automated interviews to predict the job performance of student assistants, similar to the ones already offered by providers of automated interview solutions (e.g., HireVue, 2018;Precire, 2017). ...
Article
Full-text available
Technological advancements in Artificial Intelligence allow the automation of every part of job interviews (information acquisition, information analysis, action selection, action implementation) resulting in highly automated interviews. Efficiency advantages exist, but it is unclear how people react to such interviews (and whether reactions depend on the stakes involved). Participants (N = 123) in a 2 (highly automated, videoconference) × 2 (high‐stakes, low‐stakes situation) experiment watched and assessed videos depicting a highly automated interview for high‐stakes (selection) and low‐stakes (training) situations or an equivalent videoconference interview. Automated high‐stakes interviews led to ambiguity and less perceived controllability. Additionally, highly automated interviews diminished overall acceptance through lower social presence and fairness. To conclude, people seem to react negatively to highly automated interviews and acceptance seems to vary based on the stakes. OPEN PRACTICES This study was pre‐registered on the Open Science Framework (osf.io/hgd5r) and on AsPredicted (https://AsPredicted.org/i52c6.pdf).
... Specifically, machine learning could ostensibly be used to leverage a variety of objective performance data and automatically identify the most impactful information to provide to trainees as task feedback (Jordan & Mitchell, 2015). Indeed, there are now a plethora of objective team performance indicators, including various sensing technologies and wearables, which could, when combined with machine-learning algorithms, be used to provide trainees with task performance feedback (Langer, Schmid Mast, Meyer, Maass, & König, 2019). More broadly, machine learning could automate the AAR administration and process and therefore eliminate the use of an expert facilitator (e.g., Core et al., 2006). ...
Conference Paper
The after-action review (AAR), also termed debrief, is a special type of work meeting that commonly encompasses some form of technology because of the contexts in which it is most frequently applied (i.e., for action teams in simulated task environments). However, technology is largely treated as a tangential consideration in the extant AAR literature, which serves as the impetus for the present study. Based on a systematic review of 73 empirical articles, a variety of nuances were identified about (1) where in the AAR administration or process technology is used, and the (2) users, (3) type, and (4) use of that technology. Technology is indeed common to AARs, but largely relegated to either aid in the task performance episode that precedes the AAR (91%) or in the provision of task feedback (61%). More broadly, the findings from the present review reflect the inherent complexity of determining how best to use technology in AARs with little extant guidance. Researchers have responded to this complexity often by relying on a single type of technology to provide trainees with feedback (i.e., a video review) and on expert facilitators to dictate how the objective data collected via technology is provided to trainees. The present review provides the basis for subsequent conceptual development and empirical research to help explain the effect of technology on the effectiveness of AARs.
... It might be worthwhile to extend these datasets by using data from alternative and new sources. In particular, such new data could come from using sensors like wearables (for an overview see Langer, Schmid Mast, et al., 2019). ...
Chapter
Full-text available
Machine learning (ML) approaches, a subfield of artificial intelligence (AI), promise advancements in the field of personnel selection. This chapter introduces ML approaches to personnel selection practitioners and researchers in a non-technical way. We review the empirical research to date, specifically research that has looked at the potentials of ML approaches (in particular the increased prediction power) as well as the challenges and disadvantages of such approaches. We explain that the assumption of bias-free ML approaches is unwarranted, and that there might be negative reactions among applicants and users. We close this chapter by providing suggestions for highly needed research to demonstrate the validity of ML approaches to selection, to analyse the human-AI interface, and to more closely examine the reactions of applicants, users, and further stakeholders.
... Sensors in wearables and smartphones allow unobtrusive collection of large scale longitudinal and behavioral data (cf. Langer, Schmid Mast, Meyer, Maass, & König, 2019). An important advantage is that these approaches can augment self-reports, helping to overcome common methodological issues (e.g., common method bias and social desirable responding) and thus increasing ecological validity of findings (cf. ...
Article
Full-text available
This article is based on conversations from the project “Big Data in Psychological Assessment” (BDPA) funded by the European Union, which was initiated because of the advances in data science and artificial intelligence that offer tremendous opportunities for personnel assessment practice in handling and interpreting this kind of data. We argue that psychologists and computer scientists can benefit from interdisciplinary collaboration. This article aims to inform psychologists who are interested in working with computer scientists about the potentials of interdisciplinary collaboration, as well as the challenges such as differing terminologies, foci of interest, data quality standards, approaches to data analyses, and diverging publication practices. Finally, we provide recommendations preparing psychologists who want to engage in collaborations with computer scientists. We argue that psychologists should proactively approach computer scientists, learn computer scientific fundamentals, appreciate that research interests are likely to converge, and prepare novice psychologists for a data-oriented scientific future.
... We believe that one of the major barriers for psychologists embarking on this technological bandwagon is their limited understanding and command of technology. Granted, using technology for research is fairly easy (Langer, Schmid Mast, Meyer, Maass, & König, 2019), but developing technology requires a completely different set of skills and interests. Modern computer technology has a great advantage in this respect: it is to some extent reusable once it has been developed. ...
Article
Full-text available
The Electronically Activated Recorder (EAR) is a method for collecting periodic brief audio snippets of participants’ daily lives using a portable recording device. The EAR can potentially intrude into people’s privacy, alter their natural behavior, and introduce self-selection biases greater than in other types of social science methods. Previous research (Mehl and Holleran, 2007, hereafter M&H) has shown that participant non-compliance with, and perceived obtrusiveness of, an EAR protocol are both low. However, these questions have not been addressed in jurisdictions that require the consent of all parties to recording conversations. This EAR study required participants to wear a button bearing a microphone icon and the words “This conversation may be recorded” to comply with California’s all-party consent law. Results revealed self-reported obtrusiveness and non-compliance were actually lower in the present study than in the M&H study. Behaviorally assessed non-compliance did not differ between the two studies. Participants in the present study talked more about being in the study than participants in the M&H study, but such talk still comprised <2% of sampled conversations. Another potential problem with the EAR, participant self-selection bias, was addressed by comparing the EAR volunteers’ HEXACO personality dimensions to a non-volunteer sample drawn from the same student population. EAR volunteers were significantly and moderately higher in Conscientiousness, and lower in Emotionality, than non-volunteers. In conclusion, the EAR method can be successfully implemented in at least one all-party consent state (California). Interested researchers are encouraged to review this procedure with their own legal counsel.
Article
Full-text available
This article reviews the Electronically Activated Recorder (EAR) as an ambulatory ecological momentary assessment tool for the real-world observation of daily behavior. Technically, the EAR is an audio recorder that intermittently records snippets of ambient sounds while participants go about their lives. Conceptually, it is a naturalistic observation method that yields an acoustic log of a person’s day as it unfolds. The power of the EAR lies in unobtrusively collecting authentic real-life observational data. In preserving a high degree of naturalism at the level of the raw recordings, it resembles ethnographic methods; through its sampling and coding, it enables larger empirical studies. This article provides an overview of the EAR method; reviews its validity, utility, and limitations; and discusses it in the context of current developments in ambulatory assessment, specifically the emerging field of mobile sensing.
Article
Full-text available
Wrist-worn fitness and heart rate (HR) monitors are popular.¹,2 While the accuracy of chest strap, electrode-based HR monitors has been confirmed,³,4 the accuracy of wrist-worn, optically based HR monitors is uncertain.⁵,6 Assessment of the monitors’ accuracy is important for individuals who use them to guide their physical activity and for physicians to whom these individuals report HR readings. The objective of this study was to assess the accuracy of 4 popular wrist-worn HR monitors under conditions of varying physical exertion.
Article
Full-text available
Expanding research on employment interview training, this study introduces virtual employment interview (VI) training with focus on nonverbal behavior. In VI training, participants took part in a simulated interview with a virtual character. Simultaneously, the computer analyzed participants’ nonverbal behavior and provided real-time feedback for it. The control group received parallel interview training. Following training, participants took part in mock interviews, where interviewers rated participants’ nonverbal behavior, and interview performance. Analyses revealed (a) that participants of VI training showed better interview performance, (b) that this effect was mediated by nonverbal behavior, and (c) that VI training has a positive influence on interview anxiety. These results have important practical implications for applicants, career counseling centers, and organizations.
Article
Full-text available
Background: Wrist-worn monitors claim to provide accurate measures of heart rate and energy expenditure. People wishing to lose weight use these devices to monitor energy balance, however the accuracy of these devices to measure such parameters has not been established. Aim: To determine the accuracy of four wrist-worn devices (Apple Watch, Fitbit Charge HR, Samsung Gear S and Mio Alpha) to measure heart rate and energy expenditure at rest and during exercise. Methods: Twenty-two healthy volunteers (50% female; aged 24 ± 5.6 years) completed ~1-hr protocols involving supine and seated rest, walking and running on a treadmill and cycling on an ergometer. Data from the devices collected during the protocol were compared with reference methods: electrocardiography (heart rate) and indirect calorimetry (energy expenditure). Results: None of the devices performed significantly better overall, however heart rate was consistently more accurate than energy expenditure across all four devices. Correlations between the devices and reference methods were moderate to strong for heart rate (0.67-0.95 [0.35 to 0.98]) and weak to strong for energy expenditure (0.16-0.86 [-0.25 to 0.95]). All devices underestimated both outcomes compared to reference methods. The percentage error for heart rate was small across the devices (range: 1-9%) but greater for energy expenditure (9-43%). Similarly, limits of agreement were considerably narrower for heart rate (ranging from -27.3 to 13.1 bpm) than energy expenditure (ranging from -266.7 to 65.7 kcals) across devices. Conclusion: These devices accurately measure heart rate. However, estimates of energy expenditure are poor and would have implications for people using these devices for weight loss.
Article
Full-text available
Continuous respiratory monitoring is important to assess adequate ventilation. We present a fiber optic-based smart textile for respiratory monitoring able to work during Magnetic Resonance (MR) examinations. The system is based on the conversion of chest wall movements into strain of two fiber Bragg grating (FBG) sensors, placed on the upper thorax (UT). FBGs are glued on the textile by an adhesive silicon rubber. To increase the system sensitivity, the FBGs positioning was led by preliminary experiments performed using an optoelectronic system: FBGs placed on the chest surface experienced the largest strain during breathing. System performances, in terms of respiratory period (TR), duration of inspiratory (TI) and expiratory (TE) phases, as well as left and right UT volumes, were assessed on four healthy volunteers. The comparison of results obtained by the proposed system and an optoelectronic plethysmography highlights the high accuracy in the estimation of TR, TI, and TE: Bland-Altman analysis shows mean of difference values lower than 0.045 s, 0.33 s, and 0.35 s for TR, TI, and TE, respectively. The mean difference of UT volumes between the two systems is about 8.3%. The promising results foster further development of the system to allow routine use during MR examinations.
Article
Full-text available
In recent years, researchers in work and organizational psychology have increasingly become interested in short-term processes and everyday experiences of working individuals. Diaries provide the necessary means to examine these processes. Although diary studies have become more popular in recent years, researchers not familiar with this method still find it difficult to get access to the required knowledge. In this paper, we provide an introduction to this method of data collection. Using two diary study examples, we discuss methodological issues researchers face when planning a diary study, examine recent methodological developments, and give practical recommendations. Topics covered include different types of diary studies, the research questions to be examined, compliance and the issue of missing data, sample size, and issues of analyses.
Article
Full-text available
In this article, we show how the use of state-of-the-art methods in computer science based on machine perception and learning allows the unobtrusive capture and automated analysis of interpersonal behavior in real time (social sensing). Given the high ecological validity of the behavioral sensing, the ease of behavioral-cue extraction for large groups over long observation periods in the field, the possibility of investigating completely new research questions, and the ability to provide people with immediate feedback on behavior, social sensing will fundamentally impact psychology.
Conference Paper
Full-text available
Ever wondered why you have been rejected from a job despite being a qualified candidate? What went wrong? In this paper, we provide a computational framework to quantify human behavior in the context of job interviews. We build a model by analyzing 138 recorded interview videos (total duration of 10.5 hours) of 69 internship-seeking students from Massachusetts Institute of Technology (MIT) as they spoke with professional career counselors. Our automated analysis includes facial expressions (e.g., smiles, head gestures), language (e.g., word counts, topic modeling), and prosodic information (e.g., pitch, intonation, pauses) of the interviewees. We derive the ground truth labels by averaging over the ratings of 9 independent judges. Our framework automatically predicts the ratings for interview traits such as excitement, friendliness, and engagement with correlation coefficients of 0.73 or higher, and quantifies the relative importance of prosody, language, and facial expressions. According to our framework, it is recommended to speak more fluently, use less filler words, speak as " we " (vs. " I "), use more unique words, and smile more.
Article
Full-text available
Despite the importance of the concept of cultural display rules in explaining cultural differences in emotional expression, and despite the fact that it has been over 30 years since this concept was coined (Ekman & Friesen, 1969), there is yet to be a study that surveys display rules across a wide range of cultures. This article reports such a study. Over 5,000 respondents in 32 countries completed the Display Rule Assessment Inventory (Matsumoto, Yoo, Hirayama, & Petrova, 2005). We examined five hypotheses concerning the relationship between display rules and Individualism-Collectivism (IC). The findings indicated the existence of several universal effects, including greater expression toward ingroups v. outgroups, and an overall regulation effect. Individualistic and collectivistic cultures differed on overall expressivity endorsement, and in norms concerning specific emotions in ingroup and outgroup situations.
Article
Full-text available
By 2025, when most of today's psychology undergraduates will be in their mid-30s, more than 5 billion people on our planet will be using ultra-broadband, sensor-rich smartphones far beyond the abilities of today's iPhones, Androids, and Blackberries. Although smartphones were not designed for psychological research, they can collect vast amounts of ecologically valid data, easily and quickly, from large global samples. If participants download the right "psych apps," smartphones can record where they are, what they are doing, and what they can see and hear and can run interactive surveys, tests, and experiments through touch screens and wireless connections to nearby screens, headsets, biosensors, and other peripherals. This article reviews previous behavioral research using mobile electronic devices, outlines what smartphones can do now and will be able to do in the near future, explains how a smartphone study could work practically given current technology (e.g., in studying ovulatory cycle effects on women's sexuality), discusses some limitations and challenges of smartphone research, and compares smartphones to other research methods. Smartphone research will require new skills in app development and data analysis and will raise tough new ethical issues, but smartphones could transform psychology even more profoundly than PCs and brain imaging did. © The Author(s) 2012.
Article
Full-text available
Although the purpose of questionnaire items is to obtain a person’s opinion on a certain matter, a respondent’s registered opinion may not reflect his or her “true” opinion because of random and systematic errors. Response styles (RSs) are a respondent’s tendency to respond to survey questions in certain ways regardless of the content, and they contribute to systematic error. They affect univariate and multivariate distributions of data collected by rating scales and are alternative explanations for many research results. Despite this, RS are often not controlled in research. This article provides a comprehensive summary of the types of RS, lists their potential sources, and discusses ways to diagnose and control for them. Finally, areas for further research on RS are proposed.
Article
Full-text available
Abstract Epidemic levels of inactivity are associated with chronic diseases and rising healthcare costs. To address this, accelerometers have been used to track levels of activity. The Fitbit and Fitbit Ultra are some of the newest commercially available accelerometers. The purpose of this study was to determine the reliability and validity of the Fitbit and Fitbit Ultra. Twenty-three subjects were fitted with two Fitbit and Fitbit Ultra accelerometers, two industry-standard accelerometers and an indirect calorimetry device. Subjects participated in 6-min bouts of treadmill walking, jogging and stair stepping. Results indicate the Fitbit and Fitbit Ultra are reliable and valid for activity monitoring (step counts) and determining energy expenditure while walking and jogging without an incline. The Fitbit and standard accelerometers under-estimated energy expenditure compared to indirect calorimetry for inclined activities. These data suggest the Fitbit and Fitbit Ultra are reliable and valid for monitoring over-ground energy expenditure.
Article
Full-text available
More than 40 years ago, Masahiro Mori, a robotics professor at the Tokyo Institute of Technology, wrote an essay [1] on how he envisioned people's reactions to robots that looked and acted almost like a human. In particular, he hypothesized that a person's response to a humanlike robot would abruptly shift from empathy to revulsion as it approached, but failed to attain, a lifelike appearance. This descent into eeriness is known as the uncanny valley. The essay appeared in an obscure Japanese journal called Energy in 1970, and in subsequent years, it received almost no attention. However, more recently, the concept of the uncanny valley has rapidly attracted interest in robotics and other scientific circles as well as in popular culture. Some researchers have explored its implications for human-robot interaction and computer-graphics animation, whereas others have investigated its biological and social roots. Now interest in the uncanny valley should only intensify, as technology evolves and researchers build robots that look human. Although copies of Mori's essay have circulated among researchers, a complete version hasn't been widely available. The following is the first publication of an English translation that has been authorized and reviewed by Mori. (See “Turning Point” in this issue for an interview with Mori.).
Article
Full-text available
Self-reports figure prominently in organizational and management research, but there are several problems associated with their use. This article identifies six categories of self-reports and discusses such problems as common method variance, the consistency motif, and social desirability. Statistical and post hoc remedies and some procedural methods for dealing with artifactual bias are presented and evaluated. Recommendations for future research are also offered.
Article
Full-text available
Objective: Identification of stress patterns in the voice has multiple potential applications. The objective was to review literature pertaining to the effects of various forms of stress upon the healthy voice. Study design: Literature review, discussion of results, and direction for further study. Methods: This review article offers a model of stress and a review of the historical and recent research into the effects of stress on the voice. Electronic databases were searched using the key words. No studies were excluded on the basis of design; however, an attempt was made to include in the discussion studies which primarily address physiological and acoustic vocal parameters. The results of greater than 50 studies examining the effect of stressors ranging from lie and guilt to high altitude and space flight upon the voice were included in the review. Results: Increase in fundamental frequency is the most commonly reported effect of stress in well-controlled trials. The trend, however, is not universal. A reduction in noise as reflected by the diminished vocal jitter is reported, but less frequently. Conclusions: Stress types, gender, and individual differences in baseline autonomic tone may explain the primarily equivocal findings of effects of stressor exposure or perceived stress on voice; and as such, the article concludes with a discussion of directions for future study.
Article
Full-text available
The rapid growth in the use of smartphones has opened a new world of opportunities for use in behavioral health care. Mobile phone software applications (apps) are available for a variety of useful tasks to include symptom assessment, psychoeducation, resource location, and tracking of treatment progress. The latest two-way communication functionality of smartphones also brings new capabilities for telemental health. There is very little information available, however, regarding the integration of smartphone and other mobile technology into care. In this paper, we provide an overview of smartphone use in behavioral health care and discuss options for integrating mobile technology into clinical practice. We also discuss limitations, practical issues, and recommendations. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
In this article, the authors provide an empirical analysis of the obtrusiveness of and participants' compliance with a relatively new psychological ambulatory assessment method, called the electronically activated recorder or EAR. The EAR is a modified portable audio-recorder that periodically records snippets of ambient sounds from participants' daily environments. In tracking moment-to-mo-ment ambient sounds, the EAR yields an acoustic log of a person's day as it unfolds. As a naturalistic observation sampling method, it provides an observer's account of daily life and is optimized for the assessment of audible aspects of participants' naturally-occurring social behaviors and interactions. Measures of self-reported and behaviorally-assessed EAR obtrusiveness and compliance were analyzed in two samples. After an initial 2-h period of relative obtrusiveness, participants habituated to wearing the EAR and perceived it as fairly unobtrusive both in a short-term (2 days, N = 96) and a longer-term (10–11 days, N = 11) monitoring. Compliance with the method was high both during the short-term and longer-term monitoring. Somewhat reduced compliance was identified over the weekend; this effect appears to be specific to student populations. Important privacy and data confidentiality considerations around the EAR method are discussed.
Article
Full-text available
PurposeResponses provided by unmotivated survey participants in a careless, haphazard, or random fashion can threaten the quality of data in psychological and organizational research. The purpose of this study was to summarize existing approaches to detect insufficient effort responding (IER) to low-stakes surveys and to comprehensively evaluate these approaches. Design/Methodology/ApproachIn an experiment (Study 1) and a nonexperimental survey (Study 2), 725 undergraduates responded to a personality survey online. FindingsStudy 1 examined the presentation of warnings to respondents as a means of deterrence and showed the relative effectiveness of four indices for detecting IE responses: response time, long string, psychometric antonyms, and individual reliability coefficients. Study 2 demonstrated that the detection indices measured the same underlying construct and showed the improvement of psychometric properties (item interrelatedness, facet dimensionality, and factor structure) after removing IE respondents identified by each index. Three approaches (response time, psychometric antonyms, and individual reliability) with high specificity and moderate sensitivity were recommended as candidates for future application in survey research. ImplicationsThe identification of effective IER indices may help researchers ensure the quality of their low-stake survey data. Originality/valueThis study is a first attempt to comprehensively evaluate IER detection methods using both experimental and nonexperimental designs. Results from both studies corroborated each other in suggesting the three more effective approaches. This study also provided convergent validity evidence regarding various indices for IER. KeywordsCareless responding–Random responding–Inconsistent responding–Online surveys–Data screening
Article
Full-text available
The design and development of wearable biosensor systems for health monitoring has garnered lots of attention in the scientific community and the industry during the last years. Mainly motivated by increasing healthcare costs and propelled by recent technological advances in miniature biosensing devices, smart textiles, microelectronics, and wireless communications, the continuous advance of wearable sensor-based systems will potentially transform the future of healthcare by enabling proactive personal health management and ubiquitous monitoring of a patient's health condition. These systems can comprise various types of small physiological sensors, transmission modules and processing capabilities, and can thus facilitate low-cost wearable unobtrusive solutions for continuous all-day and any-place health, mental and activity status monitoring. This paper attempts to comprehensively review the current research and development on wearable biosensor systems for health monitoring. A variety of system implementations are compared in an approach to identify the technological shortcomings of the current state-of-the-art in wearable biosensor solutions. An emphasis is given to multiparameter physiological sensing system designs, providing reliable vital signs measurements and incorporating real-time decision support for early detection of symptoms or context awareness. In order to evaluate the maturity level of the top current achievements in wearable health-monitoring systems, a set of significant features, that best describe the functionality and the characteristics of the systems, has been selected to derive a thorough study. The aim of this survey is not to criticize, but to serve as a reference for researchers and developers in this scientific area and to provide direction for future research improvements.
Conference Paper
Full-text available
We present a real-time system for detecting facial action units and inferring emotional states from head and shoulder gestures and facial expressions. The dynamic system uses three levels of inference on progressively longer time scales. Firstly, facial action units and head orientation are identified from 22 feature points and Gabor filters. Secondly, Hidden Markov Models are used to classify sequences of actions into head and shoulder gestures. Finally, a multi level Dynamic Bayesian Network is used to model the unfolding emotional state based on probabilities of different gestures. The most probable state over a given video clip is chosen as the label for that clip. The average F1 score for 12 action units (AUs 1, 2, 4, 6, 7, 10, 12, 15, 17, 18, 25, 26), labelled on a frame by frame basis, was 0.461. The average classification rate for five emotional states (anger, fear, joy, relief, sadness) was 0.440. Sadness had the greatest rate, 0.64, anger the smallest, 0.11.
Conference Paper
Full-text available
Abstract This paper describes a promising ,sleepiness detection ap- proach based on prosodic and spectral speech characteristics and illustrates the validity of this method,by briefly discussing results from a sleep deprivation study (N=20). We conducted a within-subject sleep deprivation design (8.00 p.m to 4.00 a.m). During the night of sleep deprivation, a standardized self- report scale was used every hour just before the recordings to determine the sleepiness state. The speech material consisted ofsimulated,driver assistance system phrases. In order to in- vestigate sleepiness induced speech changes, a standard set of spectral and prosodic features were ,extracted from the sen- tences. After forward selection and a PCA were employed,on the feature space in an attempt to prune redundant dimensions, LDA- and ANN-based classification models,were trained. The best level-0 model (RA15, LDA) offers a mean accuracy rate of 80.0% for the two-class problem. Using an ensemble classi- fication strategy (majority voting ,as meta-classifier) we achieved a accuracy rate of 88.2%. Index Terms: spectral features, sleepiness detection, driver
Article
Background Heart rate (HR) monitors are valuable devices for fitness-orientated individuals. There has been a vast influx of optical sensing blood flow monitors claiming to provide accurate HR during physical activities. These monitors are worn on the arm and wrist to detect HR with photoplethysmography (PPG) techniques. Little is known about the validity of these wearable activity trackers. Aim Validate the Scosche Rhythm (SR), Mio Alpha (MA), Fitbit Charge HR (FH), Basis Peak (BP), Microsoft Band (MB), and TomTom Runner Cardio (TT) wireless HR monitors. Methods 50 volunteers (males: n=32, age 19–43 years; females: n=18, age 19–38 years) participated. All monitors were worn simultaneously in a randomised configuration. The Polar RS400 HR chest strap was the criterion measure. A treadmill protocol of one 30 min bout of continuous walking and running at 3.2, 4.8, 6.4, 8.0, and 9.6 km/h (5 min at each protocol speed) with HR manually recorded every minute was completed. Results For group comparisons, the mean absolute percentage error values were: 3.3%, 3.6%, 4.0%, 4.6%, 4.8% and 6.2% for TT, BP, RH, MA, MB and FH, respectively. Pearson product-moment correlation coefficient (r) was observed: r=0.959 (TT), r=0.956 (MB), r=0.954 (BP), r=0.933 (FH), r=0.930 (RH) and r=0.929 (MA). Results from 95% equivalency testing showed monitors were found to be equivalent to those of the criterion HR (±10% equivalence zone: 98.15–119.96). Conclusions The results demonstrate that the wearable activity trackers provide an accurate measurement of HR during walking and running activities.
Article
Work on voice sciences over recent decades has led to a proliferation of acoustic parameters that are used quite selectively and are not always extracted in a similar fashion. With many independent teams working in different research areas, shared standards become an essential safeguard to ensure compliance with state-of-the-art methods allowing appropriate comparison of results across studies and potential integration and combination of extraction and recognition systems. In this paper we propose a basic standard acoustic parameter set for various areas of automatic voice analysis, such as paralinguistic or clinical speech analysis. In contrast to a large brute-force parameter set, we present a minimalistic set of voice parameters here. These were selected based on a) their potential to index affective physiological changes in voice production, b) their proven value in former studies as well as their automatic extractability, and c) their theoretical significance. The set is intended to provide a common baseline for evaluation of future research and eliminate differences caused by varying parameter sets or even different implementations of the same parameters. Our implementation is publicly available with the openSMILE toolkit. Comparative evaluations of the proposed feature set and large baseline feature sets of INTERSPEECH challenges show a high performance of the proposed set in relation to its size.
Article
Hierarchy is a reality of group life, for humans and for most other group-living species. However, there remains considerable debate about whether and when hierarchy can promote group performance and member satisfaction. We suggest that progress in this debate has been hampered by a lack of clarity about hierarchy and how to conceptualize it. Whereas prevailing conceptualizations of hierarchy in the group and organization literature have focused on inequality in member power or status (i.e., centralization or steepness), we build on the ethological and social network traditions to advance a view of hierarchy as cascading relations of dyadic influence (i.e., acyclicity). We suggest that hierarchy thus conceptualized is more likely to capture the functional benefits of hierarchy, whereas hierarchy as inequality is more likely to be dysfunctional. In a study of 75 teams drawn from a range of industries, we show that whereas acyclicity in influence relations reduces conflict and thereby enhances both group performance and member satisfaction, centralization and steepness have negative effects on conflict, performance, and satisfaction, particularly in groups that perform complex tasks. The theory and results of this study can help to clarify and advance research on the functions and dysfunctions of hierarchy in task groups.
Article
Remote detection of cognitive load has many powerful applications, such as measuring stress in the workplace. Cognitive tasks have an impact on breathing and heart rate variability (HRV). We show that changes in physiological parameters during cognitive stress can be captured remotely (at a distance of 3m) using a digital camera. A study (n=10) was conducted with participants at rest and under cognitive stress. A novel five band digital camera was used to capture videos of the face of the participant. Significantly higher normalized low frequency HRV components and breathing rates were measured in the stress condition when compared to the rest condition. Heart rates were not significantly different between the two conditions. We built a person-independent classifier to predict cognitive stress based on the remotely detected physiological parameters (heart rate, breathing rate and heart rate variability). The accuracy of the model was 85% (35% greater than chance).
Article
Spoken dialogue systems are increasingly being used to facilitate and enhance human communication. While these interactive systems can process the linguistic aspects of human communication, they are not yet capable of processing the complex dynamics involved in social interaction, such as the adaptation on the part of interlocutors. Providing interactive systems with the capacity to process and exhibit this accommodation could however improve their efficiency and make machines more socially-competent interactants. At present, no automatic system is available to process prosodic accommodation, nor do any clear measures exist that quantify its dynamic manifestation. While it can be observed to be a monotonically manifest property, it is our hypotheses that it evolves dynamically with functional social aspects. In this paper, we propose an automatic system for its measurement and the capture of its dynamic manifestation. We investigate the evolution of prosodic accommodation in 41 Japanese dyadic telephone conversations and discuss its manifestation in relation to its functions in social interaction. Overall, our study shows that prosodic accommodation changes dynamically over the course of a conversation and across conversations, and that these dynamics inform about the naturalness of the conversation flow, the speakers’ degree of involvement and their affinity in the conversation.
Article
Automatically detecting human social intentions and attitudes from spoken conversation is an important task for speech processing and social computing. We describe a system for detecting interpersonal stance: whether a speaker is flirtatious, friendly, awkward, or assertive. We make use of a new spoken corpus of over 1000 4-min speed-dates. Participants rated themselves and their interlocutors for these interpersonal stances, allowing us to build detectors for style both as interpreted by the speaker and as perceived by the hearer. We use lexical, prosodic, and dialog features in an SVM classifier to detect very clear styles (the strongest 10% in each stance) with up to 75% accuracy on previously seen speakers (50% baseline) and up to 59% accuracy on new speakers (48% baseline). A feature analysis suggests that flirtation is marked by joint focus on the woman as a target of the conversation, awkwardness by decreased speaker involvement, and friendliness by a conversational style including other-directed laughter and appreciations. Our work has implications for our understanding of interpersonal stance, their linguistic expression, and their automatic extraction.
Article
The hindsight bias is the tendency for people with outcome knowledge to believe falsely that they would have predicted the reported outcome of an event. This article reviews empirical research relevant to hindsight phenomena. The influence of outcome knowledge, termed creeping determinism, was initially hypothesized to result from the immediate and automatic integration of the outcome into a person's knowledge of an event. Later research has identified at least 4 plausible, general strategies for responding to hindsight questions. These explanations postulate that outcome information affects the selection of evidence to make a judgment, the evidence evaluation, the manner in which evidence is integrated, or the response generation process. It is also likely, in some situations, that a combination of 2 or more of these mechanisms produces the observed hindsight effects. We provide an interpretation of the creeping determinism hypothesis in terms of inferences made to reevaluate case-specific evidence once the relevant outcome is known and conclude that it is the most common mechanism underlying observed hindsight effects. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
A variety of theoretical frameworks predict the resemblance of behaviors between two people engaged in communication, in the form of coordination, mimicry, or alignment. However, little is known about the time course of the behavior matching, even though there is evidence that dyads synchronize oscillatory motions (e.g., postural sway). This study examined the temporal structure of nonoscillatory actions-language, facial, and gestural behaviors-produced during a route communication task. The focus was the temporal relationship between matching behaviors in the interlocutors (e.g., facial behavior in one interlocutor vs. the same facial behavior in the other interlocutor). Cross-recurrence analysis revealed that within each category tested (language, facial, gestural), interlocutors synchronized matching behaviors, at temporal lags short enough to provide imitation of one interlocutor by the other, from one conversational turn to the next. Both social and cognitive variables predicted the degree of temporal organization. These findings suggest that the temporal structure of matching behaviors provides low-level and low-cost resources for human interaction.
Article
Sociometric badges are wearable electronic badges ca-pable of automatically measuring the amount of face-to-face interaction, conversational time, prosodic style, phys-ical proximity to other people, and physical activity levels, using social signals derived from vocal features, body mo-tion, and relative location. We present the prior and current state-of-the art in this area of wearable computing and pro-pose several applications that haven't been fully exploited.
Article
This study examines the structure of 105 work groups and management teams to address the question of whether conflict can be beneficial. Multiple methods were used to examine the effects of conflict on both individual- and group-level variables to provide a more refined model of intragroup conflict. Results show that whether conflict was beneficial depended on the type of conflict and the structure of the group in terms of task type, task interdependence, and group norms. Relationship and task conflicts were negatively associated with individuals' satisfaction, liking of other group members, and intent to remain in the group. In groups performing very routine tasks, disagreements about the task were detrimental to group functioning. In contrast, in groups performing nonroutine tasks, disagreements about the tasks did not have a detrimental effect, and in some cases, such disagreements were actually beneficial. Contrary to expectations, norms encouraging open discussion of conflict were not always advantageous. The results suggest that while such norms were associated with an increase in the number and intensity of relationship conflicts, they did not increase members' ability to deal with the conflicts constructively. The model developed here contributes to an integrated perspective on organizational conflict.
Article
In the framework of a major longitudinal study of depressive disorders, 11 female and 5 male depressives were audio-video-recorded while speaking with clinical interviewers. For selected utterances during depressed and recovered mood states several voice and speech parameters were obtained, using digital analysis techniques. As predicted, the results showed that an increase in speech rate and a decrease in pause duration are powerful indicators of mood improvement in the course of therapy (remission from depressive state). In female but not in male patients, a decrease in minimum fundamental frequency of the voice predicted mood improvement. These effects are discussed with respect to neurophysiological, cognitive, and emotional factors that have been suggested in the literature as possible causes for the patterns of motor expression observed in depressives. The data also point to the urgent need to systematically study gender differences in depressive speech behavior.
Article
Being in the numeric minority (e.g., being a solo woman in a group of men) influences how well a person performs within a work group. But being the solo member is only one way in which people can be atypical in a group; a person can also represent a social or demographic category that has not typically been associated with the task that the group is working on. Using a design with four categories of group composition (minority, balanced, majority, homogeneous) and two categories of tasks (sex-typical, sex-atypical) we found that the sex composition of the group interacted with the sex typicality of the task to influence both positive deferrals by group members and individual performance in groups. But, rather than consistently reducing performance as prior research has suggested, being numerically atypical enhanced individual performance when the task was typical for that person’s sex. Further, positive deferrals mediated between the interaction of numeric composition and task typicality in influencing individual performance suggesting that both majority group members and the solo member affect one another’s performance in groups. We conclude by discussing why understanding the interplay between these two sources of stereotyping, numeric composition and task typicality, is important for understanding the social nature of individual performance in groups.
Article
This research focuses on the development and validation of an instrument to measure the privacy concerns of individuals who use the Internet and two antecedents, perceived vulnerability and perceived ability to control information. The results of exploratory factor analysis support the validity of the measures developed. In addition, the regression analysis results of a model including the three constructs provide strong support for the relationship between perceived vulnerability and privacy concerns, but only moderate support for the relationship between perceived ability to control information and privacy concerns. The latter unexpected results suggest that the relationship among the hypothesized antecedents and privacy concerns may be one that is more complex than is captured in the hypothesized model, in light of the strong theoretical justification for the role of information control in the extant literature on information privacy.
Article
Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2007. Includes bibliographical references (p. 137-144). We present the design, implementation and deployment of a wearable computing research platform for measuring and analyzing human behavior in a variety of settings and applications. We propose the use of wearable sociometric badges capable of automatically measuring the amount of face-to-face interaction, conversational time, physical proximity to other people, and physical activity levels using social signals derived from vocal features, body motion, and relative location to capture individual and collective patterns of behavior. Our goal is to be able to understand how patterns of behavior shape individuals and organizations. We attempt to use on-body sensors in large groups of people for extended periods of time in naturalistic settings for the purpose of identifying, measuring, and quantifying social interactions, information flow, and organizational dynamics. We deployed this research platform in a group of 22 employees working in a real organization over a period of one month. Using these automatic measurements we were able to predict employees' self-assessment of productivity, job satisfaction, and their own perception of group interaction quality. An initial exploratory data analysis indicates that it is possible to automatically capture patterns of behavior using this wearable platform. by Daniel Olguín Olguín. S.M.
Article
The chameleon effect refers to nonconscious mimicry of the postures, mannerisms, facial expressions, and other behaviors of one's interaction partners, such that one's behavior passively and unintentionally changes to match that of others in one's current social environment. The authors suggest that the mechanism involved is the perception-behavior link, the recently documented finding (e.g., J. A. Bargh, M. Chen, & L. Burrows, 1996) that the mere perception of another's behavior automatically increases the likelihood of engaging in that behavior oneself. Experiment 1 showed that the motor behavior of participants unintentionally matched that of strangers with whom they worked on a task. Experiment 2 had confederates mimic the posture and movements of participants and showed that mimicry facilitates the smoothness of interactions and increases liking between interaction partners. Experiment 3 showed that dispositionally empathic individuals exhibit the chameleon effect to a greater extent than do other people.
Developing with Kinect for Windows
  • Microsoft
A step towards automatic applicant selection: Predicting job performance based on applicant nonverbal interview behavior
  • M Schmid Mast
  • D Frauendorfer
  • L S Nguyen
  • D Gatica-Perez
  • T Choudhury
  • J.-M Odobez
ResearchKit and CareKit
  • Apple
Audacity. Retrieved from www
  • Audacity
Intel RealSense technology
  • Intel
Affective Signals. Retrieved from www.affective-signals
  • Affective Signals