Mary Czerwinski

Microsoft · Visualization and Interaction for Business and Entertainment Research Group (VIBE)

Behind the scenes of maintaining the safety of technology products from harmful and illegal digital content lies unrecognized human labor. The recent rise in the use of generative AI technologies and the accelerating demands to meet responsible AI (RAI) aims necessitates an increased focus on the labor behind such efforts in the age of AI. This stu...

As concerns about employee burnout and skilled staff shortages in cybersecurity grow, our study aims to better understand the contributing factors to burnout in this field. Utilizing a mixed-methods approach, we analyze self-reported job and personal characteristics, along with digital activity data from 35 incident responders, identifying several...

The global COVID-19 pandemic has spurred on new collaborations across borders, and emphasized the importance of supporting wellbeing in the workplace, whether that workplace is hybrid, remote, or in-person. Work in CSCW, HCI, and organizational psychology has explored how people come to understand their wellbeing at work, and the role of identity,...

Work-nonwork balance is an important aspect of workplace well-being with associations to improved physical and mental health, job performance, and quality of life. However, realizing work-nonwork balance goals is challenging due to competing demands and limited resources within organizational and interpersonal contexts. These challenges are compoun...

The future of work is rapidly changing, with remote and hybrid settings blurring the boundaries between professional and personal life. To understand how work rhythms vary across different work settings, we conducted a month-long study of 65 software developers, collecting anonymized computer activity data as well as daily ratings for perceived str...

Background Integrating stress-reduction interventions into the workplace may improve the health and well-being of employees, and there is an opportunity to leverage ubiquitous everyday work technologies to understand dynamic work contexts and facilitate stress reduction wherever work happens. Sensing-powered just-in-time adaptive intervention (JITA...

BACKGROUND Integrating stress-reduction interventions into the workplace may improve the health and well-being of employees, and there is an opportunity to leverage ubiquitous everyday work technologies to understand dynamic work contexts and facilitate stress reduction wherever work happens. Sensing-powered just-in-time adaptive intervention (JITA...

Workplace stress has been increasing in recent decades and has worsened by the unique demands imposed by COVID-19 and the new remote/hybrid work settings. High-stress working conditions can be detrimental to the health and wellness of workers and can lead to significant business costs in terms of productivity loss and medical expenses. An essential...

Emotion dysregulation frequently co-occurs with chronic pain, which in turn leads to heightened emotional and physical suffering. This cycle of association has prompted a recommendation for psychological treatment of chronic pain to target mechanisms for emotion regulation. The current trial addressed this need by investigating a new internet-deliv...

Artificial agents’ smiling behaviour is likely to influence their likeability and the quality of user experience. While studies of human interaction highlight the importance of smile dynamics, this feature is often lacking in artificial agents, presenting a design opportunity. We developed a virtual motivational therapist with four smiling behaviou...

How might computing support us in becoming our better, more emotionally resilient selves? We explore this in an interview with the team from Microsoft Research's Human Understanding and Empathy group.

The Future Conversations workshop at CHIIR'21 looked to the future of search, recommendation, and information interaction to ask: where are the opportunities for conversational interactions? What do we need to do to get there? Furthermore, who stands to benefit? The workshop was hands-on and interactive. Rather than a series of technical talks, we...

In the past year, have you found yourself under stress? Have you ever wished for help coping? Imagine if, throughout the pandemic, you'd had a virtual therapist powered by an artificial intelligence (AI) system, an entity that empathized with you and gradually got to know your moods and behaviors. Therapy is just one area where we think an AI syste...

Introduction Difficulties in emotional regulation are key to the development and maintenance of chronic pain. Recent evidence shows internet-delivered dialectic behaviour therapy (iDBT) skills training can reduce emotional dysregulation and pain intensity. However, further studies are needed to provide more definitive evidence regarding the efficac...

Facial action unit recognition has many applications from market research to psychotherapy and from image captioning to entertainment. Despite its recent progress, deployment of these models has been impeded due to their limited generalization to unseen people and demographics. This work conducts an in-depth analysis of performance across several d...

We now turn to understanding the impact that COVID-19 had on the personal productivity and well-being of information workers as their work practices were impacted by remote work. This chapter overviews people's productivity, satisfaction, and work patterns, and shows that the challenges and benefits of remote work are closely linked. Looking forwar...

Virtual meetings are critical for remote work because of the need for synchronous collaboration in the absence of in-person interactions. In-meeting multitasking is closely linked to people's productivity and wellbeing. However, we currently have limited understanding of multitasking in remote meetings and its potential impact. In this paper, we pr...

The ability to monitor audience reactions is critical when delivering presentations. However, current videoconferencing platforms offer limited solutions to support this. This work leverages recent advances in affect sensing to capture and facilitate communication of relevant audience signals. Using an exploratory survey (N = 175), we assessed the...

Mobile mental health interventions have the potential to reduce barriers and increase engagement in psychotherapy. However, most current tools fail to meet evidence-based principles. In this paper, we describe data-driven design implications for translating evidence-based interventions into mobile apps. To develop these design implications, we anal...

The ability to regulate our emotions plays an important role in our psychological and physical health. Regulating emotions influences how and when emotions are expressed. We performed a large, longitudinal observational study to investigate the effect of emotion regulation ability on expressed affect. We found that expression of negative affect inc...

Circadian rhythms determine daily sleep cycles, mood, and cognition. Depending on an individual's circadian preference, or chronotype (i.e.,"early birds" and "night owls"), the rhythms shift earlier or later in the day. Early birds experience circadian arousal peaks earlier in the morning than night owls. Prior work has shown that individuals are m...

Advances in machine intelligence have enabled conversational interfaces that have the potential to radically change the way humans interact with machines. However, even with the progress in the abilities of these agents, there remain critical gaps in their capacity for natural interactions. One limitation is that the agents are often monotonic in b...

A natural conversational interface that allows longitudinal symptom tracking would be extremely valuable in health/wellness applications. However, the task of designing emotionally-aware agents for behavior change is still poorly understood. In this paper, we present the design and evaluation of an emotion-aware chatbot that conducts experience sam...

We present an end-to-end voice-based conversational agent that is able to engage in naturalistic multi-turn dialogue and align with the interlocutor's conversational style. The system uses a series of deep neural network components for speech recognition, dialogue generation, prosodic analysis and speech synthesis to generate language and prosodic...

We present an end-to-end voice-based conversational agent that is able to engage in naturalistic multi-turn dialogue and align with the interlocutor's conversational style. The system uses a series of deep neural network components for speech recognition, dialogue generation, prosodic analysis and speech synthesis to generate language and prosodic...

Humans use a host of signals to infer the emotional state of others. In general, computer systems that leverage signals from multiple modalities will be more robust and accurate in the same task. We present a multimodal affect and context sensing platform. The system is composed of video, audio and application analysis pipelines that leverage ubiqu...

The delivery of mental health interventions via ubiquitous devices has shown much promise. A natural conversational interface that allows longitudinal symptom tracking and appropriate just-in-time interventions would be extremely valuable. However, the task of designing emotionally-aware agents is still poorly understood. Furthermore, the feasibili...

TODAY, PEOPLE INCREASINGLY rely on computer agents in their lives, from searching for information, to chatting with a bot, to performing everyday tasks. These agent-based systems are our first forays into a world in which machines will assist, teach, counsel, care for, and entertain us. While one could imagine purely rational agents in these roles,...

How can we create technologies to help us reflect on and change our behavior, improving our health and overall wellbeing? In this talk, I will briefly describe the last several years of work our research team has been doing in this area. We have developed wearable technology to help families manage tense situations with their children, mobile phone...

Information workers are experiencing ever-increasing online distractions in the workplace, and software to block distractions is becoming more popular. We conducted an exploratory field study with 32 information workers in their workplace using software to block online distractions for one week. We discovered that with online distractions blocked,...

Mental health disorders are a leading cause of disability worldwide. Although evidence-based psychotherapy is effective, engagement from such programs can be low. Mobile apps have the potential to help engage and support people in their therapy. We developed Pocket Skills, a mobile web app based on Dialectical Behavior Therapy (DBT). Pocket Skills...

Emotions play a major role in how interpersonal conflicts unfold. Although several strategies and technological approaches have been proposed for emotion regulation, they often require conscious attention and effort. This often limits their efficacy in practice. In this paper, we propose a different approach inspired by self-perception theory: noti...

The World Health Organization predicts that by the year 2030, mental illnesses will be the leading disease burden globally. Advances in technology create opportunities for close collaboration between computation and mental health researchers. The intersection between ubiquitous computing and sensing, social media, data analytics and emerging techno...

Analysis of casual chit-chat indicates that differences in conversational style---the way things are said---can significantly impact a participants» impressions of the conversation and of each other. However, prior work has not systematically analyzed how important style is in task-oriented, information-seeking exchanges of the sort we might have w...

Information workers are faced with ever-increasing online distractions in the workplace. Website blockers are one solution toward preventing unwanted distractions. We conducted an in situ field study with 32 information workers in their workplace to test if the use of blocking software can increase focus and productivity by preventing non-work-rela...

A formative study of a mobile application for introducing skills related to Dialectical Behavior Therapy (DBT) is reviewed with an eye toward next iteration design. DBT is considered the gold standard for the treatment of suicide and borderline personality disorder, among other complex behavioral disorders. A multidisciplinary design team took lice...

Designing virtual personal assistants that are able to engage users in an interaction have been a challenge for HCI researchers for the past 20 years. In this work we investigated how a set of vocal characteristics known as “conversational style” could play role in engaging users in an interaction with a virtual agent. We also examined whether the...

Given the importance of creativity for both personal and societal achievements, there have been consistent efforts to stimulate creative ability. But an important environmental factor --- blue (i.e., short wavelength) light --- has been relatively unexplored to date. Blue light improves a number of cognitive processes (e.g., attention, working memo...

Affective computing systems blend technical and social elements and present challenges for designing notifications. As the number of applications expand that utilize notifications, we need to consider that competing for attention is a concern. With respect to promoting positive health behaviors, notifications need to be adapted to times when a user...

Interactivity is a unique forum of the ACM CHI conference that showcases more than 50 cutting-edge demonstrations and inspirational interactive installations from the diverse group of researchers, designers, and artists that make up the CHI community. These works were chosen to engage audiences intellectually, imaginatively, and physically to creat...

In HCI research, attention has focused on understanding external influences on workplace multitasking. We explore instead how multitasking might be influenced by individual factors: personality, stress, and sleep. Forty information workers' online activity was tracked over two work weeks. The median duration of online screen focus was 40 seconds. T...

While email provides numerous benefits in the workplace, it is unclear how patterns of email use might affect key workplace indicators of productivity and stress. We investigate how three email use patterns: duration, interruption habit, and batching, relate to perceived workplace productivity and stress. We tracked email usage with computer loggin...

We present Tattio, a fabrication process that draws from current body decoration processes (i.e., jewelry-like metallic temporary tattoos) for the creation of on-skin technology. The fabrication process generates functional components such as NFC tags, circuitry, to Thermochromic tattoos, while maintaining the aesthetics and user experience of exis...

Various wearable sensors capturing body vibration, jaw movement, hand gesture, etc., have shown promise in detecting when one is currently eating. However, based on existing literature and user surveys conducted in this study, we argue that a Just-in-Time eating intervention, triggered upon detecting a current eating event, is sub-optimal. An eatin...

Positive wellbeing in the workplace is tied to better health. However, lack of wellbeing in the workplace is a serious problem in the U.S, is rising continually, and can lead to poor health conditions. In this study we investigate factors that might be associated with workplace wellbeing. We report on an in situ study in the workplace of 40 informa...

In this paper the authors present the BioCrystal – a biofeedback device that uses physiological data to evaluate user's affective states in real-time and signals the states via an ambient display. The authors evaluated the BioCrystal during a 2-week, in situ multi-method study during which ten users collected over 115 hours of usable data. Users' c...

A common assumption in studies of interruptions is that one is focused in an activity and then distracted by other stimuli. We take the reverse perspective and examine whether one might first be in an attentional state that makes one susceptible to communications typically associated with distraction. We explore the confluence of multitasking and w...

We present “Lightwear”, a series of garment-based, lightweight, light-emitting wearables designed to administer light therapy for on-the-go treatment of Seasonal Affective Disorder (SAD). Bright Light Therapy (BLT) has been used to treat SAD for more than 25 years. While light boxes continue to serve as the predominant method of treatment, it often...

We present SWARM, a wearable affective technology designed to help a user to reflect on their own emotional state, modify their affect, and interpret the emotional states of others. SWARM aims for a universal design (inclusive of people with various disabilities), with a focus on modular actuation components to accommodate users' sensory capabiliti...

Recording of various events in a video format that facilitates viewing and selective editing are provided. The video can be presented in a wiki-format that allows a multitude of subsequent users to add, modify and/or delete content to the original recorded event or a revision of that event. As edits and annotations are applied, either automatically...

A single device that receives communications intended for a user having multiple contact addresses associated with different user roles is provided. The single device can receive communications intended for multiple telephone numbers, email aliases, screen names, aliases, other means of contact, or combinations thereof. Since a user engaged in a pa...

This special issue turns the spotlight on business intelligence (BI) as an area of inquiry and explores beyond the current standard practices. The articles in the issue describe the processes that practitioners currently follow in this area and how new BI techniques and capabilities will help users understand and act on widely disparate types of da...

Parenting is always demanding, but families coping with neurodevelopmental disorders, such as ADHD, experience unique challenges. To address these challenges, research in the area of Parental Behavioral Therapy is accelerating. This type of therapy focuses on behavioral strategies that, if practiced regularly, can have a positive impact on the chil...

A collaborative phone-based file exchange technique that wirelessly synchronizes a user's work context with his or her mobile phone and makes this context readily available and sharable in collaborative situations. When the user is away from their desk, collaboration with others is possible by sharing/downloading between the mobile phone and a pair...

Introduction: This article is part of the focus theme of Methods of Information in Medicine on "Pervasive Intelligent Technologies for Health". Background: Effective nonverbal communication between patients and clinicians fosters both the delivery of empathic patient-centered care and positive patient outcomes. Although nonverbal skill training...

Stress is considered to be a modern day " global epidemic"; so given the widespread nature of this problem, it would be beneficial if solutions that help people to learn how to cope better with stress were scalable beyond what individual or group therapies can provide today. Therefore, in this work, we study the potential of smart-phones as a perva...

A user entering information in an application might desire functionality not supported by that application. Based on an input or a subset of the input, a functionality associated with a different application can be automatically applied within a current application. Sometimes, a particular application or mode of communication can be used to send in...

While distractions using digital media have received attention in HCI, understanding engagement in workplace activities has been little explored. We logged digital activity and continually probed perspectives of 32 information workers for five days in situ to understand how attentional states change with context. We present a framework of how engag...

Recognizing when computer users are stressed can help reduce their frustration and prevent a large variety of negative health conditions associated with chronic stress. However, measuring stress non-invasively and continuously at work remains an open challenge. This work explores the possibility of using a pressure-sensitive keyboard and a capaciti...

What makes people feel happy, engaged and challenged at work? We conducted an in situ study of Facebook and face-to-face interactions examining how they influence people's mood in the workplace. Thirty-two participants in an organization were each observed for five days in their natural work environment using automated data capture and experience s...

Described are systems and user interfaces for facilitating asynchronous communication with video threads. Implementations may enable members of a team to have threaded view of video messages, from which they can capture video, record their screen, and send, receive, and reply to a video message. The screen recording feature enables team members to...

Techniques for visualizing sets are described. Arbitrary subsets of data elements are represented by corresponding graphic lines. The data elements in a set are connected up sequentially by a corresponding graphic line, the graphic line passing through each data element once with minimal or no self-overlapping. The graphic lines may be curved, for...

Over-eating is often the result of consuming food for reasons other than needing fuel and nutrients: we eat for pleasure, from habit, for emotional support. Such non-homeostatic eating is an established contributor to overeating towards overweight or obese weight gain that in turn is correlated with numerous non-communicable diseases such as Type I...

Behavior modification in health is difficult, as habitual behaviors are extremely well-learned, by definition. This research is focused on building a persuasive system for behavior modification around emotional eating. In this paper, we make strides towards building a just-in-time support system for emotional eating in three user studies. The first two...

This paper presents novel human computation experiments geared towards uncovering the structure of affect. Using Mechanical Turk workers across 2 separate studies, we empirically verified some of the popular beliefs about the structure of affect, but also provide some new evidence. We replicate and reve