Rosalind W Picard

Singapore-MIT Alliance

Approximately 400,000 youth in the US are living in foster care due to experiences with abuse or neglect at home. For multiple reasons, these youth often don't receive adequate social support from those around them. Despite technology's potential, very little work has explored how these tools can provide more support to foster-involved youth. To be...

This paper explores the design, development and evaluation of a digital platform that aims to assist young people who have experienced trauma in understanding and expressing their emotions and fostering social connections. Integrating principles from expressive arts and narrative-based therapies, we collaborate with lived experts to iteratively des...

Objective This study aimed to assess whether population‐level patterns in seizure occurrence previously observed in self‐reported diaries, medical records, and electroencephalographic recordings were also present in tonic–clonic seizure (TCS) diaries produced via the combined input of a US Food and Drug Administration‐cleared wristband with an arti...

Objective: Work is ongoing to advance seizure forecasting, but the performance metrics used to evaluate model effectiveness can sometimes lead to misleading outcomes. For example, some metrics improve when tested on patients with a particular range of seizure frequencies (SF). This study illustrates the connection between SF and metrics. Additional...

Introduction Respiratory diseases such as chronic obstructive pulmonary disease, obstructive sleep apnea syndrome, and COVID-19 may cause a decrease in arterial oxygen saturation (SaO2). The continuous monitoring of oxygen levels may be beneficial for the early detection of hypoxemia and timely intervention. Wearable non-invasive pulse oximetry dev...

Rationale: Epilepsy patients are advised to maintain an adequate amount of sleep every day to reduce seizure risk. Despite the long-lasting notion that sleep deficiency may induce seizures, existing literature has not clarified the relationship between them with objective evidence. We performed a large-scale longitudinal observational ambulatory s...

Background Epilepsy patients are advised to maintain adequate sleep to reduce seizure risk. However, few studies have clarified the relationship between sleep parameters and seizure risk. We performed a longitudinal observational ambulatory study to investigate sleep parameters and the occurrence of seizures using wearable monitoring. Materials &...

Nonverbal vocalizations, such as sighs, grunts, and yells, are informative expressions within typical verbal speech. Likewise, individuals who produce 0–10 spoken words or word approximations (“minimally speaking” individuals) convey rich affective and communicative information through nonverbal vocalizations even without verbal speech. Yet, despit...

As we move closer to real-world social AI systems, AI agents must be able to deal with multiparty (group) conversations. Recognizing and interpreting multiparty behaviors is challenging, as the system must recognize individual behavioral cues, deal with the complexity of multiple streams of data from multiple people, and recognize the subtle contin...

Suicide is among the most devastating problems facing clinicians, who currently have limited tools to predict and prevent suicidal behavior. Here we report on real-time, continuous smartphone and sensor data collected before, during, and after a suicide attempt made by a patient during a psychiatric inpatient hospitalization. We observed elevated a...

Nonverbal vocalizations from non- and minimally speaking individuals (mv*) convey important communicative and affective information. While nonverbal vocalizations that occur amidst typical speech and infant vocalizations have been studied extensively in the literature, there is limited prior work on vocalizations by mv* individuals. Our work is amo...

Despite the increase in awareness and support for mental health, college students’ mental health is reported to decline every year in many countries. Several interactive technologies for mental health have been proposed and are aiming to make therapeutic service more accessible, but most of them only provide one-way passive contents for their users...

To assist people in practicing mindful breathing and regulate their perceived workload while not disturbing the ongoing foreground task during daily routines, we developed a mobile and personalizable pneumatic-haptic feedback device that provides programmable subtle tactile feedback. The device consists of three soft inflatable actuators embedded w...

Background Predicting future states of psychopathology such as depressive episodes has been a hallmark initiative in mental health research. Dynamical systems theory has proposed that rises in certain ‘early warning signals’ (EWSs) in time-series data (e.g. auto-correlation, temporal variance, network connectivity) may precede impending changes in...

Background: Using machine learning to combine wrist accelerometer (ACM) and electrodermal activity (EDA) has been shown effective to detect primarily and secondarily generalized tonic-clonic seizures, here termed as convulsive seizures (CS). A prospective study was conducted for the FDA clearance of an ACM and EDA-based CS-detection device based on...

Pre-emptive recognition of the ethical implications of study design and algorithm choices in artificial intelligence (AI) research is an important but challenging process. AI applications have begun to transition from a promising future to clinical reality in neurology. As the clinical management of neurology is often concerned with discrete, often...

Explaining deep learning model inferences is a promising venue for scientific understanding, improving safety, uncovering hidden biases, evaluating fairness, and beyond, as argued by many scholars. One of the principal benefits of counterfactual explanations is allowing users to explore "what-if" scenarios through what does not and cannot exist in...

Standard machine learning approaches require centralizing the users' data in one computer or a shared database, which raises data privacy and confidentiality concerns. Therefore, limiting central access is important, especially in healthcare settings, where data regulations are strict. A potential approach to tackling this is Federated Learning (FL...

Objective: Digital monitoring technologies (e.g., smart-phones and wearable devices) provide unprecedented opportunities to study potentially harmful behaviors such as suicide, violence, and alcohol/substance use in real-time. The use of these new technologies has the potential to significantly advance the understanding, prediction, and prevention...

Background: While preliminary evidence suggests that sensors may be employed to detect presence of low mood it is still unclear whether they can be leveraged for measuring depression symptom severity. This study evaluates the feasibility and performance of assessing depressive symptom severity by using behavioral and physiological features obtained...

The purpose of the present work is to examine, on a clinically diverse population of older adults (N=46) sleeping at home, the performance of two actigraphy-based sleep tracking algorithms (i.e., Actigraphy-based Sleep algorithm, ACT-S1 and Sadeh’s algorithm) compared to manually scored electroencephalography-based PSG (PSG-EEG). ACT-S1 allows for...

Faster, more reliable, and comfortably wearable personal devices are producing data from biosensors on an unprecedented scale. Combined with context and analytics, these signals hold great promise to advance neuroscience via real-world data. Johnson and Picard discuss wearable technology broadly and provide specific examples of activity patterns fr...

2020 Owner/Author. Driving can occupy a considerable part of our daily lives and is often associated with high levels of stress. Motivated by the effectiveness of controlled breathing, this work studies the potential use of breathing interventions while driving to help manage stress. In particular, we implemented and evaluated a closed-loop system...

We examine the problem of forecasting tomorrow morning's three self-reported levels (on scales from 0 to 100) of stressed-calm, sad-happy, and sick-healthy based on physiological data (skin conductance, skin temperature, and acceleration) from a sensor worn on the wrist from 10am-5pm today. We train automated forecasting regression algorithms using...

Driving can occupy a large portion of daily life and often can elicit negative emotional states like anger or stress, which can significantly impact road safety and long-term human health. In recent decades, the arrival of new tools to help recognize human affect has inspired increasing interest in how to develop emotion-aware systems for cars. To...

2020 Owner/Author. A Facebook Messenger chatbot, Sunny, was designed and deployed to promote positive social connections and enhance psychological wellbeing. A 10-day study was conducted with three pre-existing social groups of four members each in control (n=12) and experimental groups (n=12). Both groups completed initial assessments and daily re...

2020 Owner/Author. Current augmentative communication technology has had limited success conveying the needs of some individuals with minimally verbal autism spectrum disorder (mvASD). Primary caregivers report being able to better understand these individuals' non-traditional utterances than those less familiar with the individual such as teachers...

2020 IEEE. We examine the problem of forecasting tomorrow morning's three self-reported levels (on scales from 0 to 100) of stressed-calm, sad-happy, and sick-healthy based on physiological data (skin conductance, skin temperature, and acceleration) from a sensor worn on the wrist from 10am-5pm today. We train automated forecasting regression algor...

Study objectives: Sleep regularity, in addition to duration and timing, is predictive of daily variations in well-being. One possible contributor to changes in these sleep dimensions are early morning scheduled events. We applied a composite metric - the Composite Phase Deviation (CPD) - to assess mistiming and irregularity of both sleep and event...

The "fixed and frozen" AI-based GTCS detection algorithm complies with FDA requirements (lower bound of CI for PPA>70% and FAR<2) for both pediatric and adult populations. • The FAR for pediatric is significantly (p-value<0.01) higher FAR, most likely because children were more active in the EMU. • During rest, the overall FAR drops dramatically to...

Supporting model interpretability for complex phenomena where annotators can legitimately disagree, such as emotion recognition, is a challenging machine learning task. In this work, we show that explicitly quantifying the uncertainty in such settings has interpretability benefits. We use a simple modification of a classical network inference using...

We engineered an interactive music system that influences a user's breathing rate to induce a relaxation response. This system generates ambient music containing periodic shifts in loudness that are determined by the user's own breathing patterns. We evaluated the efficacy of this music intervention for participants who were engaged in an attention...

Emotional contagion in online social networks has been of great interest over the past years. Previous studies have focused mainly on finding evidence of affect contagion in homophilic atmospheres. However, these studies have overlooked users' awareness of the sentiments they share and consume online. In this paper, we present an experiment with Tw...

Accurately forecasting well-being may enable people to make desirable behavioral changes that could improve their future well-being. In this paper, we evaluate how well an automated model can forecast the next-day's well-being (specifically focusing on stress, health, and happiness) from static models (support vector machine and logistic regression...

2019 IEEE. Accurately forecasting well-being may enable people to make desirable behavioral changes that could improve their future well-being. In this paper, we evaluate how well an automated model can forecast the next-day's well-being (specifically focusing on stress, health, and happiness) from static models (support vector machine and logistic...

Human behavior expression and experience are inherently multi-modal, and characterized by vast individual and contextual heterogeneity. To achieve meaningful human-computer and human-robot interactions, multi-modal models of the users states (e.g., engagement) are therefore needed. Most of the existing works that try to build classifiers for the us...

We introduce a novel personalized Gaussian Process Experts (pGPE) model for predicting per-subject ADAS-Cog13 cognitive scores -- a significant predictor of Alzheimer's Disease (AD) in the cognitive domain -- over the future 6, 12, 18, and 24 months. We start by training a population-level model using multi-modal data from previously seen subjects...

Life stress is a well-established risk factor for a variety of mental and physical health problems, including anxiety disorders, depression, chronic pain, heart disease, asthma, autoimmune diseases, and neurodegenerative disorders. The purpose of this article is to describe emerging approaches for assessing stress using speech, which we do by revie...

Wearable automated seizure detection devices offer a high potential to improve seizure management, through continuous ambulatory monitoring, accurate seizure counts, and real-time alerts for prompt intervention. More importantly, these devices can be a life-saving help for people with a higher risk of sudden unexpected death in epilepsy (SUDEP), es...

2019 IEEE. We engineered an interactive music system that influences a user's breathing rate to induce a relaxation response. This system generates ambient music containing periodic shifts in loudness that are determined by the user's own breathing patterns. We evaluated the efficacy of this music intervention for participants who were engaged in a...

2019 IEEE. Perceiving users' engagement accurately is important for technologies that need to respond to learners in a natural and intelligent way. In this paper, we address the problem of automated estimation of engagement from videos of child-robot interactions recorded in unconstrained environments (kindergartens). This is challenging due to div...

2019 IEEE. Accurately forecasting stress may enable people to make behavioral changes that could improve their future health. For example, accurate stress forecasting might inspire people to make changes to their schedule to get more sleep or exercise, in order to reduce excessive stress tomorrow night. In this paper, we examine how accurately the...

2019 Neural information processing systems foundation. All rights reserved. Building an open-domain conversational agent is a challenging problem. Current evaluation methods, mostly post-hoc judgments of static conversation, do not capture conversation quality in a realistic interactive context. In this paper, we investigate interactive human evalu...

Rationale: Embrace is the first FDA approved wrist-worn device combining Accelerometer (ACM) and Electrodermal Activity (EDA) to detect and alert to generalized tonic-clonic seizures (GTCSs). A multi-site clinical evaluation of the machine learning classifier embedded in Embrace validated 94.5% sensitivity of the GTCS detector with a false alarm ra...

Rationale There is a growing need for non-EEG, wearable devices to automatically monitor epileptic seizures in ambulatory settings. Efforts to date have focused on the development of devices to detect convulsive seizures, such as the FDA-approved smart watch Embrace by Empatica (Onorati et al, Epilepsia 2017, 58, 11). However, Embrace was not desig...

This work studies the feasibility of using low-cost motion sensors to provide opportunistic heart rate assessments from ballistocardiographic signals during restful periods of daily life. Three wearable devices were used to capture peripheral motions at specific body locations (head, wrist and trouser pocket) of 15 participants during five regular...

Objective: Non-contact physiological measurement is a growing research area that allows capturing vital signs such as heart rate (HR) and breathing rate (BR) comfortably and unobtrusively with remote devices. However, most of the approaches work only in bright environments in which subtle photoplethysmographic and ballistocardiographic signals can...

2018 IEEE. Many children on autism spectrum have atypical behavioral expressions of engagement compared to their neu-rotypical peers. In this paper, we investigate the performance of deep learning models in the task of automated engagement estimation from face images of children with autism. Specifically, we use the video data of 30 children with d...

This work explores the use of pressure-sensing to capture cues of the stress of smartphone users while typing. In a controlled laboratory study, 11 participants were asked to write about a recent stressful and relaxing experience in counterbalanced order. Preliminary results show a significant positive correlation between the increase in typing pre...

2018 Copyright is held by the owner/author(s). This work explores the use of pressure-sensing to capture cues of the stress of smartphone users while typing. In a controlled laboratory study, 11 participants were asked to write about a recent stressful and relaxing experience in counterbalanced order. Preliminary results show a significant positive...

Unobtrusive and accurate ambulatory methods are needed to monitor long-term sleep patterns for improving health. Previously developed ambulatory sleep detection methods rely either in whole or in part on self-reported diary data as ground truth, which is a problem, since people often do not fill them out accurately. This paper presents an algorithm...

Purpose The Embrace smartwatch combines accelerometer and electrodermal activity sensors to alert to generalized tonic-clonic seizures (GTCSs). While a multi-site clinical study validated 94.5% sensitivity (Se) of the GTCS detector with a false alarm rate (FAR) typically less than 0.2/day (Onorati et al, Epilepsia 2017), longitudinal analysis of Em...

We designed, developed, and evaluated a novel system, QuantifyMe, for novice self-experimenters to conduct proper-methodology single-case self-experiments in an automated and scientific manner using their smartphones. In this work we evaluate its use with four different kinds of personalized investigations, examining how variables such as sleep dur...

We investigate the personalization of deep convolutional neural networks for facial expression analysis from still images. While prior work has focused on population-based (“one-size-fits-all”) approaches, we formulate and construct personalized models via a mixture of experts and supervised domain adaptation approach, showing that it improves grea...

This exploratory study examined the effects of varying g-forces, including feelings of weightlessness, on an individual's physiology during parabolic flight. Specifically, we collected heart rate, accelerometer, and skin conductance measurements from 16 flyers aboard a parabolic flight using wearable, wireless sensors. The biosignals were then corr...

This exploratory study examined the effects of varying g-forces, including feelings of weightlessness, on an individual's physiology during parabolic flight. Specifically, we collected heart rate, accelerometer, and skin conductance measurements from 16 flyers aboard a parabolic flight using wearable, wireless sensors. The biosignals were then corr...

We investigate the personalization of deep convolutional neural networks for facial expression analysis from still images. While prior work has focused on population-based (“one-size-fits-all”) approaches, we formulate and construct personalized models via a mixture of experts and supervised domain adaptation approach, showing that it improves grea...

Objective: Non-contact physiological measurement is a growing research area that allows capturing vital signs such as heart rate (HR) and breathing rate (BR) comfortably and unobtrusively with remote devices. However, most of the approaches work only in bright environments in which subtle photoplethysmographic and ballistocardiographic signals can...

Introduction We recently developed the Composite Phase Deviation (CPD), a metric that quantifies mistiming of sleep along two dimensions: (i) relative to an individual’s habitual sleep timing, and (ii) relative to their sleep timing on the previous day. Given recent findings that sleep regularity can be predictive of mood, we examined whether CPD i...

We appreciate the comments from Dr. Stewart on our article,1 and agree that obstructive apnea and laryngospasm are potentially relevant factors in some sudden unexpected death in epilepsy (SUDEP) cases. We also agree that laryngospasm can occur during seizures. Well-documented cases have been rarely reported in witnessed SUDEPs.

Smartphones and wearable sensors have enabled unprecedented data collection, with many products now providing feedback to users about recommended step counts or sleep durations. However, these recommendations do not provide personalized insights that have been shown to be best suited for a specific individual. A scientific way to find individualize...

Objective Common data elements (CDEs) are currently unavailable for mobile health (mHealth) in epilepsy devices and related applications. As a result, despite expansive growth of new digital services for people with epilepsy, information collected is often not interoperable or directly comparable. We aim to correct this problem through development...

We describe an initiative to bring mental health researchers, computer scientists, human-computer interaction researchers, and other communities together to address the challenges of the global mental ill health epidemic. Two face-to-face events and one special issue of the Journal of Medical Internet Research were organized. The works presented in...

ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2018. We designed, developed, and evaluated a novel system, QuantifyMe, for novice self-experimenters to conduct proper-methodology single-case self-experiments in an automated and scientific manner using their smartphones. In this work we evaluate its use w...

Clinically validated automated mobile seizure detection devices have been poorly tested in outpatient settings to date. Detectors relying on motion-related signals to recognize convulsive seizures (CSs) are likely to yield higher false alarm rates (FAR) in real life than in a clinical environment, due to a wider range of possible physical activitie...