Conference Paper

Objective assessment of depressive symptoms with machine learning and wearable sensors data

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... We chose a E4 wearable band from Empatica [18] as our sensing device. It has photoplethysmogram (PPG), accelerometer (ACC), electrodermal activity (EDA), and temperature sensor within a device so that they can provide variety of biometric data for analysis.The other work used E4 wearable bands and smartphones to predict depression [19,20]. Those previous works explored features such as EDA sensor data, sleep characteristics, physiological data, and smartphone usage. ...
... Paper [34,35] showed EDA and heart rate can be used for stress. The other work [19,20] is similar to our work in that they try to predict depression with the band. In the paper [32], the authors classified the arousal levels using EDA. ...
... However, our goal is to monitor depressed mood. In the paper [19], the author showed the efficacy of machine learning techniques for predicting depression with EDA and acceleration. They combined the E4 wearable band sensor data and the smartphone information. ...
Article
Full-text available
Depression in the elderly is an important social issue considering the population aging of the world. In particular, elderly living alone who has narrowed social relationship due to bereavement and retirement are more prone to be depressed. Long-term depressed mood can be a precursor to eventual depression as a disease. Our goal is how to predict the depressed mood of single household elderly from unobtrusive monitoring of their daily life. We have selected a wearable band with multiple sensors for monitoring elderly people. Depression questionnaire has been surveyed periodically to be used as the labels. Instead of working with depression patients, we recruited 14 single household elderly people from a nearby community. The wearable band provided daily activity and biometric data for 71 days. From the data, we generate a depressed mood prediction model. Multiple features from the collected sensor data are exploited for model generation. One general model is generated to be used as the baseline for the initial model deployment. Personal models are also generated for model refinement. The general model has a high recall of 80% in an MLP model. Individual models achieved an average recall of 82.7%. In this study, we have demonstrated that we can generate depressed mood prediction models with data collected from real daily living. Our work has shown the feasibility of using a wearable band as an unobtrusive depression monitoring sensor even for elderly people.
... Emotions during healthcare are mainly triggered by invasive or frightening treatment experiences and pain during surgical procedures [6,7]. Technological development of the equipment and systems for monitoring and recording psychophysiological functions enables continuous observation of the changes in responding to a situation, making it possible to supplement the psychological data obtained through self-descriptive questionnaires with physiological ones [8][9][10]. ...
... The effectiveness of using machine learning methods and data from physiological signals (EDA), compared to the subjective assessment of depression was tested [8,42]. A multimodal approach was using similar bio-signals like in the presented article (BVP, EDA, EMG, temperature, and respiration) to develop a classifier to recognize negative emotions [43]. ...
... Ghandeharioun et al., in their analyses, by using different machine-learning methods -Random Forest and adaptive boosting (AdaBoost) obtained a low RMSE (Root Mean Square Error) value. It concluded that depression symptoms could be measured continuously using data from sensors [8]. An analogy can be discovered between the studies mentioned by Ghandeharioun and those presented in this paper. ...
Article
Full-text available
Invasive or uncomfortable procedures especially during healthcare trigger emotions. Technological development of the equipment and systems for monitoring and recording psychophysiological functions enables continuous observation of changes to a situation responding to a situation. The presented study aimed to focus on the analysis of the individual’s affective state. The results reflect the excitation expressed by the subjects’ statements collected with psychological questionnaires. The research group consisted of 49 participants (22 women and 25 men). The measurement protocol included acquiring the electrodermal activity signal, cardiac signals, and accelerometric signals in three axes. Subjective measurements were acquired for affective state using the JAWS questionnaires, for cognitive skills the DST, and for verbal fluency the VFT. The physiological and psychological data were subjected to statistical analysis and then to a machine learning process using different features selection methods (JMI or PCA). The highest accuracy of the kNN classifier was achieved in combination with the JMI method (81.63%) concerning the division complying with the JAWS test results. The classification sensitivity and specificity were 85.71% and 71.43%.
... World health organization (WHO) estimated that more than 300 million people are suffering from depressive illness and an increase of more than 18% between 2005 and 2015 [1] [2]. However, the diagnosis, monitoring, tracking of depression symptoms and therapy still rely mainly on old practices; which are developed approximately more than fifty years ago [3]. ...
... The significance of using microservices is the reusing of samilar microservices data for many services and applications, complexity driven in data formatting and translations, loose coupling, lightweight, plug & play, and strong modularization. In the articles [3] [16], the sensors data, mobile phone data, electrodermal activity data, electroencephalography activity data, sleep behavior data, location change behavior data, SMS, phone calls data, and questionnaire data have been used to detect and monitor depression symptoms. ...
Conference Paper
Full-text available
Internet of Things (IoT) has the most significance in the intelligent healthcare service provision because of the wealth of data from disparate sources. By using the applications and tools of the IoT and web of data we can collect large of data and provide better care and monitoring services to the patients while reducing the cost and time. Depressive disorder (DD) is a common disease worldwide and it is estimated by the World Health Organization that in 2020, depression will be the largest medical burden on the society and government expenditures. To reduce the depression period is the important aspect for the practitioner psychiatric by using cognitive behavior therapy (CBT). To reduce cost and time we proposed an IoT based CBT system based on Web of Objects (WoO) data-driven microservices. We used data-driven microservices for the depressive person data accumulation, representation and assistance services. We also used semantically interoperable data representation microservices for IoT based DD symptoms monitoring. Our proposed model and use case scenario significance show that this approach is useful for the DD assistance and care.
... EDA, recorded by measuring the conductivity of human skin, provides a good prediction for mental states. Setz et al. used a wearable EDA device to discriminate stress from the cognitive load, with an accuracy larger than 80% [141,142]. Ghandeharioun et al. performed an experiment that predicts the Hamilton depression rating scale using sensor data captured by E4 wearable wristbands and embedded sensors in an Android phone [142]. By applying machine learning techniques on the users' objective data such as EDA, sleep behaviors, phone usage, etc., the experiment found that poor mental health is caused by irregular sleep, less motion, and higher asymmetry of EDA between right and left wrists. ...
... Setz et al. used a wearable EDA device to discriminate stress from the cognitive load, with an accuracy larger than 80% [141,142]. Ghandeharioun et al. performed an experiment that predicts the Hamilton depression rating scale using sensor data captured by E4 wearable wristbands and embedded sensors in an Android phone [142]. By applying machine learning techniques on the users' objective data such as EDA, sleep behaviors, phone usage, etc., the experiment found that poor mental health is caused by irregular sleep, less motion, and higher asymmetry of EDA between right and left wrists. ...
Article
Wearable technologies provide a non-invasive way to monitor user's activity, identity, and health in real-time, which have attracted tremendous interests from both academia and industry. Due to constraints in form factor and power consumption, the sensing capabilities and functionalities of the wearables are usually limited by the available sensors. In the past decade, researchers have committed to realizing the sensing capability of multiple sensors via the signal from one sensor, which expanded the functionalities and sensing domains of traditional sensors. For the first time, we defined such sensing approach as “cross-sensing” and provided a comprehensive review on the cross-sensing towards wearable applications (i.e., human-machine interface, health services, and security). Specifically, this paper summarized the applied signal processing and machine learning algorithms, and discussed how cross-sensing would affect the development and innovation trends of wearable electronics.
... [93] showed that social media engagement was predictive of performance on some cognitive attributes such as working memory, attentional control, and others. [92] used [101], [105], [106] [41], [107], [108] Alcohol [109], [110] [111] Tobacco [112] [113] Physical Activity [114], [115] [116] Sleep [117]- [120] [120] ...
... [104] showed that there is a relation between possitive affect and conscientiousness. Anxiety is predicted by [101], [105]. Techniques that predict stress and anxiety based on ECG monitoring have also been proposed [106]. ...
Article
Assessment of individuals' job performance, personalized health and psychometric measures are domains where data-driven ubiquitous computing will have a profound impact in the near future. Existing work in these domains focus on techniques that use data extracted from questionnaires, sensors (wearable, computer, etc.), or other traits to assess well-being and cognitive attributes of individuals. However, these techniques can neither predict individuals' well-being and psychological traits in a global manner nor consider the challenges associated with processing the often incomplete and noisy data available. In this paper, we create a benchmark for the predictive analysis of individuals from a perspective that integrates physical and physiological behavior, psychological states and traits, and job performance. We develop a novel data mining framework that can extract meaningful predictors from noisy and incomplete data derived from wearable, mobile and social media sensors to predict nineteen constructs based on twelve standardized and well-validated tests. The framework can be used to build a predictive model of outcomes of interest. We validate the framework using data from 757 knowledge workers in organizations across the United States with varied work roles. Our framework and resulting model provides the first benchmark that combines these various instrument-derived variables in a single framework to understand people's behavior. The results show that our framework is reliable and capable of predicting our chosen variables better than the baselines when prediction includes the noisy and incomplete data.
... [93] showed that social media engagement was predictive of performance on some cognitive attributes such as working memory, attentional control, and others. [92] used [101], [105], [106] [41], [107], [108] Alcohol [109], [110] [111] Tobacco [112] [113] Physical Activity [114], [115] [116] Sleep [117]- [120] [120] ...
... [104] showed that there is a relation between possitive affect and conscientiousness. Anxiety is predicted by [101], [105]. Techniques that predict stress and anxiety based on ECG monitoring have also been proposed [106]. ...
Preprint
Full-text available
Assessment of job performance, personalized health and psychometric measures are domains where data-driven and ubiquitous computing exhibits the potential of a profound impact in the future. Existing techniques use data extracted from questionnaires, sensors (wearable, computer, etc.), or other traits, to assess well-being and cognitive attributes of individuals. However, these techniques can neither predict individual's well-being and psychological traits in a global manner nor consider the challenges associated to processing the data available, that is incomplete and noisy. In this paper, we create a benchmark for predictive analysis of individuals from a perspective that integrates: physical and physiological behavior, psychological states and traits, and job performance. We design data mining techniques as benchmark and uses real noisy and incomplete data derived from wearable sensors to predict 19 constructs based on 12 standardized well-validated tests. The study included 757 participants who were knowledge workers in organizations across the USA with varied work roles. We developed a data mining framework to extract the meaningful predictors for each of the 19 variables under consideration. Our model is the first benchmark that combines these various instrument-derived variables in a single framework to understand people's behavior by leveraging real uncurated data from wearable, mobile, and social media sources. We verify our approach experimentally using the data obtained from our longitudinal study. The results show that our framework is consistently reliable and capable of predicting the variables under study better than the baselines when prediction is restricted to the noisy, incomplete data.
... They studied only depression in a limited number of 28 subjects and used EEG in a controlled environment. Ghandeharioun et al. [17] conducted a study to predict depressive symptoms based on the Hamilton Depression Rating Scale (HDRS) by machine learning techniques applied to data acquired from sensors on E4 wearable wristbands and Android phones. They used additional input data, including electrodermal activity (EDA), location changes, phone-based communication and phone usage patterns. ...
... They used additional input data including electrodermal activity (EDA), location changes, phone-based communication and phone usage patterns. Ghandeharioun et al. [17] and Rykov et al. [18] achieved a successful work in developing a classification model for depression using digital biomarkers such as steps, heart rate, energy expenditure and sleep data from consumer-grade wearable sensors. Their work is limited to finding depression only, and, to the best of authors' knowledge, there have been no other machine learning-based studies to identify the comorbid problem of geriatric depression and anxiety using a low-cost activity tracker. ...
Article
Full-text available
The identification of geriatric depression and anxiety is important because such conditions are the most common comorbid mood problems that occur in older adults. The goal of this study was to build a machine learning framework that identifies geriatric mood disorders of depression and anxiety using low-cost activity trackers and minimal geriatric assessment scales. We collected activity tracking data from 352 mild cognitive impairment patients, from 60 to 90 in age, by having them wear activity trackers on their wrist for more than a month. We then extracted the features of 24-h activity rhythms and sleep patterns from the time-series activity tracking data. To increase the accuracy, we designed a novel method to incorporate additional features from questionnaire-based assessments of the geriatric depression scale and geriatric anxiety inventory into the activity tracking features. In the multi-label classification, we applied the binary relevance method to develop two single-label classifiers for depression and anxiety. The best hyper-parameters of classification algorithms for each label were selected by comparing the classification performance. We finally selected the combination of classifiers for depression and anxiety with the lowest Hamming loss as a multi-label classifier. This study successfully demonstrated the possibility of identifying geriatric depression and anxiety using low-cost activity trackers and minimal geriatric assessment scales for use in the real fields.
... Digital traces (data from social media and other digital platforms) present a promising new approach to studying adolescents' well-being that is fast, inexpensive, nonintrusive, and with high resolution. Previous literature has demonstrated that mental health conditions, such as depression and anxiety, can be predicted from mobile sensor data [5][6][7][8], social media engagement [9], language [9][10][11][12], and photos [13]. An increasing number of studies have analyzed emotive trends based on social media data [14][15]. ...
... A team of linguists adjusted it for Russian language use. 5 ...
Preprint
Digital traces are often used as a substitute for survey data. However, it is unclear whether and how digital traces actually correspond to the survey-based traits they purport to measure. This paper examines correlations between self-reports and digital trace proxies of depression, anxiety, mood, social integration and sleep among high school students. The study is based on a small but rich multilayer data set (N = 144). The data set contains mood and sleep measures, assessed daily over a 4-month period, along with survey measures at two points in time and information about online activity from VK, the most popular social networking site in Russia. Our analysis indicates that 1) the sentiments expressed in social media posts are correlated with depression; namely, adolescents with more severe symptoms of depression write more negative posts, 2) late-night posting indicates less sleep and poorer sleep quality, and 3) students who were nominated less often as somebody's friend in the survey have fewer friends on VK and their posts receive fewer "likes." However, these correlations are generally weak. These results demonstrate that digital traces can serve as useful supplements to, rather than substitutes for, survey data in studies on adolescents' well-being. These estimates of correlations between survey and digital trace data could provide useful guidelines for future research on the topic.
... EDA, recorded by measuring the conductivity of human skin, provides a good prediction for mental states. Setz et al. used a wearable EDA device to discriminate stress from the cognitive load, with an accuracy larger than 80% [141,142]. Ghandeharioun et al. performed an experiment that predicts the Hamilton depression rating scale using sensor data captured by E4 wearable wristbands and embedded sensors in an Android phone [142]. By applying machine learning techniques on the users' objective data such as EDA, sleep behaviors, phone usage, etc., the experiment found that poor mental health is caused by irregular sleep, less motion, and higher asymmetry of EDA between right and left wrists. ...
... Setz et al. used a wearable EDA device to discriminate stress from the cognitive load, with an accuracy larger than 80% [141,142]. Ghandeharioun et al. performed an experiment that predicts the Hamilton depression rating scale using sensor data captured by E4 wearable wristbands and embedded sensors in an Android phone [142]. By applying machine learning techniques on the users' objective data such as EDA, sleep behaviors, phone usage, etc., the experiment found that poor mental health is caused by irregular sleep, less motion, and higher asymmetry of EDA between right and left wrists. ...
Article
Wearable technologies provide a non-invasive way to monitor user's activity, identity, and health in real-time, which have attracted tremendous interests from both academia and industry. Due to constraints in form factor and power consumption, the sensing capabilities and functionalities of the wearables are usually limited by the available sensors. In the past decade, researchers have committed to realizing the sensing capability of multiple sensors via the signal from one sensor, which expanded the functionalities and sensing domains of traditional sensors. For the first time, we defined such sensing approach as “cross-sensing” and provided a comprehensive review on the cross-sensing towards wearable applications (i.e., human-machine interface, health services, and security). Specifically, this paper summarized the applied signal processing and machine learning algorithms, and discussed how cross-sensing would affect the development and innovation trends of wearable electronics.
... Outcomes captured by providers, patients, and digital technologies for high-burden health conditions or risk factors[45][46][47][48][49] Table 2. Proposed and potential digital outcome measure tools for inflammatory bowel disease [24, 37, 42, 50-63] Technology International Consortium for Health Outcomes Measurement measures by domain ...
Article
Full-text available
Improving clinical outcomes remains the gold standard in advancing healthcare. Focusing on outcomes holds the potential to unite all clinical stakeholders including payers, industry, providers, and patients. Yet, the dominant ways in which outcomes are captured, provider-collected or patient-reported, have significant limitations. The emerging field of biosensors and wearables, which aims to capture many types of health data, holds promise to specifically capture outcomes while complementing existing outcome collection methods. A digital outcome measure, unlike a traditional provider-collected or patient-reported outcome measure, depends less on active patient or provider participation. Thus, digital outcome measures may be more amenable to standardization as well as greater collection consistency, frequency, and accuracy.
... However, some implementations of linear regression in R throw warnings and give worse results when the number of predictors is much higher than the number of observations. Example implementations of ridge regression in psychological research are provided by a range of authors (Dana & Dawes, 2004;Eichstaedt et al., 2015;Ghandeharioun et al., 2017). ...
Article
Full-text available
Machine learning methods for prediction and pattern detection are increasingly prevalent in psychological research. We provide an introductory overview of machine learning, its applications, and describe how to implement models for research. We review fundamental concepts of machine learning, such as prediction accuracy and out‐of‐sample evaluation, and summarize standard prediction algorithms including linear regressions, ridge regressions, decision trees, and random forests (plus additional algorithms in the supplementary materials). We demonstrate each method with examples and annotated R code, and discuss best practices for determining sample sizes; comparing model performances; tuning prediction models; preregistering prediction models; and reporting results. Finally, we discuss the value of machine learning methods in maintaining psychology’s status as a predictive science.
... For future research, we suggest applying more objective measures. For example, widely used electronic devices can be used for measuring symptoms of depression and anxiety (Ghandeharioun et al. 2017), biomarkers can be used in the assessment of excessive alcohol and drug use (Peterson 2004;Kwako et al. 2018). Similar to Milosevic (2011) and Moon et al. (2017), objective measures for impulsivity assessment could be applied more often. ...
Article
Full-text available
Gamblers are a heterogenous group in terms of the presence of comorbid psychopathology, maladaptive personality traits, and motivation to gamble. The Pathways Model, the most promising comprehensive framework to explain this heterogeneity, classifies gamblers into three subtypes. The aim of this review was to determine whether or not subtyping of gamblers based on the Pathways Model of problem and disordered gambling is valid. A literature review was conducted using the following online databases: Academic Search Complete, PubMed, Web of Science, and PsychINFO. Studies were selected or excluded based on meeting predetermined criteria. Fourteen studies examining subtyping of gamblers based on the Pathways Model were reviewed and evaluated. Results suggest that in the adult population there are three subtypes of gamblers that largely coincide with the subtypes defined in the Pathways Model. Of these, the emotionally vulnerable subtype is the most problematic and inconsistent. In contrast, for adolescents, at least four gambler subtypes have been identified. The extant literature on subtyping of gamblers suffers from some severe limitations. Further research is required to fully validate the Pathways Model.
... All features were preprocessed, transformed and calculated for four 6-h intervals and for daily aggregates (over the course of 24-h). We built upon and expanded our previous work (37) and encoded a comprehensive list of physiological and behavioral features including EDA, motion, sleep, phone usage, call and messaging behavior, app usage, and location change patterns (see comprehensive list in Supplementary Table 1). Preprocessing of EDA, motion, and sleep features was improved relative to our previous work by adding EDA features calculated during the time when there is no motion (identified by the accelerometer sensor) and by adding normalized EDA features (see additional information in Section A1 in Supplementary Material). ...
Article
Full-text available
Background: While preliminary evidence suggests that sensors may be employed to detect presence of low mood it is still unclear whether they can be leveraged for measuring depression symptom severity. This study evaluates the feasibility and performance of assessing depressive symptom severity by using behavioral and physiological features obtained from wristband and smartphone sensors. Method: Participants were thirty-one individuals with Major Depressive Disorder (MDD). The protocol included 8 weeks of behavioral and physiological monitoring through smartphone and wristband sensors and six in-person clinical interviews during which depression was assessed with the 17-item Hamilton Depression Rating Scale (HDRS-17). Results: Participants wore the right and left wrist sensors 92 and 94% of the time respectively. Three machine-learning models estimating depressive symptom severity were developed–one combining features from smartphone and wearable sensors, one including only features from the smartphones, and one including features from wrist sensors–and evaluated in two different scenarios. Correlations between the models' estimate of HDRS scores and clinician-rated HDRS ranged from moderate to high (0.46 [CI: 0.42, 0.74] to 0.7 [CI: 0.66, 0.74]) and had moderate accuracy with Mean Absolute Error ranging between 3.88 ± 0.18 and 4.74 ± 1.24. The time-split scenario of the model including only features from the smartphones performed the best. The ten most predictive features in the model combining physiological and mobile features were related to mobile phone engagement, activity level, skin conductance, and heart rate variability. Conclusion: Monitoring of MDD patients through smartphones and wrist sensors following a clinician-rated HDRS assessment is feasible and may provide an estimate of changes in depressive symptom severity. Future studies should further examine the best features to estimate depressive symptoms and strategies to further enhance accuracy.
... Recent work in affective computing has begun to explore the potential in predicting mood and emotion from mobile data. In particular, they have found that typing patterns [2,12,15,34], self-reporting apps [28], and wearable sensors [11,13,25] are particularly predictive for affect. In addition, multimodal modeling of multiple sensors (e.g. ...
Preprint
Mental health conditions remain under-diagnosed even in countries with common access to advanced medical care. The ability to accurately and efficiently predict mood from easily collectible data has several important implications towards the early detection and intervention of mental health disorders. One promising data source to help monitor human behavior is from daily smartphone usage. However, care must be taken to summarize behaviors without identifying the user through personal (e.g., personally identifiable information) or protected attributes (e.g., race, gender). In this paper, we study behavioral markers or daily mood using a recent dataset of mobile behaviors from high-risk adolescent populations. Using computational models, we find that multimodal modeling of both text and app usage features is highly predictive of daily mood over each modality alone. Furthermore, we evaluate approaches that reliably obfuscate user identity while remaining predictive of daily mood. By combining multimodal representations with privacy-preserving learning, we are able to push forward the performance-privacy frontier as compared to unimodal approaches.
... Empatica E4 is a wearable device [9] that can capture motion (through accelerometer (ACC)), and physiological indicators (blood volume pulse (BVP), electrodermal activity (EDA), and skin temperature (TEMP)). Accuracy, reliability and validity of the sensor data from Empatica E4 device have been confirmed for stress detection [22,28], assessment of depressive symptoms [12] and for regular daily activities [29]. In our study, the E4 was removed at night before the participants sleep by the nurse for charging. ...
Preprint
Full-text available
People Living with Dementia (PLwD) often exhibit behavioral and psychological symptoms, such as episodes of agitation and aggression. Agitated behaviour in PLwD causes distress and increases the risk of injury to both the patients and the caregivers. In this paper, we present the use of a multi-modal wearable device that captures motion and physiological indicators to detect agitation in PLwD. We hypothesize that combining multi-modal sensor data will be more effective to identify agitation in PLwD in comparison to a single sensor. This paper presents the results of a unique pilot study to collect motion and physiological data from PLwD admitted to a Specialized Dementia Unit. The classification results on 14 participants from 481 days of data collected from PLwD show strong evidence to support our hypothesis and highlight the importance of using multi-modal sensor data for detecting agitation events in this population.
... Ubiquitous data has been proved to be useful in detecting mental health conditions. Mobile sensor data such as GPS logs [41,42]; electrodermal activity; and sleep behavior, motion, and phone use patterns [43,44] have been applied in investigating depressive symptoms. Zaman et al [45] found that individual private Google Search histories can be used to detect low self-esteem conditions among college students. ...
Article
Background Depression and anxiety disorders among the global population have worsened during the COVID-19 pandemic. Yet, current methods for screening these two issues rely on in-person interviews, which can be expensive, time-consuming, and blocked by social stigma and quarantines. Meanwhile, how individuals engage with online platforms such as Google Search and YouTube has undergone drastic shifts due to COVID-19 and subsequent lockdowns. Such ubiquitous daily behaviors on online platforms have the potential to capture and correlate with clinically alarming deteriorations in depression and anxiety profiles of users in a noninvasive manner. Objective The goal of this study is to examine, among college students in the United States, the relationships of deteriorating depression and anxiety conditions with the changes in user behaviors when engaging with Google Search and YouTube during COVID-19. Methods This study recruited a cohort of undergraduate students (N=49) from a US college campus during January 2020 (prior to the pandemic) and measured the anxiety and depression levels of each participant. The anxiety level was assessed via the General Anxiety Disorder-7 (GAD-7). The depression level was assessed via the Patient Health Questionnaire-9 (PHQ-9). This study followed up with the same cohort during May 2020 (during the pandemic), and the anxiety and depression levels were assessed again. The longitudinal Google Search and YouTube history data of all participants were anonymized and collected. From individual-level Google Search and YouTube histories, we developed 5 features that can quantify shifts in online behaviors during the pandemic. We then assessed the correlations of deteriorating depression and anxiety profiles with each of these features. We finally demonstrated the feasibility of using the proposed features to build predictive machine learning models. Results Of the 49 participants, 49% (n=24) of them reported an increase in the PHQ-9 depression scores; 53% (n=26) of them reported an increase in the GAD-7 anxiety scores. The results showed that a number of online behavior features were significantly correlated with deteriorations in the PHQ-9 scores (r ranging between –0.37 and 0.75, all P values less than or equal to .03) and the GAD-7 scores (r ranging between –0.47 and 0.74, all P values less than or equal to .03). Simple machine learning models were shown to be useful in predicting the change in anxiety and depression scores (mean squared error ranging between 2.37 and 4.22, R2 ranging between 0.68 and 0.84) with the proposed features. Conclusions The results suggested that deteriorating depression and anxiety conditions have strong correlations with behavioral changes in Google Search and YouTube use during the COVID-19 pandemic. Though further studies are required, our results demonstrate the feasibility of using pervasive online data to establish noninvasive surveillance systems for mental health conditions that bypasses many disadvantages of existing screening methods.
... Ubiquitous data has been proved to be useful in detecting mental health conditions. Mobile sensor data, such as GPS logs [ 36,37 ], electrodermal activity, sleep behavior, motion, and phone usage patterns [ 38 ] has been applied in investigating depressive symptoms. [ 39 ] found that individual private Google Search histories can be used to detect low self-esteem conditions among college students. ...
Preprint
Background Mental health problems among the global population are worsened during the coronavirus disease (COVID-19). Yet, current methods for screening mental health issues rely on in-person interviews, which can be expensive, time-consuming, blocked by social stigmas and quarantines. Meanwhile, how individuals engage with online platforms such as Google Search and YouTube undergoes drastic shifts due to COVID-19 and subsequent lockdowns. Such ubiquitous daily behaviors on online platforms have the potential to capture and correlate with clinically alarming deteriorations in mental health profiles of users through a non-invasive manner. Objective The goal of this study is to examine, among college students in the United States, the relationship between deteriorating mental health conditions and changes in user behaviors when engaging with Google Search and YouTube during COVID-19. Methods This study recruited a cohort of undergraduate students (N = 49) from a U.S. college campus during January 2020 (prior to the pandemic) and measured the anxiety and depression levels of each participant. The anxiety level was assessed via the General Anxiety Disorder-7 (GAD-7). The depression level was assessed via the Patient Health Questionnaire-9 (PHQ-9). This study followed up with the same cohort during May 2020 (during the pandemic), and the anxiety and depression levels were assessed again. The longitudinal Google Search and YouTube history data of all participants were anonymized and collected. From individual-level Google Search and YouTube histories, we developed 5 signals that can quantify shifts in online behaviors during the pandemic. We then assessed the differences between groups with and without deteriorating mental health profiles in terms of these features. Results Of the 49 participants, 41% (n = 20) of them reported a significant increase (increase in the PHQ-9 score ≥ 5) in depression, denoted as DEP; 45% (n = 22) of them reported a significant increase (increase in the GAD-7 score ≥ 5) in anxiety, denoted as ANX. Of the 5 features proposed to quantify online behavior changes, statistical significances were found between the DEP and non-DEP groups for all of them ( P ≤.01, effect sizes ranging between 0.130 to 0.320); statistical significances were found between the ANX and non-ANX groups for 4 of them ( P ≤.01, effect sizes ranging between 0.115 to 0.231). Significant features included late-night online activities, continuous usages and time away from the internet, porn consumptions, and keywords associated with negative emotions, social activities, and personal affairs. Conclusions The results suggested strong discrepancies between college student groups with and without deteriorating mental health conditions in terms of behavioral changes in Google Search and YouTube usages during the COVID-19. Though further studies are required, our results demonstrated the feasibility of utilizing pervasive online data to establish non-invasive surveillance systems for mental health conditions that bypasses many disadvantages of existing screening methods.
... Similarly, Stamatakis et al. employed a logistic regression algorithm to predict PD severity levels based on a finger tapping task implemented on a smartphone [33] . Other regression models have been employed in the prediction of depression severity [34] and mental health states [35] using smartphones. ...
Chapter
Full-text available
Recent studies have demonstrated how novel technology-based frameworks allow for evaluation of individual behaviour in health and disease in unprecedented detail. We summarize how this technology-based approach is increasingly adopted in the study of neurological and cognitive disorders to allow for objective quantification of behavior that are otherwise not measurable or only captured by subjective reports. In addition, we outline the outcomes of efforts to adopt this approach in psychiatry with a particular focus on social behavior. We illustrate how a technology-based framework can be extended in a key-lock fashion to research in psychiatry to provide observer-independent evaluations of specific symptom domains. Here, we suggest a framework to quantify disturbances of social interaction that are ubiquitously observed across various psychiatric disorders and play a crucial role in their development and treatment. Importantly, the framework allows for an investigation of both the behavioral and neural mechanisms of social interaction in different settings and can promote a cross-fertilization of methods to focus on those ecologically valid phenomena that are, indeed, relevant to treatment success.
... Empatica E4 is a wearable device [25] that can capture motion (through accelerometer (ACC)), and physiological indicators (blood volume pulse (BVP), electrodermal activity (EDA), and skin temperature (TEMP)). Accuracy, reliability, and validity of sensor data from Empatica E4 device have been confirmed for stress detection [26,27], assessment of depressive symptoms [28], and regular daily activities [29]. In our study, the E4 was removed at night before participants sleep by the nurse for charging. ...
Article
Full-text available
People living with dementia (PLwD) often exhibit behavioral and psychological symptoms, such as episodes of agitation and aggression. Agitated behavior in PLwD causes distress and increases the risk of injury to both patients and caregivers. In this paper, we present the use of a multi-modal wearable device that captures motion and physiological indicators to detect agitation in PLwD. We identify features extracted from sensor signals that are the most relevant for agitation detection. We hypothesize that combining multi-modal sensor data will be more effective to identify agitation in PLwD in comparison to a single sensor. The results of this unique pilot study are based on 17 participants’ data collected during 600 days from PLwD admitted to a Specialized Dementia Unit. Our findings show the importance of using multi-modal sensor data and highlight the most significant features for agitation detection.
... Other, less intrusive physiological measurements are of course possible, with smartwatches being a great example of these. They are able to measure numerous factors, including heart rate, location, skin temperature, perspiration, and activity [15], making them a widespread tool in research studies [16,17,18,19]. ...
Article
Full-text available
In the last few decades, the energy consumption of individual buildings has been steadily improving. As a result, research efforts are shifting towards acquiring a deeper understanding of occupant comfort, health, and well-being in the built environment. However, existing techniques used to measure and predict the comfort of occupants have seen little change since Fanger. New research attempts are hence focusing on methods to gather more data, more frequently, and less intrusively. A little explored source of data is the one gathered from real-time videos of occupants, the so-called facial action units (FAU), which is the focus of this paper. These are the facial movements and positions that constitute the basic elements of emotions. Using software developed in the realm of affective computing, seven building occupants were monitored for a period of 2 weeks, whilst also completing surveys that gathered information about the office environment, and their work and personal life. Results found that participants that were happy with their office space showed significantly higher average values of the Cheek Raiser (AU06) and Lid Tightener (AU07) facial action units. These findings show the potential of using FAUs to assist in the control and design of buildings in a human-centric manner.
... A substantial amount of research has been carried out with wearable EDA sensors that can provide better insight into how affect and stress interact with daily life [27]- [29]. However, wearable sensors often suffer from poor signal quality as well as motion artifacts. ...
Article
Full-text available
The autonomic nervous system (ANS) stimulates various sweat glands for maintaining body temperature as well as in response to various psychological events. Variations in skin conductance (SC) measurements due to salty sweat secretion can be used to infer the underlying ANS activity. Recovering both ANS activity and the underlying system from noisy single-channel recordings is challenging. As the same ANS activity drives all the sweat glands throughout the skin, the same information is encoded in different SC recordings. We perform system identification and develop a physiological model for multi-channel SC recordings relating them to ANS activation events. Using a multi-rate formulation, we estimate the number, timings, and amplitudes of ANS activity and the unknown model parameters from multi-channel SC data.We incorporate a generalized-cross-validation-based sparse recovery approach to balance between the sparsity level of the inferred ANS activity and the goodness of fit to the multi-channel SC data. We successfully deconvolve multi-channel experimental auditory stimulation SC data from human participants.We analyze experimental and simulated data to validate the performance of our concurrent deconvolution algorithm; we illustrate that we can recover the ANS activity due to the underlying auditory stimuli. Furthermore, we estimate stress using inferred ANS activity based on multi-channel deconvolution of SC data collected during different driving conditions and at rest. We propose a model for multi-channel SC recordings. Moreover, we develop a multi-channel deconvolution approach to perform robust sparse inference in the presence of noise. The proposed approach could potentially improve stress state estimation using wearables.
... Much of the work in this area has leveraged smartphones rather than wearable devices, including smartphone-based sensing of geolocation, vocal quality, accelerometry, communication logs, and social interactions as based on co-presence of other devices and sampling of ambient noise [5,33,55,78]. Other work has explored using physiological measurements not routinely available in commercial wearables, such as electrodermal activity [43]. In addition, digital trace data, such as social media posting content has been applied to predict mental health states [93]. ...
Article
The increasing ubiquity of health sensing technology holds promise to enable patients and health care providers to make more informed decisions based on continuously-captured data. The use of sensor-captured patient-generated data (sPGD) has been gaining greater prominence in the assessment of physical health, but we have little understanding of the role that sPGD can play in mental health. To better understand the use of sPGD in mental health, we interviewed care providers in an intensive treatment program (ITP) for veterans with post-traumatic stress disorder. In this program, patients were given Fitbits for their own voluntary use. Providers identified a number of potential benefits from patients' Fitbit use, such as patient empowerment and opportunities to reinforce therapeutic progress through collaborative data review and interpretation. However, despite the promise of sensor data as offering an "objective" view into patients' health behavior and symptoms, the relationships between sPGD and therapeutic progress are often ambiguous. Given substantial subjectivity involved in interpreting data from commercial wearables in the context of mental health treatment, providers emphasized potential risks to their patients and were uncertain how to adjust their practice to effectively guide collaborative use of the FitBit and its sPGD. We discuss the implications of these findings for designing systems to leverage sPGD in mental health care.?
... (yaseminm, sarikayameh, gokhan.ince)@itu.edu.tr have used Electrodermal Activity (EDA) as an indicator of the people's depressive symptoms [7]. ...
Conference Paper
Emotions potentially have a significant impact on human actions and recognizing affective states is an effective way of implementing Brain-Computer Interface (BCI) systems which process brain signals to allow direct communication and interaction with the environment. In this paper, a real-time emotion recognition model was developed on the basis of physiological signals. A sensor fusion method is developed to detect human emotion by using data acquired from ElectroEncephaloGraphy (EEG) and ElectroDermal Activity (EDA) sensors. The proposed physiology-based emotion recognition system using a neural network was implemented and tested on human subjects, and a classification accuracy of 94% on three different emotions was achieved.
... Further, the utility of contextual data outside the current digital medicine system should be evaluated, e.g., wearables and mobile passive sensing, potentially in conjunction with self-report ecological momentary assessments or brief clinical questionnaires. [19][20][21] In conclusion, this work has demonstrated that a hybrid Bayesian modeling framework is capable of characterizing temporal patterns of successful and unsuccessful ingestion events from a digital medicine system in patients with serious mental illness. We have also identified immediate next steps and additional opportunities for research in the space. ...
Article
Full-text available
The objective of this work was to adapt and evaluate the performance of a Bayesian hybrid model to characterize objective temporal medication ingestion parameters from two clinical studies in patients with serious mental illness (SMI) receiving treatment with a digital medicine system. This system provides a signal from an ingested sensor contained in the dosage form to a patient-worn patch and transmits this signal via the patient’s mobile device. A previously developed hybrid Markov-von Mises model was used to obtain maximum-likelihood estimates for medication ingestion behavior parameters for individual patients. The individual parameter estimates were modeled to obtain distribution parameters of priors implemented in a Markov chain-Monte Carlo framework. Clinical and demographic covariates associated with model ingestion parameters were also assessed. We obtained individual estimates of overall observed ingestion percent (median:75.9%, range:18.2–98.3%, IQR:32.9%), rate of excess dosing events (median:0%, range:0–14.3%, IQR:3.0%) and observed ingestion duration. The modeling also provided estimates of the Markov-dependence probabilities of dosing success following a dosing success or failure. The ingestion-timing deviations were modeled with the von Mises distribution. A subset of 17 patients (22.1%) were identified as prompt correctors based on Markov-dependence probability of a dosing failure followed by a dosing success of unity. The prompt corrector sub-group had a better overall digital medicine ingestion parameter profile compared to those who were not prompt correctors. Our results demonstrate the potential utility of a Bayesian Hybrid Markov-von Mises model for characterizing digital medicine ingestion patterns in patients with SMI.
... March 2020 → a non-invasive manner (Zhang et al. 2020, p. 1; see Saeb et al. 2016, Ghandeharioun et al. 2017, Wang et al. 2018). ...
Article
Full-text available
Understanding how people react to the COVID-19 crisis, and what the consequences are of the COVID-19 pandemic is key to enable public health and other agencies to develop optimal intervention strategies. Because the timely identification of new cases of infection has proven to be the key to timely respond to the spread of infection within a particular region, we have developed a method that can detect and predict the emergence of new cases of COVID-19 at an early stage. Further, this method can give useful insights into a family's life during the pandemic and give the prediction of birth rates. The basic methodological concept of our approach is to monitor the digital trace of language searches with the Google Trends analytical tool (GT). We divided the keyword frequency for selected words giving us a search frequency index and then compared searches with official statistics to prove the significations of results. 1) Google Trends tools are suitable for predicting the emergence of new COVID-19 cases in Croatia. The data collected by this method correlate with official data. In Croatia search activities using GT for terms such as "PCR +COVID", and symptoms "cough + corona", "pneumonia + corona"; "muscle pain + corona" correlate strongly with officially reported cases of the disease. 2) The method also shows effects on family life, increase in stress, and domestic violence. 3) Birth rate in 2021 will be just 87% of what it would be "a normal year" in Croatia. 4) This tool can give useful insights into domestic violence. Unquestionably, there are still significant open methodological issues and the questionable integrity of the data obtained using this source. The fact is also a problem that GT does not provide data on which population was sampled or how it was structured. Although these open-ended issues pose serious challenges for making clear estimates, statistics offer a range of tools available to deal with imperfect data as well as to develop controls that take data quality into account. All these insights show that GT has the potential to capture attitudes in the broad spectrum of family life themes. The benefit of this method is reliable estimates that can enable public health officials to prepare and better respond to the possible return of a pandemic in certain parts of the country and the need for responses to protect family well-being.
... However, inferring the ANS stimulation as well as physiological system parameters using only one channel of SC data is challenging in presence of noise. Extensive research has been conducted to collect EDA using wearable sensors that can provide better insight into how affect and stress interact with daily life [13], [14], [15]. Nevertheless, wearable sensors often suffer from poor signal acquisition and motion artifacts. ...
... Examples include using smartphone data to model social interactions ( [13]), to study the relationship between mood and sleep ( [38]), to detect stress, happiness, and mood ( [53,5,2,29,6,24]), and to predict depressive symptoms ( [50]). Others have also attempted prediction of fine grained symptoms on a continuous scale using smartphone data and wearable sensors ( [19]). Though not perfect, personal sensing -"collection and analysis of data from sensors embedded in the context of daily life with the aim of identifying human behaviors, thoughts, feelings, and traits" [35] -has shown potential for monitoring mental health and providing a platform for just-in-time interventions. ...
Preprint
The delivery of mental health interventions via ubiquitous devices has shown much promise. A natural conversational interface that allows longitudinal symptom tracking and appropriate just-in-time interventions would be extremely valuable. However, the task of designing emotionally-aware agents is still poorly understood. Furthermore, the feasibility of automating the delivery of just-in-time mHealth interventions via such an agent has not been fully studied. In this paper, we present the design and evaluation of EMMA (EMotion-Aware mHealth Agent) through two human-subject experiments with N=39 participants (one-week, and two-week long respectively). EMMA conducts experience sampling in an empathetic manner and provides emotionally appropriate micro-activities. We show the system can be extended to detect a user's mood purely from smartphone sensor data. Our results show that extraverts preferred EMMA significantly more than introverts and that our personalized machine learning model worked as well as relying on gold-standard self-reports of emotion from users. Finally, we provide a set of guidelines for the design of bots for mHealth.
... Pulmonary disease, such as Chronic Obstructive Pulmonary Disease (COPD), asthma, and Covid-19, is one of leading causes of morbidity and mortality. Some recent works use wearables to detect cough activity, a major symptom of pulmonary diseases [199,311,329,330] Other works have introduced methods for monitoring stroke in infants using wearable accelerometers [100] and methods for assessing depressive symptoms utilizing wrist-worn sensors [104]. In addition, detecting muscular activities and hand motions using electromyography (EMG) sensors has been widely applied to enable improved prostheses control for people with missing or damaged limbs [186,209,211,240]. ...
Preprint
Full-text available
Mobile and wearable devices have enabled numerous applications, including activity tracking, wellness monitoring, and human-computer interaction, that measure and improve our daily lives. Many of these applications are made possible by leveraging the rich collection of low-power sensors found in many mobile and wearable devices to perform human activity recognition (HAR). Recently, deep learning has greatly pushed the boundaries of HAR on mobile and wearable devices. This paper systematically categorizes and summarizes existing work that introduces deep learning methods for wearables-based HAR and provides a comprehensive analysis of the current advancements, developing trends, and major challenges. We also present cutting-edge frontiers and future directions for deep learning--based HAR.
... March 2020 → a non-invasive manner (Zhang et al. 2020, p. 1; see Saeb et al. 2016, Ghandeharioun et al. 2017, Wang et al. 2018). ...
Article
Understanding how people react to the COVID-19 crisis, and what the consequences are of the COVID-19 pandemic is key to enable public health and other agencies to develop optimal intervention strategies. Because the timely identification of new cases of infection has proven to be the key to timely respond to the spread of infection within a particular region, we have developed a method that can detect and predict the emergence of new cases of COVID-19 at an early stage. Further, this method can give useful insights into a family’s life during the pandemic and give the prediction of birth rates. The basic methodological concept of our approach is to monitor the digital trace of language searches with the Google Trends analytical tool (GT). We divided the keyword frequency for selected words giving us a search frequency index and then compared searches with official statistics to prove the significations of results. 1) Google Trends tools are suitable for predicting the emergence of new COVID-19 cases in Croatia. The data collected by this method correlate with official data. In Croatia search activities using GT for terms such as “PCR +COVID”, and symptoms “cough + corona”, “pneumonia + corona”; “muscle pain + corona” correlate strongly with officially reported cases of the disease. 2) The method also shows effects on family life, increase in stress, and domestic violence. 3) Birth rate in 2021 will be just 87% of what it would be “a normal year” in Croatia. 4) This tool can give useful insights into domestic violence. Unquestionably, there are still significant open methodological issues and the questionable integrity of the data obtained using this source. The fact is also a problem that GT does not provide data on which population was sampled or how it was structured. Although these open-ended issues pose serious challenges for making clear estimates, statistics offer a range of tools available to deal with imperfect data as well as to develop controls that take data quality into account. All these insights show that GT has the potential to capture attitudes in the broad spectrum of family life themes. The benefit of this method is reliable estimates that can enable public health officials to prepare and better respond to the possible return of a pandemic in certain parts of the country and the need for responses to protect family well-being. Keywords: Google trends, COVID-19, birth rates, domestic violence, Croatia, predicting demographic trends, family
... Both classification and regression models showed predictive value, however, many of them lacked information regarding the handling of missing sensor data and calibration. Those that do, report simple imputation methods such as mean imputation, with two studies using multiple imputation methods 6,18 . ...
Article
Full-text available
The use of digital tools to measure physiological and behavioural variables of potential relevance to mental health is a growing field sitting at the intersection between computer science, engineering, and clinical science. We summarised the literature on remote measuring technologies, mapping methodological challenges and threats to reproducibility, and identified leading digital signals for depression. Medical and computer science databases were searched between January 2007 and November 2019. Published studies linking depression and objective behavioural data obtained from smartphone and wearable device sensors in adults with unipolar depression and healthy subjects were included. A descriptive approach was taken to synthesise study methodologies. We included 51 studies and found threats to reproducibility and transparency arising from failure to provide comprehensive descriptions of recruitment strategies, sample information, feature construction and the determination and handling of missing data. The literature is characterised by small sample sizes, short follow-up duration and great variability in the quality of reporting, limiting the interpretability of pooled results. Bivariate analyses show consistency in statistically significant associations between depression and digital features from sleep, physical activity, location, and phone use data. Machine learning models found the predictive value of aggregated features. Given the pitfalls in the combined literature, these results should be taken purely as a starting point for hypothesis generation. Since this research is ultimately aimed at informing clinical practice, we recommend improvements in reporting standards including consideration of generalisability and reproducibility, such as wider diversity of samples, thorough reporting methodology and the reporting of potential bias in studies with numerous features.
... Some recent works use wearables to detect cough activity, a major symptom of pulmonary diseases [37][38][39][40]. Other works have introduced methods for monitoring stroke in infants using wearable accelerometers [41] and methods for assessing depressive symptoms utilizing wrist-worn sensors [42]. In addition, detecting muscular activities and hand motions using electromyography (EMG) sensors has been widely applied to enable improved prostheses control for people with missing or damaged limbs [43][44][45][46]. ...
Article
Full-text available
Mobile and wearable devices have enabled numerous applications, including activity tracking, wellness monitoring, and human–computer interaction, that measure and improve our daily lives. Many of these applications are made possible by leveraging the rich collection of low-power sensors found in many mobile and wearable devices to perform human activity recognition (HAR). Recently, deep learning has greatly pushed the boundaries of HAR on mobile and wearable devices. This paper systematically categorizes and summarizes existing work that introduces deep learning methods for wearables-based HAR and provides a comprehensive analysis of the current advancements, developing trends, and major challenges. We also present cutting-edge frontiers and future directions for deep learning-based HAR.
... The systematic use of large-scale data collected via commercially available fitness trackers and smartwatches is a rapidly growing line of data-driven, medical research [39][40][41]. Wearable sensor data has been applied to study physiological markers of depression [42,43], characterize daily physiology and circadian rhythms [44], and improve surveillance of influenza-like illness [45]. In the COVID-19 context, this approach has been applied to the early detection of COVID-19 in individuals [46], predicting overall case numbers and changes in trends [47], and discriminating COVID-19 positive from negative individuals [48,49]. ...
Preprint
Vaccines are among the most powerful tools used to combat the COVID-19 pandemic. They are highly effective against infection and substantially reduce the risk of severe disease, hospitalization, ICU admission, and death. However, their potential for attenuating long-term effects of a SARS-CoV-2 infection, commonly denoted as Long COVID, remains elusive and is still subject of debate. Such long-term effects can be effectively monitored at the individual level by analyzing physiological data collected by consumer-grade wearable sensors. Here, we investigate changes in resting heart rate, daily physical activity, and sleep duration in response to a SARS-CoV-2 infection stratified by vaccination status. Data was collected over a period of two years in the context of the German Corona Data Donation Project with currently around 190,000 monthly active donors. Compared to their unvaccinated counterparts, we find that vaccinated individuals on average experience smaller changes in their vital data that also return to normal levels more quickly. Likewise, extreme changes in vitals during the acute phase of the disease occur less frequently in vaccinated individuals. Our results solidify evidence that vaccines can mitigate long-term detrimental effects of SARS-CoV-2 infections both in terms of duration and magnitude. Furthermore, they demonstrate the value of large scale, high-resolution wearable sensor data in public health research.
... This poor consistency is likely a result of the heterogeneous presentation and multifactorial etiology of MDD, which substantially complicates research on this order 20 . Recently, Picard and colleagues predicted the severity of depressive symptoms in 12 patients with MDD by analyzing EDA, sleep patterns, motion, and other activities assessed via built-in smartphone sensors 21 . While this work is encouraging, the study was conducted without a healthy control comparison group and data were collected across multiple modalities to test prediction models. ...
Article
Full-text available
Abstract Major depressive disorder (MDD) is a common psychiatric disorder and the leading cause of disability worldwide. However, current methods used to diagnose depression mainly rely on clinical interviews and self-reported scales of depressive symptoms, which lack objectivity and efficiency. To address this challenge, we present a machine learning approach to screen for MDD using electrodermal activity (EDA). Participants included 30 patients with MDD and 37 healthy controls. Their EDA was measured during five experimental phases consisted of baseline, mental arithmetic task, recovery from the stress task, relaxation task, and recovery from the relaxation task, which elicited multiple alterations in autonomic activity. Selected EDA features were extracted from each phase, and differential EDA features between two distinct phases were evaluated. By using these features as input data and performing feature selection with SVM-RFE, 74% accuracy, 74% sensitivity, and 71% specificity could be achieved by our decision tree classifier. The most relevant features selected by SVM-RFE included differential EDA features and features from the stress and relaxation tasks. These findings suggest that automatic detection of depression based on EDA features is feasible and that monitoring changes in physiological signal when a subject is experiencing autonomic arousal and recovery may enhance discrimination power.
... Previous studies have explored different types of machine learning models in affect estimation. In this work, we select four most popular and effective types of models [18,29,30]: bagging model, boosting model, personalized model, and mixed effects model. ...
Article
Full-text available
Affective states are associated with people’s mental health status and have profound impact on daily life, thus unobtrusively understanding and estimating affects have been brought to the public attention. The pervasiveness of wearable sensors makes it possible to build automatic systems for affect tracking. However, constructing such systems is a challenging task due to the complexity of human behaviors. In this work, we focus on the problem of estimating daily self-reported affects from sensor-generated data. We first analyze the intra- and inter-subject differences of self-reported affect labels. Second, we explore different machine learning models as well as label transformation techniques to overcome the individual differences in self-reported responses estimation. We conceptualize three experimental settings including long-term and short-term estimation scenarios. Our experimental results show that the mixed effects model and label transformation can yield better estimation of individual daily affect. This work poses the basis for future sensor-based individualized and real-time affective digital and/or clinical interventions.
Chapter
Medical Cyber Physical Systems (MCPSs) prescribe a platform in which patient health information is acquired by the emerging Internet of Things (IoT) sensors, pre-processed locally, and processed via advanced machine intelligence algorithms in the cloud. The emergence of MCPSs holds the promise to revolutionize remote patient healthcare monitoring, accelerate the development of new drugs or treatments, and improve the quality-of-life for patients who are suffering from various medical conditions among other various applications. The amount of raw medical data gathered through the IoT sensors in an MCPS provides a rich platform that artificial intelligence algorithms can use to provide decision support for either medical experts or patients. In this paper, we provide an overview of MCPSs and the data flow through these systems. This includes how raw physiological signals are converted into features and are used by machine intelligence algorithms, the types of algorithms available for the healthcare domain, how the data and the decision support output are presented to the end user, and how all of these steps are completed in a secure fashion to preserve the privacy of the users.
Chapter
This chapter explores the most relevant aspects in relation to the outcomes and performance of the different components of a healthcare system with a particular focus on mobile healthcare applications. In detail, we discuss the six quality principles to be satisfied by a generic healthcare system and the main international and European projects, which have supported the dissemination of these systems. This diffusion has been encouraged by the application of wireless and mobile technologies, through the so-called m-Health systems. One of the main fields of application of an m-Health system is telemedicine, for which reason we will address an important challenge encountered during the realization of an m-Health application: the analysis of the functionalities that an m-Health app has to provide. To achieve this latter aim, we will present an overview of a generic m-Health application with its main functionalities and components. Among these, the use of a standardized method for the treatment of a massive amount of patient data is necessary in order to integrate all the collected information resulting from the development of a great number of new m-Health devices and applications. Electronic Health Records (EHR), and international standards, like Health Level 7 (HL7) and Fast Healthcare Interoperability Resources (FHIR), aims at addressing this important issue, in addition to guaranteeing the privacy and security of these health data. Moreover, the insights that can be discerned from an examination of this vast repository of data can open up unparalleled opportunities for public and private sector organizations. Indeed, the development of new tools for the analysis of data, which on occasions may be unstructured, noisy, and unreliable, is now considered a vital requirement for all specialists who are involved in the handling and using of information. These new tools may be based on rule, machine or deep learning, or include question answering, with cognitive computing certainly having a key role to play in the development of future m-Health applications.
Article
With the rapid development of Internet of Things (IoT) technologies, traditional disease diagnoses carried out in medical institutions can now be performed remotely at home or even ambient environments, yielding the concept of the Internet of Health Things (IoHT). Among the diverse IoHT applications, inertial measurement unit (IMU)-based systems play a significant role in the detection of diseases in many fields, such as neurological, musculoskeletal, and mental. However, traditional numerical interpretation methods have proven to be challenging to provide satisfying detection accuracies owing to the low quality of raw data, especially under strong electromagnetic interference (EMI). To address this issue, in recent years, machine learning (ML)-based techniques have been proposed to smartly map IMU-captured data on disease detection and progress. After a decade of development, the combination of IMUs and ML algorithms for assistive disease diagnosis has become a hot topic, with an increasing number of studies reported yearly. A systematic search was conducted in four databases covering the aforementioned topic for articles published in the past six years. Eighty-one articles were included and discussed concerning two aspects: different ML techniques and application scenarios. This review yielded the conclusion that, with the help of ML technology, IMUs can serve as a crucial element in disease diagnosis, severity assessment, characteristic estimation, and monitoring during the rehabilitation process. Furthermore, it summarizes the state-of-the-art, analyzes challenges, and provides foreseeable future trends for developing IMU-ML systems for IoHT.
Conference Paper
Abstract — The increasing use of technology can cause a life- style of less physical work. Also, the constant pressure on an individual can create a risk of mental disorder. These vulnerabilities include peer pressure, heart attack, depression, and many other effects. In this paper different approaches to predict depression are studied in detail. The mechanisms include collecting dataset through questionnaires asked to the person, posts on social media, text used throughout verbal communication and expressions on face. Result is derived from extracted information. Here output expected is that the person needs attention or not. In this research work different algorithms and classifiers of machine learning such as Decision Trees , SVM, Naive Bayes Classifier, Logistic Regression and KNN Classifier are analyzed to identify state of mental health in a target group. Target groups used for this identification process are public like students of high school, college students and working professionals. The paper also demonstrates an example in which Twitter scrapping tool Twint is used detect whether given Twit is depressive or not.
Chapter
Digital traces are often used as a substitute for survey data. However, it is unclear whether and how digital traces actually correspond to the survey-based traits they purport to measure. This paper examines correlations between self-reports and digital trace proxies of depression, anxiety, mood, social integration and sleep among high school students. The study is based on a small but rich multilayer data set (N = 144). The data set contains mood and sleep measures, assessed daily over a 4-month period, along with survey measures at two points in time and information about online activity from VK, the most popular social networking site in Russia. Our analysis indicates that 1) the sentiments expressed in social media posts are correlated with depression; namely, adolescents with more severe symptoms of depression write more negative posts, 2) late-night posting indicates less sleep and poorer sleep quality, and 3) students who were nominated less often as somebody’s friend in the survey have fewer friends on VK and their posts receive fewer “likes.” However, these correlations are generally weak. These results demonstrate that digital traces can serve as useful supplements to, rather than substitutes for, survey data in studies on adolescents’ well-being. These estimates of correlations between survey and digital trace data could provide useful guidelines for future research on the topic.
Article
People interact with their physical environments every day by visiting different places and moving between them. Such mobility behaviours likely influence and are influenced by people's subjective well-being. However, past research examining the links between mobility behaviours and well-being has been inconclusive. Here, we provide a comprehensive investigation of these relationships by examining individual differences in two types of mobility behaviours (movement patterns and places visited) and their relationship to six indicators of subjective well-being (depression, loneliness, anxiety, stress, affect, and energy) at two different temporal levels of analysis (two-week tendencies and daily level). Using data from a large smartphone-based longitudinal study (N = 1765), we show that (i) movement patterns assessed via GPS data (distance travelled, entropy, and irregularity) and (ii) places visited assessed via experience sampling reports (home, work, and social places) are associated with subjective well-being at the between and within person levels. Our findings suggest that distance travelled is related to anxiety, affect, and stress, irregularity is related to depression and loneliness, and spending time in social places is negatively associated with loneliness. We discuss the implications of our work and highlight directions for future research on the generalizability to other populations as well as the characteristics of places. © 2020 European Association of Personality Psychology.
Chapter
Wearable devices have evolved over the years and shown significant increase in popularity. With the advances in sensor technologies, data collection capabilities, and data analytics, wearable devices now enable interaction among users, devices, and their environment seamlessly. Multifunctional nature of this technology enables users to track their daily physical activities, engage with other users through social networking capabilities, and log their lifestyle habits. In this chapter, the authors discuss the types of sensor technologies embedded in wearable devices and how the data collected through such devices can be further interpreted by data analytics. In parallel with abundance of personal data that can be collected via wearable devices, they also discuss issues related to data privacy, suggestions for users, developers, and policymakers regarding how to protect data privacy are also discussed.
Article
Background Proactive detection of mental health needs among people with diabetes mellitus could facilitate early intervention, improve overall health and quality of life, and reduce individual and societal health and economic burdens. Passive sensing and ecological momentary assessment are relatively newer methods that may be leveraged for such proactive detection. Objective The primary aim of this study was to conceptualize, develop, and evaluate a novel machine learning approach for predicting mental health risk in people with diabetes mellitus. Methods A retrospective study was designed to develop and evaluate a machine learning model, utilizing data collected from 142,432 individuals with diabetes enrolled in the Livongo for Diabetes program. First, participants’ mental health statuses were verified using prescription and medical and pharmacy claims data. Next, four categories of passive sensing signals were extracted from the participants’ behavior in the program, including demographics and glucometer, coaching, and event data. Data sets were then assembled to create participant-period instances, and descriptive analyses were conducted to understand the correlation between mental health status and passive sensing signals. Passive sensing signals were then entered into the model to train and test its performance. The model was evaluated based on seven measures: sensitivity, specificity, precision, area under the curve, F1 score, accuracy, and confusion matrix. SHapley Additive exPlanations (SHAP) values were computed to determine the importance of individual signals. Results In the training (and validation) and three subsequent test sets, the model achieved a confidence score greater than 0.5 for sensitivity, specificity, area under the curve, and accuracy. Signals identified as important by SHAP values included demographics such as race and gender, participant’s emotional state during blood glucose checks, time of day of blood glucose checks, blood glucose values, and interaction with the Livongo mobile app and web platform. Conclusions Results of this study demonstrate the utility of a passively informed mental health risk algorithm and invite further exploration to identify additional signals and determine when and where such algorithms should be deployed.
Article
****THIS IS AN OPEN ACCESS ARTICLE. VISIT URL: https://mental.jmir.org/2021/6/e24668 **** Background Uncertainty surrounds the ethical and legal implications of algorithmic and data-driven technologies in the mental health context, including technologies characterized as artificial intelligence, machine learning, deep learning, and other forms of automation. Objective This study aims to survey empirical scholarly literature on the application of algorithmic and data-driven technologies in mental health initiatives to identify the legal and ethical issues that have been raised. Methods We searched for peer-reviewed empirical studies on the application of algorithmic technologies in mental health care in the Scopus, Embase, and Association for Computing Machinery databases. A total of 1078 relevant peer-reviewed applied studies were identified, which were narrowed to 132 empirical research papers for review based on selection criteria. Conventional content analysis was undertaken to address our aims, and this was supplemented by a keyword-in-context analysis. Results We grouped the findings into the following five categories of technology: social media (53/132, 40.1%), smartphones (37/132, 28%), sensing technology (20/132, 15.1%), chatbots (5/132, 3.8%), and miscellaneous (17/132, 12.9%). Most initiatives were directed toward detection and diagnosis. Most papers discussed privacy, mainly in terms of respecting the privacy of research participants. There was relatively little discussion of privacy in this context. A small number of studies discussed ethics directly (10/132, 7.6%) and indirectly (10/132, 7.6%). Legal issues were not substantively discussed in any studies, although some legal issues were discussed in passing (7/132, 5.3%), such as the rights of user subjects and privacy law compliance. Conclusions Ethical and legal issues tend to not be explicitly addressed in empirical studies on algorithmic and data-driven technologies in mental health initiatives. Scholars may have considered ethical or legal matters at the ethics committee or institutional review board stage. If so, this consideration seldom appears in published materials in applied research in any detail. The form itself of peer-reviewed papers that detail applied research in this field may well preclude a substantial focus on ethics and law. Regardless, we identified several concerns, including the near-complete lack of involvement of mental health service users, the scant consideration of algorithmic accountability, and the potential for overmedicalization and techno-solutionism. Most papers were published in the computer science field at the pilot or exploratory stages. Thus, these technologies could be appropriated into practice in rarely acknowledged ways, with serious legal and ethical implications.
Article
Background: Mood disorders are commonly underrecognized and undertreated, as diagnosis is reliant on self-reporting and clinical assessments that are often not timely. Speech characteristics of those with mood disorders differs from healthy individuals. With the wide use of smartphones, and the emergence of machine learning approaches, smartphones can be used to monitor speech patterns to help the diagnosis and monitoring of mood disorders. Objective: The aim of this review is to synthesize research on using speech patterns from smartphones to diagnose and monitor mood disorders. Methods: Literature searches of major databases, Medline, PsycInfo, EMBASE, and CINAHL, initially identified 832 relevant articles using the search terms "mood disorders", "smartphone", "voice analysis", and their variants. Only 13 studies met inclusion criteria: use of a smartphone for capturing voice data, focus on diagnosing or monitoring a mood disorder(s), clinical populations recruited prospectively, and in the English language only. Articles were assessed by 2 reviewers, and data extracted included data type, classifiers used, methods of capture, and study results. Studies were analyzed using a narrative synthesis approach. Results: Studies showed that voice data alone had reasonable accuracy in predicting mood states and mood fluctuations based on objectively monitored speech patterns. While a fusion of different sensor modalities revealed the highest accuracy (97.4%), nearly 80% of included studies were pilot trials or feasibility studies without control groups and had small sample sizes ranging from 1 to 73 participants. Studies were also carried out over short or varying timeframes and had significant heterogeneity of methods in terms of the types of audio data captured, environmental contexts, classifiers, and measures to control for privacy and ambient noise. Conclusions: Approaches that allow smartphone-based monitoring of speech patterns in mood disorders are rapidly growing. The current body of evidence supports the value of speech patterns to monitor, classify, and predict mood states in real time. However, many challenges remain around the robustness, cost-effectiveness, and acceptability of such an approach and further work is required to build on current research and reduce heterogeneity of methodologies as well as clinical evaluation of the benefits and risks of such approaches.
Article
Depression and bipolar disorder are mood disorders affecting millions of people worldwide that can have severe impacts on one’s quality of life. Our ability to detect these illnesses is directly tied to our ability to intervene; however, there has not yet been a systematic review of the technological aspects of the most studied mood disorder detection mechanisms. This survey summarizes detection methods from clinical sensing solutions, such as electroencephalograms and functional near infrared spectroscopy, to ubiquitous sensing solutions, such as scraping social media data and utilizing GPS data. We provide break downs of the algorithms and feature processing techniques of the state of the art methods across these various category and discuss the current challenges affecting mood disorder detection technology, setting the stage for further research.
Article
Full-text available
Introduction: Behavioral and psychological symptoms of dementia (BPSD) signal distress or unmet needs and present a risk to people with dementia and their caregivers. Variability in the expression of these symptoms is a barrier to the performance of digital biomarkers. The aim of this study was to use wearable multimodal sensors to develop personalized machine learning models capable of detecting individual patterns of BPSD. Methods: Older adults with dementia and BPSD (n = 17) on a dementia care unit wore a wristband during waking hours for up to 8 weeks. The wristband captured motion (accelerometer) and physiological indicators (blood volume pulse, electrodermal activity, and skin temperature). Agitation or aggression events were tracked, and research staff reviewed videos to precisely annotate the sensor data. Personalized machine learning models were developed using 1-minute intervals and classifying the presence of behavioral symptoms, and behavioral symptoms by type (motor agitation, verbal aggression, or physical aggression). Results: Behavioral events were rare, representing 3.4% of the total data. Personalized models classified behavioral symptoms with a median area under the receiver operating curve (AUC) of 0.87 (range 0.64-0.95). The relative importance of the different sensor features to the predictive models varied both by individual and behavior type. Discussion: Patterns of sensor data associated with BPSD are highly individualized, and future studies of the digital phenotyping of these behaviors would benefit from personalization.
Chapter
Wearable devices have evolved over the years and shown significant increase in popularity. With the advances in sensor technologies, data collection capabilities, and data analytics, wearable devices now enable interaction among users, devices, and their environment seamlessly. Multifunctional nature of this technology enables users to track their daily physical activities, engage with other users through social networking capabilities, and log their lifestyle habits. In this chapter, the authors discuss the types of sensor technologies embedded in wearable devices and how the data collected through such devices can be further interpreted by data analytics. In parallel with abundance of personal data that can be collected via wearable devices, they also discuss issues related to data privacy, suggestions for users, developers, and policymakers regarding how to protect data privacy are also discussed.
Article
Full-text available
Major depression is common, and imposes a high burden in terms of cost, morbidity, and suffering. Most people with depression are treated in general medicine using antidepressant medication. Outcomes are poor due to failure points across the care system, including patient non-adherence, failure of physicians to optimize the treatment regimens, and lack of patient-physician communication. This study reports on the 4-week pilot deployment of MedLink, a mobile intervention aimed at systemically addressing each of these failure points. A mobile app provides the patient with information and collects data on symptoms and side-effects. A cellularly enabled pill bottle monitors medication adherence. Data from these are provided to the physician and patient to foster communication and medication adjustments. Usability evaluation was generally favorable. Medication adherence rates in this first deployment were high with no patients discontinuing, and 84% of doses taken. Depressive symptom severity was significantly reduced. This study supports the use of a comprehensive, systemic approach to mHealth solutions to enhance processes of care for depression by general medicine physicians.
Article
Full-text available
Mood disorders are inherently related to emotion. In particular, the behaviour of people suffering from mood disorders such as unipolar depression shows a strong temporal correlation with the affective dimensions valence, arousal and dominance. In addition to structured self-report questionnaires, psychologists and psychiatrists use in their evaluation of a patient's level of depression the observation of facial expressions and vocal cues. It is in this context that we present the fourth Audio-Visual Emotion recognition Challenge (AVEC 2014). This edition of the challenge uses a subset of the tasks used in a previous challenge, allowing for more focussed studies. In addition, labels for a third dimension (Dominance) have been added and the number of annotators per clip has been increased to a minimum of three, with most clips annotated by 5. The challenge has two goals logically organised as sub-challenges: the first is to predict the continuous values of the affective dimensions valence, arousal and dominance at each moment in time. The second is to predict the value of a single self-reported severity of depression indicator for each recording in the dataset. This paper presents the challenge guidelines, the common data used, and the performance of the baseline system on the two tasks.
Article
Full-text available
Mobile sensor data-to-knowledge (MD2K) was chosen as one of 11 Big Data Centers of Excellence by the National Institutes of Health, as part of its Big Data-to-Knowledge initiative. MD2K is developing innovative tools to streamline the collection, integration, management, visualization, analysis, and interpretation of health data generated by mobile and wearable sensors. The goal of the big data solutions being developed by MD2K is to reliably quantify physical, biological, behavioral, social, and environmental factors that contribute to health and disease risk. The research conducted by MD2K is targeted at improving health through early detection of adverse health events and by facilitating prevention. MD2K will make its tools, software, and training materials widely available and will also organize workshops and seminars to encourage their use by researchers and clinicians.
Article
Full-text available
Bipolar disorder is one of the most common mood disorders characterized by large and invalidating mood swings. Several projects focus on the development of decision support systems that monitor and advise patients, as well as clinicians. Voice monitoring and speech signal analysis can be exploited to reach this goal. In this study, an Android application was designed for analyzing running speech using a smartphone device. The application can record audio samples and estimate speech fundamental frequency, F0, and its changes. F0-related features are estimated locally on the smartphone, with some advantages with respect to remote processing approaches in terms of privacy protection and reduced upload costs. The raw features can be sent to a central server and further processed. The quality of the audio recordings, algorithm reliability and performance of the overall system were evaluated in terms of voiced segment detection and features estimation. The results demonstrate that mean F0 from each voiced segment can be reliably estimated, thus describing prosodic features across the speech sample. Instead, features related to F0 variability within each voiced segment performed poorly. A case study performed on a bipolar patient is presented.
Article
Full-text available
Background Depression is a common, burdensome, often recurring mental health disorder that frequently goes undetected and untreated. Mobile phones are ubiquitous and have an increasingly large complement of sensors that can potentially be useful in monitoring behavioral patterns that might be indicative of depressive symptoms. Objective The objective of this study was to explore the detection of daily-life behavioral markers using mobile phone global positioning systems (GPS) and usage sensors, and their use in identifying depressive symptom severity. MethodsA total of 40 adult participants were recruited from the general community to carry a mobile phone with a sensor data acquisition app (Purple Robot) for 2 weeks. Of these participants, 28 had sufficient sensor data received to conduct analysis. At the beginning of the 2-week period, participants completed a self-reported depression survey (PHQ-9). Behavioral features were developed and extracted from GPS location and phone usage data. ResultsA number of features from GPS data were related to depressive symptom severity, including circadian movement (regularity in 24-hour rhythm; r=-.63, P=.005), normalized entropy (mobility between favorite locations; r=-.58, P=.012), and location variance (GPS mobility independent of location; r=-.58, P=.012). Phone usage features, usage duration, and usage frequency were also correlated (r=.54, P=.011, and r=.52, P=.015, respectively). Using the normalized entropy feature and a classifier that distinguished participants with depressive symptoms (PHQ-9 score ≥5) from those without (PHQ-9 score
Conference Paper
Full-text available
Much of the stress and strain of student life remains hidden. The StudentLife continuous sensing app assesses the day-to- day and week-by-week impact of workload on stress, sleep, activity, mood, sociability, mental well-being and academic performance of a single class of 48 students across a 10 week term at Dartmouth College using Android phones. Results from the StudentLife study show a number of significant cor- relations between the automatic objective sensor data from smartphones and mental health and educational outcomes of the student body. We also identify a Dartmouth term lifecycle in the data that shows students start the term with high pos- itive affect and conversation levels, low stress, and healthy sleep and daily activity patterns. As the term progresses and the workload increases, stress appreciably rises while posi- tive affect, sleep, conversation and activity drops off. The StudentLife dataset is publicly available on the web.
Article
Full-text available
Reduced frequency range in vowel production is a well documented speech characteristic of individuals with psychological and neurological disorders. Affective disorders such as depression and post-traumatic stress disorder (PTSD) are known to influence motor control and in particular speech production. The assessment and documentation of reduced vowel space and reduced expressivity often either rely on subjective assessments or on analysis of speech under constrained laboratory conditions (e.g. sustained vowel production, reading tasks). These constraints render the analysis of such measures expensive and impractical. Within this work, we investigate an automatic unsupervised machine learning based approach to assess a speaker’s vowel space. Our experiments are based on recordings of 253 individuals. Symptoms of depression and PTSD are assessed using standard self-assessment questionnaires and their cut-off scores. The experiments show a significantly reduced vowel space in subjects that scored positively on the questionnaires. We show the measure’s statistical robustness against varying demographics of individuals and articulation rate. The reduced vowel space for subjects with symptoms of depression can be explained by the common condition of psychomotor retardation influencing articulation and motor control. These findings could potentially support treatment of affective disorders, like depression and PTSD in the future.
Conference Paper
Full-text available
Major depression is common, and imposes a high burden in terms of cost, morbidity, and suffering. Most people with depression are treated in general medicine using antidepressant medication. Outcomes are poor due to failure points across the care system, including patient non-adherence, failure of physicians to optimize the treatment regimens, and lack of patient-physician communication. This study reports on the 4-week pilot deployment of MedLink, a mobile intervention aimed at systemically addressing each of these failure points. A mobile app provides the patient with information and collects data on symptoms and side-effects. A cellularly enabled pill bottle monitors medication adherence. Data from these are provided to the physician and patient to foster communication and medication adjustments. Usability evaluation was generally favorable. Medication adherence rates in this first deployment were high with no patients discontinuing, and 84% of doses taken. Depressive symptom severity was significantly reduced. This study supports the use of a comprehensive, systemic approach to mHealth solutions to enhance processes of care for depression by general medicine physicians.
Article
Full-text available
This paper reports on a novel algorithm for the analysis of electrodermal activity (EDA) using methods of convex optimization. EDA can be considered one of the most common observation channels of sympathetic nervous system activity, and manifests itself as a change in electrical properties of the skin, such as skin conductance (SC). The proposed model describes SC as the sum of three terms: the phasic component, the tonic component, and an additive white Gaussian noise term incorporating model prediction errors as well as measurement errors and artifacts. This model is physiologically inspired and fully explains EDA through a rigorous methodology based on Bayesian statistics, mathematical convex optimization and sparsity. The algorithm was evaluated in three different experimental sessions to test its robustness to noise, its ability to separate and identify stimulus inputs, and its capability of properly describing the activity of the autonomic nervous system in response to strong affective stimulation. Results are very encouraging, showing good performance of the proposed method and suggesting promising future applicability, e.g., in the field of affective computing.
Conference Paper
Full-text available
The monitoring of sleep by quantifying sleeping time and quality is pivotal in many preventive health care scenarios. A substantial amount of wearable sensing products have been introduced to the market for just this reason, detecting whether the user is either sleeping or awake. Assessing these devices for their accuracy in estimating sleep is a daunting task, as their hardware design tends to be different and many are closed-source systems that have not been clinically tested. In this paper, we present a challenging benchmark dataset from an open source wrist-worn data logger that contains relatively high-frequent (100Hz) 3D inertial data from 42 sleep lab patients, along with their data from clinical polysomnography. We analyse this dataset with two traditional approaches for detecting sleep and wake states and propose a new algorithm specifically for 3D acceleration data, Estimation of Stationary Sleep-segments (ESS). Results show that all three methods generally over-estimate for sleep, with our method performing slightly better (74% overall mean accuracy) than the traditional activity count-based methods.
Article
Full-text available
This paper presents a series of challenges for developing mobile health solutions for mental health as a result of MONARCA project three-year activities. The lessons learnt on the design, development and evaluation of a mobile health system for supporting the treatment of bipolar disorder. The findings presented here are the result of over 3 years of activity within the MONARCA EU project. The challenges listed and detailed in this paper may be used in future research as a starting point for identifying important non-functional requirements involved in mobile health provisioning that are fundamental for the successful implementation of mobile health services in real life contexts.
Conference Paper
Full-text available
The idea of continuously monitoring well-being using mobile-sensing systems is gaining popularity. In-situ measurement of human behavior has the potential to overcome the short comings of gold-standard surveys that have been used for decades by the medical community. However, current sensing systems have mainly focused on tracking physical health; some have approximated aspects of mental health based on proximity measurements but have not been compared against medically accepted screening instruments. In this paper, we show the feasibility of a multi-modal mobile sensing system to simultaneously assess mental and physical health. By continuously capturing fine-grained motion and privacy-sensitive audio data, we are able to derive different metrics that reflect the results of commonly used surveys for assessing well-being by the medical community. In addition, we present a case study that highlights how errors in assessment due to the subjective nature of the responses could potentially be avoided by continuous mobile sensing.
Article
Full-text available
Experience Sampling Methodology (ESM) is ideally suited to test the predictions, and inform the development of contemporary cognitive models of depression. Yet there has been no systematic examination of ESM in depression research. A search of databases (PsychARTICLES, PsycINFO, AMED, Ovid Medline and CINAHL) was conducted to identify studies published within the last 25 years investigating major depressive disorder (MDD) using ESM. Altogether, 19 studies using ESM, or comparable methodologies, with clinically depressed individuals were identified and critically reviewed. The identified studies examined six aspects of MDD: methodological issues; positive and negative affect; cortisol secretion; antidepressant treatment; work performance; genetic risk factors. Despite some methodological limitations of existing studies, ESM has made a significant contribution to our current understanding of depression by consolidating existing theories, uncovering new and clinically relevant findings and identifying questions for future research. This review concludes by introducing the possibility of using ESM as an intervention tool in clinical practice and proposing that ESM could be useful for furthering knowledge of the causes of MDD.
Article
Full-text available
Understanding the neurocircuitry that underlies normal and adaptive behaviors is a foundation for elucidating the pathology and pathophysiology of psychiatric diseases. The rich history of anatomical observations forms the backbone for what we now consider to be the key structures and pathways associated with aberrant behaviors and psychiatric disorders. Given the focus of psychiatry on the interface of motivation, emotion, and cognition, psychiatric neuroscience has progressively highlighted brain networks that subserve these functional domains and their interactions. The convergence of findings from the variety of techniques in animal studies to human postmortem, neurosurgical, and brain imaging methods have led to major advances that constitute the groundwork for the neurocircuitry pertinent to psychiatric diseases.
Article
Full-text available
Use of real-time data collection is rapidly expanding in the medical sciences and questions have been raised as to whether frequent ratings of disease symptoms could evoke depressed mood. This study investigated the effect of an intensive momentary assessment protocol on depressed mood. Community rheumatology patients (N = 105) were recruited to participate in a 30-day momentary assessment protocol of pain and fatigue. Patients were randomly signaled and completed approximately 6 ratings per day and at bedtime. Beck Depression Inventory-II scores were obtained prior to and at the completion of the protocol. Thirty-six percent of patients were classified initially as mild to severely depressed, and 31% percent at the end of the protocol. Depression scores were significantly lower following the protocol (p < .001). Whereas 10% of patients shifted into a more depressed category at the end of the protocol, 20% shifted into a less depressed category. These findings suggest frequent assessment of pain and fatigue may not induce depressed mood, and may in some instances be associated with a small reduction in depressed mood.
Article
Full-text available
In this paper we describe LiveNet, a flexible wearable platform intended for long-term ambulatory health monitoring with real-time data streaming and context classification. Based on the MIT Wearable Computing Group's distributed mobile system architecture, LiveNet is a stable, accessible system that combines inexpensive, commodity hardware; a flexible sensor/peripheral interconnection bus; and a powerful, light-weight distributed sensing, classification, and inter-process communications software architecture to facilitate the development of distributed real-time multi-modal and context-aware applications. LiveNet is able to continuously monitor a wide range of physiological signals together with the user's activity and context, to develop a personalized, data-rich health profile of a user over time. We demonstrate the power and functionality of this platform by describing a number of health monitoring applications using the LiveNet system in a variety of clinical studies that are underway. Initial evaluations of these pilot experiments demonstrate the potential of using the LiveNet system for real-world applications in rehabilitation medicine.
Article
In order to model students' happiness, we apply machine learning methods to data collected from undergrad students monitored over the course of one month each. The data collected include physiological signals, location, smartphone logs, and survey responses to behavioral questions. Each day, participants reported their wellbeing on measures including stress, health, and happiness. Because of the relationship between happiness and depression, modeling happiness may help us to detect individuals who are at risk of depression and guide interventions to help them. We are also interested in how behavioral factors (such as sleep and social activity) affect happiness positively and negatively. A variety of machine learning and feature selection techniques are compared, including Gaussian Mixture Models and ensemble classification. We achieve 70% classification accuracy of self-reported happiness on held-out test data.
Article
Depression is one of the most common psychiatric disorders worldwide, with over 350 million people affected. Current methods to screen for and assess depression depend almost entirely on clinical interviews and self-report scales. While useful, such measures lack objective, systematic, and efficient ways of incorporating behavioral observations that are strong indicators of depression presence and severity. Using dynamics of facial and head movement and vocalization, we trained classifiers to detect three levels of depression severity. Participants were a community sample diagnosed with major depressive disorder. They were recorded in clinical interview (Hamilton Rating Scale for Depression, HRSD) at 7-week intervals over a period of 21 weeks. At each interview, they were scored by HRSD as moderately to severely depressed, mildly depressed, or remitted. Logistic regression classifiers using leave-one-participant-out validation were compared for facial movement, head movement, and vocal prosody individually and in combination. Accuracy of depression severity measurement from facial movement dynamics was higher than that for head movement dynamics; and each was substantially higher than that for vocal prosody. Accuracy using all three modalities combined only marginally exceeded that of face and head combined. These findings suggest that automatic detection of depression severity from behavioral indicators in patients is feasible and that multimodal measures afford most powerful detection.
Conference Paper
In order to model students' happiness, we apply machine learning methods to data collected from undergrad students monitored over the course of one month each. The data collected include physiological signals, location, smartphone logs, and survey responses to behavioral questions. Each day, participants reported their wellbeing on measures including stress, health, and happiness. Because of the relationship between happiness and depression, modeling happiness may help us to detect individuals who are at risk of depression and guide interventions to help them. We are also interested in how behavioral factors (such as sleep and social activity) affect happiness positively and negatively. A variety of machine learning and feature selection techniques are compared, including Gaussian Mixture Models and ensemble classification. We achieve 70% classification accuracy of self-reported happiness on held-out test data.
Article
There is growing amount of scientific evidence that motor activity is the most consistent indicator of bipolar disorder. Motor activity includes several areas such as body movement, motor response time, level of psychomotor activity, and speech related motor activity. Studies of motor activity in bipolar disorder have typically used self-reported questionnaires with clinical observer-rated scales, which are therefore subjective and have often limited effectiveness. Motor activity information can be used to classify episode type inbipolar patients, which is highly relevant, since severe depression and manic states can result in mortality. This paper introduces a system able to classify the state of patients suffering from bipolar disorder using sensed information from smartphones. We collected audio, accelerometer and self-assessment data from five patients over a time-period of 12 weeks during their real-life activities. In this research we evaluated the performance of several classifiers, different sets of features and the role of the questionnaires for classifying bipolar disorder episodes. In particular, we have shown that it is possible to classify with high confidence ( 85%) the course of mood episodes or relapse in bipolar patients. To our knowledge, no research to date has focused on naturalistic observation of day-to-day phone conversation to classify impaired life functioning in individuals with bipolar disorder.
Article
Using “big data” from sensors worn continuously outside the lab, researchers have observed patterns of objective physiology that challenge some of the long-standing theoretical concepts of emotion and its measurement. One challenge is that emotional arousal, when measured as sympathetic nervous system activation through electrodermal activity, can sometimes differ significantly across the two halves of the upper body. We show that traditional measures on only one side may lead to misjudgment of arousal. This article presents daily life and controlled study data, as well as existing evidence from neuroscience, supporting the influence of multiple emotional substrates in the brain causing innervation on different sides of the body. We describe how a theory of multiple arousals explains the asymmetric EDA findings.
Article
Serious mental illnesses are among the most pressing public healthcare concerns. Continuous and unobtrusive sensing of social and physical functioning has tremendous potential to support lifelong health management by acting as an early warning system to detect changes in mental well-being, delivering context-aware micro-interventions to patients when and where they need them and significantly accelerating patients' understanding of their illness. We have presented a range of techniques from our previous and ongoing work to suggest how technologists can make significant advances in this area. While we are still shaping and evaluating these solutions, there is an opportunity for other researchers in our field to help ease the burden of SMIs by striking a balance between cutting-edge sensing and patient needs. This could empower patients by giving them a hand in their own treatment and ultimately lead to more effective, lower-cost treatment.
Article
The nine-item Patient Health Questionnaire depression scale is a dual-purpose instrument that can establish provisional depressive disorder diagnoses as well as grade depression severity.
Article
In recent studies of the structure of affect, positive and negative affect have consistently emerged as two dominant and relatively independent dimensions. A number of mood scales have been created to measure these factors; however, many existing measures are inadequate, showing low reliability or poor convergent or discriminant validity. To fill the need for reliable and valid Positive Affect and Negative Affect scales that are also brief and easy to administer, we developed two 10-item mood scales that comprise the Positive and Negative Affect Schedule (PANAS). The scales are shown to be highly internally consistent, largely uncorrelated, and stable at appropriate levels over a 2-month time period. Normative data and factorial and external evidence of convergent and discriminant validity for the scales are also presented. (PsycINFO Database Record (c) 2010 APA, all rights reserved)
Article
The mean resting skin conductance level (SCL) in 31 inpatients with major unipolar depression was 2.63 mumho/cm2 +/- SD 1.41 compared to 7.81 mumho/cm2 +/- SD 3.0 in the control group. The best criterion below which an SCL could be considered abnormal was 4.3 mumho/cm2 with a resulting sensitivity of 87% and specificity of 89%. There was a significant difference in SCL between males and females but none between drug-free and medicated patients. SCL was abnormal in all depression subtypes. The SCL may represent a biological final common pathway in depression and may prove to be a very effective diagnostic test for depression.
Multi-modal behavioral biometrics based on hci and electrophysiology
  • H F S Gamboa