Ermal Toto’s research while affiliated with Worcester Polytechnic Institute and other places

Mental illnesses are often undiagnosed, highlighting the need for an effective alternative to traditional screening surveys. We propose our Early Mental Health Uncovering (EMU) framework that conducts rapid mental illness screening with active and passive modalities. We designed, deployed, and evaluated the EMU app to passively collect retrospective digital phenotype data and actively collect short voice recordings. The EMU app also administered a depression screening survey to label the data. We collected data from crowdsourced and student populations, both of whom shared sufficient voice recordings for modeling. We thus assess the classification ability of machine learning and deep learning models trained with scripted and unscripted voice recordings. For the crowdsourced participants, machine learning models screened for depression with an AUC of 0.78 and suicidal ideation with an AUC of 0.73. For the student participants, deep learning models screened for depression with an AUC of 0.70 and suicidal ideation with an AUC of 0.72. Combining datasets did not improve screening capabilities, though the best performing models on the combined dataset notably required voice transcripts. This research facilitates a better understanding of modality selection for mobile screening. We will make the features publicly available to further advance mental illness screening research.

Depression is a common mental health disorder with large social and economic consequences. It can be costly and difficult to detect, traditionally requiring hours of assessment by a trained clinical. Recently, machine learning models have been trained to screen for depression with patient voice recordings collected during an interview with a virtual agent. To engage the patient in a conversation and increase the quantity of responses, the virtual interviewer asks a series of follow-up questions. However, asking fewer questions would reduce the time burden of screening for the participant. We, therefore, assess if these follow-up questions have a tangible impact on the performance of deep learning models for depression classification. Specifically, we study the effect of including the vocal and transcribed replies to one, two, three, four, five, or all follow-up questions in the depression screening models. We notably achieve this using unimodal and multimodal pre-trained transfer learning models. Our findings reveal that follow-up questions can help increase F1 scores for the majority of the interview questions. This research can be leveraged for the design of future mental illness screening applications by providing important information about both question selection and the best number of follow-up questions.

The growing prevalence of depression and suicidal ideation among college students further exacerbated by the Coronavirus pandemic is alarming, highlighting the need for universal mental illness screening technology. With traditional screening questionnaires too burdensome to achieve universal screening in this population, data collected through mobile applications has the potential to rapidly identify at-risk students. While prior research has mostly focused on collecting passive smartphone modalities from students, smartphone sensors are also capable of capturing active modalities. The general public has demonstrated more willingness to share active than passive modalities through an app, yet no such dataset of active mobile modalities for mental illness screening exists for students. Knowing which active modalities hold strong screening capabilities for student populations is critical for developing targeted mental illness screening technology. Thus, we deployed a mobile application to over 300 students during the COVID-19 pandemic to collect the Student Suicidal Ideation and Depression Detection (StudentSADD) dataset. We report on a rich variety of machine learning models including cutting-edge multimodal pretrained deep learning classifiers on active text and voice replies to screen for depression and suicidal ideation. This unique StudentSADD dataset is a valuable resource for the community for developing mobile mental illness screening tools.

Depression is a leading cause of disability and is associated with suicide risk. However, a quarter of patients with major depression remain undiagnosed. Prior work has demonstrated that a smartphone user's depression level can be detected by analyzing data gathered from their smartphone's sensors or from their social media posts over a few weeks after enrollment in a user study. These studies typically utilize a prospective study design, which is burdensome as it requires participants smartphone data to be gathered for prolonged periods before their depression level can be assessed. In contrast, we present a feasibility study of our Mood Assessment Capable Framework (Moodable) that facilitates almost instantaneous mood assessment by analyzing instantaneous voice samples provided by the user as well as historical sensor data harvested (scraped) from their smartphone and recent social media posts. Our retrospective, low-burden approach means that Moodable no longer requires study participants to engage with their phone for weeks before a depression score can be inferred. Moodable has the potential to minimize user data collection burden, increase user compliance, avoid study awareness bias and offer a near instantaneous depression screening. To lay a solid foundation for Moodable, we first surveyed 202 volunteer participants about their willingness to share voice samples and various smartphone and social media data types for mental health assessment. Based on these findings, we then developed the Moodable app. Thereafter, we utilized Moodable to collect short voice samples, and a rich array of retrospectively harvested data from users' smartphones (location, browser history, call logs) and social media accounts (instagram, twitter and facebook), with appropriate permissions, of 335 volunteer participants who also responded to 9 depression related questions of the Patient Health Questionaire (PHQ-9). Moodable then used machine learning to build classification models and classify the user's depression and suicidal ideation, for users which scores where unknown to the models. Results of Moodable's screening capability are promising. In particular, for the depression classification task we achieved F1 scores (the harmonic mean of the precision and recall) of 0.766, sensitivity of 0.750, and specificity of 0.792. For the suicidal ideation task we achieved F1 scores of 0.848, sensitivity of 0.864, and specificity of 0.725. This work could significantly increase depression-screening at the population level and opens numerous avenues for further research into this newly proposed paradigm of instantaneously screening depression and suicide risk levels from voice samples and retrospective smartphone and social media data.