Thomas Ploetz

Thomas Ploetz
Georgia Institute of Technology | GT · School of Interactive Computing

PhD (Computer Science)

About

204
Publications
83,042
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
5,317
Citations
Additional affiliations
February 2017 - present
Georgia Institute of Technology
Position
  • Professor (Associate)
August 2016 - January 2017
Newcastle University
Position
  • Professor (Full)
February 2014 - July 2016
Newcastle University
Position
  • Lecturer

Publications

Publications (204)
Article
Deviant eating behavior such as skipping meals and consuming unhealthy meals has a significant association with mental well-being in college students. However, there is more to what an individual eats. While eating patterns form a critical component of their mental well-being, insights and assessments related to the interplay of eating patterns and...
Preprint
The emergence of self-supervised learning in the field of wearables-based human activity recognition (HAR) has opened up opportunities to tackle the most pressing challenges in the field, namely to exploit unlabeled data to derive reliable recognition systems from only small amounts of labeled training samples. Furthermore, self-supervised methods...
Article
Sleep is a fundamental physiological process that is essential for sustaining a healthy body and mind. The gold standard for clinical sleep monitoring is polysomnography(PSG), based on which sleep can be categorized into five stages, including wake/rapid eye movement sleep (REM sleep)/Non-REM sleep 1 (N1)/Non-REM sleep 2 (N2)/Non-REM sleep 3 (N3)....
Article
Full-text available
Supervised training of human activity recognition (HAR) systems based on body-worn inertial measurement units (IMUs) is often constrained by the typically rather small amounts of labeled sample data. Systems like IMUTube have been introduced that employ cross-modality transfer approaches to convert videos of activities of interest into virtual IMU...
Preprint
Full-text available
Sleep is a fundamental physiological process that is essential for sustaining a healthy body and mind. The gold standard for clinical sleep monitoring is polysomnography(PSG), based on which sleep can be categorized into five stages, including wake/rapid eye movement sleep (REM sleep)/Non-REM sleep 1 (N1)/Non-REM sleep 2 (N2)/Non-REM sleep 3 (N3)....
Article
Full-text available
Today's smartphones and wearable devices come equipped with an array of inertial sensors, along with IMU-based Human Activity Recognition models to monitor everyday activities. However, such models rely on large amounts of annotated training data, which require considerable time and effort for collection. One has to recruit human subjects, define c...
Article
Recently, IMUTube introduced a paradigm change for bootstrapping human activity recognition (HAR) systems for wearables. The key idea is to utilize videos of activities to support training activity recognizers based on inertial measurement units (IMUs). This system retrieves video from public repositories and subsequently generates virtual IMU data...
Article
With the widespread proliferation of (miniaturized) sensing facilities and the massive growth and popularity of the field of machine learning (ML) research, new frontiers in automated sensor data analysis have been explored that lead to paradigm shifts in many application domains. In fact, many practitioners now employ and rely more and more on ML...
Article
Feature extraction is crucial for human activity recognition (HAR) using body-worn movement sensors. Recently, learned representations have been used successfully, offering promising alternatives to manually engineered features. Our work focuses on effective use of small amounts of labeled data and the opportunistic exploitation of unlabeled data t...
Chapter
Full-text available
Building activity recognition systems conventionally involves training a common model from all data of training users and utilizing this model to recognize activities of unseen subjects. However, participants come from diverse demographics, so that different users can perform the same actions in diverse ways. Each subject might exhibit user-specifi...
Article
Full-text available
Human activity recognition is progressing from automatically determining what a person is doing and when, to additionally analyzing the quality of these activities—typically referred to as skill assessment. In this chapter, we propose a new framework for skill assessment that generalizes across application domains and can be deployed for near-real-...
Article
Research in sensor based human activity recognition (HAR) has been a core concern of the mobile and ubiquitous computing community. Sophisticated systems have been developed with the main view on applications of HAR methods in research settings. This work addresses a related yet practically different problem that mainly focuses on users of HAR tech...
Article
Full-text available
Background: Eating behavior has a high impact on the well-being of an individual. Such behavior involves not only when an individual is eating, but also various contextual factors such as with whom and where an individual is eating and what kind of food the individual is eating. Despite the relevance of such factors, most automated eating detection...
Preprint
Feature extraction is crucial for human activity recognition (HAR) using body-worn movement sensors. Recently, learned representations have been used successfully, offering promising alternatives to manually engineered features. Our work focuses on effective use of small amounts of labeled data and the opportunistic exploitation of unlabeled data t...
Article
The lack of large-scale, labeled data sets impedes progress in developing robust and generalized predictive models for on-body sensor-based human activity recognition (HAR). Labeled data in human activity recognition is scarce and hard to come by, as sensor data collection is expensive, and the annotation is time-consuming and error-prone. To addre...
Conference Paper
Full-text available
Self-esteem encompasses how individuals evaluate themselves and is an important contributor to their success. Self-esteem has been traditionally measured using survey-based methodologies. However , surveys suffer from limitations such as retrospective recall and reporting biases, leading to a need for proactive measurement approaches. Our work uses...
Preprint
While activity recognition from inertial sensors holds potential for mobile health, differences in sensing platforms and user movement patterns cause performance degradation. Aiming to address these challenges, we propose a transfer learning framework, TransFall, for sensor-based activity recognition. TransFall's design contains a two-tier data tra...
Preprint
Full-text available
The lack of large-scale, labeled data sets impedes progress in developing robust and generalized predictive models for on-body sensor-based human activity recognition (HAR). Labeled data in human activity recognition is scarce and hard to come by, as sensor data collection is expensive, and the annotation is time-consuming and error-prone. To addre...
Preprint
BACKGROUND Eating behavior has a significant impact on the wellbeing of an individual. Such behavior comprises not only when an individual is eating, but also various contextual factors such as with whom and where an individual is eating, what kind of food they are having, to name but a few. Despite the significance of such factors, most automated...
Article
Full-text available
Background Eating behavior has a high impact on the well-being of an individual. Such behavior involves not only when an individual is eating, but also various contextual factors such as with whom and where an individual is eating and what kind of food the individual is eating. Despite the relevance of such factors, most automated eating detection...
Preprint
BACKGROUND This paper describes a semi-automated eating detection system that leverages Ecological Momentary Assessment (EMA) questions to capture contextual factors upon detecting when an individual is eating. Our validation study demonstrates the efficacy of the system by deploying it in-the-wild among college students. OBJECTIVE This study buil...
Preprint
Full-text available
On university campuses, social interactions among students can explain their academic experiences. However, assessing these interactions with surveys fails to capture their dynamic nature. While these behaviors can be captured with client-based passive sensing, these techniques are limited in scalability. By contrast, infrastructure-based approache...
Article
Full-text available
Background Late-life depression (LLD) is associated with poor social functioning. However, previous research uses bias-prone self-report scales to measure social functioning and a more objective measure is lacking. We tested a novel wearable device to measure speech that participants encounter as an indicator of social interaction. Methods Twenty...
Chapter
Perinatal stroke (PS) is a serious condition that often leads to life-long disability, in particular cerebral palsy (CP). Early detection and early intervention could improve motor outcome. In clinical settings, Prechtl’s general movement assessment (GMA) can be used to classify infant movements using a Gestalt approach, identifying infants at high...
Conference Paper
Affective computing aims to detect a person's affective state (e.g. emotion) based on observables. The link between affective states and biophysical data, collected in lab settings, has been established successfully. However, the number of realistic studies targeting affect detection in the wild is still limited. In this paper we present an explora...
Conference Paper
Full-text available
Developing systems for Human Activity Recognition (HAR) using wearables typically relies on datasets that were manually annotated by human experts with regards to precise timings of instances of relevant activities. However, obtaining such data annotations is often very challenging in the predominantly mobile scenarios of Human Activity Recognition...
Conference Paper
Full-text available
Traditionally, the sliding window based activity recognition chain (ARC) has been dominating practical applications, in which features are carefully optimized towards scenario specifics. Recently, end-to-end, deep learning methods, that do not discriminate between representation learning and classifier optimization, have become very popular also fo...
Article
Full-text available
Mental health issues, which can be difficult to diagnose, are a growing concern worldwide. For effective care and support, early detection of mood-related health concerns is of paramount importance. Typically, survey based instruments including Ecologically Momentary Assessments (EMA) and Day Reconstruction Method (DRM) are the method of choice for...
Article
Full-text available
Physical contact is critical for children's physical and emotional growth and well-being. Previous studies of physical contact are limited to relatively short periods of direct observation and self-report methods. These methods limit researchers' understanding of the natural variation in physical contact across families, and its specific impacts on...
Article
Full-text available
Perinatal stroke (PS) is a serious condition that, if undetected and thus untreated, often leads to life-long disability, in particular Cerebral Palsy (CP). In clinical settings, Prechtl's General Movement Assessment (GMA) can be used to classify infant movements using a Gestalt approach, identifying infants at high risk of developing PS. Training...
Preprint
Full-text available
Perinatal stroke (PS) is a serious condition that, if undetected and thus untreated, often leads to life-long disability, in particular Cerebral Palsy (CP). In clinical settings, Prechtl's General Movement Assessment (GMA) can be used to classify infant movements using a Gestalt approach, identifying infants at high risk of developing PS. Training...
Article
We address the use of accelerometery to automatically monitor lying behaviour in free-farrowing sows; due to their freedom of movement and the consequent increased variety of movements the sows are able to exhibit, the challenges in automating this are greater than in sows housed in movement restricting farrowing environments. The methodology devel...
Article
Full-text available
Mobile phone data are an interesting new data source for official statistics. However, multiple problems and uncertainties need to be solved before these data can inform, support or even become an integral part of statistical production processes. In this article, we focus on arguably the most important problem hindering the application of mobile p...
Preprint
Full-text available
Gait as a biometric trait has attracted much attention in many security and privacy applications such as identity recognition and authentication, during the last few decades. Because of its nature as a long-distance biometric trait, gait can be easily collected and used to identify individuals non-intrusively through CCTV cameras. However, it is ve...
Conference Paper
We present SeeSaw, a synchronous gesture interface for commodity smartwatches to support watch-hand only input with no additional hardware. Our algorithm, which uses correlation to determine whether the user is rotating their wrist in synchrony with a tactile and visual prompt, minimizes false-trigger events while maintaining fast input during situ...
Conference Paper
Full-text available
Deep Learning methods have become very attractive in the wider, wearables-based human activity recognition (HAR) research community. The majority of models are based on either convolutional or explicitly temporal models, or combinations of both. In this paper we introduce attention models into HAR research as a data driven approach for exploring re...
Conference Paper
Full-text available
Washing hands is one of the easiest yet most effective ways to prevent spreading illnesses and diseases. However, not adhering to thorough handwashing routines is a substantial problem worldwide. For example, in hospital operations lack of hygiene leads to healthcare associated infections. We present WristWash, a wrist-worn sensing platform that in...
Conference Paper
Sliding window based activity recognition chains represent the state-of-the-art for many mobile and embedded scenarios as they are common in wearable computing. The length of the analysis frames is a crucial system parameter that directly influences the effectiveness of the overall approach. In this paper we present a method that optimizes the wind...
Preprint
Full-text available
In this contribution we summarize insights on the geographical veracity of using mobile phone data to create (statistical) indicators. We focus on problems that persist with spatial allocation, spatial delineation and spatial aggregation of information obtained from mobile phone data. For each of the cases, we offer insights from our works on a Fre...
Preprint
Full-text available
Mobile phone data are an interesting new data source for official statistics. However, multiple problems and uncertainties need to be solved before these data can inform, support or even become an integral part of statistical production processes. In this paper, we focus on arguably the most important problem hindering the application of mobile pho...
Preprint
Full-text available
Non-continuous location traces inferred from Call Detail Records (CDR) at population scale are increasingly becoming available for research and show great potential for automated detection of meaningful places. Yet, a majority of Home Detection Algorithms (HDAs) suffer from "blind" deployment of criteria to define homes and from limited possibiliti...
Conference Paper
Full-text available
Feature extraction is a critical step in sliding-window based standard activity recognition chains. Recently, distribution based features have been introduced that showed excellent generalization capabilities across a wide range of application domains in human activity recognition scenarios based on body-worn sensors. These features capture the dat...
Article
Full-text available
We designed and evaluated an assumption-free, deep learning-based methodology for animal health monitoring, specifically for the early detection of respiratory disease in growing pigs based on environmental sensor data. Two recurrent neural networks (RNNs), each comprising gated recurrent units (GRUs), were used to create an autoencoder (GRU-AE) in...
Article
Extensions to auto-context segmentation are proposed and applied to segmentation of multiple organs in porcine offal as a component of an envisaged system for post-mortem inspection at abbatoir. In common with multi-part segmentation of many biological objects, challenges include variations in configuration, orientation, shape, and appearance, as w...
Article
Introduction/Background Encouraging impaired limb use during routine and rehabilitation activities after stroke is challenging. We evaluated changes in stroke arm activity related to vibration prompts delivered by a wrist-worn accelerometer (“CueS wristband”). Material and method A pilot randomised controlled trial was conducted with adults < 3 mo...
Article
Full-text available
In this paper, we propose a correction of the Mobility Entropy indicator (ME) used to describe the diversity of individual movement patterns as can be captured by data from mobile phones. We argue that a correction is necessary because standard calculations of ME show a structural dependency on the geographical density of observation points, render...
Preprint
Taller and sleeker smartphone devices are becoming the new norm. More screen space and very responsive touchscreens have made for enjoyable experiences available to us at all times. However, after years of interacting with smaller, portable devices, we still try to use these large smartphones on the go, and do not want to change how, where, and whe...
Preprint
Full-text available
Most approaches that model time-series data in human activity recognition based on body-worn sensing (HAR) use a fixed size temporal context to represent different activities. This might, however, not be apt for sets of activities with individ- ually varying durations. We introduce attention models into HAR research as a data driven approach for ex...
Conference Paper
FingerPing is a novel sensing technique that can recognize various fine-grained hand poses by analyzing acoustic resonance features. A surface-transducer mounted on a thumb ring injects acoustic chirps (20Hz to 6,000Hz) to the body. Four receivers distributed on the wrist and thumb collect the chirps. Different hand poses of the hand create distinc...
Article
The annual International Symposium on Wearable Computers (ISWC) is the flagship conference on all topics related to wearable computing and the ideal venue to present and learn about cutting-edge research in the field. The authors share their observations from the most recent gathering, held in September 2017 in Maui, Hawaii.
Article
Full-text available
Background: Frequent practice of functional movements after stroke may optimise motor recovery; however, it is challenging for patients to remember to integrate an impaired limb into daily activities. We report the activity responses of stroke patients receiving a vibrating alert delivered by a tri-axial accelerometer wristband to prompt movement...
Article
Full-text available
User identification on interactive surfaces is a desirable feature that is not inherently supported by existing technologies. We have conducted an extensive survey of existing identification techniques, which led us to formulate a unified model for user identification. We start by introducing this model that (1) classifies existing user identificat...
Article
Full-text available
In this work, we discuss how an existing algorithm to extract long-distance trips from mobile phone data (Janzen et al., 2016 a,b) can be supplemented with man-made heuristics to arrive at plausible domestic tourism trips. In total, we detect 18,380 domestic tourism trips from mobile phone data of 69,000 users sampled in 32 cities in France. By ana...
Article
Full-text available
Since animals express their internal state through behaviour, changes in said behaviour may be used to detect early signs of problems, such as in animal health. Continuous observation of livestock by farm staff is impractical in a commercial setting to the degree required to detect behavioural changes relevant for early intervention. An automated m...
Article
Full-text available
Background There is increasing interest in the definition, measurement and use of traits associated with water use and drinking behaviour, mainly because water is a finite resource and its intake is an important part of animal health and well-being. Analysis of such traits has received little attention, due in part to the lack of appropriate techno...
Conference Paper
We present FingOrbits, a wearable interaction technique using synchronized thumb movements. A thumb-mounted ring with an inertial measurement unit and a contact microphone are used to capture thumb movements when rubbing against the other fingers. Spectral information of the movements are extracted and fed into a classification backend that facilit...
Article
We introduce FingerSound, an input technology to recognize unistroke thumb gestures, which are easy to learn and can be performed through eyes-free interaction. The gestures are performed using a thumb-mounted ring comprising a contact microphone and a gyroscope sensor. A K-Nearest-Neighbor(KNN) model with a distance function of Dynamic Time Warpin...
Article
Full-text available
Quality assessment in cricket is a complex task that is performed by understanding the combination of individual activities a player is able to perform and by assessing how well these activities are performed. We present a framework for inexpensive and accessible, automated recognition of cricketing shots. By means of body-worn inertial measurement...