Paul Lukowicz

Deutsches Forschungszentrum für Künstliche Intelligenz | DFKI · Embedded Intelligence

Sensor-based 3D hand tracking is still challenging despite the massive exploration of different sensing modalities in the past decades. This work describes the design, implementation, and evaluation of a novel induced magnetic field-based 3D hand tracking system, aiming to address the shortcomings of existing approaches and supply an alternative so...

While sports activity recognition is a well studied subject in mobile, wearable and ubiquitous computing, work to date mostly focuses on recognition and counting of specific exercise types. Quality assessment is a much more difficult problem with significantly less published results. In this work, we present Quali-Mat: a method for evaluating the q...

Human activity recognition (HAR) has become an intensive research topic in the past decade because of the pervasive user scenarios and the overwhelming development of advanced algorithms and novel sensing approaches. Previous HAR-related sensing surveys were primarily focused on either a specific branch such as wearable sensing and video-based sens...

We present a novel intelligent garment design approach for body posture/gesture detection in the form of a loose-fitting blazer prototype, “the MoCaBlazer.” The design is realized by leveraging conductive textile antennas with the capacitive sensing modality, supported by an open-source electronic theremin system (OpenTheremin). The use of soft tex...

The estimation of 3D human body shape and clothing measurements is crucial for virtual try-on and size recommendation problems in the fashion industry but has always been a challenging problem due to several conditions, such as lack of publicly available realistic datasets, ambiguity in multiple camera resolutions, and the undefinable human shape s...

Inertial measurement unit (IMU) is currently the dominant sensing modality in sensor-based wearable human activity recognition. In this work, we explored an alternative wearable motion-sensing approach: inferring motion information of various body parts from the human body capacitance (HBC). While being less robust in tracking the body motions, HBC...

Document image enhancement and binarization methods are often used to improve the accuracy and efficiency of document image analysis tasks such as text recognition. Traditional non-machine-learning methods are constructed on low-level features in an unsupervised manner but have difficulty with binarization on documents with severely degraded backgr...

The reliable assessment of muscle states, such as contracted muscles vs. non-contracted muscles or relaxed muscles vs. fatigue muscles, is crucial in many sports and rehabilitation scenarios, such as the assessment of therapeutic measures. The goal of this work was to deploy machine learning (ML) models based on one-dimensional (1-D) sonomyography...

User dependence remains one of the most difficult general problems in Human Activity Recognition (HAR), in particular when using wearable sensors. This is due to the huge variability of the way different people execute even the simplest actions. In addition, detailed sensor fixtures and placement will be different for different people or even at di...

Multiple debates were held on TV in the 2021 German federal election campaign between the chancellor candidates Armin Laschet (CDU/CSU), Olaf Scholz (SPD) and Annelena Baerbock (Bündnis 90/Die Grünen) . For the last three televised debates (so-called „Trielle“), surveying viewers immediately before and immediately after the events and conducting re...

Bearing is a key component in industrial machinery and its failure may lead to unwanted downtime and economic loss. Hence, it is necessary to predict the remaining useful life (RUL) of bearings. Conventional data-driven approaches of RUL prediction require expert domain knowledge for manual feature extraction and may suffer from data distribution d...

The rapid dynamics of COVID-19 calls for quick and effective tracking of virus transmission chains and early detection of outbreaks, especially in the “phase 2” of the pandemic, when lockdown and other restriction measures are progressively withdrawn, in order to avoid or minimize contagion resurgence. For this purpose, contact-tracing apps are bei...

Body capacitance change is an interesting signal for a variety of body sensor network applications in activity recognition. Although many promising applications have been published, capacitive on body sensing is much less understood than more dominant wearable sensing modalities such as IMUs and has been primarily studied in individual, constrained...

User dependence remains one of the most difficult general problems in Human Activity Recognition (HAR), in particular when using wearable sensors. This is due to the huge variability of the way different people execute even the simplest actions. In addition, detailed sensor fixtures and placement will be different for different people or even at di...

Inertial Measurement Unit (IMU) is currently the dominant sensing modality in sensor-based wearable human activity recognition. In this work, we explored an alternative wearable motion-sensing approach: inferring motion information of various body parts from the human body capacitance (HBC). While being less robust in tracking the body motions, HBC...

Bearing is a key component in industrial machinery and its failure may lead to unwanted downtime and economic loss. Hence, it is necessary to predict the remaining useful life (RUL) of bearings. Conventional data-driven approaches of RUL prediction require expert domain knowledge for manual feature extraction and may suffer from data distribution d...

Although hand gesture recognition has been widely explored with sensing modalities like IMU, electromyography and camera, it is still a challenge of those modalities to provide a compact, power-efficient on-board inferencing solution. In this work, we present a capacitive-sensing wristband surrounded by four single-end electrodes for on-board hand...

The data imbalance problem is a frequent bottleneck in the classification performance of neural networks. In this paper, we propose a novel supervised discriminative feature generation (DFG) method for a minority class dataset. DFG is based on the modified structure of a generative adversarial network consisting of four independent networks: genera...

We propose to use ambient sound as a privacy-aware source of information for COVID-19-related social distance monitoring and contact tracing. The aim is to complement currently dominant Bluetooth Low Energy Received Signal Strength Indicator (BLE RSSI) approaches. These often struggle with the complexity of Radio Frequency (RF) signal attenuation,...

Human activity recognition (HAR) using wearable sensors has benefited much less from recent advances in Deep Learning than fields such as computer vision and natural language processing. This is, to a large extent, due to the lack of large scale (as compared to computer vision) repositories of labeled training data for sensor-based HAR tasks. Thus,...

Autonomous underwater vehicles (AUV) are seen as an emerging technology for maritime exploration but are still restricted by the availability of short range, accurate positioning methods necessary, e.g., when docking remote assets. Typical techniques used for high-accuracy positioning in indoor use case scenarios, such as systems using ultra-wide b...

State of the art internet of things (IoT) and mobile monitoring systems promise to help gathering real time progress information from construction sites. However, on remote sites the adaptation of those technologies is frequently difficult due to a lack of infrastructure and often harsh and dynamic environments. On the other hand, visual inspection...

This paper has a threefold contribution. First, it presents a novel online handwriting database captured using a digital/sensor pen (Apple pencil) and digital/sensor screen (iPad). The captured data are continuous streams of multi-dimensional points, analyzed and processed to classify handwritten sequences into plain text, mathematical expressions,...

The data imbalance problem in classification is a frequent but challenging task. In real-world datasets, numerous class distributions are imbalanced and the classification result under such condition reveals extreme bias in the majority data class. Recently, the potential of GAN as a data augmentation method on minority data has been studied. In th...

In this paper, we present and evaluate a method for trajectory reconstruction from IMU signals generated when a person ”air writes” text with a finger worn IMU to make the resulting text as human-readable as possible. The vision is to provide a virtual ”sticky note” allowing people to digitally attach simple texts to locations. Thus, for example, w...

THE NEW CORONAVIRUS pandemic has promoted the new development of mobile and wearable computing in unprecedented ways. We discuss how on-body devices can help to fight the pandemic and may stay as a toolset to effectively deal with infectious diseases in the future. WHY WEARABLES? Researchers and health policy managers turned to smartphones and on-b...

Human activity recognition (HAR) using wearable sensors has benefited much less from recent advances in Machine Learning than fields such as computer vision and natural language processing. This is to a large extent due to the lack of large scale repositories of labeled training data. In our research we aim to facilitate the use of online videos, w...

The data imbalance problem is a frequent bottleneck in the classification performance of neural networks. In this paper, we propose a novel supervised discriminative feature generation (DFG) method for a minority class dataset. DFG is based on the modified structure of a generative adversarial network consisting of four independent networks: genera...

Document image enhancement and binarization methods are often used to improve the accuracy and efficiency of document image analysis tasks such as text recognition. Traditional non-machine-learning methods are constructed on low-level features in an unsupervised manner but have difficulty with binarization on documents with severely degraded backgr...

We propose a novel hybrid approach that fuses traditional computer vision techniques with deep learning models to detect figures and formulas from document images. The proposed approach first fuses the different computer vision based image representations, i.e., color transform, connected component analysis, and distance transform, termed as Fi-Fo...

Empirical research has shown that augmented reality (AR) has the potential to promote learning in different contexts. In particular, this has been shown for AR-supported physics experiments, where virtual elements (e.g., measurement data) were integrated into the learners’ visual reality in real time: compared to traditional experimentation, AR red...

Figure 1: Use case examples: a.b. front and back obstacle warning via directional red LED and scalp haptic feedback. c. automatic flashlight under in dark ambience (the obstacle warning also functions in darkness) ABSTRACT This work demonstrates a connected smart helmet platform, Head-gearX, aimed at improving personnel safety and real-time monitor...

This paper investigates the possibility of using soft smart textiles over the hair regions to detect chewing activities under episodes of snacking in a simulated scenario with everyday activities. The planar pressure textile sensors are used to perform mechanomyography of the temporalis muscles in the form of a cap. 10 participants contributed 30 r...

Social distancing and contact/exposure tracing are accepted to be critical strategies in the fight against the COVID-19 epidemic. They are both closely connected to the ability to reliably establish the degree of proximity between people in real-world environments. We proposed, implemented, and evaluated a wearable proximity sensing system based on...

We present a wearable, oscillating magnetic field-based proximity sensing system to monitor social distancing as suggested to prevent COVID 19 spread (being between 1.5 and 2.0m) apart. We evaluate the system both in controlled lab experiments and in a real life large hardware store setting. We demonstrate that, due physical properties of the magne...

Many human activities and states are related to the facial muscles’ actions: from the expression of emotions, stress, and non-verbal communication through health-related actions. such as coughing and sneezing to nutrition and drinking. In this work, we describe, in detail, the design and evaluation of a wearable system for facial muscle activity mo...

The demands of automated shipping address recognition and verification have increased to handle a large number of packages and to save costs associated with misdelivery. A previous study proposed a deep learning system where the shipping address is recognized and verified based on a camera image capturing the shipping address and barcode area. Beca...

Cardiorespiratory (CR) signals are crucial vital signs for fitness condition tracking, medical diagnosis, and athlete performance evaluation. Monitoring such signals in real-life settings is among the most widespread applications of wearable computing. We investigate how miniaturized barometers can be used to perform accurate spirometry in a wearab...

Multilingualism is a cultural cornerstone of Europe and firmly anchored in the European treaties including full language equality. However, language barriers impacting business, cross-lingual and cross-cultural communication are still omnipresent. Language Technologies (LTs) are a powerful means to break down these barriers. While the last decade h...

The rapid dynamics of COVID-19 calls for quick and effective tracking of virus transmission chains and early detection of outbreaks, especially in the phase 2 of the pandemic, when lockdown and other restriction measures are progressively withdrawn, in order to avoid or minimize contagion resurgence. For this purpose, contact-tracing apps are being...

The rapid dynamics of COVID-19 calls for quick and effective tracking of virus transmission chains and early detection of outbreaks, especially in the “phase 2” of the pandemic, when lockdown and other restriction measures are progressively withdrawn, in order to avoid or minimize contagion resurgence. For this purpose, contact-tracing apps are bei...

Multilingualism is a cultural cornerstone of Europe and firmly anchored in the European treaties including full language equality. However, language barriers impacting business, cross-lingual and cross-cultural communication are still omnipresent. Language Technologies (LTs) are a powerful means to break down these barriers. While the last decade h...

Recent studies emphasize a positive impact of learning with augmented reality (AR) systems in various instructional scenarios. Especially combining real and virtual learning components according to spatial and temporal contiguity principles is claimed to foster learning and to reduce extraneous cognitive processing. We applied these principles to a...

We investigate how pressure-sensitive smart textiles, in the form of a headband, can detect changes in facial expressions that are indicative of emotions and cognitive activities. Specifically, we present the Expressure system that performs surface pressure mechanomyography on the forehead using an array of textile pressure sensors that is not depe...

In this work, we present a novel and generic approach, Figure and Formula Detector (FFD) to detect the formulas and figures from document images. Our proposed method employs traditional computer vision approaches in addition to deep models. We transform input images by applying connected component analysis (CC), distance transform, and colour trans...

We demonstrate how a combination of a wrist-worn and stationary barometer can be used to track the vertical position of the user’s Hand with an accuracy in the range of 30 cm. To this end, the two barometers synchronized each time an RFID reader detects them being in proximity of each other. The accuracy is sufficient to detect a specific shelve of...

Learning is known to be a highly individual process affected by learners’ individual previous experience and self-directed action. Especially during laboratory courses in university science, technology, engineering and mathematics (STEM) education, all channels of knowledge construction become relevant: students have to match their theoretical back...

We present the system CoRSA to incorporate integrated sensors in millimeter-scale packages for continuous cardiorespiratory (CR) evaluation in sports activities. CoRSA retrofits trending sports apparel to add on CR sensing capability. The system uses an air pressure sensor inside a vented mask to approximate a spirometer, and an earlobe pulse-oxime...

Being able to reliably predict muscle contractions is important for athletes and rehabilitation patients alike. Numerous techniques and surrogates exist for this task. However, they are in general not well suited for everyday use and not able to extract information of muscles located in deeper body layers. To address this shortcoming, we present an...

Recent advances in Machine Learning, in particular Deep Learning have been driving rapid progress in fields such as computer vision and natural language processing. Human activity recognition (HAR) using wearable sensors, which has been a thriving research field for the last 20 years, has benefited much less from such advances. This is largely due...

During the dataset creation process for activity and context recognition research, manual annotation of ground truth events can be a time-consuming and error-prone task. In the typical use case, one or more annotators have to go over the videos recorded during the experiments and label what happens at what time of the experiment. In this paper, we...

Crowdsensing applications are a popular and common research tool, because they allow volunteering participants to provide valuable data via their mobile phones with minimal effort. In most scenarios, it is an important goal to gather data in a reliable and continuous way, while the app runs in the background to avoid disturbing the user. However, i...

Construction sites for large civil engineering projects consist of very different workflows, depending on the size and type of the project (e.g., highway construction). Managing and coordinating such complex projects is a difficult task. Lack of proper digital and reliable data makes the near permanent physical presence of a project manager necessa...

Monitoring of human activities is an essential capability of many smart systems. In recent years much progress has been achieved. One of the key remaining challenges is the availability of labeled training data, in particular taking into account the degree of variability in human activities. A possible solution is to leverage large scale online dat...

Advances in information technology are changing public health at an unprecedented rate. Participatory surveillance systems are contributing to public health by actively engaging digital (eg, Web-based) communities of volunteer citizens to report symptoms and other pertinent information on public health threats and also by empowering individuals to...