About
612
Publications
211,339
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
16,449
Citations
Introduction
Current institution
Additional affiliations
Publications
Publications (612)
Web agents are systems capable of navigating and interacting autonomously with the World Wide Web. While this concept is almost as old as the web itself, the recent success of large language models (LLMs) and their accompanying capabilities has accelerated the development of web agents.As the adoption of web agents continues, the need arises to com...
We present the CASE framework, an open-source platform for adaptive, context-aware participatory research, and pandemic preparedness. CASE implements an event-driven architecture that enables dynamic survey workflows, allowing real-time adaptation based on participant responses, external data, temporal conditions, and evolving user states. The fram...
Quantum Information Science (QIS) is a vast, diverse, and abstract field. In consequence, learners face many challenges. Science, Technology, Engineering, and Mathematics (STEM) education research has found that visualizations are valuable to aid learners in complex matters. The conditions under which visualizations pose benefits are largely unexpl...
Latent space representations are critical for understanding and improving the behavior of machine learning models, yet they often remain obscure and intricate. Understanding and exploring the latent space has the potential to contribute valuable human intuition and expertise about respective domains. In this work, we present HILL, an interactive fr...
Sensor-based human activity recognition (HAR) has predominantly focused on Inertial Measurement Units and vision data, often overlooking the capabilities unique to pressure sensors, which capture subtle body dynamics and shifts in the center of mass. Despite their potential for postural and balance-based activities, pressure sensors remain underuti...
Wearable augmented reality (AR) systems have significant potential to enhance surgical outcomes through in-situ visualization of patient-specific data. Yet, efforts to develop AR-based systems for open surgery have been limited, lacking comprehensive interdisciplinary research and actual clinical evaluations in real surgical environments.
Our rese...
Interactive systems powered by artificial intelligence (AI) are gaining incredible momentum among everyday computing devices. Recent research works on so-called web agents, which are large language models (LLM) capable of interacting with the world wide web on their own, promise the possibility to develop universally applicable systems. These syste...
Integrated into websites, LLM-powered chatbots offer alternative means of navigation and information retrieval, leading to a shift in how users access information on the web. Yet, predominantly closed-sourced solutions limit proliferation among web hosts and suffer from a lack of transparency with regard to implementation details and energy efficie...
Human activity recognition (HAR) with deep learning models relies on large amounts of labeled data, often challenging to obtain due to associated cost, time, and labor. Self-supervised learning (SSL) has emerged as an effective approach to leverage unlabeled data through pretext tasks, such as masked reconstruction and multitask learning with signa...
While human body capacitance ($HBC$) has been explored as a novel wearable motion sensing modality, its competence has never been quantitatively demonstrated compared to that of the dominant inertial measurement unit ($IMU$) in practical scenarios. This work is thus motivated to evaluate the contribution of $HBC$ in wearable motion sensing. A real-...
Using oscillating magnetic fields for indoor positioning is a robust way to resist dynamic environments. This work presents the hard- and software-related optimizations of an induced magnetic field positioning system. We describe a new coil architecture for both the transmitter and receiver, reducing inter-axes cross-talk. A new analog circuit desi...
Alleviating high workloads for teachers is crucial for continuous high quality education. To evaluate if Large Language Models (LLMs) can alleviate this problem in the quantum computing domain, we conducted two complementary studies exploring the use of GPT-4 to automatically generate tips for students. (1) A between-subject survey in which student...
Improvements in the area of large language models have shifted towards the construction of models capable of using external tools and interpreting their outputs. These so-called web agents have the ability to interact autonomously with the internet. This allows them to become powerful daily assistants handling time-consuming, repetitive tasks while...
Human activity recognition (HAR) ideally relies on data from wearable or environment-instrumented sensors sampled at regular intervals, enabling standard neural network models optimized for consistent time-series data as input. However, real-world sensor data often exhibits irregular sampling due to, for example, hardware constraints, power-saving...
Conventional human activity recognition (HAR) relies on classifiers trained to predict discrete activity classes, inherently limiting recognition to activities explicitly present in the training set. Such classifiers would invariably fail, putting zero likelihood, when encountering unseen activities. We propose Open Vocabulary HAR (OV-HAR), a frame...
The passive body-area electrostatic field has recently been aspiringly explored for wearable motion sensing, harnessing its two thrilling characteristics: full-body motion sensitivity and environmental sensitivity, which potentially empowers human activity recognition both independently and jointly from a single sensing front-end and theoretically...
Over the last decade, representation learning, which embeds complex information extracted from large amounts of data into dense vector spaces, has emerged as a key technique in machine learning. Among other applications, it has been a key building block for large language models and advanced computer vision systems based on contrastive learning. A...
The distribution network relies on a critical electric component, the cable joint (CJ), for expansion and post-fault repairs. In some parts of the network, a CJ can be found every few hundred meters, making it a crucial element that requires thorough study. Additionally, the CJ is highly vulnerable, and its failure often results in explosions follo...
Understanding human-to-human interactions, especially in contexts like public security surveillance, is critical for monitoring and maintaining safety. Traditional activity recognition systems are limited by fixed vocabularies, predefined labels, and rigid interaction categories that often rely on choreographed videos and overlook concurrent intera...
Wearable Augmented Reality (AR) technologies are gaining recognition for their potential to transform surgical navigation systems. As these technologies evolve, selecting the right interaction method to control the system becomes crucial. Our work introduces a voice-controlled user interface (VCUI) for surgical AR assistance systems (ARAS), designe...
The research of machine learning (ML) algorithms for human activity recognition (HAR) has made significant progress with publicly available datasets. However, most research prioritizes statistical metrics over examining negative sample details. While recent models like transformers have been applied to HAR datasets with limited success from the ben...
Supervised machine learning often operates on the data-driven paradigm, wherein internal model parameters are autonomously optimized to converge predicted outputs with the ground truth, devoid of explicitly programming rules or a priori assumptions. Although data-driven methods have yielded notable successes across various benchmark datasets, they...
A fundamental step in the development of machine learning models commonly involves the tuning of hyperparameters, often leading to multiple model training runs to work out the best-performing configuration. As machine learning tasks and models grow in complexity, there is an escalating need for solutions that not only improve performance but also a...
Human Activity Recognition using time-series data from wearable sensors poses unique challenges due to complex temporal dependencies, sensor noise, placement variability, and diverse human behaviors. These factors, combined with the nontransparent nature of black-box Machine Learning models impede interpretability and hinder human comprehension of...
In the many years since the inception of wearable sensor-based Human Activity Recognition (HAR), a wide variety of methods have been introduced and evaluated for their ability to recognize activities. Substantial gains have been made since the days of hand-crafting heuristics as features, yet, progress has seemingly stalled on many popular benchmar...
Objective
The present study aimed to evaluate the safety of the first wearable augmented reality assistance system (ARAS) specifically designed for pancreatic surgery and its impact on perioperative outcomes.
Background
Pancreatic surgery remains highly complex and is associated with a high rate of perioperative complications. ARAS, as an intraope...
The integration of Augmented Reality (AR) technology into surgical procedures offers significant potential to enhance clinical outcomes. While there are plenty of lab-proven prototypes, systems employed in actual clinical settings require specialized design and rigorous clinical evaluation of these AR-based solutions to meet the high demands of com...
The quest for enhanced cognition has been a driving force behind human advancement, fostering innovation and personal fulfillment. Cognition Altering Technologies (CAT) holds immense promise in elevating the quality of life across diverse domains including education, decision-making, healthcare, and fitness. The current proliferation of Artificial...
As distance learning becomes increasingly important and artificial intelligence tools continue to advance, automated systems for individual learning have attracted significant attention. However, the scarcity of open-source online tools that are capable of providing personalized feedback has restricted the widespread implementation of research-base...
Disordered Quantum many-body Systems (DQS) and Quantum Neural Networks (QNN) have many structural features in common. However, a DQS is essentially an initialized QNN with random weights, often leading to non-random outcomes. In this work, we emphasize the possibilities of random processes being a deceptive quantum-generating model effectively hidd...
Distractions caused by digital devices are increasingly causing dangerous situations on the road, particularly for more vulnerable road users like cyclists. While researchers have been exploring ways to enable richer interaction scenarios on the bike, safety concerns are frequently neglected and compromised. In this work, we propose Head 'n Shoulde...
Password sharing is a convenient means to access shared resources, save on subscription costs, provide emergency access, and avoid forgetting vital account details. However, it also raises significant privacy concerns, especially in digital communication contexts where content may be inadvertently exposed to unintended recipients. In this paper, we...
Artificial Intelligence (AI) methods are powerful tools for various domains, including critical fields such as avionics, where certification is required to achieve and maintain an acceptable level of safety. General solutions for safety-critical systems must address three main questions: Is it suitable? What drives the system's decisions? Is it rob...
In the human activity recognition research area, prior studies predominantly concentrate on leveraging advanced algorithms on public datasets to enhance recognition performance, little attention has been paid to executing real-time kitchen activity recognition on energy-efficient, cost-effective edge devices. Besides, the prevalent approach of segr...
This workshop brings researchers together to discuss and explore how artificial intelligence (AI) can be used to improve general health. During our workshop at the MuC conference, we will focus on three main areas: developing ethical AI health recommendations, exploring how smart technologies in our homes can influence our health habits, and unders...
Smaller machine learning models, with less complex architectures and sensor inputs, can benefit wearable sensor-based human activity recognition (HAR) systems in many ways, from complexity and cost to battery life. In the specific case of smart factories, optimizing human-robot collaboration hinges on the implementation of cutting-edge, human-centr...
Despite the widespread integration of ambient light sensors (ALS) in smart devices commonly used for screen brightness adaptation, their application in human activity recognition (HAR), primarily through body-worn ALS, is largely unexplored. In this work, we developed ALS-HAR, a robust wearable light-based motion activity classifier. Although ALS-H...
Diet is an inseparable part of good health, from maintaining a healthy lifestyle for the general population to supporting the treatment of patients suffering from specific diseases. Therefore it is of great significance to be able to monitor people’s dietary activity in their daily life remotely. While the traditional practices of self-reporting an...
Large Language Models (LLMs) have made significant advances in natural language processing, but their underlying mechanisms are often misunderstood. Despite exhibiting coherent answers and apparent reasoning behaviors, LLMs rely on statistical patterns in word embeddings rather than true cognitive processes. This leads to vulnerabilities such as "h...
Lithium-ion batteries are widely used in various applications, including electric vehicles and renewable energy storage. The prediction of the remaining useful life (RUL) of batteries is crucial for ensuring reliable and efficient operation, as well as reducing maintenance costs. However, determining the life cycle of batteries in real-world scenar...
In this work, we propose a novel single-end morphing capacitive sensing method for shape tracking, FxC, by combining Folding origami structures and Capacitive sensing to detect the morphing structural motions using state-of-the-art sensing circuits and deep learning. It was observed through embedding areas of origami structures with conductive mate...
Individual teaching is among the most successful ways to impart knowledge. Yet, this method is not always feasible due to large numbers of students per educator. Quantum computing serves as a prime example facing this issue, due to the hype surrounding it. Alleviating high workloads for teachers, often accompanied with individual teaching, is cruci...
Hybrid intelligence aims to enhance decision-making, problem-solving, and overall system performance by combining the strengths of both, human cognitive abilities and artificial intelligence. With the rise of Large Language Models (LLM), progressively participating as smart agents to accelerate machine learning development, Hybrid Intelligence is b...
This work presents a novel and versatile approach to employ textile capacitive sensing as an effective solution for capturing human body movement through fashionable and everyday-life garments. Conductive textile patches are utilized for sensing the movement, working without the need for strain or direct body contact, wherefore the patches can sens...
In this work, we explore the use of a novel neural network architecture, the Kolmogorov-Arnold Networks (KANs) as feature extractors for sensor-based (specifically IMU) Human Activity Recognition (HAR). Where conventional networks perform a parameterized weighted sum of the inputs at each node and then feed the result into a statically defined nonl...
This work described a novel non-contact, wearable, real-time eye blink detection solution based on capacitive sensing technology.
A custom-built prototype employing low-cost and low-power consumption capacitive sensors was integrated into standard glasses, with a copper tape electrode affixed to the frame. The blink of an eye induces a variation i...
Purpose
The retroperitoneal nature of the pancreas, marked by minimal intraoperative organ shifts and deformations, makes augmented reality (AR)-based systems highly promising for pancreatic surgery. This study presents preliminary data from a prospective study aiming to develop the first wearable AR assistance system, ARAS, for pancreatic surgery...
Human Activity Recognition is a longstanding problem in AI with applications in a broad range of areas: from healthcare, sports and fitness, security, and human computer interaction to robotics. The performance of HAR in real-world settings is strongly dependent on the type and quality of the input signal that can be acquired. Given an unobstructed...
The general availability of large language models and thus unrestricted usage in sensitive areas of everyday life, such as education, remains a major debate. We argue that employing generative artificial intelligence (AI) tools warrants informed usage and examined their impact on problem solving strategies in higher education. In a study, students...
Due to the scarcity of labeled sensor data in HAR, prior research has turned to video data to synthesize Inertial Measurement Units (IMU) data, capitalizing on its rich activity annotations. However, generating IMU data from videos presents challenges for HAR in real-world settings, attributed to the poor quality of synthetic IMU data and its limit...
This work proposes an incremental learning (IL) framework for wearable sensor human activity recognition (HAR) that tackles two challenges simultaneously: catastrophic forgetting and non-uniform inputs. The scalable framework, iKAN, pioneers IL with Kolmogorov-Arnold Networks (KAN) to replace multi-layer perceptrons as the classifier that leverages...
The proliferation of deep learning has significantly advanced various fields, yet Human Activity Recognition (HAR) has not fully capitalized on these developments, primarily due to the scarcity of labeled datasets. Despite the integration of advanced Inertial Measurement Units (IMUs) in ubiquitous wearable devices like smartwatches and fitness trac...
In the field of quantum information science and technology, the representation and visualization of quantum states and related processes are essential for both research and education. In this context, a focus lies especially on ensembles of few qubits. There exist many powerful representations for single-qubit and multiqubit systems, such as the fa...
Hand-over-face gestures can provide important implicit interactions during conversations, such as frustration or excitement. However, in situations where interlocutors are not visible, such as phone calls or textual communication, the potential meaning contained in the hand-over-face gestures is lost. In this work, we present iFace, an unobtrusive,...
This Special Interest Group (SIG) explores the transformative impact of Generative Artificial Intelligence (GenAI) on Human-Computer Interaction (HCI) research processes. The theme here is to answer “question zero”: when to use and when to refrain from using AI tools during the research cycle? The discussion is guided by five research phases common...
Automatic and precise fitness activity recognition can be beneficial in aspects from promoting a healthy lifestyle to personalized preventative healthcare. While IMUs are currently the prominent fitness tracking modality, through iMove, we show bio-impedence can help improve IMU-based fitness tracking through sensor fusion and contrastive learning....
Due to the fact that roughly sixty percent of the human body is essentially composed of water, the human body is inherently a conductive object, being able to, firstly, form an inherent electric field from the body to the surroundings and secondly, deform the distribution of an existing electric field near the body. Body-area capacitive sensing, al...
Recent advancements in Artificial Neural Networks have significantly improved human activity recognition using multiple time-series sensors. While employing numerous sensors with high-frequency sampling rates usually improves the results , it often leads to data inefficiency and unnecessary expansion of the ANN, posing a challenge for their practic...
As distance learning becomes increasingly important and artificial intelligence tools continue to advance, automated systems for individual learning have attracted significant attention. However, the scarcity of open-source online tools that are capable of providing personalized feedback has restricted the widespread implementation of research-base...
This work examines the effects of variations in machine learning training regimes and learning paradigms on the corresponding energy consumption. While increasing data availability and innovation in high-performance hardware fuels the training of sophisticated models, it also supports the fading perception of energy consumption and carbon emission....
The increased presence of large language models (LLMs) in educational settings has ignited debates concerning negative repercussions, including overreliance and inadequate task reflection. Our work advocates moderated usage of such models, designed in a way that supports students and encourages critical thinking. We developed two moderated interact...
