No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
A high-accuracy, low-budget Sensor Glove for Trajectory Model Learning
Biomechanical sensors are essential components for wearable robots because of their broad spectrum of industrial and medical applications. A wide range of these sensors uses soft materials with deformation-related electrical properties, namely variable resistors and capacitors. Previously, conductive materials were injected into silicone structures that require several molding steps to build the microchannels. This paper proposes a versatile soft sensing glove, using a simple process for preparing sensors of different sizes without molding. We used commercially available silicone tubes to host the conductive liquid. Ten sensors were attached to the back side of the glove to measure flexion-extension and four sensors were placed on the glove at the interdigital folds between the fingers to measure abduction-adduction. The sensory glove successfully replicates hand motion. We used machine learning algorithms to estimate the angles of the joints in the hand and also to identify 15 gestures. The system’s robustness was evaluated in two experiments. The gesture prediction is robust to shocks from contact with a punching ball and also submersion of the glove in water. The proposed sensory glove overcomes challenges of comfort, rigidity, and robustness. It can be used, therefore, to replicate human hand motion in industrial and medical applications.
This paper presents a design that jointly provides hand pose sensing, hand localization, and haptic feedback to facilitate real-time stable grasps in Virtual Reality (VR). The design is based on an easy-to-replicate glove-based system that can reliably perform (i) a high-fidelity hand pose sensing in real time through a network of 15 IMUs, and (ii) the hand localization using a Vive Tracker. The supported physics-based simulation in VR is capable of detecting collisions and contact points for virtual object manipulation, which drives the collision event to trigger the physical vibration motors on the glove to signal the user, providing a better realism inside virtual environments. A caging-based approach using collision geometry is integrated to determine whether a grasp is stable. In the experiment, we showcase successful grasps of virtual objects with large geometry variations. Comparing to the popular LeapMotion sensor, we demonstrate the proposed glove-based design yields a higher success rate in various tasks in VR. We hope such a glove-based system can simplify the data collection of human manipulations with VR.
Capturing hand motions for hand function evaluations is essential in the medical field. Various data gloves have been developed for rehabilitation and manual dexterity assessments. This study proposed a modular data glove with 9-axis inertial measurement units (IMUs) to obtain static and dynamic parameters during hand function evaluation. A sensor fusion algorithm is used to calculate the range of motion of joints. The data glove is designed to have low cost, easy wearability, and high reliability. Owing to the modular design, the IMU board is independent and extensible and can be used with various microcontrollers to realize more medical applications. This design greatly enhances the stability and maintainability of the glove.
Physical interaction requires robots to accurately follow kinematic trajectories while modulating the interaction forces to accomplish tasks and to be safe to the environment. However, current approaches rely on accurate physical models or iterative learning approaches. We present a versatile approach for physical interaction tasks, based on Movement Primitives (MPs) that can learn physical interaction tasks solely by demonstrations, without explicitly modeling the robot or the environment. We base our approach on the Probabilistic Movement Primitives (ProMPs), which utilizes the variance of the demonstrations to provide better generalization of the encoded skill, combine skills, and derive a controller that follows exactly the encoded trajectory distribution. However, the ProMP controller requires the system dynamics to be known. We present a reformulation of the ProMPs that allows accurate reproduction of the skill without modeling the system dynamics and, further, we extent our approach to incorporate external sensors, as for example, force/torque sensors. Our approach learns physical interaction tasks solely from demonstrations and online adapts the movement to force–torque sensor input. We derive a variable-stiffness controller in closed form that reproduces the trajectory distribution and the interaction forces present in the demonstrations. We evaluate our approach in simulated and real-robot tasks.
In this study, a soft sensor-based three-dimensional (3-D) finger motion measurement system is proposed. The sensors, made of the soft material Ecoflex, comprise embedded microchannels filled with a conductive liquid metal (EGaln). The superior elasticity, light weight, and sensitivity of soft sensors allows them to be embedded in environments in which conventional sensors cannot. Complicated finger joints, such as the carpometacarpal (CMC) joint of the thumb are modeled to specify the location of the sensors. Algorithms to decouple the signals from soft sensors are proposed to extract the pure flexion, extension, abduction, and adduction joint angles. The performance of the proposed system and algorithms are verified by comparison with a camera-based motion capture system.
Objective analysis of hand and finger kinematics is important to increase understanding of hand function and to quantify motor symptoms for clinical diagnosis. The aim of this paper is to compare a new 3D measurement system containing multiple miniature inertial sensors (PowerGlove) with an opto-electronic marker system during specific finger tasks in three healthy subjects. Various finger movements tasks were performed: flexion, fast flexion, tapping, hand open/closing, ab/adduction and circular pointing. 3D joint angles of the index finger joints and position of the thumb and index were compared between systems. Median root mean square differences of the main joint angles of interest ranged between 3.3 and 8.4deg. Largest differences were found in fast and circular pointing tasks, mainly in range of motion. Smallest differences for all 3D joint angles were observed in the flexion tasks. For fast finger tapping, the thumb/index amplitude showed a median difference of 15.8mm. Differences could be explained by skin movement artifacts caused by relative marker movements of the marker system, particularly during fast tasks; large movement accelerations and angular velocities which exceeded the range of the inertial sensors; and by differences in segment calibrations between systems. The PowerGlove is a system that can be of value to measure 3D hand and finger kinematics and positions in an ambulatory setting. The reported differences need to be taken into account when applying the system in studies understanding the hand function and quantifying hand motor symptoms in clinical practice.
Human motor skill learning is driven by the necessity to adapt to new situations. While supportive contacts are essential for many tasks, little is known about their impact on motor learning. To study the effect of contacts an innovative full-body experimental paradigm was established. The task of the subjects was to reach for a distant target while postural stability could only be maintained by establishing an additional supportive hand contact. To examine adaptation, non-trivial postural perturbations of the subjects’ support base were systematically introduced. A novel probabilistic trajectory model approach was employed to analyze the correlation between the motions of both arms and the trunk. We found that subjects adapted to the perturbations by establishing target dependent hand contacts. Moreover, we found that the trunk motion adapted significantly faster than the motion of the arms. However, the most striking finding was that observations of the initial phase of the left arm or trunk motion (100–400 ms) were sufficient to faithfully predict the complete movement of the right arm. Overall, our results suggest that the goal-directed arm movements determine the supportive arm motions and that the motion of heavy body parts adapts faster than the light arms.
Achieving accurate and reliable kinematic hand pose reconstructions represents a challenging task. The main reason for this is the complexity of hand biomechanics, where several degrees of freedom are distributed along a continuous deformable structure. Wearable sensing can represent a viable solution to tackle this issue, since it enables a more natural kinematic monitoring. However, the intrinsic accuracy (as well as the number of sensing elements) of wearable hand pose reconstruction (HPR) systems can be severely limited by ergonomics and cost considerations. In this paper, we combined the theoretical foundations of the optimal design of HPR devices based on hand synergy information, i.e., the inter-joint covariation patterns, with textile goniometers based on knitted piezoresistive fabrics (KPF) technology, to develop, for the first time, an optimally-designed under-sensed glove for measuring hand kinematics. We used only five sensors optimally placed on the hand and completed hand pose reconstruction (described according to a kinematic model with 19 degrees of freedom) leveraging upon synergistic information. The reconstructions we obtained from five different subjects were used to implement an unsupervised method for the recognition of eight functional grasps, showing a high degree of accuracy and robustness.
Sensor gloves are widely adopted input devices for several kinds of human-robot interaction applications. Existing glove concepts differ in features and design, but include limitations concerning the captured finger kinematics, position/orientation sensing, wireless operation, and especially economical issues. This paper presents the DAGLOVE which addresses the mentioned limitations with a low-cost design (ca. 300 e). This new sensor glove allows separate measurements of proximal and distal finger joint motions as well as position/orientation detection with an inertial measurement unit (IMU). Those sensors and tactile feedback induced by coin vibration motors at the fingertips are integrated within a wireless, easy-to-use, and open-source system. The design and implementation of hardware and software as well as proof-of-concept experiments are presented. An experimental evaluation of the sensing capabilities shows that proximal and distal finger motions can be acquired separately and that hand position/orientation can be tracked. Further, teleoperation of the iCub humanoid robot is investigated as an exemplary application to highlight the potential of the extended low-cost glove in human-robot interaction.
This paper describes the design and manufacturing of soft artificial skin with an array of embedded soft strain sensors for detecting various hand gestures by measuring joint motions of five fingers. The proposed skin was made of a hyperelastic elastomer material with embedded microchannels filled with two different liquid conductors, an ionic liquid and a liquid metal. The ionic liquid microchannels were used to detect the mechanical strain changes of the sensing material, and the liquid metal microchannels were used as flexible and stretchable electrical wires for connecting the sensors to an external control circuit. The two heterogeneous liquid conductors were electrically interfaced through flexible conductive threads to prevent the two liquid from being intermixed. The skin device was connected to a computer through a microcontroller instrumentation circuit for reconstructing the 3-D hand motions graphically. The paper also presents preliminary calibration and experimental results.
Movement primitives (MPs) provide a powerful framework for data driven movement generation that has been successfully applied for learning from demonstrations and robot reinforcement learning. In robotics we often want to solve a multitude of different, but related tasks. As the parameters of the primitives are typically high dimensional, a common practice for the generalization of movement primitives to new tasks is to adapt only a small set of control variables, also called meta parameters, of the primitive. Yet, for most MP representations, the encoding of these control variables is pre-coded in the representation and can not be adapted to the considered tasks. In this paper, we want to learn the encoding of task-specific control variables also from data instead of relying on fixed meta-parameter representations. We use hierarchical Bayesian models (HBMs) to estimate a low dimensional latent variable model for probabilistic movement primitives (ProMPs), which is a recent movement primitive representation. We show on two real robot datasets that ProMPs based on HBMs outperform standard ProMPs in terms of generalization and learning from a small amount of data and also allows for an intuitive analysis of the movement. We also extend our HBM by a mixture model, such that we can model different movement types in the same dataset.
We report a new method, embedded-3D printing (e-3DP), for fabricating strain sensors within highly conformal and extensible elastomeric matrices. e-3DP allows soft sensors to be created in nearly arbitrary planar and 3D motifs in a highly programmable and seamless manner. Several embodiments are demonstrated and sensor performance is characterized.
Movement Primitives (MP) are a well-established approach for representing modular and re-usable robot movement generators. Many state-of-the-art robot learning successes are based MPs, due to their compact representation of the inherently continuous and high dimensional robot movements. A major goal in robot learning is to combine multiple MPs as building blocks in a modular control architecture to solve complex tasks. To this effect, a MP representation has to allow for blending between motions, adapting to altered task variables, and co-activating multiple MPs in parallel. We present a probabilistic formulation of the MP concept that maintains a distribution over trajectories. Our probabilistic approach allows for the derivation of new operations which are essential for implementing all aforementioned properties in one framework. In order to use such a trajectory distribution for robot movement control, we analytically derive a stochastic feedback controller which reproduces the given trajectory distribution. We evaluate and compare our approach to existing methods on several simulated as well as real robot scenarios.
This paper presents the design and validation of a wearable glove-based multi-finger motion capture device (SmartGlove) with a specific focus on the development of a new optical linear encoder (OLE). The OLE specially designed for this project has a compact size, light weight and low power consumption. The characterization tests also show that the OLE's digital output has good linearity and accuracy. The first prototype of SmartGlove which uses ten OLEs to capture the flexion/extension motion of the 14 finger joints is constructed based on the multi-point sensing method. A user study for the evaluation of SmartGlove using a standard protocol shows high repeatability and reliability in both the gripped and flat hand positions compared with four other evaluated data gloves using the same protocol.
We present a careful evaluation of the sensor characteristics of the CyberGlove(TM) model CG1801 whole hand input device. In particular, we conducted an experimental study that investigated level of sensitivity of the sensors, their performance in recognized angles, and factors that affect accuracy of recognition of flexion measurements. Among our results, we show that hand size differences between the subjects of the study did not have a statisical affect on the accuracy of the device. We also analyzed the effect of differect software calibration approaches on accuracy of the sensors.
With growing numbers of intelligent autonomous systems in human environments, the ability of such systems to perceive, understand, and anticipate human behavior becomes increasingly important. Specifically, predicting future positions of dynamic agents and planning considering such predictions are key tasks for self-driving vehicles, service robots, and advanced surveillance systems. This article provides a survey of human motion trajectory prediction. We review, analyze, and structure a large selection of work from different communities and propose a taxonomy that categorizes existing methods based on the motion modeling approach and level of contextual information used. We provide an overview of the existing datasets and performance metrics. We discuss limitations of the state of the art and outline directions for further research.
We propose a stretch-sensing soft glove to interactively capture hand poses with high accuracy and without requiring an external optical setup. We demonstrate how our device can be fabricated and calibrated at low cost, using simple tools available in most fabrication labs. To reconstruct the pose from the capacitive sensors embedded in the glove, we propose a deep network architecture that exploits the spatial layout of the sensor itself. The network is trained only once, using an inexpensive off-the-shelf hand pose reconstruction system to gather the training data. The per-user calibration is then performed on-the-fly using only the glove. The glove's capabilities are demonstrated in a series of ablative experiments, exploring different models and calibration methods. Comparing against commercial data gloves, we achieve a 35% improvement in reconstruction accuracy.
In this paper, a wearable sensing glove for measuring the motion of the fingers is proposed. The system consists of linear potentiometers, flexible wires, and linear springs, which makes it compact and lightweight so that it does not interfere with the natural motion of the fingers. Inspired by the way wrinkles on finger joints are smoothed out when the finger is flexed, a flexible wire is attached to the back of each finger. As the flexible wire moves due to the motion of the finger, the joint angles are calculated by measuring the change in length of wire. Linear potentiometers with linear springs were used to maintain the tension of the wires in order to measure the wire length change consistently. Because the motion of the proximal interphalangeal (PIP) joint is dependent on that of the distal interphalangeal (DIP) joint, only two linear potentiometers were used for each finger. A compact sensing module including 10 linear potentiometers and springs was attached to a glove. The proposed system can widely be applied for the systems, which require to measure finger motions accurately, e.g., virtual reality or teleoperation systems. Such feasible applications were actually implemented and introduced in this paper.
Optimal control theory and its more recent extension, optimal feedback control theory, provide valuable insights into the flexible and task-dependent control of movements. Here, we focus on the problem of coordination, defined as movements that involve multiple effectors (muscles, joints or limbs). Optimal control theory makes quantitative predictions concerning the distribution of work across multiple effectors. Optimal feedback control theory further predicts variation in feedback control with changes in task demands and the correlation structure between different effectors. We highlight two crucial areas of research, hierarchical control and the problem of movement initiation, that need to be developed for an optimal feedback control theory framework to characterise movement coordination more fully and to serve as a basis for studying the neural mechanisms involved in voluntary motor control.
Sensor gloves for measurements of finger movements are a promising tool for objective assessments of kinematic parameters and new rehabilitation strategies. Here, a novel low-cost sensor glove equipped with resistive bend sensors is described and evaluated. Resistive bend sensors were modified in order to optimize measurement accuracy (quantified as the stability of sensor signal after a fast and constant bending) and to increase sensor linearity, reducing calibration time from several minutes to only approximately 10s. Reliability analysis of the sensor glove in five subjects showed an intraclass correlation coefficient (ICC) of 0.93+/-0.05, a mean standard deviation of 1.59 degrees and an overall error of 4.96 degrees , comparable to previously evaluated sensor gloves. User acceptance and applicability, assessed by a user feedback questionnaire, was high. Thus, with minor modifications, resistive bend sensors are suitable for accurate assessments of human finger movements. The low material costs (<US$ 500) and easy manufacturing make this solution interesting for widespread use in research, clinical and rehabilitative settings.
Unifying principles of movement have emerged from the computational study of motor control. We review several of these principles and show how they apply to processes such as motor planning, control, estimation, prediction and learning. Our goal is to demonstrate how specific models emerging from the computational approach provide a theoretical framework for movement neuroscience.
Optitrack, Optitrack-motion capture systems
- Optitrack
Valve index vr system
- V Corporation
Leap motion controller
- Ultraleap
Probabilistic trajectory model toolbox
- E Rueckert
S. Electronics. Sparkfun bend sensor
- S Electronics
Performance optimization of a flex sensor based glove for hand gestures recognition and translation
- O Nisar
- M A Imtiaz
- S Hussain
- O Saleem
Movement primitives toolbox
- A Paraschos
F. S. Systems. Flexpoint bend sensor
- F S Systems