Conference Paper

A wearable, ultralight interface for bimanual teleoperation of a compliant, whole-body-controlled humanoid robot

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Although different objects and instructions were obviously used, the bimanual capability and the intuitive control of our setup would, in all likelihood, be the reason behind the shorter times obtained in our experiment. While bimanual teleoperation has been widely studied [34,35,36,37,38,39], the tracking method used is often wired or dependent on external tracking [40,41], including also our previously published video 2 [42], which has initiated this study. In this work, we use a wearable, wireless and independent device both for hand movement recognition and upper-body tracking. ...
Article
Full-text available
Objective. Bimanual humanoid platforms for home assistance are nowadays available, both as academic prototypes and commercially. Although they are usually thought of as daily helpers for non-disabled users, their ability to move around, together with their dexterity, makes them ideal assistive devices for upper-limb disabled persons, too. Indeed, teleoperating a bimanual robotic platform via muscle activation could revolutionize the way stroke survivors, amputees and patients with spinal injuries solve their daily home chores. Moreover, with respect to direct prosthetic control, teleoperation has the advantage of freeing the user from the burden of the prosthesis itself, overpassing several limitations regarding size, weight, or integration, and thus enables a much higher level of functionality. Approach. In this study, nine participants, two of whom suffer from severe upper-limb disabilities, teleoperated a humanoid assistive platform, performing complex bimanual tasks requiring high precision and bilateral arm/hand coordination, simulating home/office chores. A wearable body posture tracker was used for position control of the robotic torso and arms, while interactive machine learning applied to electromyography of the forearms helped the robot to build an increasingly accurate model of the participant’s intent over time. Main results. All participants, irrespective of their disability, were uniformly able to perform the demanded tasks. Completion times, subjective evaluation scores, as well as energy- and time- efficiency show improvement over time on short and long term. Significance. This is the first time a hybrid setup, involving myoeletric and inertial measurements, is used by disabled people to teleoperate a bimanual humanoid robot. The proposed setup, taking advantage of interactive machine learning, is simple, non-invasive, and offers a new assistive solution for disabled people in their home environment. Additionnally, it has the potential of being used in several other applications in which fine humanoid robot control is required.
... Rehabilitation based upon Virtual Reality is also a target to this procedure (Nissler et al., 2019). Robotic control based upon muscle activity can be also transferred to teleoperated scenarios (Porges et al., 2019) and, probably, in space. In all these scenarios it is worth investigating the usefulness and feasibility of the procedure described in this paper. ...
Article
Full-text available
Natural myocontrol is the intuitive control of a prosthetic limb via the user's voluntary muscular activations. This type of control is usually implemented by means of pattern recognition, which uses a set of training data to create a model that can decipher these muscular activations. A consequence of this approach is that the reliability of a myocontrol system depends on how representative this training data is for all types of signal variability that may be encountered when the amputee puts the prosthesis into real use. Myoelectric signals are indeed known to vary according to the position and orientation of the limb, among other factors, which is why it has become common practice to take this variability into account by acquiring training data in multiple body postures. To shed further light on this problem, we compare two ways of collecting data: while the subjects hold their limb statically in several positions one at a time, which is the traditional way, or while they dynamically move their limb at a constant pace through those same positions. Since our interest is to investigate any differences when controlling an actual prosthetic device, we defined an evaluation protocol that consisted of a series of complex, bimanual daily-living tasks. Fourteen intact participants performed these tasks while wearing prosthetic hands mounted on splints, which were controlled via either a statically or dynamically built myocontrol model. In both cases all subjects managed to complete all tasks and participants without previous experience in myoelectric control manifested a significant learning effect; moreover, there was no significant difference in the task completion times achieved with either model. When evaluated in a simulated scenario with traditional offline performance evaluation, on the other hand, the dynamically-trained system showed significantly better accuracy. Regardless of the setting, the dynamic data acquisition was faster, less tiresome, and better accepted by the users. We conclude that dynamic data acquisition is advantageous and confirm the limited relevance of offline analyses for online myocontrol performance.
... Rehabilitation based upon Virtual Reality is also a target for this procedure [27]. Robotic control based upon muscle activity can be also transferred to teleoperated scenarios [28]. In future work we aim to explore the use and feasibility of the procedure described in this paper in some of these scenarios. ...
Conference Paper
Natural myocontrol employs pattern recognition to allow users to control a robotic limb intuitively using their own voluntary muscular activations. The reliability of myocontrol strongly depends on the signals initially collected from the users, which must appropriately capture the variability encountered later on during operation. Since myoelectric signals can vary based on the position and orientation of the limb, it has become best practice to gather data in multiple body postures. We hereby concentrate on this acquisition protocol and investigate the relative merits of collecting data either statically or dynamically. In the static case, data for a desired hand configuration is collected while the users keep their hand still in certain positions, whereas in the dynamic case, data is collected while users move their limbs, passing through the required positions with a roughly constant velocity. Fourteen able-bodied subjects were asked to naturally control two dexterous hand prostheses mounted on splints, performing a set of complex, realistic bimanual activities of daily living. We could not find any significant difference between the protocols in terms of the total execution times, although the dynamic data acquisition was faster and less tiring. This would indicate that dynamic data acquisition should be preferred over the static one.
Article
Full-text available
Stable myoelectric control of hand prostheses remains an open problem. The only successful human-machine interface is surface electromyography, typically allowing control of a few degrees of freedom. Machine learning techniques may have the potential to remove these limitations, but their performance is thus far inadequate: myoelectric signals change over time under the influence of various factors, deteriorating control performance. It is therefore necessary, in the standard approach, to regularly retrain a new model from scratch. We hereby propose a non-linear incremental learning method in which occasional updates with a modest amount of novel training data allow continual adaptation to the changes in the signals. In particular, Incremental Ridge Regression and an approximation of the Gaussian Kernel known as Random Fourier Features are combined to predict finger forces from myoelectric signals, both finger-by-finger and grouped in grasping patterns. We show that the approach is effective and practically applicable to this problem by first analyzing its performance while predicting single-finger forces. Surface electromyography and finger forces were collected from 10 intact subjects during four sessions spread over two different days; the results of the analysis show that small incremental updates are indeed effective to maintain a stable level of performance. Subsequently, we employed the same method on-line to teleoperate a humanoid robotic arm equipped with a state-of-the-art commercial prosthetic hand. The subject could reliably grasp, carry and release everyday-life objects, enforcing stable grasping irrespective of the signal changes, hand/arm movements and wrist pronation and supination.
Article
This work presents a new control approach to multi-contact balancing for torque-controlled humanoid robots. The controller includes a non-strict task hierarchy, which allows the robot to use a subset of its end effectors for balancing while the remaining ones can be used for interacting with the environment. The controller creates a passive and compliant behavior for regulating the center of mass (CoM) location, hip orientation and the poses of each end effector assigned to the interaction task. This is achieved by applying a suitable wrench (force and torque) at each one of the end effectors used for interaction. The contact wrenches at the balancing end effectors are chosen such that the sum of the balancing and interaction wrenches produce the desired wrench at the CoM. The algorithm requires the solution of an optimization problem, which distributes the CoM wrench to the end effectors taking into account constraints for unilaterality, friction and position of the center of pressure. Furthermore, the feedback controller is combined with a feedforward control in order to improve performance while tracking a predefined trajectory, leading to a control structure similar to a PD+ control. The controller is evaluated in several experiments with the humanoid robot TORO.
Article
The paradigm of simultaneous and proportional myocontrol of hand prostheses is gaining momentum in the rehabilitation robotics community. As opposed to the traditional surface electromyography classification schema, in simultaneous and proportional control the desired force/torque at each degree of freedom of the hand/wrist is predicted in real-time, giving to the individual a more natural experience, reducing the cognitive effort and improving his dexterity in daily-life activities. In this study we apply such an approach in a realistic manipulation scenario, using ten non-linear incremental regression machines to predict the desired torques for each motor of two robotic hands. The prediction is enforced using two sets of surface electromyography electrodes and an incremental, non-linear machine learning technique called Incremental Ridge Regression with Random Fourier Features. Nine able-bodied subjects were engaged in a functional test with the aim to evaluate the performance of the system. The robotic hands were mounted on two hand/wrist orthopedic splints worn by healthy subjects and controlled online. An average completion rate of more than 95% was achieved in single-handed tasks and 84% in bimanual tasks. On average, five minutes of retraining were necessary on a total session duration of about one hour and forty minutes. This work sets a beginning in the study of bimanual manipulation with prostheses and will be carried on through experiments in unilateral and bilateral upper limb amputees thus increasing its scientific value.
Conference Paper
This paper gives an overview on the torquecontrolled humanoid robot TORO, which has evolved from the former DLR Biped. In particular, we describe its mechanical design and dimensioning, its sensors, electronics and computer hardware. Additionally, we give a short introduction to the walking and multi-contact balancing strategies used for TORO.