Project

H2020 ColRobot - Collaborative Robotics for Assembly and Kitting in Smart Manufacturing

Goal: ColRobot creates an integrated system for collaborative robotics in which a mobile manipulator acts as a “third hand” by delivering kits, tools, parts, and holding work pieces while the operator works on it. Humans will cognitively and physically interact with ColRobot robots using gestures, touch commands and demonstrations. The robot will be able to navigate autonomously in the factory floor to pick up the required parts and tools and prepare kits for assembly. Two use cases in automobile and aerospace industry will be validated in real-world operational environments.

ColRobot project is coordinated by Prof Olivier Gibaru (ENSAM) and funded by the H2020 ICT programme of the EU GA N. 688807.

Stay informed at https://colrobot.eu and follow us @Col_Robot and join our LinkedIn Group https://www.linkedin.com/groups/8514731

Date: 1 February 2016 - 31 January 2019

Updates
0 new
22
Recommendations
0 new
3
Followers
0 new
27
Reads
1 new
678

Project log

Pedro Neto
added 2 research items
We propose a novel way of solving the issue of classification of out-of-vocabulary gestures using Artificial Neural Networks (ANNs) trained in the Generative Adversarial Network (GAN) framework. A generative model augments the data set in an online fashion with new samples and stochastic target vectors, while a discriminative model determines the class of the samples. The approach was evaluated on the UC2017 SG and UC2018 DualMyo data sets. The generative models’ performance was measured with a distance metric between generated and real samples. The discriminative models were evaluated by their accuracy on trained and novel classes. In terms of sample generation quality, the GAN is significantly better than a random distribution (noise) in mean distance, for all classes. In the classification tests, the baseline neural network was not capable of identifying untrained gestures. When the proposed methodology was implemented, we found that there is a trade-off between the detection of trained and untrained gestures, with some trained samples being mistaken as novelty. Nevertheless, a novelty detection accuracy of 95.4% or 90.2% (depending on the data set) was achieved with just 5% loss of accuracy on trained classes.
The paradigm for robot usage has changed in the last few years, from a scenario in which robots work isolated to a scenario where robots collaborate with human beings, exploiting and combining the best abilities of robots and humans. The development and acceptance of collaborative robots is highly dependent on reliable and intuitive human-robot interaction (HRI) in the factory floor. This paper proposes a gesture-based HRI framework in which a robot assists a human co-worker delivering tools and parts, and holding objects to/for an assembly operation. Wearable sensors, inertial measurement units (IMUs), are used to capture the human upper body gestures. Captured data are segmented in static and dynamic blocks recurring to an unsupervised sliding window approach. Static and dynamic data blocks feed an artificial neural network (ANN) for static, dynamic, and composed gesture classification. For the HRI interface, we propose a parameterization robotic task manager (PRTM), in which according to the system speech and visual feedback, the co-worker selects/validates robot options using gestures. Experiments in an assembly operation demonstrated the efficiency of the proposed solution.
Jose Saenz
added 2 research items
The EU-funded project ColRobot focuses on the use of mobile manipulators for collaborative kitting and assembly tasks in automobile and satellite production. Kitting tasks in particular rely on machine vision for part detection and for visual servoing, but recent research, has shown the advantages of multi-modal picking for the completion of complex and flexible picking tasks. This paper focuses on new developments toward a high-resolution tactile sensor that can be mounted on to different grippers. The sensor described in this paper has been integrated into a commercially available robotic gripper, and is used for object classification and pose estimation. The sensor components, including the electronics, the integrated design, and interfacing will be presented in this paper. We further present our results object classification, based on neuronal networks. Initial experimental results to validate the sensor's operability will be described in this article.
The EU-funded project ColRobot focuses on the use of mobile manipulators for collaborative kitting and assembly tasks in automobile and satellite production. This paper focuses on the application and experimental validation of the workspace monitoring system built by the IFF in the ColRobot project. This includes the validation of machine learning techniques for the detection of humans in a shared workspace with the aim of offering soft-safety functionality (e.g. functionality which supports the process cycle time and at the same time increases human acceptance and satisfaction), as well as the validation of a visual workspace monitoring system for hard-safety functionality under adverse lighting conditions. Soft-safety functionality is complementary to hard-safety aspects, which can be defined as the requirements expressed in the relevant standards and which ensure that hazardous situations are mitigated in most cases by stopping the robot’s motion. The robotic hardware – with a focus on the vision system, the application of the machine learning techniques to the task of human detection, and initial experimental results to validate the system will be described in this article.
Nicola Dorigo Salamon
added an update
Next 5th October 2018, Pedro Neto (University of Coimbra) will partake in the IROS 2018 Workshop Robot Safety: filling the gap between technology offer and industry needs, for a fully deployable human-robot collaboration. The participation will include a talk on 'Collaborative robotics challenges for safe and intuitive interaction featuring the ColRobot use cases for automotive and spacecraft industrial applications'. The talk will be followed by a final roundtable. The panel will include speakers from AIRBUS, Loccioni Group, CNR ITIA, and LAAS-CNRS. The workshop (14.30-19.00) will be held at the Madrid Municipal Conference Centre (Madrid/Spain).
For more information http://robotsafety-iros18.com
ColRobot results will be also exhibited at Technaid S.L stand N.106 and disseminated through the papers authored by partners that investigated the following technological challenges:
- Modelling, representation implementation and execution of high performance and composable tasks. Collaborative robot with precision hand-guiding abilities;
- Semantically Meaningful View Selection;
- Reducing the computational complexity of mass-matrix calculation for high DOF robots.
 
Nicola Dorigo Salamon
added an update
On 26th September 2018, Olivier Gibaru together with Gabriel Audry presented the ColRobot RENAULT automotive usecase to 25 officers of the European Commission Directorate-General for Internal Market, Industry, Entrepreneurship and SMEs that visited the Group Renault's plant in Douai (FR).
 
Mario Sposato
added a research item
Collaborative robots must operate safely and efficiently in ever-changing unstructured environments, grasping and manipulating many different objects. Artificial vision has proved to be collaborative robots' ideal sensing technology and it is widely used for identifying the objects to manipulate and for detecting their optimal grasping. One of the main drawbacks of state of the art robotic vision systems is the long training needed for teaching the identification and optimal grasps of each object, which leads to a strong reduction of the robot productivity and overall operating flexibility. To overcome such limit, we propose an engineering method, based on deep learning techniques, for the detection of the robotic grasps of unknown objects in an unstructured environment, which should enable collaborative robots to autonomously generate grasping strategies without the need of training and programming. A novel loss function for the training of the grasp prediction network has been developed and proved to work well also with low resolution 2-D images, then allowing the use of a single, smaller and low cost camera, that can be better integrated in robotic end-effectors. Despite the availability of less information (resolution and depth) a 75% of accuracy has been achieved on the Cornell data set and it is shown that our implementation of the loss function does not suffer of the common problems reported in literature. The system has been implemented using the ROS framework and tested on a Baxter collaborative robot.
Nicola Dorigo Salamon
added an update
#ColRobot releases new open data-sets on "UC2018 DualMyo Hand Gesture" obtained from two consumer-market EMG sensors (Myo) with a subject performing 8 distinct hand gestures. Data have been generated by Pedro Neto and Olivier Gibaru
 
Jose Saenz
added a research item
While collaborative robots have made headlines through recent industrial applications, they are not as widespread in industry as it may seem. The authors of this paper believe that one reason for this slow uptake is due to the high requirements on the safety and the lack of engineering tools for analyzing collaborative robotics applications. Systems engineering provides a good framework for creating the engineering tools needed for faster and more reliable deployment, but has only recently been applied to robotics challenges. In this paper, we discuss the state of the art for designing robotics applications featuring human-robot collaboration (HRC) and then review existing systems engineering approaches, which could offer support. Our review aims to support the robotics community in the future development of engineering tools to better understand, plan, and implement applications featuring collaborative robotics.
Nicola Dorigo Salamon
added an update
AIMEN Centro Tecnológico unveils ColRobot prototype assembly for the automotive use-case.
This video shows the automotive prototype performing the final mechanical and electrical assessment and assembling using the mobile manipulator. The experiment has been conducted by the ColRobot team led by Diego Pérez Losada Losada at AIMEN (Porriño, ES).
The video has been produced by the ColRobot project funded by the European Union's Horizon 2020 research and innovation programme under grant agreement No. 688807.
Find out more about ColRobot at https://colrobot.eu
 
Nicola Dorigo Salamon
added an update
New video to demonstrate the tactile sensing technology developed by the Fraunhofer IFF to support dextrous manipulation for in-hand classification of screws.
 
Nicola Dorigo Salamon
added an update
The video documents the ColRobot experiments at the University of Modena and Reggio Emilia (UNIMORE) using two Convolutional Neural Networks (CNN). The first CNN recognizes an object to pick from low-resolution images and a second one predicts the robot arm best pose and trajectory to deliver the object to the operator.
The experiment has been conducted by the UNIMORE ColRobot team led by Prof Marcello Pelliciari at the University of Modena and Reggio Emilia.
The video will be presented at the 25th International Conference on Transdisciplinary Engineering (TE2018) in Modena next 3-6 July 2018.
 
Richard Béarée
added 2 research items
Hand-guiding is a main functionality of collaborative robots, allowing to rapidly and intuitively interact and program a robot. Many applications require end-effector precision positioning during the teaching process. This paper presents a novel method for precision hand-guiding at the end-effector level. From the end-effector force/torque measurements the hand-guiding force/torque (HGFT) is achieved by compensating for the tools weight/inertia. Inspired by the motion properties of a passive mechanical system, mass subjected to coulomb/viscous friction, it was implemented a control scheme to govern the linear/angular motion of the decoupled end-effector. Experimental tests were conducted in a KUKA iiwa robot in an assembly operation.
Considering a set of robotic tasks which involve physical interaction with the environment, the theoretical knowledge of the full force capacity of the manipulator is a key factor in the design or development of an efficient and economically attractive solution. Carrying its own weight while countering forces may be too much for a robot in certain configurations. Kinematic redundancy with regard to a task allows a robot to perform it in a continuous space of articular configurations; space in which the payload of the robot may vary dramatically. It may be impossible to withstand a physical interaction in some configurations, while it may be easily sustainable in others that bring the end-effector to the same location. This becomes obviously more prevalent for a limited payload robot. This paper describes a framework for this kind of operations, in which kinematic redundancy is used to explore the full extent of a force capacity for a given manipulator and task. A pragmatic force capacity index (FCI) is proposed. The FCI offers a sound basis for redundancy resolution via optimization or complete redundancy exploration, and may provide good hints for end-effector design. A practical use case involving 7-DOFs KUKA LBR iiwa was used to demonstrate the relevance of the proposed method.
Nicola Dorigo Salamon
added an update
Papers presented at the session on “Collaborative Robots for Smart Manufacturing” at FAIM 2017 27th International Conference on Flexible Automation and Intelligent Manufacturing (27-30 June 2017, Modena, Italy) have been published in Procedia Manufacturing Vol. 11, 2017
The volume edited by Marcello Pellicciari and Margherita Peruzzini includes the following papers authored by ColRobot partners:
- “Safeguarding Collaborative Mobile Manipulators - Evaluation of the VALERI Workspace Monitoring System”, Jose Saenz , C. Vogel, F. Penzlin, N. Elkmann pp. 47-54;
- "3D metrology using a collaborative robot with a laser triangulation sensor”, G. Boyé de Sousaa, A. Olabia, J. Palosa & Olivier Gibaru , pp. 132-140;
- “From the Internet of Things to Cyber-Physical Systems: the Holonic Perspective”, Luca Pazzi and Marcello Pellicciari , pp. 989-995.
 
Nicola Dorigo Salamon
added an update
Check-out the video of the talk of Mohammad Safeea at TEDxAveiro on collaborative robotics and the fourth industrial revolution. At the University of Coimbra, as part of the H2020 ColRobot project, Mohammad is researching and developing a safety system for avoiding collisions between robots and human coworkers.
 
Richard Béarée
added a research item
In the context of human–robot manipulation interaction for service or industrial robotics, the robot controller must be able to quickly react to unpredictable events in dynamic environments. In this paper, a FIR filter-based trajectory generation methodology is presented, combining the simplicity of the analytic second-order trajectory generation, i.e. acceleration-limited trajectory, with the flexibility and computational efficiency of FIR filtering, to generate on the fly smooth jerk-constrained trajectories. The proposed methodology can generate synchronized (fixed-time) and time-optimal jerk-limited trajectories from arbitrary initial velocity and acceleration conditions within 20 microsecond. Other jerk-constrained trajectories such as jerk-time fixed trajectories, which are particularly suitable for vibration reduction, can be easily generated. Experimental validations carried out on a seven axis Kuka LBR iiwa are presented.
Nicola Dorigo Salamon
added a project reference
Nicola Dorigo Salamon
added an update
ColRobot combines cutting-edge robot technology and end-user requirements for assembly processes to create an integrated system for collaborative robotics in which a mobile manipulator acts as a “third hand” by delivering kits, tools, parts, and holding workpieces while the operator works on it.
Humans will cognitively and physically interact with ColRobot using gestures, touch commands and demonstrations. The robot will be able to navigate autonomously on the factory floor to pick up the required parts and tools and prepare kits for assembly. A safety system that pushes the limits of standardization in collaborative robotics supervises the process.
The technology readiness level (TRL) will be scaled-up through continuous iterative testing (performance, usability, manufacturing relevance), validation and improvements. Two use cases in automotive and aerospace industry are being demonstrated in real-world operational environments. This video shows a demonstration in the aerospace industry.
The video has been produced by the ColRobot project funded by the European Union's Horizon 2020 research and innovation programme under grant agreement No. 688807. Stay informed about @Col_robot visiting https://colrobot.eu
 
Nicola Dorigo Salamon
added an update
From 6th to 8th December 2017, ColRobot partners gathered at ENSAM campus and at the CITC IOTcluster in Lille to advance the preparation of the automotive and aerospace demonstration scenarios for assembly and kitting operations. As part of the IoTWeek, CITC organised a public workshop for industry audience to provide insights on ColRobot aerospace demonstrator for satellite assembly and kitting operations.
Stay informed visit https://colrobot.eu
 
Nicola Dorigo Salamon
added an update
This week, the ColRobot Consortium successfully completed the first review meeting held in Luxembourg at DG CONNECT. Great Work! Thanks to all the partners' delegates. #H2020 #InvestEUresearch .Find out more at https://colrobot.eu @col_robot
 
Pedro Neto
added 2 research items
The ability of efficient and fast calculation of the minimum distance between humans and robots is vitally important for realizing a safe human robot interaction (HRI), where robots and human co-workers share the same workspace. The minimum distance is the main input for most of collision avoidance methods, HRI, robot decision making, as well as robot navigation. In this study it is presented a novel methodology to analytically compute of the minimum distance between cylindrical primitives with spherical ends. Such primitives are very important since that there geometrical shape is suitable for representing the co-worker and the robots structures. The computational cost of the minimum distance between cylinders is of order. In this study QR factorization is proposed to achieve the computational efficiency in calculating the minimum distance mutually between each pair of cylinders. Experimental tests demonstrated the effectiveness of the proposed approach.
This paper presents an intuitive human-robot interaction (HRI) framework for gesture and human behavior recognition. It relies on a vision-based system as interaction technology to classify gestures and a 3-axis accelerometer for behavior classification (stand, walking, etc.). An intelligent system integrates static gesture recognition recurring to artificial neural networks (ANNs) and dynamic gesture recognition using hidden Markov models (HMM). Results show a recognition rate of 95% for a library of 22 gestures and 97% for a library of 6 behaviors. Experiments show a robot controlled using gestures in a HRI process.
Nicola Dorigo Salamon
added an update
On 18 July, seventeen students participated in the Summer School in Mechanical Engineering at the University of Coimbra. A full morning dedicated to collaborative robotics and its importance for industry. The concept of Industry 4.0 was also discussed with the participants. The students had also the opportunity to interact and program a collaborative robot.
 
Nicola Dorigo Salamon
added an update
ColRobot video animation to explain how collaborative robotics can improve productivity, flexibility and ergonomics in automotive and aerospace assembly operations.
ColRobot is a mobile robotic manipulator for industrial assembly and assistance in kitting .
Check-out our animation https://youtu.be/8zpYzVEw-Io
The animation conceptualises visually a robot interacting naturally, collaboratively and safely with a factory operator preparing kits, sorting, passing and holding parts and tools.
The video has been produced by the ColRobot project funded by the EU Horizon 2020 Programme.
 
Nicola Dorigo Salamon
added an update
ColRobot Consortium getting together for the mid term meeting in Coimbra
 
Nicola Dorigo Salamon
added 5 project references
Nicola Dorigo Salamon
added a project goal
ColRobot creates an integrated system for collaborative robotics in which a mobile manipulator acts as a “third hand” by delivering kits, tools, parts, and holding work pieces while the operator works on it. Humans will cognitively and physically interact with ColRobot robots using gestures, touch commands and demonstrations. The robot will be able to navigate autonomously in the factory floor to pick up the required parts and tools and prepare kits for assembly. Two use cases in automobile and aerospace industry will be validated in real-world operational environments.
ColRobot project is coordinated by Prof Olivier Gibaru (ENSAM) and funded by the H2020 ICT programme of the EU GA N. 688807.
Stay informed at https://colrobot.eu and follow us @Col_Robot and join our LinkedIn Group https://www.linkedin.com/groups/8514731