No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Automated Plan Refinement for Improving Efficiency of Robotic Layup of Composite Sheets
ResearchGate has not been able to resolve any citations for this publication.
Early detection and correction of defects are critical in additive manufacturing (AM) to avoid build failures. In this paper, we present a multisensor fusion-based digital twin for in-situ quality monitoring and defect correction in a robotic laser-directed energy deposition process. Multisensor fusion sources consist of an acoustic sensor, an infrared thermal camera, a coaxial vision camera, and a laser line scanner. The key novelty and contribution of this work are to develop a spatiotemporal data fusion method that synchronizes and registers the multisensor features within the part's 3D volume. The fused dataset can be used to predict location-specific quality using machine learning. On-the-fly identification of regions requiring material addition or removal is feasible. Robot toolpath and auto-tuned process parameters are generated for defect correction. In contrast to traditional single-sensor-based monitoring, multisensor fusion allows for a more in-depth understanding of underlying process physics, such as pore formation and laser-material interactions. The proposed methods pave the way for self-adaptation AM with higher efficiency, less waste, and cleaner production.
Robotic layup is a novel process developed to face the increasing demand for automation, flexibility, repeatability, and achieving high-quality composite materials in relevant industry fields, such as aerospace and automotive. This process is based on laying prepreg tissues on a mold using the action of a robotic arm equipped with a specific end-effector, which is usually composed of rollers and punches. The main drawback of the robotic layup is the occurrence of wrinkles and defects while moving, placing, and processing the pre-impregnated tissues. This issue is particularly evident in the processing of complex-shaped surfaces. The robotic arm cannot replicate exactly the movement of a human operator, following the geometry of the surfaces with a proper angulation like a human wrist. Moreover, operator hands can be set in a different shape just changing the configuration of the fingers, adapting themself in different curvatures. The demand of the industry to improve automation requires that the robotic manufacturing systems replicate as much as possible the gesture of the operator. In this study, an end-effector capable to replicate the human movements during the hand layup of fibrous tissues or fabrics has been conceptualized.
In industry, several operations require sheet-like materials to be transported from a loading station to the desired location. Such applications are prevalent in the aerospace and textile industry where composite prepreg sheets or fabrics are placed over a tool or fed to a machine. Using robots for sheet transport operations offers a flexible solution for such highly complex tasks. To create high-quality parts, sheets need to be accurately placed at the correct location. This paper presents automated trajectory planning and control algorithms for a robot to pick up sheets from the input station using suction grippers and, transport and place them over the tool surface. Machine vision is used at the pick location for estimating the sheet pose. Unfortunately, pick-up accuracy is not sufficiently high due to sheet movement during suction-based grasping and localization errors. We employ ideas inspired by visual servo techniques to accurately place the sheet on the tool. Our method uses an Eye-to-Hand camera configuration to align the desired image features with the reference markings on the tool. We introduce a sampling-based Jacobian estimation scheme that can reliably achieve the desired accuracy while minimizing the operation time. Experiments are performed to validate our methodology and compute the placement accuracy on an industrial tool.
Automation of high-performance manufacturing processes such as prepreg composite layup has been gaining a lot of interest lately. Reliable and accurate defect detection methods play a crucial role in the automation of such processes to maintain the desired quality. The composite prepreg layup process involves manipulation of sheet-like material. Traditional machine vision-based defect detection techniques are inept in detecting defects for such complex processes due to the nature of the defects. Advanced defect detection techniques enabled by deep learning are the key for such applications. However, Deep learning usually requires an enormous amount of physical images of the process which is infeasible in high-mix manufacturing applications. In this paper, we resolve the data generation problem for deep learning by presenting an approach where with a combination of finite element-based simulation and advanced graphics techniques we generate a dataset of photorealistic images of the defects. Approximately, 10000 synthetic images are generated and combined with around 1000 images of real sheets to train a ResNeSt-based deep learning model. We have also devised an efficient 2-stage methodology for training the deep learning network to detect wrinkle-like defects. With the trained model and data augmentation techniques, our method can achieve a mean Average Precision (mAP) of 0.98 on actual production data for detecting defects. The code and the entire dataset are available at: https://github.com/RROS-Lab/DeepSynthDefectDetector
Deformable object manipulation (DOM) is an emerging research problem in robotics. The ability to manipulate deformable objects endows robots with higher autonomy and promises new applications in the industrial, services, and health-care sectors. However, compared to rigid object manipulation, the manipulation of deformable objects is considerably more complex and is still an open research problem. Addressing DOM challenges demands breakthroughs in almost all aspects of robotics: hardware design, sensing, (deformation) modeling, planning, and control. In this article, we review recent advances and highlight the main challenges when considering deformation in each subfield. A particular focus of our article lies in the discussions of these challenges and proposing future directions of research.
In this study an automated composite layup end effector is presented which is the first to be able to find defects in real time during layup using tactile shape sensing. Based around an existing sensor concept developed by the Bristol Robot Laboratory known as the ‘TacTip’, a new end effector is developed, replacing the soft gel core of the original sensor was replaced by a much firmer elastomer, enabling it to apply up to 400N of compaction force. In this paper it is shown to successfully detect typical defects such as wrinkles, foreign objects, layup errors or incorrect material types while simultaneously compacting preimpregnated composite materials over complex mould shapes.
Surface monitoring is an essential part of quality assurance for additive manufacturing (AM). Surface defects need to be identified early in the AM process to avoid further deterioration of the part quality. In this paper, a rapid surface defect identification method for directed energy deposition (DED) is proposed. The main contribution of this work is the development of an in-situ point cloud processing with machine learning methods that enable automatic surface monitoring without sensor intermittence. An in-house software platform with a multi-nodal architecture is developed. In-situ point cloud processing steps, including filtering, segmentation, surface-to-point distance calculation, point clustering, and machine learning feature extraction, are performed by multiple subprocesses running simultaneously. The combined unsupervised and supervised machine learning techniques are applied to detect and classify surface defects. The proposed method is experimentally validated, and a surface defect identification accuracy of 93.15% is achieved.
Hand layup is a commonly used process for making composite structures from several plies of carbon-fiber prepreg. The process involves multiple human operators manipulating and conforming layers of prepreg to a mold. The manual layup process is ergonomically challenging, tedious, and limits throughput. Moreover, different operators may perform the process differently, and hence introduce inconsistency. We have developed a multi-robot cell to automate the layup process. A human expert provides a sequence to conform to the ply and types of end-effectors to be used as input to the system. The system automatically generates trajectories for the robots that can achieve the specified layup. Using the cell requires the automated generation of different types of plans. This paper addresses two main planning problems: (a) generating plans to grasp and manipulate the ply and (b) generating feasible robot trajectories. We use a hybrid-physics based simulator coupled with a state space search to find grasp plans. The system employs a strategy that applies constraints successively in a non-linear optimization formulation to identify suitable placements of the robots around the mold so that feasible trajectories can be generated. Our system can generate plans in a computationally efficient manner, and it can handle a wide variety of complex parts. We demonstrate the automated layup by conducting physical experiments on an industry-inspired mold using the generated plans.
High-performance composites are widely used in industry because of specific mechanical properties and lightweighting opportunities. Current automation solutions to manufacturing components from prepreg (pre-impregnated precursor material) sheets are limited. Our previous work has demonstrated the technical feasibility of a robotic cell to automate the sheet layup process. Many decisions are required for the cell to function correctly, and the time necessary to make these decisions must be reduced to utilize the cell effectively. Robot placement with respect to the mold is a significant and complex decision problem. Ensuring that robots can collaborate effectively requires addressing multiple constraints related to the robot workspace, singularity, and velocities. Solving this problem requires developing computationally efficient algorithms to find feasible robot placements in the cell. We describe an approach based on successive solution refinement strategy to identify a cell design that satisfies all constraints related to robot placement.
This work considers an autonomous multi-robot pick-and-place process for the manufacturing of aerospace structures, which consists of the steps picking, transfer, dropping and post-drop treatment. Autonomous production is achieved by combining computer vision assisted gripping, automated transfer path generation and generic process execution in one system. Test scenario is an airplane skin demonstrator made of tailored carbon fiber multiaxial fabrics that are placed in a half-shell shaped jig of approximately 4 m in diameter by two cooperating robots mounted on a common linear axis.
The use of depth cameras, such as a laser profilometer, in defect detection during the automated fiber placement (AFP) process is of great importance as they improve AFP’s layup quality and efficiency. However, the collected point clouds are large and of heterogeneous densities. Several works have projected 3D point clouds onto 2D images, but the method has only led to loss of key geometric details. In this study, we proposed a two-stage segmentation method for collected point clouds during the AFP process, named AFP-Seg, to realize AFP in-process inspection. In the first stage, the point clouds fused with collected laser lines and sampling position data are fed into a semantic segmentation network, and the semantic label of each point can be obtained, and in the second stage, point clouds with specified semantic labels are clustered using the post-process algorithm. Through the AFP-Seg method, information about defects regarding types, sizes, and positions are acquired eventually. Compared with our previous method, the AFP-Seg method can reduce the data processing time by 33-66% and obtain more robust and better inspection results. Furthermore, it can be well integrated into a real-time AFP in-process inspection system and easily adjusted based on actual engineering inspection specifications.
We present a framework for deformable object manipulation that interleaves planning and control, enabling complex manipulation tasks without relying on high-fidelity modeling or simulation. The key question we address is when should we use planning and when should we use control to achieve the task? Planners are designed to find paths through complex configuration spaces, but for highly underactuated systems, such as deformable objects, achieving a specific configuration is very difficult even with high-fidelity models. Conversely, controllers can be designed to achieve specific configurations, but they can be trapped in undesirable local minima owing to obstacles. Our approach consists of three components: (1) a global motion planner to generate gross motion of the deformable object; (2) a local controller for refinement of the configuration of the deformable object; and (3) a novel deadlock prediction algorithm to determine when to use planning versus control. By separating planning from control we are able to use different representations of the deformable object, reducing overall complexity and enabling efficient computation of motion. We provide a detailed proof of probabilistic completeness for our planner, which is valid despite the fact that our system is underactuated and we do not have a steering function. We then demonstrate that our framework is able to successfully perform several manipulation tasks with rope and cloth in simulation, which cannot be performed using either our controller or planner alone. These experiments suggest that our planner can generate paths efficiently, taking under a second on average to find a feasible path in three out of four scenarios. We also show that our framework is effective on a 16-degree-of-freedom physical robot, where reachability and dual-arm constraints make the planning more difficult.
We present a novel approach to deformable object manipulation that does not rely on highly-accurate modeling. The key contribution of this paper is to formulate the task as a Multi-Armed Bandit problem, with each arm representing a model of the deformable object. To “pull” an arm and evaluate its utility, we use the arm’s model to generate a velocity command for the gripper(s) holding the object and execute it. As the task proceeds and the object deforms, the utility of each model can change. Our framework estimates these changes and balances exploration of the model set with exploitation of high-utility models. We also propose an approach based on Kalman Filtering for Non-stationary Multi-armed Normal Bandits (KF-MANB) to leverage the coupling between models to learn more from each arm pull.We demonstrate that our method outperforms previous methods on synthetic trials, and performs competitively on several manipulation tasks in simulation.
The thesis deals with the optimal motion planning in redundant robotic systems for automation of the composite lay-up processes. The primary goalis to improve the lay-up workcell productivity by developing a novel methodology of optimizing coordinated motions of the robotic manipulator,workpiece positioner and workspace extension unit,which ensure the shortest processing time and smooth movements of all mechanical components. In contrast to the previous works, the proposed methodology provides high computational efficiencyand also takes into account both the technological constraints and the robotic system constraints, which describe capacities of the actuators and are expressed by the maximum allowable velocities and accelerations in the actuated joints. The developed technique is based on conversion of the original continuous problem into a combinatorial one, where all possible configurations of the mechanical components are represented as a directed multi layergraph and the desired time-optimal motion is generated using dynamic programming principle for searching the shortest path on the graph satisfying the smoothness constraints. It is also proposed an enhancement of this technique by dividing the optimization procedure in two stages combining global and local searches. At the first stage, the developed algorithm is applied in the global search space generated with large discretization step. Then,the same technique is applied in the local search space, which is created with smaller step in the neighborhood of the obtained trajectory. The advantages of the developed methodology are confirmed by industrial implementation on the factory floor that deals with manufacturing of the high pressure vessel.
full-text: https://tel.archives-ouvertes.fr/tel-01980728v2/document/
With increasing use of fiber reinforced polymer composites follows a natural pursuit for more rational and effective manufacturing. Robotic pick-and-place systems can be used to automate handling of a multitude of materials used in the manufacturing of composite parts. There are systems developed for automated layup of prepreg, dry fibers and thermoplastic blanks as well as to handle auxiliary materials used in manufacturing. The aim of this paper is to highlight the challenges associated with automated handling of these materials and to analyze the main design principles that have been employed for pick-and-place systems in terms of handling strategy, reconfigurability, gripping technology and distribution of gripping points etc. The review shows that it is hard to find generic solutions for automated material handling due to the great variety in material properties. Few cases of industrial applications in full-scale manufacturing could be identified.
The use of composite materials in aerospace, automotive and ship industry allows manufacturing lighter and more efficient mechanical structures. The major limiting factors are the high-manufacturing costs and low-production rates. Automated tape placement is one alternative process to overcome the limiting factors. Due to the complex contour of the mandrel, the lay-up paths of the contiguous tapes are not parallel along their lengths, which eventually introduce gaps or overlaps between the edges of tapes. Overlaps and excessive gaps are undesirable as they will decrease the strength of the resulting composite member. In this paper, a novel trajectory planning method for automated tape placement, entitled improved geodesic strategy, is proposed. The strategy aims to optimize the relationship between adjacent tapes. At the same time, the distortion of composite tape is also specified in order to prevent wrinkling. The principle and characteristic of the algorithm are presented. The method is investigated on the airfoil of an unmanned aerial vehicle and some of the results are presented in this paper.
In this paper, we present a general approach to automatically visual-servo control the position and shape of a deformable object whose deformation parameters are unknown. The servo-control is achieved by online learning a model mapping between the robotic end-effector's movement and the object's deformation measurement. The model is learned using the Gaussian Process Regression (GPR) to deal with its highly nonlinear property, and once learned, the model is used for predicting the required control at each time step. To overcome GPR's high computational cost while dealing with long manipulation sequences, we implement a fast online GPR by selectively removing uninformative observation data from the regression process. We validate the performance of our controller on a set of deformable object manipulation tasks and demonstrate that our method can achieve effective and accurate servo-control for general deformable objects with a wide variety of goal settings. Videos are available at https://sites.google.com/view/mso-fogpr.
High throughput automated techniques are nowadays playing a key role in polymer composite manufacturing in a number of industries such as automotive and aerospace. There is a need to produce high volume parts efficiently. Automated manufacturing methods such as automated tape layup and automated fiber placement can produce composite parts efficiently, and with the advent of additive manufacturing the complexity of these components are increasing. This paper will review contemporary composite manufacturing methods filament winding, automated tape layup, and automated fiber placement, and the newer automation techniques of robotic pick-and-place and continuous tow shearing. It also addresses recent advances in composite additive manufacturing using vat photopolymerization, binder jetting, material extrusion, sheet lamination and powder bed fusion. Methods, materials and testing results of the manufactured components will be discussed.
Exploring and modeling heterogeneous elastic surfaces requires multiple interactions with the environment and a complex selection of physical material parameters. The most common approaches model deformable properties from sets of offline observations using computationally expensive force-based simulators. In this work we present an online probabilistic framework for autonomous estimation of a deformability distribution map of heterogeneous elastic surfaces from few physical interactions. The method takes advantage of Gaussian Processes for constructing a model of the environment geometry surrounding a robot. A fast Position-based Dynamics simulator uses focused environmental observations in order to model the elastic behavior of portions of the environment. Gaussian Process Regression maps the local deformability on the whole environment in order to generate a deformability distribution map. We show experimental results using a PrimeSense camera, a Kinova Jaco2 robotic arm and an Optoforce sensor on different deformable surfaces.
The hand layup of pre-impregnated woven materials is still a large part of the composite manufacturing industry, requiring the skills and experience of a human workforce to form flat plies into complex shapes. It is capable of producing high performance and complex parts, but can be an expensive and variable process. Despite its importance, there appears to have been very little research into the actual methods and techniques used by workers to manipulate flat sheets of composite material into shape during layup. This work presents the first known detailed study of the approach and techniques used by laminators. Four participants laid up onto 15 different shaped molds that replicated features commonly found on composite components. The actions used in layup were grouped into eight distinct techniques. Use of these techniques across tasks of different geometry, ramp angles, radii and drape path was identified using video analysis techniques from the ergonomics field. This revealed strong links between specific features and techniques, revealing a systematic approach to layup. This has enabled the first step toward producing a design for manufacture knowledge base surrounding hand layup. This could then be used to inform the development of the layup process, improve training methods and assist in the design of future automated solutions.
Due to the outstanding material properties the use of carbon fiber reinforced plastics in aerospace applications has grown rapidly during the last years. However, the manual process of creating a preform out of dry cut-outs is still very time-consuming and error-prone and thus limits an efficient use of this technology. Especially the high diversity of variants, the material properties and the complexity of the process limited an automation of the preforming process so far. In this paper an automation system is presented, which consists of a robot-based preforming end-effector and its offline path-planning. The end-effector has a highly modular and flexible design and integrates the three essential functions of the preforming process: gripping, draping and heating. Based on a detailed analysis of the geometric parameters of the preforms and its layers the task-specific layout of the end-effector is conducted. To achieve a preform in high-quality a solution for controlling the end-effector and planning the robot-path is necessary. Hence, a semi-automatic approach is developed, which incorporates the know-how of experts and automatically generates layup-curves with path-synchronous trigger signals for the end-effector. In an experimental set up the feasibility and flexibility of the automation solution could be successfully tested. The evaluation on three industrial moulds showed that the challenging requirements and the high quality standards could be met.
Due to their high stiffness and strength, composites are widely used in the aerospace industry. To manufacture composites, especially composites of free-form surface structure, process of robotic fibre placement (RFP) is widely used in industry. However, due to the complex geometry of the free-form surface, it is quite challenging to generate accurate roller paths for placing fibre on the surface for high composites quality. To address this problem, this work proposes an accurate roller path planning method using the differential geometry. The roller paths can ensure the specified small gaps and overlaps between two tows for high composite quality. This approach is applied to several examples, and their results verify the validity of this approach. It has great potential to be adopted in industry.
Automated composite draping: a review
- M Elkington
- C Ward
- A Sarkytbayev
- Sampe Seattle
Design of a hyper-flexible cell for handling 3d carbon fiber fabric
- R Molfino
- M Zoppi
- F Cepolina
- J Yousef
- E Cepolina
A simulation-based grasp planner for enabling robotic grasping during composite sheet layup
- O M Manyar
- J Desai
- N Deogaonkar
- R J Joesph
- R Malhan
- Z Mcnulty
Learning predictive representations for deformable objects using contrastive estimation
- W Yan
- A Vangipuram
- P Abbeel
- L Pinto
Learning-based feedback controller for deformable object manipulation
- B Jia
- Z Hu
- Z Pan
- D Manocha
- J Pan
Automated composite draping: a review
- Elkington
Design of a hyper-flexible cell for handling 3d carbon fiber fabric
- Molfino
Learning predictive representations for deformable objects using contrastive estimation
- Yan
Learning-based feedback controller for deformable object manipulation
- Jia