Article

Development and evaluation of a robust soft robotic gripper for apple harvesting

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

Fruit harvesting is facing challenges due to the labour shortage, which has been more severe since the rapid pandemic. Robotic harvesting has been attempted in autonomous fruit harvesting tasks, such as apple harvesting. However, current apple harvesting robots show limited harvesting performance in the orchard environment due to the inefficiency of the robotic grippers. This research presents a fruit harvesting method that includes a novel soft robotic gripper and a detachment strategy to achieve apple harvesting in the natural orchard. The soft robotic gripper includes four tapered soft robotic fingers (SRF) and one multi-mode suction cup. The SRF is customised to avoid interference with obstacles during grasping, and its compliance and force exertion are comprehensively evaluated with FEA and experiments. The multi-mode suction cup can provide suction adhesion force, show active extrusion/withdrawal, and present passive compliance mode. The simultaneously twist-pulling motion is finally proposed and implemented to detach the apples from the trees. The proposed robotic gripper is compact, compliant with apple grasping and generates a large grasping force. Our proposed method is finally validated in a natural orchard and achieves a detachment, damage and harvesting rate of 75.6%, 4.55%, and 70.77%, respectively.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Reference [2] also reviews end-effector designs, and Ref. [1] compares eight end-effectors. Notable examples of recent gripper designs include a three-finger gripper based on the fin-ray concept [4], a gripper with four pneumatically actuated soft robotic fingers [5], and a bistable gripper with two fingers [6]. ...
... Some of the studies on these highly adaptable soft robotic grippers in the agro-industry include Refs. [4,5]. Lumped compliance grippers, on the other hand, are designed to deform only at specific hinge locations, similar to rigid link mechanisms. ...
... 16Grasping process. From the undeformed state (1), the gripper opens (2), after which it moves to the object (3, 4) and closes(5,6). ...
Article
The design of grippers for the agro-industry is challenging. To be cost-effective, the picked object should be moved around fast requiring a firm grip on fruit of different hardnesses, shapes, and sizes without causing damage. This paper presents a self-adaptive flexure-based gripper design optimized for high acceleration loads. A main novelty is that it is actuated through a push-pull flexure that is loaded in tension when the gripper closes, allowing it to handle high actuation forces without the risk of buckling. To create a robust gripper that can handle relatively high loads, flexures are used that are reinforced and have a thickness variation over the length. The optimal thickness distribution of these flexures is derived analytically to facilitate the design process. The derived principles are generally applicable to flexure hinges. The resulting advanced cartwheel flexure joint, as used in this gripper, has a 2.5 times higher support stiffness and a 1.5 times higher buckling load when compared to a conventional cartwheel joint of the same size and actuation stiffness. The PP-gripper is numerically optimized for a high pull-out force, using analytical design insights as starting point. The gripper can grip circular objects with radii between 30 and 40 mm. The pull-out force is 21.4 N, with a maximum actuation force of 100 N. Good correspondence is found between the geometric design approach, the numerically optimized design, and the results of the experimental validation.
... Robots in Traptic 2 and FFRobotics 3 use conveyors to stop fruit damage while being collected, while Kang et al. [37] robotic apple harvesting system, Wei et al. [38] apple-picking robot's trial base for gripping, and Wang et al. [39] apple picking technique at Monash. The overview offered by ) gave a clear understanding of the apple harvesting concept, structure, and primary outcomes [40]. ...
... The maximum bending angle and contact force were 22.6 and 5.72 N, respectively; better ability and more considerable tensile strength were achieved when grasping the finger's base. The unique soft robotic gripper proposed by Xing et al. [39] has suction cups with active motion, passive compliance, and tapered delicate robotic fingers (Fig. 5C). The tapered SRF is a single-step, 3D-printed, customized, flexible bending actuator. ...
... The development of the components that make up the apple harvesting equipment and their capacity to boost the machine and robot's efficiency is crucial. The design and manufacturing of arms and grippers with superior specifications have been the focus of research [106][107][108]39], and each of their efforts has improved the performance of harvesting robots, which has sped up and facilitated the process of harvesting apples. ...
Article
Full-text available
Harvesting apples is one of the most apple-challenging operations; its process is labor-intensive, and for various reasons, automation has yet to advance as swiftly as it might. Researchers have concentrated on developments in robotics and automated apple harvesting, two domains with a plethora of opportunities and difficulties that require more evaluation for future growth and quality. In this paper, we provide an overview of apple harvesting by beginning with a perspective that focuses on vision techniques and recognition systems. We then cover the outcomes, methods, time, and observations via robust analysis, including visible light, spectral, and thermal imaging. After that, we were followed by the localization of the apple, which aids in detaching apples from branches, leaves, and other overlapping apples, besides directing end-effectors to grip and remove apples. Next, the harvester robots progress includes developments in machinery and equipment that contain grippers, arms, and manipulators, which speed up operations and upgrade performance. Additionally, the platforms that provide aid for harvesting boost productivity, reduce the demand for strength, and lower the danger of accidents at work. Furthermore, the discussion part includes a comprehensive analysis covering works on apple detection systems and automated apple harvesting robot technology. Finally, we summarize the challenges, limitations, opportunities, and future perspectives and provide the trends and technologies. This research offers several avenues for future automated apple harvesting advancement and interaction with other fields that attract investment firms, such as sorting and bagging. Assist in sustaining the expansion of research communities and offering services to raise the yield and quality of apple fruit.
... Unlike electrically controlled robots often consisting of rigid materials, robots with soft materials are better adapted to unpredictable interactions with unknown objects [1]. Recently, various types of robotic grippers have been developed for applications such as humanoid robots, advanced manufacturing, and agricultural automation [2][3][4]. Especially, many researchers have focused on pneumatic-powered grippers due to their fast and reliable operation, low cost, simple design, and effective forceto-weight ratio. However, a fundamental limitation of these pneumatic grippers is their precise control. ...
... To understand the bending curvature and inner pressure of the pneumatic gripper, a FEA is often used [3,15,[21][22][23][24]. Given that our pneumatic gripper was designed using Ecoflex 100% for the main structure, while three distinct combinations (Ecoflex 60%/PDMS 40%, Ecoflex 50%/PDMS 50%, Ecoflex 40%/PDMS 60%) were adopted for the cover structure, it was necessary to obtain preliminary data from strainstress relationships to attempt simulation analyses with the Yeoh hyper-elastic model [25][26][27][28]. ...
Article
Full-text available
A pneumatic soft gripper, composed of multiple elastomeric materials, was designed and manufactured based on gripping simulations. The simulation results demonstrated that the different stiffnesses of the elastomeric materials can influence the internal air pressure required for the proper actuation of the gripper. Considering the substrate properties and the morphological changes of the soft gripper during gripping, a piezoresistive force sensor was developed using elastomers and conductive filaments with the aid of additive manufacturing techniques. We confirmed the reproducibility and stability of the proposed piezoresistive force sensor through evaluations under various fabrication conditions. Results from touch experiments and compressive force measurements indicated that the stiffness of the sensor substrate and the thickness of the conductive part of the sensor affected the sensitivity and reliability of the sensor with respect to different levels of applied forces. Incorporating a rigid panel between the soft gripper and the piezoresistive force sensor diminished the effect of the variable stiffness and curvature of the gripper on the measurement of electrical resistance generated by the piezoresistive force sensor. Our 3D-printed sensor combined with elastomeric materials showed the possibility of differentiating the simple actuation and the gripping demonstration of the soft gripper.
... Fan et al. [27] investigated two gripping modes, vertical and horizontal, as well as four picking modes, vertical pulling, horizontal pulling, verticalrotary pulling, and horizontal-rotary pulling, of a three-finger soft gripper, as shown in Figure 2b. The experiment showed that the lowest three-finger grip force required for horizontal-rotational pull picking under horizontal grasping conditions was 10.33 N. Wang et al. [12] designed a four-finger soft gripper for apple picking, as shown in Figure 2c. The robot structure consists of four tapered soft fingers and a multi-modal suction cup. ...
... (d) The soft gripper based on collective mechanics. Figure taken from[12,[26][27][28]. ...
Article
Full-text available
This review summarizes the important properties required for applying soft grippers to agricultural harvesting, focusing on their actuation methods and structural types. The purpose of the review is to address the challenges of limited load capacity and stiffness, which significantly hinder the broader application of soft grippers in agriculture. This paper examines the research progress on variable stiffness methods for soft grippers over the past five years. We categorize various variable stiffness techniques and analyze their advantages and disadvantages in enhancing load capacity, stiffness, dexterity, degree of integration, responsiveness, and energy consumption of soft grippers. The applicability and limitations of these techniques in the context of agricultural harvesting are also discussed. This paper concludes that combined material variable stiffness technology with a motor actuation claw structure in soft grippers is better suited for agricultural harvesting operations of woody crops (e.g., apples, citrus) and herbaceous crops (e.g., tomatoes, cucumbers) in unstructured environments.
... Though the harvest-assist platform can help increase workers' harvest efficiency, it still requires manual involvement in harvesting (Bu et al., 2020), and cannot eliminate the labour shortage issue. Therefore, researchers have shifted to the development of harvesting robotics to fully replace workers from tedious, dangerous, and laborious harvest operations (Subeesh & Mehta, 2021;Wang et al., 2023;Xu et al., 2022;Zhou et al., 2022). ...
... However, the excessive tilt angle of the end-effector when picking fruit in high places made it challenging to establish a good suction with the apples, resulting in a successful picking rate of only 47%. Wang et al. (2023) developed a new gripper comprising four tapered soft robotic fingers (SRF) and a multi-mode suction cup. The major advantage of the SRF gripper is that it is flexible and allows better contact with the curved apple surface (Elfferich et al., 2022). ...
Article
Timely harvesting of fresh apples faces challenges due to labour shortage, and the modern approach of robotic harvesting has the potential to address this issue. The prevailing process of apple harvest robotics could not meet the demands of practical applications, mainly due to the lack of a suitable manipulator, because the existing ones are associated with low picking rates, fruit damage, and high costs. A prototype apple harvesting manipulator was developed, which includes a vacuum three-revolute-degrees-of-freedom end-effector, a three-prismatic-degrees-of-freedom Cartesian system, an RGB-D camera, and system integration. The vision positioning system and controller were designed to realise precise positioning and detachment of the manipulator. The major contribution of the current study is the three-revolute-degrees-of-freedom vacuum suction end-effector, whose performance evaluation was conducted in a commercial apple orchard. Experimental results showed that a 33ϕ mm diameter suction cup achieved superior performance over a 43ϕ mm cup. The method of rotation followed by pull proved to be more effective than only pulling for apple detachment. The results indicated that the apple’s equatorial region was the optimal area for suction. Furthermore, the vacuum pressure should be at least −65 kPa to guarantee successful detachment. Experimental results showed that 83.1% of harvested apples had stems intact. For the developed manipulator, a 33ϕ mm diameter suction cup, a rotate-and-pull separation method, and −65 kPa were recommended for practical applications. With the integrated new manipulator, the developed apple harvest robot has been demonstrated to have the potential to realise robotic apple harvesting.
... So in ref. [6], the authors propose a three-finger gripping device for harvesting apples, which has flexible fingers that are unable to squeeze the apple by damaging it, while the gripping force is sufficient to separate the fruit. There are promising studies aimed at creating soft gripping devices that do not have rigid elements capable of damaging fruits [7,8]. These gripping devices work on the basis of pneumatics, which makes it possible to reduce the dimensions of the gripping device by moving the motors outside the manipulator. ...
... The choice of the compression force parameter of the fruit F E is based on the data given in refs. [7,25,26], which presents studies on the force to separate an apple from a tree for five varieties. It is taken into account that the gripping force at the torque of separation of the apple from the tree branch varied from 2.4 to 57.6 N, according to which the gripping device should provide a gripping force of at least 57.6 N. Also in ref. [26], the value of the permissible surface pressure on the apple was obtained equal to 56.97 Pa, which will be used by us for the most effective modeling. ...
Article
Gripping devices for harvesting fruits have such types of work as cutting, tearing and unscrewing. For apples, it is preferable to use slicing or unscrewing, while the fruit leg should not remain, damaging the apple during storage. In this article, we are developing a grab for harvesting apples. The gripper is used both for holding the fruit and for jamming, followed by unscrewing. One of the advantages is that the proposed method of collecting apples allows you not to waste time moving the manipulator from the tree to the basket, but only to grab and tear them off. The fruit enters the gripper device; after which it enters the fruit collection container through a rigid or flexible pipe. The gripper device is built on the basis of a ball-screw transmission, which is supplemented by a gear drive along the helical surface. This allows for rotation and rectilinear movement of the held fruit. The gripping device has a ratchet mechanism that allows you to fix the fruit. A mathematical model of the gripper device has been developed, which allows determining the torque of the engine depending on the position of the fingers. The parameters of the mechanism were optimized using a genetic algorithm, and the results are presented in the form of a Pareto set. A 3D model of the gripper device has been built and a layout has been developed using 3D printing. Experimental laboratory and field tests of the gripping device were carried out.
... However, soft grippers are challenged when it comes to picking heavier fruits such as avocados, and special designs to regulate how soft fingers deform are needed. Quite recently, an alternative method employing hybrid actuators that combine soft fingers and suction cups was proposed to pick heavier fruits like apples [21]. Another commonly used type of grippers includes rigid-based ones. ...
... Moreover, we tried to use a soft finger structure (made of Formlabs Form 3 Flex-80A resin) in place of the rigid one, but it was impossible to hold the avocado tightly. We note that a hybrid design with fingers and a suction cup (like in related work for picking apples [21]) may be possible. However, this would come at a higher mechanism design and control operation complexity, whereas our proposed endeffector affords direct and intuitive operation. ...
... However, soft grippers are challenged when it comes to picking heavier fruits such as avocados, and special designs to regulate how soft fingers deform are needed. Quite recently, an alternative method employing hybrid actuators that combine soft fingers and suction cups was proposed to pick heavier fruits like apples [21]. Another commonly used type of grippers includes rigid-based ones. ...
... Moreover, we tried to use a soft finger structure (made of Formlabs Form 3 Flex-80A resin) in place of the rigid one, but it was impossible to hold the avocado tightly. We note that a hybrid design with fingers and a suction cup (like in related work for picking apples [21]) may be possible. However, this would come at a higher mechanism design and control operation complexity, whereas our proposed endeffector affords direct and intuitive operation. ...
Preprint
Full-text available
Robot-assisted fruit harvesting has been a critical research direction supporting sustainable crop production. One important determinant of system behavior and efficiency is the end-effector that comes in direct contact with the crop during harvesting and directly affects harvesting success. Harvesting avocados poses unique challenges not addressed by existing end-effectors (namely, they have uneven surfaces and irregular shapes grow on thick peduncles, and have a sturdy calyx attached). The work reported in this paper contributes a new end-effector design suitable for avocado picking. A rigid system design with a two-stage rotational motion is developed, to first grasp the avocado and then detach it from its peduncle. A force analysis is conducted to determine key design parameters. Preliminary experiments demonstrate the efficiency of the developed end-effector to pick and apply a moment to an avocado from a specific viewpoint (as compared to pulling it directly), and in-lab experiments show that the end-effector can grasp and retrieve avocados with a 100% success rate.
... Robotic harvesters are custom agricultural systems typically equipped with robotic arms, mobile platforms, sensors, and end-effectors [12][13][14]. Since studies dating back to the 1980s [15,16], manipulator designs [17,18], grasping strategies [19][20][21], motion control techniques [22,23], and sensing methods [24,25] for robotic pickers have continued to evolve to adapt to current technological trends. However, unlike traditional industrial robotic manipulators that work in well-structured indoor environments, harvesting robots are exposed to various environmental effects. ...
Preprint
This paper presents the challenges agricultural robotic harvesters face in detecting and localising fruits under various environmental disturbances. In controlled laboratory settings, both the traditional HSV (Hue Saturation Value) transformation and the YOLOv8 (You Only Look Once) deep learning model were employed. However, only YOLOv8 was utilised in outdoor experiments, as the HSV transformation was not capable of accurately drawing fruit contours. Experiments include ten distinct fruit patterns with six apples and six oranges. A grid structure for homography (perspective) transformation was employed to convert detected midpoints into 3D world coordinates. The experiments evaluated detection and localisation under varying lighting and background disturbances, revealing accurate performance indoors, but significant challenges outdoors. Our results show that indoor experiments using YOLOv8 achieved 100% detection accuracy, while outdoor conditions decreased performance, with an average accuracy of 69.15% for YOLOv8 under direct sunlight. The study demonstrates that real-world applications reveal significant limitations due to changing lighting, background disturbances, and colour and shape variability. These findings underscore the need for further refinement of algorithms and sensors to enhance the robustness of robotic harvesters for agricultural use.
... Indeed, in addressing the complex challenges posed by the diverse sizes, irregular shapes, and varying orientation of cassava main stems, the utilization of a robotic manipulator stands out as a promising solution. Moreover, robotics is widely used in agriculture, can solve the high labor requirement problem, and avoid waste from unharvested products [16,17]. Robotic manipulators are composed of individual components that enable them to perform specific functions [18]. ...
... Harvesting an apple from an orchard, for instance, requires a pre-programmed path planning and grasping sequence. The robotic arm must first navigate through dense branches and leaves, then delicately grasp and twist the fruit to detach it from the tree, and finally withdraw it with the robotic arm [5]. In industrial packing, re-grasps are usually required to correctly orient the arbitrarily placed objects for downstream assembly or kitting [6]. ...
Preprint
Full-text available
The ability of robotic grippers to not only grasp but also re-position and re-orient objects in-hand is crucial for achieving versatile, general-purpose manipulation. While recent advances in soft robotic grasping has greatly improved grasp quality and stability, their manipulation capabilities remain under-explored. This paper presents the DexGrip, a multi-modal soft robotic gripper for in-hand grasping, re-orientation and manipulation. DexGrip features a 3 Degrees of Freedom (DoFs) active suction palm and 3 active (rotating) grasping surfaces, enabling soft, stable, and dexterous grasping and manipulation without ever needing to re-grasp an object. Uniquely, these features enable complete 360 degree rotation in all three principal axes. We experimentally demonstrate these capabilities across a diverse set of objects and tasks. DexGrip successfully grasped, re-positioned, and re-oriented objects with widely varying stiffnesses, sizes, weights, and surface textures; and effectively manipulated objects that presented significant challenges for existing robotic grippers.
... The apple-harvesting robot introduced in Zhang's paper [32] also utilized fluid pipelines and pneumatic principles to pick apples. In contrast, Wang et al. [33] developed a more novel and advanced apple-harvesting robot gripper, consisting of four conical soft fingers and a suction cup. After the fingers grasp the apple, the suction cup attaches to the target, and the claws close, rotating and pulling to pick the apple from the tree. ...
Article
Full-text available
To address the issue of automated apple harvesting in orchards, we propose a YOLOv5-RACF algorithm for identifying apples and calculating apple diameters. This algorithm employs the robot operating dystem (ROS) to control the robot’s locomotion system, Lidar mapping, and navigation, as well as the robotic arm’s posture and grasping operations, achieving automated apple harvesting and placement. The tests were conducted in an actual orchard environment. The algorithm model achieved an average apple detection accuracy (mAP@0.5) of 98.748% and a (mAP@0.5:0.95) of 90.02%. The time to calculate the diameter of one apple was 0.13 s, with a measurement accuracy within an error range of 1–3 mm. The robot takes an average of 9 s to pick an apple and return to the initial pose. These results demonstrate the system’s efficiency and reliability in real agricultural environments.
... This is mainly due to the complexity of the unstructured agricultural environment, the intrinsic challenge posed by soft materials, and the need to demonstrate the economic viability of robotic harvesting in the sector [89]. Results of a recent study on the evaluation of a robust soft robotic gripper for apple harvesting in a natural orchard have shown a detachment, damage, and harvesting rate of 75.6%, 4.55%, and 70.77%, respectively [156]. Some of the main barriers that soft robotics, and more particularly soft grippers, face against their possible application in agriculture can be categorized as the design process [101], standardizing manufacturing methods [89], and features quantification (i.e., repeatability, reliability, and controllability). ...
... However, the grasping reliability should be improved due to the limited load capacity. To enhance the grasping reliability, X. Wang et al. (2023) developed a robust 3D-printed pneumatic soft bellow gripper with a suction cup for an apple grasping robot. The proposed gripper and the suction cup achieved high compliance and stability when grasping apples. ...
Conference Paper
Current grippers show limited performance when grasping mushrooms with an umbrella shape, raising the demand for a balance between non-destructivity and stability. Soft grippers, integrated with biomimetic approaches, are promising solutions for grasping mushrooms. Inspired by the variable-curvature structure of umbrella-shaped mushrooms, this study presented a biomimetic pneumatic soft gripper (BPSG) that mainly comprised three soft variable-curvature fiber-reinforced bending actuators (VCFRBAs), dedicated to grasping umbrella-shaped mushrooms. The finite element method was employed to predict the deformation of the VCFRBA, and the results showed good matches with the experiments. Step response tests demonstrated the rising time of VCFRBA was 0.46 s, suggesting the proposed BPSG had a fast response speed. Moreover, force experiments indicated that each VCFRBA can generate a clamping force of 1.52 N with an additional lifting force as high as 2.11 N under the actuation pressure of 0.30 MPa. The pull-off forces of 65.23 N were also measured for the BPSG at this pressure. Finally, the results of the grasping experiment showed a 100% success rate and 0% damage rate when the proposed BPSG grasped mushrooms positioned in an upright position, demonstrating good non-destructivity and stability. This study showcased a novel BPSG, providing a unique solution to grasping umbrella-shaped mushrooms for agricultural robots.
... Activated by heat and temperature, navigated by light beam [11], drove by compressed fluids [12], grasp can be realized in different ways. Advanced structures are also born in laboratories from time to time, Xing Wang and his research team have created a gripper with four tapered soft fingers and one multi-mode suction cup to augment the productivity of fruit collecting in agriculture field by increasing the grasping force during harvest [13]. The range of its application and its reliability have also been greatly improved [14]. ...
Article
Full-text available
Soft robotics and soft robots are gaining popularity these days due to their advantage like adaptability. Multistable structure, a special structure that can stay stable under more than one state, provides people with the capacity of the researching open-loop control method in the domain of robotics. Thus, the characteristics of multistable structure can benefit the design and simplify the control of robots. This article introduces an energy-based analytical model for soft multistable grippers. It can design the configuration of the gripper, predict its stable states and set the force of grasping action and this model can save much time spent on calculation comparing to traditional method of Finite Element (FE) simulations under a sufficiently low error rate. The idea of topology, certain algorithms and simulations are applied and experiments are also conducted later to certify the feasibility and accuracy of this model. This elaborate model is a new inspiration for soft robotics.
... For example, Chen et al. used a compliant structure gripper with fin effect [11] to develop a robotic apple harvesting system, which can realize adaptive picking of apples with a picking success rate of 62.8 % [12]. They then designed a flexible tapered finger and, based on this, developed a soft gripper, increasing the picking success rate to 70.77% [13]. Inspired by duck feet and octopus tentacles, Cai et al. [14] proposed a pneumatically webbed soft gripper, which added four webs to the four pneumatically flexible fingers. ...
Article
Full-text available
Adaptability to unstructured objects and the avoidance of target damage are critical challenges for flexible grippers in fruit-picking robots. Most existing flexible grippers have many problems in terms of control complexity, stability and cost. This paper proposes a flexible finger design method that considers contact behavior. The new approach incorporates topological design of contact targets and introduces contact stress constraints to directly obtain a flexible finger structure with low contact stress and good adaptability. The study explores the effects of design parameters, including virtual spring stiffness, volume fraction, design domain size, and discretization, on the outcomes of the flexible finger topology optimization. Two flexible finger structures were selected for comparative analysis. The experimental results verified the effectiveness of the design method and the maximum contact stress was reduced by about 70%. An adaptive two-finger gripper was developed. This design allows the gripper to achieve damage-free grasping without additional sensors and control systems. The adaptive and contact performances of the grippers with different driving modes were analyzed. Practical grasping tests were also performed, including evaluation of adaptive performance, stability, and maximum grasping weight. The results indicate that gripper 2 with flexible finger 2 excelled in contact stress and adaptive wrapping, making it well-suited for grasping unstructured and fragile objects. This paper provides valuable insights for the design and application of flexible grippers for picking robots, offering a promising solution to enhance adaptability while minimizing target damage.
... 16 To address challenges such as the small size and similar color of strawberry seedling flowers and fruits, researchers proposed an improved algorithm based on YOLOv7 to recognize seedling flowers and fruits accurately, achieving average accuracies of 94.7% and 89.5%, respectively. 17 The skin texture of strawberry is fragile, and using a grasping method similar to that used for apples, 18 citrus, 19 and kiwifruits 20 can easily cause damage. To ensure the quality and freshness of strawberry, cutting the peduncle is considered an effective method to separate the fruit from the plant. ...
Article
Full-text available
Background Strawberry, being an important economic crop, requires a large amount of human labor for harvesting operations. Efficient and non‐destructive harvesting by strawberry harvesting robots requires the precise location of the picking points. Current algorithms for locating picking points encounter significant issues with location errors and minimal effective information in complex situations. Results To improve the accuracy of the location of picking points, this study proposes a visual location method based on composite models. This method employs object detection and instance segmentation models to detect fruits and segment peduncles sequentially, thereby enabling the identification of picking points and inclination on the peduncle. Different object detection algorithms and instance segmentation models were validated to explore the optimal model combination, and the Convolutional Block Attention Module (CBAM) was integrated into YOLOv8s‐seg to construct YOLOv8s‐seg‐CBAM. Test results show that the composite model built with YOLOv8s and YOLOv8s‐seg‐CBAM achieved a peduncle detection accuracy of 86.2%, with an inference time of 30.6 ms per image. Conclusion The picking point visual location method based on YOLOv8s and YOLOv8s‐seg‐CBAM composite models can better balance accuracy and efficiency and can provide more accurate guidance for automated harvesting. © 2024 Society of Chemical Industry.
... Deformation conditioning in PneuNets silicon was proposed by Bhat et al. (2020) using a 3D-printed TPU framework with strain limiting and PLA plates (Dragon Skin 10 A hollow interior was 3D-printed with water-soluble PVA. A unique soft robotic gripper and detachment mechanism were presented by Wang et al. (2023) for apple picking in a real plantation. It has four tapered fingers which were 3D printed using super-elastic Ninjaflex and a suction grip. ...
Article
Full-text available
Many robotic systems face substantial challenges when trying to grasp and manipulate objects. Thought of initially as humanoid automata a century ago, this viewpoint is still influential in modern robot design. Many robotic grippers are inspired by the deftness of the human hand. The perceptual, processing, and control issues that conventional grippers have are also experienced by soft-fingered grippers. Precise finger placement, dictated by the shape and attitude of the object, is necessary to accomplish force closure when using soft fingertips to grasp. Soft robotic end-effectors have several advantages, such as a good interface with humans, the capacity to adapt to different environments, a number of degrees of freedom, and the ability to non-destructively grasp items of various shapes. Adding to earlier research that looked at the soft robot in a theoretical way, this study creates an optimized model based on the deformation in terms of bending of the channel cavity under applied pneumatic pressure. A correlation between pneumatic pressure and the pneumatic soft actuator's bending angle has been demonstrated. This research looks at how different design factors affect the bending of a multi-chambered soft actuator that is pneumatically networked. The finite element approach involves fine-tuned (optimised) actuator construction. Using FEM to evaluate aspects affecting actuator mechanical output, the ideal design parameters were discovered using DoE, resulting in a bending angle of ~ 104 degrees at 30 kPa. This study used ANOVA at a 5% significant level to identify which variables most affected the pneumatic actuator's deformation (bending angle). The significant R-square value of 96.42% supports the study's conclusions that the parameters utilised explain an immense percentage of bending angle deviations. Experimental verification of the optimized finite element model findings was conducted. The verification of the actuators' bending angles and output forces reveals that the discrepancy between the two sets of data stayed below 9%. Also, the average gripping success rate attained in the grasping evaluation, which involved four distinct types of items, was almost 97%.
Chapter
Smart manufacturing does not aim for a fully unmanned approach. The role of humans in complex manufacturing processes may never be completely replaceable.
Article
Full-text available
This study presents a novel 3D-printed grabber designed to assess the mechanical properties of pineapples, crucial for optimizing harvesting and handling processes. This research represents a key component of a larger project aimed at developing a semi-automatic pineapple harvester tailored for the challenging terrain of hilly regions, such as those found in Manipur, India, where the pineapple data for this study was collected. Key physical parameters like including transverse diameter, weight, longitudinal diameter, and compression force were evaluated. The grabber was designed using 3D modelling software and fabricated via 3D printing, enabling rapid prototyping and customization. A compression force sensor unit integrated into the grabber allowed for real-time force measurement during testing. Experimental validation involved applying controlled compression forces to pineapples of varying sizes and maturities. The results demonstrated that the grabber could effectively measure compression force while ensuring minimal fruit damage, even when exceeding the required gripping force. The study successfully determined a safe grabbing force range for pineapples, informing the development of robotic or semi-automatic harvesting systems. Furthermore, the adaptable design of the force-sensing grabber presents a promising approach for assessing the mechanical properties of other fruits and vegetables, potentially reducing damage and improving handling practices across the agricultural sector.
Article
In order to address the issues of poor flexibility and complex control systems in existing picking manipulators, the present work designs a multi-degree-of-freedom cable-driven underactuated manipulator. Firstly, an end-effector drive module based on the cam mechanism is designed, which can control gripping force without the need for force sensor feedback, and enables the opening, closing, and rotary movements of the end-effector to be controlled by the same motor. Secondly, the influence of the structural parameters of the fingers on the contact force at the phalange is analyzed, and then the end-effector is optimized using a genetic algorithm. Next, the cable pulling force control model is established. Subsequently, a prototype manipulator is fabricated for testing, the results show that the adjustment range of cable pulling force is 11.1–27.4 N, and the output torque of manipulator is 0.64 N·m. Finally, the results of picking test in the orchard demonstrate that the manipulator is capable of accomplishing non-destructive picking operations.
Article
Full-text available
Smart dry adhesives, a rapidly growing class of intelligent materials and structures, are engineered to provide strong, robust adhesion when needed while also allowing for controlled, easy detachment in response to specific stimuli. Traditional smart adhesives, often exemplified by fibrillar structures made of elastomers, face a number of challenges. These include limitations on maximum adhesion strength imposed by microstructural dimensions, restricted adaptability to surfaces with varying degrees of roughness, and an inherent trade-off between adhesion strength and switchability. This review explores how shape memory polymers (SMPs) can address these challenges and, through their rubber-to-glass (R2G) transition capability, provide a powerful foundation for the next generation of smart dry adhesives. Specifically, we summarize and elucidate the mechanisms by which SMPs enhance adhesion strength and switchability through material characteristics such as tunable stiffness, shape-locking, and shape-memory effects. Additionally, we discuss a wide range of innovative designs and applications of SMP adhesives, offering insights into the ongoing challenges and emerging opportunities in this rapidly evolving field.
Article
Full-text available
Robotic technologies are affording opportunities to revolutionize the production of specialty crops (fruits, vegetables, tree nuts, and horticulture). They offer the potential to automate tasks and save inputs such as labor, fertilizer, and pesticides. Specialty crops are well known for their high economic value and nutritional benefits, making their production particularly impactful. While previous review papers have discussed the evolution of agricultural robots in a general agricultural context, this review uniquely focuses on their application to specialty crops, a rapidly expanding area. Therefore, we aimed to develop a state-of-the-art review to scientifically contribute to the understanding of the following: (i) the primary areas of robots' application for specialty crops; (ii) the specific benefits they offer; (iii) their current limitations; and (iv) opportunities for future investigation. We formulated a comprehensive search strategy, leveraging Scopus ® and Web of Science™ as databases and selecting "robot" and "specialty crops" as the main keywords. To follow a critical screening process, only peer-reviewed research papers were considered, resulting in the inclusion of 907 papers covering the period from 1988 to 2024. Each paper was thoroughly evaluated based on its title, abstract, keywords, methods, conclusions, and declarations. Our analysis revealed that interest in agricultural robots for specialty crops has significantly increased over the past decade, mainly driven by technological advancements in computer vision and recognition systems. Harvesting robots have arisen as the primary focus. Robots for spraying, pruning, weed control, pollination, transplanting, and fertilizing are emerging subjects to be addressed in further research and development (R&D) strategies. Ultimately, our findings serve to reveal the dynamics of agricultural robots in the world of specialty crops while supporting suitable practices for more sustainable and resilient agriculture, indicating a new era of innovation and efficiency in agriculture.
Chapter
Currently, apple harvesting primarily relies on manual picking, which presents issues such as low efficiency, high costs, and frequent occupational health problems. As the labor shortage intensifies, the timely harvest of fresh apples faces significant challenges, making the use of robots to replace human labor an inevitable trend. With the groundbreaking advancements in standardized orchard production techniques and artificial intelligence technology, apple harvesting automation has achieved new breakthroughs. The key technologies of apple picking robots include apple recognition, apple positioning, robotic arms, motion planning, and end effectors. The performance of these technologies directly determines the efficiency and quality of the harvesting process. This paper provides an overview of the current development status of apple-picking robots. It reviews methods for apple recognition, introduces various sensor-based positioning technologies, studies robotic arms used for apple harvesting, discusses related motion planning algorithms, and analyzes end effectors and their detachment methods. The paper also introduces apple-picking robots that have already been commercialized and identifies the existing issues such as long picking cycles, low success rates, severe fruit damage, and high costs. It points out future directions for commercial development, specifically focusing on reducing the complexity of orchard environments, optimizing recognition and motion planning algorithms, integrating multiple sensors, and enhancing multi-robotic arm cooperation.
Chapter
The timely harvesting of fresh apples faces challenges due to labor shortages. Robotic harvesting has the potential to alleviate the labor intensity on operators and liberate additional agricultural labor. However, the current success rate of apple-picking robots is low, resulting in fruit damage, high operational costs, and an inability to meet market demand. In this research, a prototype apple harvesting manipulator was designed and constructed, providing a solid foundation for future advancements. The manipulator prototype includes a vacuum three-rotational (3R) degrees-of-freedom (DoF) end-effector, a 3-DoF Cartesian system, an RGB-D camera, and a mobile vehicle. Vision-assisted positioning system and manipulator controller were designed to achieve precise positioning and separation motions of the manipulator. The harvesting experiment was conducted in a dwarf, densely planted spindle apple orchard. Results from the experiment indicate that the rotation followed by pull mode proved more effective than the pull mode for apple picking. In the critical negative pressure environment of − 65 kPa employed for apple picking, utilizing 33 mm suction cups has demonstrated an 81.7% separation success rate. The diameter center as the optimal theoretical point for adsorption with an allowable two-dimensional error mean value of 9.62%. The manipulator ensures essentially damage-free picking of target fruits, demonstrating a stem damage rate of 17.1%. The developed picking prototype successfully accomplishes the task of apple harvesting, boasting main advantages of compact size, low cost, non-destructiveness, and high flexibility.
Article
Full-text available
The food industry increasingly depends on technological assets to improve the efficiency and accuracy of fruit processing and quality control. This article enhances the application of computer vision with collaborative robotics to create a non-destructive system. The system can automate the detection and handling of fruits, particularly tomatoes, reducing the reliance on manual labor and minimizing damage during processing. This system was developed with a Raspberry Pi 5 to capture images of the fruit using a PiCamera module 3. After detecting the object, a command is sent to a Universal Robotics UR3e robotic arm via Ethernet cable, using Python code that integrates company functions and functions developed specifically for this application. Four object detection models were developed using the TensorFlow Object Detection API, converted to TensorFlow Lite, to detect two types of fruit (tomatoes) using deep learning techniques. Each fruit had two versions of the models. The models obtained 67.54% mAP for four classes and 64.66% mAP for two classes, A rectangular work area was created for the robotic arm and computer vision to work together. After 640 manipulation tests, a reliable area of 262 × 250 mm was determined for operating the system. In fruit sorting facilities, this system can be employed to automatically classify fruits based on size, ripeness, and quality. This ensures consistent product standards and reduces waste by sorting fruits according to pre-defined criteria. The system’s ability to detect multiple fruit types with high accuracy enables it to integrate into existing workflows, thereby increasing productivity and profitability for food processing companies. Additionally, the non-destructive nature of this technology allows for the inspection of fruits without causing any damage, ensuring that only the highest-quality produce is selected for further processing. This application can enhance the speed and precision of quality control processes, leading to improved product quality and customer satisfaction.
Article
Apples are among the most popular fruits globally due to their health and nutritional benefits for humans. Artificial intelligence in agriculture has advanced, but vision, which improves machine efficiency, speed, and production, still needs to be improved. Managing apple development from planting to harvest affects productivity, quality, and economics. In this study, by establishing a vision system platform with a range of camera types that conforms with orchard standard specifications for data gathering, this work provides two new apple collections: Orchard Fuji Growth Stages (OFGS) and Orchard Apple Varieties (OAV), with preliminary benchmark assessments. Secondly, this research proposes the orchard apple vision transformer method (POA-VT), incorporating novel regularization techniques (CRT) that assist us in boosting efficiency and optimizing the loss functions. The highest accuracy scores are 91.56 % for OFGS and 94.20 % for OAV. Thirdly, an ablation study will be conducted to demonstrate the importance of CRT to the proposed method. Fourthly, the CRT outperforms the baselines by comparing it with the standard regularization functions. Finally, time series analyses predict the ‘Fuji’ growth stage, with the outstanding training and validation RMSE being 19.29 and 19.26, respectively. The proposed method offers high efficiency via multiple tasks and improves the automation of apple systems. It is highly flexible in handling various tasks related to apple fruits. Furthermore, it can integrate with real-time systems, such as UAVs and sorting systems. This research benefits the growth of apple’s robotic vision, development policies, time-sensitive harvesting schedules, and decision-making.
Preprint
Full-text available
Robotic technologies are bringing opportunities to revolutionize the production of specialty crops through task automation along with the potential of saving valuable inputs such as labor, fertilizer, and pesticides. Considering the prominence of agricultural robots and their recent expansion into specialty crops, we aimed to develop a state-of-the-art review to scientifically contribute to the understanding of: i) the primary areas of robots’ application for specialty crops; ii) the specific benefits they offer; iii) their current limitations; and iv) suggest opportunities for future investigation. Therefore, we initially formulated a comprehensive search strategy, leveraging Scopus® and Web of ScienceTM as databases and selecting "robot" and "specialty crops" as the main keywords. As a critical screening process, only peer-reviewed papers and original research were considered, resulting in the inclusion of 706 papers covering the period from 1988 to 2023. Each paper was thoroughly evaluated regarding its title, abstract, keywords, methods, conclusions, and declarations. As a result, interest in agricultural robots for specialty crops has significantly increased over the past decade, driven by advancements in vision and recognition systems. Harvesting robots arose as the primary focus. Conversely, robots for spraying, weed control, pruning, pollination, transplanting, and fertilizing are emerging subjects to be addressed in further research and development strategies. Ultimately, our findings serve to reveal the dynamics of agricultural robots in the world of specialty crops, while supporting suitable practices for more sustainable and resilient agriculture, undoubtedly indicating a new era of innovation and efficiency in agriculture.
Article
Full-text available
The growing interest in the flexibility and operational capabilities of soft manipulators in confined spaces emphasizes the need for precise modeling and accurate motion control. Conventional control methods encounter difficulties in modeling and involve intricate computations. This work introduces a novel deep reinforcement learning (DRL) control algorithm based on neural network modeling. Using the Whale Optimization Algorithm, an approximate dynamic model for the soft manipulator is established. The twin delayed deterministic policy gradient is employed for DRL control. Domain randomization is applied during pretraining in a simulated environment. The algorithm addresses issues related to dependency on measurement data quality and redundant mappings, outperforming other methods by 8–15 mm in control accuracy. The trained DRL controller achieves precise trajectory tracking within the soft manipulator's task space, enabling successful grasping tasks in various complex environments, including pipelines and other narrow spaces. Experimental results confirm the autonomy of our controller in performing these tasks without human intervention.
Article
Full-text available
Due to a shortage of labor, the harvesting of fruits and vegetables faces significant challenges. Soft robotic hands, adaptable to variable environments due to their high-curvature bending and twisting, have garnered widespread attention and usage. However, their application in Sichuan pepper picking remains largely unexplored. Therefore, this study proposes a picking soft robotic hand composed of a rigid skeleton and flexible skin for pepper harvesting. Through analyzing the characteristics of peppers, the structure of the robotic hand is determined. Inflatable airbags are employed to drive finger bending, utilizing a rotating–twisting method for Sichuan pepper picking. Structural parameters influencing the bending angle of the airbags were determined through theoretical analysis and validated via simulation. Optimal parameter combinations for the airbags were obtained through response surface experiments to establish the robotic hand model. To assess the feasibility of the robotic hand’s movement, dynamic simulations were conducted using R D (RecurDyn V9R2) software. Subsequently, a discrete element model of Sichuan pepper clusters was established and coupled with the simulation of the Sichuan pepper picking process. The results indicate that the robotic hand does not cause damage to the Sichuan peppers during picking. Finally, field tests were conducted in pepper orchards to validate the success rate of the robotic hand in picking, yielding an 85% success rate and a 0.3% damage rate.
Article
This study presents a novel 3D-printed grabber designed to assess the mechanical properties of pineapples, crucial for optimizing harvesting and handling processes. This research represents a key component of a larger project aimed at developing a semi-automatic pineapple harvester tailored for the challenging terrain of hilly regions, such as those found in Manipur, India, where the pineapple data for this study was collected. Key physical parameters like including transverse diameter, weight, longitudinal diameter, and compression force were evaluated. The grabber was designed using 3D modelling software and fabricated via 3D printing, enabling rapid prototyping and customization. A compression force sensor unit integrated into the grabber allowed for real-time force measurement during testing. Experimental validation involved applying controlled compression forces to pineapples of varying sizes and maturities. The results demonstrated that the grabber could effectively measure compression force while ensuring minimal fruit damage, even when exceeding the required gripping force. The study successfully determined a safe grabbing force range for pineapples, informing the development of robotic or semi-automatic harvesting systems. Furthermore, the adaptable design of the force-sensing grabber presents a promising approach for assessing the mechanical properties of other fruits and vegetables, potentially reducing damage and improving handling practices across the agricultural sector.
Article
We propose a novel multisection cable-driven soft robotic arm inspired by octopus tentacles along with a new modeling approach. Each section of the modular manipulator is made of a soft tubing backbone, a soft silicone arm body, and two rigid endcaps, which connect adjacent sections and decouple the actuation cables of different sections. The soft robotic arm is made with casting after the rigid endcaps are 3D-printed, achieving low-cost and convenient fabrication. To capture the nonlinear effect of cables pushing into the soft silicone arm body, which results from the absence of intermediate rigid cable guides for higher compliance, an analytical static model is developed to describe the relationship between the bending curvature and the cable lengths. The proposed model shows superior prediction performance in experiments over that of a baseline model, especially under large bending conditions. Based on the nonlinear static model, a kinematic model of a multisection arm is further developed and used to derive a motion planning algorithm. Experiments show that the proposed soft arm has high flexibility and a large workspace, and the tracking errors under the algorithm based on the proposed modeling approach are up to 52 %\% smaller than those with the algorithm derived from the baseline model.
Article
Full-text available
Intelligent apple-picking robots can significantly improve the efficiency of apple picking, and the realization of fast and accurate recognition and localization of apples is the prerequisite and foundation for the operation of picking robots. Existing apple recognition and localization methods primarily focus on object detection and semantic segmentation techniques. However, these methods often suffer from localization errors when facing occlusion and overlapping issues. Furthermore, the few instance segmentation methods are also inefficient and heavily dependent on detection results. Therefore, this paper proposes an apple recognition and localization method based on RGB-D and an improved SOLOv2 instance segmentation approach. To improve the efficiency of the instance segmentation network, the EfficientNetV2 is employed as the feature extraction network, known for its high parameter efficiency. To enhance segmentation accuracy when apples are occluded or overlapping, a lightweight spatial attention module is proposed. This module improves the model position sensitivity so that positional features can differentiate between overlapping objects when their semantic features are similar. To accurately determine the apple-picking points, an RGB-D-based apple localization method is introduced. Through comparative experimental analysis, the improved SOLOv2 instance segmentation method has demonstrated remarkable performance. Compared to SOLOv2, the F1 score, mAP, and mIoU on the apple instance segmentation dataset have increased by 2.4, 3.6, and 3.8%, respectively. Additionally, the model’s Params and FLOPs have decreased by 1.94M and 31 GFLOPs, respectively. A total of 60 samples were gathered for the analysis of localization errors. The findings indicate that the proposed method achieves high precision in localization, with errors in the X, Y, and Z axes ranging from 0 to 3.95 mm, 0 to 5.16 mm, and 0 to 1 mm, respectively.
Article
This study reviews the advancement of apple-related technology in six distinct domains: pruning, pollinating, thinning, bagging, harvesting, and sorting. Firstly, the pruning section discusses machine vision systems and moves on to mechanical branch-cutting methods, followed by automated apple pruning. Secondly, concerning pollination, we primarily focus on the growth of apple fruits, beginning with identifying blossoms, using artificial pollen, and mechanized pollination. Thirdly, in the thinning section, a review of achievements of the vision systems and mechanical and robotic thinning technologies. Fourthly, in apple bagging, we address the effects of apple bagging strategies, including the primary procedures, varieties of bags, and influencing aspects. Next, in the harvesting part, the view focuses on the vision strategies and recognition systems that include visible light, spectral, and thermal imaging by means of thorough analysis that considers the time, observations, techniques, and findings. After that, the localization of the apple assists in separating apples such as leaves, branches, and other overlapping apples, in addition to instructing end-effectors on how to grasp and extract apples. Furthermore, harvester robots’ progress includes developments in machinery, grippers, arms, and manipulators, besides the assistance platforms for harvesting. Developing the machinery for sorting apples is the final area of focus. The machinery includes conveyors, sorting executors, and bin fillers. Vision systems imaging types and methods are followed by illumination systems, and identification strategies such as machine and deep learning algorithms are considered. In limitations and future views, we summarize the challenges of the six distinct domains and provide opportunities and future perspectives. By confirming the economic advantages and sustaining the expansion of academic organizations for technology development and industry, the study assists in boosting apple fruit quality and production.
Article
Full-text available
Advancements in smart manufacturing have embraced the adoption of soft robots for improved productivity, flexibility, and automation as well as safety in smart factories. Hence, soft robotics is seeing a significant surge in popularity by garnering considerable attention from researchers and practitioners. Bionic soft robots, which are composed of compliant materials like silicones, offer compelling solutions to manipulating delicate objects, operating in unstructured environments, and facilitating safe human–robot interactions. However, despite their numerous advantages, there are some fundamental challenges to overcome, which particularly concern motion precision and stiffness compliance in performing physical tasks that involve external forces. In this regard, enhancing the operation performance of soft robots necessitates intricate, complex structural designs, compliant multifunctional materials, and proper manufacturing methods. The objective of this literature review is to chronicle a comprehensive overview of soft robot design, manufacturing, and operation challenges in conjunction with recent advancements and future research directions for addressing these technical challenges.
Article
Full-text available
Accurate depth-sensing is crucial in securing a high success rate of robotic harvesting in natural orchard environments. The solid-state LiDAR technique, a recently introduced LiDAR sensor, can perceive high-resolution geometric information of the scenes, which can be utilised to receive accurate depth information. Meanwhile, the fusion of the sensory data from LiDAR and the camera can significantly enhance the sensing ability of the harvesting robots. This work first introduces a LiDAR-camera fusion-based visual sensing and perception strategy to perform accurate fruit localisation in the apple orchards. Two SOTA LiDAR-camera extrinsic calibration methods are evaluated to obtain the accurate extrinsic matrix between the LiDAR and camera. After that, the point clouds and colour images are fused to perform fruit localisation using a one-stage instance segmentation network. In addition, comprehensive experiments show that LiDAR-camera achieves better visual sensing performance in natural environments. Meanwhile, introducing the LiDAR-camera fusion can largely improve the accuracy and robustness of the fruit localisation. Specifically, the standard deviations of fruit localisation using LiDAR-camera at 0.5, 1.2, and 1.8 m are 0.253, 0.230, and 0.285 cm, respectively, during the afternoon with intensive sunlight. This measurement error is much smaller compared with that from Realsense D455. Lastly, visualised point cloud² of the apple trees have been attached to demonstrate the highly accurate sensing results of the proposed Lidar-camera fusion method.
Article
Full-text available
Nowadays, harvesting delicate and high-value fruits, vegetables, and edible fungi requires a large input of manual human labor. The relatively low wages and many health problems the workforce faces make this profession increasingly unpopular. Meanwhile, robotic systems that selectively harvest crops are being developed. Whilst the moving platform, manipulator, and image recognition systems of such robots have been studied the past few decades, research on the gripping end of such robots is only since recently growing. This study analyzes the state-of-the-art of soft grippers for crop handling and harvesting, reporting on their quantitative and qualitative characteristics. Seventy-eight grippers are retrieved from the academic literature and compared with each other in terms of their design and reported performance, more specifically grasping and detachment methods, materials used, type of actuators and sensors employed, and the control of the gripping procedure. In addition, the identified grippers are classified into 13 distinct soft grasping technology categories. Moreover, the retrieved papers are analyzed with respect to their publication date and country of origin to observe trends in the recent growth in the field. Furthermore, a subset of soft grippers is identified that was tested on the task of selectively harvesting crops, where grip and detachment success rates and plant and crop damage are compared.
Article
Full-text available
In order to enable the picking robot to detect and locate apples quickly and accurately in the orchard natural environment, we propose an apple object detection method based on Shufflenetv2-YOLOX. This method takes YOLOX-Tiny as the baseline and uses the lightweight network Shufflenetv2 added with the convolutional block attention module (CBAM) as the backbone. An adaptive spatial feature fusion (ASFF) module is added to the PANet network to improve the detection accuracy, and only two extraction layers are used to simplify the network structure. The average precision (AP), precision, recall, and F1 of the trained network under the verification set are 96.76%, 95.62%, 93.75%, and 0.95, respectively, and the detection speed reaches 65 frames per second (FPS). The test results show that the AP value of Shufflenetv2-YOLOX is increased by 6.24% compared with YOLOX-Tiny, and the detection speed is increased by 18%. At the same time, it has a better detection effect and speed than the advanced lightweight networks YOLOv5-s, Efficientdet-d0, YOLOv4-Tiny, and Mobilenet-YOLOv4-Lite. Meanwhile, the half-precision floating-point (FP16) accuracy model on the embedded device Jetson Nano with TensorRT acceleration can reach 26.3 FPS. This method can provide an effective solution for the vision system of the apple picking robot.
Article
Full-text available
Intelligent robots for fruit harvesting have been actively developed over the past decades to bridge the increasing gap between feeding a rapidly growing population and limited labour resources. Despite significant advancements in this field, widespread use of harvesting robots in orchards is yet to be seen. To identify the challenges and formulate future research and development directions, this work reviews the state-of-the-art of intelligent fruit harvesting robots by comparing their system architectures, visual perception approaches, fruit detachment methods and system performances. The potential reasons behind the inadequate performance of existing harvesting robots are analysed and a novel map of challenges and potential research directions is created, considering both environmental factors and user requirements.
Article
Full-text available
Field robotic harvesting is a promising technique in recent development of agricultural industry. It is vital for robots to recognise and localise fruits before the harvesting in natural orchards. However, the workspace of harvesting robots in orchards is complex: many fruits are occluded by branches and leaves. It is important to estimate a proper grasping pose for each fruit before performing the manipulation. In this study, a geometry-aware network, A3N, is proposed to perform end-to-end instance segmentation and grasping estimation using both color and geometry sensory data from a RGB-D camera. Besides, workspace geometry modelling is applied to assist the robotic manipulation. Moreover, we implement a global-to-local scanning strategy, which enables robots to accurately recognise and retrieve fruits in field environments with two consumer-level RGB-D cameras. We also evaluate the accuracy and robustness of proposed network comprehensively in experiments. The experimental results show that A3N achieves 0.873 on instance segmentation accuracy, with an average computation time of 35 ms. The average accuracy of grasping estimation is 0.61 cm and 4.8° in centre and orientation, respectively. Overall, the robotic system that utilizes the global-to-local scanning and A3N, achieves success rate of harvesting ranging from 70% to 85% in field harvesting experiments.
Article
Full-text available
The emerging field of soft robotics presents a new paradigm for robot design in which “precision through rigidity” is replaced by “cognition through compliance.” Lightweight and flexible, soft robots have vast potential to interact with fragile objects and navigate unstructured environments. Like octopuses and worms in nature, soft robots’ flexible bodies conform to hard objects and reconfigure for different tasks, delegating the burden of control from brain to body through embodied cognition. However, because of the lack of efficient modeling and simulation tools, soft robots are primarily designed by hand. Typically, hard components from rigid robots or living creatures are heuristically substituted for comparable soft ones. Autonomous design and manufacturing methodologies are urgently required to produce bespoke, high‐performing robots. Currently, design methodologies exist between simple but realistic parametric optimizations, and evolutionary algorithms which simulate morphology and control coevolution. To find high‐performing designs, novel high‐fidelity simulators and high‐throughput manufacturing and testing processes are required to explore the complex soft material, morphology and control landscape, blending simulation, and experimental data. This article reviews the state of the art in autonomous soft robotic design. Existing manual and automated designs are surveyed and future directions to automate soft robot design and manufacturing are presented. By using soft and functional materials to deform around objects and adapt to new environments, soft robotics has the potential to revolutionize material handling and terrain navigation. But in the absence of accurate modeling tools, they are still laboriously designed manually. This article reviews progress toward autonomous modeling, simulation, and design.
Article
Full-text available
Soft bending actuators, as one of the most important components of soft robotics, have attracted significantly increasing attention due to their robustness, compliance, inherent safety, and ease of manufacture. However, the key disadvantages can be the low output force, slow response speed, large deformation and vibration, which can potentially be addressed by introducing a bistable mechanism enabled by a prestressed steel shell. This work proposes a novel soft actuator with bistable property, which can maintain the predefined initial state and enhance bending motion at the corresponding stable state. A novel dual-actuation mechanism, which utilises pneumatic pressure for closing and tendon-driven for opening process, is proposed for autonomous transition between both states, and for a fast response. Mathematical model is proposed and compared with the experimental result for triggering pressure, which serves as a threshold to activate the transition of the stable state. Experimental results also indicate that closing and opening speeds are enhanced by 9.82% and more than ten times, respectively, as compared with the existing pneumatic bistable reinforced actuator design. Mathematical and experimental results suggest that a programmable bending angle at the second stable state can also be achieved by adjusting the preset tendon extension. The tendon arrangement also acts as a passive damping mechanism to reduce the oscillation while closing. The damping ratio is increased by more than four times, indicating that the oscillation decay is significantly accelerated for quick stabilization. Finally, a three-finger soft gripper is developed based on the proposed actuator design, which demonstrates promising performance in grasping objects with various shapes and sizes. The experimental results also show that the proposed bistable gripper can grasp the object with a weight up to 2067 g, which is more than 17 times heavier than that of three actuators.
Article
Full-text available
Agriculture 4.0 is transforming farming livelihoods thanks to the development and adoption of technologies such as artificial intelligence, the Internet of Things and robotics, traditionally used in other productive sectors. Soft robotics and soft grippers in particular are promising approaches to lead to new solutions in this field due to the need to meet hygiene and manipulation requirements in unstructured environments and in operation with delicate products. This review aims to provide an in-depth look at soft end-effectors for agricultural applications, with a special emphasis on robotic harvesting. To that end, the current state of automatic picking tasks for several crops is analysed, identifying which of them lack automatic solutions, and which methods are commonly used based on the botanical characteristics of the fruits. The latest advances in the design and implementation of soft grippers are also presented and discussed, studying the properties of their materials, their manufacturing processes, the gripping technologies and the proposed control methods. Finally, the challenges that have to be overcome to boost its definitive implementation in the real world are highlighted. Therefore, this review intends to serve as a guide for those researchers working in the field of soft robotics for Agriculture 4.0, and more specifically, in the design of soft grippers for fruit harvesting robots.
Article
Full-text available
CONTEXT The COVID-19 pandemic, caused by the SARS-CoV-2 virus, has affected global agricultural chains and intensified the issue of food insecurity worldwide. OBJECTIVE We propose a short retrospective of the reported effects of the COVID-19 pandemic from its beginning in March 2020 in Europe, on orchard management and harvest, sales and agricultural chains, monitoring and research in orchards, and we expose some of the solutions undertaken to tackle down these issues. RESULTS AND CONCLUSION In Europe, the fruit and vegetable sector has been affected by the pandemic in terms of production, distribution, and disturbance in market performance. Concerning apple, the most harvested and exported fruit in Europe, national governments, European institutions, the industry and producers have undertaken actions to ensure production and supply demand. Yet, stakeholders have faced several difficulties and additional costs for growth and harvest, sales, but also monitoring and research. However, European Union demand for fresh apples has increased during the pandemic. In addition, apple harvest has started a couple of months after the end of the first lockdown in most countries, and European apple orchards are usually in more flexible smallholder or family farms. Finally, the fruit itself has relatively long shelf-life comparatively to other fruits. For these main reasons, we argue that the apple sector might be more resilient than other fruit sectors or other cultures, despite the negative effects of seasonal workforce shortage and unstable market. The apple sector may suffer more from side-effects such as increasing labor, distribution, and packaging costs, than from actual stock and production issues. SIGNIFICANCE The pandemic could be an opportunity to reconsider production modes and to innovate for the future of food production in different crop systems in Europe, including apple orchards.
Article
Full-text available
Many soft robots are composed of soft fluidic actuators that are fabricated from silicone rubbers and use hydraulic or pneumatic actuation. The strong nonlinearities and complex geometries of soft actuators hinder the development of analytical models to describe their motion. Finite element modeling provides an effective solution to this issue and allows the user to predict performance and optimize soft actuator designs. Herein, the literature on a finite element analysis of soft actuators is reviewed. First, the required nonlinear elasticity concepts are introduced with a focus on the relevant models for soft robotics. In particular, the procedure for determining material constants for the hyperelastic models from material testing and curve fitting is explored. Then, a comprehensive review of constitutive model parameters for the most widely used silicone rubbers in the literature is provided. An overview of the procedure is provided for three commercially available software packages (Abaqus, Ansys, and COMSOL). The combination of modeling procedures, material properties, and design guidelines presented in this article can be used as a starting point for soft robotic actuator design.
Article
Full-text available
Robotic harvesting shows a promising aspect in future development of agricultural industry. However, there are many challenges which are still presented in the development of a fully functional robotic harvesting system. Vision is one of the most important keys among these challenges. Traditional vision methods always suffer from defects in accuracy, robustness, and efficiency in real implementation environments. In this work, a fully deep learning-based vision method for autonomous apple harvesting is developed and evaluated. The developed method includes a light-weight one-stage detection and segmentation network for fruit recognition and a PointNet to process the point clouds and estimate a proper approach pose for each fruit before grasping. Fruit recognition network takes raw inputs from RGB-D camera and performs fruit detection and instance segmentation on RGB images. The PointNet grasping network combines depth information and results from the fruit recognition as input and outputs the approach pose of each fruit for robotic arm execution. The developed vision method is evaluated on RGB-D image data which are collected from both laboratory and orchard environments. Robotic harvesting experiments in both indoor and outdoor conditions are also included to validate the performance of the developed harvesting system. Experimental results show that the developed vision method can perform highly efficient and accurate to guide robotic harvesting. Overall, the developed robotic harvesting system achieves 0.8 on harvesting success rate and cycle time is 6.5 s.
Article
Full-text available
This article reviews the research and development progress of mechanical harvest technologies for fresh market apples over the past decades with a focus on the predominant technologies of shake-and-catch, robots, and harvest-assist platform methods. In addition, based on the review it points out the bottlenecks and future trends of these three technology categories. Major progress in the shake-and-catch method is related to theoretical studies on the effective removal of apples and catching mechanisms to minimize bruising. The unacceptable bruising conditions hinder the shake-and-catch method from commercial application. Two startups of apple harvesting robots are in the stage of commercializing their products based on vacuum and three-finger end-effectors, respectively. Economic benefits, as well as technology reliability and robustness of both robots, are pending for validation before they are on the market. In addition, a key obstacle faced by both robots before commercial use is to find a solution to pick apples grown in clusters. Harvest-assist platforms are gradually adopted by apple growers, but at a very low rate due to their doubts on economic benefits. Validation of harvest-assist platforms’ economic benefits and incorporation with more functions (e.g., sorting) would enhance their adoption. With the rapid development of sensing and automation technologies, such as novel sensors, embedded systems, and machine learning algorithms, and the progress in new tree canopy structures that are friendlier for fruit visibility and accessibility, it is believed the robots for fresh market apple harvest would be realized and commercialized in the near future. Currently, more efforts should be invested in analyzing and validating the economic benefits of harvest-assist platforms, as well as adding more functions to the harvest-assist platforms, to increase their application rate for the benefit of the apple industry.
Article
Full-text available
Soft actuators, as an important part of soft robotics, have attracted significant attention due to their inherent compliance, flexibility and safety. However, low capacity in force and load limits their applications. Prestored elastic energy can improve the capacity in output force and load of soft actuators. This work introduces a soft pneumatic bistable reinforced actuator inspired by the Venus Flytrap’s bistable mechanism that allows for the storage of elastic energy. The proposed actuator consists of two separated 3D-printed actuation chambers that are attached to a central prestressed steel shell to achieve bistability. The pressure triggering the pneumatic bistable reinforced actuator from one stable state to the other is derived and validated by experiments. Further experimental comparisons between the proposed actuator and a dual chamber actuator show that the proposed design significantly improves the block tip force, load capacity and stiffness. The pneumatic bistable reinforced actuator also demonstrates superior performance in the actuation speed and bending angle under the same input pressure. Finally, a two-finger gripper is developed using the proposed actuator, which is demonstrated to grasp and hold various objects.
Article
Full-text available
The utilization of machine vision and its associated algorithms improves the efficiency, functionality, intelligence, and remote interactivity of harvesting robots in complex agricultural environments. Machine vision and its associated emerging technology promise huge potential in advanced agricultural applications. However, machine vision and its precise positioning still have many technical difficulties, making it difficult for most harvesting robots to achieve true commercial applications. This article reports the application and research progress of harvesting robots and vision technology in fruit picking. The potential applications of vision and quantitative methods of localization, target recognition, 3D reconstruction, and fault tolerance of complex agricultural environment are focused, and fault-tolerant technology designed for utilization with machine vision and robotic systems are also explored. The two main methods used in fruit recognition and localization are reviewed, including digital image processing technology and deep learning-based algorithms. The future challenges brought about by recognition and localization success rates are identified: target recognition in the presence of illumination changes and occlusion environments; target tracking in dynamic interference-laden environments, 3D target reconstruction, and fault tolerance of the vision system for agricultural robots. In the end, several open research problems specific to recognition and localization applications for fruit harvesting robots are mentioned, and the latest development and future development trends of machine vision are described.
Article
Full-text available
Detachment of fruit from the plants is a basic and essential step either in manual or in robotic harvesting. Usually there are two methods involved in fruit detachment one is fruit cutting and the other is fruit plucking. Compared with fruit cutting, fruit plucking is more simple, low cost, and flexible in robotic harvesting. Different plucking patterns have been adopted in various prototypes for tomato, cherry tomato, apple and strawberry fruit harvesting. Although, it was found that there were differences in the success rate, efficiency and damage of different plucking patterns, moreover in-depth analysis still not carried out. In this paper, combining the classical load-deformation and fatigue break theory with the flexibility of the peduncle and pedicel are presented, theoretical modeling and discussion of different plucking based experimental results were performed. It was found from force transmission, that the load applied on the abscission layer was different from the motive power. Large deformation may happen in the whole peduncle and pedicel and lead to failure, inefficiency or even collision with obstacles in the canopy. Complex combined loading lead to the final break of the abscission layer. For reciprocating twisting pattern, long tomato peduncle and pedicel may take excessive swing times to realize the fatigue break of the abscission layer. In addition, the potential damage to fruit or the whole cluster can be well explained by this theory. The performance of different fruit plucking patterns may be related to the different species and cultivation modes, this research provides a good analytical basis for revealing the complex mechanisms of robotic fruit plucking and making a practical decision.
Article
Full-text available
Robotic harvesting offers a solution to reducing labor costs, optimizing harvest scheduling, enabling selective harvesting, and increasing operation efficiency. These attributes allow the users of robotic harvesters to maximize production efficiency and profits. This article reviews automated fruit harvesting systems of sweet pepper, tomato, apple and kiwifruit as an example to demonstrate the recent advances in intelligent automatic harvesting robots in horticulture.
Article
Full-text available
Agriculture provides an unique opportunity for the development of robotic systems; robots must be developed which can operate in harsh conditions and in highly uncertain and unknown environments. One particular challenge is performing manipulation for autonomous robotic harvesting. This paper describes recent and current work to automate the harvesting of iceberg lettuce. Unlike many other produce, iceberg is challenging to harvest as the crop is easily damaged by handling and is very hard to detect visually. A platform called Vegebot has been developed to enable the iterative development and field testing of the solution, which comprises of a vision system, custom end effector and software. To address the harvesting challenges posed by iceberg lettuce a bespoke vision and learning system has been developed which uses two integrated convolutional neural networks to achieve classification and localization. A custom end effector has been developed to allow damage free harvesting. To allow this end effector to achieve repeatable and consistent harvesting, a control method using force feedback allows detection of the ground. The system has been tested in the field, with experimental evidence gained which demonstrates the success of the vision system to localize and classify the lettuce, and the full integrated system to harvest lettuce. This study demonstrates how existing state‐of‐the art vision approaches can be applied to agricultural robotics, and mechanical systems can be developed which leverage the environmental constraints imposed in such environments.
Article
Full-text available
In this study, three different soft pneumatic actuators (SPA) are designed and directly fabricated through additive manufacturing using thermoplastic polyurethane (TPU) filaments. The equal total inner volume size is used in the three varied designs to compare their effect on the bending response. A material model is selected and implemented according to the uniaxial tensile test parameters. The experimental results obtained from three different soft pneumatic actuators are compared with numerical model results. Especially, the experimentally measured bending forces are compared with the numerical model counterparts. The highest continuous bending deformation is determined among the three different soft pneumatic actuators. Additionally, a new integrated design and manufacturing approach is presented aiming to maximize the potential bending capability of the actuator through additive manufacturing.
Article
Full-text available
Conventionally, engineers have employed rigid materials to fabricate precise, predictable robotic systems, which are easily modelled as rigid members connected at discrete joints. Natural systems, however, often match or exceed the performance of robotic systems with deformable bodies. Cephalopods, for example, achieve amazing feats of manipulation and locomotion without a skeleton; even vertebrates such as humans achieve dynamic gaits by storing elastic energy in their compliant bones and soft tissues. Inspired by nature, engineers have begun to explore the design and control of soft-bodied robots composed of compliant materials. This Review discusses recent developments in the emerging field of soft robotics.
Article
Full-text available
Harvesting is a crucial component of sugarcane production system. In this paper we conducted a critical review of harvesting technologies currently being used in commercial sugarcane production. It presented a brief introduction of two harvesting practices (burnt cane harvesting and green cane harvesting) and two harvesting modes (whole stalk harvesting and chopper harvesting) including a brief discussion on their advantages, limitations, and technical challenges. It is followed by a comprehensive review of two core technologies of current sugarcane harvesters: cane base cutting and feeding mechanisms. The influence of base cutter kinematic and geometric parameters, such as blade cutting velocity, disc tilt angle, blade number, blade oblique angle, and blade shape on harvesting efficiency and cutting quality, were reviewed. Relevant inventions and technologies used to adjust feeding rate, arrange stalks into uniform orientation, and maintain uniform billet lengths were reviewed and summarized. In addition, anti-plugging apparatus used to improve feeding efficiency were discussed. Finally, potential new technologies were recommended for improving the performance and adaptability of sugarcane harvesters.
Article
Full-text available
Neo-Hookean model and Mooney-Rivlin model are hyperelastic material models where the strain energy density function is made from invariants of the left Cauchy-Green deformation tensor. Even though Ogden model is a hyperelastic material model, its strain energy density function is expressed by principal stretch ratio. These three models have been widely used in industries. Recently, Ogden model, especially Ogden 3rd model, shows better agreement with the test data than others. In spite of the limitations to describe particular stress states, it is known that reasonable results using these models can be obtained for various structural components. In this research, three kinds of models are considered for Chloroprene rubber. Three kinds of tests (Uniaxial tension test, Biaxial tension test, and Planar shear test) are performed for Chloroprene specimen and through four kinds of test combinations (Uni+Bi, Uni+Pl, Bi+Pl, Uni+Bi+Pl), numerical simulations are carried out for Neo-Hookean model, Mooney-Rivlin model, and Ogden model. It is shown that Mooney-Rivlin model and Ogden model can be used for Chloroprene Rubber in the specific ranges for Isotropic Hyperelastic model.
Article
Fruit harvesting is time-consuming and laborious. Robots can harvest fruits with a high degree of automation, greatly reducing labor requirements, but are limited by their efficiency and high costs. In this research, a prototype apple harvesting robot was designed and constructed. The robotic prototype integrated a binocular camera, a 4-degree-of-freedom manipulator, a vacuum-based end-effector, and a mobile vehicle. The robot detected, positioned, gripped, detached, and placed apples. A manipulator controller was designed to realize rapid control execution. Picking experiments were conducted in a spindle apple orchard. Picking was tested using rotation-pull and pull patterns. The test results showed that the rotation-pull pattern was more effective picking apples than the pull pattern. The picking success rate of the rotation-pull pattern was 47.37% in the field orchard and 78% in the simulated orchard environment, with picking cycle time of ∼ 4 s. The stem damage rate in the field orchard was 11.11%. The developed picking prototype realized the task of picking apples. Its primary advantage was its competitive picking cycle time, which provides a solid foundation for the future improvement of harvesting efficiency.
Article
A Two-step Division-Merge (TDM) based inverse kinematics (IK) mechanism is proposed for robots with multiple degrees of freedom (multi-DOFs) in unstructured environments. For any robot with n-DOFs (n>6), a division point is selected first to divide it into a lower part subsystem from its base to the division point, together with an upper part one from the point to its end-effector. Once a collision-free configuration is found for the upper part subsystem to match with the target, the lower part subsystem is then planned to merge with the upper part subsystem at the division point. Specifically, the division point is selected such that the lower part subsystem has 6-DOFs, while the upper part subsystem has (n-6)-DOFs. Furthermore, a Reverse & Random Rotation (RRR) algorithm is also proposed to solve the IK of the upper part subsystem in unstructured environments, while the IK of the lower part subsystem is solved analytically. The proposed TDM based mechanism is verified via experiments using a lab-customized humanoid apple harvesting robot with 11- and 13-DOFs, respectively. Under the various scenarios with different target and obstacle configurations, errors in position coordinates and orientation angles between the target pose and those of the end-effector obtained with forward kinematics (FK) are calculated. Results show that the errors in position and orientation are less than 0.005 mm and 0.007°, respectively, and the average computational time is less than 2 s for the IK solution of the entire robot. Such results indicate that the proposed mechanism is reliable and capable of reducing the complexity of IK for multi-DOFs humanoid robots. With the proposed algorithm, the apple harvesting success rate of a lab-customized 11-DOFs robot reaches 89.2%, which can be further increased up to 93.3% with error compensation, while the average time starting from the robot action to the end-effector reaching the target pose of the apple is around 12.3 s.
Article
In order to realize the compliant grasping of apple harvesting robot and reduce the mechanical damage in the process of grasping, a variable damping impedance control strategy for gripper grasping force tracking is proposed based on the establishment of contact force model in this paper. Firstly, the Burgers model is used to characterize the biological characteristics of apple. Then, according to Hertz contact theory, the contact force model of apple during grasping was established to represent the dynamic relationship between grasping force and deformation. Finally, based on the position impedance control theory and the analysis of the influence of impedance parameters on the control performance, a variable damping impedance controller is designed. The dynamic tracking of the grasping force is realized by adjusting the damping coefficient online to modify the desired position. The simulation and experimental results show that compared with the traditional impedance control, the variable damping impedance control has a smoother tracking effect on the grasping force and its dynamic performance is significantly improved. This method can provide reference for the research of compliant grasping control of apple harvesting robot end-effector.
Article
This paper presents the development and evaluation of a pneumatic finger-like end-effector for cherry tomato harvesting robot. The end-effector is pneumatically controlled and has the capability of picking cherry tomatoes continuously and steadily. The end-effector is compact in overall structure, which consists of finger-like clamping finger, rotating and telescopic cylinders, and RGB-D camera, etc. Another important feature is that the clamping finger uses a combination of clamping and rotating and has good adaptability in dense operating environments. Further, a hand-picking dynamic measurement system is developed to measure and analyze the applied force and disturbance for simplified picking methods during picking. The hand-picking test shows that both the applied force and the disturbance are smaller for rotating compared to pulling, so the method of rotating is chosen for the design of the end-effector in this study. The field test shows the average cycle time of picking single cherry tomato is 6.4 s. The harvest success rates for pickable cherry tomatoes in different directions are 84% (right), 83.3% (back), 79.8% (left), and 69.4% (front), respectively. The failure cases are analyzed and collision and localization failure are the main causes of unsuccessful picking during picking.
Article
Gripping control of the manipulator is still a technical problem in its research field. Accurate, efficient, and lossless crawling remains a challenge due to the difficulty of controlling grip mode, joint movement, and optimal implementation. Human hands can accurately perceive objects and grasp flexibly, so understanding and simulating the gripping behavior of human hands is crucial for controlling manipulators. In this study, an electronic glove was designed and assembled based on manual grasping and mechanical grip perception, combined with force sensors, bending sensors, and inertial sensor (Inertial Measurement Unit, IMU). The mechanism and characteristics of handgrip are analyzed from the four aspects, including force and bending relationship, hand grasping patterns, finger cooperation and correlation, and the law of external interaction acceleration. The pressure generally increases with the increase of bending, so the force can be controlled by controlling the degree of bending. Moreover, mode A shows a more complex grip mechanism compared with mode B. The pose position and orientation of the manipulator can be controlled by analyzing the gripping characteristics between the various modes. Besides, the correlation between the index finger, middle finger, and ring finger is relatively high. By understanding the correlation law between the fingers can control the cooperative movement of the manipulator when grasping. And the angular velocity of the X and Y axes changes significantly during the grasping process. The direction of movement and position of the manipulator can be determined by the changing state of the angular velocity. Thus, the analysis of these aspects could provide theoretical guidance for the structural design, surface material selection, and grasping planning of bionic manipulators.
Article
Operational efficiency is an important bottleneck in the industrialization of apple picking robots. The action planning of grasping and picking is key to improving the operational efficiency of picking robots. In this study, first, from the perspective of grip stability, the grasping positions of a three-finger end-effector were planned in two cases to determine an optimal three-finger grip posture: one that is parallel to the apple stem–calyx axis and the other being centripetal to the apple equatorial plane. Second, based on the gripping posture parallel to the apple stem-calyx axis, four picking patterns were designed: vertical pull, horizontal pull, vertical rotation, and rotation–horizontal pull. With the horizontal displacement, vertical displacement, and rotation angle as the main factors influencing the picking process, it was found through ABAQUS simulation software and the response surface analysis method that the rotation–horizontal pull pattern with the combination of horizontal displacement and rotation angle yields the minimum separation force for apple piking. With the three-fingered hand in the optimal grasping posture parallel to the stem–calyx, the four picking patterns were tested on a dedicated apple picking experimental device equipped with a grip force acquisition sensor, and the test results showed that the average three-finger grip force required for rotary tug-pull picking was the lowest (10.33 N). The proposed rotation–horizontal pull pattern with the three-finger grip posture was proven to be the best pattern for apple picking. The results of this study provide a theoretical basis for further research on apple picking robots.
Article
In order to reduce the mechanical damage caused by apple harvesting robot to apples and realize the non-destructive grasping during the grasping process, an adaptive impedance control strategy for apple harvesting robot compliant grasping is proposed based on the analysis of grasping modes in this paper. Firstly, burgers viscoelastic model is established to describe the rheological characteristics of apples. Then taking the plastic deformation of apples as the damage standard, three types of grasping modes of the gripper for picking apples are compared and analyzed. Finally, in view of the fact that the traditional impedance controller cannot satisfy the grasping force control under the condition of uncertain environmental stiffness and uncertain environmental position, an adaptive controller is designed to adjust the desired position online based on the error between desired force and actual grasping force so that it can track the desired force. Simulation and experiment results show that the adaptive impedance control strategy is more compliant with the actual grasping force, the overshoot and the adjustment time has been significantly improved, which provides a reference for actual apple picking.
Article
Robotics has undergone a profound revolution in the past 50 years, moving from the laboratory and research institute to the factory and home. Kinematics and dynamics theories have been developed as the foundation for robot design and control, based on the conventional definition of robots: a kinematic chain of rigid links.
Article
A methodology for planning the sequence of tasks for a harvesting robot is presented. The fruit targets are situated at unknown locations and must be detected by the robot through a sequence of sensing tasks. Once the targets are detected, the robot must execute a harvest action at each target location. The traveling salesman paradigm (TSP) is used to plan the sequence of sensing and harvesting tasks taking into account the costs of the sensing and harvesting actions and the traveling times. Sensing is planned online. The methodology is validated and evaluated in both laboratory and greenhouse conditions for a case study of a sweet pepper harvesting robot. The results indicate that planning the sequence of tasks for a sweet pepper harvesting robot results in 12% cost reduction. Incorporating the sensing operation in the planning sequence for fruit harvesting is a new approach in fruit harvesting robots and is important for cycle time reduction. Furthermore, the sequence is re-planned as sensory information becomes available and the costs of these new sensing operations are also considering in the planning.
Article
Purpose The purpose of this paper is to describe recent fruit picking robot developments with an emphasis on corporate activity rather than academic research. It also aims to provide a view on the commercial prospects for these developments. Design/methodology/approach Following a short introduction, this first discusses strawberry and other soft fruit picking robot developments conducted principally by commercial organisations. It then provides similar details of robots for harvesting apples and other hard fruits. This is followed by a discussion and concluding comments. Findings The shortage of seasonal fruit pickers has stimulated the need for automation. Accordingly, a growing community of companies, many founded in the past five years, are developing fruit picking robots. These are aimed at both soft and hard fruits, such as strawberries and apples, respectively, and exploit advanced vision systems, image processing techniques and AI. Some products are already on the market, whereas many more are due for commercial release during the next two years into what is expected to be a highly competitive market. Originality/value This provides details of the emerging fruit picking robot business by describing the products and manufacturing companies.
Article
As labour requirements in horticultural become more challenging, automated solutions are becoming an effective approach to maintain productivity and quality. This paper presents the design and performance evaluation of a novel multi-arm kiwifruit harvesting robot designed to operate autonomously in pergola style orchards. The harvester consists of four robotic arms that have been designed specifically for kiwifruit harvesting, each with a novel end-effector developed to enable safe harvesting of the kiwifruit. The vision system leverages recent advances in deep neural networks and stereo matching for reliably detecting and locating kiwifruit in real-world lighting conditions. Furthermore, a novel dynamic fruit scheduling system is presented that has been developed to coordinate the four arms throughout the harvesting process. The performance of the harvester has been measured through a comprehensive and realistic field-trial in a commercial orchard environment. The results show that the presented harvester is capable of successfully harvesting 51.0% of the total number of kiwifruit within the orchard with an average cycle time of 5.5s/fruit.
Article
Fresh market apple harvesting is a difficult task that relies entirely on manual labor. Much research has been done on the development of mechanical harvesting techniques. Several selective harvesting robots have been developed for research studies, but there are no commercially available robotic systems. This article discusses the design and development of a novel pneumatic 3D-printed soft-robotic end-effector to facilitate apple separation. The end-effector was integrated into a robotic system with five degrees of freedom that was designed to simplify the picking sequence and reduce costs compared to previous versions. Apples were successfully harvested using the low-cost robotic system in a commercial orchard during the fall 2017 harvest. A detachment success rate on attempted apples of 67% was achieved, with an average time of 7.3 s per fruit from separation to storage bin. By conducting this study in an orchard where problematic apples were not removed to increase the detachment success rate, current pruning and thinning practices were assessed to help lay the foundation for future studies and develop strategies for successfully harvesting apples that are difficult to detach. Keywords: Apple catching, Apples, Automated harvesting, Field experimentation, Harvesting robot, Soft-robotic gripper.
Article
Fruit and vegetable picking grippers are important technology to achieve rapid and labor-saving harvest. However, most of the existing fruit and vegetable picking grippers still use traditional rigid or underactuated grippers, which often cause fruits and vegetables damage by the heavy mass and lack of high-precision control, and have poor compliance in the operation process. In recent years, inspired by soft creature's tentacles, soft robotic grippers have appeared and been used in robotics due to the emergence of soft robots. Soft robotic gripper, which is made of flexible material, is a new type of gripper for general purposes with grasping and holding capabilities enabled by a simple control scheme. Under the infinite degrees of freedom, the soft robotic gripper can change its shape and size corresponding to the load in a large range. These advantages overcome the defects of traditional fruit and vegetable picking robots, such as rigidity and poor adaptability. Moreover, soft robotic gripper is easy to manufacture and can be integrated with manual operation without any large-scale safe requirement. This paper gives a research on limitations of the traditional grippers and summarizes the characteristics of ideal picking grippers of fruit and vegetable. In addition, the concept of soft robotic grippers is introduced in detail, and the current development status about soft robotic grippers is described. The manuscript also summarizes the progress and superiority of soft robotic grippers in fruit and vegetable picking as well as strong adaptability to the environment. The characteristics of the soft robotic grippers driven by the common driving mode including pneumatic, cable, shape memory alloy and electroactive polymer mode have been analyzed in the process of fruit and vegetable grasping and picking. Compared with the rigid grippers, in addition to the simple control and mechanism, the soft robotic grippers have high-degree flexibility, adaptability and versatility. Based on the related work, the problems of modeling and control of fruit and vegetable picking devices have been discussed, and possible solutions of soft robotic grippers are also summarized by means of analysis and classification. We can choose the appropriate control methods according to the surface characteristics of picked fruits and vegetables. Finally, we conclude that multi-sensing, variable stiffness, multi-functional composite materials, as well as control strategies of fusion intelligence will be future development directions of fruit and vegetable picking grippers with the progress of micro-sensors and biomaterials. This research will provide theoretical and technical guidance for the development of fruit and vegetable picking grippers. It is expected that more fruit and vegetable picking operations could be carried out by soft robotic grippers, and the application will effectively reduce the damage rate of fruit and vegetable picking. As a new-generation operating device, the soft robotic grippers involve the development of materials, chemistry, machinery and other multidiscipline. Further studies are required to improve its design, controllability, manipulating methods, and so on. © 2018, Editorial Department of the Transactions of the Chinese Society of Agricultural Engineering. All right reserved.
Article
A robotic device consisting of a manipulator, end-effector and image-based vision servo control system was developed for harvesting apple. The manipulator with 5 DOF PRRRP structure was geometrically optimised to provide quasi-linear behaviour and to simplify the control strategy. The spoon-shaped end-effector with the pneumatic actuated gripper was designed to satisfy the requirements for harvesting apple. The harvesting robot autonomously performed its harvesting task using a vision-based module. By using a support vector machine with radial basis function, the fruit recognition algorithm was developed to detect and locate the apple in the trees automatically. The control system, including industrial computer and AC servo driver, conducted the manipulator and the end-effector as it approached and picked the apples. The effectiveness of the prototype robot device was confirmed by laboratory tests and field experiments in an open field. The success rate of apple harvesting was 77%, and the average harvesting time was approximately 15 s per apple. Crown Copyright
Hyperelastic Mooney-Rivlin model: determination and physical interpretation of material constants
  • Kumar
Kumar, N., Rao, V.V., 2016. Hyperelastic Mooney-Rivlin model: determination and physical interpretation of material constants. Parameters 2 (10), 01.
Labor issues and COVID-19
  • Larue
Larue, B., 2020. Labor issues and COVID-19. Can. J. Agric. Econ./Revue Canad. D'Agroeconomie 68 (2), 231-237.