System-level description and evaluation of a robot-aided strawberry harvesting system Bio Automation Lab

Abstract

Manual harvesting of fresh-market crops, like strawberries, table grapes, cherry tomatoes, and berry fruits, is costly and labor-intensive. These crops are planted in parallel rows with furrows/aisles between them. Pickers walk inside rows or furrows as they harvest selectively, based primarily on ripeness criteria. Each picker places the harvested crop in a small container (e.g., tray, bag, or wheelbarrow). When the container becomes full, the picker walks and transports it to a collection station at the field’s headland and takes an empty container to resume picking. Selective harvesting requires advanced perception, to detect and select harvestable fruits; significant dexterity and skill, to harvest them without damaging them, and speed, to work fast, in a cost-effective manner. Because of the demanding task constraints, mechanical or robotic harvesters have not successfully replaced pickers yet, in real-world production.

Walking time is non-productive and takes up to 20% of workers’ time, thus contributing to harvest inefficiency. As an alternative to mechanical picking, the concept of a system consisting of instrumented picking carts and mobile tray-transporting robots were introduced to reduce the time pickers spend walking for container transportation during harvesting. During robot-aided harvesting, each picker collects fruits and puts them in a tray located on an instrumented cart. Each picking cart has two load cells to measure the weight of the tray, a GPS module to track its location while the picker walks in the furrows, and a LoRa wireless module to transmit data to the server in real-time. Software on the server generates/predicts tray pickup requests and computes the robot scheduling and dispatching for all available robots waiting at the collection station. The robots travel between the collection station and the pickers to bring empty trays to pickers and transport the full trays with the harvested produce from pickers to the collection station. Each robot plans its path to the tray pickup location and back and drives autonomously on that path. Robot localization is implemented by sensor fusion of the GPS-RTK modules, IMU, and wheel encoders installed on the robot. Robot operation is sequenced and supervised using a Finite State Machine on each robot. The entire system is modular and was developed using the Robot Operation System (ROS). A ROS network was built for communication among the picking carts, the server computer, and the computers on the robots.

The whole system is currently being tested in fields on the UC Davis campus. The navigation module has been implemented and tested in the mapped field obtaining less than 10cm tracking error compared to the planned path. It will be further tested in a commercial strawberry field in Salinas during the harvesting season to evaluate the improvement in harvest efficiency. Based on the simulation, the improved efficiency under the assistance of the crop-transporting robots is up to 19% when the robot/picker ratio is around ⅓. In the real field, we expect the efficiency improvement is above 15% for the same robot/picker ratio. The results will be presented at the conference.

Full text

System-level description and evaluation of a robot-aided
strawberry harvesting system
Chen Peng1, Stavros G. Vougioukas2,
Zhenghao Fei2, Benjamin Gatten1
UC Davis
1Department of Mechanical and Aerospace Engineering
2Department of Biological and Agricultural Engineering
2020 ASABE Annual International Meeting
Bio Automation Lab
Department of Biological & Agricultural Engineering
University of California, Davis, CA 95616
Sponsors

Outline
•Introduction
•Background;
•System components;
•System Diagram;
•Crop-transport Robots
•Robots Hardware;
•Robot localization;
•Path planning and tracking;
•Multiple robots' coordination;
•Evaluation of harvest-aiding system
•Experiment Design;
•Results analysis;
•Summary and Future work
2

Background: Manual Strawberry Harvesting Process
Field Setting
Collection station
Manual Harvesting
3
Tray delivery
•Pickers spend 15%-25% of
their time walking.
•This non-productive time
decreases efficiency.

Background: Crop-transport Robot
Field Setting
4
Transporting Robots
GOALS:
•Efficiency improvement.
•Labor reduction.
Collection station
Manual Picking with
instrumented cart

Laptop Server
5
System Components
Wi-Fi Range
extender
ROS-Network
ROS-Network
Simulated pickers
Seyyedhasani, H., Peng, C., Jang, W.J. and Vougioukas, S.G., 2020. Collaboration of human pickers and crop-
transporting robots during harvesting–Part II: Simulator evaluation and robot-scheduling case-study. Computers and
Electronics in Agriculture,172, p.105323.

Field Operations
Dispatching System
System Architecture
Pickers Operation Robots Operation
Transport Request
Prediction
Predictive
Scheduling
Pickers State
Observation Module
Carrito
messages Robots
States Dispatch
Commands
Robot State
Observation Module
Transport Requests
Return Buttons
Serve Flags
Reject Flags
Return Buttons

Crop-transport Robots: Hardware
7
GPS Antennas
Control Box:
- GPS Module + IMU;
- Router;
- Motor controllers;
- Mini computer;
- Batteries;
Emergency
Button
Steering
System Wheel
Encoder
Row Stride-
over chassis
Return
Button

Robot Localization: Field Mapping
8
1. Features of the field:
•Plant beds;
•Furrows;
2. Map of the field:
•RTK coordinates of head points of each bed and furrow;
•RTK coordinates of collection stations;
3. Local field frame:
•Localization;
•Navigation (path planning and path tracking);

Frame Transformation
Robot Localization: Sensor Fusion
9
GPS Modules
5Hz
Wheel Encoder
40 Hz
IMU
50 Hz EKF Node
Local Frame
IMU frame
Robot Frame
Robot Localization in
Local Frame

Path planning and tracking
10
Parking Path
•Type: 1.5 m linear path
•Speed: 0.3 m/s
Headland Path
•Type: Smooth Dubin path from parking
location to furrow entering point
•Speed: 0.5 m/s
Row entering Path
•Type: 3 m linear path
•Speed: 0.4 m/s
In-row path
•Type: linear path
•Speed: 1.5 m/s
Pre-picker path
•Type: linear path
•Speed: 0.3 m/s
Robot Parking
Location
Robot
Track Point
Target Point
Pure-Pursuit Path
Tracking
Rule-based Path Planner

Multiple Robots Coordination
11

12
System Evaluation: Experiment design
Harvesting configuration
•2 Crop-transport Robots;
•8/12/16 rows×50 meters;
•4/6/8 Simulated pickers (two rows per picker);

13
System Evaluation: Results without rejection
Field experiment ROS simulation
Number of pickers
Mean wait time of
pickers
Mean of robot
speed
Mean wait time of
pickers
Mean of robot
speed
4 19.5s
0.42m/s
18.7s
0.48 m/s
6 38.6s 37.2s
8 96.7s 95.8s

14
Pickers number Served trays ratio1Mean wait time2Mean transport time3Efficiency improvement4
4 24/38 7.88s 52.93s 16.32
6 31/55 12.91s 86.97s 13.64
8 30/68 19.35s 92.76s 11.58
System Evaluation: Results with rejection
1. Number of trays served/total harvested trays.
2. Mean of wait time of the pickers served by robots.
3. Mean transport time of trays if not served.
4. Efficiency improvement relative to manual harvesting.

15
Summary
1. A workable harvest aiding system was built and evaluated in the agricultural field;
2. Crop-transporting robots are evaluated in the system with less than 5% discrepancy to the
simulator;
3. Scheduling performance: given the robot/picker ratio to 1/3, 56% of pickers transport request can
be served and their working efficiency can be improved 13.64%;
Future work
1. Integrating the instrumented carrito into the system;
2. Scheduling with uncertain prediction of spatiotemporal full trays;

16
Acknowledgements
This work is part of the National Robotics Initiative project titled "FRAIL-bots:
Fragile cRop hArvest-aIding mobiLe robots", that is funded by NIFA-USDA

17
Thanks for your attention!