Proof-of-concept robot platform for exploring automated harvesting of sugar snap peas

Abstract

Currently, sugar snap peas are harvested manually. In high-cost countries like Norway, such a labour-intensive practise implies particularly large costs for the farmer. Hence, automated alternatives are highly sought after. This project explored a concept for robotic autonomous identification and tracking of sugar snap pea pods. The approach was based on a combination of visible–near infrared reflection measurements and image analysis, along with visual servoing. A proof-of-concept harvesting platform was implemented by mounting a robotic arm with hand-mounted sensors on a mobile unit. The platform was tested under plastic greenhouse conditions on potted plants of the sugar snap pea variety Cascadia using LED-lights and a partial shade. The results showed that it was feasible to differentiate the pods from the surrounding foliage using the light reflection at the spectral range around 970 nm combined with elementary image segmentation and shape modelling methods. The proof-of-concept harvesting platform was tested on 48 representative agricultural environments comprising dense canopy, varying pod sizes, partial occlusions and different working distances. A set of 104 images were analysed during the teleoperation experiment. The true positive detection rate was 93 and 87% for images acquired at long distances and at close distances, respectively. The robot arm achieved a success rate of 54% for autonomous visual servoing to a pre-grasp pose around targeted pods on 22 untouched scenarios. This study shows the potential of developing a prototype robot for semi-automated sugar snap pea harvesting.