Learning Robot Harvests Peppers

Learning Robot Harvests Peppers The agricultural industry continually faces challenges, including labor shortages and the need to improve productivity and sustainability. Automation through robotics is a promising solution. Traditional control algorithms are often inadequate for the variability of agricultural tasks, whereas “imitation learning” enables robots to learn from human demonstrations, improving adaptability in complex environments. “Unlike earlier projects that needed specialized modules for vision and control, our method uses imitation learning, collecting demonstrations and training one algorithm to process sensory data and guide pepper harvesting,” says John Kim, a PhD student. “It’s showing good results.” Kim used a custom handheld shear-gripper with sensors to teach the robot to pick peppers by mimicking human actions and using advanced software (Universal Manipulation Interface framework) to guide its movements. The ergonomic gripper with pose tracking and a fisheye camera enabled data collection, while the UR5e robot on a Husky platform was tested outdoors. Trained diffusion policy software and a grasp detector managed harvesting and pepper placement. “We went to pepper fields and manually harvested 300 demonstrations,” Kim explains. “It was enough to get us started, but not close enough to get us to the point where we could confidently say there’s a very good chance of harvesting 99% accuracy.” Kim completed the demonstrations at the Iowa State University Horticulture Research Station over two days. “Our main goal during testing wasn’t how fast the robot picks, but how well the algorithm works. Speeding up the harvesting is a downstream process when we want to deploy it on a larger scale,” Kim says. The harvesting policy reached a 29% success rate across 221 trials, with results impacted by peduncle shape and occlusion. The grasp detector was 83% accurate, demonstrating strong precision and recall in identifying successful grasps. Compared with previous greenhouse trials, the system performed similarly under challenging outdoor conditions, demonstrating its adaptability to environmental changes and dynamic lighting. Failures mainly involved gripper positioning and pepper drops after harvest, highlighting areas needing improvement. Future upgrades could include better cameras for peduncle targeting, improved foliage management, and expansion to other crops. “It’s easy to get lost in the technical detail and application and miss the big picture,” says George Kantor, a professor at Carnegie Mellon University who oversaw the trials. “We taught a robot to do an agricultural task by demonstration, which is something that has never been done before. “If we believe robots will do 10,000 different things in agriculture, there won’t be computer scientists available to program that number of robots for so many things. A general-purpose robot and a method to technically program it with a demonstration would be game-changing. We’re still far away from a farmer picking peppers, but it’s a first step in that direction.” Contact: FARM SHOW Followup, George Kantor, Carnegie Mellon University, Robotics Institute, 2100F Newell-Simon Hall, Pittsburgh, Pa. 15213 (kantor@cmu.edu; www.ri.cmu.edu).

To read the rest of this story, download this issue below or click here to register with your account number.