A Practical Approach for Picking Items in an Online Shopping Warehouse
Source abstract
Abstract
Commercially viable automated picking in unstructured environments by a robot arm remains a difficult challenge. The problem of robot grasp planning has long been around but the existing solutions tend to be limited when it comes to deploy them in open-ended realistic scenarios. Practical picking systems are called for that can handle the different properties of the objects to be manipulated, as well as the problems arising from occlusions and constrained accessibility. This paper presents a practical solution to the problem of robot picking in an online shopping warehouse by means of a novel approach that integrates a carefully selected method with a new strategy, the centroid normal approach (CNA), on a cost-effective dual-arm robotic system with two grippers specifically designed for this purpose: a two-finger gripper and a vacuum gripper. Objects identified in the scene point cloud are matched to the grasping techniques and grippers to maximize success. Extensive experimentation provides clues as to what are the reasons for success and failure. We chose as benchmark the scenario proposed by the 2017 Amazon Robotics Challenge, since it represents a realistic description of a retail shopping warehouse case; it includes many challenging constraints, such as a wide variety of different product items with a diversity of properties, which are also presented with restricted visibility and accessibility.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robot Manipulation and Learning · Robotic Path Planning Algorithms · Modular Robots and Swarm Intelligence
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Robotic Intelligence Lab., Universitat Jaume I, 12071 Castellón, Spain
Department of Interaction Science, Sungkyunkwan University, Seoul 03063, Korea
References (38)
Show source-provided references
1. Data-Driven Grasp Synthesis—A Survey10.1109/TRO.2013.2289018 ↗
2. Strand, M., Dillmann, R., Menegatti, E., and Ghidoni, S. (2019). Grasping Strategies for Picking Items in an Online Shopping Warehouse. Intelligent Autonomous Systems 15, Springer International Publishing.
3. Task-Based Robot Grasp Planning Using Probabilistic Inference10.1109/TRO.2015.2409912 ↗
4. Madry, M., Song, D., and Kragic, D. (2012, January 14–18). From object categories to grasp transfer using probabilistic reasoning. Proceedings of the 2012 IEEE International Conference on Robotics and Automation (ICRA), Saint Paul, MN, USA.10.1109/ICRA.2012.6225052 ↗
5. Huebner, K., and Kragic, D. (2008, January 22–26). Selection of robot pre-grasps using box-based shape approximation. Proceedings of the 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, Nice, France.10.1109/IROS.2008.4650722 ↗
6. Semantic grasping: Planning task-specific stable robotic grasps10.1007/s10514-014-9391-2 ↗
7. Category-based task specific grasping10.1016/j.robot.2015.04.002 ↗
8. Miller, A.T., Knoop, S., Christensen, H.I., and Allen, P.K. (2003, January 14–19). Automatic grasp planning using shape primitives. Proceedings of the Robotics and Automation, 2003. Proceedings. ICRA’03, IEEE International Conference, Taipei, Taiwan.
9. Saxena, A., Wong, L.L., and Ng, A.Y. (2008, January 13–17). Learning Grasp Strategies with Partial Shape Information. Proceedings of the AAAI-08, Chicago, IL, USA.
10. Fischinger, D., Vincze, M., and Jiang, Y. (2013, January 6–10). Learning grasps for unknown objects in cluttered scenes. Proceedings of the 2013 IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany.10.1109/ICRA.2013.6630636 ↗
11. Learning grasps with topographic features10.1177/0278364915577105 ↗
12. Gualtieri, M., ten Pas, A., Saenko, K., and Platt, R. (2016, January 9–14). High precision grasp pose detection in dense clutter. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea.10.1109/IROS.2016.7759114 ↗
13. Jiang, Y., Moseson, S., and Saxena, A. (2011, January 9–13). Efficient grasping from rgbd images: Learning using a new rectangle representation. Proceedings of the 2011 IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China.
14. Yoneyama, R., Duran, A.J., and del Pobil, A.P. (2021). Integrating Sensor Models in Deep Learning Boosts Performance: Application to Monocular Depth Estimation in Warehouse Automation. Sensors, 21.10.3390/s21041437 ↗
15. Hasegawa, S., Wada, K., Niitani, Y., Okada, K., and Inaba, M. (2017, January 24–28). A three-fingered hand with a suction gripping system for picking various objects in cluttered narrow space. Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.10.1109/IROS.2017.8202288 ↗
16. Tobin, J., Biewald, L., Duan, R., Andrychowicz, M., Handa, A., Kumar, V., McGrew, B., Ray, A., Schneider, J., and Welinder, P. (2018, January 1–5). Domain Randomization and Generative Models for Robotic Grasping. Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.10.1109/IROS.2018.8593933 ↗
17. Mahler, J., Matl, M., Liu, X., Li, A., Gealy, D., and Goldberg, K. (2018, January 21–25). Dex-Net 3.0: Computing Robust Vacuum Suction Grasp Targets in Point Clouds Using a New Analytic Model and Deep Learning. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia.10.1109/ICRA.2018.8460887 ↗
18. Zeng, A., Song, S., Yu, K.T., Donlon, E., Hogan, F.R., Bauza, M., Ma, D., Taylor, O., Liu, M., and Romo, E. (2018, January 21–25). Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia.10.1109/ICRA.2018.8461044 ↗
19. Learning robots to grasp by demonstration10.1016/j.robot.2020.103474 ↗
20. Schwarz, M., Lenz, C., García, G.M., Koo, S., Periyasamy, A.S., Schreiber, M., and Behnke, S. (2018, January 21–25). Fast Object Learning and Dual-arm Coordination for Cluttered Stowing, Picking, and Packing. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia.10.1109/ICRA.2018.8461195 ↗
21. Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
22. Self-Supervised Learning for Precise Pick-and-Place Without Object Model10.1109/LRA.2020.3003865 ↗
23. Mallick, A., del Pobil, A.P., and Cervera, E. (2018). Deep Learning Based Object Recognition for Robot Picking Task. Proceedings of the 12th International Conference on Ubiquitous Information Management and Communication, IMCOM ’18, ACM.10.1145/3164541.3164628 ↗
24. del Pobil, A.P., Kassawat, M., Duran, A.J., Arias, M.A., Nechyporenko, N., Mallick, A., Cervera, E., Subedi, D., Vasilev, I., and Cardin, D. (2017, January 16–18). UJI RobInLab’s approach to the Amazon Robotics Challenge 2017. Proceedings of the 2017 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), Daegu, Korea.10.1109/MFI.2017.8170448 ↗
25. A study on picking objects in cluttered environments: Exploiting depth features for a custom low-cost universal jamming gripper10.1016/j.rcim.2019.101888 ↗
26. The amazonpicking challenge 201510.1109/MRA.2015.2452071 ↗
27. Yu, K.T., Fazeli, N., Chavan-Dafle, N., Taylor, O., Donlon, E., Lankenau, G.D., and Rodriguez, A. (2016). A Summary of Team MIT’s Approach to the Amazon Picking Challenge 2015. arXiv.
28. Hernandez, C., Bharatheesha, M., Ko, W., Gaiser, H., Tan, J., van Deurzen, K., de Vries, M., Van Mil, B., van Egmond, J., and Burger, R. (2016). Team delft’s robot winner of the amazon picking challenge 2016. RoboCup 2016: Robot World Cup XX, Springer.10.1007/978-3-319-68792-6_51 ↗
29. Morrison, D., Tow, A.W., McTaggart, M., Smith, R., Kelly-Boxall, N., Wade-McCue, S., Erskine, J., Grinover, R., Gurman, A., and Hunn, T. (2017). Cartman: The low-cost cartesian manipulator that won the amazon robotics challenge. arXiv.10.1109/ICRA.2018.8463191 ↗
30. Analysis and Observations from the First Amazon Picking Challenge10.1109/TASE.2016.2600527 ↗
31. Eppner, C., Höfer, S., Jonschkowski, R., Martín-Martín, R., Sieverling, A., Wall, V., and Brock, O. (2018). Four Aspects of Building Robotic Systems: Lessons from the Amazon Picking Challenge 2015. Auton. Robot., 1–17.10.1007/s10514-018-9761-2 ↗
32. The UJI Librarian Robot10.1007/s11370-008-0028-1 ↗
33. Fischinger, D., and Vincze, M. (2012, January 7–12). Empty the basket-a shape based learning approach for grasping piles of unknown objects. Proceedings of the Intelligent Robots and Systems (IROS), IEEE/RSJ International Conference, Vilamoura-Algarve, Portugal.10.1109/IROS.2012.6386137 ↗
34. Nguyen, V.D. (1986, January 7–10). Constructing force-closure grasps. Proceedings of the 1986 IEEE International Conference on Robotics and Automation, San Francisco, CA, USA.10.1109/ROBOT.1986.1087483 ↗
35. Dalal, N., and Triggs, B. (2005, January 20–25). Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA.
36. ten Pas, A., and Platt, R. (2015, January 12–15). Using Geometry to Detect Grasps in 3d Point Cloud. Proceedings of the International Symposium on Robotics Research (ISRR), Genova, Italy.
37. Choi, S., Kim, T., and Yu, W. (2009, January 7–10). Performance Evaluation of RANSAC Family. Proceedings of the British Machine Vision Conference 2009, London, UK.10.5244/C.23.81 ↗
38. Chum, O., and Matas, J. (2005). Matching with PROSAC-Progressive Sample Consensus. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05)-Volume 1-Volume 01; CVPR ’05, IEEE Computer Society.10.1109/CVPR.2005.221 ↗