6-DoF Pose Estimation from Single RGB Image and CAD Model Retrieval Using Feature Similarity Measurement
Source abstract
Abstract
This study presents six degrees of freedom (6-DoF) pose estimation of an object from a single RGB image and retrieval of the matching CAD model by measuring the similarity between RGB and CAD rendering images. The 6-DoF pose estimation of an RGB object is one of the important techniques in 3D computer vision. However, in addition to 6-DoF pose estimation, retrieval and alignment of the matching CAD model with the RGB object should be performed for various industrial applications such as eXtended Reality (XR), Augmented Reality (AR), robot’s pick and place, and so on. This paper addresses 6-DoF pose estimation and CAD model retrieval problems simultaneously and quantitatively analyzes how much the 6-DoF pose estimation affects the CAD model retrieval performance. This study consists of two main steps. The first step is 6-DoF pose estimation based on the PoseContrast network. We enhance the structure of PoseConstrast by adding variance uncertainty weight and feature attention modules. The second step is the retrieval of the matching CAD model by an image similarity measurement between the CAD rendering and the RGB object. In our experiments, we used 2000 RGB images collected from Google and Bing search engines and 100 CAD models from ShapeNetCore. The Pascal3D+ dataset is used to train the pose estimation network and DELF features are used for the similarity measurement. Comprehensive ablation studies about the proposed network show the quantitative performance analysis with respect to the baseline model. Experimental results show that the pose estimation performance has a positive correlation with the CAD retrieval performance.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: 3D Surveying and Cultural Heritage · Image Processing and 3D Reconstruction · Robotics and Sensor-Based Localization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electronic and Electrical Engineering, Kyungpook National University, 80 Daehak-ro, Puk-gu, Daegu 41566, Republic of Korea
Funding
Korea government (MSIT) · Ministry of Culture, Sports and Tourism
References (33)
Show source-provided references
1. Wang, S., Clark, R., Wen, H., and Trigoni, N. (June, January 29). DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.10.1109/ICRA.2017.7989236 ↗
2. Distinctive Image Features from Scale-Invariant Keypoints10.1023/B:VISI.0000029664.99615.94 ↗
3. “Speeded-Up Robust Features (SURF)10.1016/j.cviu.2007.09.014 ↗
4. Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. (2011, January 6–13). ORB: An Efficient Alternative to SIFT or SURF. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.10.1109/ICCV.2011.6126544 ↗
5. A Method for Registration of 3-D Shapes10.1109/34.121791 ↗
6. Random Sample Consensus10.1145/358669.358692 ↗
7. Xiang, Y., Schmidt, T., Narayanan, V., and Fox, D. (2018). PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes. arXiv.10.15607/RSS.2018.XIV.019 ↗
8. Do, T.-T., Cai, M., Pham, T., and Reid, I. (2018). Deep-6DPose: Recovering 6D Object Pose from a Single RGB Image. arXiv.
9. Kendall, A., Grimes, M., and Cipolla, R. (2015, January 7–13). PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV) IEEE, Santiago, Chile.10.1109/ICCV.2015.336 ↗
10. Dong, Z., Liu, S., Zhou, T., Cheng, H., Zeng, L., Yu, X., and Liu, H. (2019, January 3–8). PPR-Net:Point-Wise Pose Regression Network for Instance Segmentation and 6D Pose Estimation in Bin-Picking Scenarios. Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China.10.1109/IROS40897.2019.8967895 ↗
11. Wang, C., Xu, D., Zhu, Y., Martin-Martin, R., Lu, C., Fei-Fei, L., and Savarese, S. (2019, January 16–20). DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00346 ↗
12. Zakharov, S., Shugurov, I., and Ilic, S. (November, January 27). DPOD: 6D Pose Object Detector and Refiner. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.10.1109/ICCV.2019.00203 ↗
13. Xu, Y., Lin, K.-Y., Zhang, G., Wang, X., and Li, H. (2022, January 21–24). RNNPose: Recurrent 6-DoF Object Pose Refinement with Robust Correspondence Field Estimation and Pose Optimization. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01446 ↗
14. Collaborative Distribution Alignment for 2D image-based 3D shape retrieval10.1016/j.jvcir.2021.103426 ↗
15. Park, S.-Y., Son, C.-M., Jeong, W.-J., and Park, S. (2023). Relative Pose Estimation between Image Object and ShapeNet CAD Model for Automatic 4-DoF Annotation. Appl. Sci., 13.10.3390/app13020693 ↗
16. Gümeli, C., Dai, A., and Nießner, M. (2022, January 19–24). ROCA: Robust CAD Model Retrieval and Alignment from a Single Image. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.00399 ↗
17. Xiao, Y., Du, Y., and Marlet, R. (2021, January 1–3). PoseContrast: Class-Agnostic Object Viewpoint Estimation in the Wild with Pose-Aware Contrastive Learning. Proceedings of the 2021 International Conference on 3D Vision (3DV), London, UK.10.1109/3DV53792.2021.00018 ↗
18. ROV6D: 6D Pose Estimation Benchmark Dataset for Underwater Remotely Operated Vehicles10.1109/LRA.2023.3331624 ↗
19. Noh, H., Araujo, A., Sim, J., Weyand, T., and Han, B. (2017, January 22–29). Large-Scale Image Retrieval with Attentive Deep Local Features. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.374 ↗
20. Qiao, S., Chen, L.-C., and Yuille, A. (2021, January 18–24). DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolution. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.01008 ↗
21. (2023, June 01). Home of the Blender Project—Free and Open 3D Creation Software. Available online: https://www.blender.org/.
22. Xiang, Y., Mottaghi, R., and Savarese, S. (2014, January 24–26). Beyond PASCAL: A Benchmark for 3D Object Detection in the Wild. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Steamboat Springs, CO, USA.10.1109/WACV.2014.6836101 ↗
23. Sun, X., Wu, J., Zhang, X., Zhang, Z., Zhang, C., Xue, T., Tenenbaum, J.B., and Freeman, W.T. (2018, January 18–22). Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00314 ↗
24. ObjectNet3D: A Large Scale Database for 3D Object Recognition10.1007/978-3-319-46484-8_10 ↗
25. Lin, T.-Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C.L., and Dollár, P. (2014, January 6–12). Microsoft COCO: Common Objects in Context. Proceedings of the Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland.10.1007/978-3-319-10602-1_48 ↗
26. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
27. A Mathematical Theory of Communication10.1002/j.1538-7305.1948.tb01338.x ↗
28. Girshick, R. (2015, January 7–13). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.169 ↗
29. Bangunharcana, A., Cho, J.W., Lee, S., Kweon, I.S., Kim, K.-S., and Kim, S. (2021, January 27). Correlate-and-Excite: Real-Time Stereo Matching via Guided Cost Volume Excitation. Proceedings of the 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic.10.1109/IROS51168.2021.9635909 ↗
30. Xu, G., Zhou, H., and Yang, X. (2023). CGI-Stereo: Accurate and Real-Time Stereo Matching via Context and Geometry Interaction. arXiv.
31. CBAM: Convolutional Block Attention Module10.1007/978-3-030-01234-2_1 ↗
32. Su, H., Qi, C.R., Li, Y., and Guibas, L.J. (2015, January 7–13). Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.308 ↗
33. Tulsiani, S., and Malik, J. (2015, January 7–12). Viewpoints and Keypoints. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298758 ↗