A Learnable Viewpoint Evolution Method for Accurate Pose Estimation of Complex Assembled Product
Source abstract
Abstract
Balancing adaptability, reliability, and accuracy in vision technology has always been a major bottleneck limiting its application in appearance assurance for complex objects in high-end equipment production. Data-driven deep learning shows robustness to feature diversity but is limited by interpretability and accuracy. The traditional vision scheme is reliable and can achieve high accuracy, but its adaptability is insufficient. The deeper reason is the lack of appropriate architecture and integration strategies between the learning paradigm and empirical design. To this end, a learnable viewpoint evolution algorithm for high-accuracy pose estimation of complex assembled products under free view is proposed. To alleviate the balance problem of exploration and optimization in estimation, shape-constrained virtual–real matching, evolvable feasible region, and specialized population migration and reproduction strategies are designed. Furthermore, a learnable evolution control mechanism is proposed, which integrates a guided model based on experience and is cyclic-trained with automatically generated effective trajectories to improve the evolution process. Compared to the 1.69°,55.67 mm of the state-of-the-art data-driven method and the 1.28°,77.67 mm of the classic strategy combination, the pose estimation error of complex assembled product in this study is 0.23°,23.71 mm, which proves the effectiveness of the proposed method. Meanwhile, through in-depth exploration, the robustness, parameter sensitivity, and adaptability to the virtual–real appearance variations are sequentially verified.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robotics and Sensor-Based Localization · 3D Surveying and Cultural Heritage · Robot Manipulation and Learning
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Mechanical Engineering and Automation, Beihang University, 37 College Road, Haidian District, Beijing 100191, China
Funding
National Natural Science Foundation of China
References (50)
Show source-provided references
1. Pose Estimation for Augmented Reality: A Hands-On Survey10.1109/TVCG.2015.2513408 ↗
2. Wang, Z., Shang, Y., and Zhang, H. (2018, January 10–12). A Survey on Approaches of Monocular CAD Model-Based 3D Objects Pose Estimation and Tracking. Proceedings of the 2018 IEEE CSAA Guidance, Navigation and Control Conference (CGNCC), Xiamen, China.10.1109/GNCC42960.2018.9019029 ↗
3. A review of deep learning-based approaches for defect detection in smart manufacturing10.1007/s12596-023-01340-5 ↗
4. Augmented reality-based guidance in product assembly and maintenance/repair perspective: A state of the art review on challenges and opportunities10.1016/j.eswa.2022.118983 ↗
5. Hao, J.C., He, D., Li, Z.Y., Hu, P., Chen, Y., and Tang, K. (2023). Efficient cutting path planning for a non-spherical tool based on an iso-scallop height distance field. Chin. J. Aeronaut., in press.10.1016/j.cja.2023.12.005 ↗
6. Coverage path planning with targeted viewpoint sampling for robotic free-form surface inspection10.1016/j.rcim.2019.101843 ↗
7. Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: A review10.1007/s10462-020-09888-5 ↗
8. Wang, H.Y., Shen, Q., Deng, Z.L., Cao, X., and Wang, X. Absolute pose estimation of UAV based on large-scale satellite image. Chin. J. Aeronaut., 2023. in press.10.1016/j.cja.2023.12.028 ↗
9. Zhang, M., Zhang, C.C., Wang, W., Du, R., and Meng, S. (2021, January 28–30). Research on Automatic Assembling Method of Large Parts of Spacecraft Based on Vision Guidance. Proceedings of the 2021 2nd International Conference on Artificial Intelligence and Information Systems, ACM, Chongqing, China.
10. Design and research of automobile anti-collision warning system based on monocular vision sensor with license plate cooperative target10.1007/s11042-016-4042-6 ↗
11. A Novel Dual-Robot Accurate Calibration Method Using Convex Optimization and Lie Derivative10.1109/TRO.2023.3344025 ↗
12. A Fast and Flexible Projector-Camera Calibration System10.1109/TASE.2020.2994223 ↗
13. Absolute calibration of an ABB IRB 1600 robot using a laser tracker10.1016/j.rcim.2012.06.004 ↗
14. Three-dimensional shape measurement technique for large-scale objects based on line structured light combined with industrial robot10.1016/j.ijleo.2019.163656 ↗
15. 3D Object Recognition and Pose Estimation from Point Cloud Using Stably Observed Point Pair Feature10.1109/ACCESS.2020.2978255 ↗
16. Wuest, H., Vial, F., and Stricker, D. (2005, January 5–8). Adaptive line tracking with multiple hypotheses for augmented reality. Proceedings of the Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR’05), Vienna, Austria.10.1109/ISMAR.2005.8 ↗
17. A review of edge-based 3D tracking of rigid objects10.1016/j.vrih.2019.10.001 ↗
18. An Occlusion-aware Edge-Based Method for Monocular 3D Object Tracking using Edge Confidence10.1111/cgf.14154 ↗
19. Jau, Y.-Y., Zhu, R., Su, H., and Chandraker, M. (2020–24, January 24). Deep keypoint-based camera pose estimation with geometric constraints. Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.10.1109/IROS45743.2020.9341229 ↗
20. EPnP: An accurate O(n) solution to the PnP problem10.1007/s11263-008-0152-6 ↗
21. Ferraz, L., Binefa, X., and Moreno-Noguer, F. (2014, January 1). Leveraging Feature Uncertainty in the PnP Problem. Proceedings of the British Machine Vision Conference 2014, British Machine Vision Association, Nottingham, UK.10.5244/C.28.83 ↗
22. ASPnP: An Accurate and Scalable Solution to the Perspective-n-Point Problem10.1587/transinf.E96.D.1525 ↗
23. Garro, V., Crosilla, F., and Fusiello, A. (2012, January 13–15). Solving the PnP Problem with Anisotropic Orthogonal Procrustes Analysis. Proceedings of the 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission, Zurich, Switzerland.10.1109/3DIMPVT.2012.40 ↗
24. MLPnP-A Real-Time Maximum Likelihood Solution to the Perspective-n-Point Problem
25. NetVLAD: CNN Architecture for Weakly Supervised Place Recognition10.1109/TPAMI.2017.2711011 ↗
26. Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016). Computer Vision—ECCV 2016, Springer International Publishing.10.1007/978-3-319-46454-1 ↗
27. Humenberger, M., Cabon, Y., Guerin, N., Morat, J., Leroy, V., Revaud, J., Rerole, P., Pion, N., de Souza, C., and Csurka, G. (2022). Robust Image Retrieval-based Visual Localization using Kapture. arXiv.
28. Kendall, A., Grimes, M., and Cipolla, R. (2015, January 7–13). PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.336 ↗
29. Kendall, A., and Cipolla, R. (2017, January 21–26). Geometric Loss Functions for Camera Pose Regression with Deep Learning. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.694 ↗
30. PVNet: Pixel-wise voting network for 6dof pose estimation10.1109/TPAMI.2020.3047388 ↗
31. Ferrari, V., Hebert, M., Sminchisescu, C., and Weiss, Y. (2018). Computer Vision-ECCV 2018, Springer International Publishing.10.1007/978-3-030-01252-6 ↗
32. Xu, Y., Lin, K.-Y., Zhang, G., Wang, X., and Li, H. (2022, January 18–24). RNNPose: Recurrent 6-DoF Object Pose Refinement with Robust Correspondence Field Estimation and Pose Optimization. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01446 ↗
33. Bukschat, Y., and Vetter, M. (2020). EfficientPose-An efficient, accurate and scalable end-to-end 6D multi object pose estimation approach. arXiv.
34. Labbe, Y., Manuelli, L., Mousavian, A., Tyree, S., Birchfield, S., Tremblay, J., Carpentier, J., Aubry, M., Fox, D., and Sivic, J. (2022). Megapose: 6d pose estimation of novel objects via render & compare. arXiv.
35. Brachmann, E., Krull, A., Michel, F., Gumhold, S., Shotton, J., and Rother, C. (2014). Computer Vision—ECCV 2014, Springer. Lecture Notes in Computer Science.10.1007/978-3-319-12096-6 ↗
36. Brachmann, E., Krull, A., Nowozin, S., Shotton, J., Michel, F., Gumhold, S., and Rother, C. (2017, January 21–26). DSAC-Differentiable RANSAC for Camera Localization. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.267 ↗
37. Sarlin, P.-E., Cadena, C., Siegwart, R., and Dymczyk, M. (2019, January 15–20). From Coarse to Fine: Robust Hierarchical Localization at Large Scale. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.01300 ↗
38. Ben Abdallah, H., Jovančević, I., Orteu, J.-J., and Brèthes, L. (2019). Automatic Inspection of Aeronautical Mechanical Assemblies by Matching the 3D CAD Model and Real 2D Images. J. Imaging, 5.10.3390/jimaging5100081 ↗
39. Regularization Based Iterative Point Match Weighting for Accurate Rigid Transformation Estimation10.1109/TVCG.2015.2410272 ↗
40. Importance sampling in reinforcement learning with an estimated behavior policy10.1007/s10994-020-05938-9 ↗
41. Tjaden, H., Schwanecke, U., and Schömer, E. (2017, January 22–29). Real-time monocular pose estimation of 3d objects using temporally consistent local color histograms. Proceedings of the IEEE International Conference on Computer Vision, Institute of Electrical and Electronics Engineers Inc., Venice, Italy.10.1109/ICCV.2017.23 ↗
42. Carlile, B., Delamarter, G., Kinney, P., Marti, A., and Whitney, B. (2017). Improving Deep Learning by Inverse Square Root Linear Units (ISRLUs). arXiv.
43. Lee, J., and Leyffer, S. (2012). Mixed Integer Nonlinear Programming, Springer.10.1007/978-1-4614-1927-3 ↗
44. Reduced Hessian Successive Quadratic Programming for Realtime Optimization
45. Pixel-Wise Weighted Region-Based 3D Object Tracking Using Contour Constraints10.1109/TVCG.2021.3085197 ↗
46. Fast and Robust Iterative Closest Point
47. Tekin, B., Sinha, S.N., and Fua, P. (2018, January 18–23). Real-Time Seamless Single Shot 6D Object Pose Prediction. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00038 ↗
48. Robust and Accurate Monocular Pose Tracking for Large Pose Shift10.1109/TIE.2022.3217598 ↗
49. Tian, X., Lin, X., Zhong, F., and Qin, X. (2022, January 23–27). Large-Displacement 3D Object Tracking with Hybrid Non-local Optimization. Proceedings of the Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel.10.1007/978-3-031-20047-2_36 ↗
50. Stoiber, M., Pfanne, M., Strobl, K.H., Triebel, R., and Albu-Schäffer, A. (2020, January 30). A sparse gaussian approach to region-based 6DoF object tracking. Proceedings of the Computer Vision-ACCV 2020: 15th Asian Conference on Computer Vision, Kyoto, Japan.10.1007/978-3-030-69532-3_40 ↗