A Survey on Deep Learning in 3D CAD Reconstruction
Source abstract
Abstract
Three-dimensional CAD reconstruction is a long-standing and important task in fields such as industrial manufacturing, architecture, medicine, film and television, research, and education. Reconstructing CAD models remains a persistent challenge in machine learning. There have been many studies on deep learning in the field of 3D reconstruction. In recent years, with the release of CAD datasets, there have been more and more studies on 3D CAD reconstruction using deep learning. With the continuous deepening of research, deep learning has significantly improved the performance of tasks in the field of CAD reconstruction. However, this task remains challenging due to data scarcity and labeling difficulties, model complexity, and lack of generality and adaptability. This paper reviews both classic and recent research results on 3D CAD reconstruction tasks based on deep learning. To the best of our knowledge, this is the first investigation focusing on the CAD reconstruction task in the field of deep learning. Since there are relatively few studies related to 3D CAD reconstruction, we also investigate the reconstruction and generation of 2D CAD sketches. According to the different input data, we divide all investigations into the following categories: point cloud input to 3D CAD models, sketch input to 3D CAD models, other input to 3D CAD models, reconstruction and generation of 2D sketches, characterization of CAD data, CAD datasets, and related evaluation indicators. Commonly used datasets are outlined in our taxonomy. We provide a brief overview of the current research background, challenges, and recent results. Finally, future research directions are discussed.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: 3D Shape Modeling and Analysis · 3D Surveying and Cultural Heritage · Advanced Numerical Analysis Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Advanced Manufacturing, Fuzhou University, Quanzhou 362251, China
Funding
Enterprise Commissioned Research Project · Fuzhou University Graduate Education and Teaching Reform Project
References (100+)
Show source-provided references
1. Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling
2. Fan, H., Su, H., and Guibas, L.J. (2017, January 21–26). A point set generation network for 3D object reconstruction from a single image. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.264 ↗
3. Wohlers, T. (2014). 3D printing and additive manufacturing state of the industry. Annual Worldwide Progress Report, Wohlers Associates.
4. 3D printing in pharmaceutical and medical applications–recent achievements and challenges
5. Prince, S. (2011). Digital Visual Effects in Cinema: The Seduction of Reality, Rutgers University Press.
6. Towards the definition of best 3D practices in archaeology: Assessing 3D documentation techniques for intra-site data recording10.1016/j.culher.2015.07.005 ↗
7. Chang, A.X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., and Su, H. (2015). ShapeNet: An information-rich 3D model repository. arXiv.
8. Koch, S., Matveev, A., Jiang, Z., Williams, F., Artemov, A., Burnaev, E., Alexa, M., Zorin, D., and Panozzo, D. (2019, January 16–20). ABC: A big CAD model dataset for geometric deep learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00983 ↗
9. Fusion 360 gallery: A dataset and environment for programmatic CAD construction from human design sequences10.1145/3450626.3459818 ↗
10. Wu, R., Xiao, C., and Zheng, C. (2021, January 10–17). DeepCAD: A deep generative network for computer-aided design models. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00670 ↗
11. Sharma, G., Goyal, R., Liu, D., Kalogerakis, E., and Maji, S. (2018, January 18–22). CSGNet: Neural shape parser for constructive solid geometry. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00578 ↗
12. Sharma, G., Liu, D., Maji, S., Kalogerakis, E., Chaudhuri, S., and Měch, R. (2020, January 23–28). ParSeNet: A parametric surface fitting network for 3D point clouds. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK. Proceedings, Part VII 16.10.1007/978-3-030-58571-6_16 ↗
13. Complexgen: CAD reconstruction by B-rep chain complex generation
14. Jayaraman, P.K., Lambourne, J.G., Desai, N., Willis, K.D., Sanghi, A., and Morris, N.J. (2022). SolidGen: An autoregressive model for direct B-rep synthesis. arXiv.
15. Xu, X., Willis, K.D., Lambourne, J.G., Cheng, C.Y., Jayaraman, P.K., and Furukawa, Y. (2022). SkexGen: Autoregressive generation of CAD construction sequences with disentangled codebooks. arXiv.
16. Jayaraman, P.K., Sanghi, A., Lambourne, J.G., Willis, K.D., Davies, T., Shayani, H., and Morris, N. (2021, January 19–25). UV-Net: Learning from boundary representations. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Virtual, USA.10.1109/CVPR46437.2021.01153 ↗
17. Lambourne, J.G., Willis, K.D., Jayaraman, P.K., Sanghi, A., Meltzer, P., and Shayani, H. (2021, January 19–25). BRepNet: A topological message passing system for solid models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Virtual, USA.10.1109/CVPR46437.2021.01258 ↗
18. Chen, Z., Tagliasacchi, A., and Zhang, H. (2020, January 13–19). BSP-Net: Generating compact meshes via binary space partitioning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00012 ↗
19. InverseCSG: Automatic conversion of 3D models to CSG trees10.1145/3272127.3275006 ↗
20. UCSG-NET-unsupervised discovering of constructive solid geometry tree
21. A neural network system feature recognition for two-dimensional10.1080/095119298130859 ↗
22. Novel ANN-based feature recognition incorporating design by features10.1016/j.compind.2004.02.002 ↗
23. Boundary representation-based feature identification10.1016/B978-0-444-81600-9.50007-9 ↗
24. Application of neural networks in feature recognition of mould reinforced plastic parts10.1177/1063293X9900700204 ↗
25. Cao, W., Robinson, T., Hua, Y., Boussuge, F., Colligan, A.R., and Pan, W. (2020, January 17–19). Graph representation of 3D CAD models for machining feature recognition with deep learning. Proceedings of the International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Virtual.10.1115/DETC2020-22355 ↗
26. Li, P., Guo, J., Zhang, X., and Yan, D.M. (2023, January 18–22). SECAD-Net: Self-supervised CAD reconstruction by learning sketch-extrude operations. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.01613 ↗
27. Ren, D., Zheng, J., Cai, J., Li, J., and Zhang, J. (2022, January 23–27). ExtrudeNet: Unsupervised inverse sketch-and-extrude for shape parsing. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.10.1007/978-3-031-20086-1_28 ↗
28. Zhou, S., Tang, T., and Zhou, B. (2023, January 19–25). CADParser: A learning approach of sequence modeling for B-rep CAD. Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), Macau, China.10.24963/ijcai.2023/200 ↗
29. PIE-NET: Parametric inference of point cloud edges
30. Uy, M.A., Chang, Y.Y., Sung, M., Goel, P., Lambourne, J.G., Birdal, T., and Guibas, L.J. (2022, January 19–24). Point2Cyl: Reverse engineering 3D objects from point clouds to extrusion cylinders. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01155 ↗
31. Hu, W., Zheng, J., Zhang, Z., Yuan, X., Yin, J., and Zhou, Z. (2023, January 2–6). PlankAssembly: Robust 3D Reconstruction from Three Orthographic Views with Learnt Shape Programs. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France.10.1109/ICCV51070.2023.01695 ↗
32. Willis, K.D., Jayaraman, P.K., Lambourne, J.G., Chu, H., and Pu, Y. (2021, January 19–25). Engineering sketch generation for computer-aided design. Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPRW53098.2021.00239 ↗
33. Reverse engineering modeling methods and tools: A survey10.1080/16864360.2017.1397894 ↗
34. Topological operators and Boolean operations for complex-based nonmanifold geometric models10.1016/0010-4485(93)90097-8 ↗
35. Generic primitive detection in point clouds using novel minimal quadric fits10.1109/TPAMI.2019.2900309 ↗
36. Li, L., Sung, M., Dubrovina, A., Yi, L., and Guibas, L.J. (2019, January 16–20). Supervised fitting of geometric primitives to 3D point clouds. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00276 ↗
37. Sommer, C., Sun, Y., Bylow, E., and Cremers, D. (August, January 31). PrimiTect: Fast continuous hough voting for primitive detection. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.10.1109/ICRA40945.2020.9196988 ↗
38. Application of local-feature-based 3-D point cloud stitching method of low-overlap point cloud to aero-engine blade measurement
39. Deng, J., Liu, S., Chen, H., Chang, Y., Yu, Y., Ma, W., Wang, Y., and Xie, H. (2025). A Precise Method for Identifying 3D Circles in Freeform Surface Point Clouds. IEEE Trans. Instrum. Meas.10.1109/TIM.2025.3547492 ↗
40. Three-dimensional reconstruction of bubble geometry from single-perspective images based on ray tracing algorithm10.1088/1361-6501/ad7e43 ↗
41. Xu, X., Fu, X., Zhao, H., Liu, M., Xu, A., and Ma, Y. (2023). Three-dimensional reconstruction and geometric morphology analysis of lunar small craters within the patrol range of the Yutu-2 Rover. Remote Sens., 15.10.3390/rs15174251 ↗
42. Birdal, T., Busam, B., Navab, N., Ilic, S., and Sturm, P. (2018, January 18–22). A minimalist approach to type-agnostic detection of quadrics in point clouds. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00372 ↗
43. Paschalidou, D., Ulusoy, A.O., and Geiger, A. (2019, January 15–20). Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.01059 ↗
44. Segmentation methods for smooth point regions of conventional engineering objects10.1016/S0010-4485(03)00159-3 ↗
45. Xu, X., Peng, W., Cheng, C.Y., Willis, K.D., and Ritchie, D. (2021, January 20–25). Inferring CAD modeling sequences using zone graphs. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.00600 ↗
46. Atzmon, M., and Lipman, Y. (2020, January 14–19). SAL: Sign agnostic learning of shapes from raw data. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00264 ↗
47. Park, J.J., Florence, P., Straub, J., Newcombe, R., and Lovegrove, S. (2019, January 16–20). DeepSDF: Learning continuous signed distance functions for shape representation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00025 ↗
48. Cherenkova, K., Aouada, D., and Gusev, G. (2020, January 25–28). Pvdeconv: Point-voxel deconvolution for autoencoding CAD construction in 3D. Proceedings of the 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates.10.1109/ICIP40778.2020.9191095 ↗
49. Yang, B., Jiang, H., Pan, H., Wonka, P., Xiao, J., and Lin, G. (2024). PS-CAD: Local Geometry Guidance via Prompting and Selection for CAD Reconstruction. arXiv.10.1145/3733595 ↗
50. Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., and Lo, W.Y. (2023, January 1–7). Segment anything. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France.10.1109/ICCV51070.2023.00371 ↗
51. Liu, Y., D’Aronco, S., Schindler, K., and Wegner, J.D. (2021). PC2WF: 3D wireframe reconstruction from raw point clouds. arXiv.
52. Zhou, Y., Qi, H., Zhai, Y., Sun, Q., Chen, Z., Wei, L.Y., and Ma, Y. (November, January 27). Learning to reconstruct 3D manhattan wireframes from a single image. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.
53. Zhou, Y., Qi, H., and Ma, Y. (November, January 27). End-to-end wireframe parsing. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.
54. Xue, N., Wu, T., Bai, S., Wang, F., Xia, G.S., Zhang, L., and Torr, P.H. (2020, January 14–19). Holistically-attracted wireframe parsing. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00286 ↗
55. Wu, X., Jiang, L., Wang, P.S., Liu, Z., Liu, X., Qiao, Y., Ouyang, W., He, T., and Zhao, H. (2024, January 18–22). Point transformer v3: Simpler faster stronger. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52733.2024.00463 ↗
56. Automated semantic segmentation of industrial point clouds using ResPointNet++10.1016/j.autcon.2021.103874 ↗
57. Deep learning-based pipe segmentation and geometric reconstruction from poorly scanned point clouds using BIM-driven data alignment10.1016/j.autcon.2025.106071 ↗
58. Building Information Modeling (BIM) for existing buildings—Literature review and future needs10.1016/j.autcon.2013.10.023 ↗
59. BIM generation from 3D point clouds by combining 3D deep learning and improved morphological approach10.1016/j.autcon.2022.104422 ↗
60. On the shape of a set of points in the plane10.1109/TIT.1983.1056714 ↗
61. Automatic segmentation and 3D modeling of pipelines into constituent parts from laser-scan data of the built environment10.1016/j.autcon.2016.05.010 ↗
62. Automated 3D wireframe modeling of indoor structures from point clouds using constrained least-squares adjustment for as-built BIM10.1061/(ASCE)CP.1943-5487.0000556 ↗
63. Avetisyan, A., Dahnert, M., Dai, A., Savva, M., Chang, A.X., and Nießner, M. (2019, January 15–20). Scan2CAD: Learning CAD model alignment in rgb-d scans. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00272 ↗
64. A 2D sketch-based user interface for 3D CAD model retrieval10.1080/16864360.2005.10738335 ↗
65. Sketch-based interaction and modeling: Where do we stand?
66. Lun, Z., Gadelha, M., Kalogerakis, E., Maji, S., and Wang, R. (2017, January 10–12). 3D shape reconstruction from sketches via multi-view convolutional networks. Proceedings of the 2017 International Conference on 3D Vision (3DV), Qingdao, China.10.1109/3DV.2017.00018 ↗
67. Sketch2CAD: Sequential CAD modeling by sketching in context10.1145/3414685.3417763 ↗
68. Shape synthesis from sketches via procedural models and convolutional networks10.1109/TVCG.2016.2597830 ↗
69. Bae, S.H., Balakrishnan, R., and Singh, K. (2008, January 19–22). ILoveSketch: As-natural-as-possible sketching system for creating 3d curve models. Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology (UIST), Monterey, CA, USA.10.1145/1449715.1449740 ↗
70. Han, W., Xiang, S., Liu, C., Wang, R., and Feng, C. (2020, January 13–19). SPARE3D: A dataset for spatial reasoning on three-view line drawings. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01470 ↗
71. Fast 3D solid model reconstruction from orthographic views10.1016/S0010-4485(97)00054-7 ↗
72. Nash, C., Ganin, Y., Eslami, S.A., and Battaglia, P. (2020, January 13–18). PolyGen: An autoregressive generative model of 3D meshes. Proceedings of the International Conference on Machine Learning (ICML), Washington, DC, USA.
73. A survey of 3D solid reconstruction from 2D projection line drawings10.1111/1467-8659.1220137 ↗
74. Solid model input through orthographic views10.1145/964967.801155 ↗
75. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the 31st Advances in Neural Information Processing Systems (NeurIPS 2017), Long Beach, CA, USA.
76. Fan, Z., Zhu, L., Li, H., Chen, X., Zhu, S., and Tan, P. (2021, January 11–17). FloorPlanCAD: A large-scale CAD drawing dataset for panoptic symbol spotting. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Virtual Conference.10.1109/ICCV48922.2021.00997 ↗
77. Fan, Z., Chen, T., Wang, P., and Wang, Z. (2022, January 18–24). Cadtransformer: Panoptic symbol spotting transformer for CAD drawings. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01071 ↗
78. Geosemantic snapping for sketch-based modeling10.1111/cgf.12044 ↗
79. Yang, G., Huang, X., Hao, Z., Liu, M.Y., Belongie, S., and Hariharan, B. (November, January 27). PointFlow: 3D point cloud generation with continuous normalizing flows. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.
80. Wang, N., Zhang, Y., Li, Z., Fu, Y., Liu, W., and Jiang, Y.G. (2018, January 8–14). Pixel2Mesh: Generating 3D mesh models from single rgb images. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01252-6_4 ↗
81. Xu, X., Jayaraman, P.K., Lambourne, J.G., Willis, K.D., and Furukawa, Y. (2023). Hierarchical neural coding for controllable CAD model generation. arXiv.
82. Yu, F., Chen, Z., Li, M., Sanghi, A., Shayani, H., Mahdavi-Amiri, A., and Zhang, H. (2022, January 18–24). Capri-Net: Learning compact CAD shapes with adaptive primitive assembly. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01147 ↗
83. Neural Shape Parsers for Constructive Solid Geometry
84. Xu, X., Lambourne, J.G., Jayaraman, P.K., Wang, Z., Willis, K.D., and Furukawa, Y. (2024). BrepGen: A B-rep Generative Diffusion Model with Structured Latent Geometry. arXiv.10.1145/3658129 ↗
85. Denoising diffusion probabilistic models
86. Jones, B.T., Hu, M., Kodnongbua, M., Kim, V.G., and Schulz, A. (2023, January 18–24). Self-supervised representation learning for CAD. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.02043 ↗
87. Druc, S., Balu, A., Wooldridge, P., Krishnamurthy, A., and Sarkar, S. (2022, January 18–24). Concept activation vectors for generating user-defined 3D shapes. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPRW56347.2022.00338 ↗
88. Parametric CAD modeling: An analysis of strategies for design reusability10.1016/j.cad.2016.01.003 ↗
89. Exchange of CAD part models based on the macro-parametric approach
90. A benchmark for rough sketch cleanup
91. SketchGen: Generating constrained CAD sketches
92. Computer-aided design as language
93. Seff, A., Zhou, W., Richardson, N., and Adams, R.P. (2021). Vitruvion: A generative model of parametric CAD sketches. arXiv.
94. Model-guided 3D sketching10.1109/TVCG.2018.2860016 ↗
95. Karadeniz, A.S., Mallis, D., Mejri, N., Cherenkova, K., Kacem, A., and Aouada, D. (2024). DAVINCI: A Single-Stage Architecture for Constrained CAD Sketch Inference. arXiv.
96. Sarcar, M., Rao, K.M., and Narayan, K.L. (2008). Computer Aided Design and Manufacturing, PHI Learning Pvt. Ltd.
97. Radhakrishnan, P., Subramanyan, S., and Raju, V. (2008). Cad/Cam/Cim, New Age International.
98. Geometric modeling of solid objects by using a face adjacency graph representation10.1145/325165.325218 ↗
99. A topological enabled three-dimensional model based on constructive solid geometry and boundary representation10.1007/s10586-016-0634-1 ↗
100. Industrial geometry: Recent advances and applications in CAD10.1016/j.cad.2004.08.013 ↗