4D Building Reconstruction with Machine Learning and Historical Maps
Source abstract
Abstract
The increasing importance of three-dimensional (3D) city modelling is linked to these data’s different applications and advantages in many domains. Images and Light Detection and Ranging (LiDAR) data availability are now an evident and unavoidable prerequisite, not always verified for past scenarios. Indeed, historical maps are often the only source of information when dealing with historical scenarios or multi-temporal (4D) digital representations. The paper presents a methodology to derive 4D building models in the level of detail 1 (LoD1), inferring missing height information through machine learning techniques. The aim is to realise 4D LoD1 buildings for geospatial analyses and visualisation, valorising historical data, and urban studies. Several machine learning regression techniques are analysed and employed for deriving missing height data from digitised multi-temporal maps. The implemented method relies on geometric, neighbours, and categorical attributes for height prediction. Derived elevation data are then used for 4D building reconstructions, offering multi-temporal versions of the considered urban scenarios. Various evaluation metrics are also presented for tackling the common issue of lack of ground-truth information within historical data.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: 3D Modeling in Geospatial Applications · Remote Sensing and LiDAR Applications · 3D Surveying and Cultural Heritage
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
3D Optical Metrology (3DOM) Unit, Bruno Kessler Foundation (FBK), Via Sommarive 18, 38123 Trento, Italy
Skolkovo Institute of Technology (Skoltech), Bolshoy Boulevard 30, 121205 Moscow, Russia
Funding
Fondazione Cassa Di Risparmio Di Trento E Rovereto
References (62)
Show source-provided references
1. Döllner, J., Kolbe, T.H., Liecke, F., Sgouros, T., and Teichmann, K. (2006, January 15–17). The virtual 3D city model of Berlin: Managing, integrating, and communicating complex urban information. Proceedings of the 25th Urban Data Management Symposium, Aalborg, Denmark.
2. Kersten, T.P., Keller, F., Sänger, J., and Schiewe, J. (2012). Automated Generation of an Historic 4D City Model of Hamburg and Its Visualisation with the GE Engine, Springer.10.1007/978-3-642-34234-9_6 ↗
3. Singh, S., Jain, K., and Mandla, V.R. (2013). Virtual 3D city modeling: Techniques and applications. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci., 73–91.10.5194/isprsarchives-XL-2-W2-73-2013 ↗
4. Kocaman, S., Akca, D., Poli, D., and Remondino, F. (2020). 3D/4D City Modelling—From Sensors to Applications, Whittles Publishing.
5. Applications of 3D city models: State of the art review10.3390/ijgi4042842 ↗
6. Multi-temporal analysis of landscapes and urban areas
7. An update on automatic 3D building reconstruction10.1016/j.isprsjprs.2010.09.006 ↗
8. Haala, N., Rothermel, M., and Cavegn, S. (April, January 30). Extracting 3D urban models from oblique aerial images. Proceedings of the 2015 Joint Urban Remote Sensing Event (JURSE), Lausanne, Switzerland.10.1109/JURSE.2015.7120479 ↗
9. Buyukdemircioglu, M., Kocaman, S., and Isikdag, U. (2018). Semi-automatic 3D city model generation from large-format aerial images. ISPRS Int. J. Geo-Inf., 7.10.3390/ijgi7090339 ↗
10. Chen, R. (2011, January 24–26). The development of 3D city model and its applications in urban planning. Proceedings of the 2011 19th International Conference on Geoinformatics, Shanghai, China.10.1109/GeoInformatics.2011.5981007 ↗
11. Ghassoun, Y., Löwner, M.-O., and Weber, S. (2015). Exploring the benefits of 3D city models in the field of urban particles distribution modelling—A comparison of model results. 3D Geoinformation Science, Springer.10.1007/978-3-319-12181-9_12 ↗
12. Willenborg, B., Sindram, M., and Kolbe, T.H. (2018). Applications of 3D city models for a better understanding of the built environment. Trends in Spatial Analysis and Modelling, Springer.10.1007/978-3-319-52522-8_9 ↗
13. Building extraction from airborne laser scanning data: An analysis of the state of the art10.3390/rs70403826 ↗
14. 3D city modelling with Oblique Photogrammetry Method10.1016/j.protcy.2015.02.060 ↗
15. Breunig, M., Al-Doori, M., Butwilowski, E., Kuper, P.V., Benner, J., and Haefele, K.H. (2015). 3D Geoinformation Science, Lecture Notes in Geoinformation and Cartographyitle, Springer.10.1007/978-3-319-12181-9 ↗
16. A dimension-independent extrusion algorithm using generalisedgeneralised maps10.1080/13658816.2015.1010535 ↗
17. A survey on procedural modelling for virtual worlds10.1111/cgf.12276 ↗
18. Towards defining a framework for the automatic derivation of 3D CityGML models from volunteered geographic information
19. An improved LOD specification for 3D building models10.1016/j.compenvurbsys.2016.04.005 ↗
20. CityGML—Interoperable semantic 3D city models10.1016/j.isprsjprs.2012.04.004 ↗
21. Gröger, G., Kolbe, T.H., Nagel, C., and Häfele, K.H. (2012). OGC City Geography Markup Language (CityGML) Encoding Standard, Open Geospatial Consortium. Version 2.0.0, OGC 08-007r2.
22. CityJSON: A compact and easy-to-use encoding of the CityGML data model10.1186/s40965-019-0064-0 ↗
23. Topologically consistent 3D city models obtained by extrusion10.1080/13658811003623277 ↗
24. Shi, Y., and He, B. (2012, January 19–21). Creating Topologically Consistent 3D City Models of LOD+ with Extrusion. Proceedings of the International Conference on Computer and Computing Technologies in Agriculture, Zhangjiajie, China.10.1007/978-3-642-36137-1_25 ↗
25. Fan, H., and Zipf, A. (2016). Modelling the world in 3D from VGI/Crowdsourced data. European Handbook of Crowdsourced Geographic Information, Ubiquity Press.10.5334/bax.ae ↗
26. Brasebin, M., Perret, J., Mustière, S., and Weber, C. (2016). A generic model to exploit urban regulation knowledge. ISPRS Int. J. Geo-Inf., 5.10.3390/ijgi5020014 ↗
27. A GIS-based method for modeling urban-climate parameters using automated recognition of shadows cast by buildings10.1016/j.compenvurbsys.2016.05.006 ↗
28. Generating 3D city models without elevation data10.1016/j.compenvurbsys.2017.01.001 ↗
29. Rook, M., Biljecki, F., and Diakité, A.A. (2016). Towards Automatic Semantic Labelling of 3D City Models. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci., 4.10.5194/isprs-annals-III-4-W1-51-2016 ↗
30. Roofn3D: Deep Learning Training Data for 3D Building Reconstruction10.5194/isprs-archives-XLII-2-1191-2018 ↗
31. Generating 3D City Models Based on the Semantic Segmentation of Lidar Data Using Convolutional Neural Networks10.5194/isprs-annals-IV-4-W8-3-2019 ↗
32. Tripodi, S., Duan, L., Trastour, F., Poujad, V., Laurore, L., and Tarabalka, Y. (2019, January 9–11). Deep learning-based extraction of building contours for large-scale 3D urban reconstruction. Proceedings of the Image and Signal Processing for Remote Sensing XXV, Strasbourg, France.10.1117/12.2533149 ↗
33. Lee, S., Jung, S., and Lee, J. (2019). Prediction model based on an artificial neural network for user-based building energy consumption in South Korea. Energies, 12.10.3390/en12040608 ↗
34. Benavente-Peces, C., and Ibadah, N. (2020). Buildings Energy Efficiency Analysis and Classification Using Various Machine Learning Technique Classifiers. Energies, 13.10.3390/en13133497 ↗
35. Mohammadiziazi, R., and Bilec, M.M. (2020). Application of machine learning for predicting building energy use at different temporal and spatial resolution under climate change in USA. Buildings, 10.10.3390/buildings10080139 ↗
36. Wu, Y., Filippovska, Y., Schmidt, V., and Kada, M. (2019, January 15–20). Application of Deep Learning for 3D building generalization. Proceedings of the 29th International Cartographic Conference (ICC 2019), Tokyo, Japan.10.5194/ica-proc-2-147-2019 ↗
37. Predicting building ages from LiDAR data with random forests for building energy modeling10.1016/j.enbuild.2013.10.004 ↗
38. Biljecki, F., and Sindram, M. (2017, January 26–27). Estimating building age with 3D GIS. Proceedings of the 12th International 3D GeoInfo Conference 2017, Melbourne, Australia.
39. Zeppelzauer, M., Despotovic, M., Sakeena, M., Koch, D., and Döller, M. (2018, January 11–14). Automatic prediction of building age from photographs. Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval, Yokohama, Japan.10.1145/3206025.3206060 ↗
40. Prediction of Building Structure Age Using Machine Learning
41. Mou, L., and Zhu, X.X. (2018). IM2HEIGHT: Height estimation from single monocular imagery via fully residual convolutional-deconvolutional network. arXiv.
42. Creating 3D city models with building footprints and LIDAR point cloud classification: A machine learning approach10.1016/j.compenvurbsys.2019.01.004 ↗
43. Liu, C.-J., Krylov, V.A., Kane, P., Kavanagh, G., and Dahyot, R. (2020). IM2ELEVATION: Building Height Estimation from Single-View Aerial Imagery. Remote Sens., 12.10.3390/rs12172719 ↗
44. Mahmud, J., Price, T., Bapat, A., and Frahm, J.-M. (2020, January 14–19). Boundary-Aware 3D Building Reconstruction from a Single Overhead Image. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00052 ↗
45. Kapoor, A., Larco, H., and Kiveris, R. (2019, January 4–8). Nostalgin: Extracting 3D City Models from Historical Image Data. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.10.1145/3292500.3330743 ↗
46. Anh, P., Thanh, N.T.N., Vu, C.T., Ha, N.V., and Hung, B.Q. (2018, January 23–24). Preliminary Result of 3D City Modelling for Hanoi, Vietnam. Proceedings of the 2018 5th NAFOSTED Conference on Information and Computer Science (NICS), Ho Chi Minh City, Vietnam.10.1109/NICS.2018.8606867 ↗
47. Introduction to machine learning, neural networks, and deep learning
48. Scikit-learn: Machine learning in Python
49. Rong, S., and Bao-wen, Z. (2018, January 18–20). The research of regression model in machine learning field. Proceedings of the MATEC Web of Conferences 2018 6th International Forum on Industrial Design (IFID 2018), Luoyang, China.
50. Random Forests10.1023/A:1010933404324 ↗
51. CatBoost for big data: An interdisciplinary review10.1186/s40537-020-00369-8 ↗
52. Awad, M., and Khanna, R. (2015). Support vector regression. Efficient Learning Machines, Springer.10.1007/978-1-4302-5990-9 ↗
53. Multilayer perceptrons for classification and regression10.1016/0925-2312(91)90023-5 ↗
54. Wong, S.C., Gatt, A., Stamatescu, V., and McDonnell, M.D. (December, January 30). Understanding data augmentation for classification: When to warp?. Proceedings of the 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA), Gold Coast, Australia.10.1109/DICTA.2016.7797091 ↗
55. Perez, L., and Wang, J. (2017). The effectiveness of data augmentation in image classification using deep learning. arXiv.
56. A survey on image data augmentation for deep learning10.1186/s40537-019-0197-0 ↗
57. A Full Stage Data Augmentation Method in Deep Convolutional Neural Network for Natural Image Classification
58. Paschali, M., Simson, W., Roy, A.G., Göbl, R., Wachinger, C., and Navab, N. (2019, January 2–7). Manifold Exploring Data Augmentation with Geometric Transformations for Increased Performance and Robustness. Proceedings of the International Conference on Information Processing in Medical Imaging, Hong Kong, China.10.1007/978-3-030-20351-1_40 ↗
59. Kim, E.K., Lee, H., Kim, J.Y., and Kim, S. (2020). Data Augmentation Method by Applying Color Perturbation of Inverse PSNR and Geometric Transformations for Object Recognition Based on Deep Learning. Appl. Sci., 10.10.3390/app10113755 ↗
60. Single view geometry10.1023/A:1026598000963 ↗
61. Remondino, F. (2003, January 22–25). Recovering metric information from old monocular video sequences. Proceedings of the 6th Conference on Optical 3D Measurement Techniques, Zürich, Switzerland.
62. Semple, J.G., and Kneebone, G.T. (1998). Algebraic Projective Geometry, Oxford University Press.10.1093/oso/9780198503637.001.0001 ↗