Optimizing Geospatial Data for ML/CV Applications: A Python-Based Approach to Streamlining Map Processing by Removing Irrelevant Areas
Source abstract
Abstract
Massive image datasets are often required for the proper functioning of Machine Learning (ML) and Computer Vision (CV) applications. This paper offers a solution to computational challenges in the Image Processing of satellite imagery, by proposing an optimization procedure. The presented approach is verified by an exemplary Python implementation, constituting a standalone tool for automating the dataset creation and labeling, including the extraction of road network data from the national satellite cartography provider. The collected data include detailed road maps along with the parcel information obtained via WebMapService endpoints. The method presented in this paper involves three basic steps: road segmentation (using the Shapely module) to facilitate handling high-resolution orthoimagery, and then a modified Region-of-Interest approach, i.e., removing irrelevant areas, with only roads remaining. This results in obtaining file sizes that are significantly smaller. The presented algorithm also involves asynchronous tile downloading, which, combined with the masking of irrelevant areas, improves not only the efficiency but surprisingly also the accuracy of subsequent ML/CV procedures. The research results of the paper reveal substantial file size reduction, and improved processing efficiency, thus making the optimized geospatial graphical data more practical for ML/CV applications, while still maintaining the original data quality and relevance of the analyzed parcels or infrastructure.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Computational Physics and Python Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science, Opole University of Technology, Proszkowska 76, 45-758 Opole, Poland
References (55)
Show source-provided references
1. A survey on dataset quality in Machine Learning10.1016/j.infsof.2023.107268 ↗
2. Guo, W., Yang, W., Zhang, H., and Hua, G. (2018). Geospatial Object Detection in High Resolution Satellite Images Based on Multi-Scale Convolutional Neural Network. Remote Sens., 10.10.3390/rs10010131 ↗
3. A review of high-definition map creation methods for autonomous driving10.1016/j.engappai.2023.106125 ↗
4. A comparison of urban mapping methods using high-resolution digital imagery10.14358/PERS.69.9.963 ↗
5. Stuart, M.B., Davies, M., Hobbs, M.J., Pering, T.D., McGonigle, A.J.S., and Willmott, J.R. (2022). High-Resolution Hyperspectral Imaging Using Low-Cost Components: Application within Environmental Monitoring Scenarios. Sensors, 22.10.3390/s22124652 ↗
6. Hurricane Disaster Assessments With Image-Driven Data Mining in High-Resolution Satellite Imagery10.1109/TGRS.2007.890808 ↗
7. A fuzzy decision making system for building damage map creation using high resolution satellite imagery10.5194/nhess-13-455-2013 ↗
8. Performance of high-resolution satellite precipitation products over China10.1029/2009JD012097 ↗
9. BigDataViewer: Visualization and processing for large image data sets10.1038/nmeth.3392 ↗
10. On Creating Benchmark Dataset for Aerial Image Interpretation: Reviews, Guidances, and Million-AID10.1109/JSTARS.2021.3070368 ↗
11. Gu, S., Lugmayr, A., Danelljan, M., Fritsche, M., Lamour, J., and Timofte, R. (2019, January 27–28). DIV8K: DIVerse 8K Resolution Image Dataset. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Republic of Korea.10.1109/ICCVW.2019.00435 ↗
12. Gomes, V.C., Queiroz, G.R., and Ferreira, K.R. (2020). An overview of platforms for big earth observation data management and analysis. Remote Sens., 12.10.3390/rs12081253 ↗
13. Dyke, G., Rosenqvist, A., Killough, B., and Yuan, F. (2021, January 11–16). Intercomparison of Sentinel-1 datasets from Google earth engine and the sinergise Sentinel Hub Card4L Tool. Proceedings of the 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, Brussels, Belgium.10.1109/IGARSS47720.2021.9554039 ↗
14. Geoportal usability evaluation
15. The European geoportal—-one step towards the establishment of a European Spatial Data Infrastructure10.1016/S0198-9715(04)00049-3 ↗
16. Meinel, G., Lippold, R., and Netzband, M. (1998, January 7–10). The potential use of new high resolution satellite data for urban and regional planning. Proceedings of the ISPRS Commission IV Symposium on GIS—Between Visions and Applications, Stuttgart, Germany.
17. Current status and future directions of precision aerial application for site-specific crop management in the USA10.1016/j.compag.2010.07.001 ↗
18. Reference scenarios for deforestation and forest degradation in support of REDD: A review of data and methods10.1088/1748-9326/3/2/025011 ↗
19. Urban development analysis using built-up area maps based on multiple high-resolution satellite data
20. A near real-time satellite-based global drought climate data record10.1088/1748-9326/7/4/044037 ↗
21. Detection of cars in high-resolution aerial images of complex urban environments10.1109/TGRS.2017.2716984 ↗
22. Coastal and environmental remote sensing from unmanned aerial vehicles: An overview10.2112/JCOASTRES-D-15-00005.1 ↗
23. Road extraction methods in high-resolution remote sensing images: A comprehensive review10.1109/JSTARS.2020.3023549 ↗
24. Detecting road obstacles by erasing them10.1109/TPAMI.2023.3335152 ↗
25. Ma, K., Hoai, M., and Samaras, D. (2017, January 4–7). Large-scale Continual Road Inspection: Visual Infrastructure Assessment in the Wild. Proceedings of the BMVC, London, UK.10.5244/C.31.151 ↗
26. Courtial, A., El Ayedi, A., Touya, G., and Zhang, X. (2020). Exploring the potential of deep learning segmentation for mountain roads generalisation. ISPRS Int. J.-Geo-Inf., 9.10.3390/ijgi9050338 ↗
27. Geoportal (2024, September 30). Information About Geoportal, Available online: https://www.geoportal.gov.pl/en/about-geoportal/information/.
28. Openstreetmap: User-generated street maps10.1109/MPRV.2008.80 ↗
29. Zhang, H., and Malczewski, J. (2019). Quality evaluation of volunteered geographic information: The case of OpenStreetMap. Crowdsourcing: Concepts, Methodologies, Tools, and Applications, IGI Global.10.4018/978-1-5225-8362-2.ch058 ↗
30. Quality analysis of OpenStreetMap data based on application needs
31. Testing the positional accuracy of OpenStreetMap data for mapping applications10.3846/20296991.2015.1160493 ↗
32. Zhao, Q., Yu, L., Li, X., Peng, D., Zhang, Y., and Gong, P. (2021). Progress and trends in the application of Google Earth and Google Earth Engine. Remote Sens., 13.10.3390/rs13183778 ↗
33. Google Earth as a resource10.1080/00167487.2015.12093954 ↗
34. A procedure for semi-automated cadastral boundary feature extraction from high-resolution satellite imagery10.1080/14498596.2017.1345667 ↗
35. Satellite imagery cadastral features extractions using image processing algorithms: A viable option for cadastral science
36. Survey on Land Use/Land Cover (LU/LC) change analysis in remote sensing and GIS environment: Techniques and Challenges10.1007/s11356-020-09091-7 ↗
37. Comparison of land use/land cover change and landscape patterns in Honghe National Nature Reserve and the surrounding Jiansanjiang Region, China10.1016/j.ecolind.2014.11.025 ↗
38. Land suitability analysis for agriculture in the Abbay basin using remote sensing, GIS and AHP techniques10.1007/s40808-016-0167-x ↗
39. Infrastructure mapping and performance assessment of irrigation system using GIS and remote sensing10.1051/e3sconf/202126403005 ↗
40. Creating cadastral maps in rural and urban areas of using high resolution satellite imagery
41. Kwong, I.H., Wong, F.K., Fung, T., Liu, E.K., Lee, R.H., and Ng, T.P. (2021). A multi-stage approach combining very high-resolution satellite image, gis database and post-classification modification rules for habitat mapping in Hong Kong. Remote Sens., 14.10.3390/rs14010067 ↗
42. Basheer, S., Wang, X., Farooque, A.A., Nawaz, R.A., Liu, K., Adekanmbi, T., and Liu, S. (2022). Comparison of land use land cover classifiers using different satellite imagery and machine learning techniques. Remote Sens., 14.10.3390/rs14194978 ↗
43. Shizan, M.M.I., Aurgho, A.S., Ani, F.H., Bondhon, A.R., and Kalpoma, K.A. (2023). Dataset for Road Roughness Assessment Using Image Classification Techniques and Deep Learning Models: A Case Study on Bangladeshi National Highways. International Conference on Big Data, IoT and Machine Learning, Springer.10.1007/978-981-99-8937-9_19 ↗
44. Abdollahi, A., Pradhan, B., Shukla, N., Chakraborty, S., and Alamri, A. (2020). Deep learning approaches applied to remote sensing datasets for road extraction: A state-of-the-art review. Remote Sens., 12.10.3390/rs12091444 ↗
45. Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Patnaik, V., Tsui, P., Guo, J., Zhou, Y., Chai, Y., and Caine, B. (2020, January 14–19). Scalability in perception for autonomous driving: Waymo open dataset. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00252 ↗
46. RDD2022: A multi-national image dataset for automatic road damage detection
47. Wang, W., Xu, Z., Sun, D., and Lan, T. (2021). Spatial optimization of mega-city fire stations based on multi-source geospatial data: A case study in Beijing. ISPRS Int. J.-Geo-Inf., 10.10.3390/ijgi10050282 ↗
48. Leclerc, G., Ilyas, A., Engstrom, L., Park, S.M., Salman, H., and Mądry, A. (2023, January 20–22). FFCV: Accelerating training by removing data bottlenecks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01156 ↗
49. Lawhead, J. (2019). Learning Geospatial Analysis with Python: Understand GIS Fundamentals and Perform Remote Sensing Data Analysis Using Python 3.7, Packt Publishing Ltd.
50. Drogowe, P. (2024, September 30). Dom. Available online: https://www.pomiarydrogowe.pl/dom.
51. Kłodzko, P. (2024, September 30). Powiat Kłodzko Portal. Available online: https://powiatklodzko.lp-portal.pl.
52. i Kartografii, G.U.G. (2024, September 30). ULDK, Available online: https://uldk.gugik.gov.pl.
53. Geoportal (2024, September 30). View Services: WMS and WMTS, Available online: https://www.geoportal.gov.pl/en/services/view-services-wms-and-wmts/.
54. K.T.G. (2024, September 30). WGS 84/Pseudo-Mercator-Spherical Mercator, Google Maps, OpenStreetMap, Bing, ArcGIS, ESRI-EPSG:3857. Available online: https://epsg.io/3857.
55. K.T.G. (2024, September 30). ETRF2000-PL/CS92-EPSG:2180. Available online: https://epsg.io/2180.