journal article·2024-03-01Open access

A Practical Deep Learning Architecture for Large-Area Solid Wastes Monitoring Based on UAV Imagery

Yang Liu, Bo Zhao, Xuepeng Zhang, Wei Nie, Peng Gou, Jiachun Liao, Kunxin Wang

Applied Sciences· Volume 14· Issue 5

Abstract

The development of global urbanization has brought about a significant amount of solid waste. These untreated wastes may be dumped in any corner, causing serious pollution to the environment. Thus, it is necessary to accurately obtain their distribution locations and detailed edge information. In this study, a practical deep learning network for recognizing solid waste piles over extensive areas using unmanned aerial vehicle (UAV) imagery has been proposed and verified. Firstly, a high-resolution dataset serving to solid waste detection was created based on UAV aerial data. Then, a dual-branch solid waste semantic segmentation model was constructed to address the characteristics of the integration of solid waste distribution with the environment and the irregular edge morphology. The Context feature branch is responsible for extracting high-level semantic features, while the Spatial feature branch is designed to capture fine-grained spatial details. After information fusion, the model obtained more comprehensive feature representation and segmentation ability. The effectiveness of the improvement was verified through ablation experiments and compared with 13 commonly used semantic segmentation models, demonstrating the advantages of the method in solid waste segmentation tasks, with an overall accuracy of over 94%, and a recall rate of 88.6%—much better than the best performing baselines. Finally, a spatial distribution map of solid waste over Jiaxing district, China was generated by the model inference, which assisted the environmental protection department in completing environmental management. The proposed method provides a feasible approach for the accurately monitoring of solid waste, so as to provide policy support for environmental protection.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Advanced Neural Network Applications · Advanced Image Fusion Techniques · Remote-Sensing Image Classification

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Research Center of Big Data Technology, Nanhu Laboratory, Jiaxing 314000, China

Funding

Science and Technology Program of Zhejiang Province

References (55)

Show source-provided references
1. Current status of solid waste management in small island developing states: A review10.1016/j.wasman.2015.06.012 ↗
2. Assessing the variables affecting on the rate of solid waste generation and recycling: An empirical analysis in Prespa Park10.1016/j.wasman.2015.09.028 ↗
3. Incekara, A.H., Delen, A., Seker, D.Z., and Goksel, C. (2019). Investigating the Utility Potential of Low-Cost Unmanned Aerial Vehicles in the Temporal Monitoring of a Landfill. ISPRS Int. J. Geo-Inf., 8.10.3390/ijgi8010022 ↗
4. Top-down approach from satellite to terrestrial rover application for environmental monitoring of landfills10.1016/j.scitotenv.2017.01.033 ↗
5. Solid Waste Detection in Cities Using Remote Sensing Imagery Based on a Location-Guided Key Point Network With Multiple Enhancements10.1109/JSTARS.2022.3224555 ↗
6. Identifying sustainable solid waste management barriers in practice using the fuzzy Delphi method10.1016/j.resconrec.2019.104625 ↗
7. Effect of glass content on sintering kinetics, microstructure and mechanical properties of glass-ceramics from coal fly ash and waste glass10.1016/j.matchemphys.2020.124120 ↗
8. Malche, T., Maheshwary, P., Tiwari, P.K., Alkhayyat, A.H., Bansal, A., and Kumar, R. (2023). Efficient solid waste inspection through drone-based aerial imagery and TinyML vision model. Trans. Emerg. Telecommun. Technol., e4878.10.1002/ett.4878 ↗
9. Padubidri, C., Kamilaris, A., and Karatsiolis, S. (2022, January 21–25). Accurate Detection of Illegal Dumping Sites Using High Resolution Aerial Photography and Deep Learning. Proceedings of the 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), Pisa, Italy.10.1109/PerComWorkshops53856.2022.9767451 ↗
10. A video processing and machine vision-based automatic analyzer to determine sequentially total suspended and settleable solids in wastewater10.1016/j.aca.2021.339411 ↗
11. Detection of waste dumping locations in landfill using multi-temporal Landsat thermal images10.1177/0734242X18821808 ↗
12. Begur, H., Dhawade, M., Gaur, N., Dureja, P., Gao, J., Mahmoud, M., Huang, J., Chen, S., and Ding, X. (2017, January 4–8). An edge-based smart mobile service system for illegal dumping detection and monitoring in San Jose. Proceedings of the 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), San Francisco, CA, USA.10.1109/UIC-ATC.2017.8397575 ↗
13. Rapid and robust monitoring of flood events using Sentinel-1 and Landsat data on the Google Earth Engine10.1016/j.rse.2020.111664 ↗
14. Enegbuma, W.I., Bamgbade, J.A., Ming, C.P.H., Ohueri, C.C., Tanko, B.L., Ojoko, E.O., Dodo, Y.A., and Kori, S.A. (2020). Handbook of Research on Resource Management for Pollution and Waste Treatment, IGI Global.
15. A review on deep learning in UAV remote sensing
16. Manfreda, S., McCabe, M.F., Miller, P.E., Lucas, R., Pajuelo Madrigal, V., Mallinis, G., Ben Dor, E., Helman, D., Estes, L., and Ciraolo, G. (2018). On the Use of Unmanned Aerial Systems for Environmental Monitoring. Remote Sens., 10.10.20944/preprints201803.0097.v1 ↗
17. Tmušić, G., Manfreda, S., Aasen, H., James, M.R., Gonçalves, G., Ben-Dor, E., Brook, A., Polinova, M., Arranz, J.J., and Mészáros, J. (2020). Current Practices in UAS-based Environmental Monitoring. Remote Sens., 12.10.3390/rs12061001 ↗
18. A transdisciplinary review of deep learning research and its relevance for water resources scientists10.1029/2018WR022643 ↗
19. Mapping marine litter using UAS on a beach-dune system: A multidisciplinary approach10.1016/j.scitotenv.2019.135742 ↗
20. Automatic detection and classification system of domestic waste via multimodel cascaded convolutional neural network10.1109/TII.2021.3085669 ↗
21. An internet of things based smart waste management system using LoRa and tensorflow deep learning model10.1109/ACCESS.2020.3016255 ↗
22. A deep learning approach based hardware solution to categorise garbage in environment10.1007/s40747-021-00529-0 ↗
23. A Survey on Waste Detection and Classification Using Deep Learning10.1109/ACCESS.2022.3226682 ↗
24. Deep learning-based waste detection in natural and urban environments10.1016/j.wasman.2021.12.001 ↗
25. Vision-based garbage dumping action detection for real-world surveillance platform10.4218/etrij.2018-0520 ↗
26. A modified YOLOv3 detection method for vision-based water surface garbage capture robot10.1177/1729881420932715 ↗
27. Multilayer hybrid deep-learning method for waste classification and recycling10.1155/2018/5060857 ↗
28. Shahab, S., and Anjum, M. (2022). Solid waste management scenario in india and illegal dump detection using deep learning: An AI approach towards the sustainable waste management. Sustainability, 14.10.3390/su142315896 ↗
29. Shi, C., Tan, C., Wang, T., and Wang, L. (2021). A waste classification method based on a multilayer hybrid convolution neural network. Appl. Sci., 11.10.3390/app11188572 ↗
30. Fasana, C., and Pasini, S. (2022, December 20). Learning to Detect Illegal Landfills in Aerial Images with Scarce Labeling Data. Available online: https://hdl.handle.net/10589/196992.
31. AerialWaste dataset for landfill discovery in aerial and satellite images10.1038/s41597-023-01976-9 ↗
32. Yu, X., Chen, Z., Zhang, S., and Zhang, T. (2018, January 19). A street rubbish detection algorithm based on Sift and RCNN. Proceedings of the MIPPR 2017: Automatic Target Recognition and Navigation, Xiangyang, China.10.1117/12.2285043 ↗
33. Solid waste mapping based on very high resolution remote sensing imagery and a novel deep learning approach10.1080/10106049.2022.2164361 ↗
34. Intelligent solid waste classification using deep convolutional neural networks10.1007/s13762-021-03179-4 ↗
35. Municipal solid waste classification and real-time detection using deep learning methods10.1016/j.uclim.2023.101462 ↗
36. Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (2016, January 27–30). The cityscapes dataset for semantic urban scene understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.350 ↗
37. Semantic object classes in video: A high-definition ground truth database10.1016/j.patrec.2008.04.005 ↗
38. Neuhold, G., Ollmann, T., Rota Bulo, S., and Kontschieder, P. (2017, January 22–29). The mapillary vistas dataset for semantic understanding of street scenes. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.534 ↗
39. A review of the impacts of defogging on deep learning-based object detectors in self-driving cars
40. He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
41. Adaptive deformable convolutional network10.1016/j.neucom.2020.06.128 ↗
42. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L. (2018, January 18–22). Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗
43. Bisenet v2: Bilateral network with guided aggregation for real-time semantic segmentation10.1007/s11263-021-01515-2 ↗
44. Qi, J., Nguyen, M., and Yan, W. (2022). Image and Video Technology: Proceedings of the 10th Pacific-Rim Symposium, PSIVT 2022, Virtual Event, 12–14 November 2022, Springer International Publishing.
45. Image segmentation using deep learning: A survey
46. Hong, Y., Pan, H., Sun, W., and Jia, Y. (2021). Deep dual-resolution networks for real-time and accurate semantic segmentation of road scenes. arXiv.
47. Encoder-decoder with atrous separable convolution for semantic image segmentation10.1007/978-3-030-01234-2_49 ↗
48. U-net: Convolutional networks for biomedical image segmentation
49. Chao, P., Kao, C.-Y., Ruan, Y.-S., Huang, C.-H., and Lin, Y. (November, January 27). Hardnet: A low memory traffic network. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
50. Peng, C., Zhang, X., Yu, G., Luo, G., and Sun, J. (2017, January 21–26). Large kernel matters–improve semantic segmentation by global convolutional network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.189 ↗
51. UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation10.1109/TMI.2019.2959609 ↗
52. Zhu, X., Cheng, Z., and Wang, S. (2021). Coronary angiography image segmentation based on PSPNet. Comput. Methods Programs Biomed., 200.10.1016/j.cmpb.2020.105897 ↗
53. Lee, J., Kim, D., Ponce, J., and Ham, B. (2019, January 15–20). Sfnet: Learning object-aware semantic correspondence. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00238 ↗
54. SegFormer: Simple and efficient design for semantic segmentation with transformers
55. Aleissaee, A., Kumar, A., Anwer, R., Khan, S., and Cholakkal, H. (2023). Transformers in remote sensing: A survey. Remote Sens., 15.10.3390/rs15071860 ↗