Deep Learning Semantic Segmentation for Water Level Estimation Using Surveillance Camera
Source abstract
Abstract
The interest in visual-based surveillance systems, especially in natural disaster applications, such as flood detection and monitoring, has increased due to the blooming of surveillance technology. In this work, semantic segmentation based on convolutional neural networks (CNN) was proposed to identify water regions from the surveillance images. This work presented two well-established deep learning algorithms, DeepLabv3+ and SegNet networks, and evaluated their performances using several evaluation metrics. Overall, both networks attained high accuracy when compared to the measurement data but the DeepLabv3+ network performed better than the SegNet network, achieving over 90% for overall accuracy and IoU metrics, and around 80% for boundary F1 score (BF score), respectively. When predicting new images using both trained networks, the results show that both networks successfully distinguished water regions from the background but the outputs from DeepLabv3+ were more accurate than the results from the SegNet network. Therefore, the DeepLabv3+ network was used for practical application using a set of images captured at five consecutive days in the study area. The segmentation result and water level markers extracted from light detection and ranging (LiDAR) data were overlaid to estimate river water levels and observe the water fluctuation. River water levels were predicted based on the elevation from the predefined markers. The proposed water level framework was evaluated according to Spearman’s rank-order correlation coefficient. The correlation coefficient was 0.91, which indicates a strong relationship between the estimated water level and observed water level. Based on these findings, it can be concluded that the proposed approach has high potential as an alternative monitoring system that offers water region information and water level estimation for flood management and related activities.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Flood Risk Assessment and Management · Image Enhancement Techniques · Water Quality Monitoring Technologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Biological and Agricultural Engineering, Faculty of Engineering, Universiti Putra Malaysia, Serdang 43400, Malaysia
Institute of Aquaculture and Aquatic Sciences, Batu 7, Jalan Kemang 6, Teluk Kemang, Port Dickson 71050, Malaysia
Department of Civil and Environmental Engineering, South Kensington Campus, Imperial College London, London SW7 2AZ, UK
Funding
Universiti Putra Malaysia
References (81)
Show source-provided references
1. Experience of extreme weather affects climate change mitigation and adaptation responses10.1007/s10584-016-1837-4 ↗
2. Extreme weather and floods in Kelantan state, Malaysia in December 2014
3. Disaster risks from economic perspective: Malaysian scenario10.1088/1755-1315/244/1/012009 ↗
4. State of the Climate in 2020
5. UNDRR (2020). The Human Cost of Disasters—An Overview of the Last 20 Years 2000–2019, UNDRR.
6. Integration of SAR-derived river inundation areas, high-precision topographic data and a river flow model toward near real-time flood management
7. Near real-time flood detection in urban and rural areas using high-resolution synthetic aperture radar images10.1109/TGRS.2011.2178030 ↗
8. Flood Hazard and Flood Risk Assessment Using a Time Series of Satellite Images: A Case Study in Namibia10.1111/risa.12156 ↗
9. Satellite-supported flood forecasting in river networks: A real case study10.1016/j.jhydrol.2015.01.084 ↗
10. GIS A near real-time flood-mapping approach by integrating social media and post-event satellite imagery10.1080/19475683.2018.1450787 ↗
11. DID (2009). DID Manual (Volume 1—Flood Management), Jabatan Pengairan dan Saliran Malaysia.
12. A real-time measurement system for long-life flood monitoring and warning applications10.3390/s120404213 ↗
13. Microwave-Photonic Sensor for Remote Water-Level Monitoring Based on Chaotic Laser10.1142/S0218127414500321 ↗
14. Cyber surveillance for flood disasters10.3390/s150202369 ↗
15. Kang, W., Xiang, Y., Wang, F., Wan, L., and You, H. (2018). Flood detection in gaofen-3 SAR images via fully convolutional networks. Sensors, 18.10.3390/s18092915 ↗
16. Yang, C., Wei, Y., Wang, S., Zhang, Z., and Huang, S. (November, January 29). Extracting the flood extent from satellite SAR image with the support of topographic data. Proceedings of the 2001 International Conferences on Info-Tech and Info-Net: A Key to Better Life, ICII 2001—Proceedings, Beijing, China.
17. Cao, H., Zhang, H., Wang, C., and Zhang, B. (2019). Operational flood detection using Sentinel-1 SAR data over large areas. Water, 11.10.3390/w11040786 ↗
18. Calibration of two-dimensional floodplain modeling in the central Atchafalaya Basin Floodway System using SAR interferometry10.1029/2012WR011951 ↗
19. The accuracy of sequential aerial photography and SAR data for observing urban flood dynamics, a case study of the UK summer 2007 floods10.1016/j.rse.2011.04.039 ↗
20. Urban flood mapping based on unmanned aerial vehicle remote sensing and random forest classifier-A case of yuyao, China10.3390/w7041437 ↗
21. Technical note: Advances in flash flood monitoring using unmanned aerial vehicles (UAVs)10.5194/hess-20-4005-2016 ↗
22. Langhammer, J., Bernsteinová, J., and Miřijovský, J. (2017). Building a high-precision 2D hydrodynamic flood model using UAV photogrammetry and sensor network monitoring. Water, 9.10.3390/w9110861 ↗
23. UAV monitoring and documentation of a large landslide10.1007/s12518-015-0165-0 ↗
24. Thermal infrared imaging of geothermal environments and by an unmanned aerial vehicle (UAV): A case study of the Wairakei—Tauhara geothermal field, Taupo, New Zealand10.1016/j.renene.2015.09.042 ↗
25. Assesment of the influence of UAV image quality on the orthophoto production10.5194/isprsarchives-XL-1-W4-1-2015 ↗
26. Muhadi, N.A., Abdullah, A.F., Bejo, S.K., Mahadi, M.R., and Mijic, A. (2020). The Use of LiDAR-Derived DEM in Flood Applications: A Review. Remote Sens., 12.10.3390/rs12142308 ↗
27. Filonenko, A., Hernandez, D.C., Seo, D., and Jo, K.H. (2015). Real-time flood detection for video surveillance. Proceedings of the IECON 2015—41st Annual Conference of the IEEE Industrial Electronics Society, Yokohama, Japan, 9–12 November 2015, IEEE.10.1109/IECON.2015.7392736 ↗
28. Muhadi, N.A., Abdullah, A.F., Bejo, S.K., Mahadi, M.R., and Mijic, A. (2020). Image Segmentation Methods for Flood Monitoring System. Water, 12.10.3390/w12061825 ↗
29. Lopez-Fuentes, L., Rossi, C., and Skinnemoen, H. (2017). River segmentation for flood monitoring. Proceedings of the 2017 IEEE International Conference on Big Data (Big Data), Boston, MA, USA, 11–14 December 2017, IEEE.10.1109/BigData.2017.8258373 ↗
30. Measurement of the flood discharge of a small-sized river using an existing digital video recording system10.1016/j.jher.2010.12.004 ↗
31. Use of LSPIV in assessing urban flash flood vulnerability10.1007/s11069-017-2768-8 ↗
32. Adaptive monitoring relevance in camera networks for critical surveillance applications10.1155/2013/836721 ↗
33. Arshad, B., Ogie, R., Barthelemy, J., Pradhan, B., Verstaevel, N., and Perez, P. (2019). Computer vision and iot-based sensors in flood monitoring and mapping: A systematic review. Sensors, 19.10.3390/s19225012 ↗
34. Water-Body Area Extraction From High Resolution Satellite Images—An Introduction, Review, and Comparison
35. Geetha, M., Manoj, M., Sarika, A.S., Mohan, M., and Rao, S.N. (2017). Detection and estimation of the extent of flood from crowd sourced images. Proceedings of the 2017 International Conference on Communication and Signal Processing (ICCSP), Melmaruvathur, India, 6–8 April 2017, IEEE.10.1109/ICCSP.2017.8286429 ↗
36. Jyh-Horng, W., Chien-Hao, T., Lun-Chi, C., Shi-Wei, L., and Fang-Pang, L. (2015). Automated Image Identification Method for Flood Disaster Monitoring In Riverine Environments: A Case Study in Taiwan. Proceedings of the AASRI International Conference on Industrial Electronics and Applications (IEA 2015), London, UK, 27–28 June 2015, Atlantis Press.10.2991/iea-15.2015.65 ↗
37. Visual sensing for urban flood monitoring10.3390/s150820006 ↗
38. Buscombe, D., and Ritchie, A.C. (2018). Landscape classification with deep neural networks. Geosciences, 8.10.3390/geosciences8070244 ↗
39. Spectral–spatial multi-feature-based deep learning for hyperspectral remote sensing image classification10.1007/s00500-016-2246-3 ↗
40. Deep learning in environmental remote sensing: Achievements and challenges10.1016/j.rse.2020.111716 ↗
41. Tapu, R., Mocanu, B., and Zaharia, T. (2017). DEEP-SEE: Joint object detection, tracking and recognition with application to visually impaired navigational assistance. Sensors, 17.10.3390/s17112473 ↗
42. Feng, J., Wei, Y., Tao, L., Zhang, C., and Sun, J. (2011, January 6–13). Salient object detection by composition. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.
43. Automated defect detection tool for closed circuit television (cctv) inspected sewer pipelines10.1016/j.autcon.2018.01.004 ↗
44. Varfolomeev, I., Yakimchuk, I., and Safonov, I. (2019). An application of deep neural networks for segmentation of microtomographic images of rock samples. Computers, 8.10.3390/computers8040072 ↗
45. Waterline and obstacle detection in images from low-cost autonomous boats for environmental monitoring10.1016/j.robot.2019.103346 ↗
46. Land-cover classification with high-resolution remote sensing images using transferable deep models10.1016/j.rse.2019.111322 ↗
47. Deep learning on edge: Extracting field boundaries from satellite images with a convolutional neural network10.1016/j.rse.2020.111741 ↗
48. Bischke, B., Bhardwaj, P., Gautam, A., Helber, P., Borth, D., and Dengel, A. (2017, January 13–15). Detection of Flooding Events in Social Multimedia and Satellite Imagery using Deep Neural Networks. Proceedings of the MediaEval, Dublin, Ireland.
49. Convolutional Neural Networks for Water Body Extraction from Landsat Imagery10.1142/S1469026817500018 ↗
50. Amit, S.N.K.B., and Aoki, Y. (2017, January 26–27). Disaster detection from aerial imagery with convolutional neural network. Proceedings of the Proceedings—International Electronics Symposium on Knowledge Creation and Intelligent Computing, IES-KCIC 2017, Surabaya, Indonesia.10.1109/KCIC.2017.8228593 ↗
51. Surface water mapping by deep learning10.1109/JSTARS.2017.2735443 ↗
52. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs10.1109/TPAMI.2017.2699184 ↗
53. Water Body Extraction from Very High-Resolution Remote Sensing Imagery Using Deep U-Net and a Superpixel-Based Conditional Random Field Model10.1109/LGRS.2018.2879492 ↗
54. Urban flood mapping with an active self-learning convolutional neural network based on TerraSAR-X intensity and interferometric coherence10.1016/j.isprsjprs.2019.04.014 ↗
55. Scalable Flood Level Trend Monitoring with Surveillance Cameras using a Deep Convolutional Neural Network10.5194/hess-23-4621-2019 ↗
56. Deep Learning Applied to Water Segmentation10.5194/isprs-archives-XLIII-B2-2020-1189-2020 ↗
57. Deep learning for the estimation of water-levels using river cameras10.5194/hess-25-4435-2021 ↗
58. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
59. Chen, L.C., Papandreou, G., Schroff, F., and Adam, H. (2017). Rethinking atrous convolution for semantic image segmentation. arXiv.
60. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation10.1109/TPAMI.2016.2644615 ↗
61. Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8–14). Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. Proceedings of the Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_49 ↗
62. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L. (2014). Semantic image segmentation with deep convolutional nets and fully connected CRFs. arXiv.
63. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
64. Feurer, M., and Hutter, F. (2020). Hyperparameter Optimization. Automated Machine Learning, Springer Nature.10.1007/978-3-030-05318-5_1 ↗
65. ImageNet Classification with Deep Convolutional Neural Networks Alex
66. Csurka, G., Larlus, D., and Perronnin, F. (2013, January 9–13). What is a good evaluation measure for semantic segmentation?. Proceedings of the BMVC 2013—British Machine Vision Conference 2013, Bristol, UK.10.5244/C.27.32 ↗
67. Apple Tree Tree Trunk Trunk and Branch Segmentation Segmentation for for Automatic Automatic Trellis Trellis Training Training Apple Tree Trunk and Segmentation for Automatic Trellis Training Using Convolutional Neural Network Based Semantic Segmentation10.1016/j.ifacol.2018.08.064 ↗
68. Learning to detect natural image boundaries using local brightness, color, and texture cues10.1109/TPAMI.2004.1273918 ↗
69. The Proof and Measurement of Association between Two Things10.2307/1412159 ↗
70. Vandaele, R., Dance, S.L., and Ojha, V. (2021). Automated Water Segmentation and River Level Detection on Camera Images Using Transfer Learning. DAGM German Conference on Pattern Recognition, Springer.10.1007/978-3-030-71278-5_17 ↗
71. Boonpook, W., Tan, Y., Ye, Y., Torteeka, P., Torsri, K., and Dong, S. (2018). A deep learning approach on building detection from unmanned aerial vehicle-based images in riverbank monitoring. Sensors, 18.10.3390/s18113921 ↗
72. Automated brain tumor segmentation on multi-modal MR image using SegNet10.1007/s41095-019-0139-y ↗
73. Torres, D.L., Queiroz Feitosa, R., Nigri Happ, P., Elena Cué La Rosa, L., Marcato Junior, J., Martins, J., Olã Bressan, P., Gonçalves, W.N., and Liesenberg, V. (2020). Applying Fully Convolutional Architectures for Semantic Segmentation of a Single Tree Species in Urban Environment on High Resolution UAV Optical Imagery. Sensors, 20.10.3390/s20020563 ↗
74. Talal, M., Panthakkan, A., Mukhtar, H., Mansoor, W., Almansoori, S., and Ahmad, H. (2018, January 7–8). Al Detection of Water-Bodies Using Semantic Segmentation. Proceedings of the 2018 International Conference on Signal Processing and Information Security (ICSPIS), Dubai, UAE.10.1109/CSPIS.2018.8642743 ↗
75. Semantic scene segmentation in unstructured environment with modified DeepLabV3+10.1016/j.patrec.2020.07.029 ↗
76. Semantic segmentation of endangered tree species in Brazilian savanna using DEEPLABV3+ variants10.5194/isprs-archives-XLII-3-W12-2020-355-2020 ↗
77. Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., and Garcia-Rodriguez, J. (2017). A Review on Deep Learning Techniques Applied to Semantic Segmentation. arXiv.10.1016/j.asoc.2018.05.018 ↗
78. Pashaei, M., Kamangir, H., Starek, M.J., and Tissot, P. (2020). Review and evaluation of deep learning architectures for efficient land cover mapping with UAS hyper-spatial imagery: A case study over a wetland. Remote Sens., 12.10.3390/rs12060959 ↗
79. Multiscale features supported deeplabv3+ optimization scheme for accurate water semantic segmentation10.1109/ACCESS.2019.2949635 ↗
80. Lo, S.W., Wu, J.H., Chen, L.C., Tseng, C.H., Lin, F.P., and Hsu, C.H. (2016). Uncertainty comparison of visual sensing in adverse weather conditions. Sensors, 16.10.3390/s16071125 ↗
81. Hsu, S.-Y.Y., Chen, T.-B.B., Du, W.-C.C., Wu, J.-H.H., and Chen, S.-C.C. (2019). Integrate Weather radar and monitoring devices for urban flooding surveillance. Sensors, 19.10.3390/s19040825 ↗