Image Segmentation Framework for Detecting Adversarial Attacks for Autonomous Driving Cars
Source abstract
Abstract
The widespread deployment of deep neural networks (DNNs) in critical real-time applications has spurred significant research into their security and robustness. A key vulnerability identified is that DNN decisions can be maliciously altered by introducing carefully crafted noise into the input data, leading to erroneous predictions. This is known as an adversarial attack. In this paper, we propose a novel detection framework leveraging segmentation masks and image segmentation techniques to identify adversarial attacks on DNNs, particularly in the context of autonomous driving systems. Our defense technique considers two levels of adversarial detection. The first level mainly detects adversarial inputs with large perturbations using the U-net model and one-class support vector machine (SVM). The second level of defense proposes a dynamic segmentation algorithm based on the k-means algorithm and a verifier model that controls the final prediction of the input image. To evaluate our approach, we comprehensively compare our method to the state-of-the-art feature squeeze method under a white-box attack, using eleven distinct adversarial attacks across three benchmark and heterogeneous data sets. The experimental results demonstrate the efficacy of our framework, achieving overall detection rates exceeding 96% across all adversarial techniques and data sets studied. It is worth mentioning that our method enhances the detection rates of FGSM and BIM attacks, reaching average detection rates of 95.65% as opposed to 62.63% in feature squeezing across the three data sets.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Adversarial Robustness in Machine Learning · Advanced Malware Detection Techniques · Anomaly Detection Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electrical and Computer Engineering, American University of Beirut, Beirut 1107-2020, Lebanon
Funding
Maroun Semaan Faculty of Engineering and Architecture (MSFEA) at the American University of Beirut (AUB)
References (77)
Show source-provided references
1. A Trustworthy Internet of Vehicles: The DAO to Safe, Secure, and Collaborative Autonomous Driving10.1109/TIV.2023.3337345 ↗
2. Parallel Learning-Based Steering Control for Autonomous Driving10.1109/TIV.2022.3173448 ↗
3. End-to-End Autonomous Driving with Semantic Depth Cloud Mapping and Multi-Agent10.1109/TIV.2022.3185303 ↗
4. Robust Cognitive Capability in Autonomous Driving Using Sensor Fusion Techniques: A Survey10.1109/TITS.2023.3327949 ↗
5. Multi-Sensor Fusion Technology for 3D Object Detection in Autonomous Driving: A Review10.1109/TITS.2023.3317372 ↗
6. Zeng, L., Zhang, K., Han, Q., Chen, S., Ye, L., Wang, R., Lei, J., and Xie, Q. (2019, January 9–12). Research of Path Planning Model Based on Hotspots Evaluation. Proceedings of the 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France.10.1109/IVS.2019.8814163 ↗
7. PEP: Policy-Embedded Trajectory Planning for Autonomous Driving10.1109/LRA.2024.3490377 ↗
8. Poddar, S.D., M, M., P, R., and N, P. (2024, January 1–2). A comprehensive Study on Security Threats in Autonomous Vehicles: Safeguarding the Future. Proceedings of the 2024 1st International Conference on Cognitive, Green and Ubiquitous Computing (IC-CGU), Bhubaneswar, India.10.1109/IC-CGU58078.2024.10530760 ↗
9. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. (2013). Intriguing Properties of Neural Networks. arXiv.
10. Xiang, C., Qi, C.R., and Li, B. (2019, January 15–20). Generating 3D Adversarial Point Clouds. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00935 ↗
11. Sitawarin, C., Bhagoji, A.N., Mosenia, A., Chiang, M., and Mittal, P. (2018). Darts: Deceiving Autonomous Cars with Toxic Signs. arXiv.
12. Morgulis, N., Kreines, A., Mendelowitz, S., and Weisglass, Y. (2019). Fooling a Real Car with Adversarial Traffic Signs. arXiv.
13. Sato, T., Shen, J., Wang, N., Jia, Y.J., Lin, X., and Chen, Q.A. (2021, January 27). Demo: Security of Deep Learning based Automated Lane Centering under Physical-World Attack. Proceedings of the 2021 IEEE Security and Privacy Workshops (SPW), San Francisco, CA, USA.10.1109/SPW53761.2021.00041 ↗
14. Wu, H., Yunas, S., Rowlands, S., Ruan, W., and Wahlström, J. (2023, January 4–7). Adversarial Driving: Attacking End-to-end Autonomous Driving. Proceedings of the 2023 IEEE Intelligent Vehicles Symposium (IV), Anchorage, AK, USA.10.1109/IV55152.2023.10186386 ↗
15. Light can be dangerous: Stealthy and Effective Physical-world Adversarial Attack by Spot Light10.1016/j.cose.2023.103345 ↗
16. Sato, T., Bhupathiraju, S.H., Clifford, M., Sugawara, T., Chen, Q.A., and Rampazzi, S. (2023, January 27). WIP: Infrared Laser Reflection Attack Against Traffic Sign Recognition Systems. Proceedings of the ISOC Symposium on Vehicle Security and Privacy (VehicleSec), San Diego, CA, USA.10.14722/vehiclesec.2023.23055 ↗
17. Yan, C., Xu, Z., Yin, Z., Mangard, S., Ji, X., Xu, W., Zhao, K., Zhou, Y., Wang, T., and Gu, G. (2022, January 10–12). Rolling Colors: Adversarial Laser Exploits against Traffic Light Recognition. Proceedings of the 31st USENIX Security Symposium (USENIX Security 22), Boston, MA, USA.
18. Duan, R., Ma, X., Wang, Y., Bailey, J., Qin, A.K., and Yang, Y. (2020, January 13-19). Adversarial Camouflage: Hiding Physical-World Attacks with Natural Styles. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00108 ↗
19. Wang, W., Yao, Y., Liu, X., Li, X., Hao, P., and Zhu, T. (2021, January 15–19). I Can See The Light: Attacks on Autonomous Vehicles Using Invisible Lights. Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, Virtual.10.1145/3460120.3484766 ↗
20. Sayles, A., Hooda, A., Gupta, M., Chatterjee, R., and Fernandes, E. (2021, January 20–25). Invisible Perturbations: Physical Adversarial Examples Exploiting the Rolling Shutter Effect. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.01443 ↗
21. Zhong, Y., Liu, X., Zhai, D., Jiang, J., and Ji, X. (2022, January 18–24). Shadows can be Dangerous: Stealthy and Effective Physical-world Adversarial Attack by Natural Phenomenon. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01491 ↗
22. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. (2017). Towards Deep Learning Models Resistant to Adversarial Attacks. arXiv.
23. Kurakin, A., Goodfellow, I.J., and Bengio, S. (2018). Adversarial Examples in The Physical World. Artificial Intelligence Safety and Security, Chapman and Hall/CRC.10.1201/9781351251389-8 ↗
24. Samangouei, P., Kabkab, M., and Chellappa, R. (2018). Defense-gan: Protecting Classifiers against Adversarial Attacks Using Generative Models. arXiv.
25. Xu, W., Evans, D., and Qi, Y. (2018, January 18–21). Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks. Proceedings of the 2018 Network and Distributed Systems Security Symposium (NDSS), San Diego, CA, USA.10.14722/ndss.2018.23198 ↗
26. Lin, W.A., Balaji, Y., Samangouei, P., and Chellappa, R. (2019). Invert and Defend: Model-based Approximate Inversion of Generative Adversarial Networks for Secure Inference. arXiv.
27. Goodfellow, I.J., Shlens, J., and Szegedy, C. (2014). Explaining and Harnessing Adversarial Examples. arXiv.
28. Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z.B., and Swami, A. (2016, January 21–24). The Limitations of Deep Learning in Adversarial Settings. Proceedings of the 2016 IEEE European Symposium on Security and Privacy (EuroS&P), Saarbruecken, Germany.10.1109/EuroSP.2016.36 ↗
29. Visualising Image Classification Models and Saliency Maps
30. Moosavi-Dezfooli, S.M., Fawzi, A., and Frossard, P. (2016, January 27–30). Deepfool: A Simple and Accurate Method to Fool Deep Neural Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.282 ↗
31. Carlini, N., and Wagner, D. (2017, January 22–26). Towards Evaluating The Robustness of Neural Networks. Proceedings of the 2017 IEEE Symposium on Security and Privacy (sp), San Jose, CA, USA.10.1109/SP.2017.49 ↗
32. Kingma, D.P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv.
33. Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling The Knowledge In a Neural Network. arXiv.
34. Papernot, N., McDaniel, P., Wu, X., Jha, S., and Swami, A. (2016, January 22–26). Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks. Proceedings of the 2016 IEEE Symposium on Security and Privacy (SP), San Jose, CA, USA.10.1109/SP.2016.41 ↗
35. Meng, D., and Chen, H. (November, January 30). Magnet: A Two-pronged Defense against Adversarial Examples. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, Dallas, TX, USA.10.1145/3133956.3134057 ↗
36. Vincent, P., Larochelle, H., Bengio, Y., and Manzagol, P.A. (2008, January 5–9). Extracting and Composing Robust Features with Denoising Autoencoders. Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland.10.1145/1390156.1390294 ↗
37. Robey, A., Latorre, F., Pappas, G.J., Hassani, H., and Cevher, V. (2023). Adversarial Training Should Be Cast as a Non-Zero-Sum Game. arXiv.
38. Generative Adversarial Nets
39. Arjovsky, M., Chintala, S., and Bottou, L. (2017, January 6–11). Wasserstein Generative Adversarial Networks. Proceedings of the International Conference on Machine Learning, Sydney, Australia.
40. Non-Local Means Denoising10.5201/ipol.2011.bcm_nlm ↗
41. A Novel Lane-Change Decision-Making with Long-Time Trajectory Prediction for Autonomous Vehicle10.1109/ACCESS.2023.3337046 ↗
42. A Multimodality Fusion Deep Neural Network and Safety Test Strategy for Intelligent Vehicles10.1109/TIV.2020.3027319 ↗
43. Athalye, A., Engstrom, L., Ilyas, A., and Kwok, K. (2018, January 10–15). Synthesizing Robust Adversarial Examples. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden.
44. Jia, W., Lu, Z., Zhang, H., Liu, Z., Wang, J., and Qu, G. (2022). Fooling The Eyes of Autonomous Vehicles: Robust Physical Adversarial Examples against Traffic Sign Recognition Systems. arXiv.10.14722/ndss.2022.24130 ↗
45. Kurakin, A., Goodfellow, I., and Bengio, S. (2016). Adversarial Machine Learning at Scale. arXiv.
46. Xiao, C., Li, B., Zhu, J.Y., He, W., Liu, M., and Song, D. (2018). Generating Adversarial Examples with Adversarial Networks. arXiv.10.24963/ijcai.2018/543 ↗
47. Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., and Song, D. (2018, January 18–23). Robust Physical-world Attacks on Deep Learning Visual Classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00175 ↗
48. Lu, J., Sibai, H., Fabry, E., and Forsyth, D. (2017). No Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles. arXiv.
49. Kong, Z., Guo, J., Li, A., and Liu, C. (2020, January 13–19). Physgan: Generating Physical-world-resilient Adversarial Examples for Autonomous Driving. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01426 ↗
50. Udacity (2024, December 22). Udacity Steering Wheel Dataset. Available online: https://www.udacity.com/.
51. Pan, X., You, Y., Wang, Z., and Lu, C. (2017). Virtual to Real Reinforcement Learning for Autonomous Driving. arXiv.10.5244/C.31.11 ↗
52. Vision Meets Robotics: The Kitti Dataset10.1177/0278364913491297 ↗
53. Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L.D., Monfort, M., Muller, U., and Zhang, J. (2016). End to End Learning for Self-driving Cars. arXiv.
54. Shah, S., Dey, D., Lovett, C., and Kapoor, A. (2017, January 12–15). Airsim: High-fidelity Visual and Physical Simulation for Autonomous Vehicles. Proceedings of the Field and Service Robotics: Results of the 11th International Conference, Zurich, Switzerland.10.1007/978-3-319-67361-5_40 ↗
55. Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017, January 13–15). CARLA: An Open Urban Driving Simulator. Proceedings of the Conference on Robot Learning, Mountain View, CA, USA.
56. Robot Operating System 2: Design, Architecture, and Uses in The Wild10.1126/scirobotics.abm6074 ↗
57. Cao, Y., Xiao, C., Yang, D., Fang, J., Yang, R., Liu, M., and Li, B. (2019). Adversarial Objects against Lidar-based Autonomous Driving Systems. arXiv.
58. Remote Attacks on Automated Vehicles Sensors: Experiments on Camera and Lidar
59. Shin, H., Kim, D., Kwon, Y., and Kim, Y. (2017, January 25–28). Illusion and Dazzle: Adversarial Optical Channel Exploits against Lidars for Automotive Applications. Proceedings of the Cryptographic Hardware and Embedded Systems—CHES 2017: 19th International Conference, Taipei, Taiwan.10.1007/978-3-319-66787-4_22 ↗
60. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015: 18th International Conference, Munich, Germany.10.1007/978-3-319-24574-4_28 ↗
61. One-class Support Vector Machines—An Application in Machine Fault Detection and Classification10.1016/j.cie.2005.01.009 ↗
62. Wang, C., Sun, Y., Lv, S., Wang, C., Liu, H., and Wang, B. (2023). Intrusion Detection System Based on One-class Support Vector Machine and Gaussian Mixture Model. Electronics, 12.10.3390/electronics12040930 ↗
63. Bishop, C.M., and Nasrabadi, N.M. (2006). Pattern Recognition and Machine Learning, Springer.
64. Akbari, M., Liang, J., and Han, J. (2019, January 12–17). DSSLIC: Deep Semantic Segmentation-based Layered Image Compression. Proceedings of the ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.10.1109/ICASSP.2019.8683541 ↗
65. Li, F., Zhang, H., Xu, H., Liu, S., Zhang, L., Ni, L.M., and Shum, H.Y. (2023, January 22–23). Mask dino: Towards a Unified Transformer-based Framework for Object Detection and Segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00297 ↗
66. Image Segmentation Using K-means Clustering Algorithm and Subtractive Clustering Algorithm10.1016/j.procs.2015.06.090 ↗
67. LeCun, Y. (2024, December 22). The MNIST Database of Handwritten Digits. Available online: http://yann.lecun.com/exdb/mnist/.
68. Krizhevsky, A., and Hinton, G. (2009). Learning Multiple Layers of Features from Tiny Images, University of Toronto.
69. Stallkamp, J., Schlipsing, M., Salmen, J., and Igel, C. (August, January 31). The German Traffic Sign Recognition Benchmark: A Multi-class Classification Competition. Proceedings of the The 2011 International Joint Conference on Neural Networks, San Jose, CA, USA.10.1109/IJCNN.2011.6033395 ↗
70. Papernot, N., McDaniel, P., and Goodfellow, I. (2016). Transferability in Machine Learning: From Phenomena to Black-box Attacks Using Adversarial Samples. arXiv.
71. CVAT (2024, August 05). Online Annotation Tool. Available online: https://www.cvat.ai/.
72. Buslaev, A., Iglovikov, V.I., Khvedchenya, E., Parinov, A., Druzhinin, M., and Kalinin, A.A. (2018). Albumentations: Fast and Flexible Image Augmentations. arXiv.
73. Segmentation (2024, August 07). Cat2 Dataset. Available online: https://universe.roboflow.com/segmentation-w6eax/cat2-6nv4t.
74. (2025, January 09). Recognition of Sheep Dataset. Available online: https://universe.roboflow.com/recognition-of-sheep/recognition-of-sheep.
75. Carlini, N. (2024, August 16). Robust Evasion Attacks against Neural Network to Find Adversarial Examples. Available online: https://github.com/carlini/nn_robust_attacks.
76. Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21–26). Densely Connected Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
77. Majumdar, S. (2024, August 16). DenseNet Implementation in Keras. Available online: https://github.com/titu1994/DenseNet.