Boosting Adversarial Training with Learnable Distribution
Source abstract
Abstract
In recent years, various adversarial defense methods have been proposed to improve the robustness of deep neural networks. Adversarial training is one of the most potent methods to defend against adversarial attacks. However, the difference in the feature space between natural and adversarial examples hinders the accuracy and robustness of the model in adversarial training. This paper proposes a learnable distribution adversarial training method, aiming to construct the same distribution for training data utilizing the Gaussian mixture model. The distribution centroid is built to classify samples and constrain the distribution of the sample features. The natural and adversarial examples are pushed to the same distribution centroid to improve the accuracy and robustness of the model. The proposed method generates adversarial examples to close the distribution gap between the natural and adversarial examples through an attack algorithm explicitly designed for adversarial training. This algorithm gradually increases the accuracy and robustness of the model by scaling perturbation. Finally, the proposed method outputs the predicted labels and the distance between the sample and the distribution centroid. The distribution characteristics of the samples can be utilized to detect adversarial cases that can potentially evade the model defense. The effectiveness of the proposed method is demonstrated through comprehensive experiments.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Adversarial Robustness in Machine Learning · Advanced Neural Network Applications · Anomaly Detection Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (38)
Show source-provided references
1. Deep learning-based action classification using one-shot object detection
2. Anomalous situations recognition in surveillance images using deep learning
3. Deep learning enabled semantic communications with speech recognition and synthesis10.1109/TWC.2023.3240969 ↗
4. DeepOCL: A deep natural network for object constraint language generation from unrestricted nature language
5. J. K. Devlin, M. W. Chang, and L. K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
6. I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” arXiv preprint arXiv:1412.6572, 2014.
7. Universal adversarial attacks on deep neural networks for medical image classification10.1186/s12880-020-00530-y ↗
8. WilDect-YOLO: An efficient and robust computer vision-based accurate object localization model for automated endangered wildlife detection10.1016/j.ecoinf.2022.101919 ↗
9. MTSDet: Multi-scale traffic sign detection with attention and path aggregation10.1007/s10489-022-03459-7 ↗
10. Light projection-based physical-world vanishing attack against car detection
11. ESGAN for generating high-quality enhanced samples10.1007/s00530-022-00953-3 ↗
12. Adversarial example detection by predicting adversarial noise in the frequency domain10.1007/s11042-023-14608-6 ↗
13. Deep image prior-based defense against adversarial examples10.1016/j.patcog.2021.108249 ↗
14. A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” arXiv preprint arXiv:1706.06083, 2017.
15. Adversarial training for free!
16. F. Tramèr, A. Kurakin, N. Papernot, I. Goodfellow, D. Boneh and P. McDaniel, “Ensemble adversarial training: Attacks and defenses,” arXiv preprint arXiv:1705.07204, 2017.
17. Entropy minimization and domain adversarial training guided by label distribution similarity for domain adaptation10.1007/s00530-023-01106-w ↗
18. Theoretically principled trade-off between robustness and accuracy
19. H. Kannan, A. Kurakin, and I. Goodfellow, “Adversarial logit pairing,” arXiv preprint arXiv:1803.06373, 2018.
20. C. Xie and A. Yuille, “Intriguing properties of adversarial training at scale,” arXiv preprint arXiv:1906.03787, 2019.
21. Adversarial examples improve image recognition
22. Towards evaluating the robustness of neural networks
23. Q. Z. Cai, M. Du, C. Liu, and D. Song, “Curriculum adversarial training,” arXiv preprint arXiv:1805.04807, 2018.
24. Attacks which do not kill training make adversarial learning stronger
25. Y. Wang, X. Ma, J. Bailey, J. Yi, B. Zhou and Q. Gu, “On the convergence and robustness of adversarial training,” arXiv preprint arXiv:2112.08304, 2021.
26. A. Kurakin, I. Goodfellow, and S. Bengio, “Adversarial machine learning at scale,” arXiv preprint arXiv:1611.01236, 2016.
27. Learning multiple layers of features from tiny images
28. Identity mappings in deep residual networks
29. S. Zagoruyko and N. Komodakis, “Wide residual networks,” arXiv preprint arXiv:1605.07146, 2016.10.5244/C.30.87 ↗
30. Improving adversarial robustness requires revisiting misclassified examples
31. Learnable boundary guided adversarial training
32. J. Rauber, W. Brendel, and M. Bethge, “Foolbox: A python toolbox to benchmark the robustness of machine learning models,” arXiv preprint arXiv:1707.04131, 2017.
33. H. Kim, “Torchattacks: A pytorch repository for adversarial attacks,” arXiv preprint arXiv:2010.01950, 2020.
34. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
35. Minimally distorted adversarial examples with a fast adaptive boundary attack
36. Deepfool: A simple and accurate method to fool deep neural networks
37. Decoupling direction and norm for efficient gradient-based L2 adversarial attacks and defenses
38. Boosting adversarial attacks with momentum