Fast-M Adversarial Training Algorithm for Deep Neural Networks
Source abstract
Abstract
Although deep neural networks have been successfully applied in many fields, research studies show that neural network models are easily disrupted by small malicious inputs, greatly reducing their performance. Such disruptions are known as adversarial attacks. To reduce the impact of adversarial attacks on models, researchers have proposed adversarial training methods. However, compared with standard training, adversarial training results in additional computational overhead and training time. To improve the training effect without significantly increasing the training time, an improved Fast-M adversarial training algorithm based on the fast adversarial training algorithm is proposed in this paper. Extensive comparative experiments are conducted with the MNIST, CIFAR10, and CIFAR100 datasets. The results show that the Fast-M algorithm achieves the same training effect as the commonly used projected gradient descent (PGD) adversarial training method, with a training time that is only one-third of that of PGD and a performance comparable to that of fast adversarial training, demonstrating the proposed algorithm’s effectiveness.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Anomaly Detection Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electronic and Control Engineering, Chang’an University, Xi’an 710018, China
MOE Key Laboratory for Intelligent Networks and Network Security, Xi’an Jiaotong University, Xi’an 710049, China
Faculty of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an 710049, China
Funding
National Natural Science Foundation of China · National Postdoctoral Innovative Talents Support Program of China · Natural Science Basic Research Program of Shaanxi · Fundamental Research Funds for the Central Universities, CHD
References (20)
Show source-provided references
1. Threat of adversarial attacks on deep learning in computer vision: A survey10.1109/ACCESS.2018.2807385 ↗
2. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. (2013). Intriguing properties of neural networks. arXiv.
3. Goodfellow, I.J., Shlens, J., and Szegedy, C. (2014, January 14–16). Explaining and harnessing adversarial examples. Proceedings of the International Conference on Learning Representations, Banff, AB, Canada.
4. Wong, E., Rice, L., and Kolter, J.Z. (2019, January 6–9). Fast is better than free: Revisiting adversarial training. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA.
5. Papernot, N., McDaniel, P., Wu, X., Jha, S., and Swami, A. (2016, January 22–26). Distillation as a defense to adversarial perturbations against deep neural networks. Proceedings of the 2016 IEEE Symposium on Security and Privacy (SP), San Jose, CA, USA.10.1109/SP.2016.41 ↗
6. Metzen, J.H., Genewein, T., Fischer, V., and Bischoff, B. (2017, January 24–26). On detecting adversarial perturbations. Proceedings of the International Conference on Learning Representations, Toulon, France.
7. Feinman, R., Curtin, R.R., Shintre, S., and Gardner, A.B. (2017). Detecting adversarial samples from artifacts. arXiv.
8. Pinot, R., Ettedgui, R., Rizk, G., Chevaleyre, Y., and Atif, J. (2020, January 13–18). Randomization matters how to defend against strong adversarial attacks. Proceedings of the 37th International Conference on Machine Learning, PMLR, Virtual Event.
9. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. (May, January 30). Towards deep learning models resistant to adversarial attacks. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada.
10. Shafahi, A., Najibi, M., Ghiasi, M.A., Xu, Z., Dickerson, J., Studer, C., Davis, L.S., Taylor, G., and Goldstein, T. Adversarial training for free! In Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada, 8–14 December 2019; Volume 32.
11. Smith, L.N., and Topin, N. (2019, January 14–18). Super-convergence: Very fast training of neural networks using large learning rates. Proceedings of the Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, SPIE, Baltimore, MD, USA.10.1117/12.2520589 ↗
12. Micikevicius, P., Narang, S., Alben, J., Diamos, G., Elsen, E., Garcia, D., Ginsburg, B., Houston, M., Kuchaiev, O., and Venkatesh, G. (May, January 30). Mixed precision training. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada.
13. Ye, N., Li, Q., Zhou, X.Y., and Zhu, Z. (2021, January 2–9). Amata: An annealing mechanism for adversarial training acceleration. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada. No. 12.10.1609/aaai.v35i12.17278 ↗
14. Sriramanan, G., Addepalli, S., and Baburaj, A. (2020, January 6–12). Guided adversarial attack for evaluating and enhancing adversarial defenses. Proceedings of the Advances in Neural Information Processing Systems, Virtual.
15. Kurakin, A., Goodfellow, I.J., and Bengio, S. (2018). Adversarial examples in the physical world. Artificial Intelligence Safety and Security, Chapman and Hall/CRC.10.1201/9781351251389-8 ↗
16. Moosavi-Dezfooli, S.M., Fawzi, A., and Frossard, P. (2016, January 27–30). Deepfool: A simple and accurate method to fool deep neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.282 ↗
17. Carlini, N., and Wagner, D. (2017, January 22–24). Towards evaluating the robustness of neural networks. Proceedings of the 2017 IEEE Symposium on Security and Privacy (SP), San Jose, CA, USA.10.1109/SP.2017.49 ↗
18. One pixel attack for fooling deep neural networks10.1109/TEVC.2019.2890858 ↗
19. Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z.B., and Swami, A. Practical black-box attacks against deep learning systems using adversarial examples. arXiv, 2016.10.1145/3052973.3053009 ↗
20. Chen, P.Y., Zhang, H., Sharma, Y., Yi, J., and Hsieh, C.J. (2017, January 3). Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, Dallas, TX, USA.10.1145/3128572.3140448 ↗