WordBlitz: An Efficient Hard-Label Textual Adversarial Attack Method Jointly Leveraging Adversarial Transferability and Word Importance
Source abstract
Abstract
Existing textual attacks mostly perturb keywords in sentences to generate adversarial examples by relying on the prediction confidence of victim models. In practice, attackers can only access the prediction label, meaning that the victim model can easily defend against such hard-label attacks by denying access based on the attack’s frequency. In this paper, we propose an efficient hard-label attack approach, called WordBlitz. First, based on the adversarial transferability, we train a substitute model to initialize the attack parameter set, including a candidate pool and two weight tables of keywords and candidate words. Then, adversarial examples are generated and optimized under the guidance of the two weight tables. During optimization, we design a hybrid local search algorithm with word importance to find the globally optimal solution while updating the two weight tables according to the attack results. Finally, the non-adversarial text generated during perturbation optimization is added to the training of the substitute model as data augmentation to improve the adversarial transferability. Experimental results show that WordBlitz surpasses the baseline in terms of better effectiveness, higher efficiency, and lower cost. Its efficiency is especially pronounced in scenarios with broader search spaces, and its attack success rate on a Chinese dataset is higher than on baselines.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Adversarial Robustness in Machine Learning · Advanced Malware Detection Techniques · Hate Speech and Cyberbullying Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing 100876, China
Funding
National Natural Science Foundation of China
References (33)
Show source-provided references
1. Nguyen, A., Yosinski, J., and Clune, J. (2015, January 7–12). Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298640 ↗
2. Adversarial attacks on deep-learning models in natural language processing: A survey
3. Wang, W., Wang, R., Wang, L., Wang, Z., and Ye, A. (2019). Towards a robust deep neural network in texts: A survey. arXiv.
4. Ye, M., Miao, C., Wang, T., and Ma, F. (2022, January 12–17). TextHoaxer: Budgeted hard-label adversarial attacks on text. Proceedings of the AAAI Conference on Artificial Intelligence, Philadelphia, PA, USA.10.1609/aaai.v36i4.20303 ↗
5. Maheshwary, R., Maheshwary, S., and Pudi, V. (2021, January 2–9). Generating natural language attacks in a hard label black box setting. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual Event.10.1609/aaai.v35i15.17595 ↗
6. Emmery, C., Kádár, Á., and Chrupała, G. (2021, January 19–23). Adversarial Stylometry in the Wild: Transferable Lexical Substitution Attacks on Author Profiling. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, Online.10.18653/v1/2021.eacl-main.203 ↗
7. Toward understanding and boosting adversarial transferability from a distribution perspective10.1109/TIP.2022.3211736 ↗
8. Xie, C., Zhang, Z., Zhou, Y., Bai, S., Wang, J., Ren, Z., and Yuille, A.L. (2019, January 15–20). Improving transferability of adversarial examples with input diversity. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00284 ↗
9. Li, L., Ma, R., Guo, Q., Xue, X., and Qiu, X. (2020, January 16–20). BERT-ATTACK: Adversarial Attack Against BERT Using BERT. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online.10.18653/v1/2020.emnlp-main.500 ↗
10. Garg, S., and Ramakrishnan, G. (2020, January 16–20). BAE: BERT-based Adversarial Examples for Text Classification. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online.10.18653/v1/2020.emnlp-main.498 ↗
11. Li, D., Zhang, Y., Peng, H., Chen, L., Brockett, C., Sun, M.T., and Dolan, W.B. (2021, January 6–11). Contextualized Perturbation for Textual Adversarial Attack. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online.10.18653/v1/2021.naacl-main.400 ↗
12. Maheshwary, R., Maheshwary, S., and Pudi, V. (2021). A strong baseline for query efficient attacks in a black box setting. arXiv.10.18653/v1/2021.emnlp-main.661 ↗
13. Lee, D., Moon, S., Lee, J., and Song, H.O. (2022, January 17–23). Query-efficient and scalable black-box adversarial attacks on discrete sequential data via bayesian optimization. Proceedings of the International Conference on Machine Learning, PMLR, Baltimore, MD USA.
14. Alzantot, M., Sharma, Y., Elgohary, A., Ho, B.J., Srivastava, M., and Chang, K.W. (November, January 31). Generating Natural Language Adversarial Examples. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.10.18653/v1/D18-1316 ↗
15. Zhao, Z., Dua, D., and Singh, S. (May, January 30). Generating Natural Adversarial Examples. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada.
16. Ribeiro, M.T., Singh, S., and Guestrin, C. (2018, January 15–20). Semantically equivalent adversarial rules for debugging NLP models. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Melbourne, Australia.10.18653/v1/P18-1079 ↗
17. Yu, Z., Wang, X., Che, W., and He, K. (2022). TextHacker: Learning based Hybrid Local Search Algorithm for Text Hard-label Adversarial Attack. arXiv.10.18653/v1/2022.findings-emnlp.44 ↗
18. Vijayaraghavan, P., and Roy, D. (2019, January 16–20). Generating black-box adversarial examples for text classifiers using a deep reinforced model. Proceedings of the Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2019, Würzburg, Germany. Proceedings, Part II.10.1007/978-3-030-46147-8_43 ↗
19. Liu, Y., Chen, X., Liu, C., and Song, D. (2017, January 24–26). Delving into Transferable Adversarial Examples and Black-box Attacks. Proceedings of the International Conference on Learning Representations, Toulon, France.
20. Wu, D., Wang, Y., Xia, S.T., Bailey, J., and Ma, X. (2019, January 6–9). Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA.
21. Lin, J., Song, C., He, K., Wang, L., and Hopcroft, J.E. (2020, January 26–30). Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks. Proceedings of the International Conference on Learning Representations, Addis Ababa, Ethiopia.
22. Wang, X., Ren, J., Lin, S., Zhu, X., Wang, Y., and Zhang, Q. (2021, January 3–7). A Unified Approach to Interpreting and Boosting Adversarial Transferability. Proceedings of the International Conference on Learning Representations, Vienna, Austria.
23. Wang, X., and He, K. (2021, January 20–25). Enhancing the transferability of adversarial attacks through variance tuning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00196 ↗
24. Eberhart, R., and Kennedy, J. (December, January 27). Particle swarm optimization. Proceedings of the IEEE International Conference on Neural Networks. Citeseer, Perth, Australia.
25. Cer, D., Yang, Y., Kong, S.y., Hua, N., Limtiaco, N., John, R.S., Constant, N., Guajardo-Cespedes, M., Yuan, S., and Tar, C. (November, January 31). Universal sentence encoder for English. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Brussels, Belgium.10.18653/v1/D18-2029 ↗
26. Kim, Y. (2014, January 25–29). Convolutional Neural Networks for Sentence Classification. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.10.3115/v1/D14-1181 ↗
27. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
28. Kenton, J.D.M.W.C., and Toutanova, L.K. (2019, January 3–5). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the NAACL-HLT, Minneapolis, MN, USA.
29. Sun, Y., Wang, S., Li, Y., Feng, S., Tian, H., Wu, H., and Wang, H. (2020, January 7–12). Ernie 2.0: A continual pre-training framework for language understanding. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i05.6428 ↗
30. Zeng, A., Liu, X., Du, Z., Wang, Z., Lai, H., Ding, M., Yang, Z., Xu, Y., Zheng, W., and Xia, X. (2022, January 25). GLM-130B: An Open Bilingual Pre-trained Model. Proceedings of the Eleventh International Conference on Learning Representations, Vienna, Austria.
31. Maas, A., Daly, R.E., Pham, P.T., Huang, D., Ng, A.Y., and Potts, C. (2011, January 19–24). Learning word vectors for sentiment analysis. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, OR, USA.
32. Zhang, X., Zhao, J., and LeCun, Y. (2015). Character-level convolutional networks for text classification. arXiv.
33. Pang, B., and Lee, L. (2005, January 25). Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, Ann Arbor, MI, USA.10.3115/1219840.1219855 ↗