Prevention Is Better than Cure: Exposing the Vulnerabilities of Social Bot Detectors with Realistic Simulations
Source abstract
Abstract
The evolution of social bots, i.e., accounts on social media platforms controlled by malicious software, is making them increasingly more challenging to discover. A practical solution is to explore the adversarial nature of novel bots and find the vulnerability of bot detectors in simulations in advance. However, current studies fail to realistically simulate the environment and bots’ actions, thus not effectively representing the competition between novel bots and bot detectors. Hence, we propose a new method for modeling the impact of bot actions and develop a new bot strategy to simulate various evolved bots within a large social network. Specifically, a bot influence model and a user engagement model are introduced to simulate the growth of followers, retweets, and mentions. Additionally, a profile editor and a target preselection mechanism are proposed to more accurately simulate the behavior of evolved bots. The effectiveness of the bots and two representative bot detectors are verified using adversarial simulations and the real-world dataset. In simulated adversarial scenarios against both RF-based and GNN-based detection models, the proposed approach achieves survival rates of 99.7% and 85.9%, respectively. The simulation results indicate that, despite utilizing the bots’ profile data, user-generated content, and graph information, the detectors failed to identify all variations of the bots and mitigate their impact. More importantly, for the first time, it is found that certain types of bots outperform those usually deemed more advanced in ablation experiments, demonstrating that such “penetration testing” can indeed reveal vulnerabilities in the detectors.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Spam and Phishing Detection · Advanced Malware Detection Techniques · Network Security and Intrusion Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Cyber Science and Technology, University of Science and Technology of China, Hefei 230026, China
References (36)
Show source-provided references
1. Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., and Tesconi, M. (2017, January 3–7). The paradigm-shift of social spambots: Evidence, theories, and tools for the arms race. Proceedings of the 26th International Conference on World Wide Web Companion, Perth, Australia.10.1145/3041021.3055135 ↗
2. Social bots and the spread of disinformation in social media: The challenges of artificial intelligence10.1111/1467-8551.12554 ↗
3. Rise of the machines? Examining the influence of social bots on a political discussion network10.1177/0894439320908190 ↗
4. Feng, S., Tan, Z., Wan, H., Wang, N., Chen, Z., Zhang, B., Zheng, Q., Zhang, W., Lei, Z., and Yang, S. (2022). TwiBot-22: Towards graph-based Twitter bot detection. arXiv.
5. AudioGuard: Speech Recognition System Robust against Optimized Audio Adversarial Examples10.1007/s11042-023-15961-2 ↗
6. Using machine learning to detect fake identities: Bots vs humans10.1109/ACCESS.2018.2796018 ↗
7. Wang, L., Qiao, X., Xie, Y., Nie, W., Zhang, Y., and Liu, A. (November, January 29). My brother helps me: Node injection based adversarial attack on social bot detection. Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, ON, Canada.
8. Abreu, J.V.F., Fernandes, J.H.C., Gondim, J.J.C., and Ralha, C.G. (2020). Bot development for social engineering attacks on Twitter. arXiv.
9. Cresci, S., Petrocchi, M., Spognardi, A., and Tognazzi, S. (July, January 30). Better safe than sorry: An adversarial approach to improve social bot detection. Proceedings of the 10th ACM Conference on Web Science, Boston, MA, USA.
10. Le, T., Tran-Thanh, L., and Lee, D. (2022, January 25–29). Socialbots on fire: Modeling adversarial behaviors of socialbots via multi-agent hierarchical reinforcement learning. Proceedings of the ACM Web Conference 2022, Lyon, France.10.1145/3485447.3512215 ↗
11. Zeng, X., Peng, H., and Li, A. (2024, January 20–27). Adversarial socialbots modeling based on structural information principles. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.
12. Davis, C.A., Varol, O., Ferrara, E., Flammini, A., and Menczer, F. (2016, January 11–15). Botornot: A system to evaluate social bots. Proceedings of the 25th International Conference Companion on World Wide Web, Montreal, QC, Canada.10.1145/2872518.2889302 ↗
13. Varol, O., Ferrara, E., Davis, C., Menczer, F., and Flammini, A. (2017, January 15–18). Online human-bot interactions: Detection, estimation, and characterization. Proceedings of the International AAAI Conference on Web and Social Media, Montreal, QC, Canada.10.1609/icwsm.v11i1.14871 ↗
14. Arming the public with artificial intelligence to counter social bots10.1002/hbe2.115 ↗
15. Deep neural networks for bot detection10.1016/j.ins.2018.08.019 ↗
16. Heidari, M., Jones, J.H., and Uzuner, O. (2020, January 17–20). Deep contextualized word embedding for text-based online user profiling to detect social bots on twitter. Proceedings of the 2020 International Conference on Data Mining Workshops (ICDMW), Sorrento, Italy.10.1109/ICDMW51313.2020.00071 ↗
17. Feng, S., Wan, H., Wang, N., Li, J., and Luo, M. (2021, January 1–5). Satar: A self-supervised approach to twitter account representation learning and its application in bot detection. Proceedings of the 30th ACM International Conference on Information & Knowledge Management, Virtual.10.1145/3459637.3481949 ↗
18. Cardaioli, M., Conti, M., Di Sorbo, A., Fabrizio, E., Laudanna, S., and Visaggio, C.A. (2021, January 19–22). It’s a matter of style: Detecting social bots through writing style consistency. Proceedings of the 2021 International Conference on Computer Communications and Networks (ICCCN), Athens, Greece.10.1109/ICCCN52240.2021.9522339 ↗
19. Contrast pattern-based classification for bot detection on twitter10.1109/ACCESS.2019.2904220 ↗
20. Wei, F., and Nguyen, U.T. (2019, January 12–14). Twitter bot detection using bidirectional long short-term memory neural networks and word embeddings. Proceedings of the 2019 First IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA), Los Angeles, CA, USA.10.1109/TPS-ISA48467.2019.00021 ↗
21. Dukić, D., Keča, D., and Stipić, D. (2020, January 6–9). Are you human? Detecting bots on Twitter Using BERT. Proceedings of the 2020 IEEE 7th International Conference on Data Science and Advanced Analytics (DSAA), Sydney, Australia.10.1109/DSAA49011.2020.00089 ↗
22. Jia, J., Wang, B., and Gong, N.Z. (2017, January 26–29). Random walk based fake account detection in online social networks. Proceedings of the 2017 47th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), Denver, CO, USA.10.1109/DSN.2017.55 ↗
23. Breuer, A., Eilat, R., and Weinsberg, U. (2020, January 20–24). Friend or faux: Graph-based early detection of fake accounts on social networks. Proceedings of the Web Conference 2020, Taipei, Taiwan.10.1145/3366423.3380204 ↗
24. Feng, S., Wan, H., Wang, N., and Luo, M. (2021, January 8–11). BotRGCN: Twitter bot detection with relational graph convolutional networks. Proceedings of the 2021 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, Virtual.10.1145/3487351.3488336 ↗
25. Feng, S., Tan, Z., Li, R., and Luo, M. (March, January 22). Heterogeneity-aware twitter bot detection with relational graph transformers. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual.
26. Liu, Y., Tan, Z., Wang, H., Feng, S., Zheng, Q., and Luo, M. (2023, January 23–27). Botmoe: Twitter bot detection with community-aware mixtures of modal-specific experts. Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, Taipei, Taiwan.10.1145/3539618.3591646 ↗
27. Bot2Vec: A general approach of intra-community oriented representation learning for bot detection in different types of social networks10.1016/j.is.2021.101771 ↗
28. Detecting bots in social-networks using node and structural embeddings10.1186/s40537-023-00796-3 ↗
29. Social fingerprinting: Detection of spambot groups through DNA-inspired behavioral modeling
30. Gao, C., Lan, X., Lu, Z., Mao, J., Piao, J., Wang, H., Jin, D., and Li, Y. (2023). S3: Social-network Simulation System with Large Language Model-Empowered Agents. arXiv.10.2139/ssrn.4607026 ↗
31. Feng, S., Wan, H., Wang, N., Tan, Z., Luo, M., and Tsvetkov, Y. (2024). What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detection. arXiv.10.18653/v1/2024.acl-long.196 ↗
32. The relationship between the judged desirability of a trait and the probability that the trait will be endorsed10.1037/h0058073 ↗
33. Varol, O., Davis, C.A., Menczer, F., and Flammini, A. (2018). Feature engineering for social bot detection. Feature Engineering for Machine Learning and Data Analytics, CRC Press.10.1201/9781315181080-12 ↗
34. Ratner, A., Bach, S.H., Ehrenberg, H., Fries, J., Wu, S., and Ré, C. (September, January 28). Snorkel: Rapid training data creation with weak supervision. Proceedings of the VLDB Endowment. International Conference on Very Large Data Bases, Munich, Germany.
35. Pettingzoo: Gym for multi-agent reinforcement learning
36. Stable-Baselines3: Reliable Reinforcement Learning Implementations