Investigating the Influence of Feature Sources for Malicious Website Detection
Source abstract
Abstract
Malicious websites in general, and phishing websites in particular, attempt to mimic legitimate websites in order to trick users into trusting them. These websites, often a primary method for credential collection, pose a severe threat to large enterprises. Credential collection enables malicious actors to infiltrate enterprise systems without triggering the usual alarms. Therefore, there is a vital need to gain deep insights into the statistical features of these websites that enable Machine Learning (ML) models to classify them from their benign counterparts. Our objective in this paper is to provide this necessary investigation, more specifically, our contribution is to observe and evaluate combinations of feature sources that have not been studied in the existing literature—primarily involving embeddings extracted with Transformer-type neural networks. The second contribution is a new dataset for this problem, GAWAIN, constructed in a way that offers other researchers not only access to data, but our whole data acquisition and processing pipeline. The experiments on our new GAWAIN dataset show that the classification problem is much harder than reported in other studies—we are able to obtain around 84% in terms of test accuracy. For individual feature contributions, the most relevant ones are coming from URL embeddings, indicating that this additional step in the processing pipeline is needed in order to improve predictions. A surprising outcome of the investigation is lack of content-related features (HTML, JavaScript) from the top-10 list. When comparing the prediction outcomes between models trained on commonly used features in the literature versus embedding-related features, the gain with embeddings is slightly above 1% in terms of test accuracy. However, we argue that even this somewhat small increase can play a significant role in detecting malicious websites, and thus these types of feature categories are worth investigating further.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Spam and Phishing Detection · Advanced Malware Detection Techniques · Network Security and Intrusion Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science, University of Guelph, 50 Stone Rd E, Guelph, ON N1G 2W1, Canada
Arctic Wolf Networks, 455 Phillip St #100, Waterloo, ON N2L 3X2, Canada
References (39)
Show source-provided references
1. (2020). IC3 Internet Crime Report.
2. Sahoo, D., Liu, C., and Hoi, S. (2017). Malicious URL detection using machine learning: A survey. arXiv.
3. Kulkarni, A., and Brown, L.L. (2019). Phishing Websites Detection using Machine Learning. IJACSA, 10.10.14569/IJACSA.2019.0100702 ↗
4. Phishing website detection based on effective machine learning approach
5. Detection of Phishing Attacks: A Machine Learning Approach
6. Phishing Website Detection Based on Multidimensional Features Driven by Deep Learning10.1109/ACCESS.2019.2892066 ↗
7. Amiri, I.S., Akanbi, O.A., and Fazeldehkordi, E. (2014). A Machine-Learning Approach to Phisihing Detection and Defense, Syngress.
8. Ndichu, S., Ozawa, S., Misu, T., and Okada, K. (2018, January 8–13). A Machine Learning Approach to Malicious JavaScript Detection using Fixed Length Vector Representation. Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil.10.1109/IJCNN.2018.8489414 ↗
9. Yaokai, Y. (2019). Effective Phishing Detection Using Machine Learning Approach, Case Western Reserve University.
10. Rudd, E.M., and Abdallah, A. (2020). Training Transformers for Information Security Tasks: A Case Study on Malicious URL Prediction. arXiv.
11. Machine learning based phishing detection from URLs
12. Ensemble-based Feature Selection and Classification Model for DNS Typo-squatting Detection
13. Dalgic, F.C., Bozkir, A.S., and Aydos, M. (2018, January 19–21). Phish-IRIS: A New Approach for Vision Based Brand Prediction of Phishing Web Pages via Compact Visual Descriptors. Proceedings of the 2018 2nd International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), Ankara, Turkey.10.1109/ISMSIT.2018.8567299 ↗
14. Shahrivari, V., Darabi, M.M., and Izadi, M. (2020). Phishing Detection Using Machine Learning Techniques. arXiv.
15. Malicious Website Feature Study (2021, August 10). GitHub Repository. Available online: https://github.com/AhmadChaiban/Malicious-Website-Feature-Study.
16. Apruzzese, G., Colajanni, M., Ferretti, L., Guido, A., and Marchetti, M. (June, January 30). On the effectiveness of machine and deep learning for cyber security. Proceedings of the 10th International Conference on Cyber Conflict (CyCon), Tallinn, Estonia.
17. Gu, J., and Oelke, D. (2019). Understanding bias in machine learning. arXiv.
18. Lin, Y., Liu, R., Divakaran, D., Ng, J., Chan, Q., Lu, Y., Si, Y., Zhang, F., and Dong, J. (2021, January 11–13). Phishpedia: A Hybrid Deep Learning Based Approach to Visually Identify Phishing Webpages. Proceedings of the 30th USENIX Security Symposium (USENIX) Security 21, Vancouver, BC, Canada.
19. Web Phishing Detection Using a Deep Learning Framework10.1155/2018/4678746 ↗
20. Discovering features for detecting malicious websites: An empirical study10.1016/j.cose.2021.102374 ↗
21. Malicious and Benign Webpages Dataset10.1016/j.dib.2020.106304 ↗
22. Singh, A., and Goyal, N. (2021, January 7–10). Malcrawler: A crawler for seeking and crawling malicious websites. Proceedings of the International Conference on Distributed Computing and Internet Technology, Bhubaneswar, India.
23. Detecting Malicious Web Links and Identifying Their Attack Types
24. Malicious javascript detection by features extraction
25. Enhanced XSS Defensive Framework for Web Applications Deployed in the Virtual Machines of Cloud Computing Environment10.1016/j.protcy.2016.05.152 ↗
26. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L. (2018, January 18–22). Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗
27. Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., and Jiang, D. (2020). Codebert: A pre-trained model for programming and natural languages. arXiv.10.18653/v1/2020.findings-emnlp.139 ↗
28. Hess, S., Satam, P., Ditzler, G., and Hariri, S. (November, January 28). Malicious HTML File Prediction: A Detection and Classification Perspective with Noisy Data. Proceedings of the 2018 IEEE/ACS 15th International Conference on Computer Systems and Applications (AICCSA), Aqaba, Jordan.10.1109/AICCSA.2018.8612855 ↗
29. Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2019). DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv.
30. Simple Transformers (2021, December 15). GitHub Repository. Available online: https://github.com/ThilinaRajapakse/simpletransformers.
31. Beltagy, I., Peters, M., and Cohan, A. (2020). Longformer: The long-document transformer. arXiv.
32. Joshi, A., Lloyd, L., Westin, P., and Seethapathy, S. (2019). Using Lexical Features for Malicious URL Detection–A Machine Learning Approach. arXiv.
33. Rozi, M., Ban, T., Kim, S., Ozawa, S., Takahashi, T., and Inoue, D. (2021, January 21–25). Detecting Malicious Websites Based on JavaScript Content Analysis. Proceedings of the Computer Security Symposium 2021, Dubrovnik, Croatia.
34. Schütt, K., Bikadorov, A., Kloft, M., and Rieck, K. (2012, January 19). Early Detection of Malicious Behavior in JavaScript Code. Proceedings of the 5th ACM Workshop on Security and Artificial Intelligence, Raleigh, NC, USA.10.1145/2381896.2381901 ↗
35. Curtsinger, C., Livshits, B., Zorn, B., and Seifert, C. (2011, January 8–12). ZOZZLE: Fast and Precise In-Browser JavaScript Malware Detection. Proceedings of the 20th USENIX Security Symposium (USENIX Security 11), San Francisco, CA, USA.
36. MALURLS: A lightweight malicious website classification based on URL features
37. JS Auto DeObfuscator (2021, December 15). GitHub Repository. Available online: https://github.com/lucianogiuseppe/JS-Auto-DeObfuscator.
38. Fass, A., Krawczyk, R., Backes, M., and Stock, B. (2018). JaSt: Fully Syntactic Detection of Malicious (Obfuscated) JavaScript. Detection of Intrusions and Malware, and Vulnerability Assessment, SpringerLink.10.1007/978-3-319-93411-2_14 ↗
39. JaSt—JS AST-Based Analysis (2021, December 15). GitHub Repository. Available online: https://github.com/Aurore54F/JaSt.