A Payload Based Malicious HTTP Traffic Detection Method Using Transfer Semi-Supervised Learning
Source abstract
Abstract
Malicious HTTP traffic detection plays an important role in web application security. Most existing work applies machine learning and deep learning techniques to build the malicious HTTP traffic detection model. However, they still suffer from the problems of huge training data collection cost and low cross-dataset generalization ability. Aiming at these problems, this paper proposes DeepPTSD, a deep learning method for payload based malicious HTTP traffic detection. First, it treats the malicious HTTP traffic detection as a text classification problem and trains the initial detection model using TextCNN on a public dataset, and then adapts the initial detection model to the target dataset based on a transfer learning algorithm. Second, in the transfer learning procedure, it uses a semi-supervised learning algorithm to accomplish the model adaptation task. The semi-supervised learning algorithm enhances the target dataset based on a HTTP payload data augmentation mechanism to exploit both the labeled and unlabeled data. We evaluate DeepPTSD on two real HTTP traffic datasets. The results show that DeepPTSD has competitive performance under the small data condition.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Internet Traffic Analysis and Secure E-voting · Network Security and Intrusion Detection · Advanced Malware Detection Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Computer Science, Zhejiang University of Technology, Hangzhou 310023, China
Funding
National Natural Science Foundation of China
References (44)
Show source-provided references
1. Rotsos, C., Van Gael, J., Moore, A.W., and Ghahramani, Z. (July, January 28). Probabilistic graphical models for semi-supervised traffic classification. Proceedings of the 6th International Wireless Communications and Mobile Computing Conference, Caen, France.
2. Jadidi, Z., Muthukkumarasamy, V., Sithirasenan, E., and Singh, K. (2015, January 14–16). Flow-based anomaly detection using semisupervised learning. Proceedings of the 2015 9th International Conference on Signal Processing and Communication Systems (ICSPCS), Cairns, QLD, Australia.10.1109/ICSPCS.2015.7391760 ↗
3. Draper-Gil, G., Lashkari, A.H., Mamun, M.S.I., and Ghorbani, A.A. (2016, January 19–21). Characterization of encrypted and vpn traffic using time-related. Proceedings of the 2nd International Conference on Information Systems Security and Privacy (ICISSP), Rome, Italy.10.5220/0005740704070414 ↗
4. Bapat, R., Mandya, A., Liu, X., Abraham, B., Brown, D.E., Kang, H., and Veeraraghavan, M. (2018, January 27). Identifying malicious botnet traffic using logistic regression. Proceedings of the 2018 Systems and Information Engineering Design Symposium (SIEDS), Charlottesville, VA, USA.10.1109/SIEDS.2018.8374749 ↗
5. Feature analysis of encrypted malicious traffic10.1016/j.eswa.2019.01.064 ↗
6. Zhenqi, W., and Xinyu, W. (2008, January 30–31). Netflow based intrusion detection system. Proceedings of the 2008 International Conference on Multimedia and Information Technology, Yichang, China.10.1109/MMIT.2008.213 ↗
7. A survey of techniques for internet traffic classification using machine learning10.1109/SURV.2008.080406 ↗
8. Machlica, L., Bartos, K., and Sofka, M. (2017). Learning detectors of malicious web requests for intrusion detection in network traffic. arXiv.
9. Verma, R., and Das, A. (2017, January 24). What’s in a url: Fast feature extraction and malicious url detection. Proceedings of the 3rd ACM on International Workshop on Security and Privacy Analytics, Scottsdale, AZ, USA.10.1145/3041008.3041016 ↗
10. Liu, C., Wang, L., Lang, B., and Zhou, Y. (2018, January 13–15). Finding effective classifier for malicious URL detection. Proceedings of the 2018 2nd International Conference on Management Engineering, Software Engineering and Service Sciences, Wuhan, China.10.1145/3180374.3181352 ↗
11. Patgiri, R., Katari, H., Kumar, R., and Sharma, D. Empirical study on malicious URL detection using machine learning. Proceedings of the International Conference on Distributed Computing and Internet Technology.
12. Anomaly detection for HTTP using convolutional autoencoders10.1109/ACCESS.2018.2881003 ↗
13. A joint approach to detect malicious URL based on attention mechanism10.1142/S1469026819500214 ↗
14. Le, H., Pham, Q., Sahoo, D., and Hoi, S.C. (2018). URLNet: Learning a URL representation with deep learning for malicious URL detection. arXiv.
15. Detecting malicious urls via a keyword-based convolutional gated-recurrent-unit neural network10.1109/ACCESS.2019.2895751 ↗
16. Liu, H., and Lang, B. (2019). Machine learning and deep learning methods for intrusion detection systems: A survey. Appl. Sci., 9.10.3390/app9204396 ↗
17. Addressing the train–test gap on traffic classification combined subflow model with ensemble learning10.1016/j.knosys.2020.106192 ↗
18. Ma, J., Saul, L.K., Savage, S., and Voelker, G.M. (July, January 28). Beyond blacklists: Learning to detect malicious web sites from suspicious URLs. Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, France.
19. Yang, J., Wang, L., and Xu, Z. A novel semantic-aware approach for detecting malicious web traffic. Proceedings of the International Conference on Information and Communications Security.
20. Yang, J., Yang, P., Jin, X., and Ma, Q. (2017, January 21–24). Multi-classification for malicious URL based on improved semi-supervised algorithm. Proceedings of the 2017 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC), Guangzhou, China.10.1109/CSE-EUC.2017.34 ↗
21. Saxe, J., and Berlin, K. (2017). eXpose: A character-level convolutional neural network with embeddings for detecting malicious URLs, file paths and registry keys. arXiv.
22. Yu, Y., Liu, G., Yan, H., Li, H., and Guan, H. (2018, January 21–22). Attention-based Bi-LSTM model for anomalous HTTP traffic detection. Proceedings of the 2018 15th International Conference on Service Systems and Service Management (ICSSSM), Hangzhou, China.10.1109/ICSSSM.2018.8465034 ↗
23. Li, Z., Yao, H., and Ma, F. (2020, January 3–7). Learning with Small Data. Proceedings of the 13th International Conference on Web Search and Data Mining, Houston, TX, USA.10.1145/3336191.3371874 ↗
24. Zero-shot learning using synthesised unseen visual data with diffusion regularisation10.1109/TPAMI.2017.2762295 ↗
25. Antoniou, A., Storkey, A., and Edwards, H. (2017). Data augmentation generative adversarial networks. arXiv.
26. Chen, Z., Fu, Y., Zhang, Y., Jiang, Y.G., Xue, X., and Sigal, L. (2018). Semantic feature augmentation in few-shot learning. arXiv.
27. Deep multimodal learning: A survey on recent advances and trends10.1109/MSP.2017.2738401 ↗
28. Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. (2014). How transferable are features in deep neural networks?. arXiv.
29. Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv.
30. Commonsense reasoning and commonsense knowledge in artificial intelligence10.1145/2701413 ↗
31. Stewart, R., and Ermon, S. (2017, January 4–9). Label-free supervision of neural networks with physics and domain knowledge. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.10.1609/aaai.v31i1.10934 ↗
32. Fang, Y., Xu, Y., Jia, P., and Huang, C. (2020). Providing Email Privacy by Preventing Webmail from Loading Malicious XSS Payloads. Appl. Sci., 10.10.3390/app10134425 ↗
33. Kim, Y. (2014). ConvolutionalNeuralNetworksforSentence Classification. arXiv.
34. Google (2021, February 28). Word2vec. Available online: https://code.google.com/p/word2vec/.
35. Qiu, S., Xu, B., Zhang, J., Wang, Y., Shen, X., de Melo, G., Long, C., and Li, X. (2020). EasyAug: An automatic textual data augmentation platform for classification tasks. Companion Proceedings of the Web Conference 2020, ACM.10.1145/3366424.3383552 ↗
36. Silfverberg, M., Wiemerslage, A., Liu, L., and Mao, L.J. (2017, January 3–4). Data augmentation for morphological reinflection. Proceedings of the CoNLL SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection, Vancouver, BC, Canada.10.18653/v1/K17-2010 ↗
37. Yu, A.W., Dohan, D., Luong, M.T., Zhao, R., Chen, K., Norouzi, M., and Le, Q.V. (2018). Qanet: Combining local convolution with global self-attention for reading comprehension. arXiv.
38. Kobayashi, S. (2018). Contextual augmentation: Data augmentation by words with paradigmatic relations. arXiv.10.18653/v1/N18-2072 ↗
39. Wei, J., and Zou, K. (2019). Eda: Easy data augmentation techniques for boosting performance on text classification tasks. arXiv.10.18653/v1/D19-1670 ↗
40. Generating text via adversarial training
41. Virtual adversarial training: A regularization method for supervised and semi-supervised learning10.1109/TPAMI.2018.2858821 ↗
42. Tarvainen, A., and Valpola, H. (2017). Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. arXiv.
43. Xie, Q., Dai, Z., Hovy, E., Luong, M.T., and Le, Q.V. (2019). Unsupervised data augmentation for consistency training. arXiv.
44. FSecurify (2021, January 31). Using-Machine-Learning-to-Detect-Malicious-URLs. Available online: http://fsecurify.com/using-machine-learning-detect-malicious-urls/.