A Comprehensive Review of Tunnel Detection on Multilayer Protocols: From Traditional to Machine Learning Approaches
Source abstract
Abstract
Tunnels, a key technology of traffic obfuscation, are increasingly being used to evade censorship. While providing convenience to users, tunnel technology poses a hidden danger to cybersecurity due to its concealment and camouflage capabilities. In contrast to previous studies of encrypted traffic detection, we perform the first measurement study of tunnel traffic and its unique characteristics and focus on the challenges and solutions in detecting tunnel traffic among traditional and machine learning techniques. This study covers an almost twenty-year research period from 2003 to 2022. First, we present the concepts of two types of tunnels, broad and narrow tunnels, respectively, as well as a framework for major tunnel applications, such as Tor (the second-generation onion router), proxy, VPN, and their relationships. Second, we analyze state-of-the-art methods from traditional to machine learning applications to systematize tunnel traffic detection, including HTTP, HTTPS, DNS, SSH, TCP, ICMP and IPSec. A quantitative evaluation is presented with five crucial indicators applied to the detection methods and reviews. We further discuss the research work based on datasets, feature engineering, and challenges that have are solved, partly solved and unsolved. Finally, by providing open questions and the potential directions, we hope to inspire future work in this area.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Internet Traffic Analysis and Secure E-voting · Network Security and Intrusion Detection · Digital and Cyber Forensics
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
State Key Laboratory of Mathematical Engineering and Advanced Computing, Zhengzhou 450000, China
Funding
National Key R&D Program of China
References (79)
Show source-provided references
1. (2022, October 01). Available online: https://www.cert.org.cn/publish/main/upload/File/CNCERTreport202112.pdf.
2. Identifying Fast-Flux Botnet With AGD Names at the Upper DNS Hierarchy10.1109/ACCESS.2018.2880884 ↗
3. (2022, October 01). Available online: https://www.secrss.com/articles/40646.
4. Do, V.T., Engelstad, P.E., Feng, B., and Do, T.V. (2017, January 20–23). Detection of DNS Tunneling in Mobile Networks Using Machine Learning. Proceedings of the International Conference on Information Science and Applications, Macau, China.10.1007/978-981-10-4154-9_26 ↗
5. Machine Learning Based Classification Accuracy of Encrypted Service Channels: Analysis of Various Factors
6. How to Write a Literature Review Paper?10.1080/01441647.2015.1065456 ↗
7. Chaum, D. (1984). Advances in Cryptology: Proceedings of Crypto 83, Springer.10.1007/978-1-4684-4730-9 ↗
8. Karagiannis, T., Broido, A., Faloutsos, M., and Claffy, K.C. (2004, January 25–27). Transport layer identification of P2P traffic. Proceedings of the ACM/SIGCOMM Internet Measurement Conference, Taormina Sicily, Italy.10.1145/1028788.1028804 ↗
9. Karagiannis, T., Broido, A., and Brownlee, N. (December, January 29). Is P2P dying or just hiding?. Proceedings of the IEEE Global Telecommunications Conference, GLOBECOM’04, Dallas, TX, USA.
10. Cotton, M., Eggert, L., Touch, J.D., Westerlund, M., and Cheshire, S. (2022, October 01). Internet Assigned Numbers Authority (IANA) Procedures for the Management of the Service Name and Transport Protocol Port Number Registry. Available online: https://www.rfc-editor.org/rfc/pdfrfc/rfc6335.txt.pdf.
11. Issues and future directions in traffic classification10.1109/MNET.2012.6135854 ↗
12. Dharmapurikar, S., Krishnamurthy, P., Sproull, T.S., and Lockwood, J.W. (2004, January 20–22). Deep packet inspection using parallel bloom filters. Proceedings of the 11th Symposium on High Performance Interconnects, Stanford, CA, USA.
13. Bortolameotti, R., Ede, T.v., Caselli, M., Everts, M.H., Hartel, P.H., Hofstede, R., Jonker, W., and Peter, A. (2017, January 4–8). DECANTeR: DEteCtion of Anomalous outbouNd HTTP TRaffic by Passive Application Fingerprinting. Proceedings of the 33rd Annual Computer Security Applications Conference, Orlando, FL, USA.
14. Entropy-based analyzing anomaly WEB traffic10.3233/JHS-200642 ↗
15. Durumeric, Z., Kasten, J., Bailey, M., and Halderman, J.A. (2013, January 23–25). Analysis of the HTTPS certificate ecosystem. Proceedings of the 2013 conference on Internet measurement conference, New York, NY, USA.10.1145/2504730.2504755 ↗
16. Wang, F., Huang, L., Chen, Z., Miao, H., and Yang, W. (2013, January 25–28). A Novel Web Tunnel Detection Method Based on Protocol Behaviors. Proceedings of the SecureComm, Sydney, NSW, Australia.10.1007/978-3-319-04283-1_15 ↗
17. Sun, G., Xue, Y., Dong, Y., Wang, D., and Li, C. (2010, January 6–10). An Novel Hybrid Method for Effectively Classifying Encrypted Traffic. Proceedings of the 2010 IEEE Global Telecommunications Conference GLOBECOM 2010, Miami, FL, USA.10.1109/GLOCOM.2010.5683649 ↗
18. Karasaridis, A., Meier-Hellstern, K.S., and Hoeflin, D.A. (2006, January 1). NIS04-2: Detection of DNS Anomalies using Flow Data Analysis. Proceedings of the 1st IEEE Workshop on Automotive Networking and Applications (AutoNet 2006), San Francisco, CA, USA.10.1109/GLOCOM.2006.280 ↗
19. A Network Steganography Lab on Detecting TCP/IP Covert Channels10.1109/TE.2016.2520400 ↗
20. An Entropy-Based Approach to Detecting Covert Timing Channels10.1109/TDSC.2010.46 ↗
21. Shon, T., Moon, J., Lee, S., Lee, D.H., and Lim, J. (2003, January 3–5). Covert Channel Detection in the ICMP Payload Using Support Vector Machine. Proceedings of the International Symposium on Computer and Information Sciences, Antalya, Turkey.10.1007/978-3-540-39737-3_103 ↗
22. Govil, J., and Jivika, G. (2007, January 17–20). Criminology of BotNets and their detection and defense methods. Proceedings of the 2007 IEEE International Conference on Electro./Information Technology, Chicago, IL, USA.10.1109/EIT.2007.4374517 ↗
23. Barbhuiya, F.A., Roopa, S.N., Ratti, R., Biswas, S., and Nandi, S. (2012, January 25–27). An Active Detection Mechanism for Detecting ICMP Based Attacks. Proceedings of the 2012 IEEE 11th International Conference on Trust, Security and Privacy in Computing and Communications, Liverpool, UK.10.1109/TrustCom.2012.68 ↗
24. Sayadi, S., Abbes, T., and Bouhoula, A. (November, January 30). Detection of Covert Channels Over ICMP Protocol. Proceedings of the 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA), Hammamet, Tunisia.10.1109/AICCSA.2017.60 ↗
25. Korczyński, M., and Duda, A. (May, January 27). Markov chain fingerprinting to classify encrypted traffic. Proceedings of the IEEE INFOCOM 2014—IEEE Conference on Computer Communications, Toronto, ON, Canada.10.1109/INFOCOM.2014.6848005 ↗
26. Detection of TCP covert channel based on Markov model10.1007/s11235-013-9737-7 ↗
27. Piraisoody, G., Huang, C., Nandy, B., and Seddigh, N. (2013, January 11–13). Classification of applications in HTTP tunnels. Proceedings of the 2013 IEEE 2nd International Conference on Cloud Networking (CloudNet), San Francisco, CA, USA.10.1109/CloudNet.2013.6710559 ↗
28. McPAD: A multiple classifier system for accurate payload-based anomaly detection10.1016/j.comnet.2008.11.011 ↗
29. Anderson, B., and McGrew, D.A. (2017, January 13–17). Machine Learning for Encrypted Malware Traffic Classification: Accounting for Noisy Labels and Non-Stationarity. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA.10.1145/3097983.3098163 ↗
30. A Support Vector Machine-Based Framework for Detection of Covert Timing Channels10.1109/TDSC.2015.2423680 ↗
31. Fu, G., Li, Q., Chen, Z., Zeng, G., and Gu, J. (2018, January 8–10). Network Storage Covert Channel Detection Based on Data Joint Analysis. Proceedings of the ICCCS, Haikou, China.10.1007/978-3-030-00015-8_30 ↗
32. Okada, Y., Ata, S., Nakamura, N., Nakahira, Y., and Oka, I. (2011, January 21–25). Application identification from encrypted traffic based on characteristic changes by encryption. Proceedings of the 2011 IEEE International Workshop Technical Committee on Communications Quality and Reliability (CQR), Seattle, WA, USA.10.1109/CQR.2011.5996087 ↗
33. Kumano, Y., Ata, S., Nakamura, N., Nakahira, Y., and Oka, I. (2014, January 3–6). Towards real-time processing for application identification of encrypted traffic. Proceedings of the 2014 International Conference on Computing, Networking and Communications (ICNC), Honolulu, HI, USA.10.1109/ICCNC.2014.6785319 ↗
34. Deep learning-based real-time VPN encrypted traffic identification methods10.1007/s11554-019-00930-6 ↗
35. Ding, S., Zhang, D., Ge, J., Yuan, X.-F., and Du, X. (October, January 30). Encrypt DNS Traffic: Automated Feature Learning Method for Detecting DNS Tunnels. Proceedings of the 2021 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom), New York, NY, USA.10.1109/ISPA-BDCloud-SocialCom-SustainCom52081.2021.00056 ↗
36. Wang, K., and Stolfo, S. (2004, January 15–17). Anomalous Payload-Based Network Intrusion Detection. Proceedings of the International Symposium on Recent Advances in Intrusion Detection, Sophia Antipolis, France.10.1007/978-3-540-30143-1_11 ↗
37. Siby, S.D., Juárez, M., Díaz, C., Vallina-Rodriguez, N., and Troncoso, C. (2019). Encrypted DNS -> Privacy? A Traffic Analysis Perspective. arXiv.10.14722/ndss.2020.24301 ↗
38. Liu, C., Cao, Z., Xiong, G., Gou, G., Yiu, S.-M., and He, L. (2018, January 4–6). MaMPF: Encrypted Traffic Classification Based on Multi-Attribute Markov Probability Fingerprints. Proceedings of the 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS), Banff, AB, Canada.10.1109/IWQoS.2018.8624124 ↗
39. Ding, Y., and Cai, W.-d. (2011, January 27–29). A method for HTTP-tunnel detection based on statistical features of traffic. Proceedings of the 2011 IEEE 3rd International Conference on Communication Software and Networks, Xi’an, China.10.1109/ICCSN.2011.6013585 ↗
40. Draper-Gil, G., Habibi Lashkari, A., Mamun, M.S.I., and Ghorbani, A.A. (2016, January 19–21). Characterization of Encrypted and VPN Traffic using Time-related Features. Proceedings of the International Conference on Information Systems Security and Privacy, Rome, Italy.10.5220/0005740704070414 ↗
41. Bilge, L., Kirda, E., Krügel, C., and Balduzzi, M. (2011, January 6–9). EXPOSURE: Finding Malicious Domains Using Passive DNS Analysis. Proceedings of the Network and Distributed System Security Symposium, San Diego, CA, USA.
42. Can encrypted traffic be identified without port numbers, IP addresses and payload inspection?10.1016/j.comnet.2010.12.002 ↗
43. Agghey, A.Z., Mwinuka, L.J., Pandhare, S.M., Dida, M.A., and Ndibwile, J.D. (2021). Detection of Username Enumeration Attack on SSH Protocol: Machine Learning Approach. Symmetry, 13.10.3390/sym13112192 ↗
44. Zhang, J., Qian, Z., Shou, G., and Hu, Y. (2010, January 20–22). Traffic identification method based on on-line density based spatial clustering algorithm. Proceedings of the 2010 2nd IEEE InternationalConference on Network Infrastructure and Digital Content, Washington, DC, USA.10.1109/ICNIDC.2010.5657786 ↗
45. A survey of covert channels and countermeasures in computer network protocols10.1109/COMST.2007.4317620 ↗
46. Zhao, P., Gou, G., Liu, C., Guan, Y., Cui, M., and Xiong, G. (2021, January 6–9). TMT-RF: Tunnel Mixed Traffic Classification Based on Random Forest. Proceedings of the Security and Privacy in Communication Networks, Virtual.10.1007/978-3-030-90019-9_21 ↗
47. A Method of Detecting Storage Based Network Steganography Using Machine Learning10.1016/j.procs.2019.06.086 ↗
48. Wang, B., Xiong, G., Fu, P., Gou, G., Qin, Y., and Li, Z. (2022, January 10–12). A Two-Stage Method for Fine-Grained DNS Covert Tunnel Behavior Detection. Proceedings of the International Conference on Science of Cyber Security, Matsue, Japan.10.1007/978-3-031-17551-0_13 ↗
49. Burghouwt, P., Spruit, M.E.M., and Sips, H.J. (2010, January 8–11). Detection of botnet collusion by degree distribution of domains. Proceedings of the 2010 International Conference for Internet Technology and Secured Transactions, London, UK.
50. Wong, M.Z. (2019). Deep Learning Models for Malicious Web Content Detection: An Enterprise Study, University of Toronto.
51. Parchekani, A., Naghadeh, S.N., and Shah-Mansouri, V. (2020). Classification of Traffic Using Neural Networks by Rejecting: A Novel Approach in Classifying VPN Traffic. arXiv.
52. Herrmann, D., Wendolsky, R., and Federrath, H. (2009, January 13). Website fingerprinting: Attacking popular privacy enhancing technologies with the multinomial naïve-bayes classifier. Proceedings of the Cloud Computing Security Workshop, New York, NY, USA.10.1145/1655008.1655013 ↗
53. Yu, Y., Liu, G., Yan, H., Li, H., and Guan, H. (2018, January 21–22). Attention-Based Bi-LSTM Model for Anomalous HTTP Traffic Detection. Proceedings of the 2018 15th International Conference on Service Systems and Service Management (ICSSSM), Hangzhou, China.10.1109/ICSSSM.2018.8465034 ↗
54. HTTP Tunnel Trojan Detection Model Based on Deep Learning10.1088/1742-6596/1187/4/042055 ↗
55. Marchal, S., François, J., Wagner, C., State, R., Dulaunoy, A., Engel, T., and Festor, O. (2012, January 25–29). DNSSM: A large scale passive DNS security monitoring framework. Proceedings of the 2012 IEEE Network Operations and Management Symposium, Budapest, Hungary.10.1109/NOMS.2012.6212019 ↗
56. Maiolini, G., Baiocchi, A., Iacovazzi, A., and Rizzi, A. (2009, January 11–15). Real Time Identification of SSH Encrypted Application Flows by Using Cluster Analysis Techniques. Proceedings of the Networking 2009: 8th International IFIP-TC 6 Networking Conference, Aachen, Germany.10.1007/978-3-642-01399-7_15 ↗
57. Pradhan, A., Behera, S., and Dash, R. (2018, January 22–23). Hybrid RBFN Based Encrypted SSH Traffic Classification. Proceedings of the 2018 5th International Conference on Signal Processing and Integrated Networks (SPIN), Amity University, Noida, India.10.1109/SPIN.2018.8474059 ↗
58. Palau, F., Catania, C.A., Guerra, J., García, S., and Rigaki, M. (2020). DNS Tunneling: A Deep Learning based Lexicographical Detection Approach. arXiv.
59. DNS tunnels detection via DNS-images10.1016/j.ipm.2022.102930 ↗
60. Wang, Z., Dong, H., Chi, Y., Zhang, J., Yang, T., and Liu, Q. (2019, January 22). DGA and DNS Covert Channel Detection System based on Machine Learning. Proceedings of the 3rd International Conference on Computer Science and Application Engineering, New York, NY, USA.10.1145/3331453.3361663 ↗
61. Alshammari, R., and Zincir-Heywood, A.N. (2007, January 7–10). A flow based approach for SSH traffic detection. Proceedings of the 2007 IEEE International Conference on Systems, Man and Cybernetics, Montréal, QC, Canada.10.1109/ICSMC.2007.4414006 ↗
62. Refined Detection of SSH Brute-Force Attackers Using Machine Learning
63. The 1999 DARPA off-line intrusion detection evaluation10.1016/S1389-1286(00)00139-0 ↗
64. Anomaly-based model for detecting HTTP-tunnel traffic using network behavior analysis
65. (2022, October 01). Available online: https://www.unb.ca/cic/datasets/vpn.html.
66. Hoffman, P.E., and McManus, P. (2022, October 01). DNS Queries over HTTPS (DoH). Available online: https://www.rfc-editor.org/rfc/pdfrfc/rfc8484.txt.pdf.
67. Kingma, D.P., and Welling, M. (2013). Auto-Encoding Variational Bayes. arXiv.
68. MontazeriShatoori, M., Davidson, L., Kaur, G., and Habibi Lashkari, A. (2020, January 7). Detection of DoH Tunnels using Time-series Classification of Encrypted Traffic. Proceedings of the 2020 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech), Calgary, Alta.10.1109/DASC-PICom-CBDCom-CyberSciTech49142.2020.00026 ↗
69. Liu, C., He, L., Xiong, G., Cao, Z., and Li, Z. (May, January 29). FS-Net: A Flow Sequence Network For Encrypted Traffic Classification. Proceedings of the IEEE INFOCOM 2019—IEEE Conference on Computer Communications, Paris, France.10.1109/INFOCOM.2019.8737507 ↗
70. Singh, A., Nordström, O., Lu, C., and Santos, A.D. (2003, January 9–11). Malicious ICMP Tunneling: Defense against the Vulnerability. Proceedings of the Australasian Conference on Information Security and Privacy, Wollongong, Australia.10.1007/3-540-45067-X_20 ↗
71. Dakhane, D.D.M., Patil, S.M., and Patil, M.V. (2011, January 3–5). Detection and elimination of covert communication in Transport and Internet layer—A Survey. Proceedings of the IJCA Proceedings on International Conference on Recent Trends in Information Technology and Computer Science (ICRTITCS-2011), Chennai, India.
72. Gober, S.Z., Javed, B., and Saqib, N.A. (2012, January 12–14). Covert channel detection: A survey based analysis. Proceedings of the High Capacity Optical Networks and Emerging/Enabling Technologies, Istanbul, Turkey.10.1109/HONET.2012.6421435 ↗
73. Pattern-Based Survey and Categorization of Network Covert Channel Techniques10.1145/2684195 ↗
74. Carrara, B., and Adams, C.M. (2016, January 20–22). A Survey and Taxonomy Aimed at the Detection and Measurement of Covert Channels. Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security, New York, NY, USA.10.1145/2909827.2930800 ↗
75. Yassine, S., Khalife, J., Chamoun, M., and Ghor, H.E. (2018, January 13–15). A Survey of DNS Tunnelling Detection Techniques Using Machine Learning. Proceedings of the 1st International Conference on Big Data and Cyber-Security Intelligence, BDCSIntell, Hadath, Lebanon.
76. A comprehensive survey on DNS tunnel detection10.1016/j.comnet.2021.108322 ↗
77. Elsadig, M.A., and Fadlalla, Y.A. (2017, January 8–11). Network Protocol Covert Channels: Countermeasures Techniques. Proceedings of the 2017 9th IEEE-GCC Conference and Exhibition (GCCCE), Manama, Bahrain.10.1109/IEEEGCC.2017.8447997 ↗
78. Mazel, J., Saudrais, M., and Hervieu, A. (2022). ML-based tunnel detection and tunneled application classification. arXiv.
79. A Survey of Key Technologies for Constructing Network Covert Channel10.1155/2020/8892896 ↗