journal article·2023-02-23Open access

Classification and Analysis of Malicious Code Detection Techniques Based on the APT Attack

Kyungroul Lee, Jaehyuk Lee, Kangbin Yim

Applied Sciences· Volume 13· Issue 5

Abstract

According to the Fire-eye’s M-Trends Annual Threat Report 2022, there are many advanced persistent threat (APT) attacks that are currently in use, and such continuous and specialized APT attacks cause serious damages attacks. As APT attacks continue to be active, there is a need for countermeasures to detect new and existing malicious codes. An APT attack is a type of intelligent attack that analyzes the target and exploits its vulnerabilities. It attempts to achieve a specific purpose, and is persistent in continuously attacking and threatening the system. With this background, this paper analyzes attack scenarios based on attack cases by malicious code, and surveys and analyzes attack techniques used in attack cases. Based on the results of the analysis, we classify and analyze malicious code detection techniques into security management systems, pattern-based detection, heuristic-based detection, reputation-based detection, behavior-based detection, virtualization-based detection, anomaly detection, data analysis-based detection (big data-based, machine learning-based), and others. This paper is expected to serve as a useful reference for detecting and preventing malicious codes. Specifically, this article is a surveyed review article.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Advanced Malware Detection Techniques · Network Security and Intrusion Detection · Information and Cyber Security

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Information Security, Mokpo National University, Mokpo 58554, Republic of Korea

Interdisciplinary Program of Information & Protection, Mokpo National University, Mokpo 58554, Republic of Korea

Department of Information Security Engineering, Soonchunhyang University, Asan 31538, Republic of Korea

Funding

National Research Foundation of Korea (NRF) · Korea government (MSIT) · Soonchunhyang University Research Fund

References (100+)

Show source-provided references
1. Khalid, A., Zainal, A., Maarof, M.A., and Ghaleb, F.A. (2021, January 5). Advanced Persistent Threat Detection: A Survey. Proceedings of the 2021 3rd International Cyber Resilience Conference (CRC), Langkawi Island, Malaysia.10.1109/CRC50527.2021.9392626 ↗
2. A comprehensive study on APT attacks and countermeasures for future networks and communications: Challenges and solutions
3. (2011). Managing Information Security Risk: Organization, Mission, and Information System View, National Institute of Standards and Technology.
4. Kim, G., Choi, C., and Choi, J. (2018, January 9). Ontology Modeling for APT Attack Detection in an IoT-Based Power System. Proceedings of the 2018 Conference on Research in Adaptive and Convergent Systems, Honolulu, Hawaii.10.1145/3264746.3264786 ↗
5. Fei, Y., Ning, J., and Jiang, W. (2018, January 12–14). A Quantifiable Attack-Defense Trees Model for APT Attack. Proceedings of the 2018 IEEE 3rd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), Chongqing, China.10.1109/IAEAC.2018.8577817 ↗
6. Gao, P., Xiao, X., Li, Z., Xu, F., Kulkarni, S.R., and Mittal, P. (, January July). AIQL: Enabling Efficient Attack Investigation from System Monitoring Data. Proceedings of the 2018 USENIX Conference on Usenix Annual Technical Conference, Boston, MA, USA.
7. APTGuard: Advanced Persistent Threat (APT) Detections and Predictions Using Android Smartphone10.1007/978-981-13-2622-6_53 ↗
8. Nicho, M., Oluwasegun, A., and Kamoun, F. (2018, January 26–28). Identifying Vulnerabilities in APT Attacks: A Simulated Approach. Proceedings of the 2018 9th IFIP International Conference on New Technologies, Mobility and Security (NTMS), Paris, France.10.1109/NTMS.2018.8328696 ↗
9. (2023, January 05). FireEye, M-Trends. Available online: https://content.fireeye.com/m-trends/rpt-m-trends-2022.
10. Security Framework for the Semiconductor Supply Chain Environment10.1007/978-3-030-05195-2_16 ↗
11. Kim, N., Lee, S., Cho, H., Kim, B.-I., and Jun, M. (2018, January 29–31). Design of a Cyber Threat Information Collection System for Cyber Attack Correlation. Proceedings of the 2018 International Conference on Platform Technology and Service (PlatCon), Jeju, Republic of Korea.10.1109/PlatCon.2018.8472775 ↗
12. Host-Based Intrusion Detection System for Secure Human-Centric Computing10.1007/s11227-015-1506-9 ↗
13. Kim, Y., and Kim, I. (2014, January 16–20). Involvers Behavior-based Modeling in Cyber Targeted Attack. Proceedings of the Eighth International Conference on Emerging Security Information, Systems and Technologies (SECURWARE), Lisbon, Portugal.
14. Jonathan, A.P., Kim, K.H., Park, J.-H., Kim, C.-H., and Lee, H.-J. (2013, January 24–25). Analysis of the 2013.3.20 South Korea APT Attack. Proceedings of the Korean Institute of Information and Commutation Sciences Conference, Mokpo, Republic of Korea.
15. Preemptive Response Strategy for Attacker Origin for National Cybersecurity
16. Kim, Y.S., Kim, I., and Park, N. (2014). Mobile, Ubiquitous, and Intelligent Computing, Springer.
17. Unified threat management
18. (2023, January 25). McAfee, Gold Dragon Widens Olympics Malware Attacks, Gains Permanent Presence on Victims’ Systems. Available online: https://vxug.fakedoma.in/archive/APTs/2018/2018.02.02/Gold%20Dragon.pdf.
19. Secure transmission technique for data in IoT edge computing infrastructure10.1007/s40747-021-00576-7 ↗
20. Swain, S.C. (2021). Strategies for e-Service, e-Governance, and Cybersecurity: Challenges and Solutions for Efficiency and Sustainability, CRC Press.
21. Vardalaki, A., and Vlachos, V. (2021). Cybersecurity Issues in Emerging Technologies, CRC Press.
22. Ghafir, I., Prenosil, V., Hammoudeh, M., Aparicio-Navarro, F.J., Rabie, K., and Jabban, A. (2018, January 26–27). Disguised executable files in spear-phishing emails: Detecting the point of entry in advanced persistent threat. Proceedings of the 2nd International Conference on Future Networks and Distributed Systems (ICFNDS), New York, NY, USA.10.1145/3231053.3231097 ↗
23. Protection against semantic social engineering attacks10.1007/978-3-319-97643-3_4 ↗
24. Kim, J. (2020, January 16–18). North Korea’s cyber attack threat analysis research(Based on the type of attack technology). Proceedings of the Korean Society of Computer Information Conference, Jeju, South Korea.
25. Rath, S., Zografopoulos, I., and Konstantinou, C. (2021, January 22). Stealthy Rootkit Attacks on Cyber-Physical Microgrids: Poster. Proceedings of the Twelfth ACM International Conference on Future Energy Systems, Virtual Event, Italy.10.1145/3447555.3466576 ↗
26. Li, M., Huang, W., Wang, Y., Fan, W., and Li, J. (2016, January 26–29). The Study of APT Attack Stage Model. Proceedings of the 2016 IEEE/ACIS 15th International Conference on Computer and Information Science (ICIS), Okayama, Japan.10.1109/ICIS.2016.7550947 ↗
27. Bermejo Higuera, J., Abad Aramburu, C., Bermejo Higuera, J.-R., Sicilia Urban, M.A., and Sicilia Montalvo, J.A. (2020). Systematic Approach to Malware Analysis (SAMA). Appl. Sci., 10.10.3390/app10041360 ↗
28. Attack Provenance Tracing in Cyberspace: Solutions, Challenges and Future Directions10.1109/MNET.2018.1700469 ↗
29. ProPatrol: Attack Investigation via Extracted High-Level Tasks10.1007/978-3-030-05171-6_6 ↗
30. Daimi, K. (2018). Computer and Network Security Essentials, Springer International Publishing.10.1007/978-3-319-58424-9 ↗
31. Faruki, P., Laxmi, V., Gaur, M.S., and Vinod, P. Behavioural Detection with API Call-Grams to Identify Malicious PE Files. Proceedings of the First International Conference on Security of Internet of Things-SecurIT ’12, Kollam, India.
32. In-Execution Dynamic Malware Analysis and Detection by Mining Information in Process Control Blocks of Linux OS10.1016/j.ins.2011.09.016 ↗
33. Secured and privacy-based IDS for healthcare systems on E-medical data using machine learning approach
34. A hybrid approach for network intrusion detection
35. DL-IDS: A deep learning–based intrusion detection framework for securing IoT10.1002/ett.3803 ↗
36. Ahmed, W., Shahzad, F., Javed, A.R., Iqbal, F., and Ali, L. (2021, January 19). WhatsApp Network Forensics: Discovering the IP Addresses of Suspects. Proceedings of the 2021 11th IFIP International Conference on New Technologies, Mobility and Security (NTMS), Paris, France.10.1109/NTMS49979.2021.9432677 ↗
37. Qureshi, S., Tunio, S., Akhtar, F., Wajahat, A., Nazir, A., and Ullah, F. (2021). Network Forensics: A Comprehensive Review of Tools and Techniques. IJACSA, 12.10.14569/IJACSA.2021.01205103 ↗
38. Design for Zombie PCs and APT Attack Detection based on traffic analysis
39. Detecting APT Malware Infections Based on Malicious DNS and Traffic Analysis10.1109/ACCESS.2015.2458581 ↗
40. Negi, P.S., Garg, A., and Lal, R. (2020, January 29–31). Intrusion detection and prevention using honeypot network for cloud security. Proceedings of the 2020 10th International Conference on Cloud Computing, Data Science & Engineering (Confluence), Noida, India.10.1109/Confluence47617.2020.9057961 ↗
41. Sun, X., Wang, Y., Ren, J., Zhu, Y., and Liu, S. (2008, January 18–21). Collecting Internet Malware Based on Client-Side Honeypot. Proceedings of the 2008 the 9th International Conference for Young Computer Scientists.10.1109/ICYCS.2008.257 ↗
42. Collecting Autonom s Spreading Malware Using High-Interaction Honeypots10.1007/978-3-540-77048-0_34 ↗
43. Wang, Y.-M., Beck, D., Jiang, X., Roussev, R., Verbowski, C., Chen, S., and King, S. (2006, January 3–5). Automated Web Patrol with Strider HoneyMonkeys. Proceedings of the 2006 Network and Distributed System Security Symposium, San Diego, CA, USA.
44. Radek, H. (2009). The Capture-HPC Client Architecture, Victoria University of Wellington. Technical report.
45. Monkey-spider: Detecting malicious websites with low-interaction honeyclients
46. Mohr, G., Stack, M., Ranitovic, I., Avery, D., and Kimpton, M. (2004, January 16). An Introduction to Heritrix. Proceedings of the 4th International Web Archiving Workshop, Bath, UK.
47. Xie, M., Wu, Z., and Wang, H. (2007, January 10–14). HoneyIM: Fast Detection and Suppression of Instant Messaging Malware in Enterprise-Like Networks. Proceedings of the Twenty-Third Annual Computer Security Applications Conference (ACSAC 2007), Miami Beach, FL, USA.10.1109/ACSAC.2007.24 ↗
48. Secure Instant Messaging in Enterprise-like Networks10.1016/j.comnet.2011.09.006 ↗
49. A Smart Agent Design for Cyber Security Based on Honeypot and Machine Learning
50. Devi, B.T., Shitharth, S., and Jabbar, M.A. (2020, January 5–7). An Appraisal over Intrusion Detection Systems in Cloud Computing Security Attacks. Proceedings of the 2020 2nd International Conference on Innovative Mechanisms for Industry Applications (ICIMIA), Bangalore, India.10.1109/ICIMIA48430.2020.9074924 ↗
51. Ghafir, I., Prenosil, V., Hammoudeh, M., Han, L., and Raza, U. (2017, January 19). Malicious SSL Certificate Detection: A Step Towards Advanced Persistent Threat Defence. Proceedings of the Proceedings of the International Conference on Future Networks and Distributed Systems, Cambridge, UK.10.1145/3102304.3102331 ↗
52. A Study on a Secure USB Mechanism That Prevents the Exposure of Authentication Information for Smart Human Care Services10.1155/2018/2089626 ↗
53. Vulnerability Analysis of a Secure USB Memory: Based on a Commercial Product D10.1007/978-3-030-90072-4_30 ↗
54. Lee, J.D.M. (2021, December 23). FingerTool v1.19 in the DM PD065 Secure USB is Susceptible to Improper Authentication by a Replay Attack. The MITRE Corporation. Available online: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-26824.
55. Firstbrook, P., and MacDonald, N. (2021, December 23). A buyers guide to endpoint protection platforms. Available online: https://www.gartner.com/doc/2973617/buyers-guide-endpoint-protection-platform.
56. Jung, B., Kim, T., and Im, E.G. (2018, January 9). Malware Classification Using Byte Sequence Information. Proceedings of the 2018 Conference on Research in Adaptive and Convergent Systems, Honolulu, Hawaii.10.1145/3264746.3264775 ↗
57. Tian, R., Batten, L., Islam, R., and Versteeg, S. (2009, January 13–14). An Automated Classification System Based on the Strings of Trojan and Virus Families. Proceedings of the 2009 4th International Conference on Malicious and Unwanted Software (MALWARE), Montreal, QC, USA.10.1109/MALWARE.2009.5403021 ↗
58. Opcodes as Predictor for Malware10.1504/IJESDF.2007.016865 ↗
59. Bonfante, G., Kaczmarek, M., and Marion, J.-Y. (2008, January 13). Morphological Detection of Malware. Proceedings of the 2008 3rd International Conference on Malicious and Unwanted Software (MALWARE), Fairfax, VI, USA.10.1109/MALWARE.2008.4690851 ↗
60. Pektaş, A., Eriş, M., and Acarman, T. (2011, January 21–27). Proposal of N-Gram Based Algorithm for Malware Classification. Proceedings of the the Fifth International Conference on Emerging Security Information, Systems and Technologies, Nice/Saint Laurent du Var, France.
61. Idea: Opcode-Sequence-Based Malware Detection10.1007/978-3-642-11747-3_3 ↗
62. A Statistical Pattern Based Feature Extraction Method on System Call Traces for Anomaly Detection10.1016/j.infsof.2020.106348 ↗
63. Zhou, L., and Liu, F. (2003, January 8). Research on Computer Network Security Based on Pattern Recognition. Proceedings of the SMC’03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme-System Security and Assurance (Cat. No.03CH37483), Washington, DC, USA.
64. Li, P., Teng, W.-D., Zheng, W., and Zhang, K.-H. (2010, January 13–15). Formalized Answer Extraction Technology Based on Pattern Learning. Proceedings of the International Forum on Strategic Technology 2010, Ulsan, Republic of Korea.
65. Shreeranga, P.R., Vig, A., and Narayana, V.S.A. (2007, January 17–20). An Efficient Classification Algorithm Based on Pattern Range Tree Prototypes. Proceedings of the 10th International Conference on Information Technology (ICIT 2007), Rourkela, Orissa, India.10.1109/ICIT.2007.51 ↗
66. Mitigating Drive-By Download Attacks: Challenges and Open Problems10.1007/978-3-642-05437-2_5 ↗
67. Niki, A. (2009). Drive-by Download Attacks: Effects and Detection Methods. [Ph.D. thesis, Master’s thesis, Royal Holloway University of London].
68. Zhou, R., Pan, J., Tan, X., and Xi, H. (2008, January 13–17). Application of CLIPS Expert System to Malware Detection System. Proceedings of the 2008 International Conference on Computational Intelligence and Security, Suzhou, China.10.1109/CIS.2008.100 ↗
69. Computer virus strategies and detection methods
70. Malware Target Recognition via Static Heuristics10.1016/j.cose.2011.09.002 ↗
71. Novel Active Learning Methods for Enhanced PC Malware Detection in Windows OS10.1016/j.eswa.2014.02.053 ↗
72. ENDMal: An Anti-Obfuscation and Collaborative Malware Detection System Using Syscall Sequences10.1016/j.mcm.2013.03.008 ↗
73. A Heuristic Local Community Detection Method (HLCD)10.1007/s10489-016-0824-9 ↗
74. Akhtar, M.S., and Feng, T. (2022). Malware Analysis and Detection Using Machine Learning Algorithms. Symmetry, 14.10.3390/sym14112304 ↗
75. Williams, C. (2021, July 06). Applications of Genetic Algorithms to Malware Detection and Creation. Available online: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=32fe925452da3a44ba92dc3df051d0ccc9061980.
76. Opcode Sequences as Representation of Executables for Data-Mining-Based Unknown Malware Detection10.1016/j.ins.2011.08.020 ↗
77. Permission-Based Android Malware Detection System Using Feature Selection with Genetic Algorithm10.1142/S0218194019500116 ↗
78. Wang, L., Gao, Y., Gao, S., and Yong, X. (2021). A New Feature Selection Method Based on a Self-Variant Genetic Algorithm Applied to Android Malware Detection. Symmetry, 13.10.3390/sym13071290 ↗
79. Sinha, S., Bailey, M., and Jahanian, F. (2008, January 7–8). Shades of Grey: On the Effectiveness of Reputation-Based;Blacklists. Proceedings of the 2008 3rd International Conference on Malicious and Unwanted Software (MALWARE), Alexandria, VA, USA.10.1109/MALWARE.2008.4690858 ↗
80. Choi, Y.H., Han, B.J., Bae, B.C., Oh, H.G., and Sohn, K.W. (2012). Toward extracting malware features for classification using static and dynamic analysis. Comput. Netw. Technol., 126–129.
81. HDM-Analyser: A Hybrid Analysis Approach Based on Data Mining Techniques for Malware Detection10.1007/s11416-013-0181-8 ↗
82. A Comparison of Static, Dynamic, and Hybrid Analysis for Malware Detection10.1007/s11416-015-0261-z ↗
83. A Survey of Intrusion Detection Techniques in Cloud10.1016/j.jnca.2012.05.003 ↗
84. SCARECROW: Scalable Malware Reporting, Detection and Analysis
85. Qiao, Y., Yang, Y., Ji, L., and He, J. (2013, January 16–18). Analyzing Malware by Abstracting the Frequent Itemsets in API Call Sequences. Proceedings of the 2013 12th IEEE International Conference on Trust, Security and Privacy in Computing and Communications, Melbourne, Australia.10.1109/TrustCom.2013.36 ↗
86. Deriving Common Malware Behavior through Graph Clustering10.1016/j.cose.2013.09.006 ↗
87. Classification of Malware Based on Integrated Static and Dynamic Features10.1016/j.jnca.2012.10.004 ↗
88. Malware Characteristics and Threats on the Internet Ecosystem10.1016/j.jss.2012.02.015 ↗
89. A Survey of Insider Attack Detection Research10.1007/978-0-387-77322-3_5 ↗
90. DeBarr, D., Ramanathan, V., and Wechsler, H. (2013, January 4–7). Phishing Detection Using Traffic Behavior, Spectral Clustering, and Random Forests. Proceedings of the 2013 IEEE International Conference on Intelligence and Security Informatics, Seattle, WA, USA.10.1109/ISI.2013.6578788 ↗
91. Architectural Mechanisms for Dynamic Changes of Behavior Selection Strategies in Behavior-Based Systems10.1109/TSMCB.2004.837309 ↗
92. Lauf, A.P., Peters, R.A., and Robinson, W.H. Embedded Intelligent Intrusion Detection: A Behavior-Based Approach. Proceedings of the 21st International Conference on Advanced Information Networking and Applications Workshops (AINAW’07), Niagara Falls, ON, Canada.
93. DTB-IDS: An Intrusion Detection System Based on Decision Tree Using Behavior Analysis for Preventing APT Attacks10.1007/s11227-015-1604-8 ↗
94. A Feature Extraction Method of Hybrid Gram for Malicious Behavior Based on Machine Learning
95. Differential Area Analysis for Ransomware Attack Detection within Mixed File Datasets10.1016/j.cose.2021.102377 ↗
96. Ransomware Detection Method Based on Context-Aware Entropy Analysis10.1007/s00500-018-3257-z ↗
97. McIntosh, T., Jang-Jaccard, J., Watters, P., and Susnjak, T. (2019). Proceedings of the International Conference on Neural Information Processing, Sydney, Australia, 12–15 December 2019, Springer.
98. A CNN-Based Encrypted Data Detection for Ransomware Defense10.5626/KTCP.2019.25.5.279 ↗
99. Wavelet Decomposition of Software Entropy Reveals Symptoms of Malicious Code10.1016/j.jides.2016.10.009 ↗
100. Ko, J., and Kwak, J. (2018, January 11). Accuracy Enhancement of Determining File Encryption Status through Divided Shannon Entropy. Proceedings of the Korea Information Processing Society Conference, Jeju, Republic of korea.