MemCatcher: An In-Depth Analysis Approach to Detect In-Memory Malware
Source abstract
Abstract
Recent advancements in cyber threats have led to increasingly sophisticated attack methods that evade traditional malware detection systems. In-memory malware, a particularly challenging variant, operates by modifying volatile memory, leaving minimal traces on secondary storage. This paper presents an in-depth analysis of in-memory malware characteristics, behavior, and evasion strategies. We propose “MemCatcher”, a novel detection algorithm that integrates real-time system activity monitoring and memory analysis to effectively identify these threats from the Windows 10 system. Experimental validation using real-world and synthetic in-memory malware samples demonstrates the effectiveness of our approach. Additionally, we analyze evasion tactics using “Volatility3” and “PEview”, providing insights into countermeasures. Future work will focus on enhancing in-memory malware detection using “Processor-in-Memory (PIM) hardware”.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Security and Verification in Computing · Advanced Malware Detection Techniques · Digital and Cyber Forensics
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science, Hanyang University, Seoul 04763, Republic of Korea
Funding
IITP Korea
References (47)
Show source-provided references
1. Kumar, A., Choi, B.J., Kuppusamy, K., and Aghila, G. (2021). Malware Attacks: Dimensions, Impact, and Defenses. Advances in Nature-Inspired Cyber Security and Resilience, Springer.10.1007/978-3-030-90708-2_9 ↗
2. Fileless malware and programmatic method of detection
3. Bucevschi, A.G., Balan, G., and Prelipcean, D.B. (2019, January 4–7). Preventing file-less attacks with machine learning techniques. Proceedings of the 2019 21st International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), Timisoara, Romania.10.1109/SYNASC49474.2019.00042 ↗
4. Bajpai, P., and Enbody, R. (2020, January 15–19). Memory Forensics Against Ransomware. Proceedings of the 2020 International Conference on Cyber Security and Protection of Digital Services (Cyber Security), Dublin, Ireland.10.1109/CyberSecurity49315.2020.9138853 ↗
5. Beaman, C., Barkworth, A., Akande, T.D., Hakak, S., and Khan, M.K. (2021). Ransomware: Recent advances, analysis, challenges and future research directions. Comput. Secur., 111.10.1016/j.cose.2021.102490 ↗
6. Sanjay, B., Rakshith, D., Akash, R., and Hegde, V.V. (2018, January 20–22). An approach to detect fileless malware and defend its evasive mechanisms. Proceedings of the 2018 3rd International Conference on Computational Systems and Information Technology for Sustainable Solutions (CSITSS), Bengaluru, India.10.1109/CSITSS.2018.8768769 ↗
7. Saad, S., Mahmood, F., Briguglio, W., and Elmiligi, H. (2019, January 26–28). Jsless: A tale of a fileless javascript memory-resident malware. Proceedings of the Information Security Practice and Experience: 15th International Conference, ISPEC 2019, Kuala Lumpur, Malaysia. Proceedings 15.10.1007/978-3-030-34339-2_7 ↗
8. Fileless cyberattacks: Analysis and classification10.4218/etrij.2020-0086 ↗
9. and Kumar, S. (2020). An emerging threat Fileless malware: A survey and research challenges. Cybersecurity, 3.10.1186/s42400-019-0043-x ↗
10. (2023, February 07). Intezer. Memory Analysis Forensics Tools: A Practical Guide, 2021, Available online: https://intezer.com/blog/memory-analysis-forensic-tools/.
11. Shah, S.S.H., Ahmad, A.R., Jamil, N., and Khan, A.u.R. (2022). Memory Forensics-Based Malware Detection Using Computer Vision and Machine Learning. Electronics, 11.10.3390/electronics11162579 ↗
12. Dener, M., Ok, G., and Orman, A. (2022). Malware Detection Using Memory Analysis Data in Big Data Environment. Appl. Sci., 12.10.3390/app12178604 ↗
13. Shah, S.S.H., Jamil, N., and Khan, A.u.R. (2022). Memory Visualization-Based Malware Detection Technique. Sensors, 22.10.3390/s22197611 ↗
14. Sihwail, R., Omar, K., Zainol Ariffin, K.A., and Al Afghani, S. (2019). Malware Detection Approach Based on Artifacts in Memory Image and Dynamic Execution. Appl. Sci., 9.10.3390/app9183680 ↗
15. Hamid, I., and Rahman, M.H. (2024). A comprehensive literature review on volatile memory forensics. Electronics, 13.10.3390/electronics13153026 ↗
16. Alshamsi, O., Shaalan, K., and Butt, U. (2024). Towards securing smart homes: A systematic literature review of malware detection techniques and recommended prevention approach. Information, 15.10.3390/info15100631 ↗
17. Berrios, S., Leiva, D., Olivares, B., Allende-Cid, H., and Hermosilla, P. (2025). Systematic review: Malware detection and classification in cybersecurity. Appl. Sci., 15.10.3390/app15147747 ↗
18. Obfuscated privacy malware classifiers based on memory dumping analysis10.1109/ACCESS.2024.3358840 ↗
19. Owoh, N., Adejoh, J., Hosseinzadeh, S., Ashawa, M., Osamor, J., and Qureshi, A. (2024). Malware detection based on api call sequence analysis: A gated recurrent unit–generative adversarial network model approach. Future Internet, 16.10.3390/fi16100369 ↗
20. Mele Pottaraikkal, S., and Sujeer Sugatha, A. (2023, January 11–12). Effectiveness of Multiple Memory-Images in detecting Fileless Malware. Proceedings of the 2023 11th International Symposium on Digital Forensics and Security (ISDFS), Chattanooga, TN, USA.10.1109/ISDFS58141.2023.10131728 ↗
21. Nair, S.J., and Syam, S.R. (2024, January 24–28). Automated Malware Detection Using Memory Forensics. Proceedings of the 2024 15th International Conference on Computing Communication and Networking Technologies (ICCCNT), Kamand, India.10.1109/ICCCNT61001.2024.10725096 ↗
22. Leng, T., Pan, Y., Zhao, L., Yu, A., Zhu, Z., Cai, L., and Meng, D. (2023, January 1–3). MemInspect: Memory Forensics for investigating Fileless Attacks. Proceedings of the 2023 IEEE 22nd International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), Exeter, UK.10.1109/TrustCom60117.2023.00134 ↗
23. Novak, P., Oujezsky, V., Kaura, P., Horvath, T., and Holik, M. (2024). Multistage malware detection method for backup systems. Technologies, 12.10.3390/technologies12020023 ↗
24. Dora, J.R., Hluchỳ, L., and Staňo, M. (2025). In-Memory Shellcode Runner Detection in Internet of Things (IoT) Networks: A Lightweight Behavioral and Semantic Analysis Framework. Sensors, 25.10.3390/s25175425 ↗
25. Naeem, H., Dong, S., Falana, O.J., and Ullah, F. (2023). Development of a deep stacked ensemble with process based volatile memory forensics for platform independent malware detection and classification. Expert Syst. Appl., 223.10.1016/j.eswa.2023.119952 ↗
26. Bozkir, A.S., Tahillioglu, E., Aydos, M., and Kara, I. (2021). Catch them alive: A malware detection approach through memory forensics, manifold learning and computer vision. Comput. Secur., 103.10.1016/j.cose.2020.102166 ↗
27. Hardware-assisted MMU redirection for in-guest monitoring and API profiling10.1109/TIFS.2020.2969514 ↗
28. Palutke, R., Block, F., Reichenberger, P., and Stripeika, D. (2020). Hiding process memory via anti-forensic techniques. Forensic Sci. Int. Digit. Investig., 33.10.1016/j.fsidi.2020.301012 ↗
29. Lyles, S., Desantis, M., Donaldson, J., Gallegos, M., Nyholm, H., Taylor, C., and Monteith, K. (2022, January 27–30). Machine Learning Analysis of Memory Images for Process Characterization and Malware Detection. Proceedings of the 2022 52nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W), Baltimore, MD, USA.10.1109/DSN-W54100.2022.00035 ↗
30. Carrier, T., Victor, P., Tekeoglu, A., and Lashkari, A.H. (2022, January 9–11). Detecting Obfuscated Malware using Memory Feature Engineering. Proceedings of the Icissp, Online.10.5220/0010908200003120 ↗
31. Lashkari, A.H., Li, B., Carrier, T.L., and Kaur, G. (2021, January 18–19). Volmemlyzer: Volatile memory analyzer for malware classification using feature engineering. Proceedings of the 2021 Reconciling Data Analytics, Automation, Privacy, and Security: A Big Data Challenge (RDAAPS), Hamilton, ON, Canada.10.1109/RDAAPS48126.2021.9452028 ↗
32. Yücel, Ç., and Koltuksuz, A. (2020). Imaging and evaluating the memory access for malware. Forensic Sci. Int. Digit. Investig., 32.10.1016/j.fsidi.2019.200903 ↗
33. Varlioglu, S., Elsayed, N., ElSayed, Z., and Ozer, M. (April, January 26). The dangerous combo: Fileless malware and cryptojacking. Proceedings of the SoutheastCon 2022, Mobile, AL, USA.10.1109/SoutheastCon48659.2022.9764043 ↗
34. Panker, T., and Nissim, N. (2021). Leveraging malicious behavior traces from volatile memory using machine learning methods for trusted unknown malware detection in Linux cloud environments. Knowl.-Based Syst., 226.10.1016/j.knosys.2021.107095 ↗
35. Botacin, M., Grégio, A., and Alves, M.A.Z. (October, January 28). Near-memory & in-memory detection of fileless malware. Proceedings of the International Symposium on Memory Systems, Washington, DC, USA.
36. Exterro (2023, May 18). FTK Imager. FTk Imager Software 2023. Available online: https://www.exterro.com/ftk-imager.
37. Memory forensics tools: A comparative analysis10.1080/23742917.2022.2100036 ↗
38. Volatility (2023, June 10). Volatility Foundation. Volatility 3 2020. Available online: https://www.volatilityfoundation.org/3.
39. Arfeen, A., Asim Khan, M., Zafar, O., and Ahsan, U. (2022). Process based volatile memory forensics for ransomware detection. Concurr. Comput. Pract. Exp., 34.10.1002/cpe.6672 ↗
40. Visualization in testing a volatile memory forensic tool10.1016/j.diin.2011.05.006 ↗
41. Memory forensic challenges under misused architectural features10.1109/TIFS.2018.2819119 ↗
42. Live memory forensics of mobile phones10.1016/j.diin.2010.05.010 ↗
43. Radburn, W.J. (2023, July 11). PEviewer. Software by Wayne J. Radburn; Utilities, 2022; p. 1. Available online: http://wjradburn.com/software/.
44. Kerns, Q., Payne, B., and Abegaz, T. (2021, January 28–29). Double-extortion ransomware: A technical analysis of maze ransomware. Proceedings of the Future Technologies Conference (FTC) 2021, Volume 3, Vancouver, BC, Canada.10.1007/978-3-030-89912-7_7 ↗
45. Al-Sofyani, S., Alelayani, A., Al-zahrani, F., and Monshi, R. (2023, January 23–25). A Survey of Malware Forensics Analysis Techniques and Tools. Proceedings of the 2023 1st International Conference on Advanced Innovations in Smart Cities (ICAISC), Jeddah, Saudi Arabia.10.1109/ICAISC56366.2023.10085474 ↗
46. Prajapati, M., and Dave, D. (2019, January 1–3). Host-based Forensic Artefacts of Botnet Infection. Proceedings of the 2019 International Carnahan Conference on Security Technology (ICCST), Chennai, India.10.1109/CCST.2019.8888438 ↗
47. Andri Rai, E.G.I. (2025, November 02). In-Memory-Malware-Detecion, Available online: https://github.com/Freaky-Coder07/In-Memory-Malware-Detecion-.