GUARD: Generic API De-obfuscation and Obfuscated Malware Unpacking with sIAT

GUARD: Generic API De-obfuscation and Obfuscated Malware Unpacking with sIAT

연구 분야: Safety

학회: SAC '25: Proceedings of the 40th ACM/SIGAPP Symposium on Applied Computing

Within the field of malware analysis, the application programming interface (API) is pivotal for identifying and understanding threats, thereby enabling the development of effective countermeasures. In particular, API obfuscation presents significant challenges in malware analysis, obscuring the malware's inner operations and hindering effective analysis. Despite the importance of resolving obfuscated API, there exists a notable research gap, as recent efforts have overlooked the challenges posed by API obfuscation. Additionally, previous unpacking studies have not made their executable files and data public, hindering replication and follow-up research. To address this research gap, we propose an emulation-based generic API de-obfuscation and unpacking method, called GUARD. Our method employs an obfuscated call emulation combined with a stack-layout analysis algorithm and a scattered import address table (sIAT), effectively restoring original APIs from packed files. Our evaluations against sophisticated commercial packers, including Themida and VMProtect, demonstrate the method's capability to successfully restore APIs and unpack files previously unaddressed by existing research while improving malware detection rate by as much as 24%.