Original Entry Point Detection Based on Graph Similarity

Original Entry Point Detection Based on Graph Similarity

연구 분야: Cryptography

학회: International Symposium on Foundations and Practice of Security

This paper proposes a method for packer identification and OEP (Original Entry Point) detection based on the graph similarity on control flow graphs of packed codes. Packed code consists of an unpacking stub and a packed payload, which is recovered to the original after the unpacking stub executes. In this paper, the CFGs of packed code are generated by a DSE (Dynamic Symbolic Execution) tool BE-PUM on x86-32/Windows. We define the template of the unpacking stub as the pair of the average of Weisfeiler-Lehman histogram vectors and the tail jump sequence. Next, each template is computed packer-wise (i.e., processing packed codes by the same packer) for the ease of covering a new packer. We use the total of 71 samples packed by 12 packers. For unknown packed code, we will find the templates in its CFG generated by BE-PUM. Among them, the CFG fragment with the highest cosine similarity is regarded as the unpacking stub, which also detects the used packer and the OEP as the jump destination from the exit. Our first experiment is performed on 700 non-malware samples (of which the original payload is also known) packed by 12 packers above. The used packer is correctly identified for 689 and the OEP is correctly detected for 688. Further, we apply the method to 1239 malware samples. Among them, 1089 samples are detected packed by unknown packer and among them 150 samples are detected as packed by the 11 packers (except for TELOCK) and their OEPs are detected. We conclude that our method is highly effective as long as we have access to an executable of a target packer to compute its templates.