On the Properness of Large Language Models for Malware Detection

On the Properness of Large Language Models for Malware Detection

---

연구 분야: Safety

---

---

학회: International Conference on Risks and Security of Internet and Systems

---

초록

The last couple of years were marked by the rise of LLMs (Large Language Models) that are shown to bring important developments in various sectors such as technology, education or customer service. Generative AI also impacts other areas such as content generation, data analysis, and can be used to automate various systems. From a security point of view, initial applications of LLMs improve customer support services - be it direct customer support or security operation centers - and also the way an IT administrator connects with SIEM (Security Information and Event Management) or XDR (Extended Detection and Response) products. However, there were fewer practical attempts to use LLMs to improve the detection rate or a security product at large. The current paper explores this direction by analyzing the detection rate of several LLMs (ChatGPT, LLama, Mistral, Orca2, Falcon) against a recent set of macro VBA (Visual Basic for Application) malware. The respective performance levels are discussed. Other relevant issues, such as inference cost, memory and space requirements and practicality from a security product point of view, are also considered. Even if the models used in our research do not reach adequate results in terms of detection rate they can still be used in security operation centers to assist a security researcher to analyse malicious content. Furthermore, additional fine-tuning might improve the accuracy of such models to a point where they may become a good choice for various security tasks.

---