Evaluation of Static Analysis and Transformer-Based LLMs for IoT Firmware Security

Evaluation of Static Analysis and Transformer-Based LLMs for IoT Firmware Security

연구 분야: Analysis

학회: 2025 21st International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT)

The widespread adoption of Internet of Things (IoT) devices has heightened the need for robust firmware security measures. Static analysis tools, such as the Firmware Analysis and Comparison Tool (FACT), automatically detect vulnerabilities using predefined rules and signatures. However, these rules must be manually updated over time to remain effective. In parallel, transformer-based Large Language Models (LLMs) can capture contextual relationships within code and metadata, enabling the detection of novel threats without predefined signatures. This paper presents a comprehensive comparison of these two approaches by detailing independent research environments, curated datasets, and performance metrics including detection accuracy, scalability, energy consumption, and environmental impact. FACT achieved a detection accuracy of 93%, effectively identifying well-documented vulnerabilities, while LLMs demonstrated a slightly higher accuracy of 96%, leveraging contextual reasoning to uncover novel threats. Additionally, the energy consumption analysis revealed that FACT operates with lower power requirements, whereas LLMs, despite higher energy usage, can be powered entirely by renewable energy sources. The ease of deployment and maintenance for both systems was evaluated, highlighting FACT's straightforward setup versus LLMs' more complex infrastructure needs. The results underscore the complementary roles of static analysis and LLMs in addressing both well-documented and emerging IoT vulnerabilities, providing valuable insights for practitioners seeking to secure diverse firmware ecosystems sustainably and efficiently.