Automatic Inspection of Static Application Security Testing (SAST) Reports via Large Language Model Reasoning

Automatic Inspection of Static Application Security Testing (SAST) Reports via Large Language Model Reasoning

연구 분야: Verification

학회: International Conference on AI Logic and Applications

Static Application Security Testing (SAST) tools are widely employed to detect bugs and vulnerabilities in software due to their extensive coverage and independence from their execution environment. However, these tools often produce numerous false positives, necessitating manual inspection and confirmation by developers, which is both arduous and time-consuming. Large Language Models (LLMs) have exhibited superior code semantic understanding, presenting an opportunity to effectively act as human experts in analyzing source code and SAST reports. This paper introduces a versatile and easily extensible approach that leverages LLM reasoning to automatically inspect extensive SAST reports. We developed a GPT-based prototype, named FPShield, to automatically identify and eliminate potential false positives in SAST reports. Our experiments, conducted on the smartbugs-curated dataset containing 207 vulnerabilities across 31 Solidity smart contracts, demonstrate the practical effectiveness of our approach. By integrating FPShield with the prominent SAST tool Slither, we improved its precision from 9.43% to 52.11% and its F1 score from 13.41% to 26.62%. Moreover, FPShield reduced the number of issues reported by Slither by approximately 86%, decreasing from 509 to 71. Our research presents a novel perspective on harnessing the capabilities of LLMs to markedly improve the precision of SAST tools and significantly reduce the manual effort required by developers.