Generating Software Tests for Mobile Applications Using Fine-Tuned Large Language Models

Generating Software Tests for Mobile Applications Using Fine-Tuned Large Language Models

연구 분야: Verification

학회: 2024 IEEE/ACM International Conference on Automation of Software Test (AST)

Motivation. Software tests are a necessity in the development of software to secure functionality, reliability, and usability [10]; however, these tests are costly and time-consuming [6]. Although tool support for software testing has advanced, there remains considerable potential for enhancement. Many software tests are still devised manually, with the creation of unit tests being particularly laborious. Automating the generation of test cases is promising for streamlining this aspect of software testing [6].Large Language Models (LLMs) have exhibited capabilities in code generation [11, 13–15], test case generation [17], and various other domains [11]. The advancement of model performance of transformer-based LLMs is mainly achieved by expanding the model size in line with an increase in training data size [7, 8]. However, this approach leads to high computational costs which can only be afforded by corporations with significant financial resources. This highlights the need for transformer-based LLMs that perform well on a specific downstream task and are also cost-efficient. Addressing this, we focused on supervised fine-tuning (SFT) of more resource-efficient transformer-based LLMs LLaMA 2 13B, Code Llama 13B, and Mistral 7B for the specific downstream task of generating test cases for mobile applications.