Main FaaS Providers Behavior Under High Concurrency: An Evaluation with Orama Framework Distributed Architecture

Main FaaS Providers Behavior Under High Concurrency: An Evaluation with Orama Framework Distributed Architecture

연구 분야: Databases

학회: SN Computer Science

With the increasing adoption of Function-as-a-Service (FaaS) oriented solutions, there is a growing interest in tools that facilitate benchmarking in these environments. The Orama framework has emerged as a standout solution, offering high configurability and scalability for provisioning, running benchmarks, and performing statistical analysis. This research presents the use of the distributed architecture of the Orama framework in evaluating the main public FaaS providers in situations of high competition. The experimental results demonstrate the framework’s effectiveness in orchestrating load tests with high levels of concurrency. Notably, the study reveals specific characteristics of the providers involved, with Alibaba Function showing excellent performance, registering the lowest average execution time and remaining free of failures. Google Cloud Function and AWS Lambda provide intermediate results at average runtime and report some failures. Meanwhile, Azure Function performs worst in average execution time and cold start. A deeper investigation into the Alibaba Function provider was conducted to verify if its performance maintains a more complex usage scenario, in this case, a use case merging FaaS and Database-as-a-Service.