Scalable Continuous Integration using Remote Execution

Scalable Continuous Integration using Remote Execution

연구 분야: Software Development

학회: 2024 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)

Continuous integration at scale is a common problem for any large project. Multiple dimensions need to be optimized in order to provide a consistent experience for a large set of users. In particular, cost and performance are the typical ends of the spectrum that a CI maintainer needs to balance. Namely, for a non-optimized CI setup it is very tempting to get great performance by simply adding more machines to the fleet, however, maintaining those machines persistently running comes at a high infrastructure cost. Therefore, it is not a surprise that one of the key difficulties of a scalable CI is providing performant builds and tests at a reasonable and predictable cost. At EngFlow, we specialize in optimizing the build and test cycles for large scale organizations for both interactive user builds and CI workflows. A typical scenario is a company that has adopted GitHub actions as their CI solution where they would have dozens or hundreds of workflows. The first step is to optimize the compilation, linking and test times by using the scalability of the cloud. This is accomplished by implementing Remote Execution API which requires that the client machine (e.g. GitHub runner) to break down the steps in a declarative manner (note: this can be easily accomplished by any build system supporting the API such as Bazel). The server can then optimally allocate server resources to execute the workflows, namely, the server can decide to scale up or down the number of machines based on the incoming work. In our most recent work, we have experimented with using the same remote execution infrastructure and optimal allocation to run the CI workflows. This means that CI machines (e.g. GitHub action runners) can be auto-scaled and become cost effective instead of needing to persistently run a set of machines for a given CI workload. In this talk we present our remote execution implementation of CI runners, demonstrate the infrastructure cost savings and discuss recommended migratio... Show More