A highly efficient distributed database knobs tuning method based on multi-agent deep reinforcement learning

A highly efficient distributed database knobs tuning method based on multi-agent deep reinforcement learning

연구 분야: Databases

학회: The Journal of Supercomputing

The performance of distributed databases critically depends on effective configuration tuning, with knob tuning playing a pivotal role. However, most existing methods predominantly adopt single-agent reinforcement learning approaches, which fail to capture the complex interactions among database nodes. This limitation leads to environmental non-stationarity and often results in suboptimal local solutions. To address these challenges, we propose a general multi-agent deep reinforcement learning (MARL) architecture for distributed database knob tuning. Specifically, we employ multi-agent deep deterministic policy gradient (MADDPG) with a centralized policy gradient estimator, termed C-MADDPG, and formulate the tuning problem as a decentralized partially observable Markov decision process (Dec-POMDP). This formulation explicitly models both cooperative and competitive interactions among nodes through a carefully designed reward function, enabling comprehensive exploration of the knob space while maintaining system-wide coordination. Experimental evaluations on both MySQL NDB Cluster and TiDB Cluster demonstrate that MARL-based tuning methods significantly outperform single-agent baselines. Notably, FACMAC achieves superior performance in read-intensive workloads, albeit with higher training time, while C-MADDPG is more effective for write-intensive workloads, offering a better trade-off between training efficiency and tuning accuracy. The proposed approach has good generalization capabilities across different distributed database architectures and workload types.