Quantum infidelity codistillation for fast and accurate distributed quantum machine learning

Quantum infidelity codistillation for fast and accurate distributed quantum machine learning

연구 분야: Cryptography

학회: The Journal of Supercomputing

This article aims to develop a scalable, accurate, and fast-converging method for knowledge transfer in quantum machine learning (QML), which is essential for reducing QML model sizes to fit within a limited number of qubits. Quantum knowledge distillation (QKD) offers a promising approach that transfers knowledge from a pre-trained large teacher model to a smaller model. However, its dependency on the pre-trained large teacher model restricts its applicability to offline training scenarios and cases where a large number of qubits is already available. To address these limitations, we propose quantum infidelity codistillation (QICD), inspired by co-distillation (CD) in classical machine learning and quantum state fidelity in QML. In QICD, following CD, each student learns from the ensemble knowledge of the other students, eliminating the need for a pre-trained teacher. However, the diversity among student models may lead to excessively high KL divergence, used in traditional CD for regularization by measuring knowledge differences, thus hindering training convergence. To resolve this, QICD employs quantum state infidelity as a bounded regularization measure, ensuring stable training. Using the quantum neural tangent kernel framework, we theoretically prove the asymptotic convergence of QICD and demonstrate that the convergence speed increases with the number of students. By numerical simulation on an image classification task, we show that QICD with four students achieves competitive accuracy compared to a baseline QKD using a pre-trained teacher model. QICD also achieves higher accuracy and faster convergence than a baseline quantum codistillation (QCD) that uses the KL divergence, highlighting the effectiveness of quantum infidelity regularization.