DPFedSub: A Differentially Private Federated Learning with Randomized Subspace Descend

DPFedSub: A Differentially Private Federated Learning with Randomized Subspace Descend

연구 분야: Cryptography

학회: Australasian Conference on Information Security and Privacy

Differentially private federated learning (DPFL), which introduces stochastic perturbations to the updated parameters, has been explored as an effective strategy to mitigate additional privacy breaches in ongoing research. However, an inescapable reduction in performance arises when such techniques are implemented and the delicate balance between privacy and performance is under intense scrutiny. To mitigate this performance decline, in this study, we introduce DPFedSub, a novel differentially private federated learning framework that harnesses the benefits of randomized subspace descent optimization. In detail, our methodology and scheme intricately partition the parameter space into a shared common subspace and a discreet private subspace. This common subspace assumes the role of a noise reduction stratum for the privacy subspace. In parallel, our approach encompasses the distribution of lower layers of the network amongst all clients, thereby capturing universally adaptable feature representations. The upper layers of the network are tailored to the specific task for personalized on-client refinement, thereby effectively addressing concerns related to statistical heterogeneity. Moreover, we provide a comprehensive convergence analysis of the formulated scheme and present extensive experimental results derived from five distinct datasets. These findings confirm the ability of the proposed scheme to achieve subtle performance degradation while maintaining a high privacy guarantee.