A study of two-branch fusion network model for low-quality deepfake detection in face videos

A study of two-branch fusion network model for low-quality deepfake detection in face videos

연구 분야: Cryptography

학회: Signal, Image and Video Processing

Existing deepfake detection methods frequently struggle to achieve reliable results when processing low-quality facial videos, primarily due to artifacts, noise, and other interfering factors. To address this issue, we propose a dual-branch fusion-network model based on physiological features (DPFFNet). The DPFFNet exploits spatio-temporal inconsistencies while emphasizing fine-grained extraction of facial lip motion features. This design significantly enhances the integrity and accuracy of lip motion feature representation, facilitating the capture of potential subtle anomalies in forgery videos and providing a robust foundation for subsequent forgery detection. The architecture incorporates an innovative multi-scale traceblock module (TBM) that efficiently captures local and global texture features from facial videos. This module fully exploits the complementarity among physiological features at different scales. Through multi-scale feature fusion, the model can focus on both local details and the overall structure, thereby further improving detection accuracy. Moreover, the DPFFNet model incorporates the dynamic fusion module (DFM) to more effectively fuse local and global physiological features. This module utilizes backpropagation to dynamically generate and refine the final feature representation, which can significantly enhance the adaptability of the model to multiscale features while optimizing the utilization efficiency of global information. This enables the model to demonstrate stronger robustness when handling complex and variable-forgery videos, especially those of different qualities and types, and allows for flexible adjustment of the feature fusion strategy to achieve more precise detection. To validate the effectiveness of the DPFFNet model comprehensively, we conducted numerous validation experiments on several widely used datasets, including FaceForensics++, DFDC, CelebDF, and WildDeepfake. The experimental results demonstrate that compared with existing methods, the DPFFNet model achieves significant improvements in several evaluation metrics, such as Acc and Auc, when detecting low-quality deepfake face videos. This indicates excellent detection performance and outstanding generalization ability.