Dynamic multi-stream graph neural networks for efficient interactive action recognition

Dynamic multi-stream graph neural networks for efficient interactive action recognition

연구 분야: Artificial Intelligence

학회: The Visual Computer

Interactive action recognition aims to automatically recognize and understand behaviors involving interactions between humans and objects. In this paper, we propose a multi-stream graph neural network approach that efficiently recognizes interactive actions from skeletal motion sequences. Our method consists of three targeted deep neural networks: the Priori Interaction Graph Convolutional Network (PI-GCN), the Dynamic Interaction Hypergraph Convolutional Network (DIH-GCN), and the Global Interaction Attention (GI-Attention). PI-GCN improves feature aggregation by incorporating potential interactions into the skeletal adjacency matrix. DIH-GCN dynamically constructs hypergraphs using the K-nearest neighbor algorithm to capture local interaction features. GI-Attention integrates attention computation across spatial and temporal dimensions to capture global interaction features. Experiments on the NTU-RGB+D 120 interaction dataset demonstrate the superiority of our model, achieving state-of-the-art performance. Our work highlights the importance of modeling dynamic interaction information for effective interactive action recognition. The source code is accessible via https://github.com/HENRYDJ520/DM-GCN.