Real-time computer vision-based gestures recognition system for bangla sign language using multiple linguistic features analysis

Real-time computer vision-based gestures recognition system for bangla sign language using multiple linguistic features analysis

연구 분야: Artificial Intelligence

학회: Multimedia Tools and Applications

Dynamic gestures recognition is a challenging task in the computer vision research area and still difficult to incorporate all linguistic features of a sign language in case of recognition. The proposed method has addressed only three linguistic features (hand-shape, position, and movement) among five more features for a real-time computer vision-based gestures recognition system for Bangla sign language (GRS-BdSL). The system uses Normalized Outer Boundary Vector (NOBV) and proposed Binary Window-Grid Vector (BWGV) of binary hand gestures to classify hand-shapes. Hand position is identified by using the proposed model of hand Position Mapping Filter (PMF). In parallel, the system tracks the movement path of hand-shape using the Adaptive Kalman Filter (AKF). After getting those three linguistic features, the system converts these into corresponding encoding patterns which are used to train and test the system. The proposed system recognizes each gesture by measuring the maximum Inter-Correlation Coefficient (ICC) between the encoding patterns of the test and pre-trained gestures. The system is trained and tested for 100 gestures of Bangla sign language (BdSL) achieving a mean recognition accuracy of 95.43% with the computational costs of 56.013 ms/f. We have also compared the performance of the proposed method with existing methods and have demonstrated that the proposed method has outperformed them under a similar experimental setup.