Semantic-aware dynamic scene tracking and reconstruction for mobile robots leveraging visual SLAM

Semantic-aware dynamic scene tracking and reconstruction for mobile robots leveraging visual SLAM

연구 분야: Strategies

학회: The Journal of Supercomputing

Mobile robots operating in dynamic environments often encounter challenges due to interference from moving objects, compromising the performance of their SLAM systems. To address this issue, we propose a depth camera based semantic-aware visual SLAM approach for mobile robots, aimed at dynamic scene tracking and reconstruction. By leveraging a lightweight Yolact++ network, we segment and detect objects in the scene, eliminate dynamic features through prior knowledge and epipolar constraints, and utilize high-confidence static features for robot’s pose estimation. Subsequently, the overall similarity between images is evaluated based on the similarity of the matched semantic object regions between images to select high-quality candidate keyframes, effectively enhancing the precision of loop closure detection. Furthermore, we employ Delta Generalized Labeled Multi-Bernoulli (Delta-GLMB) filtering to track the masks of prior dynamic objects provided by Yolact++. A global semantic octree map is then constructed, integrating both the semantic knowledge of the static scene and the tracking information of dynamic targets, which in turn realizes the tracking & reconstruction of the dynamic scene. This allows mobile robots to mimic human behavior and cognitive abilities, not merely to construct a map of unknown environments more efficiently, but also to track the state of moving targets in the field-of-view(FoV) in real-time, enabling self-localization and navigation functions. Experimental results demonstrate the effectiveness of our approach in improving the robustness, accuracy, and adaptability of mobile robots in dynamic environments.