Enhanced virtual gesture generation through joint contact representation and contact area alignment

Enhanced virtual gesture generation through joint contact representation and contact area alignment

연구 분야: Strategies

학회: The Visual Computer

Virtual gesture generation, a pivotal technology in virtual reality, faces challenges in accurately capturing and representing the intricate details of hand-object interactions. Existing methods often lack comprehensive contact representations, resulting in inadequate capture of contact process nuances. Furthermore, the alignment of hand and object contact positions during grasp pose generation is frequently overlooked, leading to incoordinated poses. To address these issues, we propose a novel virtual gesture generation approach integrating Joint Contact Representation (HCR) and Contact Area Alignment (CAA). HCR jointly models and represents precise contact positions between the hand and object, hand contact parts, and grasping directions, enhancing the representation of hand-object interactions. The CAA method imposes alignment constraints on contact positions both globally and locally during pose generation, using a global contact distance loss and a local SDF-based matching loss, enhancing grasp pose diversity and coordination. Experimental results demonstrate that our approach reduces penetration volume to 2.60mm on the GRAB dataset and 4.50mm on the HO3D dataset, surpassing state-of-the-art methods. This study underscores the importance of detailed contact modeling and alignment in achieving realistic and diverse virtual grasp poses.