Pipeline of Our Annotation System. The left part sketches the hand pose estimation process in Real World, while the right part sketches the collection process of the dexterous functional grasp pose in Simulation.
Dalian University of Technology
† corresponding author
Pipeline of Our Annotation System. The left part sketches the hand pose estimation process in Real World, while the right part sketches the collection process of the dexterous functional grasp pose in Simulation.
DexFuncGrasp Dataset : A dataset with diverse annotations of dexterous functional grasp, which covers 559 instances across 12 categories and more than 14k grasp poses
Robot grasp dataset is the basis of designing the robot’s grasp generation model. Compared with the building grasp dataset for Low-DOF grippers, it is harder for High-DOF dexterous robot hand. Most current datasets meet the needs of generating stable grasps, but they are not suitable for dexterous hands to complete human-like functional grasp, such as grasp the handle of a cup or pressing the button of a flashlight, so as to enable robots to complete subsequent functional manipulation action autonomously, and there is no dataset with functional grasp pose annotations at present. This paper develops a unique Cost-Effective Real-Simulation Annotation System by leveraging natural hand’s actions. The system is able to capture a functional grasp of a dexterous hand in a simulated environment assisted by human demonstration in real world. By using this system, dexterous grasp data can be collected efficiently as well as cost-effective. Finally, we construct the first dexterous functional grasp dataset with rich pose annotations. A Functional Grasp Synthesis Model is also provided to validate the effectiveness of the proposed system and dataset.
Simulation results in Isaacgym of each category.
Illustration of real experiment.
<@inproceedings{hang2024dexfuncgrasp,
title={DexFuncGrasp: A Robotic Dexterous Functional Grasp Dataset Constructed from a Cost-Effective Real-Simulation Annotation System.},
author={Hang, Jinglue and Lin, Xiangbo and Zhu, Tianqiang and Li, Xuanheng and Wu, Rina and Ma, Xiaohong and Sun, Yi},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
volume={38},
number={9},
pages={10306-10313},
year={2024}
}>
If you have any questions, please feel free to contact Jinglue Hang at 13942210907@163.com.