3D Hand Pose Estimation Based on Graph Convolution Network

NAVIGATE

Table of Contents

STATISTICS

Viewed27

Downloads31

3D Hand Pose Estimation Based on Graph Convolution Network

[HTML] PDF下载 (31)

[1]PENG Chunyan,WANG Xuan,CHEN Yangbo,et al.3D Hand Pose Estimation Based on Graph Convolution Network[J].Journal of Zhengzhou University (Engineering Science),2026,47(5):9-16.[doi:10.13705/j.issn.1671-6833.2026.02.013]

Copy

Journal of Zhengzhou University (Engineering Science)[ISSN 1671-6833/CN 41-1339/T] Volume: 47 Number of periods: 2026 Issue 5 Page number: 9-16 Column: Public date: 2026-09-09

Title:: 3D Hand Pose Estimation Based on Graph Convolution Network

Author(s):: PENG Chunyan^1,2, WANG Xuan^1,2, CHEN Yangbo^1,2, HE Gangbo^1,2; 1. College of Computer, Qinghai Normal University, Xining 810016, China; 2. The State Key Laboratory of Tibetan Intelligence, Qinghai Normal University, Xining 810016, China

Keywords:: 3D hand pose estimation; graph convolution networks; feature extraction; optimisation of graph kernel learning; dynamic adjustment of assessment indicators

CLC:: TP391TP751

DOI:: 10.13705/j.issn.1671-6833.2026.02.013

Abstract:: In the task of 3D hand pose estimation from a single image in color, challenges such as occlusion and high self‑similarity of hand parts might lead to large prediction errors and unnatural hand structures. To address these issues, a graph convolution‑based 3D hand pose estimation method was firstly proposed. Visual features and 2D keypoint positions were extracted from the input image using Keypoint R‑CNN. These features were then fed into an improved adaptive kernel graph convolution module (AK_GraFormer). Subsequently, a residual‑connected AKGNN graph kernel was introduced to adaptively process graph‑structured data, thereby enhancing the model’s feature learning and representation. Finally, a dynamic training strategy was employed, which was monitored by a proposed evaluation metric, to optimize estimation performance. Experimental results on the HO‑3D_v3 and FreiHand datasets demonstrated that the proposed method outperformed existing approaches in monocular 3D hand pose estimation. Specifically, the procrustes‑aligned mean per joint position error (PA‑MPJPE) was reduced by up to 12.50 percentage points, and the area under the curve (AUC) of the percentage of correct keypoints metric was improved by up to 3.44 percentage points compared to state‑of‑the‑art methods.

References:: [1] Sridhar S, Feit A M, Theobalt C, et al. Investigating the dexterity of multi‑finger input for mid‑air text entry[C]//Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. New York: ACM, 2015: 3643‑3652.
[2] Oikonomidis I, Kyriazis N, Argyros A. Tracking the articulated motion of two strongly interacting hands[C]//Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2012: 1862‑1869.
[3] Tkach A, Pauly M, Tagliasacchi A. Sphere‑meshes for real‑time hand modeling and tracking[J]. ACM Transactions on Graphics, 2016, 35(6): 1‑11.
[4] Romero J, Tzionas D, Black M J. Embodied hands: modeling and capturing hands and bodies together[J]. ACM Transactions on Graphics, 2017, 36(6): 1‑17.
[5] Pavlakos G, Choutas V, Ghorbani N, et al. Expressive body capture: 3D hands, face, and body from a single image[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2019: 10967‑10977.
[6] Keskin C, Kıraç F, Kara Y E, et al. Hand pose estimation and hand shape classification using multi‑layered randomized decision forests[C]//Proceedings of the 12th European conference on Computer Vision. New York: ACM, 2012: 852‑863.
[7] Tompson J, Stein M, Lecun Y, et al. Real‑time continuous pose recovery of human hands using convolutional networks[J]. ACM Transactions on Graphics, 2014, 33(5): 1‑10.
[8] Pan Xiaoying, Li Shoukun, Wang Hao, et al. LGCANet: lightweight hand pose estimation network based on HRNet[J]. The Journal of Supercomputing, 2024, 80(13): 19351‑19373.
[9] Hoang D C, Xuan Tan P, Pham D L, et al. Efficient multimodal fusion for hand pose estimation with hourglass network[J]. IEEE Access, 2024, 12: 113810‑113825.
[10] Zhan Zhi, Luo Guang. Multiscale feature fusion network for monocular complex hand pose estimation[J]. Electronics Letters, 2023, 59(24): e13044.
[11] Panteleris P, Oikonomidis I, Argyros A. Using a single RGB frame for real time 3D hand pose estimation in the wild[C]//Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). Piscataway: IEEE, 2018: 436‑445.
[12] Doosti B, Naha S, Mirbagheri M, et al. HOPE‑Net: a graph‑based model for hand‑object pose estimation[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2022: 13618‑13627.
[13] Zhao Weixi, Wang Weiqiang, Tian Yunjie. GraFormer: graph‑oriented transformer for 3D pose estimation[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2022: 20406‑20415.
[14] Li Zhixin, Shang Fanqi, Huan Zhan, et al. Human activity recognition based on hybrid feature graph convolutional neural network[J]. Journal of Zhengzhou University (Engineering Science), 2024, 45(4): 46‑52.［李志新，尚繁琦，宦展，等. 基于混合特征图卷积神经网络的人体行为识别方法［J］. 郑州大学学报(工学版), 2024, 45(4):46‑52.］
[15] Cai Yujun, Ge Lihao, Liu Jun, et al. Exploiting spatial‑temporal relationships for 3D pose estimation via graph convolutional networks[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2019: 2272‑2281.
[16] Aboukhaddra A T, Malik J, Robertini N, et al. Shape‑GraFormer: GraFormer‑based network for hand‑object reconstruction from a single depth map[J]. IEEE Access, 2024, 12: 124021‑124031.
[17] Zhuang Nan, Mu Yadong. Joint hand‑object pose estimation with differentiably‑learned physical contact point analysis[C]//Proceedings of the 2021 International Conference on Multimedia Retrieval. New York: ACM, 2021: 420‑428.
[18] Zhang Maomao, Li Ao, Liu Honglei, et al. Coarse‑to‑fine hand‑object pose estimation with interaction‑aware graph convolutional network[J]. Sensors, 2021, 21(23): 8092.
[19] Ma Shenglei, Li Jinghua, Kong Dehui, et al. 3D hand pose estimation based on double branches with multi‑scale attention[J]. Chinese Journal of Computers, 2023, 46(7): 1383‑1395.［马胜蕾，李敬华，孔德慧，等. 基于双分支多尺度注意力的手三维姿态估计［J］. 计算机学报, 2023, 46(7):1383‑1395.］
[20] Yang Wenji, Xie Liping, Qian Wenbin, et al. Coarse‑to‑fine cascaded 3D hand reconstruction based on SSGC and MHSA[J]. The Visual Computer, 2025, 41(1): 11‑24.
[21] He Kaiming, Gkioxari G, Dollár P, et al. Mask R‑CNN[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2017: 2980‑2988.
[22] Ju Mingxuan, Hou Shifu, Fan Yujie, et al. Adaptive kernel graph neural network[C]//Proceedings of the Thirty‑Sixth AAAI Conference on Artificial Intelligence (AAAI‑22). Washington D C: AAAI, 2022:7051‑7058.
[23] Vasconcelos C, Birodkar V, Dumoulin V. Proper reuse of image classification features improves object detection[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2022: 2740‑2750.
[24] Hampali S, Sarkar S D, Lepetit V. HO‑3D_v3: improving the accuracy of hand‑object annotations of the HO‑3D dataset[PP/OL]. V1. arXiv (2021‑07‑02)[2025‑12‑19]. https://doi.org/10.48550/arXiv.2107.00887.
[25] Zimmermann C, Ceylan D, Yang Jimei, et al. Frei‑HAND: a dataset for markerless capture of hand pose and shape from single RGB images[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2019: 813‑822.
[26] Yang Bing, Xu Chuyang, Yao Jinliang, et al. 3D hand pose estimation method based on monocular RGB images[J]. Journal of Zhejiang University (Engineering Science), 2025, 59(1): 18‑26.［杨冰，徐楚阳，姚金良，等. 基于单目RGB图像的三维手部姿态估计方法［J］. 浙江大学学报(工学版), 2025, 59(1):18‑26.］
[27] Chen Yujin, Tu Zhigang, Kang Di, et al. Model‑based 3D hand reconstruction via self‑supervised learning[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2021: 10446‑10455.
[28] Yang Lixin, Li Kailin, Zhan Xinyu, et al. ArtiBoost: boosting articulated 3D hand‑object pose estimation via online exploration and synthesis[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2022: 2740‑2750.
[29] Zhang Hongwen, Tian Yating, Zhang Yuxiang, et al. PyMAF‑X: towards well‑aligned full‑body model regression from monocular images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(10): 12287‑12303.
[30] Duran E, Kocabas M, Choutas V, et al. HMP: hand motion priors for pose and shape estimation from video[C]//Proceedings of the 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Piscataway: IEEE, 2024: 6341‑6351.
[31] Chen Ping, Chen Yujin, Yang Dong, et al. I2UV‑Hand‑Net: image‑to‑UV prediction network for accurate and high‑fidelity 3D hand mesh modeling[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2021: 12909‑12918.
[32] Lin K, Wang Lijuan, Liu Zicheng. Mesh graphormer[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2021: 12919‑12928.
[33] Liu Zuyan, Lin Gaojie, Wang Congyi, et al. HandMIM: pose‑aware self‑supervised learning for 3D hand mesh estimation[PP/OL]. V1. arXiv (2023‑07‑29)[2025‑12‑19]. https://doi.org/10.48550/arXiv.2307.16061.
[34] Pavlakos G, Shan Dandan, Radosavovic I, et al. Reconstructing hands in 3D with transformers[C]//Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2024: 9826‑9836.

Similar References:

Memo

Last Update: 2026-09-03