Human Pose Prediction Based on LSTM and MediaPipe
DOI:
https://doi.org/10.61173/4vyed650Keywords:
Unity3D, Mediapipe, LSTM, UCF101, YoYo, Deep learning, Motion CaptureAbstract
Motion capture technology is a crucial digital animation production method in the fields of film and television and games. Unmarked motion capture technology has received increasing attention and wide application in recent years due to its ability to significantly reduce the time for data collection and processing. However, this technology is prone to generating incorrect human posture information when dealing with occlusion issues. Therefore, this paper attempts to combine Long Short-Term Memory networks (LSTM) for human posture prediction to correct or eliminate incorrect posture data, thereby enhancing the accuracy and reliability of motion capture.
References
[1] Liu, S., Zhang, J., Zhang, Y. et al. A wearable motion capture device able to detect dynamic motion of human limbs. Nat Commun 11, 5615 (2020). https://doi.org/10.1038/s41467-020- 19424-2
[2] [2]Robert M. Kanko, Elise K. Laende, Elysia M. Davis, W. Scott Selbie, Kevin J. Deluzio,Concurrent assessment of gait kinematics using marker-based and markerless motion capture,Journal of Biomechanics,Volume 127,(2021),110665,ISSN 0021-9290,https://doi.org/10.1016/ j.jbiomech.2021.110665.
[3] Takeda, I., Yamada, A., & Onodera, H. (2020). Artificial Intelligence-Assisted motion capture for medical applications: a comparative study between markerless and passive marker motion capture. Computer Methods in Biomechanics and Biomedical Engineering, 24(8), 864–873. https://doi.org/10.108 0/10255842.2020.1856372
[4] Robert M. Kanko, Elise Laende, W. Scott Selbie, Kevin J. Deluzio,Inter-session repeatability of markerless motion capture gait kinematics,Journal of Biomechanics,Volume 121,2021,110422,ISSN 0021-9290,https://doi.org/10.1016/ j.jbiomech.2021.110422.
[5] Wade L, Needham L, McGuigan P, Bilzon J. 2022. Applications and limitations of current markerless motion capture methods for clinical gait biomechanics. PeerJ 10:e12995 https://doi.org/10.7717/peerj.12995
[6] Soshi Shimada, Vladislav Golyanik, Weipeng Xu, and Christian Theobalt. 2020. PhysCap: physically plausible monocular 3D motion capture in real time. ACM Trans. Graph. 39, 6, Article 235 (December 2020), 16 pages. https://doi. org/10.1145/3414685.3417877
[7] [10]L. Malekian and R. Lapeer, “Self-Occluded Human Pose Recovery in Monocular Video Motion Capture,” 2024 14th International Conference on Pattern Recognition Systems (ICPRS), London, United Kingdom, 2024, pp. 1-6, doi: 10.1109/ICPRS62101.2024.10677815. keywords: {Optical filters;Measurement;Three-dimensional displays;Motion estimation;Pose estimation;Predictive models;Cameras;human pose estimation;self-occlusion;single view video;SMPL model;machine learning;deep learning},
[8] Chenchu Xu, Zhifan Gao, Heye Zhang, Shuo Li, Victor Hugo C. de Albuquerque,Video salient object detection using dualstream spatiotemporal attention,Applied Soft Computing,Volume 108,2021,107433,ISSN 1568-4946,https://doi.org/10.1016/ j.asoc.2021.107433.
[9] Kim, J.-W.; Choi, J.-Y.; Ha, E.-J.; Choi, J.-H. Human Pose Estimation Using MediaPipe Pose and Optimization Method Based on a Humanoid Model. Appl. Sci. 2023, 13, 2700. https:// doi.org/10.3390/app13042700
[10] L. Xie and X. Guo, “Object Detection and Analysis of Human Body Postures Based on TensorFlow,” 2019 IEEE International Conference on Smart Internet of Things (SmartIoT), Tianjin, China, 2019, pp. 397-401, doi: 10.1109/ SmartIoT.2019.00070. keywords: {object detection;body postures;TensorFlow;Faster R-CNN},
[11] Kim, J.-W.; Choi, J.-Y.; Ha, E.-J.; Choi, J.-H. Human Pose Estimation Using MediaPipe Pose and Optimization Method Based on a Humanoid Model. Appl. Sci. 2023, 13, 2700. https:// doi.org/10.3390/app13042700
[12] Doan T N. An efficient patient activity recognition using LSTM network and high-fidelity body pose tracking[J]. International Journal of Advanced Computer Science and Applications, 2022, 13(8).
[13] Shrestha M, Pandey S P. Human action recognition using deep learning methods[C]//Proceedings of the International Conference on Machine Learning and Data Engineering. Berlin/ Heidelberg, Germany: Springer Nature, 2023: 345-356.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 by the authors.
This work is licensed under a Creative Commons Attribution 4.0 International License.
