A 3D Maze Rehabilitation Game for Upper Limb Stroke Recovery Based on MediaPipe and SVM

Abstract

This study addresses key challenges in upper limb rehabilitation for stroke patients, including the dependence on third-party assistance, the high cost of smart devices, and the lack of multisensory stimulation in traditional methods. This paper proposes a 3D maze rehabilitation game system based on gesture recognition. The system is built on a dual architecture consisting of a Python-based gesture recognition endpoint and a Unity-based game interaction endpoint. To reduce costs, the hardware relies on a standard computer camera, eliminating the need for depth sensors. On the software side, the MediaPipe framework is employed to track 21 3D hand keypoints in real-time. Three feature types, relative coordinates, finger angles, and relative distances, are extracted and normalized to mitigate distance and scale interference. Using publicly available datasets and an Support Vector Machine algorithm, this study developed a model to classify 10 distinct gestures. The system ensures cross-platform communication via UDP, and Unity is used to create dual scenes, fully mapping gestures to game controls, enabling patients to interact with the game through hand gestures during rehabilitation training. Experimental results show that the system achieves 100% accuracy at an optimal recognition distance of 25 cm. However, accuracy decreases for complex gestures when the distance increases to 45 cm or in low-light conditions, due to the impact of image quality. This system offers a low-cost, immersive rehabilitation solution for stroke patients and provides insights into the development of intelligent rehabilitation technologies.