Exploring Human Action Recognition for Rehabilitation Game Application

Through computer vision algorithms motion can be computed, which can be a crucial element to be integrated with serious game environments. To evaluate the efficacy of motion detection algorithms, the information of these algorithms can be used to perform Human Action Recognition (HAR). There are several algorithms to perform HAR, although skeleton approaches can be seen as the best way to isolate human motion. To extract the human skeleton representation, the work described in this paper evaluates three distinct methods: OpenPose (2D), YOLO-Pose (2D) and BlazePose (3D). The information translated by the skeleton representations is normalized by lightweight normalization algorithms (for further real-time application). To classify the video sequence and further action identification, a Long Short Term Memory network (LSTM), was used. Using the N-UCLA dataset, the highest F1 score of 0.745 was achieved using OpenPose skeleton extraction (2D), followed by the computation of the angles in each joint, demonstrating that the OpenPose skeleton representation can be the most viable solution for computing human motion in serious games.

Exploring Human Action Recognition for Rehabilitation Game Application Artigo de Conferência