1. Home
  2. Vol 4, No 2 (2023)
  3. Wang

A Study of Dance Movement Capture and Posture Recognition Method Based on Vision Sensors

Qun Wang, Gang Tong, Sichao Zhou

Abstract


With the development of technology, posture recognition methods have been applied in more and more fields. However, there is relatively little research on posture recognition in dance. Therefore, this paper studied the capture and posture recognition of dance movements to understand the usability of the proposed method in dance posture recognition. Firstly, the Kinect V2 visual sensor was used to capture dance movements and obtain human skeletal joint data. Then, a three-dimensional convolutional neural network (3D CNN) model was designed by fusing joint coordinate features with joint velocity features as general features for recognizing different dance postures. Through experiments on NTU60 and self-built dance datasets, it was found that the 3D CNN performed best with a dropout rate of 0.4, a ReLU activation function, and fusion features. Compared to other posture recognition methods, the recognition rates of the 3D CNN on CS and CV in NTU60 were 88.8% and 95.3%, respectively, while the average recognition rate on the dance dataset reached 98.72%, which was higher than others. The experimental results demonstrate the effectiveness of our proposed method for dance posture recognition, providing a new approach for posture recognition research and making contributions to the inheritance of folk dances.

 

Doi: 10.28991/HIJ-2023-04-02-03

Full Text: PDF


Keywords


Vision Sensor; Dance; Movement Capture; Gesture Recognition; Kinect V2.

References


Senthil Murugan, A., Suganya Devi, K., Sivaranjani, A., & Srinivasan, P. (2018). A study on various methods used for video summarization and moving object detection for video surveillance applications. Multimedia Tools and Applications, 77(18), 23273–23290. doi:10.1007/s11042-018-5671-8.

Fuggle, N. R., Lu, S., Breasail, M. Ó., Westbury, L. D., Ward, K. A., Dennison, E., Mahmoodi, S., Niranjan, M., & Cooper, C. (2022). OA22 Machine learning and computer vision of bone microarchitecture can improve the fracture risk prediction provided by DXA and clinical risk factors. Rheumatology, 61(Supplement_1). doi:10.1093/rheumatology/keac132.022.

Mahbubur Rahman, M., & Gurbuz, S. Z. (2021). Multi-Frequency RF Sensor Data Adaptation for Motion Recognition with Multi-Modal Deep Learning. 2021 IEEE Radar Conference (RadarConf21). doi:10.1109/radarconf2147009.2021.9455204.

Moreau, P., Durand, D., Bosche, J., & Lefranc, M. (2020). A motion recognition algorithm using polytopic modeling. 2020 7th International Conference on Control, Decision and Information Technologies (CoDIT). doi:10.1109/codit49905.2020.9263883.

Lee, K.-T., Yoon, H., & Lee, Y.-S. (2018). Implementation of smartwatch user interface using machine learning based motion recognition. 2018 International Conference on Information Networking (ICOIN). doi:10.1109/icoin.2018.8343229.

Balmik, A., Paikaray, A., Jha, M., & Nandy, A. (2022). Motion recognition using deep convolutional neural network for Kinect-based NAO teleoperation. Robotica, 40(9), 3233–3253. doi:10.1017/S0263574722000169.

Pribadi, T. W., & Shinoda, T. (2020). Hand Motion Recognition of Shipyard Welder Using 9-DOF Inertial Measurement Unit and Multi-Layer Perceptron Approach. IOP Conference Series: Earth and Environmental Science, 557(1). doi:10.1088/1755-1315/557/1/012009.

Rotoni, G. M., Unabia, S. A., & Villaverde, J. F. (2020). Wireless Accelerometer-based Motion Recognition Sensors for Limb Movement Analysis in Babies. Proceedings of the 2020 10th International Conference on Biomedical Engineering and Technology. doi:10.1145/3397391.3397399.

Asmaul Husna, R., Achmad, A., Ilham, A. A., Zainuddin, Z., & Jaya, A. K. (2020). Early Childhood Gymnastic Motion Recognition System Using Image Processing Technology. 2020 27th International Conference on Telecommunications (ICT). doi:10.1109/ict49546.2020.9239482.

Li, J., Zhu, K., & Pan, L. (2022). Wrist and finger motion recognition via M-mode ultrasound signal: A feasibility study. Biomedical Signal Processing and Control, 71, 103112. doi:10.1016/j.bspc.2021.103112.

Liu, Q. (2022). Human motion state recognition based on MEMS sensors and Zigbee network. Computer Communications, 181, 164–172. doi:10.1016/j.comcom.2021.10.018.

Kurban, O. C., Calik, N., & Yildirim, T. (2022). Human and action recognition using adaptive energy images. Pattern Recognition, 127, 108621. doi:10.1016/j.patcog.2022.108621.

Ding, C., Wen, S., Ding, W., Liu, K., & Belyaev, E. (2022). Temporal segment graph convolutional networks for skeleton-based action recognition. Engineering Applications of Artificial Intelligence, 110, 104675. doi:10.1016/j.engappai.2022.104675.

Li, J., Zhao, X., Zhou, G., & Zhang, M. (2022). Standardized use inspection of workers’ personal protective equipment based on deep learning. Safety Science, 150. doi:10.1016/j.ssci.2022.105689.

Lee, T. J., Kim, C. H., & Cho, D. I. D. (2019). A Monocular Vision Sensor-Based Efficient SLAM Method for Indoor Service Robots. IEEE Transactions on Industrial Electronics, 66(1), 318–328. doi:10.1109/TIE.2018.2826471.

Ayed, I., Jaume-I-capó, A., Martínez-Bueso, P., Mir, A., & Moyà-Alcover, G. (2021). Balance measurement using microsoft kinect v2: Towards remote evaluation of patient with the functional reach test. Applied Sciences (Switzerland), 11(13), 6073. doi:10.3390/app11136073.

Akhtar, M. B. (2022). The use of a convolutional neural network in detecting soldering faults from a printed circuit board assembly. HighTech and Innovation Journal, 3(1), 1-14. doi:10.28991/HIJ-2022-03-01-01.

Kurdthongmee, W., Kurdthongmee, P., Suwannarat, K., & Kiplagat, J. K. (2022). A YOLO Detector Providing Fast and Accurate Pupil Center Estimation using Regions Surrounding a Pupil. Emerging Science Journal, 6(5), 985-997. doi:10.28991/ESJ-2022-06-05-05.

Basak, H., Kundu, R., Singh, P. K., Ijaz, M. F., Woźniak, M., & Sarkar, R. (2022). A union of deep learning and swarm-based optimization for 3D human action recognition. Scientific Reports, 12(1). doi:10.1038/s41598-022-09293-8.

Yan, S., Xiong, Y., & Lin, D. (2018). Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). doi:10.1609/aaai.v32i1.12328.

Si, C., Chen, W., Wang, W., Wang, L., & Tan, T. (2019). An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/cvpr.2019.00132.

Shi, L., Zhang, Y., Cheng, J., & Lu, H. (2019). Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/cvpr.2019.01230.

Zhang, P., Lan, C., Zeng, W., Xing, J., Xue, J., & Zheng, N. (2020). Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/cvpr42600.2020.00119.


Full Text: PDF

DOI: 10.28991/HIJ-2023-04-02-03

Refbacks

  • There are currently no refbacks.


Copyright (c) 2023 Qun Wang, Gang Tong, Sichao Zhou