Turkish Sign Language Recognition Using Kinect Skeleton and Convolutional Neural Network


Unutmaz B., KARACA A. C., GÜLLÜ M. K.

27th Signal Processing and Communications Applications Conference (SIU), Sivas, Türkiye, 24 - 26 Nisan 2019 identifier identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Cilt numarası:
  • Doi Numarası: 10.1109/siu.2019.8806380
  • Basıldığı Şehir: Sivas
  • Basıldığı Ülke: Türkiye
  • Kocaeli Üniversitesi Adresli: Evet

Özet

In this paper, a system that converts Turkish signs to words using convolutional neural networks is presented. Skeleton data obtained by Microsoft's Kinect device is used in the proposed system. User located in front of Kinect sensor makes the signs of corresponding word for a limited time. Afterwards, skeleton joints are extracted. Finally, skeleton points on consecutive frames are merged, and Word classification is performed by convulational neural network. In experimental studies, evaluations are carried out with our own dataset and performance of the proposed method is compared with various classification methods. Moreover, the effect of being closer or further to the camera and movements in different speeds are also investigated. In the lights of the experimental results, it is seen that proposed convolutional neural network gives better performances than other classifiers such as Gaussian SVM, Linear SVM, weighted KNN and decision trees.