Artificial Neural Networks based Baby Sign Language Recognition via Wearable Sensors


Sevindik E., Ergin E., Erat K., Onay Durdu P.

10th Language & Technology Conference: Human Language Technologies as a Challenge for Computer Science and Linguistics, Poznan, Polonya, 21 - 23 Nisan 2023, ss.261-266

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Basıldığı Şehir: Poznan
  • Basıldığı Ülke: Polonya
  • Sayfa Sayıları: ss.261-266
  • Kocaeli Üniversitesi Adresli: Evet

Özet

Baby sign language (BSL), is a non-verbal language, which is used to communicate between parents and their toddlers. Unlike the sign language used by hearing and speech-impaired individuals, BSL is used by hearing toddlers to communicate their emotions and desires before their verbally speaking period. In this study, a deep learning-based baby sign language recognition research has been conducted using a single Myo armband, which is a wearable technology with surface Electromyography (EMG) sensors and an Inertial Measurement Unit (IMU) sensor. The dataset containing signals from the sensors consisting of six words representing the basic needs of babies are classified using Artificial Neural Networks (ANN). The results show that the proposed method has the highest accuracy, which is 98.23%, when EMG and IMU sensors are used together. In addition, it is observed that only the IMU sensor data can also be sufficient for the selected BSL words’ recognition.