Enhancing Quranic Learning: A Multimodal Deep Learning Approach for Arabic Phoneme Recognition
APPLIED SCIENCES, cilt.16, sa.19, ss.1-22, 2026 (Hakemli Dergi)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 16 Sayı: 19
- Basım Tarihi: 2026
- Doi Numarası: 10.3390/app16199905
- Dergi Adı: APPLIED SCIENCES
- Derginin Tarandığı İndeksler: Applied Science & Technology Source, Central & Eastern European Academic Source (CEEAS), zbMATH
- Sayfa Sayıları: ss.1-22
- Kocaeli Üniversitesi Adresli: Evet
Özet
Arabic phoneme recognition is relevant to computer-assisted Qur’anic learning, but short isolated recordings provide limited context and subtle acoustic distinctions remain difficult to model. This study presents an intermediate-fusion framework combining UniSpeech acoustic representations with BERT embeddings of Whisper-generated text. Early and staged late fusion are evaluated as alternative integration strategies using two configurations of an existing 1015-recording corpus with 29 target classes. Dataset A uses 783 recordings for training and 232 for testing, whereas Dataset B uses 812 and 203 recordings, respectively. In the reported experiments, the proposed intermediate-fusion model achieves accuracies of 95.69% and 98.03% on Dataset A and Dataset B, respectively. Early fusion reaches 96.55% on Dataset A, showing that the proposed method is not uniformly the numerical leader. On Dataset B, this corresponds to 199 correct predictions, compared with 196 for early fusion and 193 for staged late fusion. Exact paired McNemar tests show no significant difference among the three strategies after Holm correction at the 0.05 level. The contribution is an empirical comparison of fusion strategies using pretrained models, rather than a new transformer architecture. The results support further investigation of audio-derived textual representations for Arabic phoneme recognition, while the small corpus, incomplete speaker metadata, and absence of a separate pronunciation-error evaluation limit conclusions about speaker-independent pronunciation assessment and practical learning outcomes.