Fine-Tuning Open-Weight LLMs for Connected Health Management: A Case Study for Diabetes Management
2026 IEEE International Black Seas Conference on Communications and Networking, BlackSeaCom 2026, Bucharest, Romanya, 8 - 11 Haziran 2026, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Doi Numarası: 10.1109/blackseacom69760.2026.11598306
- Basıldığı Şehir: Bucharest
- Basıldığı Ülke: Romanya
- Anahtar Kelimeler: Catastrophic Forgetting, Diabetes Management, Google Gemma, Low-Resource Languages, Medical Informatics, QLoRA, Synthetic Data Generation, Turkish Natural Language Processing
- Kocaeli Üniversitesi Adresli: Evet
Özet
Diabetes management requires individuals to have access to accurate medical information as well as to continuously self-monitor. Although Large Language Models (LLMs) have significant potential as digital health assistants, their roles in low-resource languages such as Turkish remain limited due to training data typically being English-heavy. This study investigates the fine-tuning of the Google Gemma 3 (12B) model for Turkish diabetes management using the Parameter-Efficient Fine-Tuning (PEFT) approach and the QLoRA method. The dataset was created using synthetic data generated via the Gemini Pro API and content compiled from various medical forums. Experimental results show that aggressive fine-tuning strategies lead to substantial catastrophic forgetting. During this process, the model lost a great deal of linguistic consistency in Turkish while attempting to internalize medical patterns. Furthermore, it was observed that excessive dependence on synthetic data increased hallucinations and clinical inconsistencies. According to the findings, traditional fine-tuning approaches in medical domains and low-resource languages may weaken the model's reasoning capabilities. For this reason, transitioning to the Retrieval-Augmented Generation (RAG) architecture in healthcare applications is recommended to increase clinical reliability and minimize the risk of hallucinations.