A Confidence-Aware Hybrid Vision–Language Framework for Food Recognition and Nutritional Monitoring
Nutrients, cilt.18, sa.15, 2026 (SCI-Expanded, Scopus)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 18 Sayı: 15
- Basım Tarihi: 2026
- Doi Numarası: 10.3390/nu18152449
- Dergi Adı: Nutrients
- Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, BIOSIS, CAB Abstracts, CINAHL, EMBASE, Food Science & Technology Abstracts, MEDLINE, Biomedical Reference Collection: Corporate Edition (EBSCO), Health Research Premium Collection (ProQuest)
- Anahtar Kelimeler: confidence-aware routing, digital nutrition, food image recognition, mobile health, multimodal large language models, nutrition estimation, personalized dietary assessment
- Kocaeli Üniversitesi Adresli: Evet
Özet
Background/Objectives: The objective evaluation of regional dietary intake remains a core challenge in personalized health management due to complex plate presentations and a lack of culturally specific dataset benchmarks. Methods: This study introduces a confidence-aware hybrid vision–language framework engineered for traditional Turkish food recognition and structured nutritional assessment. Results: We curate a balanced dataset containing 14,711 verified images spanning 40 representative Turkish culinary classes to train and evaluate seven deep learning architectures. Among the visual models, EfficientNet V2-L achieved the highest standalone performance with an accuracy of 93.47%, 0.92 macro-precision, 0.92 macro-recall, and a 0.92 F1 score. To overcome visual ambiguity and automate content analysis, a confidence-aware routing strategy escalates uncertain predictions ((Formula presented.)) or user-rejected classifications to the Google Gemini 2.5 Flash multimodal large language model (MLLM). Conclusions: This hybrid paradigm yields a combined classification accuracy of 95.50% while validating portion weight estimations within a mean absolute error (MAE) of 18.42 g and total energy within 36.75 kcal. Fully realized as a cross-platform Flutter mobile application, the end-to-end pipeline demonstrates localized plate detection, adaptive portion analysis, and structured nutrient tracking, providing a scalable design for consumer-facing digital nutrition platforms.