Evaluating the quality of artificial intelligence responses to psoriasis-related clinical and patient questions: a comparative study of ChatGPT, Gemini, and Microsoft Copilot
Baylor University Medical Center Proceedings, cilt.39, sa.5, ss.837-844, 2026 (Scopus)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 39 Sayı: 5
- Basım Tarihi: 2026
- Doi Numarası: 10.1080/08998280.2026.2683945
- Dergi Adı: Baylor University Medical Center Proceedings
- Derginin Tarandığı İndeksler: Scopus
- Sayfa Sayıları: ss.837-844
- Anahtar Kelimeler: Artificial intelligence, ChatGPT, clinical decision support, dermatology, Gemini, large language models, Microsoft Copilot, patient education, psoriasis
- Kocaeli Üniversitesi Adresli: Evet
Özet
Background: Patient use of artificial intelligence (AI) chatbots for dermatologic information is increasing, but their performance on psoriasis-related questions across clinically distinct domains remains unclear. We compared ChatGPT (GPT-5.3 Instant), Gemini (Gemini 3 Flash), and Microsoft Copilot using a multidimensional scoring framework. Methods: Fifty-four psoriasis-related questions were submitted to each model across diagnostic (n = 12), treatment (n = 12), and patient-question (n = 30) categories. Three board-certified dermatologists independently scored responses for accuracy, evidence consistency, completeness, and clinical safety (maximum score, 8). Results: Interrater agreement was substantial to almost perfect (κ = 0.743 for ChatGPT, 0.830 for Gemini, and 0.844 for Copilot). Overall mean scores differed significantly: 7.36 ± 0.97, 7.77 ± 0.77, and 7.05 ± 1.09, respectively (Friedman P < 0.001). No difference was observed for diagnostic questions (P = 0.37). Gemini outperformed both models in treatment (8.00 ± 0.00) and patient questions (P < 0.001), while ChatGPT and Copilot did not differ in treatment. Differences were driven by completeness and accuracy, not clinical safety or evidence consistency. Gemini also had the highest rate of high-reliability responses (92.6%). Conclusions: All models showed high clinical safety, but Gemini provided the most complete and highest-quality responses. The observation that models may provide accurate yet clinically incomplete responses, particularly for treatment content, emphasizes the need for physician oversight when AI-generated information is used in dermatological practice.