Akademik Veri Yönetim Sistemi
INFORMATION PROCESSING & MANAGEMENT, cilt.63, sa.8, ss.1-19, 2026 (Scopus)
This study presents a Large Language Model (LLM)–assisted decision-support system to help higher-education instructors write course learning outcomes (CLOs) aligned with Bloom’s Taxonomy. The system is primarily based on GPT-4o, while additional language models were employed for comparative and evaluative purposes. The system was tested in practice with 22 faculty members across different departments. Following use of the system, usability and user perceptions were examined using the System Usability Scale, which yielded an average score of 79.64 (SD = 13.33), well above the accepted threshold of 68; 80.95% of participants rated the system as having a high level of usability. In parallel, qualitative analysis of open-ended faculty responses identified five dominant themes through manual thematic coding, capturing perceptions related to Bloom’s alignment, user experience, expectation fulfillment, institutional quality assurance, and system dissemination. To further triangulate these manually derived themes, Latent Dirichlet Allocation was employed as a complementary analytical technique. Subsequently, the generated CLOs were evaluated by three subject-matter experts with extensive teaching experience. Quantitative analyses indicated that LLM-assisted CLOs received higher average expert ratings for alignment with Bloom’s Taxonomy (M = 4.60) compared to instructor-written CLOs (M = 3.64) with statistically significant differences (p < 0.001). Beyond participant perceptions, inter-rater reliability analyses highlighted the inherently interpretive and criterion-dependent nature of evaluations based on Bloom’s Taxonomy. Agreement levels varied substantially across evaluation dimensions. Expert consensus was limited for conceptually abstract criteria, particularly alignment with Bloom’s Taxonomy, whereas stronger agreement was observed for linguistic clarity and aspects related to measurability. Accordingly, observed mean score differences for criteria exhibiting low or near-zero inter-rater reliability may be interpreted as exploratory rather than confirmatory. Thus, these findings frame the system’s contribution not as enforcing uniform expert agreement, but rather as supporting clearer, more operationally defined learning outcomes. In this respect, the results suggest that LLMs can function as effective decision-support systems for enhancing the clarity, measurability, and usability of CLOs, thereby supporting curriculum design and quality assurance processes in higher education.