Mohmand H., Kilimci Z. H., Küçükmanisa A.
Sensors, cilt.26, sa.21, ss.1-41, 2026 (Scopus)
-
Yayın Türü:
Makale / Tam Makale
-
Cilt numarası:
26
Sayı:
21
-
Basım Tarihi:
2026
-
Doi Numarası:
10.3390/s26217047
-
Dergi Adı:
Sensors
-
Derginin Tarandığı İndeksler:
Scopus
-
Sayfa Sayıları:
ss.1-41
-
Kocaeli Üniversitesi Adresli:
Evet
Özet
Disaster-related social media posts combine images and text, but the benefits of large language model (LLM)-derived representations for crisis classification remain uncertain. We evaluate three pretrained vision–language pipelines—CLIP, LLM2CLIP, and LLaVA-enhanced CLIP—on CrisisMMD informativeness, humanitarian, and image-derived damage-severity classification. Development experiments examine fusion and class-imbalance strategies; frozen-feature probes assess representation pathways, and selected configurations undergo five-seed held-out testing. CLIP achieves test macro-F1 of 92.32±0.55% and 87.52±1.09% for informativeness and humanitarian classification, respectively; development-selected LLaVA-enhanced CLIP achieves 64.06±0.98% for damage severity. Exact seed-blocked tests show no significant differences among pipeline families after Holm correction across nine contrasts (minimum adjusted 𝑝=0.5625). Frozen-feature probes show no consistent predictive advantage for LLM2CLIP-derived text under matched visual-feature controls. Because the released damage-severity partitions share tweet identifiers across splits, we also conduct five-seed tweet-disjoint retraining, yielding macro-F1 between 62.46% and 63.80% across the pipelines. A matched frozen CLIP control shows negligible change from adding the original tweet to the image input. Image–text retrieval, exploratory cross-event evaluation, and computational profiling characterize task-dependent alignment, transfer, and resource trade-offs. The study provides a disaster-domain comparison of established pipelines without isolating the effects of LLM-derived representations or establishing universal model-family superiority.