International Journal of Physical Medicine & Rehabilitation
Open Access
ISSN: 2329-9096
ISSN: 2329-9096
Ozlem Kuculmez
Department of Physical Medicine and Rehabilitation, Baskent University School of Medicine, Turkey
Scientific Tracks Abstracts: Int J Phys Med Rehabil
Objective: Clinical guidelines are essential for diagnosing and managing rheumatic conditions like rheumatoid arthritis, which can result in deformities and impairments. The aim of this research is to assess and contrast the answers provided by different large language models to open-ended questions based on the 2022 EULAR guideline. Method: This was a comparative, cross-sectional study. Five overarching principles and eleven recommendations from the 2022 EULAR guidelines were rephrased into open-ended clinical questions. These questions were presented to ChatGPT-3.5, GPT-4o, and Gemini 2.5 Flash. The responses were assessed using a 7-point Likert scale for usability and reliability, readability scores (Flesch-Kincaid Reading Ease (FKRE) and Flesch-Kincaid Grade Level (FKGL)), and similarity metrics (Universal Sentence Encoder (USE) and ROUGE-L). Different models were compared statistically and p < 0.05 was revealed as statistically significant. Results: ChatGPT-3.5 and ChatGPT-4.0 had significantly higher FKGL and FKRE scores than Google Gemini 2.5 (p < 0.05). Reliability did not differ between models (p > 0.05), but ChatGPT-4.0 had a significantly higher usefulness score than Gemini 2.5 (p < 0.05). ChatGPT4o and Gemini 2.5 showed significantly higher semantic similarity than ChatGPT-3.5 (p < 0.05). ROUGE-L F1 scores showed no significant differences (p > 0.05). FKRE and FKGL were strongly negatively correlated, while reliability and usefulness were strongly positively correlated (p < 0.05).
Ozlem Kuculmez has completed her medicine education at the age of 24 years from Istanbul University and became specialist in physical medicine and rehabilitation department in Katip Celebi University. She has been still working assist. Prof. Dr. in Baskent University since 2017 and has been serving as a board member of Turkish Physical Medicine and Rehabilitation Society.