The American Society of Anesthesiologists Physical Status (ASA-PS) classification is a cornerstone of preoperative risk assessment, yet interrater variability among clinicians is well documented. Large language models (LLMs) have recently demonstrated expert-level performance in several clinical classification tasks, including ASA-PS assignment.
This retrospective observational study evaluates whether four widely used LLMs - ChatGPT, DeepSeek, Gemini, and Claude - can accurately and consistently assign ASA-PS classes from structured, fully anonymized clinical vignettes derived from real preoperative anesthesia evaluations, using a consensus of senior anesthesiologists as the reference standard.
No patient data will be transmitted to third-party platforms. Clinical information will be converted by the investigators into de-identified structured vignettes containing only age range, sex, body mass index range, presence or absence of systemic diseases, functional capacity, and the major/minor nature of the planned surgery, in full compliance with national data protection legislation (KVKK).
Age 18 years or older
Planned elective surgery
Completed preoperative anesthesia evaluation
Emergency surgical procedures
ASA VI (brain death)
Incomplete clinical records
Adult patients who underwent preoperative anesthesia evaluation before elective surgery at a tertiary university hospital in Istanbul, Turkey.
Cohort
Retrospective
Not listed
Agreement between LLM-assigned and reference-standard ASA-PS class
Quadratic weighted Cohen's kappa between each large language model's ASA-PS assignment (ChatGPT, DeepSeek, Gemini, Claude) and the reference standard defined by consensus of a blinded panel of at least three senior anesthesiologists. Agreement of at least "good" level (weighted kappa ≥ 0.60) is hypothesized.
Overall classification accuracy of each LLM
Proportion of vignettes in which the LLM-assigned ASA-PS class exactly matches the reference standard, with exploratory pairwise comparisons among the four models (McNemar/permutation tests, corrected for multiple comparisons)
Subgroup error patterns
Frequency and direction (over- vs. under-classification) of LLM misclassifications in prespecified subgroups: ASA III-IV boundary, multimorbidity, major vs. minor surgery
Click on the lead sponsor to view all of their trials.
This trial is not recruiting at the moment. You can still explore other options:
