Improving AI-Assisted Medical Diagnosis and Triage by the General Public
Trial statusRecruiting
Trial phaseNot applicable
Trial typeInterventional
Biological sexAll
Age18+
SponsorLahore University of Management Sciences
About this trial
This study is a randomized controlled trial (RCT) investigating whether access to a new LLM interface can improve medical triage and diagnostic accuracy for laypeople compared to access to a standard LLM interface. It addresses previous findings where laypeople using standard LLMs performed worse than those using conventional methods (e.g., web search) due to incomplete symptom sharing and poor interpretation of AI advice. To address this, the research tests a structured LLM system that proactively asks clinical history questions before providing a standardized, easy-to-read diagnostic output.
Eligibility criteria
This trial accepts healthy volunteersQualifiers
Enrolled student or employed administrative staff at LUMS.
18 years of age or older.
Able to read and understand English.
Disqualifiers
Individuals with any formal education or professional training in medicine, nursing, or any other healthcare profession.
Trial design
Design model
Parallel
Treatments tested in this trial
new GPT-4o Interface
DeviceParticipants can access any assistance methods they would typically employ (e.g., web search or health portals) in addition to a new LLM (GPT-4o) interface to complete medical scenarios. The new LLM interface uses a fixed system prompt that (a) instructs the model to ask targeted clarifying questions before providing any diagnostic or triage suggestions, and (b) requires all final responses to follow a structured template listing: possible conditions, approximate likelihood of each, and a recommended triage with brief reasoning.
Control
Other interventionParticipants use any assistance methods they would typically employ at home (e.g., Google or health portals) in addition to a standard LLM (GPT-4o) to complete medical scenarios. AI-overview in web searches will be disabled via an extension. They would not be allowed to access any LLMs other than the standard LLM interface.
Treatment groups
220 Participants
are divided into 2 treatment groupsGroup A: Treatment Arm (new GPT-4o Interface)Experimental treatment 1 intervention
Group B: ControlPlacebo comparator 1 intervention
Trial outcomes
Primary outcomes
1
Urgency Assessment Accuracy
The primary outcome will be the percentage of correct urgency assessments, ranging from 0 to 100%, where higher scores indicate better urgency assessment (or triage) performance. The rating which participants give for the urgency of each case, will be measured on a five-point scale: Self-care, Routine GP, Urgent Primary Care, Accident \& Emergency, and Ambulance. Responses will be compared against the gold-standard answers to produce an accuracy measure. The primary outcome will be compared at the case-level between the randomized groups.
Time frame
Assessed at a single time point for each case, during the scheduled diagnostic evaluation session, which takes place between 0-5 days after participant enrollment.2
Condition Identification Accuracy
The co-primary outcome is the Condition Identification Accuracy, which is the percentage of cases in which the condition was correctly identified, ranging from 0 to 100%., where higher scores indicate better medical condition identification performance. Participants name all medical conditions they considered relevant to their decision. A response is scored as correct for that scenario if at least one named condition matches the physician-generated gold-standard list of relevant conditions. The co-primary outcome will be compared at the case-level between the randomized groups.
Time frame
Assessed at a single time point for each case, during the scheduled diagnostic evaluation session, which takes place between 0-5 days after participant enrollment.Secondary outcomes
1
Self-Reported Confidence
Participants' self-reported confidence in their urgency assessments, measured using a scale ranging from 0 to 100.
Time frame
Assessed at a single time point for each case, during the scheduled diagnostic evaluation session, which takes place between 0-5 days after participant enrollment.2
Time Spent
The amount of time participants spend completing each medical scenario in milliseconds.
Time frame
Assessed at a single time point for each case, during the scheduled diagnostic evaluation session, which takes place between 0-5 days after participant enrollment.Sponsors and contacts
Click on the lead sponsor to view all of their trials.
Source ClinicalTrials.gov