This study will evaluate whether, relative to conventional information retrieval approaches, direct large language models (LLM) access and LLM use training can improve the overall clinical decision-making ability of rural physicians in low-resource grassroots healthcare settings.
Currently engaged in clinical practice at a rural primary healthcare institution in northwestern China.
Has received formal medical education and holds a relevant diploma or degree.
Able to read and understand clinical case materials in Chinese.
Able to use a computer to complete the study tasks.
Previously involved in the development of the clinical case tasks, reference answers, or scoring rubric for this study.
Previously participated in pilot testing involving the same clinical case tasks or study procedures.
Factorial
LLM-Use Training
BehavioralBefore completing the clinical cases, participants receive brief structured training on the safe and effective use of LLMs. The training covers the role and limitations of LLMs, structured prompting and follow-up questioning, identification of warning signs and referral indications, medication safety, verification of LLM-generated information, high-risk situations in which LLMs should not be relied upon, and protection of patient privacy.
Conventional Non-LLM Resources
Other interventionDuring the initial 60-minute assessment, participants complete primary care clinical cases using conventional non-LLM resources only, including clinical guidelines, textbooks, drug labels, training materials, medical websites, and standard search engines. Participants are not permitted to use LLMs during this phase.
LLM Second-Opinion Review
Other interventionAfter completing and submitting their initial responses using conventional non-LLM resources, participants receive an additional 30 minutes to use the study-provided DeepSeek-V4 as a second-opinion tool. They may review, verify, and revise their initial clinical decisions before submitting their final responses.
Direct LLM Assistance
Other interventionDuring the initial 60-minute assessment, participants may use the study-provided DeepSeek-V4 to assist with medical information retrieval, diagnostic and management reasoning, identification of warning signs, referral decisions, rational prescribing, patient education, and follow-up planning. Participants remain responsible for their final clinical decisions and responses.
Overall Clinical Decision-Making Score
Participants' responses to primary care clinical cases will be evaluated using a prespecified scoring rubric. The overall score will reflect performance across key components of clinical decision-making. Higher scores indicate better overall clinical decision-making performance.
Diagnostic Judgment Domain Score
Participants' diagnostic judgment in response to primary care clinical cases will be evaluated using a predefined scoring rubric. Higher scores indicate better diagnostic judgment.
Clinical Warning Sign Recognition Domain Score
Participants' ability to identify clinically important warning signs in primary care clinical cases will be evaluated using a predefined scoring rubric. Higher scores indicate better recognition of clinical warning signs.
Treatment Plan Domain Score
Participants' proposed treatment plans for primary care clinical cases will be evaluated using a predefined scoring rubric. Higher scores indicate better treatment planning performance.
Change in Overall Clinical Decision-Making Score After LLM Review
Among participants assigned to the LLM second-opinion phase, the change in overall clinical decision-making score will be calculated as the score after LLM-assisted review minus the score before LLM-assisted review. Positive values indicate improvement in overall clinical decision-making performance.
Click on the lead sponsor to view all of their trials.
Peking University Third Hospital
Lead sponsor
Xinjiang Second Medical College
Collaborator
This trial is not recruiting at the moment. You can still explore other options:
