About this trial

This retrospective clinical trial aims to better explore the potential of large language models in medicine by comparing the effectiveness of MDT consultations conducted by human doctors with those conducted by large language models.

The main questions to be addressed are:

Does using large language models to conduct anthropomorphic MDT consultations yield better results than using non-anthropomorphic processes? Is there a significant performance gap between MDT consultations conducted by large language models and those conducted by humans? How much greater is the economic benefit of MDT consultations from large language models compared to those conducted by humans?

Retrospectively collect MDT consultation records from the past 20 years in northern Sichuan in China, as well as anonymized patient medical records. Group 1: Different large language models are assigned to act as doctors from different departments and as MDT secretaries to summarize consultations. Group 2: The large language model directly outputs diagnostic and treatment recommendations for patients. Compare the outputs of groups 1 and 2 with human performance retrospectively, score them, and select the best model from each department for a re-evaluation through anthropomorphic MDT consultations, once again comparing them to human results.

Eligibility criteria

This trial does not accept healthy volunteers

Qualifiers

1. The medical records include interdisciplinary consultation notes, with recommendations from specialists of various departments and a well-documented final summary.

2. The medical records contain data from at least one year prior to and one year following the consultation (including intact reports and imaging records).

3. The patient&amp;#39;s discharge conditions improved due to the multidisciplinary treatment plan after the consultation.

Disqualifiers

1. The medical records do not include multidisciplinary consultation notes, or the recommendations from various departmental physicians and the final summary notes are incomplete or inadequate.

2. The medical records lack data from 1 year before and after the consultation, or miss necessary reports and imaging data, resulting in incomplete documentation.

3. The patient&amp;#39;s condition at discharge has not improved following the multidisciplinary treatment plan, or the condition has worsened.

Trial population

From hospital

Trial design

Design model

Other

Time perspective

Retrospective

Treatments tested in this trial

GPT-4o
Diagnostic test
Input all patient medical records, including text, examination reports, and imaging data, into GPT-4o. Use pre-tested prompts to establish department roles, enabling it to provide diagnostic and treatment recommendations pertinent to the respective department.
GPT-4o mini
Diagnostic test
Input all patient medical records, including text, examination reports, and imaging data, into GPT-4o mini. Use pre-tested prompts to establish department roles, enabling it to provide diagnostic and treatment recommendations pertinent to the respective department.
MedicalGPT
Diagnostic test
Input all patient medical records, including text, examination reports, and imaging data, into MedicalGPT. Use pre-tested prompts to establish department roles, enabling it to provide diagnostic and treatment recommendations pertinent to the respective department.
Claude-3.5 Sonnet
Diagnostic test
Input all patient medical records, including text, examination reports, and imaging data, into Claude-3.5 Sonnet. Use pre-tested prompts to establish department roles, enabling it to provide diagnostic and treatment recommendations pertinent to the respective department.
Claude 3 Haiku
Diagnostic test
Input all patient medical records, including text, examination reports, and imaging data, into Claude 3 Haiku. Use pre-tested prompts to establish department roles, enabling it to provide diagnostic and treatment recommendations pertinent to the respective department.
Real Doctors
Diagnostic test
Retrospectively collect the diagnostic and treatment recommendations from the corresponding departments involved in the multidisciplinary treatment of past patients, as well as the overall recommendations.

Treatment groups

300 Participants

are divided into 4 treatment groups

Group A: Anthropomorphized Process Large Language Model Multidisciplinary Treatment Group5 interventions

Group B: Non-anthropomorphized Process Large Language Model Multidisciplinary Treatment Group5 interventions

Group C: Real Doctors Multi-Disciplinary Treatment Group1 intervention

Group D: Best Large Language Model Multidisciplinary Treatment Group5 interventions

Trial outcomes