This study aims to explore the feasibility of using medical large language models to assist in chronic disease management. By setting up experimental and control group interventions and having experts blindly evaluate anonymized cases, it compares different management approaches in chronic disease care, verifying the scientific basis, effectiveness, and potential for broader use of medical large language models in supporting chronic disease management.
Work experience of 3 years or more;
Able to complete the case assessment tasks required by the study;
Have a practicing doctor qualification;
Work at a township health center or community health service center;
None
Parallel
Medical large language model
DeviceThe primary care physicians in the intervention group completed the management decisions for chronic disease cases, including disease assessment, examination suggestions, treatment plans, selection of management methods, and health education, with the assistance of the medical LLM.
expert overall scores
A total of 200 cases (50 each of coronary heart disease, atrial fibrillation, heart failure, and stroke) were grouped by disease type and independently evaluated in a blinded manner by three subspecialty experts. For each case, the two management plans were anonymized in the same way and then randomly labeled by a computer as Plan A and Plan B. Experts could only view the anonymized case information and the two corresponding anonymized management plans. They rated them using a Likert scale on several aspects: accuracy of condition assessment, reasonableness of test recommendations, reasonableness of treatment plans, appropriateness of management choices, scientific nature of patient education content, feasibility at the primary care level, overall clinical quality. Experts were instructed to rank the seven dimensions from lowest to highest quality. These ordinal rankings were then converted into standardized scores of 1 to 5. Higher scores mean better.
accuracy of condition assessment
A total of 200 cases (50 each of coronary heart disease, atrial fibrillation, heart failure, and stroke) were grouped by disease type and independently evaluated in a blinded manner by three subspecialty experts. For each case, the two management plans were anonymized in the same way and then randomly labeled by a computer as Plan A and Plan B. Experts could only view the anonymized case information and the two corresponding anonymized management plans. They rated them using a Likert scale on accuracy of condition assessment from lowest to highest quality. These ordinal rankings were then converted into standardized scores of 1 to 5. Higher scores mean better.
reasonableness of test recommendations
A total of 200 cases (50 each of coronary heart disease, atrial fibrillation, heart failure, and stroke) were grouped by disease type and independently evaluated in a blinded manner by three subspecialty experts. For each case, the two management plans were anonymized in the same way and then randomly labeled by a computer as Plan A and Plan B. Experts could only view the anonymized case information and the two corresponding anonymized management plans. They rated them using a Likert scale regarding the reasonableness of test recommendations. Experts were instructed to rank the dimension from lowest to highest quality. These ordinal rankings were then converted into standardized scores of 1 to 5. Higher scores mean better.
reasonableness of treatment plans
A total of 200 cases (50 each of coronary heart disease, atrial fibrillation, heart failure, and stroke) were grouped by disease type and independently evaluated in a blinded manner by three subspecialty experts. For each case, the two management plans were anonymized in the same way and then randomly labeled by a computer as Plan A and Plan B. Experts could only view the anonymized case information and the two corresponding anonymized management plans. They rated them using a Likert scale regarding the reasonableness of treatment plans. Experts were instructed to rank the dimension from lowest to highest quality. These ordinal rankings were then converted into standardized scores of 1 to 5. Higher scores mean better.
appropriateness of management choices
A total of 200 cases (50 each of coronary heart disease, atrial fibrillation, heart failure, and stroke) were grouped by disease type and independently evaluated in a blinded manner by three subspecialty experts. For each case, the two management plans were anonymized in the same way and then randomly labeled by a computer as Plan A and Plan B. Experts could only view the anonymized case information and the two corresponding anonymized management plans. They rated them using a Likert scale regarding the ppropriateness of management choices. Experts were instructed to rank the dimension from lowest to highest quality. These ordinal rankings were then converted into standardized scores of 1 to 5. Higher scores mean better.
Click on the lead sponsor to view all of their trials.
This trial is not recruiting at the moment. You can still explore other options:
