This multicenter prospective diagnostic accuracy study will compare the performance of three artificial intelligence (AI) models (MVCS, LungDoc, and a United Imaging AI model) for predicting the malignancy risk of pulmonary nodules on chest CT. All enrolled patients will have pulmonary nodules ≤3 cm on CT and a definitive postoperative or biopsy pathological diagnosis. The AI models will generate continuous malignancy probability scores based only on CT images. Pathology will serve as the gold standard.
The primary objective is to compare the area under the receiver operating characteristic curve (AUC) for malignancy prediction among the three AI models. Secondary objectives include comparison of sensitivity, specificity, positive and negative predictive values, accuracy, F1 score, and calibration. Exploratory analyses will evaluate the MVCS model for predicting pathological invasion degree (pre-invasive, minimally invasive, and invasive adenocarcinoma) and an extended MVCSN model that incorporates clinical and imaging features in a data-complete subset.
Age ≥ 18 years, any sex.
At least one pulmonary nodule detected on chest CT, with initial nodule diameter ≤ 3 cm.
The nodule undergoes surgical resection or biopsy with a definitive benign or malignant pathological diagnosis.
Time interval between CT examination and pathological examination ≤ 6 months.
Pathological results are unclear, inconclusive, or disputed; nodule nature or grade cannot be reliably determined.
The patient receives treatments between CT and pathology that may significantly alter nodule appearance (e.g., chemotherapy, radiotherapy, targeted therapy).
CT imaging data are incomplete (missing essential series) or have severe motion, metal, or other artifacts preventing accurate AI analysis.
Required metadata for any AI model are missing and cannot be imputed. History of other malignant tumors (malignancies other than the index non-small cell lung cancer).
Adults (≥18 years) undergoing routine clinical care at five tertiary hospitals in China who have at least one pulmonary nodule (≤3 cm) detected on chest CT and receive surgical or biopsy pathology with a definitive benign or malignant diagnosis. All participants have adequate CT image quality and complete clinicopathologic information for AI model validation
Cohort
Prospective
Not listed
Area Under the ROC Curve (AUC) for Malignancy Prediction
For each pure imaging AI model (MVCS, LungDoc, United Imaging model), the AUC of the receiver operating characteristic curve for predicting malignant versus benign pulmonary nodules, based on continuous malignancy probabilities or suspicion scores. AUCs will be reported with 95% confidence intervals, and pairwise comparisons will be conducted using DeLong's test.
Sensitivity and Specificity for Malignancy Prediction
Sensitivity and specificity for classifying nodules as malignant vs benign for each AI model, using both (a) model-predefined thresholds and (b) optimal cut-off points determined by maximizing the Youden index. 95% confidence intervals will be reported; paired comparisons will use McNemar's test.
Positive Predictive Value (PPV) and Negative Predictive Value (NPV)
PPV and NPV for malignancy prediction for each AI model at the same thresholds as above, with 95% confidence intervals.
Overall Diagnostic Accuracy and F1 Score
Proportion of correctly classified nodules (accuracy) and F1 score for each AI model in the binary task of malignant versus benign nodules, with 95% confidence intervals.
Calibration Metrics
Calibration performance of each AI model will be evaluated by calibration plots, calibration intercept, and calibration slope for predicted malignancy probability versus observed malignant proportion. Hosmer-Lemeshow goodness-of-fit test will be reported.
Multi-Class Accuracy of MVCS for Pathological Invasion Degree
For nodules with specific pathological subtypes (AAH, AIS, MIA, ADC), performance of the MVCS model in three-class classification: pre-invasive lesions (AAH-AIS), minimally invasive adenocarcinoma (MIA), and invasive adenocarcinoma (ADC). Metrics include overall three-class accuracy and weighted F1 score.
Matthews Correlation Coefficient (MCC) and Confusion Matrix for Invasion Classification
Exploratory Performance of MVCSN Model for Malignancy Prediction
In a subset with complete clinical and imaging feature data, the MVCSN model (CT + clinical + imaging features) will be evaluated for malignancy prediction using similar metrics (AUC, sensitivity, specificity, PPV, NPV, accuracy, F1, calibration). Results will be used exploratorily and not for primary hypothesis testing.
Click on the lead sponsor to view all of their trials.
This trial is not recruiting at the moment. You can still explore other options:
