Teaching Medical Students to Verify AI-Generated Citation Hallucinations
Trial statusNot yet recruiting
Trial phaseNot applicable
Trial typeInterventional
Biological sexAll
AgeNot listed
SponsorQuaid-e-Azam Medical College
About this trial
Generative artificial intelligence (GenAI) tools are increasingly used by medical students for literature searching, scientific writing, and research. However, AI-generated citations may contain errors. Some references may not exist or may contain incorrect bibliographic details, while other references may be genuine but may not actually support the claim for which they are cited.
This randomized educational study will evaluate whether a structured citation-verification framework called SARA can improve medical students' ability to identify and correct these problems. SARA teaches students to assess four elements of a citation: Source authenticity, Accuracy, Relevance, and Alignment between the cited evidence and the claim being made.
Year 3-5 medical students will be randomly assigned to either a 75-minute SARA training session or an attention-matched session on responsible use of GenAI in academic work. Participants will complete citation-verification assessments before training, immediately after training, and 12 weeks later. The main outcome will assess whether students can independently apply the verification approach to an unstructured literature-review task 12 weeks after training, without being prompted to use SARA.
The study will also examine which types of citation errors remain difficult, whether students initiate appropriate verification, the quality of their verification, handling of valid citations, and confidence in their judgments. A subgroup of participants will take part in think-aloud and follow-up interviews to explore how they approach citation verification and transfer the strategy to new tasks.
Eligibility criteria
This trial accepts healthy volunteersQualifiers
Enrolled as an MBBS student in Year 3, Year 4, or Year 5 at Quaid-e-Azam Medical College, Bahawalpur.
Has received basic curricular exposure to research methodology or scientific literature searching.
Has not previously received formal structured training specifically in verification of AI-generated citations.
Willing and able to provide written informed consent.
Disqualifiers
Previous participation in development, cognitive interviewing, piloting, or validation testing of the SARA assessment materials.
Previous formal training in the SARA citation-verification framework before enrolment.
Unwilling or unable to provide informed consent.
Trial design
Design model
Parallel
Treatments tested in this trial
SARA Citation-Verification Training
BehavioralA standardized 75-minute educational intervention using the SARA framework: Source authenticity, Accuracy, Relevance, and Alignment. The session includes facilitator modelling and think-aloud demonstration, coached practice with feedback, a one-page verification scaffold, learner articulation and reflection, and progressive fading of prompts before independent practice. The intervention is designed to teach medical students to identify both source-level citation errors and deeper evidence-claim errors and to apply verification independently.
Responsible GenAI Education Intervention Description
BehavioralA standardized 75-minute attention-matched educational session on responsible use of generative artificial intelligence in academic work. The session covers privacy, disclosure, authorship, academic integrity, and general awareness that GenAI may produce errors. It does not teach the SARA framework or another structured citation-verification process. The session is matched to the SARA intervention for duration, group size, facilitator interaction, and delivery format.
Treatment groups
320 Participants
are divided into 2 treatment groupsGroup A: SARA Citation-Verification TrainingExperimental treatment 1 intervention
Group B: Responsible GenAI Education ControlActive comparator 1 intervention
Trial outcomes
Primary outcomes
1
Independent Citation-Verification Performance on an Unstructured Scholarly Transfer Task
Participants will review a novel approximately 650-word AI-generated literature-review passage containing eight embedded citation-claim relationships. The passage includes both valid and erroneous citations spanning the study's five-level citation-error taxonomy. Participants will not be prompted to use the SARA framework and will not be told how many or what types of errors are present. Each citation-claim relationship will be scored against a prespecified gold standard for the appropriateness of the participant's final disposition. One point will be awarded for each appropriate disposition, producing a total score from 0 to 8, with higher scores indicating better independent citation-verification performance. Responses will be scored by outcome assessors blinded to intervention allocation.
Time frame
12 weeks after the interventionSecondary outcomes
1
Structured Citation-Verification Performance
Citation-verification performance will be measured using parallel 24-item structured assessment forms. Each item requires an appropriate final disposition of a citation-claim pair against a prespecified gold standard. One point will be awarded for each correct disposition, giving a total score from 0 to 24. Higher scores indicate better structured citation-verification performance.
Time frame
Immediately after the intervention and 12 weeks after the intervention2
Citation-Verification Accuracy by Citation-Error Condition
Accuracy will be assessed separately for valid/aligned citations and for Levels 1-5 of the prespecified citation-error taxonomy: non-existent citations, altered bibliographic details, irrelevant sources, evidence-claim misalignment, and contextual omission or overgeneralization. The outcome will be expressed as the proportion of citation-claim decisions with an appropriate final disposition within each condition.
Time frame
Immediately after the intervention and 12 weeks after the intervention3
Verification Initiation
Verification initiation will be measured as the proportion of citation-claim decisions for which the participant initiates an external verification action, such as consulting a bibliographic database, search engine, publisher source, abstract, or full text. Higher proportions indicate more frequent initiation of verification.
Time frame
Immediately after the intervention and 12 weeks after the intervention4
High-Quality Citation Verification
High-quality verification will be defined as examining the source element necessary to resolve the specific citation problem. Merely locating an article will not be considered adequate for evidence-claim errors if the participant does not assess whether the evidence supports the claim. The outcome will be expressed as the proportion of citation-claim decisions meeting the prespecified high-quality verification criterion.
Time frame
12 weeks after the interventionOther outcomes
1
Qualitative Exploration of Citation-Verification Strategies and Transfer Processes
A purposively selected subgroup of participants will complete think-aloud verification of novel citation-claim pairs followed by stimulated-recall interviews. Data will explore what triggers verification, how participants decide what evidence is sufficient, why verification stops, how relevance is distinguished from evidentiary support, how confidence changes, and whether SARA has become an internalized strategy or remains an explicit checklist. Interview transcripts will be analyzed using reflexive thematic analysis.
Time frame
After completion of the 12-week assessmentLocations
Sponsors and contacts
Click on the lead sponsor to view all of their trials.
Source ClinicalTrials.gov
This trial is not recruiting at the moment. You can still explore other options: