AI evidence extraction

Detecting Citation Hallucinations in Large Language Model Outputs (Student Abstract)

Nipun Misra, Vikranth Udandarao · Proceedings of the AAAI Conference on Artificial Intelligence · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

1/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1609/aaai.v40i48.42257

Methodology & findings

Study design

Mixed-methods approach combining: (1) Dataset construction through literature-style prompts issued to three frontier LLMs (GPT-4o, Claude 3.5 Sonnet, LLaMA-4 Maverick), with citations extracted and verified against bibliographic databases (Crossref, OpenAlex, Semantic Scholar) using dual annotation with adjudication; (2) Development and pilot evaluation of a three-stage detection pipeline integrating exact lookup, fuzzy string matching (Levenshtein, Jaro-Winkler, BM25, embedding-based similarity), and LLM-assisted verification using Llama-3-8B-Instruct model; (3) Manual verification of 200 pilot citations (~50 per model) against bibliographic databases..

Sample

N = 200, 6 groups

Primary method

Manual verification against bibliographic databases; dual annotation with Cohen's κ for inter-rater agreement (not computed in pilot); qualitative threshold comparisons of detection stages (exact lookup, fuzzy retrieval, LLM-assisted verification). String similarity metrics: Levenshtein distance, Jaro-Winkler; BM25 retrieval; embedding-based similarity.

Main result

The study found that the proposed hybrid detection pipeline "achieved ∼80% precision in detecting hallucinated citations, representing a 15-20% relative improvement over database-only baselines." Preliminary experiments on 200 citations showed hallucination rates of approximately 12% for GPT-4o, 16% for Claude 3.5 Sonnet, and 21% for LLaMA-4 Maverick, with exact lookup detecting ~65% of hallucinations, fuzzy retrieval increasing recall to ~75%, and the three-stage pipeline substantially enhancing both precision and recall.

Reports effect sizes.

Research paradigm

positivist/empiricist

Author conclusions

"We presented a benchmark dataset and a hybrid detection pipeline for identifying citation hallucinations in LLM-generated academic text. By combining database lookups, fuzzy retrieval, and LLM-assisted verification, our approach achieved over 80% precision in preliminary experiments and improved recall relative to database-only baselines. These results demonstrate the feasibility of scalable, automated citation verification and its potential to enhance the reliability of LLM-based scholarly writing."

Risk of bias

Selection bias: citations extracted only from responses to literature-style prompts, which may not represent all hallucination patterns. Limited sample size (~200 citations, ~50 per model) limits representativeness. Annotation bias: only two annotators; inter-rater reliability (Cohen's κ) reported but specific values not provided in the text. Fuzzy retrieval false positives from similar titles suggest potential matching errors.; Selection bias: Only three frontier LLMs tested (GPT-4o, Claude 3.5 Sonnet, LLaMA-4 Maverick); generalizability to other models unclear; Domain bias: Literature-style prompts may not represent all citation use cases; Annotation bias: Dual annotation with adjudication reduces but does not eliminate potential disagreement effects; Database coverage bias: Reliance on specific bibliographic databases (Crossref, OpenAlex, Semantic Scholar) may miss valid citations not indexed; Pilot study limitations: Small sample size (~200 citations) may not be representative; Selection bias: Citations generated from specific prompts may not represent the full range of citation hallucination patterns; Annotation bias: Reliance on two human annotators with adjudication may introduce subjective judgments in classification; Model selection bias: Testing limited to three frontier LLMs may not generalize to other models; Database coverage bias: Verification against specific bibliographic databases may miss valid citations not indexed in those systems

Open questions raised

Authors identify scaling as a future direction: "In future work, we aim to scale the dataset to thousands of citations across additional domains, extend verification to multilingual settings, and publicly release both the dataset and software to promote transparency, reproducibility, and responsible use of AI in research."
Scale: Current pilot dataset limited to ~200 citations; expansion to thousands needed
Domain coverage: Need for verification across additional domains beyond machine learning, NLP, vision, bioinformatics, and law
Multilingual settings: Extension to non-English citations required
Benchmark transparency: Meta-analyses underscore that 'dataset documentation and benchmarking practices often lack transparency, which complicates the reproducibility and systematic evaluation of hallucinations'
Integration strategies: 'The core challenge is not only generation but also reliable integration of retrieval into LLM workflows'

Data: The paper states that "the implementation and annotated dataset are publicly available on GitHub" at https://github.com/Vikranth3140/Citation-Hallucination-Detection; Citation Hallucination Detection benchmark dataset; Citation hallucination benchmark datasetCode: https://github.com/Vikranth3140/Citation-Hallucination-Detection; Citation-Hallucination-DetectionExtracted from: pdfAgreement 57%

Explore related topics

Citation Integrity Research Integrity Benchmarking