AI evidence extraction

Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents

Delip Rao, Eric W. M. Wong, Chris Callison-Burch · arXiv (Cornell University) · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

2/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

Methodology & findings

Study design

Empirical characterization study across multiple datasets (DRBench with 100 multilingual research queries from 23 models; ExpertQA with 2,177 expert-curated questions across 32 academic fields).

Sample

N = 435, 14 groups

Primary method

Bootstrap 95% confidence intervals (percentile method); two-proportion z-tests for comparing rates across models and fields; stratified random sampling for sensitivity analyses; concurrent HTTP requests with thread pools (100 workers for DRBench, 60 for ExpertQA); headless-browser auditing using Playwright for UNKNOWN URL verification; chi-square tests (reference: two-proportion z = 4.83, p < 10^-5 for field variation)

Main result

The study found that "Hallucinated URL rates range from 3% to 13%, with 5-18% non-resolving overall." Additionally, "Deep research agents exhibit the highest hallucination rates despite generating far more citations; retrieval architecture matters more than output volume." The research also demonstrated that "urlhealth reduces non-resolving URLs by 6-79× (all p < 10^-35), with postmitigation rates below 1% across all three models."

Reports effect sizes and confidence intervals.

Research paradigm

Empirical-positivist (measurement-based verification of URL existence and citation reliability)

Author conclusions

The authors conclude that "citation hallucination persists even when models have access to web search, a qualitatively different failure mode" from prior research on parametric memory-only generation. They state that "Two directions seem most pressing. First, generation-time constraints restricting URL emission to actually-visited pages could reduce fabrication at its source. Second, extending verification to bibliographic metadata and fabricated snippets would provide a more complete picture of citation reliability." Finally, they commit to "releasing urlhealth and all experimental data and scripts pertaining to this work under MIT license to further LLM/agent driven bibliography verification and mitigation research."

Risk of bias

Selection bias: Only 10 of 23 models from DRBench included due to data availability constraints; 13 models excluded based on missing text, insufficient data, or anomalous patterns; Measurement bias: HTTP 403 responses excluded from non-resolving counts, which may underestimate true failure rates; Temporal bias: Point-in-time URL checking may not reflect historical or future state of citation validity; Bot-blocking false positives: Automated checking subject to rate limiting and anti-bot mechanisms; Reddit URL handling treated conservatively as alive based on spot-checks; Archive coverage bias: Wayback Machine incompleteness affects hallucination classification; absence of snapshot may reflect incomplete crawling rather than non-existence; Domain-specific sampling: ExpertQA covers 32 fields but with potentially uneven representation; DRBench limited to finance, science, and technology domains; Incomplete Wayback Machine coverage may systematically misclassify some hallucinated URLs as stale; Bot-blocking (HTTP 403) exclusion may underestimate non-resolving rates by 6.6-17.0% depending on model; Point-in-time measurement sensitivity—rates may differ at different times; Selection of 10 models from 23 available in DRBench based on URL data availability (13 excluded for missing text, insufficient data, or anomalous patterns); Reddit URL classification as alive based on manual spot-checks rather than systematic validation; Provider-specific API configuration differences may affect comparability across models; Incomplete Wayback Machine coverage leading to underestimation of hallucinated URLs; Exclusion of HTTP 403 responses (bot-blocking) potentially removing genuinely non-resolving URLs; Bot-blocking and rate-limiting false positives in automated URL checking; Temporal sensitivity of point-in-time URL liveness measurements; Potential misclassification of soft-404 or wildcard-redirect pages in Wayback Machine; Special handling of Reddit URLs (18.2% of GPT-5.1 citations) classified as alive based on manual spot-checks rather than systematic validation; Model selection bias: only 10 of 23 models in DRBench analyzed due to availability constraints

Limitations

The authors state that "Wayback Machine coverage is substantial but incomplete and nonuniform," making hallucination estimates "conservative lower bounds." Additionally, "HTTP 403 responses account for 6.6% (gemini-2.5-pro) to 17.0% (claude-sonnet-4-5) of all URLs tested, predominantly from major academic publishers," and these are excluded from non-resolving counts, making "reported non-resolving rates lower bounds." The study also notes "URL liveness checks are point-in-time measurements" and that results "may differ if checked at a different time." Finally, the authors acknowledge that "the evaluation uses single-turn outputs
the drift finding suggests that citation hallucination may worsen in conversational workflows."

Open questions raised

Need for generation-time constraints restricting URL emission to actually-visited pages to reduce fabrication at source
Extending verification beyond URLs to bibliographic metadata (e.g., DOI validity) and fabricated snippets (quoted text not appearing at cited page)
Understanding internal retrieval architectures of commercial APIs (authors note inability to verify hypotheses about URL generation mechanisms without access)
Investigation of citation fabrication in multi-turn conversational workflows (preliminary evidence of drift across turns)
Content-based citation verification for sources behind paywalls or requiring authentication
Need for generation-time constraints that restrict URL emission to actually-visited pages

Data: DRBench: 100 multilingual research queries (Chinese and English) covering finance, science, and technology, with pre-collected outputs from 23 models (Du et al., 2025); ExpertQA: 2,177 expert-curated questions across 32 academic and professional fields (Malaviya et al., 2024); Authors commit to releasing all experimental data and scripts under MIT license (urlhealth and supplementary materials mentioned); DRBench; ExpertQA; DRBench: 100 multilingual research queries (Chinese and English) covering finance, science, and technology with pre-collected outputs from 23 models; ExpertQA: 2,177 expert-curated questions across 32 academic and professional fields; Supplementary materials containing full results for all models with extractable URLsCode: urlhealth: 83 lines of Python, available as pip-installable package and as agentskills.io skill for integration with AI coding agents; Authors commit to releasing urlhealth under MIT license; urlhealth; urlhealth: open-source, model-agnostic URL verification tool (83 lines of Python), available as pip-installable package and agentskills.io skill; URL: https://agentskills.io (integration platform); Model Context Protocol (MCP): https://modelcontextprotocol.io (standardized tool interface)Extracted from: pdfAgreement 52%

Explore related topics

Hallucination Control Research Integrity Empirical Study