AI evidence extraction

IntrAgent: An LLM Agent for Content-Grounded Information Retrieval through Literature Review

Fengbo Ma, Zixin Rao, Xiaoting Li, Zhetao Chen, Hongyue Sun, Yiping Zhao et al. · ArXiv.org · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

2/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

Methodology & findings

Study design

System design and empirical evaluation using a novel benchmark (IntraBench) with 315 test instances across five STEM domains.

Sample

N = 315, 11 groups

Primary method

Accuracy as primary evaluation metric (proportion of correct answers after LLM-based mapping to multiple-choice options); Macro averaging across five domains to compute cross-domain accuracy; Ablation studies removing specific components (hierarchy preservation, sufficiency check); Comparative analysis across seven backbone LLMs; Counting metrics for token consumption and iteration count; Agreement analysis: correctness agreement ratio comparing LLM mappings to human annotations; No formal statistical significance testing reported (e.g., no hypothesis tests, p-values, or confidence intervals)

Main result

IntrAgent achieves on average "13.2% higher cross-domain accuracy than state-of-the-art RAG and research-agent baselines." Specifically, "Averaged over physics, public health, earth science, engineering, and material science, IntrAgent achieves 70.0% with GPT-4o, 75.8% with GPT-4.1, 74.4% with DeepSeek-R1, 73.4% with o3, 73.8% with o4-mini, 75.9% with Gemini-2.5 Pro, and 68.8% with Llama-3.1-70B. Compared with the strongest baseline under each model, IntrAgent surpasses them by 7.9%, 11.6%, 8.9%, 18.2%, 17.4%, 21.0%, and 7.4%, respectively."

Reports effect sizes.

Research paradigm

Computational empiricism with system design and benchmarking

Author conclusions

The authors conclude that "IntrAgent, the first specialized solution to IntraView," successfully addresses the novel task of content-grounded information retrieval from scientific literature. They state: "We evaluate IntrAgent on IntraBench and show that it outperforms both state-of-the-art RAG and literature-agent baselines across representative backbone LLMs in average cross-domain accuracy." The paper emphasizes that the two-stage pipeline design (section ranking and iterative reading) "mimics human reading behavior" and demonstrates superior performance through explicit hallucination mitigation via sufficiency checks.

Risk of bias

Paper selection bias: Each expert manually curates papers from their familiar pool before random selection. Question generation bias: Questions are generated by domain experts based on their natural reading practices, which may not be representative of all research needs. Evaluator expertise bias: Ground-truth answers are created by domain experts who are familiar with selected papers. Mapping model bias: The evaluation protocol depends on LLM-based mapping to multiple-choice options, which could introduce systematic errors.; Selection bias in paper curation: experts manually selected papers from their familiar pools, which may not represent the full diversity of scientific literature in each domain; Evaluator bias: domain experts created both questions and answer choices, potentially introducing bias in question formulation; Mapping model dependency: evaluation relies on LLM-based mapping of free-form answers to multiple-choice options, which could introduce systematic biases; Selection bias in paper sampling: Papers were selected by domain experts from their familiar pools, then randomly chosen - expert familiarity may introduce bias toward certain paper types; Annotation bias: Questions and answer choices were created by domain experts; potential for systematic bias in what questions are deemed important; Mapping model bias: Evaluation depends on LLM-based mapping of answers to multiple-choice options; different mapping models show variable performance (Table 5-6); Evaluation bias: Multiple-choice format for evaluation differs from free-form answer generation required in practice; Domain expert bias: Only one expert per domain selected papers and created questions/answers

Limitations

The authors acknowledge several limitations: (1) The evaluation uses an LLM-based mapping approach rather than pure semantic matching, noting that "Although accuracy varies across backbone-mapping model combinations, IntrAgent consistently outperforms the vanilla RAG baseline"
(2) The benchmark focuses on five specific STEM domains which may not represent all scientific fields
(3) The paper does not extensively explore failure modes beyond stating that "analysis of failure cases reveals two key issues: 1) Incomplete retrieval" and "2) Hallucination"
(4) Computational cost and token consumption are not thoroughly analyzed across all conditions (though token counts are provided in Table 4).

Open questions raised

The authors identify that existing methods for conventional Content Question Answering (CQA) tasks are insufficient for addressing IntraView, which requires handling the structural complexity and domain-specific language of scientific literature. They note that while numerous LLM-based agents have been developed for scientific tasks, none are specifically designed for content-grounded information retrieval from a single provided paper. The authors suggest future work includes investigating stronger mapping models for evaluation.
The authors identify the lack of dedicated benchmarks and specialized approaches for fine-grained, content-grounded information retrieval from scientific literature. They note that existing methods for general scientific QA through literature search are not designed for the IntraView task and therefore underperform on their benchmark.
The paper identifies that existing CQA and literature-related tasks are insufficient for IntraView's unique challenges of handling full scientific literature with complex structure and domain-specific language
Current RAG methods rely solely on surface-level semantic similarity and ignore structural organization of scientific documents
Existing LLM-based agents for scientific QA focus on external resource exploration rather than fine-grained, content-grounded information retrieval from a provided paper
No prior benchmarks existed for evaluating content-grounded information retrieval from scientific literature - motivating the creation of IntraBench

Data: IntraBench; IntraBench: https://huggingface.co/datasets/IntrAgent/IntraBench; IntraBench Dataset: https://huggingface.co/datasets/IntrAgent/IntraBenchCode: IntrAgent; IntrAgent Code: https://github.com/FengboMa/IntrAgent; IntrAgent Project page: https://intragent.github.io/Extracted from: pdfAgreement 55%

Explore related topics

RAG for Research Literature Discovery Benchmarking