AI evidence extraction

DeepXiv-SDK: An Agentic Data Interface for Scientific Literature

Hongjin Qian, Ziyi Xia, Ze Liu, Jianlyu Chen, Kun Luo, Minghao Qin et al. · arXiv (Cornell University) · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

1/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

Methodology & findings

Study design

Design science with task-driven evaluation.

Primary method

Design science research with iterative development informed by three core principles: (1) structured and normalized data interfaces, (2) progressive disclosure by knowledge density, and (3) retrieval-oriented and conditionable access.

Main result

DeepXiv-SDK enables cheaper screening, more selective section reading, and verification only when needed. The system "reduces token overhead by avoiding default full-text ingestion, improves retrieval precision via hybrid, attribute-conditioned search and structure-aware routing, and yields higher-quality outputs by escalating to evidence-level access only when verification is needed." In evaluation, "DeepXiv improves end-to-end answer quality consistently while substantially reducing token consumption and runtime," and "DeepXiv-SDK delivers large end-to-end speedups, e.g., 54.6× (Local) and 39.6× (Remote) for warm json access" compared to conventional fetch-and-parse workflows.

Research paradigm

Pragmatist/Design Science

Author conclusions

The authors conclude: "making papers tool-callable with progressive, budget-aware access improves both efficiency and grounding quality for research-agent workflows." They further state: "we show that DeepXiv-SDK enables cheaper screening, more selective section reading, and verification only when needed, improving both efficiency and evidence grounding in AI4Science workflows."

Risk of bias

Selection bias in baseline comparison: only five academic search platforms compared; unclear if these represent the full universe of agentic search systems. Limited evaluation set sizes (50 and 47 queries respectively) may not be representative of diverse real-world query patterns. No discussion of inter-rater reliability for manual auditing (only 200 sampled links audited out of 219,717 GitHub URLs extracted).; Evaluation set limited to 50 search queries and 47 QA queries; comparison baselines may not represent all contemporary academic search systems; corpus limited to arXiv papers; evaluation focused on English-language academic literature

Open questions raised

The paper identifies paper access as a practical bottleneck for research agents: "existing pipelines often treat papers as raw PDF/HTML artifacts, leading to expensive full-text ingestion and brittle evidence lookup." The authors note that prior efforts "typically do not expose a reusable data interface API that agents can call deterministically across tasks." Future extensibility mentioned: "the pipeline is designed to extend to other open-access corpora (e.g., PMC) by swapping only the ingestion connector, while reusing the same normalization schema, enrichment, indexing, and serving protocol."
The paper identifies that "paper access in today's pipelines remains largely ad hoc" and that "a common workflow queries a general-purpose search engine, opens a paper in PDF or HTML form, heuristically extracts text, and then feeds large chunks back into the agent for retrieval or question answering. This pipeline is both inefficient and brittle." The authors propose that an agent-friendly solution should treat paper access as a data interface with structured, normalized access and progressive disclosure by knowledge density.
The paper identifies that existing research-agent pipelines treat paper access as ad hoc, with agents repeatedly incurring substantial parsing and reading overhead and depending on document-specific formatting quirks. It notes the lack of a standardized interface across venues and domains, and that intermediate representations are hard to reuse across tasks or agents.

Data: arXiv corpus (2,949,129 papers as of reporting); evaluation queries available via live demo at authors' website; manual audit of GitHub URLs (200 samples); no explicit public dataset release mentioned for evaluation sets.; DeepXiv-SDK provides access to 2,949,129 arXiv papers as of the publication date. The paper mentions evaluation sets of 50 multi-constraint search queries and 47 complex QA queries but does not provide explicit URLs for dataset download.; arXiv corpus (2,949,129 papers indexed as of paper writing); Evaluation set: 50 multi-constraint search queries with unique target papers; Evaluation set: 47 complex QA queries requiring evidence aggregationCode: DeepXiv-SDK is described as "open-source SDK" with REST APIs provided; live demo showcased at authors' website; no specific GitHub repository URL provided in the paper.; DeepXiv-SDK is released as "an open-source SDK" and the paper mentions "a live demo showcasing deep-search and deep-research workflows," but specific GitHub repository URLs are not provided in the paper text.; DeepXiv-SDK: open-source SDK released (specific GitHub URL not provided in paper); Live demo available at: https://deepxiv.org (implied from text mentioning 'this website')Extracted from: pdfAgreement 54%

Explore related topics

AI-Assisted Discovery Literature Discovery Design Science