AI evidence extraction

Agentic publications: redesigning scientific publishing in the age of thinking large language models

Roberto Pugliese, George Kourousias, Francesco Venier, G. Costa · Journal of Documentation · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

1/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1108/jd-07-2025-0207

Methodology & findings

Study design

Conceptual design architecture with accompanying prototype demonstration.

Primary method

Design science approach combining conceptual architecture design with prototype implementation. The methodology includes: system architecture specification (Figures 1-3), technology stack identification, implementation strategy outline, and a working prototype demonstration.

Main result

The paper proposes that "an LLM-powered architecture as a potential complement to the traditional scientific paper, which we term an Agentic Publication (AP)" can address systemic challenges in scientific publishing. The authors identify that "the exponential growth of scientific literature -with nearly 2 million articles published annually and annual growth rates exceeding 4% -has created systematic bottlenecks that strain peer review capacity and make comprehensive knowledge synthesis impossible for individual researchers." The proposed system demonstrates feasibility through a limited prototype where "users can interact with the knowledge system through an intuitive conversational interface, enhancing static article utility."

Research paradigm

Design science / pragmatist (building and proposing an artifact-based solution)

Author conclusions

The authors conclude that "By detailing this architecture and its feasibility, we aim to illustrate a path toward a more accessible, up-to-date and interactive model of scientific communication" that is "open and responsive, benefiting researchers and society in ways the static journal article alone cannot." They further state that "APs are designed explicitly to counter" the scenario where "AI systems may displace scientists as the primary thinkers in society." Instead, "In our model, the author's intellectual contribution-thinking, interpretation, and reasoning-remains central and irreplaceable" and "Far from diminishing scholarly thought, APs amplify it: they preserve intellectual arguments alongside data, make reasoning auditable, and allow future readers and systems to engage with the author's thought process directly."

Risk of bias

Literature bias: LLMs trained on scientific literature reflect existing publication biases, including research focus disparities, gender/racial biases in study populations, and publication bias toward positive results; Training data bias: Models may amplify existing institutional prestige bias and geographic bias against non-Anglophone research; Representation bias: Underrepresentation of specific communities and global regions in training data; Hallucination risk: LLMs may generate plausible but incorrect statements with false citations; Publication bias in scientific literature; Gender and racial biases in study populations; Underrepresentation of specific communities in research; Prestige bias favoring authors from renowned institutions; Geographic bias disadvantaging non-Anglophone researchers; Training data biases in LLMs reflecting literature biases; Training data biases from scientific literature; Publication bias in source literature (preference for positive results); Geographic and language biases in available research; Underrepresentation of non-Western research traditions; Author demographic biases in study populations

Limitations

The authors acknowledge multiple limitations: "Knowledge representation across domains requires formats suitable for both LLM and algorithm consumption, accommodating varied methodologies of different fields," and crucially, "the reliance on manuscripts and preprints as input to APs" presents a challenge because "early-stage versions may contain incomplete, inconsistent, or even erroneous information." Additionally, "some knowledge types resist current AI representations
Mathematical proofs, theoretical arguments, and visual insights may be unsuitable for text-based processing, while systems struggle with symbolic reasoning and qualitative analysis." Furthermore, the authors note that "fully removing prior information from a statistical model remains technically difficult and is not equivalent to training from scratch on final, peer-reviewed material."

Open questions raised

Ensuring accuracy across millions of ingested claims - "Near-zero tolerance for critical errors is essential, especially in medicine, requiring continuous AI refinement and tiered approaches with extra human scrutiny for high-stakes information"
Infrastructure demands for sustainable scaling - "Unlike traditional publishing's human labor and low-tech distribution, this requires cutting-edge hardware and engineering"
Defining appropriate governance models for decentralized global systems
Researcher adoption and cultural shifts in incentive structures
Handling mathematical proofs, theoretical arguments, and symbolic reasoning
Version control and managing evolving preprints and peer-reviewed updates

Data: Demo AP knowledge base: Contains the manuscript content itself (https://doi.org/10.34965/agenticpublication.3567a); Paper references dataset mentioned in demo: metadata, abstract and summaries available through the demo interface; The demo AP is accessible at https://doi.org/10.34965/agenticpublication.3567a with associated datasets referenced from the paper itselfExtracted from: pdfAgreement 62%

Explore related topics

Scholarly Publishing Scholarly Infrastructure Design Science