AI evidence extraction

Toward a Computable Scientific Corpus: Retrieval-Augmented Reasoning Systems for Scientific Discovery on Exascale Supercomputers

Ozan Gökdemir · University of Chicago · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

0/4

Quality (LMQS)

Evidence

Citations

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.6082/v976y-ymz17

Methodology & findings

Study design

Design science methodology involving the development and implementation of three integrated systems: (1) HiPerRAG, a distributed high-performance computing workflow with multimodal parsing and query-aware encoder fine-tuning; (2) a scalable automated MCQA benchmark generation pipeline producing provenance-tracked questions from full-text articles; (3) Swarm Retrieval, a paradigm treating documents as active agents in the retrieval process.

Primary method

Design science approach with iterative system implementation and evaluation using exascale supercomputing infrastructure

Main result

The dissertation presents three principal contributions demonstrating scalable retrieval-augmented reasoning systems. First, "HiPerRAG empowers open-source models with retrieval and leads them to outperform proprietary frontier LLMs on scientific question-answering tasks." Second, the work identifies "distillation through retrieval as a viable alternative to weight-based distillation" and shows that "small language models such as TinyLlama-1.1B achieve a 4× improvement in domain accuracy." Third, the authors introduce Swarm Retrieval, "a forward-looking paradigm in which documents are not treated as passive points in a vector space, but as agents embodying documents that can judge their own relevance to a given query in an interpretable manner."

Research paradigm

Design science / computational systems design

Author conclusions

The authors conclude that their work charts "a path toward a computable scientific corpus which we define as a unified, queryable interface over the full body of published science that is technically feasible with current exascale infrastructure." The contributions collectively demonstrate that "retrieval-augmented reasoning systems that operate at the scale of millions of scientific documents and leverage exascale supercomputing infrastructure" can transform how scientists interface with scientific literature.

Limitations

The paper addresses several scaling challenges inherent in the work: "scaling RAG to handle the deluge of scientific output– in the order of millions of documents– introduces challenges at every stage of the pipeline
To name a few, parsing dense multi-modal PDFs, encoding domain-specific, terminology-rich text, evaluation models on unmistakably contaminated benchmarks, and orchestrating hundreds of compute nodes with thousands of GPUs to simply overcome the sheer scale of the problem."

Open questions raised

The dissertation identifies the need to scale RAG systems to handle "the deluge of scientific output– in the order of millions of documents" and addresses gaps in making frontier LLM capabilities accessible through open-source models in scientific domains. It also positions Swarm Retrieval as a forward-looking paradigm that challenges conventional document-as-vector approaches.
The dissertation identifies the need for systems that can scale RAG to millions of scientific documents and leverage exascale computing infrastructure to improve how scientists interface with scientific literature. It addresses gaps in handling domain-specific terminology, managing multi-modal scientific documents, and creating reliable evaluation benchmarks for knowledge-intensive scientific tasks.
The dissertation addresses the fundamental gap between the rapid growth of scientific output (exceeding three articles per minute to PubMed) and human cognitive capacity, identifying that "Large Language Models emerged as powerful engines for ingesting and synthesizing this growing corpus" yet suffer from hallucination in knowledge-intensive tasks. The work positions the computable scientific corpus as an infrastructure gap requiring exascale computing to address.

Data: not_statedCode: not_statedExtracted from: pdfAgreement 63%

Explore related topics

RAG for Research Literature Discovery Design Science