AI evidence extraction

An Evidence-Grounded Research Assistant for Functional Genomics and Drug Target Assessment

Ksenia Sokolova, Dmitri Kosenkov, Keerthana Nallamotu, Sanketh Vedula, Daniil Sokolov, Guillermo Sapiro et al. · bioRxiv (Cold Spring Harbor Laboratory) · 2025

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

2/4

Quality (LMQS)

Evidence

Citations

1.10

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.64898/2025.12.30.697073

Methodology & findings

Study design

Multi-component agentic system design with modular evaluation: (1) entity recognition pipeline using ensemble methods (Claude, Flair, GLiNER, regex, sequence matching); (2) tool selection and orchestration via LangGraph StateGraph; (3) evidence-driven answer generation with citation reconstruction; (4) multi-step verification with deterministic checks and LLM-based qualitative review.

Main result

Alvessa achieved 0.964 (95% CI: 0.949-0.976) accuracy on GenomeArena, "substantially outperforming both Claude Sonnet 4.5 (0.537; 95% CI: 0.501-0.576) and ChatGPT 5.1 (0.528; 95% CI: 0.492-0.565)". The context-aware verifier detected "nearly all of the adversarial statements (100% of contradictions and wrong alphanumeric values, 92.3% of overstatements, and 92.1% of wrong numerical values)", demonstrating that "evidence-grounded verification meaningfully enhances the reliability of agent-assisted biological reasoning in practice".

Research paradigm

Computational/Engineering (artifact-building with empirical validation)

Author conclusions

"By combining biology-focused entity resolution, structured access to reliable databases, and explicit statement-level verification, Alvessa enables researchers to obtain answers traceable to the underlying evidence." The authors further conclude that "progress in applying AI systems to biomedical research may depend as much on engineering choices about evidence handling as on advances in large language model scale or fluency. By emphasizing explicit provenance, statement-level verification, and transparent access to underlying data, Alvessa offers one approach to aligning language model capabilities with the reliability demands of genomics and related biomedical domains."

Risk of bias

Benchmark construction bias: GenomeArena questions generated from databases with potential annotation errors; Tool coverage bias: dbQA comparison may partly reflect differences in tool coverage between systems; Multiple testing: evaluation across numerous question categories without multiple comparison correction stated; Adversarial evaluation: reliance on LLM-generated adversarial examples may not reflect real-world error patterns; Tool coverage differences between compared systems (Alvessa vs. Biomni) may confound accuracy comparisons; GenomeArena benchmark construction uses random sampling of distractors which may not fully represent real-world difficulty distributions; Adversarial perturbations generated by LLM-based agent may not exhaustively capture all plausible hallucination patterns; Web search baseline (Claude + web search) may have suboptimal query formulation since not specifically designed for genomic reasoning; Limited annotation of dataset construction details (number of reviewers, inter-rater reliability, validation procedures); Benchmark design: GenomeArena questions generated from existing databases may reflect database biases; Tool coverage differences: Performance comparisons with Biomni may partly reflect differences in available tool integrations; Entity naming variability: Drug entities handled through fuzzy matching which could introduce recall/precision trade-offs; Verification context truncation: Long documents may be condensed, potentially losing relevant evidence context

Limitations

"We anticipate that GenomeArena will need to be expanded in the future
at this stage, it serves as a first step for testing the essential ability of an agent to navigate core databases." Additionally, "while Alvessa's current tool set covers core genomic and proteomic resources, expanding to additional databases and other omics domains would broaden the range of questions that can be addressed." The authors note that "although accuracy differences may partly reflect differences in tool coverage" when comparing to Biomni, and acknowledge that "Alvessa is not designed to generate speculative hypotheses or to analyze user-provided data."

Open questions raised

Need to expand GenomeArena to include more comprehensive and complex questions specifically aimed at judging performance gains of evolving genomic agents
Extension to additional databases and other omics domains needed to broaden question coverage
Architecture supports user-contributed tools, pending adherence to evidence access and verification requirements
Studies of researcher interaction with verified answers and response to surfaced uncertainties needed to inform interface design
Need for expanded GenomeArena benchmark covering more comprehensive and complex questions beyond core foundational competencies
Integration of additional omics domains and databases beyond current genomic and proteomic resources

Data: GenomeArena: 720 multiple-choice questions across variant annotation, gene annotation, pathways, interactions, miRNA targets, drug-target relationships, protein structure, and gene-phenotype associations (code for benchmark generation and all questions publicly available); dbQA from LAB-Bench suite (obtained from HuggingFace Lab-Bench dataset page, August 19, 2025); CysDB: chemoproteomics data integrated into Alvessa; AlphaFold database v4: protein structures; BioLiP2: experimentally supported protein-ligand binding information; GenomeArena: 720 curated multiple-choice questions spanning eight categories (variant annotation, gene annotation, pathways, interactions, miRNA targets, drug-target relationships, protein structure, gene-phenotype associations) - publicly available; dbQA from LAB-Bench suite - independent benchmark, accessed from HuggingFace on August 19, 2025; CysDB: chemoproteomics data integrated into alvessa_proteins.db; AlphaFold Database v4: protein structures (local copy); Local SQLite snapshots: UniProtKB, ChEMBL v35, DrugCentral, ClinVar (filtered), DisProt, GWAS Catalog, ReMap 2022, IntAct Viral (downloaded Dec 15, 2025); GenomeArena: 720 multiple-choice questions spanning variants, genes, pathways, interactions, miRNA targets, drug-target relationships, protein structure, and gene-phenotype associations (publicly available, code for benchmark generation provided)Code: Code for GenomeArena benchmark generation and all questions publicly available (specific URL not provided in text); Code for GenomeArena benchmark generation and all questions publicly available (location not specified in paper); LangGraph used for workflow construction (framework referenced but no specific repository URL provided); Code for GenomeArena benchmark generation and all questions are publicly available (specific GitHub link not provided in text)Extracted from: pdfAgreement 49%

Explore related topics

RAG for Research Data Analysis Benchmarking