AI evidence extraction

OpenCLAW-P2P v6.0: Resilient Multi-Layer Persistence, Live Reference Verification, and Production-Scale Evaluation of Decentralized AI Peer Review

Francisco Angulo De Lafuente, Teerth Sharma, Vladimir Veselov, Seid Mohammed Abdu, Nirmal Tej Kumar, Guillermo Perry · ArXiv.org · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

2/2

Quality (LMQS)

Evidence

Citations

0.00

FWCI

Methodology & findings

Study design

Formal theoretical framework development with Lean4 proof verification, combined with production-scale system implementation.

Main result

The paper demonstrates that "OpenCLAW-P2P v7.0...introduces mathematical corrections to the theoretical framework, ensuring dimensional consistency, proper range constraints, and unambiguous notation throughout." Key operational achievements include: "The platform operates with 14 real autonomous agents (3 research agents, 5 architect meta-intelligence agents, and 6 recovery/specialist agents) alongside 23 labeled simulated citizens, producing 50+ scored papers with word counts ranging from 2 072 to 4 073 and leaderboard scores from 6.4 to 8.1." Additionally, "Multi-Layer Paper Retrieval Cascade reducing retrieval latency from >3 s to <50 ms for cached papers" and "Live Reference Verification system detecting fabricated citations with >85% accuracy."

Research paradigm

Formal systems and constructive mathematics with computational implementation

Author conclusions

The authors conclude: "OpenCLAW-P2P v7.0 demonstrates that decentralized, AI-driven scientific peer review is not only feasible but can be made resilient against operational failures and mathematically rigorous. The mathematical corrections introduced in this version—dimensional consistency in the progress-rate indicator, proper fixed-point conditions, fully specified reputation update formula, disambiguated notation, explicit range constraints, discrete-time PD Governor notation, HSR parameter definitions, and clarified theorem domains—ensure that the theoretical foundation matches the implementation quality of the production system." They further state: "OpenCLAW-P2P offers a viable model for the future of scientific publishing: open, transparent, continuously evaluated, resilient to infrastructure failures, mathematically grounded, and accessible to both human researchers and AI agents. All code is open-source, and we invite the research community to deploy, critique, and extend the platform."

Risk of bias

LLM judge ensemble bias despite diversity: empirical calibration correction (α=0.82, β=0.5) suggests systematic positivity bias of 1.5–2.0 points on 10-point scale; Small real agent pool (14 real agents vs. 23 simulated) may not reflect production heterogeneity; Tribunal IQ estimation (Equation 11) uses arbitrary thresholds without validation against standardized measures; Reference verification relies on three APIs; fabricated citations with sophisticated formatting may evade detection; No inter-rater reliability analysis reported for deception detectors; Sample size of papers (50+) is small for generalizable quality benchmarking; Self-published preprint without external peer review of v7.0 mathematical corrections; Global bias mitigation is addressed through "judge diversity: by running multiple independent LLMs from different providers, architectures, and training lineages, individual model biases are diluted through ensemble averaging." However, empirical bias quantification through inter-judge agreement analysis is noted as future work. The platform depends on 17+ external LLM providers, each with potential alignment biases.; LLM judge inflation bias: Empirical observation showed 1.5–2.0 point inflation on 10-point scale, partially mitigated by calibration affine correction (α=0.82, β=0.5); Cultural/political bias in training data: While judge ensemble spans multiple nations, formal quantitative analysis of bias reduction is marked as future work; Selection bias in agent cohort: 14 real agents vs. 23 simulated citizens; simulated agents may not represent realistic failure modes; Evaluation metric circularity: Tribunal scoring and multi-LLM scoring use overlapping dimensions; no external human-expert baseline comparison available; Possible favorable evaluation: Authors acknowledge they produce both the papers and evaluate them within their own system

Limitations

The authors state under Section 20.3 'Honest Limitations': "BenchClaw deployment has a known limitation: 'Newly generated judge instances are not yet visible in the production deployment
The issue is a missing push to Railway/Vercel
9 of the intended 17+ judges are confirmed operational.'" Additionally, multiple components are marked '[Future Work]', including "Dynamic question generation by examiner agents, creating an ever-expanding, agent-curated question bank" and "Formal analysis of inter-judge agreement patterns to quantify bias reduction." The paper explicitly notes that "[Future Work] Full Lean4 Compilation Verifier—Architecture defined" and "[Future Work] Inter-Judge Bias Analysis—Planned."

Open questions raised

Human-baseline comparison studies (scoring correlation with expert human reviewers)
Formal analysis of inter-judge agreement patterns using Krippendorff's α
Full Lean4 compilation verifier (currently in-process verification only)
Dynamic tribunal question generation by examiner agents
University deployment package standardization
CAJAL-27B language model training (script ready, resource-constrained)

Data: 50+ scored papers with metadata available on p2pclaw.com platform; CAJAL-4B language model: https://huggingface.co/Agnuxo/CAJAL-4B-P2PCLAW; CAJAL-9B v2 language model: Available on HuggingFace (Agnuxo/cajal-9b-v2-*); Paper repository: https://github.com/Agnuxo1/p2pclaw-papers; BenchClaw benchmark data: https://benchclaw.vercel.app/; The paper references "50+ scored papers" and provides access to the platform at "p2pclaw.com" and "benchclaw.vercel.app/". Code repositories and trained models are available on HuggingFace: "Agnuxo/CAJAL-4B-P2PCLAW" and "Agnuxo/cajal-9b-v2-*". Papers are persisted across four tiers including GitHub repository "Agnuxo1/p2pclaw-papers."; Papers in platform: 50+ scored papers available via p2pclaw.com (noted but specific URLs/download instructions not provided); Agent leaderboard scores: Table 23 provides top 10 agent scores (May 2026); Production deployment statistics: Table 22 documents 50+ papers with word counts 2,072–4,073 and scores 6.4–8.1Code: https://github.com/Agnuxo1/p2pclaw-papers (main paper storage); https://github.com/Agnuxo1/CAJAL (CAJAL language model training code); https://github.com/agnuxo1 (author's GitHub profile with related repos); cajal-p2pclaw PyPI package: https://pypi.org/project/cajal-p2pclaw/; https://gun.eco (Gun.js graph database); https://ipfs.tech (InterPlanetary File System); Agnuxo1/p2pclaw-papers (GitHub paper repository); Agnuxo/CAJAL (CAJAL language model GitHub); Agnuxo1/CAJAL-4B-P2PCLAW (HuggingFace model); https://pypi.org/project/cajal-p2pclaw/ (PyPI package); CAJAL language models: Agnuxo/CAJAL-4B-P2PCLAW on HuggingFace; CAJAL-9B v2: Agnuxo/cajal-9b-v2-* on HuggingFace; GitHub repository: Agnuxo1/p2pclaw-papers (used for GitHub sync service); BenchClaw deployment: https://benchclaw.vercel.app/; P2PCLAW platform: https://p2pclaw.com/app/benchmarkExtracted from: pdfAgreement 43%

Explore related topics

AI Peer Review Peer Review Design Science