AI evidence extraction

Position Statement on Artificial Intelligence (AI) Use in Evidence Synthesis Across Cochrane, the Campbell Collaboration, JBI, and the Collaboration for Environmental Evidence 2025

Ella Flemyng, Anna H Noel-Storr, Biljana Macura, Gerald Gartlehner, James A. Thomas, Joerg J Meerpohl et al. · Campbell Systematic Reviews · 2025

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

0/4

Quality (LMQS)

Evidence

Citations

5.00

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1002/cl2.70074

Methodology & findings

Study design

Position statement development based on joint organizational consensus and alignment with the RAISE (Responsible use of AI in evidence SynthEsis) recommendations framework.

Main result

The organizations state that "Evidence synthesists developing and publishing syntheses with Cochrane, the Campbell Collaboration, JBI, and the Collaboration for Environmental Evidence can use AI and automation as long as they can demonstrate that it will not compromise the methodological rigor or integrity of their synthesis." The statement emphasizes that while AI offers opportunities to make evidence synthesis "more timely, affordable, and sustainable," there are also "risks that misuse could erode methodological standards by exacerbating existing biases and reducing reliability."

Research paradigm

Normative/prescriptive (position statement)

Author conclusions

The authors conclude that "Our organizations and the joint Methods Group are committed to improving AI literacy for our authors and editors to help make these decisions around AI use. We are aligning our work with developments happening across the ecosystem, including the Evidence Synthesis Infrastructure Collaborative (ESIC 2025), the RAISE initiative, and the Digital Evidence Synthesis Tool INnovation for Yielding Improvements in Climate and Health (DESTINY) project, among others. This is to ensure that the members of Cochrane, the Campbell Collaboration, JBI, and CEE, as well as the wider field, have the resources and guidance they need to use AI responsibly, efficiently, and equitably within their evidence synthesis."

Risk of bias

Commercial bias in AI tool development; Lack of transparency in AI system design and validation; English-only or open-access-only data bias in AI training; Potential biases related to scope, domains, and breadth of training data; Risk of methodological standard erosion through AI misuse; Commercial interests driving AI development without transparency; Lack of validation and evaluation of proprietary AI systems; Potential biases from training data (e.g., English-only or open-access-only data); Opacity regarding AI system limitations; Risk of eroding methodological standards through misuse of AI; English-only or open-access-only data bias in AI training and testing; Commercial interests driving AI development without transparent limitation disclosure; Potential for AI misuse to exacerbate existing biases in evidence synthesis; Lack of independent validation and evaluation of AI systems

Limitations

The authors acknowledge that "While it is clear we need to make better use of AI for evidence synthesis to become more timely, affordable, and sustainable, we must also acknowledge the environmental and social costs associated with some forms of AI, particularly large-scale language models." Additionally, they note that "There are risks that misuse could erode methodological standards by exacerbating existing biases and reducing reliability" and that "current AI developments are often driven by commercial interests and, as such, are often opaque regarding limitations and lacking appropriate validation and evaluation."

Open questions raised

A key area for further guidance identified is "how to approach justifying the use of AI in the specific synthesis." The authors note that "Work is under way to develop a framework that guides evidence synthesists through these considerations, so decisions can be made transparently and consistently." They also indicate ongoing work through the Evidence Synthesis Infrastructure Collaborative (ESIC), the RAISE initiative, and the DESTINY project to improve AI literacy and provide resources for responsible AI use.
The authors identify that "A key area for further guidance is how to approach justifying the use of AI in the specific synthesis." They note that "Work is under way to develop a framework that guides evidence synthesists through these considerations, so decisions can be made transparently and consistently."
A key area identified for further guidance is "how to approach justifying the use of AI in the specific synthesis." The authors note that "Work is under way to develop a framework that guides evidence synthesists through these considerations, so decisions can be made transparently and consistently."

Extracted from: pdfAgreement 79%

Explore related topics

AI Peer Review Peer Review Position Paper