AI evidence extraction

Responsible use of large language models in manuscript authorship, peer review, and editorial processes: a Delphi consensus among editors-in-chief of anaesthesia and pain medicine journals (RULE-AP)

Alessandro De Cassai, Burhan Dost, John Augoustides, Leonard Azamfirei, Zekeriyya Alanoğlu, L. M. T. D. A. Azi et al. · British Journal of Anaesthesia · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

1/4

Quality (LMQS)

Evidence

Citations

273.51

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1016/j.bja.2026.01.029

Methodology & findings

Study design

Modified Delphi consensus process with three mandatory rounds (one qualitative open-ended round followed by two quantitative rating rounds using 9-point Likert scales) and an optional fourth validation round.

Main result

The RULE-AP Delphi consensus yielded 59 statements providing guidance on responsible LLM use in academic publishing. The working group found that "Permitted, disclosed, and supervised use of LLMs, in line with each journal's policy, does not diminish the scientific quality or integrity of a manuscript and should not be perceived negatively by authors, reviewers, or editors." Additionally, the consensus emphasized that "LLMs cannot be listed as authors because they do not meet authorship criteria" and that "Human authors retain ultimate responsibility for the accuracy, integrity, and originality of manuscripts, even when LLMs are used for support." The statements affirmed that appropriate uses include language editing, formatting, and summarization, but prohibited uses include generating primary data, references, conclusions, or entire manuscripts.

Research paradigm

Consensus-based expert judgment (Delphi method); interpretivist/constructivist

Author conclusions

The RULE-AP Delphi project provides editor-informed guidance for the responsible use of LLMs in manuscript preparation, peer review, and editorial workflows. Key principles include transparency, human accountability, and human verification of all tasks and outputs. This guidance reflects current knowledge but may quickly be superseded as technology and understanding evolve. Additional caution is recommended for areas lacking strong consensus. The working group emphasizes that "Permitted use does not diminish manuscript quality, provided that it is transparently disclosed, carefully verified by humans, and remains under full human accountability."

Risk of bias

Selection bias: 36.0% participation rate (53 of 147 eligible journals); 62.5% of journals provided no feedback; Response bias: Inherent to Delphi method—reflects opinions of participants rather than empirical evidence; Expertise bias: Panel composed solely of journal editors-in-chief without representation from AI experts or ethicists; Attrition: Variable participation across rounds (83.0% in Round 1, 86.8% in Round 2, 79.2% in Round 3); only 67.9% participated in all rounds; Geographic/institutional bias: Predominance of North America (19 members) and Europe (18 members) despite global representation; Selection bias due to 36.0% participation rate (53 of 147 eligible journals); Non-response bias from 62.5% of journals providing no feedback; Participant expertise bias: EICs and delegates only, no AI or ethics specialists; Geographic bias with predominance of North America (19/50) and Europe (18/50); Inherent subjectivity of Delphi method dependent on participants' experiences; Potential for dominant personalities despite anonymization in iterative rounds; Selection bias: 36.0% participation rate (53 of 147 eligible journals) may not be representative; Attrition bias: Only 79.2% participated in Round 3; 67.9% participated in all rounds; Expertise bias: Panel consists only of EICs and editorial board members, not AI experts or ethicists; Geographic bias: Predominantly North America (19) and Europe (18) members; limited African representation (1); Methodological bias: Delphi method is inherently subjective and consensus-dependent, not empirical

Limitations

This Delphi consensus reflects the perspectives of EICs and their delegates from anaesthesiology and pain medicine journals and "does not include experts in AI, ethics, or broader scientific publishing
Consequently, the identified risks and recommended practices should be interpreted as the viewpoint of journal editors managing LLMs in their daily editorial workflows, rather than as definitive, evidence-based recommendations for the wider community of authors, reviewers, and publishers." Additionally, "The participation rate (36.0% of eligible journals) may introduce selection bias, and although the Delphi method facilitates structured consensus, it remains inherently subjective and dependent on the expertise and experiences of participants." The authors note that "the rapidly evolving nature of LLM technology means that guidance will likely need regular updating to remain relevant and aligned with emerging evidence and ethical standards."

Open questions raised

The authors note that because LLM technology is rapidly evolving, these recommendations should be regarded as a living framework that requires regular updates. The working group highlights that education in the responsible use of LLMs is of paramount importance. Areas of consensus but not strong consensus (e.g., use of LLMs for manuscript triage and improving reviewer language) are identified as boundary zones requiring particular caution and further deliberation.
Need for regular updates to guidance as LLM technology evolves
Importance of education in responsible LLM use highlighted as paramount
Lack of expert involvement from AI and ethics specialists in this consensus
Need for clearer policies on methodological 'grey zone' applications (conceptual mapping, summarization, reporting guideline compliance)
Additional research needed on risks associated with model collapse and AI-generated content proliferation

Data: Anonymised voting data for all rounds provided in Supplementary material 2 in accordance with FAIR principles. OSF registration: osf.it/973vf; Anonymised voting data for all rounds available in Supplementary material 2; Open Science Framework repository (osf.it/973vf); Anonymized voting data for all rounds available in Supplementary material 2; Open Science Framework (osf.it/973vf)Extracted from: pdfAgreement 78%

Explore related topics

Responsible AI Use Peer Review Qualitative