AI evidence extraction

Attitudes and Perceptions Toward the Use of Artificial Intelligence Chatbots for Peer Review in Medical Journals: A Large-Scale, International Cross-Sectional Survey

Jeremy Y. Ng, Daivat Bhavsar, Neha Dhanvanthry, L.M. Bouter, Teresa M. Chan, Holger Cramer et al. · medRxiv · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

2/4

Quality (LMQS)

Evidence

Citations

8.72

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.64898/2026.04.07.26350263

Methodology & findings

Study design

Large-scale, online, closed cross-sectional survey of medical journal peer reviewers.

Sample

N = 1077, 3 groups

Primary method

Descriptive statistics (frequencies and percentages) calculated using Microsoft Excel 2007. Demographic information analyzed for trends and patterns. Qualitative data from open-ended questions underwent inductive coding and thematic analysis by two independent authors (DB and ND) with consensus-building process. Codebook developed iteratively based on first three responses for one question, then applied to all remaining responses. Individual codes grouped into themes independently by two authors and finalized through consensus; conflicts resolved through discussion with third author (JYN). No inferential statistical testing reported.

Main result

The study found that "most respondents (n=917/1064, 86.2%) were familiar with the concept of AICs, and 25.6% (n=272/1064) stated they are very familiar with the concept." Additionally, "most respondents reported that their research institution does not provide any training on using AICs appropriately in the peer review process (n=741/1067, 69.5%)." Regarding potential benefits, "the most common benefits of AICs included enhanced quality, clarity and readability of peer review reports by suggesting edits to grammar, language, tone, or structure (n=652/1013, 64.4%); reduced workload for peer reviewers (n=634/1014, 62.5%); increased speed and efficiency of conducting reviews (n=586/1014, 57.8%)." However, "the most agreed upon challenges of AICs included biased or skewed evaluations due to AIC algorithm limitations (n=794/989, 80.3%), risks of producing errors or inaccuracies due to limited comprehension of jargon and limitations inherent to the AICs (n=784/989, 79.3%)."

Reports effect sizes.

Research paradigm

Positivist/empiricist

Author conclusions

"This survey provides insight into peer reviewers' attitudes and perceptions about AICs for peer review processes in medical journals. Many peer reviewers indicated in this survey that although they had not used AICs for peer review purposes, they were interested in learning more and receiving training on how to use them responsibly to reap the potential benefits in peer review processes." The authors further conclude: "Hence, given the growing interest, academic journals and publishers may look into developing available training opportunities and policies to help their peer reviewers understand the possible impacts of using AICs in their work of peer review. These steps would be crucial in helping journals responsibly incorporate AICs by preserving the rigour and integrity of peer review processes in medical scholarly publishing."

Risk of bias

Recall bias; Selective non-response bias; Social desirability bias; Response bias; Language bias (non-English speaking researchers excluded); Selection bias (corresponding authors only; active researchers in past 2 months); Volunteer bias (invitees with strong opinions more likely to respond); Recall bias inherent to cross-sectional surveys; Social desirability bias - peer reviewers may under-report AIC use due to concerns regarding professional and legal consequences; Response bias - invitees with strong opinions for or against AICs may have been more inclined to respond, underrepresenting those with limited experience; Language bias - non-English speaking researchers largely excluded due to survey being in English; Selection bias - convenience sampling of corresponding authors from MEDLINE journals; only those from past 2 months included; Social desirability bias regarding AIC use disclosure due to professional/legal concerns; Language exclusion bias (survey in English only, excluding non-English speaking researchers); Response bias (strong opinions may have been overrepresented); Low overall response rate affecting generalizability; Convenience sampling framework targeting corresponding authors

Limitations

"As with all cross-sectional surveys, there is the potential for recall bias, and selective non-response bias, which may impact the validity of our findings
Furthermore, self-reported attitudes and perceptions may be weak predictors of implicit biases and actual behaviour." Additional limitations include: "Another major limitation is that non-English speaking researchers are largely excluded from our sample due to language constraints, which could affect the applicability of our findings to those who primarily publish in languages other than English." The authors also note: "The overall response rate was relatively low, which further impacts generalizability
However, the final total number of participants nevertheless represent a relatively large sample size due to the scale of our sampling." Furthermore, "perspectives surrounding AIC use, and AIC uses themselves, are continuously evolving" affecting the cross-sectional analysis.

Open questions raised

Scarcity of research on attitudes and perceptions of peer reviewers toward AICs use
Need for development of clear ethical frameworks to guide responsible integration of AICs in peer review
Need for academic journals and publishers to develop training opportunities and policies regarding AIC use in peer review
Need for future research to address limitations of AICs (hallucinations, generic output) and continuous improvement
Need for longitudinal assessments of the impact of AIC adoption on quality and integrity of peer review process
Need for cross-disciplinary research beyond medical journals

Data: Deidentified survey data available on OSF at https://doi.org/10.17605/OSF.IO/VARJE; Complete study protocol with data analysis plan at https://doi.org/10.17605/OSF.IO/FHC2M; Survey instrument (Appendix 2) at https://osf.io/varje/files/c748y; Thematic analysis tallies (Appendix 6) at https://osf.io/varje/files/u3ypj; Deidentified data available on Open Science Framework (OSF) at https://doi.org/10.17605/OSF.IO/VARJE; Survey materials available at https://osf.io/varje/files/c748y; Search strategy details available at https://osf.io/varje/files/ncajv; Thematic analysis results (themes and code frequencies) available at https://osf.io/varje/files/cehpj; Full thematic analysis tallies available at https://osf.io/varje/files/u3ypj; Supplemental quantitative analysis of assigned codes available at https://osf.io/varje/files/23h4y; Deidentified data available on OSF: https://doi.org/10.17605/OSF.IO/VARJE; Study protocol and data analysis plan on OSF: https://doi.org/10.17605/OSF.IO/FHC2MExtracted from: pdfAgreement 60%

Explore related topics

AI Peer Review Peer Review Survey