AI evidence extraction

AI And the Editors' Ghost: Who Is the Writer Now?

David Clark, David Nicholas, Abdullah Abrizah, John Akeroyd, Jorge Revez, Blanca Rodríguez‐Bravo et al. · Learned Publishing · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1002/leap.2051

Methodology & findings

Study design

Qualitative case study with thematic analysis.

Sample

N = 60, 1 group

Primary method

Google NotebookLM was used for generating thematic summaries of interview transcripts. The paper employs qualitative thematic analysis but does not report formal statistical methods, hypothesis testing, or quantitative analysis.

Main result

The study found that "An overarching and thematic summary of the data was produced in minutes, that would otherwise have occupied our research team for weeks." The AI-generated summaries using Google NotebookLM were "immediately plausible and coherent" and "regarded by all national interviewers as impressive," demonstrating the efficiency of AI tools in qualitative data analysis.

Reports effect sizes.

Research paradigm

interpretivist/qualitative

Author conclusions

The authors conclude that this study "is a calibration for our future data analysis," and they reflect that the paper is fundamentally "about how we used AI as an experiment, our reaction to it, how that chimes, resonates, echoes the experiences of the ECRs." This framing suggests their conclusions are exploratory and oriented toward future methodological refinement rather than definitive claims.

Risk of bias

Selection bias from convenience sampling; Small sample size reducing generalizability; Potential confirmation bias in assessing AI-generated summaries; Single AI tool (Google NotebookLM) without comparative analysis against other tools; Selection bias - sample limited to ECRs from specific countries; Researcher bias - authors' own impressions and reactions may influence interpretation; No blinding or independent verification of AI-generated summaries; Potential selection bias in ECR recruitment; Subjective impressions of AI output quality by national interviewers

Limitations

The paper uses "a relatively small, convenience sample," which represents a significant methodological limitation in terms of generalizability and representativeness
The authors acknowledge this constraint when they state they "compare the AI generated summaries both against our original data and those first impressions" using this limited sample.

Open questions raised

The authors identify the need for continued calibration and evaluation of AI tools in research data analysis, noting this paper serves as preparation for future data analysis phases of their longitudinal study.
The authors position this work as foundational for future data analysis, suggesting a need for more rigorous evaluation of AI tools in qualitative research. They indicate future phases of the 'Harbingers' project will build upon these initial findings.
The authors identify the need for future data analysis calibration and deeper exploration of how AI tools align with researchers' and early career researchers' experiences in scholarly communication.

Data: not_statedCode: not_statedExtracted from: pdf

Explore related topics

AI Manuscript Writing Academic Writing Qualitative