AI evidence extraction

Computer-generated content and the AI/ML retraction record, 2018-2026: a characterization study

Anton Sokolov · Open MIND · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

0/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.5281/zenodo.20409915

Methodology & findings

Study design

Descriptive characterization study analyzing the 13,502-record AI/ML slice of Retraction Watch preserved in the Nekropolis working corpus v0.2.

Primary method

Characterization study with curated public-record corpus construction and systematic categorization scheme

Main result

Computer-generated content is the largest single inferred cause in the AI/ML retraction slice: "4,704 records, 34.8 % of the slice, ahead of compromised peer review (22.0 %), plagiarism (18.6 %), and editorial process (13.7 %)". The dominance holds on the conservative title-level subset (3,243 records, 41.8 %) and on the broad-CS-metadata tier (32.7 %); "84 % of CGC-retracted papers were published in 2021–2023". In 2024, "the count remained an order of magnitude above the 2018–2020 baseline (487 vs 0–38), although plagiarism (562) and compromised peer review (503) edged ahead of CGC that year".

Research paradigm

Positivist/empiricist (quantitative characterization of public records)

Author conclusions

"The integrity system has responded to generated content through retraction rather than through visible prevention at submission; the first line of defence — the submission gate — is not yet visible in the public record." The authors conclude that while retraction has become the dominant response mechanism to computer-generated content, prevention mechanisms at the submission stage remain absent from the public record, indicating a reactive rather than proactive approach.

Risk of bias

Recall-oriented sampling (inclusion of broad computer-science metadata rather than title-level AI/ML evidence only); Reliance on single data source (Retraction Watch) with potential source-level biases; Coarse, non-authoritative inferred-cause labeling scheme (single label per record from fixed vocabulary); Potential indexing and coverage biases in Retraction Watch itself; Recall bias: records rest substantially on broad computer-science metadata rather than title-level AI/ML evidence; Source bias: analysis limited to Retraction Watch public records only; Classification bias: inferred-cause labels are single coarse labels from fixed vocabulary, not authoritative determinations; Temporal bias: differential publication and retraction patterns across 2018-2026 window; Recall bias in classification (substantial share of records based on broad computer-science metadata rather than title-level AI/ML evidence); Source dependency (analysis limited exclusively to Retraction Watch records); Single-coder inference risk (project inferences, not authoritative determinations); Temporal coverage gaps (2018-2026 window may miss earlier or later trends)

Limitations

The study reports "only on the Retraction Watch source family within Nekropolis v0.2
the inferred-cause labels are project inferences (a single coarse label per record from a fixed vocabulary) and not authoritative determinations"
Additionally, "the slice is recall-oriented (a substantial share of admitted records rest on broad computer-science metadata rather than on title-level AI/ML evidence)" and the authors note that "Tyche Institute is a research and education entity, not a trust-service provider".

Open questions raised

The paper identifies that prevention mechanisms at the submission stage are not yet visible in the public record. Companion papers address related gaps: Paper B addresses public-record opacity, Paper C develops a temporal awareness benchmark, and Paper E provides additional analysis of the AI-content retraction regime.
The study identifies that the submission-stage gate for preventing computer-generated content is not yet visible in the public record, suggesting a gap in preventive integrity mechanisms. Companion papers in preparation (Papers B, C, and E of the Nekropolis programme) are indicated as addressing related gaps in public-record opacity, temporal awareness, and the AI-content retraction regime.
The paper identifies that submission-gate prevention mechanisms for AI-generated content are not yet visible in the public retraction record, suggesting a research gap in understanding pre-submission quality controls and prevention strategies.

Data: not_statedCode: not_statedExtracted from: pdfAgreement 60%

Explore related topics

Citation Integrity Research Integrity Content Analysis