AI evidence extraction

Mapping the AI revolution: A bibliometric analysis of ChatGPT's role in academic writing

Sophia Binnendyk, Mansye Sekewael, Hedyan Putra · Journal on English as a Foreign Language · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

1/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.23971/jefl.v16i1.10313

Methodology & findings

Study design

Bibliometric analysis using VOSviewer software combined with systematic manual extraction and coding.

Sample

N = 237, 3 groups

Primary method

Bibliometric analysis using VOSviewer software for network visualization and cluster analysis. Keyword co-occurrence analysis with full counting method (minimum 2 occurrences threshold). Co-authorship analysis with minimum threshold of 2 published documents per author (later adjusted to 1). Manual qualitative coding for theoretical framework and methodology identification. Data exported in RIS format; preprocessing included merging synonymous terms, unifying singular/plural forms, and removing irrelevant keywords.

Main result

The analysis reveals that research on ChatGPT in academic writing has undergone a clear evolution in focus. "Early studies were largely shaped by concerns about academic integrity, plagiarism, authorship ambiguity, and the detectability of AI-generated texts, reflecting initial uncertainty regarding the implications of generative AI for higher education. As the technology became more widely adopted, research attention increasingly shifted toward pedagogical integration." The bibliometric mapping generated 13 color-coded clusters derived from 166 keywords from 237 peer-reviewed journal articles, with research progressing from ethical disruption (2023) through pedagogical negotiation (2024) to emerging investigation of cognitive dimensions (2025).

Reports effect sizes.

Research paradigm

Positivist/Quantitative (bibliometric analysis with qualitative manual coding)

Author conclusions

"The findings indicate a clear evolution in the focus of the literature. Early studies were largely shaped by concerns about academic integrity, plagiarism, authorship ambiguity, and the detectability of AI-generated texts... As the technology became more widely adopted, research attention increasingly shifted toward pedagogical integration... More recent publications extend these discussions to assessment design, institutional governance, personalization, metacognitive awareness, and self-regulated learning, indicating a gradual shift toward pedagogically grounded understandings of human-AI collaboration in writing." The authors further conclude that "the literature increasingly positions ChatGPT as a tool that can support process-oriented writing instruction and transparent assessment practices when integrated with teacher feedback and peer interaction."

Risk of bias

Language bias: Study restricted to English-language publications only; Database bias: Limited to Scopus-indexed articles; may exclude other academic databases; Publication bias: Analysis of journal articles only; may not capture conference proceedings, books, or grey literature; Subject area bias: Exclusion of publications outside Social Sciences and Computer Science categories; Temporal bias: No eligible publications retrieved for 2022; review period 2023-2025 only; Selection bias in manual coding: Subjective identification of theoretical frameworks by researchers; Language bias: restricted to English-language publications only; Database bias: Scopus-indexed articles only, excluding other databases; Publication bias: peer-reviewed journal articles only, excluding grey literature, conference proceedings, dissertations; Temporal bias: publication period 2023-2025 with no eligible 2022 publications despite search timeframe; Indexing bias: VOSviewer analysis relies on titles, abstracts, and author-assigned keywords which may not consistently represent theoretical frameworks or methodologies; Language bias (English-language only); Database bias (Scopus only, excluding non-indexed journals); Publication bias (journal articles only, excluding dissertations/grey literature); Indexing bias (keywords not consistently applied across publications)

Limitations

"The analysis is limited to English-language Scopus-indexed journal articles and therefore may not fully represent regional or multilingual scholarship." Additionally, the authors note that "VOSviewer maps keyword co-occurrences and term frequencies derived from titles, abstracts, and author-assigned keywords
However, it does not possess the analytical capacity to detect or classify theoretical frameworks or research methodologies, as such frameworks are not consistently or systematically indexed as keywords." The study also emphasizes that "much of the existing literature is based on perception-based surveys, small-scale classroom experiments, or short-term interventions."

Open questions raised

Limited longitudinal investigations: "Much of the existing literature is based on perception-based surveys, small-scale classroom experiments, or short-term interventions. Consequently, future research should move toward longitudinal investigations that examine how sustained engagement with generative AI influences writing development, disciplinary writing practices, and students' intellectual agency over time."
Underrepresentation of regional and multilingual scholarship
Need for integration of bibliometric mapping with systematic reviews and empirical studies
Limited experimental research establishing causal claims about ChatGPT's impact in authentic learning environments
Survey-based research limited by self-report biases
Limited capture of regional and multilingual scholarship due to English-language restriction

Extracted from: pdfAgreement 56%

Explore related topics

AI Manuscript Writing Academic Writing Literature Review