AI evidence extraction

ChatGPT in higher education - a synthesis of the literature and a future research agenda

Pritpal Singh Bhullar, Mahesh Joshı, Ritesh Chugh · Education and Information Technologies · 2024

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

8/10

Relevance

3/4

Quality (LMQS)

Evidence

Citations

9.93

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1007/s10639-024-12723-x

Methodology & findings

Study design

Systematic literature review using PRISMA guidelines with bibliometric analysis.

Sample

N = 47, 6 groups

Primary method

Bibliometric techniques including citation analysis, bibliographic coupling, co-occurrence analysis, and coauthorship analysis. VOSviewer clustering based on shared bibliographic references. Cohen's kappa coefficient used to assess inter-rater agreement for thematic categorization (κ = 0.69, indicating substantial agreement). Keywords with minimum frequency n = 5 included in analysis. Pairwise comparison method used for co-occurrence calculation.

Main result

The study found that "the use of ChatGPT in higher education, particularly issues of academic integrity and research, has been studied extensively by scholars in the United States, who have produced the largest volume of publications, alongside the highest number of citations." Additionally, "Four distinct thematic clusters (academic integrity, learning environment, student engagement, and scholarly research)" were identified, with plagiarism identified as "a significant concern in the use of ChatGPT, which may reduce students' ability to produce imaginative, inventive, and original material."

Reports effect sizes and confidence intervals.

Research paradigm

Mixed-methods (predominantly qualitative interpretation with bibliometric quantification)

Author conclusions

The authors conclude that "This study offers valuable insights into the current state of ChatGPT in higher education literature, providing essential guidance for scholars, researchers, and policymakers." They emphasize that while ChatGPT presents opportunities for personalized learning and educational support, "plagiarism is a significant concern in the use of ChatGPT, which may reduce students' ability to produce imaginative, inventive, and original material," and that "educational institutions should invest in comprehensive educational programs that not only teach students the ethical use of technology but also incorporate rigorous assessments of critical thinking skills."

Risk of bias

Language bias: Only English-language papers included, excluding non-English publications; Database bias: Single database (Scopus) used; Web of Science and other databases not included; Publication bias: Only peer-reviewed journal articles included; conference proceedings explicitly excluded; Temporal bias: All included articles published in 2023, reflecting emerging nature of topic with limited citation history; Geographic bias: Potential overrepresentation of certain countries in Scopus indexing; Selection bias: Search query terms may not capture all relevant ChatGPT education literature; Publication bias (only peer-reviewed journal articles included, conference proceedings excluded); language bias (English-only); database selection bias (single database); recency bias (all papers from 2023, limiting retrospective analysis); temporal bias (nascent research area with limited citation accumulation).; Single database bias (Scopus only, despite PRISMA acceptance); Language bias (English-only publications); Publication type bias (excluded conference proceedings); Temporal bias (all articles from 2023, limiting citation maturity); Selection bias in inclusion/exclusion criteria

Limitations

The study acknowledges that "reliance on a single database is acceptable within the PRISMA framework," however, the authors did not search other major databases such as Web of Science
The limitation to English-language publications only and exclusion of conference proceedings may have restricted the scope of findings
Additionally, the study notes that "the limited citation count, as anticipated, underscores that publications from 2023 are in the early stages of gaining visibility and recognition within the academic community," which constrains the analysis of impact.

Open questions raised

The authors identify several research gaps: (1) limited research on potential dangers of sophisticated chatbots like ChatGPT in higher education through systematic literature review and bibliometric techniques; (2) need for targeted training programs for students and educators on effective ChatGPT utilization for non-native English speakers; (3) requirement for strategic partnerships between institutions and AI researchers to develop context-specific ChatGPT applications; (4) need for investigation of ChatGPT's role in specific disciplines and field-specific research contexts; (5) necessity for frameworks addressing detection of AI-generated content and academic integrity verification.
The paper identifies the need for: (1) development of targeted training programs for students and educators on effective ChatGPT utilization; (2) research on context-specific applications of ChatGPT for diverse educational needs; (3) further investigation of ChatGPT's potential dangers in higher education through systematic literature review and bibliometric techniques; (4) exploration of strategic partnerships between educational institutions and AI researchers; (5) deeper investigation of various aspects of ChatGPT's potential in higher education.
Limited research on potential dangers of sophisticated chatbots like ChatGPT in higher education through systematic literature review lens
Need for deeper investigation of ChatGPT's ramifications across various disciplines from faculty and student perspectives
Lack of research on integration strategies and context-specific applications tailored to diverse educational needs
Need for targeted training programs for students and educators emphasizing effective utilization of ChatGPT

Data: The authors state "The datasets generated during and/or analysed during the current study are available from the corresponding author upon reasonable request."; "The datasets generated during and/or analysed during the current study are available from the corresponding author upon reasonable request."Code: Not mentionedExtracted from: pdfAgreement 54%

Explore related topics

Citation Integrity Research Integrity Literature Review