AI evidence extraction

Generative AI and higher education: a review of claims from the first months of ChatGPT

Lasse X Jensen, Alexandra Buhl, A. V. N. L. Sharma, Margaret Bearman · Higher Education · 2024

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

Evidence

Citations

9.53

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1007/s10734-024-01265-3

Methodology & findings

Study design

Scoping review methodology combined with critical thematic analysis.

Main result

The analysis identified ten claims about ChatGPT and higher education across three categories: claims about ChatGPT's nature, claims about changing institutional and teaching practices, and claims about new student practices. The authors found that "Overall, the claims present a positive perspective on AI in higher education. While being perceived as a disruption of the status quo, the authors generally frame AI as a catalyst for existing agendas, e.g. assessment reform, personalisation, or inclusion." The study reveals that "the claims mainly portray students as either plagiarists or victims of a failing educational system," with limited discussion of productive student uses of AI.

Research paradigm

Interpretive/hermeneutic (critical discourse analysis of grey literature)

Author conclusions

"This analysis of the early grey literature on ChatGPT alerts us to claims about the technology and its role in the future of higher education. While the examined literature contains some caveats about its possible negative influences, particularly concerning the decoupling of thinking and writing, the majority of articles were highly positive. As we move forward and improve our understanding of genAI and its effect on teaching and learning, those working in higher education may use this synthesis to be mindful of how our early assumptions can influence what and how we teach and research." The authors warn against over-optimism and emphasize the need for deeper critical consideration of risks and ethical considerations, interrogation of existing educational agendas, and ensuring students are part of conversations about generative AI.

Risk of bias

Selection bias: Only five English-language higher education sector outlets were searched; many discussions occurred on Twitter and webinars; Geographic/linguistic bias: 25 of 45 articles authored from the USA alone; only one article from a low- or middle-income country (South Africa); Temporal bias: Analysis limited to first 3 months post-launch; may not reflect more mature perspectives; Publication source bias: Grey literature (opinion pieces, blogs) rather than peer-reviewed research; Novelty effect: Authors acknowledge tendency for new technologies to be judged more positively simply because they are new; Hype cycle effect: Early inflated expectations typical of new technologies; Author bias: nearly all authors employed in higher education sector, potentially reflecting institutional perspectives; conversations in Twitter, webinars, and other platforms excluded; limited diversity of perspectives

Limitations

"The main limitation to consider relates to the way we identified the articles
Searching sector outlets captures only a subset of perspectives
In the first months after ChatGPT's release, many conversations took place in other spaces, such as Twitter or various online webinars
Furthermore, owing to our search strategy, the included papers have a strong bias towards authors from anglophone high-income countries
This means that most of the perspectives described in this review reflect fairly similar traditions of teaching and learning." The 3-month timeframe may have captured less mature perspectives than a longer observation period would provide.

Open questions raised

Gap between broad-scale concerns and fine-grained research that responds to them: While emerging empirical literature focuses on micro-issues (e.g., distinguishing ChatGPT writing from human writing), broader concerns about thinking-writing decoupling and ethical engagement are underexplored
Lack of empirical evidence on actual impacts: Most early claims are speculative rather than grounded in empirical research
Student voice absent: Claims about how students may productively use genAI were least prevalent (21 of 45 articles)
more research involving students directly is needed
Limited perspectives from non-English-speaking countries and low/middle-income contexts
Need for more mature perspectives beyond the initial 3-month period

Data: Supplementary materials table provided with article title, author name, author location, outlet, publication date, and word count for each of the 45 included articles (available at https://doi.org/10.1007/s10734-024-01265-3).Extracted from: pdf

Explore related topics

AI Manuscript Writing Academic Writing Literature Review