AI evidence extraction

ChatGPT in higher education: Considerations for academic integrity and student learning

Miriam Sullivan, Andrew Kelly, Paul McLaughlan · Journal of Applied Learning & Teaching · 2023

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

8/10

Relevance

2/4

Quality (LMQS)

Evidence

740

Citations

26.21

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.37074/jalt.2023.6.1.17

Methodology & findings

Study design

Content analysis of 100 news articles from Australia, New Zealand, the United States, and the United Kingdom published between 2020 and February 2023.

Sample

N = 100, 1 group

Primary method

Content analysis following Neuendorf et al.'s (2017) guidebook; NVivo software for coding and analysis; NVivo's Sentiment Analysis tool for positive/negative valence; NVivo's Query tool for word frequency counts. Thematic coding with cross-moderation by multiple authors for reliability checking.

Main result

The study found that "all articles contained both positive and negative language" with "relatively balanced in the number of times positive (n=912) and negative (n=1034) language was coded." The most common themes were "general concerns about academic integrity (n=87) and ways that students could be discouraged from using ChatGPT (n=87)," while "there were fewer articles that discussed how and why ChatGPT could be used productively in teaching (n=58)." Notably, "the primary voice being portrayed in the media was that of the university (n=79)" while "student voices were only quoted in 30 articles."

Reports effect sizes.

Research paradigm

Interpretivist/qualitative

Author conclusions

"While there has been plenty of controversy surrounding the release of ChatGPT and its implications for higher education, there are clear opportunities to enhance student learning and access. This content analysis of news articles highlighted that the public discussion and university responses about ChatGPT have focused mainly on academic integrity concerns and innovative assessment design." The authors emphasize that "there is potential for AI tools to enhance student success and participation from disadvantaged backgrounds" and that "academics and university representatives should be aware of the frames they choose to discuss when engaging with the media, as news coverage can influence social norms towards student cheating behaviour and public perceptions of universities."

Risk of bias

Selection bias: Only mainstream news databases searched; alternative news sources excluded; Geographic bias: Coverage limited to Western countries (Australia, New Zealand, US, UK); Publication bias: High number of duplicates and text-sharing between outlets suggests potential PR influence; Timing bias: Articles collected during February 2023; coverage may reflect early adoption phase bias; Media bias: Reliance on journalistic framing which may overrepresent academic integrity concerns over learning benefits; Selection bias: Only English language newspapers and online news sources included; limited to four Western countries; Source bias: Unclear whether media coverage was initiated by journalists versus university PR/media releases; Sampling bias: Only 100 articles analyzed; high degree of text-sharing and reuse between outlets suggests non-independent data; Temporal bias: Coverage limited to 2020-February 2023, capturing early discourse during rapid technology evolution; Geographic bias: Over-representation of United States news coverage relative to other regions; Selection bias: Study limited to mainstream news databases and indexed newspapers only; alternative news sources not explored; Publication bias: High number of duplicate articles suggests concentrated media coverage; Selection bias: Limited to English-language articles only; Geographic bias: Coverage limited to four Western countries (Australia, New Zealand, US, UK); Source bias: Unclear proportion of coverage initiated by journalists versus university PR/media releases

Limitations

"Our study covered a relatively small number of media articles (N=100)." Additionally, "we analysed coverage in mainstream news databases and did not explore alternative news sources." The authors note that "it is also unclear how much of the media coverage was initiated by journalists compared to media releases and PR from universities, which have an increasing influence on news coverage." Furthermore, "we also only examined news coverage in select Western countries, contributing to the imbalance in academic studies of Western news, particularly news from the United States."

Open questions raised

Limited academic literature on ChatGPT and generative AI tools at time of writing
Lack of student voice in public discourse about ChatGPT
Insufficient discussion of ChatGPT's potential for enhancing participation for disadvantaged students
Missing perspectives from industry and workplace context
Need for research on student perceptions through surveys and focus groups
Need for research on academic staff views on ChatGPT and actual extent of use

Data: Not mentioned; news articles sourced from public databases (Newsbank, ProQuest, etc.)Code: Not mentionedExtracted from: pdfAgreement 59%

Explore related topics

Responsible AI Use Academic Writing Content Analysis