AI evidence extraction

ChatGPT and a new academic reality: Artificial Intelligence‐written research papers and the ethics of the large language models in scholarly publishing

Brady Lund, Ting Wang, Nishith Reddy Mannuru, Bing Nie, Somipam R. Shimray, Ziang Wang · Journal of the Association for Information Science and Technology · 2023

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

0/4

Quality (LMQS)

Evidence

769

Citations

27.20

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1002/asi.24750

Methodology & findings

Study design

Narrative review and conceptual analysis of ChatGPT, GPT-3, and related large language models, with examination of ethical implications for scholarly publishing; includes analysis of capabilities, limitations, and potential applications through literature synthesis and critical discourse analysis..

Main result

The study found that "the potential for bias in the training data and coding process of AI-driven language models such as GPT-3 poses a threat to the integrity of science." Additionally, the analysis reveals that "ChatGPT can assist editors and peer reviewers in completing repetitive or tedious tasks (e.g., correcting grammatical errors) and avoid making biased judgments about articles," though "if biased individuals train ChatGPT, it is unclear whether or how ChatGPT mitigates existing issues."

Research paradigm

Critical interpretive analysis

Author conclusions

The authors conclude that "it is necessary to ensure that they [ChatGPT and GPT-3] are used ethically and responsibly for scholarly research and publishing." They further state that "Researchers, publishers, and the developers of AI-driven language models must collaborate to establish guidelines and protocols to ensure that the use of these technologies is ethical, transparent, and accountable. Failure to do so may undermine public trust in the scientific process and have far-reaching consequences for the future of research and innovation."

Risk of bias

Training data bias: Models trained on web-based datasets contain bias regarding gender, race, ethnicity, and disability status; Lack of diversity guarantee: Size of dataset does not guarantee diversity; Stochastic distortion: Models described as 'stochastic parrots' regurgitating information distorted by randomness; Misinformation propagation: NLP algorithms lack skill in identifying and resisting misinformation; Matthew Effect amplification: Citation-based ranking systems may exacerbate existing inequalities in academic publishing; Inherent bias in GPT-3 training data sourced from large web-based datasets showing bias regarding gender, race, ethnicity, and disability status; Selection bias in citation patterns (Matthew Effect) affecting which sources are cited; Risk of model perpetuating hidden and unwitting prejudice when generating academic research; Potential for distortion in outputs as models are described as 'stochastic parrots, regurgitating what they hear, often distorted by randomness'; Potential bias in training data of GPT-3 with regard to gender, race, ethnicity, and disability status; Temporal bias: analysis limited to the time of writing, may not reflect current model capabilities; Publication bias in academic literature that ChatGPT may perpetuate through citation patterns; Matthew Effect reinforcement through citation-based ranking systems

Limitations

The authors acknowledge that "the comparison of GPT/ChatGPT to other language models may only be applicable at the time of writing
As new language models are developed, the relative strengths and weaknesses of GPT/ChatGPT may change." Additionally, they note that "future research could explore the use of GPT/ChatGPT in conjunction with other language models or technologies in order to enhance their capabilities and performance" and that "it would be worthwhile to investigate the use of GPT/ChatGPT in different tasks and domains, as well as to consider the full range of existing language models."

Open questions raised

Many unanswered questions about the ethics of using GPT in academia and its impact on research productivity
Need for investigation of GPT/ChatGPT use in conjunction with other language models
Need to investigate use of GPT/ChatGPT in different tasks and domains
Need for comprehensive consideration of the full range of existing language models
Development of anti-ChatGPT detection software
Reevaluation of tenure criteria in higher education institutions

Extracted from: pdfAgreement 72%

Explore related topics

AI Manuscript Writing Academic Writing Conceptual