Large Language Models in Academia: Boosting Productivity but Reinforcing Inequality
Journal of the Association for Information Systems · 2025
AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.
Methodology & findings
Study design
Observational study using publication data analysis.
Sample
N = 218723, 3 groups
Primary method
Difference-in-differences analysis, generalized synthetic control analysis. Software and specific statistical packages not specified in abstract.
Main result
The study found that "publication rates rose by about 8%, with growth accelerating to 3.2% in 2023 and 12.8% in 2024" following the introduction of LLMs. Additionally, "Junior scholars benefited more than seniors, with the productivity gain declining roughly 1% per year of experience," while "native English-speaking (NES) researchers published more than their non-native English-speaking (NNES) peers, widening linguistic disparities."
Reports effect sizes.
Research paradigm
Positivist/empiricist
Author conclusions
The authors conclude that "LLMs boost scholarly productivity and lower barriers for early-career researchers, while also reinforcing inequities rooted in language proficiency."
Risk of bias
Selection bias: Sample limited to computer science scholars from top 194 U.S. universities only; Confounding: Unable to establish causal mechanisms distinguishing LLM adoption from other contemporary productivity changes; Measurement bias: Publication rates may not fully capture research quality or actual productivity; Geographic/institutional bias: Limited to U.S. institutions; Selection bias: Study limited to computer science scholars from top 194 U.S. universities; Temporal confounding: Publication rate changes may be attributable to factors other than LLM introduction; Missing confounders: No control for changes in research funding, institutional policies, or other technological advances; Language proficiency proxy: Native English-speaking status may not perfectly capture LLM benefits related to language; Selection bias: Analysis limited to computer science scholars from 194 top U.S. universities, which may not represent all academic disciplines or institutions; Confounding variables: Multiple factors other than LLM introduction could affect publication rates (funding changes, career stage effects, pandemic effects on 2019-2024 period); Language classification bias: Categorization of researchers as native vs. non-native English speakers may not capture variation in language proficiency or discipline-specific language barriers; Temporal confounding: The 2019-2024 period includes COVID-19 pandemic, which may have differentially affected publication patterns
Open questions raised
- Not explicitly stated in abstract
Explore related topics
Related papers
- Autonomous chemical research with large language modelsDaniil A. Boiko · 2023 · 809 citations
- To use or not to use ChatGPT in higher education? A study of students’ acceptance and use of technologyArtur Strzelecki · 2023 · 691 citations
- Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewersCatherine A. Gao · 2023 · 657 citations
- Teacher support and student motivation to learn with Artificial Intelligence (AI) based chatbotThomas K. F. Chiu · 2023 · 617 citations
- Do AI chatbots improve students learning outcomes? Evidence from a meta‐analysisRong Wu · 2023 · 469 citations
- Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performanceYizhou Fan · 2024 · 419 citations