AI evidence extraction

Epistemic Transformations: Large Language Models and the Reconfiguration of Scholarly Knowledge Production

Jonathan Westover · Preprints.org · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

2/4

Quality (LMQS)

Evidence

Citations

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.20944/preprints202603.2286.v1

Methodology & findings

Study design

Critical narrative synthesis drawing on multiple literatures including Science and Technology Studies, philosophy of science and mind, research ethics, information science, and empirical literature on AI in research contexts.

Main result

The integration of Large Language Models into scholarly production "may extend beyond enhanced efficiency to fundamental transformations in how knowledge is produced, validated, and understood." The analysis reveals that "LLMs are not merely neutral tools but participants in networks of knowledge production that may reshape practices in ways requiring careful examination." The paper concludes that "the implications of LLM integration vary substantially across disciplines, reflecting different epistemic cultures and practices" and that "what counts as appropriate use depends on what a discipline values and how it produces knowledge."

Reports effect sizes.

Research paradigm

Critical interpretive; Science and Technology Studies (STS) framework with hermeneutic and reflexive orientation

Author conclusions

The authors conclude that "The integration of Large Language Models into scholarly production represents a development whose implications extend beyond questions of efficiency and appropriate use to fundamental transformations in how knowledge is produced, validated, and understood." They emphasize that "the future is not predetermined. Different scholarly communities may adopt different practices; resistance to LLM integration may prove as significant as adoption. The choices made during this formative period will shape trajectories for decades." Most importantly, they assert that "Thoughtful integration that leverages genuine benefits while safeguarding epistemic values can strengthen research practices. However, this requires neither uncritical adoption nor reflexive rejection but sustained, critical engagement that attends to disciplinary specificity, global diversity, temporal dynamics, and the fundamental questions about scholarship that LLMs bring to the fore."

Risk of bias

Selection bias in surveyed adoption literature (researchers willing to respond may differ systematically from non-respondents); English-language predominance in source selection despite stated efforts toward inclusivity; Author's disciplinary expertise and positionality may shape case study interpretations; Temporal bias: rapid evolution of LLM capabilities makes observational claims potentially outdated; Literature selection bias toward visible/accessible sources; Selection bias in literature review: reliance on English-language sources despite efforts to broaden scope; Author disciplinary expertise limitations: case studies in three disciplines may not generalize; Temporal obsolescence: rapid LLM capability changes may outpace analysis; Interpretation bias: analysis shaped by author's disciplinary training and positionality; Language bias: English-language predominance in literature search despite targeted efforts to identify non-Anglophone scholarship; Selection bias: Literature identification through academic databases may miss non-indexed scholarship and grey literature; Author's disciplinary positioning bias: Author's expertise does not span all fields equally; case study interpretations may reflect particular disciplinary perspectives; Temporal bias: Analysis reflects snapshot of rapidly evolving technology; LLM capabilities and adoption patterns changing faster than publication cycles; Theoretical bias: STS frameworks reflect particular intellectual traditions; other theoretical perspectives not equally weighted; Citation bias potential: Discussion of LLM bias propagation notes that LLMs may over-represent frequently-cited sources, reinforcing existing citation hierarchies

Limitations

The authors acknowledge several limitations: "Rapidly evolving landscape: LLM capabilities and adoption patterns change faster than traditional publication cycles
Some observations may be outdated by publication." "Limited empirical base: Much discussion of LLM effects remains speculative
Robust empirical research on actual impacts is only beginning to emerge." "English-language predominance: Despite efforts to incorporate non-Anglophone scholarship, the analysis draws primarily on English-language sources
This reflects both the author's linguistic limitations and the structure of academic databases, but it means perspectives from other scholarly traditions are underrepresented." "Disciplinary limitations: While the article addresses disciplinary variation through case studies, the author's expertise does not span all fields equally
The case studies represent informed interpretations that scholars from those disciplines might contest or refine." The authors note that "normative uncertainty: The article proposes frameworks for 'responsible' integration, but what counts as responsible remains contested."

Open questions raised

Limited empirical evidence on actual adoption patterns and effects of LLM use across disciplines
Lack of longitudinal studies comparing researchers with and without extensive LLM use regarding skill development
Systematic evaluation of whether LLM-generated research directions lead to productive research is lacking
Empirical study of verification effectiveness - whether researchers can reliably identify errors in LLM outputs
Research on long-term effects of different training trajectories for researchers trained with LLMs from early career stages
Global evidence gaps, particularly underrepresentation of non-Anglophone scholarship and Global South perspectives

Extracted from: pdfAgreement 70%

Explore related topics

Responsible AI Use Multi-Domain Literature Review