AI evidence extraction

AI-Produced Humanities Research: On the Dangers of Technical Incrementalism

Thomas R. Metcalf · Philosophy & Technology · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

2/4

Quality (LMQS)

Evidence

Citations

13.26

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1007/s13347-026-01085-6

Methodology & findings

Study design

Philosophical argument with sociological analysis.

Main result

The study concludes that "LLM-produced humanities scholarship may help us discover important knowledge, LLMs will overproduce safe, technical, and insular work, crowding out radical innovation in humanities fields." The author argues that "if people begin to commonly submit LLM-generated humanities manuscripts, and nothing else changes (in a major and so-far-not-predicted way) about the present-day system of humanities scholarship, then technical incrementalism will be overproduced."

Reports effect sizes.

Research paradigm

Critical theory / Philosophy of technology

Author conclusions

The author concludes: "What we need to deal with today are the present and near-future capabilities of LLMs, which will soon allow for widespread, convincing LLM authorship in the humanities. We are now in a position to envision how LLM authorship will intensify existing problems with philosophy and, to some degree, with other humanities disciplines." Additionally: "if an impending problem will occur because of existing issues in a field—such as perverse incentives and unequal global access to the community of scholarship—then there's a bright side. We might at least hope that anticipating this problem also leads us to a broader conversation about how to solve those other issues now as well."

Risk of bias

Speculative/predictive argument prone to confirmation bias in premise selection; Author's philosophical positionality may favor certain disciplines (philosophy) over others; Limited empirical data collection for humanities fields beyond philosophy; Cherry-picking of supporting evidence (e.g., philosophical paradox-mongering examples); Speculative nature of predictions about future LLM behavior; Limited empirical observation of LLM-generated humanities scholarship (as of writing date); Selection of cited examples from philosophy may not generalize to all humanities; Author's apparent preference for radical innovation may introduce confirmation bias; Reliance on institutional data (acceptance rates, job markets) that may not capture actual scholarly quality or diversity; Selection bias in cited examples (paradox-mongering cases chosen to illustrate extremes); Confirmation bias in interpreting LLM capabilities; Speculative reasoning about future scenarios without empirical validation; Limited empirical data from humanities fields beyond philosophy; Author's own disciplinary perspective (philosophy) may skew analysis of other humanities

Limitations

The author explicitly acknowledges: "Still, some readers may suggest, at this point, that the argument as a whole is somewhat speculative
The premises are empirical
the argument-form is ampliative
and the thesis essentially belongs to sociology of academic philosophy, not to mainstream philosophy per se." The author further notes: "It is possible that we will observe LLMs' integration into the philosophical literature and this will not, after all, lead to the problem I have predicted." Additionally, "while it's impossible to survey and discuss every humanities field in the space we have here, I provided some evidence above from readership, historical focus, and job markets, to show that other humanities are similar to philosophy in several respects."

Open questions raised

Comprehensive data on humanities fields' institutional incentives beyond philosophy
Empirical measurement of LLM-generated manuscript prevalence over time
Detailed analysis of how different humanities traditions (Continental, non-Western) will be affected by LLM integration
Investigation of Kuhnian paradigm shifts in LLM-generated scholarship
Longitudinal studies tracking innovation versus technical incrementalism in humanities literature post-LLM integration
Relatively little attention has been paid to what the prevalence of AI-generated manuscripts is likely to mean for the overall quality (epistemic and moral) of the humanities literature

Extracted from: pdfAgreement 69%

Explore related topics

Epistemic Risk Research Integrity Position Paper