AI evidence extraction

Beyond Augmentation: Epistemic Complementarity in LLM-Assisted Qualitative Analysis

Alina I. Palimaru, Ryan Brown, Daniel Siconolfi, Michael S. Dunbar, Nipher Malika, Prateek Puri et al. · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

1/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.31235/osf.io/n5e2d_v1

Methodology & findings

Study design

Scoping review of 51 published studies on LLM-assisted qualitative analysis (2024-2026), combined with a worked example from the CHAT project using pragmatic narrative inquiry combining Framework Analysis with Codebook Thematic Analysis applied to 86 semi-structured interviews and 5,466 coded excerpts analyzed through Muse QDA..

Sample

N = 86, 3 groups

Primary method

Framework Analysis combined with Codebook Thematic Analysis; analysis conducted through Muse QDA (purpose-built LLM-assisted qualitative analysis platform); specific statistical tests not detailed in abstract

Main result

The study found that "approximately 80% do not articulate an epistemological stance" among the fifty-one studies reviewed on LLM-assisted qualitative analysis, and "those that do rarely work within their stated paradigm as a constraint on methodological decisions." The authors propose that "human and LLM epistemic capacities are non-substitutable within this workflow, such that neither agent can do the other's work without epistemic loss."

Reports effect sizes.

Research paradigm

Mixed epistemological paradigms (interpretivist, pragmatic, and others); the paper critiques lack of explicit epistemological positioning in the reviewed literature

Author conclusions

The authors conclude that "The integration question is not whether LLMs can do qualitative research, but for which tradition, under which paradigmatic commitments, do which kinds of integration satisfy that tradition's own standards." They further argue that "the integration question is methodology-specific and must be answered on the terms each tradition sets for itself."

Risk of bias

Selection bias in scoping review (only studies published 2024-2026 included); Potential researcher bias in framework development (authors developed both the theoretical framework and the case study application); Tool dependence on Muse QDA platform; Limited generalizability beyond the specific narrative inquiry tradition examined; Publication bias in scoping review (only published studies captured); Potential selection bias in choice of 51 studies reviewed; Single case study may not generalize across methodologies; Tool dependence on specific QDA platform (Muse QDA); Selection bias in literature review (studies published 2024-2026 only; language/publication bias not addressed); limited to English-language publications likely; single case example may not represent diversity of qualitative approaches; funding source influence on tool selection (Muse QDA is described as purpose-built, potential vendor bias).

Limitations

The authors acknowledge several limitations: "resource demands, tool dependence, and the question of who gets to do LLM-assisted qualitative research," and note that "QDA platform design in the LLM era is itself a methodological matter." The framework is demonstrated through a single project pathway, potentially limiting generalizability across diverse qualitative traditions.

Open questions raised

The paper identifies that current debate conflates important issues and that "the field's discussion of their integration has unfolded as though 'qualitative research' were a single tradition with a single epistemological stake." The authors call for tradition-specific approaches to LLM integration and highlight that QDA platform design in the LLM era requires methodological attention.
Lack of explicit epistemological stance in LLM-assisted qualitative analysis literature
Insufficient operationalization of tradition-specific standards for analytic adequacy
Need for clearer methodology-specific frameworks for LLM integration
Questions about equitable access to LLM-assisted qualitative research tools
The paper identifies gaps in: (1) explicit articulation of epistemological stances in LLM-assisted qualitative research; (2) operationalization of tradition-specific standards for analytic adequacy; (3) understanding of complementarity between human and LLM epistemic capacities; (4) equitable access to LLM-assisted qualitative research tools and training.

Data: not_statedCode: not_statedExtracted from: pdfAgreement 58%

Explore related topics

Qualitative Coding with AI Data Analysis Case Study