AI evidence extraction

The Role of Generative Artificial Intelligence in the Analysis of Qualitative Data Compared With Human-Led Analysis

Jasmin Dhanoa, Mark Lee, Sonaina Chopra, Quang Ngo, Elif Bilgiç · Cureus · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

2/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.7759/cureus.109620

Methodology & findings

Study design

Comparative case study using secondary qualitative data analysis.

Main result

The study found that "GenAI mainly focused on analyzing the sources of emotions and identifying the specific emotions," and that "the Copilot output was not as detailed and nuanced as the human-generated table, as even with continuous prompting, Copilot did not consider positionality and reflexivity in the outputs." Additionally, "Copilot was ineffective at utilizing the entirety of follow-up prompts based on the output it had already generated," and "Copilot failed to incorporate past inputs and outputs, making large-scale data analyses challenging."

Research paradigm

Constructivist/Interpretivist with qualitative descriptive approach

Author conclusions

"While arguments can be made that the latest versions of other GenAI tools, such as ChatGPT, have more sophisticated and advanced functions, it is important to consider that variability in output interpretation and depth of understanding of the findings is crucial in qualitative research and can depend on the capabilities and limitations of the GenAI platform." The authors emphasize that "human judgment is crucial in all areas of the qualitative research process, especially in data analysis, where researchers derive meaning from the data and can theoretically construct meaning in an infinite number of ways. However, GenAI limits interpretations by generating responses through pattern recognition in language modeling rather than through reflexivity and contextual analysis."

Risk of bias

Researcher bias: Same researchers conducted both human-led and GenAI-led analyses; Platform-specific limitations: Results specific to Microsoft Copilot may not generalize; Temporal bias: GenAI platforms continue to evolve, affecting transferability; Training data representation bias: GenAI outputs depend on training data recency and representativeness; Same researchers conducted both human-led and GenAI-led analyses (introduces comparison bias); Use of single GenAI platform (Microsoft Copilot) limits generalizability; Researcher familiarity with data from prior analysis may influence interpretation; No independent verification of analyses; Same researchers conducted both human and GenAI analyses, creating potential interpretation bias; Researchers familiar with original human analysis may have influenced GenAI prompt design; Secondary data analysis - data collected for different research question

Limitations

"A limitation of this study is the transferability of the results to other GenAI platforms or across time, as GenAI continues to learn and evolve." Additionally, "since the same researchers conducted both the human-led analysis and the GenAI-led analysis, there is potential for bias in the comparison of the findings." The authors also note that "each qualitative research design and methodology has its own steps and approaches, including the role of the research team, reflexivity, and analytic approach."

Open questions raised

Transferability to other GenAI platforms
Systematic exploration of multiple GenAI platforms to understand variability across platforms
Exploration of GenAI's role in supporting analysis across multiple qualitative research designs and methodologies
Need for standardized methods to evaluate GenAI quality performance in qualitative data analysis
Understanding how different GenAI platforms produce different analyses and capabilities
Transferability of findings across GenAI platforms and over time

Data: Secondary dataset comprising 23 interview transcripts from a previous medical education research project. Dataset not explicitly stated as publicly available.Extracted from: pdfAgreement 59%

Explore related topics

Qualitative Coding with AI Data Analysis Case Study