AI evidence extraction

Higher Education Faculty Members’ Perceptions of an AI-driven Qualitative Data Analysis Tool for Their Research: An Exploratory Study

Abigail M. Nubla-Kung, Pressley R. Rankin, Mary Dereshiwsky · Journal of Online Graduate Education · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

2/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.65201/001c.158996

Methodology & findings

Study design

Basic qualitative research using exploratory study design.

Sample

N = 23, 5 groups

Primary method

Descriptive analysis of questionnaire data: means and percentages were calculated for questionnaire items exported to Excel from SurveyMonkey. For interview data: thematic analysis including line-by-line coding, focused codes, categories, and themes. Investigator triangulation employed with two researchers (Dr. Nubla-Kung and Dr. Rankin) independently analyzing transcripts to increase validity. Interview transcripts were also uploaded to Intellectus 2.0 for comparison of AI-generated categories and themes with manual analysis.

Main result

The study found that faculty members perceived AI qualitative analysis tools as valuable but limited assistants rather than replacements for human analysis. Specifically, "the faculty still believe AI software could enhance research efficiency for routine tasks while requiring significant human oversight and expertise." Additionally, "participants viewed the AI program as helpful as a limited assistant rather than as a replacement for human analysis," with one participant noting that "it's not like it could replace a human." The research revealed that faculty recognized potential efficiency gains but maintained healthy skepticism about AI outputs, with all interviewed participants agreeing that "researchers must have a fundamental understanding of research processes to analyze the accuracy and quality of what AI tools generate."

Reports effect sizes.

Research paradigm

Qualitative; interpretivism; exploratory pragmatism

Author conclusions

The authors conclude that "While faculty members appreciate the efficiency gains AI tools can provide to their research workflows, they also harbor healthy skepticism about the outputs of these tools." They emphasize that their study "contributes to the emerging yet still limited literature on HEI faculty pressures and their perceptions and use of AI-assisted qualitative tools." Most importantly, they stress that "All of those we interviewed said that researchers must have a fundamental understanding of research processes to analyze the accuracy and quality of what AI tools generate." The authors recommend that "higher education institutions create policies and expectations that faculty must disclose which AI tool they used, how the tool was used, and how the output was verified by the researcher" to ensure "transparency and accountability."

Risk of bias

Selection bias: Purposive sampling of faculty with existing qualitative research experience limits generalizability to all faculty types; Attrition bias: 23 questionnaire respondents versus only 5 interview participants (78% dropout rate for phase 2); Self-selection bias: Interview participants self-selected based on willingness to complete training, potentially excluding those more skeptical or time-constrained; Funding/incentive bias: Interview participants were offered 6 months of free tool access, potentially biasing responses toward positive views; Small sample size for interview phase (n=5) limits transferability of qualitative findings; Training variability: No standardized training protocol; participants used different training resources (videos, monthly sessions, weekly overview sessions, FAQ, resource library) at their own pace; Selection bias: Purposive sampling may have recruited faculty with greater interest in AI tools; Attrition bias: Large drop-off from Phase 1 (n=23) to Phase 2 (n=5) interviews; Self-selection bias: Only 5 of 8 who accepted interview invitation actually participated due to scheduling/training barriers; Small sample size in Phase 2 limiting generalizability; Potential volunteer bias from recruitment through professional networks; Selection bias: Purposive sampling and self-selection into second phase may not be representative; Attrition bias: Significant dropout from questionnaire phase (23) to interview phase (5); Small sample size: 5 interview participants limits generalizability; Volunteer bias: Incentive of free tool access may have influenced participation; Investigator bias: Principal investigator conducted all interviews

Limitations

The authors state: "We were limited in the number of participants we could recruit for the study because of the number of individual subscriptions (25) we had for Intellectus 2.0." They further note: "We expected all those who completed the initial phase of data collection to train on the AI tool and be interviewed
However, only a fraction of questionnaire participants opted for the second phase
The small number of participants for the second phase of data collection was a surprise especially with the incentive of gaining free access to an AI-driven research tool for six months." The authors conclude: "while generalizability was not the purpose in this exploratory study, had we known the disparity in the number of participants between the first and second phases of data collection, we would have recruited a much larger number of participants for the questionnaire, thereby strengthening the trustworthiness of the data."

Open questions raised

The authors identify the following research gaps: (1) Limited empirical literature on the intersection of publication pressure and AI adoption in higher education; (2) Lack of studies examining how training in AI-assisted qualitative analysis tools affects faculty research perceptions; (3) Insufficient understanding of faculty knowledge regarding AI qualitative analytic tools; (4) Need for future research on reasons why faculty opt out of AI tool training despite time-saving potential; (5) Limited guidance on appropriate policies for AI use in qualitative research in higher education institutions. The authors suggest future studies should investigate the experiences of those who declined to participate in the training phase.
Limited literature on the intersection of publication expectations and emerging AI technologies in higher education
Need to understand reasons why faculty opt out of AI training despite time constraints as a reported barrier
Further research needed on how different disciplines and career stages adopt AI tools
Need for more comprehensive empirical studies directly examining the relationship between publication pressure and AI adoption
Investigation of effective training approaches (formal vs. self-paced) for AI tool adoption among faculty

Data: Not explicitly stated as availableCode: Not mentionedExtracted from: pdfAgreement 53%

Explore related topics

Qualitative Coding with AI Human-AI Collaboration Qualitative