AI evidence extraction

Generative artificial intelligence tools

Yiran Xu, Charlene Polio · Research methods in applied linguistics · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

0/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1075/rmal.15.17xu

Methodology & findings

Study design

Narrative literature review discussing applications and implications of generative AI tools in applied linguistics research

Main result

The chapter demonstrates that "GenAI can be leveraged to support various stages of the research processes in empirical studies, such as instrument design, automated coding, text annotation, and qualitative data analysis." The review emphasizes that while GenAI tools like ChatGPT show promise for research applications in applied linguistics, concerns around validity, reliability, and ethical considerations remain significant.

Reports effect sizes.

Research paradigm

Mixed (descriptive and critical analysis)

Author conclusions

The authors conclude that while GenAI tools like ChatGPT "are emerging as powerful web-based tools for research purposes," there is a need to carefully consider validity, reliability, and ethical implications. They propose that "promising directions for future studies" should address the current "capacities and limitations based on emerging empirical research."

Risk of bias

Early-stage technology with limited peer-reviewed empirical validation; Publication bias toward positive GenAI applications; Rapid technological evolution may render findings obsolete quickly; Limited discussion of negative outcomes or failed applications

Limitations

The authors note that "Given that GenAI is in the early stage of research application, we describe its current capacities and limitations based on emerging empirical research," indicating that the field lacks mature evidence bases and comprehensive empirical validation of GenAI research applications.

Open questions raised

The authors identify that given GenAI's early stage, they "propose promising directions for future studies" to better understand its applications in empirical research, particularly regarding validity, reliability, and ethical safeguards.
The chapter identifies the need for future studies to establish empirical evidence validating GenAI applications across research stages, develop standardized protocols for assessing reliability and validity of AI-generated research outputs, and address ethical considerations including transparency, data privacy protections, and bias detection mechanisms.
The authors identify the need for empirical research establishing best practices for using GenAI in various research stages, particularly regarding validity and reliability of AI-generated outputs. They call for future studies to address ethical concerns around transparency, data privacy, and bias in AI-generated content, and to develop guidelines for appropriate application in applied linguistics research.

Data: not_statedCode: not_statedExtracted from: pdfAgreement 78%

Explore related topics

Qualitative Coding with AI Data Analysis Literature Review