AI evidence extraction

Artificial Intelligence in Literature Review Synthesis: A Step-by-Step Methodological Approach for Researchers and Academics

Matolwandile Mzuvukile Mtotywa, Jeri-Lee Mowers, Wavhudi Ndou, Thabang V. Q. Moleko, Matsobane Ledwaba · Informatics · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

10/10

Relevance

0/4

Quality (LMQS)

Evidence

Citations

33.79

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.3390/informatics13030043

Methodology & findings

Study design

Theoretical framework development grounded in sociotechnical systems theory.

Main result

The study found that "AI in research processes has the potential to improve decision-making, operational efficiency, and the overall quality of research output" and that the developed "structured step-by-step approach to integrate AI tools into the literature review synthesis process" embeds iterative validation loops with human oversight to address the "AI co-piloting-hallucination paradox," where "the value AI brings as a supportive partner (a co-pilot for thinking, research and decision-making) versus the risks posed by its technical limitations (hallucinations: when AI produces false, misleading or fabricated outputs)" must be actively managed through five sociotechnical practices: treating "AI tools as collaborators, not as authorities," combining "AI with human critical judgment," maintaining "documentation for transparency," "cross-validat[ing] AI outputs using verified sources," and "respect[ing] copyright and use [of] open-access filters."

Research paradigm

Interpretive/hermeneutic with pragmatist orientation toward sociotechnical systems

Author conclusions

The authors conclude: "our proposed model makes a unique contribution by being explicitly grounded in a sociotechnical systems lens. This approach moves beyond a simple workflow to foreground the critical, ongoing tension between AI's technical capabilities and its inherent limitations, such as hallucination. By embedding an iterative validation loop (Steps 3 and 4) as the core engine of the review process, our model proposes a practical methodology for researchers to actively manage this 'co-piloting-hallucination paradox,' ensuring that human critical judgment remains central to knowledge production." They further state: "The robustness of the proposed framework is derived from the systematic integration of the five sociotechnical practices with the five procedural steps. Rather than treating ethical boundaries or human judgment as isolated preparatory tasks, this model ensures they permeate every phase of the review-from the strategic mapping in Step 2 to the rigorous auditing in Step 4."

Risk of bias

Potential researcher bias in tool selection and evaluation; Algorithmic biases embedded in AI training data; Selection bias in which AI tools were included in the comparative matrix; Over-reliance on AI outputs without critical human judgment; Publication bias favoring documented and visible AI tools over emerging alternatives; Tool selection bias: differences in AI platform algorithms and training data may affect consistency; User bias: over-reliance on AI outputs may lead researchers to undervalue human critical judgment; Confirmation bias: researchers may develop biases when assuming AI outputs are inherently accurate; Algorithmic bias: AI systems constrained by training data with potential gaps in paywalled or proprietary sources; Human bias transfer: biases in decision-making, cultural diversity, and historical/societal perspectives transferred to sociotechnical system; No empirical validation: Use-case applications are conceptual and not empirically tested; Selection bias in tool inclusion: The 20 AI tools presented were purposively selected, not exhaustively inventoried; Lack of comparative evaluation: No controlled studies comparing the proposed approach against traditional manual methods; Potential confirmation bias: Framework developers may be biased toward demonstrating feasibility of their own model; Limited evidence base for claims: Recommendations are theory-grounded but lack quantitative empirical support

Limitations

The authors explicitly state: "The developed structured step-by-step approach is not without limitations
First, there is variability in AI tools
Differences in AI platform algorithms, training data, and functionality may affect the consistency and generalisability of the results
Second, academics and researchers may create an over-reliance on AI outputs
As such, scholars may risk undervaluing critical human judgment if AI-generated syntheses are accepted uncritically
Furthermore, ethical and disclosure challenges

Open questions raised

Lack of controlled empirical studies comparing AI-integrated 5-step approach against traditional manual methods
Limited research on AI-assisted bibliographic analysis and citation parsing
Underdeveloped work on AI-assisted theory-building using NLP and generative models
Insufficient exploration of personalized AI review agents tailored to individual scholars
Need for improved interoperability with major academic databases (Scopus, Web of Science)
Absence of standardized metadata formats and interoperable APIs for AI tools

Extracted from: pdfAgreement 75%

Explore related topics

Systematic Review Automation Literature Discovery Literature Review