Trustworthy AI to conduct literature reviews
Isabelle Walsh · Journal of the Association for Information Systems · 2025
AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.
Methodology & findings
Study design
Descriptive/demonstration study with comparative benchmarking.
Main result
The paper presents that "Hybrid neuro-symbolic systems, which combine transparently logical reasoning for clustering literature with pattern recognition for synthesizing it" can address limitations in current AI approaches for literature reviews. The focus is on ARTIREV, described as "a hybrid AI tool integrating a bibliometric expert system coupled with fine-tuned generative AI (SOCRATES) offering improved transparency, exhaustivity, and reliability."
Research paradigm
pragmatist
Author conclusions
The authors conclude that hybrid neuro-symbolic systems represent an improved approach to AI-supported literature reviews. They propose that "Hybrid neuro-symbolic systems, which combine transparently logical reasoning for clustering literature with pattern recognition for synthesizing it" offer solutions to current limitations in AI-supported academic literature review workflows.
Risk of bias
Lack of systematic comparison protocol - attendees choose comparison baselines; Potential selection bias in choosing 'any other AI solution of their choice'; No mention of blinding or objective evaluation criteria; Single tool focus (ARTIREV) may introduce promotional bias
Limitations
- The abstract highlights "strengths and limitations of current AI approaches" but does not explicitly state specific limitations of the proposed ARTIREV tool itself
- The paper notes it will "critically assess the capabilities of the proposed tool," suggesting limitations exist but are not detailed in the abstract.
Open questions raised
- The abstract identifies gaps in current AI approaches for literature reviews, noting limitations in system transparency, input data handling, user control, and output presentation. The need for hybrid neuro-symbolic systems that combine logical reasoning with pattern recognition is highlighted as an unmet need.
Explore related topics
Related papers
- PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and ExplanationAndrea C. Tricco · 2018 · 40,391 citations
- Rayyan—a web and mobile app for systematic reviewsMourad Ouzzani · 2016 · 24,664 citations
- Cochrane Handbook for Systematic Reviews of Interventions2019 · 14,420 citations
- Guidance for conducting systematic scoping reviewsMicah D.J. Peters · 2015 · 7,472 citations
- Updated methodological guidance for the conduct of scoping reviewsMicah D.J. Peters · 2020 · 6,688 citations
- Which academic search systems are suitable for systematic reviews or meta‐analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resourcesMichael Gusenbauer · 2019 · 2,116 citations