AI-Assisted Systematic Literature Review of the Economic Burden of Pneumococcal Disease: Development and Validation Study
Dong Wang, Surabhi Datta, Julie Glasgow, Kyeryoung Lee, Hunki Paek, Jun Zhang et al. · JMIR AI · 2026
AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.
This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.2196/81049
Methodology & findings
Study design
Artifact validation study using gold standard comparison.
Primary method
design science research with iterative development and validation against gold standard
Main result
ISLaR 2.0 demonstrated strong performance in automated systematic literature review tasks. The platform achieved "F1-scores of 86.27 for abstract screening and 87.18 for full-text screening" and "F1-scores of 92.83 for study details and 79.76 for economic burden outcomes" in data extraction from text, with particularly high performance in tabular data extraction at "F1-score for data extraction of tabular economic burden outcome data was 94.83." The qualitative analysis identified that "2 main challenges in extracting economic burden details: misclassification of cost categories and failure to extract relevant information."
Research paradigm
positivist/empiricist
Author conclusions
The authors concluded that "ISLaR 2.0 enabled efficient execution of an SLR regarding the economic burden of PD. The platform allowed users to flexibly define and modify criteria and data elements, supporting its use across a broad range of health research topics."
Risk of bias
Potential selection bias in gold standard creation (50 expert-curated articles may not represent the full distribution of pneumococcal disease literature). No mention of inter-rater reliability for expert curation or potential funding bias.; Gold standard based on only 50 expert-curated articles may not represent broader SLR populations; Single use case (pneumococcal disease economics) may limit generalizability; LLM performance may vary with different domains, languages, or article types
Explore related topics
Related papers
- PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and ExplanationAndrea C. Tricco · 2018 · 40,391 citations
- Rayyan—a web and mobile app for systematic reviewsMourad Ouzzani · 2016 · 24,664 citations
- Cochrane Handbook for Systematic Reviews of Interventions2019 · 14,420 citations
- Guidance for conducting systematic scoping reviewsMicah D.J. Peters · 2015 · 7,472 citations
- Updated methodological guidance for the conduct of scoping reviewsMicah D.J. Peters · 2020 · 6,688 citations
- Which academic search systems are suitable for systematic reviews or meta‐analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resourcesMichael Gusenbauer · 2019 · 2,116 citations