AI evidence extraction

KNIMEZoBot: Enhancing Literature Review with Zotero and KNIME OpenAI Integration using Retrieval-Augmented Generation

Suad Alshammari, Lama Basalelah, Walaa Abu Rukbah, Ali Alsuhibani, Dayanjan S. Wijesinghe · arXiv (Cornell University) · 2023

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

1/4

Quality (LMQS)

Evidence

Citations

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.48550/arxiv.2311.04310

Methodology & findings

Study design

Design science / artifact development.

Primary method

Design science / tool development. The artifact integrates three existing platforms through KNIME's visual workflow interface.

Main result

KNIMEZoBot successfully integrates Zotero, OpenAI, and KNIME to automate literature review tasks. The system "uses a Retrieval-Augmented Generation (RAG) architecture, first conducting a semantic search to identify relevant passages from retrieved PDFs" and "enables KNIMEZoBot to provide informative responses to questions by efficiently searching academic papers and distilling salient facts and main points." The artifact demonstrates that "researchers can save significant time while benefiting from state-of-the-art AI techniques for synthesizing knowledge in a low code manner."

Research paradigm

design science / pragmatist

Author conclusions

"KNIMEZoBot marks an important step toward streamlining access to critical information in existing literature by domain experts who are not coders by training. By facilitating more rapid and comprehensive understanding of prior work, this system could substantially benefit the research community and knowledge-building process."

Risk of bias

No user study conducted - lacks empirical validation; No comparison with alternative approaches or baseline systems; No assessment of hallucination or error rates in generated responses; Potential API dependency bias (reliance on OpenAI's proprietary models); No discussion of potential biases in retrieved documents or generated summaries

Open questions raised

The authors identify the need for automated strategies to query curated literature libraries and routinely refresh domain knowledge, and note that while RAG-based systems are expedient, they "hitherto necessitated substantial coding knowledge," which the authors address through their code-free integration.
Authors identify the need for improvements in accuracy and sophistication of automated analysis. They note the broader challenge of information overload in academia and the gap between non-coding researchers' needs and availability of accessible AI tools for literature review.
The authors identify the challenge of information overload in academia and the limitations of existing large language models (context window constraints, hallucination). They propose Retrieval-Augmented Generation (RAG) as a solution but note that "the RAG-based system, although expedient in summarizing information, hitherto necessitated substantial coding knowledge," which motivated their creation of a code-free tool.

Code: https://github.com/dayanjan-lab/KNIMEZoBotExtracted from: pdfAgreement 77%

Explore related topics

Systematic Review Automation Literature Discovery Design Science