AI evidence extraction

Next-generation IS research methods – towards a better understanding of complex and dynamic phenomena … and generative AI as the elephant in the room

Ivo Blohm, Shaila Miranda, Shuk Ying Ho, Jan Marco Leimeister · Journal of Information Technology · 2025

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

3/4

Quality (LMQS)

Evidence

Citations

7.07

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1177/02683962251340699

Methodology & findings

Study design

Editorial and narrative synthesis of five accepted special issue articles on next-generation IS research methods.

Main result

The paper synthesizes next-generation IS research methods across three paradigms. Key findings indicate that "the papers accepted for this special issue are representative of all submissions that we received" and emphasize "a trend towards addressing the challenges of temporal dynamics, complexity, and ethical considerations more effectively in future research methods." The analysis proposes that "Generative AI might shorten the time-to-publication by making researchers more productive and could help democratise science to make it more equitable and approachable" (Stokel-Walker and Van Noorden, 2023; Susarla et al., 2023), though researchers also "outline substantial challenges including the truthfulness of obtained outputs and fabricated answers, ethical questions concerning the transparency and integrity of the research process, or plagiarism and copyright infringements."

Reports effect sizes.

Research paradigm

Mixed (positivist, interpretivist, critical realist)

Author conclusions

The authors conclude that "the editors of this special issue believe that these challenges and ethical considerations will continue to be controversial topics. However, it is crucial to balance the promise of GAI with an acute awareness of the associated risks and maintaining the integrity and authenticity of the research process." They further emphasize that next-generation IS research must address "temporal detail for improved understanding of dynamic phenomena" and that researchers "must be mindful of the nature of their engagement with GAI—that is, the nature of their tasks and manner in which they are delegating the tasks to the GAI—and implement controls accordingly."

Risk of bias

Selection bias in special issue submissions (not all methodological innovations captured); Temporal limitation: GAI developments occurred after special issue call deadline; Narrative synthesis vulnerability to author interpretation bias; Limited coverage of emerging methodologies due to editorial scope constraints; Selection bias in special issue articles - only accepted papers reviewed, not all submissions; Publication bias toward methodological innovation; Author perspective bias - editorial team curated and interpreted the narrative; Potential over-representation of certain paradigms depending on submission quality; Discussion of GAI perpetuating STEM bias and excluding social sciences perspectives; Selection bias in article acceptance for special issue; Confirmation bias in interpreting GAI capabilities and limitations; Publication bias toward positive framing of novel methodologies; Limited representation of non-STEM social science perspectives in GAI training data

Limitations

The authors acknowledge: "Our special issue call for papers preceded the explosion of GAI and was therefore too early to capture contributions exploring GAI support for conducting research." Additionally, they note that "these applications might not be exhaustive and that certain GAI potentials that we describe in one of the three paradigms might also be applied in the other ones." The paper also highlights that "recent research indicates that clustering and visualisation based on GAI lack theory-guiding principles, and researcher intervention and improvement are deemed necessary (Haque, 2024)."

Open questions raised

Need for research methods capturing dynamic and temporal phenomena in IS
Integration of ethical considerations into design science research methodologies
Understanding of how to effectively combine computational techniques with qualitative theory construction
Ontological stratification in agent-based modeling
Standardized metrics for validating GAI-generated research outputs
Understanding implications of GAI-enabled human-AI collaboration in knowledge creation

Extracted from: pdfAgreement 75%

Explore related topics

Responsible AI Use Human-AI Collaboration Conceptual