AI evidence extraction

How to Use Generative Artificial Intelligence in the Research Process: A Modular Course Approach for Early Career Researchers

Heike da Silva Cardoso, Raphaela Stöckl, Martin Brehmer · Proceedings of the ... Annual Hawaii International Conference on System Sciences/Proceedings of the Annual Hawaii International Conference on System Sciences · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

0/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.24251/hicss.2026.008

Methodology & findings

Study design

Mixed-methods design combining: (1) expert interviews with two senior professors (structured interviews analyzed using deductive content analysis following Mayring 2022); (2) pre-course survey (N=16) and post-course survey (N=11) measuring learning outcomes aligned with ARCS model; (3) think-aloud feedback session in final module.

Primary method

Design Science Research Methodology (DSRM) following Peffers et al. (2007)

Main result

The course effectively increased participants' confidence and ability to apply GAI tools along the research process. Specifically, "Prior to the course, 31% (n=5 of N=16) of the participants reported rarely or never using such tools. After the course (N=11), frequency of often 54.5% (n=6) or always use 10.0% (n=1) of GAI-tools increased noticeably." Additionally, "81.8% (n=9) experienced improvements in conducting literature reviews, and 72.7% (n=8) in writing or revising research papers." The evaluation revealed that "every respondent indicated that the course either met (45.5%, n=5) or exceeded personal expectations (54.5%, n=6)."

Research paradigm

Design Science Research (DSR); mixed-methods evaluation combining expert qualitative interviews with participant surveys

Author conclusions

The authors conclude: "Following the DSRM (Peffers et al., 2007), we developed a novel course prototype which was assessed through expert interviews and participant feedback, with particular attention to its pedagogical structure, responsible GAI use, and potential impact on research practices." They further state that "The findings indicate that the course effectively increased participants' confidence and ability to apply GAI tools along the RP, including literature review, writing, and other core research activities." They also emphasize that "we present a novel and evaluated instantiation as well as actionable guidelines, contributing to DSR (Gregor & Hevner, 2013) and ECRs' education, allowing others to design similar courses, building on our insights."

Risk of bias

Small sample size (N=16 pre-course, N=11 post-course for ECRs; only 2 professors interviewed); Self-reported measures without objective validation; Potential selection bias: only ECRs willing to participate in intensive course; Attrition: 5 participants dropped from pre- to post-course survey; Context-specific nature limits generalizability; No control group for comparison; Interviewer bias in qualitative interviews despite dual coding; Small sample size (N=2 expert interviews, N=11 post-course survey); Self-reported measures susceptible to social desirability bias; Limited to German academic context; Selection bias: only two senior researchers participated due to time constraints; Attrition: post-course survey (N=11) lower than pre-course (N=16); Lack of control group for comparison; Very small sample size for expert interviews (N=2 professors, both German, full professors in IS/Computer Science); Small participant sample for surveys (N=16 pre-course, N=11 post-course) with potential attrition bias; Self-reported measures without objective outcome assessment; Selection bias: only two professors willing to participate due to substantial time commitment; Limited demographic data collection (anonymity constraint) prevents subgroup analysis; No standardized, validated assessment instrument used; Context-specific nature limits generalizability across disciplines

Limitations

The study shows several limitations: "the small sample size resulting from the target group (ECRs) as well as the context-specific nature of the course
Moreover, the evaluation relied primarily on qualitative feedback and self-reported measures." The authors also note that "given the small sample we did not collect demographic data to ensure anonymity." Additionally, "both professors acknowledged the cognitive load caused by the breadth of content" and "the limited timeframe made it difficult to engage deeply."

Open questions raised

Need for broader evaluation with larger sample sizes across diverse disciplines
Module-specific and behavior-focused evaluations needed
Limited research on GAI education specifically targeting ECRs for scholarly work (most literature focuses on higher education below PhD level)
Need to explore applicability across broader range of disciplines
Lack of standardized evaluation frameworks for GAI training courses
Insufficient research on responsible delegation of tasks to GAI

Extracted from: pdfAgreement 64%

Explore related topics

AI Literacy Training Doctoral Training Design Science