AI evidence extraction

Promoting student engagement with GPTutor: An intelligent tutoring system powered by generative AI

Haoran Bai, Wing Cheung Lui, Paul Vinod Khiatani · International Journal of Educational Technology in Higher Education · 2025

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

5/10

Relevance

0/4

Quality (LMQS)

Evidence

Citations

1.73

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1186/s41239-025-00571-9

Methodology & findings

Study design

Explanatory mixed-method case study combining survey research (n=880 undergraduate students) with follow-up focus groups.

Sample

N = 880, 3 groups

Primary method

Survey research with quantitative analysis of relationships between feature usage and student engagement dimensions; focus group qualitative analysis; back-end trace data analysis from GenAI system. Specific statistical tests (e.g., correlation, regression) not detailed in abstract.

Main result

The study found that "engagement with the chatbot is significantly and positively associated with behavioral and emotional engagement, but not cognitive engagement. The exercise generator feature had no significant associations with any of the three dimensions of student engagement." Additionally, the focus group findings revealed that "GPTutor was used only when it was perceived as useful, and this perceived usefulness was shaped by the students' perception of the difficulty of the course and whether their support system could adequately address questions they may have."

Reports effect sizes.

Research paradigm

Mixed methods (pragmatist)

Author conclusions

The authors conclude that "leveraging survey data, interview data, and back-end trace data from GenAI, this research makes an original contribution to AI-supported effective learning environments and design strategies to optimize the educational experiences of higher education students." The findings suggest that perceived usefulness and course context shape how students engage with ITS powered by GenAI.

Risk of bias

Selection bias: Self-selection of students who chose to use GPTutor; Self-report bias: Reliance on self-reported student engagement measures in survey; Temporal confounding: Changing perceived usefulness as course progressed and examinations approached; Self-reported engagement measures (susceptible to social desirability bias); Single institution case study (limited generalizability); Subsample selection bias for focus groups (not specified how subsample was selected); Potential attrition bias in focus group recruitment; No randomization or control group comparison; Selection bias: Non-random sample of 880 students using GPTutor voluntarily; Self-report bias: Reliance on self-reported student engagement measures; Attrition: Subsample of survey participants used in focus groups (potential selection into focus groups); Social desirability bias: Focus group participants may provide biased responses; Contextual factors: Single institutional setting limits external validity

Open questions raised

Need for further research on design improvements for ITS powered by GenAI, particularly incorporating multimodal media capabilities such as video recordings of lectures
Need to understand differential effects of GenAI-ITS features on different dimensions of student engagement
Need to examine how course context and examination timing influence the effectiveness of GenAI-ITS features
Future research should examine multimodal capabilities for ITS (video lecture integration), explore mechanisms linking chatbot engagement to behavioral and emotional but not cognitive engagement, investigate contextual factors affecting perceived usefulness across different course types, and test design improvements identified by participants.
Need for improved multimodal capabilities in chatbot-based ITS (video recordings of lectures)
Understanding of how exercise generator features can be optimized for student engagement

Data: not_statedCode: not_statedExtracted from: pdfAgreement 59%

Explore related topics

Responsible AI Use Human-AI Collaboration Mixed Methods