AI evidence extraction

Impact of AI assistance on student agency

Ali Darvishi, Hassan Khosravi, Shazia Sadiq, Dragan Gašević, George Siemens · Computers & Education · 2023

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

6/10

Relevance

1/4

Quality (LMQS)

Evidence

384

Citations

98.25

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1016/j.compedu.2023.104967

Methodology & findings

Study design

Randomised controlled experiment (RCT) with four treatment groups across a four-week intervention period followed by a four-week post-intervention period

Sample

N = 1625, 8 groups

Main result

The study found that "students tended to rely on rather than learn from AI assistance." Furthermore, "If AI assistance was removed, self-regulated strategies could help in filling in the gap but were not as effective as AI assistance." The results also demonstrated that "hybrid human-AI approaches that complement AI assistance with self-regulated strategies (SAI) were not more effective than AI assistance on its own."

Reports effect sizes.

Research paradigm

Empirical positivism / Quantitative experimental design

Author conclusions

The authors conclude that "students tended to rely on rather than learn from AI assistance" and that while "self-regulated strategies could help in filling in the gap," they are "not as effective as AI assistance." They note that "hybrid human-AI approaches that complement AI assistance with self-regulated strategies (SAI) were not more effective than AI assistance on its own," and discuss "broader benefits, challenges and implications of relying on AI assistance in relation to student agency."

Risk of bias

Potential selection bias if student enrollment in courses was not randomized; Attrition risk across the two 4-week periods not mentioned; No mention of blinding procedures (instructor or student blinding); Potential confounding variables related to course differences across 10 courses not controlled; Hawthorne effect possible given students aware of participation in experiment; Potential selection bias in course selection; Attrition across the eight-week study period not reported; Possible confounding by course type or instructor variation; Selection bias: Student self-selection into courses; Attrition: Potential dropout across the eight-week study period; Confounders: Differences in student prior knowledge, motivation, and learning strategies across groups; Hawthorne effect: Students may have altered behavior due to being observed

Limitations

The limitations section is not explicitly provided in the abstract
The paper states it discusses "broader benefits, challenges and implications" but specific limitations are not detailed in the provided abstract text.

Open questions raised

The paper identifies that "their impact on students' agency and ability to self-regulate their learning is under-explored" and questions whether "students learn from the regular, detailed and personalised feedback provided by AI systems, and will they continue to exhibit similar behaviour in the absence of assistance?"
The paper addresses the gap that "the impact on students' agency and ability to self-regulate their learning is under-explored" despite the increasing use of AI-powered learning technologies. The authors pose research questions about whether students learn from AI feedback or instead continue to rely on AI assistance without learning from it.
The paper addresses the under-explored impact of AI-powered learning technologies on students' agency and ability to self-regulate their learning, asking whether students learn from AI feedback or become dependent on it.

Data: not_statedCode: not_statedExtracted from: pdfAgreement 57%

Explore related topics

Productivity Effects Human-AI Collaboration Experimental