AI evidence extraction

How Does LLM-powered Coding Assistance Shape Incidental Learning? Exploring Cognitive Forcing Strategies in Programming Education

Ba-Thinh Tran-Le, Patrick Thomas, Nicholas M. Stiffler, Thanh-Danh Nguyen · Proceedings of the AAAI Conference on Artificial Intelligence · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

5/10

Relevance

0/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1609/aaai.v40i48.42121

Methodology & findings

Study design

Pilot study with within-subjects design (assisted and unassisted conditions), involving novice and advanced college programmers completing LeetCode-style coding tasks with pre-test and post-test measurements.

Sample

2 groups

Main result

The study found that "Novices showed substantial post-test gains despite receiving AI guidance only during the intervention, suggesting that incidental exposure improved later performance." Additionally, "Across both groups, participants required fewer debugging attempts in the post-test compared to earlier stages, indicating improved debugging efficiency and algorithmic understanding."

Reports effect sizes.

Research paradigm

positivist/empiricist

Author conclusions

The authors conclude that "These findings provide early evidence that LLMs can be designed to promote indirect learning while shaping problem-solving strategies" and that "This work offers a proof of concept for cognitively informed tutoring systems in computer science education and discusses implications for integrating LLMs to enhance both immediate outcomes and lasting skill development."

Risk of bias

Selection bias: Pilot study with self-selected college programmers; Small sample size typical of pilot studies; Potential order effects from within-subjects design (assisted vs. unassisted conditions); Novice vs. advanced learner stratification may introduce confounding if groups differ on unmeasured characteristics; Selection bias: Self-selected college programmers (volunteers for pilot study); Lack of explicit blinding mentioned; Potential Hawthorne effect from aware participation in AI-assisted intervention; Selection bias: self-selected college programmers (volunteers); Lack of randomization mentioned in abstract; Potential confounding from individual differences in prior programming experience

Open questions raised

The authors identify the need for further research on integrating LLMs into computer science education systems and understanding how cognitive forcing strategies can enhance both immediate learning outcomes and lasting skill development in programming education.
The authors identify the need for further investigation into how LLM-assisted coding platforms can support incidental learning at scale, the generalizability of findings beyond novice and advanced programmer groups, and the long-term retention effects of cognitive forcing strategies in programming education.
The paper identifies the need for further research on how AI-based code assistants can support incidental learning through guided interaction rather than providing complete solutions, and the integration of cognitively informed design principles in LLM-powered tutoring systems.

Data: not_statedCode: not_statedExtracted from: pdfAgreement 65%

Explore related topics

AI-Assisted Discovery Doctoral Training Experimental