AI evidence extraction

AI chatbots in programming education: Students’ use in a scientific computing course and consequences for learning

S.E.A. Groothuijsen, Antoine van den Beemt, Joris C. Remmers, Ludo W. van Meeuwen · Computers and Education Artificial Intelligence · 2024

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

8/10

Relevance

1/4

Quality (LMQS)

Evidence

Citations

19.62

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1016/j.caeai.2024.100290

Methodology & findings

Study design

Mixed-methods case study consisting of 29 student questionnaires, a semi-structured group interview with three students, a semi-structured interview with the teacher, and analysis of 29 students' grades from a scientific computing course..

Sample

N = 29, 8 groups

Main result

The study found that "students used ChatGPT for error checking and debugging of code, increasing conceptual understanding, generating, and optimizing solution code, explaining code, and solving mathematical problems." However, "the teacher expressed concerns over declining code quality and student learning," and "both students and teacher perceived a negative influence from ChatGPT usage on pair programming, and consequently on student collaboration."

Reports effect sizes.

Research paradigm

Mixed methods (pragmatist)

Author conclusions

The authors conclude that "learning objectives should be formulated in more detail, to highlight essential programming skills, and be expanded to include the use of AI tools. Complex programming assignments remain appropriate in programming education, but pair programming as a didactic approach should be reconsidered in light of the growing use of AI Chatbots."

Risk of bias

Selection bias: Small sample size (n=29 students) from a single Master's program; Self-selection bias: Only 3 students participated in the group interview; Social desirability bias: Student questionnaire responses may be influenced by perceived teacher expectations; Temporal confounding: No baseline measurement before ChatGPT adoption; No control group: Unable to compare with students not using ChatGPT; Selection bias: Only 29 students surveyed from a single course; Self-report bias: Student questionnaires may not accurately reflect actual ChatGPT usage patterns; Small qualitative sample: Only 3 students in group interview and 1 teacher interviewed; Single institution: Case study at one university in the Netherlands may have limited generalizability; Temporal confounds: Cannot isolate ChatGPT impact from other course factors; Selection bias: Only students who voluntarily completed questionnaires were included; Small interview sample: Only 3 students participated in the group interview; Single institution bias: Study conducted at one university; Teacher perspective bias: Single teacher's viewpoint on code quality concerns; Self-report bias: Student questionnaires rely on self-reported usage patterns

Limitations

The study is limited in scope as "a case study of a single course at one institution," which may affect generalizability
The paper notes that the findings are based on a specific educational context and may not be directly transferable to other programming courses or institutions.

Open questions raised

The study identifies the need to adapt programming education in response to widespread ChatGPT use, particularly regarding reformulation of learning objectives, integration of AI tool use into curricula, and reconsideration of pair programming as a pedagogical strategy in the era of AI assistance.
The study identifies the need for: (1) more detailed learning objectives that explicitly address AI tool usage, (2) reconsideration of pair programming pedagogies in the context of AI chatbot availability, (3) strategies to maintain code quality and student learning outcomes despite widespread ChatGPT adoption, and (4) adaptation of programming education practices in higher engineering education.
The paper identifies the need for further research on how to adapt programming education in higher engineering education in response to student use of AI chatbots. It suggests that curriculum redesign is needed to explicitly address AI tool use in learning objectives.

Data: not_statedCode: not_statedExtracted from: pdfAgreement 67%

Explore related topics

AI-Assisted Discovery Human-AI Collaboration Mixed Methods