AI evidence extraction

TOOLS TO SUPPORT PROGRAMMING EDUCATION WITH THE HELP OF MULTI-ROLE AI AGENTS

MYKOLA LEHKYI, Hanna Shevchuk · Herald of Khmelnytskyi National University Technical sciences · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

7/10

Relevance

0/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.31891/2307-5732-2026-361-30

Methodology & findings

Study design

Pedagogical experiment with mixed methods data collection including telemetry data, rubric-based code assessment, and survey responses

Sample

< 30

Primary method

Telemetry data analysis, rubric-based assessment (Correctness, C# idioms, Clarity, Robustness, Efficiency), survey analysis; specific statistical tests not detailed in abstract

Main result

The findings demonstrate high acceptability of the system: "most students evaluated interactions with the agents as useful, the Tutor's explanations as clear, the Evaluator's grading as fair, and the Coach's recommendations as helpful for planning their learning." Additionally, "an increase in the proportion of successful solutions, a reduction in task completion time, and a decrease in repeated errors were recorded."

Reports effect sizes.

Research paradigm

Mixed methods (quantitative and qualitative)

Author conclusions

"The scientific novelty of the work lies in combining a multi-role AI architecture with rubric-based formative assessment and coaching support integrated into a real programming course." The authors conclude that the system design "ensures the individualization of learning while preserving the leading role of the instructor."

Risk of bias

Single institution study (Lviv Polytechnic National University); Self-reported survey responses subject to social desirability bias; No control group mentioned for comparison; Selection bias from voluntary participation; Self-selection bias: Students may have volunteered for the study; Response bias: Survey responses on agent usefulness may reflect social desirability bias; Single institution: Study conducted only at Lviv Polytechnic National University, limiting generalizability; No control group explicitly mentioned in abstract, limiting causal inference; Self-selection bias: student perception data based on voluntary survey responses; Lack of control group mentioned in abstract

Data: not_statedCode: not_statedExtracted from: pdfAgreement 65%

Explore related topics

AI Literacy Training Doctoral Training Experimental