AI evidence extraction

Usability and Preferences for a Personalized Adaptive Learning System for AI Upskilling

Mark G. Core, Benjamin D. Nye, Kayla Carr, Shirley Xin Li, Aaron Shiel, Daniel Auerbach et al. · Proceedings of the ... International Florida Artificial Intelligence Research Society Conference · 2025

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

6/10

Relevance

0/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.32473/flairs.38.1.138996

Methodology & findings

Study design

Design-based research approach with iterative testing and refinement across four semesters, combined with survey data collection and engagement analysis

Primary method

Descriptive statistics (percentages for usability ratings); engagement metrics based on completion rates; survey data analysis. Specific software packages not mentioned.

Main result

The study found that "Students rated the adaptive system positively overall (93% rated as a "good idea"), but more complex learning activities (tutoring dialogs, programming) were rated lower than traditional ones (e.g., multiple choice, reading)." Additionally, "Students were most likely to master topics highly aligned to the course materials, as well as self-directed learning toward easier high-interest topics (e.g., LLM Prompting)."

Reports effect sizes.

Research paradigm

Pragmatist/Design-based research

Author conclusions

The research demonstrates that while students perceive an adaptive AI learning system as "a good idea" overall, engagement and usability vary significantly by activity type, with "Students were most likely to master topics highly aligned to the course materials, as well as self-directed learning toward easier high-interest topics (e.g., LLM Prompting)", suggesting that personalized adaptive systems should balance alignment with course content with student interest-driven learning.

Risk of bias

Selection bias: optional use in course may attract self-motivated learners; Attrition: completion rates vary across topics; Lack of control group: no comparison to non-adaptive learning approaches; Selection bias: Optional participation may result in self-selected sample of more motivated or interested students; Volunteer bias: Students choosing to use an optional system may differ systematically from non-participants; Temporal confounding: Iterative changes across four semesters make it difficult to isolate effects of specific system features; Selection bias: optional use may result in self-selection of motivated students; Attrition: variable engagement across semesters not quantified; Self-report bias: usability ratings based on survey responses; Single institution: limited generalizability

Open questions raised

The authors identify a need to expand AI education beyond computer scientists and integrate it into technician-level training; the iterative refinement approach suggests ongoing investigation into optimal design of tutoring dialogs, programming activities, and topic selection for adaptive learning systems.
Future research directions implied but not explicitly stated: the need to understand factors driving lower engagement with complex learning activities (tutoring dialogs, programming) compared to traditional formats, and strategies to improve alignment of system activities with student self-directed learning preferences
The paper identifies the need to better understand how to design complex learning activities (tutoring dialogs, programming) to improve engagement and completion rates, as well as the need to balance curriculum alignment with student self-directed learning preferences.

Data: not_statedCode: not_statedExtracted from: pdfAgreement 65%

Explore related topics

AI Literacy Training Design Science