AI evidence extraction

Onco-Shikshak: An AI-Native Adaptive Learning Ecosystem for Medical Oncology Education

ASHISH MAKANI · medRxiv · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

5/10

Relevance

1/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.64898/2026.02.23.26346944

Methodology & findings

Study design

System design and technical validation.

Primary method

No statistical inference performed. Technical validation involved algorithm verification (correctness checks) against reference specifications and simulated response sequences. Methods include: (1) Citation extraction pipeline verification against source documents; (2) FSRS scheduling verification against reference specification with decay curve comparison; (3) IRT convergence testing with ability estimate stability after 20 interactions; (4) ACT-R activation formula verification; (5) Clinical reasoning phase ordering enforcement; (6) Multi-agent RAG routing isolation testing; (7) Brier score computation verification; (8) ACGME milestone mapping coverage testing.

Main result

The system integrates "ACT-R activation dynamics (illness scripts), Item Response Theory (adaptive difficulty), the Free Spaced Repetition Scheduler (FSRS v4), Zone of Proximal Development (scaffolding), and metacognitive calibration training (Brier score)" into a unified platform. Technical validation confirms "algorithmic correctness across eight subsystems" with "the first system to unify ACT-R, IRT, FSRS, ZPD, and metacognitive calibration in a single medical education platform." The platform provides "18 clinical cases with decision trees across six cancer types, maps every interaction to 13 ACGME Hematology-Oncology milestones, and implements four closed-loop feedback mechanisms."

Reports effect sizes.

Research paradigm

Design science / computational artifact development

Author conclusions

"Onco-Shikshak V7 represents a paradigm shift from isolated educational modules to an integrated cognitive ecosystem for medical oncology education. By unifying ACT-R activation dynamics, Item Response Theory, FSRS spaced repetition, Zone of Proximal Development scaffolding, and metacognitive calibration training into a single architecture, the platform addresses the fundamental challenges of oncology knowledge velocity, LLM hallucination, and automation bias." The authors conclude: "While the technical foundation is delivered and validated, the critical next step—formal evaluation with oncology trainees—will determine whether this cognitive architecture translates to measurable learning outcomes."

Risk of bias

Single-author case construction (potential author bias in decision tree design); Uncalibrated IRT parameters (may not reflect oncology learner population); No expert consensus validation of ACGME milestone thresholds; No independent technical validation beyond author verification; Single-author case development introduces selection and construction bias in case decision trees; Absence of expert consensus (e.g., Delphi) validation for clinical content; No empirical learner evaluation—technical validation only, no learning outcome data; IRT parameters use general pre-trained values, not oncology-population-calibrated, risking miscalibration bias; Unvalidated metacognitive interventions lack empirical testing for efficacy; Keyword-based retrieval for local textbooks may introduce retrieval bias compared to semantic retrieval; Single-author case construction without expert consensus; Lack of empirical validation with target population; Pre-trained IRT parameters not calibrated to oncology learner population; Metacognitive interventions unvalidated; No learner evaluation data to assess actual efficacy

Limitations

"We acknowledge several limitations: 1
No formal learner evaluation
The system has not been evaluated with actual medical trainees
A randomized controlled trial is planned (Section 9.6)
Uncalibrated IRT parameters
Ability (θ) and difficulty (β) parameters use general pre-trained values, not oncology-population-calibrated estimates

Open questions raised

Formal learner evaluation with oncology residents via randomized controlled trial (20-30 residents at 2-3 academic centers)
Semantic retrieval for local textbooks (replacing keyword matching with text embeddings)
On-Call module for oncologic emergencies (SVC syndrome, spinal cord compression, tumor lysis, etc.)
Longitudinal case chains modeling disease progression across encounters
LMS/LTI integration for institutional deployment
Expert panel validation via Modified Delphi process for case decision trees and ACGME thresholds

Data: Source code, case library, clinical trial data, and Prisma schema; "The complete source code, case library (18 JSON files), clinical trial data, and Prisma schema are available at https://github.com/showmethecode-dev/onco-shikshak under the MIT license."; source; url; contents; licenseCode: Onco-Shikshak; https://github.com/showmethecode-dev/onco-shikshak (MIT license); GitHub: https://github.com/showmethecode-dev/onco-shikshakExtracted from: pdfAgreement 53%

Explore related topics

AI Literacy Training Doctoral Training Design Science