AI evidence extraction

AI, Explainable AI and Evaluative AI: Informed Data-Driven Decision-Making in Education

Sonsoles López‐Pernas, Eduardo Oliveira, Yige Song, Mohammed Saqr · 2025

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

5/10

Relevance

0/4

Quality (LMQS)

Evidence

Citations

3.32

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1007/978-3-031-95365-1_2

Methodology & findings

Study design

Narrative review synthesizing literature on AI applications in education, XAI techniques, and evaluative AI.

Main result

The chapter establishes that "AI has had an immense impact in education through its ability to facilitate personalized learning experiences," while simultaneously highlighting critical concerns: "the increasing adoption of AI systems in education has raised concerns regarding transparency and fairness." The authors synthesize that "Explainable AI (XAI) emerges as a solution to address these issues, offering methods to make AI models interpretable and their decisions understandable to learners, educators, and other stakeholders." Furthermore, the review demonstrates that "One of the reasons for the mistrust in AI is that its performance is commonly affected by bias, as AI models are only as unbiased as the data they are trained on," with the specific finding that "adaptive systems might inadvertently favor learners whose behavior aligns with the dominant patterns in the training data, leaving those with less conventional learning trajectories underserved."

Research paradigm

Interpretivist/Constructivist

Author conclusions

The authors conclude that "The chapter aims to provide readers with an understanding of AI in education and XAI's potential, serving as a foundation for the tutorials presented in the book." They further state that "The chapter also discusses evaluative AI as a new perspective on XAI for decision-making in which evidence is provided against and in favor of human-made hypotheses, rather than providing explanations for the single most-likely AI outcome." Most significantly, they emphasize that "Fostering a culture of inquiry—where explanations are evaluated, not merely accepted—is necessary for XAI to truly empower stakeholders to make informed, ethical decisions in education."

Risk of bias

Historical inequities or imbalances in training datasets perpetuating disparities; Algorithmic bias in AI models used for educational decisions; Bias in emotion recognition algorithms with potential cultural or individual biases; Selection bias in models trained on non-representative student populations; Feature independence assumptions in PDPs that may not hold in educational data; Disproportionate advantage to learners matching dominant patterns in training data; Selection bias in reviewed literature not addressed (non-systematic review); Potential publication bias from cited studies; No discussion of author funding sources or potential conflicts

Limitations

The chapter acknowledges several limitations regarding XAI implementation in education: "PDPs rely on the assumption that features are independent
In reality features like reading and writing forum contributions may be correlated, so this assumption can lead to misleading interpretations." Additionally, regarding emotion recognition technologies, the authors note that "the accuracy of emotion recognition algorithms and the potential for cultural or individual biases remain critical challenges to address." The authors further state that "Research suggests that predictive modeling is hardly generalizable and lacks portability between courses, or even between course iterations." Critically, they emphasize that "while feature importance highlights what influences predictions, it does not fully explain how features influence prediction and how features interact with one another, underlining its value as a foundational but limited interpretability tool."

Open questions raised

The paper identifies several gaps: (1) the need for more research on mitigating biases in AI algorithms and ensuring equitable treatment of all learners; (2) the importance of enhancing interpretability of AI systems through XAI; (3) limited understanding of how students respond to AI-generated feedback and whether it promotes better learning outcomes or contributes to over-reliance on AI; (4) the need to address privacy, consent, and data security concerns with emerging technologies like facial recognition and mood detection; (5) challenges in accuracy of emotion recognition algorithms and potential cultural biases in these systems.
Need for rigorous evaluation and diverse datasets in AI systems for education
Importance of fairness-aware machine learning practices and inclusive design principles
Limited research on cultural and individual biases in emotion recognition algorithms
Gap between predictive modeling accuracy and actual portability across courses
Need for deeper understanding of how features interact in educational prediction models

Extracted from: pdfAgreement 80%

Explore related topics

Responsible AI Use AI Governance Conceptual