Intelligent Tutoring for Law Courses: Design and Evaluation of an LLM-based System
Florian Weber, Seyed Parsa Neshaei, Thiemo Wambsganß, Matthias Soellner · Journal of the Association for Information Systems · 2025
AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.
Methodology & findings
Study design
Design science research approach consisting of two phases: (1) requirements derivation through interviews with 14 law students, and (2) quantitative evaluation study with 40 business and law students.
Primary method
Design science research
Main result
The study evaluated CaseCoach's effectiveness through a quantitative study involving 40 business and law students. The abstract indicates the system was "designed to support students in solving complex legal problems through conversational tutoring," and the research "contributes to the development of interactive intelligent tutoring systems and expands design knowledge for AI-based educational technologies in technology-mediated learning environments."
Research paradigm
Design science research
Author conclusions
The authors conclude that "Our research contributes to the development of interactive intelligent tutoring systems and expands design knowledge for AI-based educational technologies in technology-mediated learning environments," indicating that CaseCoach represents a meaningful advance in AI-based educational system design.
Risk of bias
Small sample size (n=40) may limit generalizability; Potential selection bias in recruitment of law and business students; No mention of control group in abstract, suggesting possible lack of comparison condition; No information on blinding or allocation concealment; Potential self-selection bias in the interview phase (n=14); Small sample size for quantitative study (n=40); Participant selection bias from interviews (n=14 law students); Potential self-selection bias in user study participation; Small sample size (40 students) may limit generalizability; Self-selection bias in student participation; Potential Hawthorne effect from being observed in tutoring system study
Open questions raised
- The paper identifies the need for better support of legal writing and problem-solving through technology-mediated learning, particularly in addressing the challenge that "Many students, however, find legal writing challenging because of its complex reasoning and formal conventions."
Explore related topics
Related papers
- What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in educationAhmed Tlili · 2023 · 1,587 citations
- Conceptualizing AI literacy: An exploratory reviewDavy Tsz Kit Ng · 2021 · 1,492 citations
- A SWOT analysis of ChatGPT: Implications for educational practice and researchMohammadreza Farrokhnia · 2023 · 1,171 citations
- Shaping the Future of Education: Exploring the Potential and Consequences of AI and ChatGPT in Educational SettingsSimone Grassini · 2023 · 921 citations
- Revolutionizing education with AI: Exploring the transformative potential of ChatGPTTufan Adıgüzel · 2023 · 858 citations
- Embracing the future of Artificial Intelligence in the classroom: the relevance of AI literacy, prompt engineering, and critical thinking in modern educationYoshija Walter · 2024 · 805 citations