AI evidence extraction

Embracing the future of Artificial Intelligence in the classroom: the relevance of AI literacy, prompt engineering, and critical thinking in modern education

Yoshija Walter · International Journal of Educational Technology in Higher Education · 2024

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

7/10

Relevance

2/4

Quality (LMQS)

Evidence

805

Citations

544.77

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1186/s41239-024-00448-3

Methodology & findings

Study design

Two-step mixed-methods approach combining: (1) case study based on informal unstructured discussions with students and lecturers at a Swiss University of Applied Sciences (Kalaidos University of Applied Sciences), including classroom discussions, individual interviews, lunch talks with professors, and correspondence analysis; and (2) narrative literature review to address gaps and provide best practices and practical suggestions for higher education implementation..

Main result

The paper identifies three prime skills necessary for AI integration in education: "AI literacy is identified as crucial, encompassing an understanding of AI technologies and their broader societal impacts. Prompt engineering is highlighted as a key skill for eliciting specific responses from AI systems, thereby enriching educational experiences and promoting critical thinking." The study finds that students at the case-study institution are generally enthusiastic about AI tools but lack adequate training in responsible use, and that "providing AI regulations is a good first step, but creating ways for students and lecturers to engage more deeply with the topic would probably enhance these measures and might help to foster a respective culture."

Research paradigm

Pragmatist/Mixed-methods (combines case study with narrative literature review)

Author conclusions

The authors conclude that "Addressing these issues requires more than just setting guidelines; it calls for a holistic approach that includes educating students about AI, its ethical use, and limitations." They emphasize that "in order for all parties to be best prepared for using AI in education, based on a case study and a subsequent literature analysis, there are three necessary skills that can remedy these problems, which are AI literacy, knowledge about prompt engineering, and critical thinking." The paper advocates for institutional investment: "There should be a budget dedicated to helping employees to become knowledgeable in the field" and recommends "creating positions where experts have a say and can help shape the AI culture in the institution."

Risk of bias

Selection bias: Case study limited to one institution (Kalaidos University of Applied Sciences); Researcher positionality bias: Author holds leadership position in university's AI-Taskforce, creating potential insider bias; Sampling bias: Informal discussions were unstructured and collected 'where feasible', not systematic; Confirmation bias: Author's teaching experience may influence interpretation of findings; Limited representativeness: Higher education focus with adult students may not generalize to other educational levels; No control groups or comparison institutions; Selection bias: informal, unstructured discussions may not be representative of all students and faculty; Confirmation bias: author's intimate position as lead of AI-Taskforce may influence interpretation of data; Attrition bias: voluntary reading of guidelines means compliance cannot be verified; Institution-specific bias: findings from one Swiss university may not generalize to other educational contexts; Temporal bias: data collected during 'early days of AI use' (circa 2023) may not reflect current adoption patterns; Selection bias: Case study based on single institution (Kalaidos University of Applied Sciences); Respondent bias: Informal, unstructured discussions may not capture representative sample; Confirmation bias: Author holds leadership position in university's AI-Taskforce, potentially biasing observations; Attrition bias: Unclear what proportion of students/lecturers participated in discussions; Geographic bias: Swiss higher education context may not represent other educational systems; Temporal bias: Observations made during early adoption phase (2022-2023) of ChatGPT

Limitations

The paper acknowledges several limitations: "One of the most substantial issues is the fact that their effectiveness hinges on student compliance, which is not guaranteed
Many students might not thoroughly read these documents, leading to a gap in understanding and adherence." Additionally, "a significant issue is the lack of comprehensive training in AI capabilities for students
Merely providing a document on AI use is not sufficient for fostering a deep understanding of AI technology, its potential, and its limitations." The authors also note that "Monitoring the use of AI in student assignments poses another challenge
It is difficult to verify whether a piece of work has been created with the aid of AI, especially as these tools become more sophisticated." The case study is institution-specific (one Swiss university), limiting generalizability.

Open questions raised

Lack of systematic AI training programs for educators
Need for comprehensive curriculum adaptation to embed AI literacy across disciplines
Limited research on effective monitoring and verification of AI use in student work
Gap in understanding of prompt engineering as pedagogical tool
Need for more specific practical implementation guidelines beyond policy documents
Insufficient research on equity and access issues in AI integration

Extracted from: pdfAgreement 75%

Explore related topics

AI Literacy Training Doctoral Training Case Study