AI evidence extraction

“From Unseen Needs to Classroom Solutions”: Exploring AI Literacy Challenges & Opportunities with Project-Based Learning Toolkit in K-12 Education

Hanqi Li, Ruiwei Xiao, Hsuan Nieu, Ying-Jui Tseng, Guanze Liao · Proceedings of the AAAI Conference on Artificial Intelligence · 2025

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

5/10

Relevance

1/4

Quality (LMQS)

Evidence

Citations

16.49

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1609/aaai.v39i28.35187

Methodology & findings

Study design

Formative empirical evaluation combining qualitative interviews and demonstration sessions.

Primary method

Co-design with teachers; participatory design approach; design science research

Main result

The study found that "38% of teachers stated that they have a strong understanding of the AI4K12 framework or the 5 Big Ideas" and "69.23% are confident (self-rated confidence level ≥ 50%) in understanding the results of AI and recognizing their limitations." Additionally, "over 46% of teachers mentioned that using the PBL Toolkit could help solve one or more challenges in the course plans they designed," with the most common challenge being limited teaching resources. The research also revealed that "AI chatbot has the widest adoption (n=9)" among the three toolkit components, and teachers designed activities aligned to Bloom's Taxonomy with "33.3%" at the create-level, "23.8%" for concept understanding, and "19%" at the remember-level.

Research paradigm

pragmatist/design science

Author conclusions

"Our results highlight several key points about integrating the PBL AI Literacy toolkit into K-12 education. First, while many teachers are confident in understanding AI's basic capabilities, they still face challenges, especially with students' varying AI skill levels." The authors further conclude that "students' opportunities to learning AI Literacy at school might not be significantly varied by their economic status. Instructors teaching lower-income students showed success in guiding their students properly through AI tools and produce satisfying study results, despite resource constraints, which challenges assumptions about access and technological inequality."

Risk of bias

Small sample size (n=13) limiting generalizability; Limited demographic diversity despite purpose-sampling; Self-selection bias: teachers voluntarily participated; Social desirability bias in self-reported AI literacy confidence levels; Single-rater bias in initial coding (though two researchers conducted analysis); Potential observer effect from researchers conducting Zoom-based interviews; Geographic concentration (North America and East Asia only); Language barrier: interviews conducted in English and Mandarin with potential translation effects; Selection bias: Purpose sampling strategy may not represent broader K-12 teacher population; Small sample size (n=13) limits generalizability; Geographic limitation: 54% North America, 46% East Asia; limited diversity; Self-report bias: Teachers' confidence levels and perspectives self-assessed; Interview context bias: Participants aware they were being evaluated on toolkit usage; Language bias: Interviews conducted in English and Mandarin, potentially affecting non-native speakers; Limited demographic diversity despite purposive sampling; Selection bias: teachers recruited may be more interested in AI literacy than general population; Self-rated confidence measures subject to social desirability bias; Researcher facilitation of interviews could influence responses

Limitations

The authors state that "There are limitations in this research that should be considered when interpreting the findings, including a small sample size and limited diversity, which may affect the generalizability of the findings." They further note that "Future studies should involve a broader demographic for a more comprehensive understanding of AI literacy challenges." Additionally, the paper acknowledges that "Although a PBL AI Toolkit can address many of the major difficulties teachers face in the classroom, there are still some practical issues that fall outside its scope," such as widespread hardware access and equipment limitations in schools.

Open questions raised

Future studies should involve "a broader demographic for a more comprehensive understanding of AI literacy challenges." Future work could focus on: (1) improving teacher training to enhance AI literacy, (2) offering clearer classroom guidance, (3) addressing ethical concerns about AI-generated content accuracy and copyright issues, (4) making the toolkit scalable for diverse educational settings, and (5) providing better onboarding support and scaffolding for teachers.
Need for broader demographic representation in future AI literacy studies
Limited research on teacher training approaches to enhance AI literacy
Gap in understanding how to make AI literacy tools scalable across diverse educational settings
Need for addressing legal/copyright concerns with AI-generated content in educational contexts
Limited exploration of hardware and equipment accessibility issues in underserved schools

Data: Not mentioned as available; interview transcripts referenced but not stated to be publicly availableCode: Not mentionedExtracted from: pdfAgreement 58%

Explore related topics

AI Literacy Training Doctoral Training Qualitative