12,637 papers · updated 18 Sept 2026livingmeta.ai
← Browse all papers
AI evidence extraction

AI literacy and its implications for prompt engineering strategies

Nils Knoth, Antonia Tolzin, Andreas Janson, Jan Marco Leimeister · Computers and Education Artificial Intelligence · 2024

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

7/10
Relevance
E
Evidence
277
Citations
82.83
FWCI
Top 10%
Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1016/j.caeai.2024.100225

Methodology & findings

Study design

Mixed methods study combining quantitative assessment of prompt engineering skills and LLM output quality with qualitative analysis of students' intuitive behaviors towards LLM-based AI systems

Main result

The study found that "higher-quality prompt engineering skills predict the quality of LLM output, suggesting that prompt engineering is indeed a required skill for the goal-directed use of generative AI tools." Additionally, "certain aspects of AI literacy can play a role in higher quality prompt engineering and targeted adaptation of LLMs within education."

Reports effect sizes.

Research paradigm

Mixed methods (quantitative and qualitative)

Author conclusions

The authors conclude: "We, therefore, argue for the integration of AI educational content into current curricula to enable a hybrid intelligent society in which students can effectively use generative AI tools such as ChatGPT." This conclusion is based on findings that "prompt engineering is indeed a required skill for the goal-directed use of generative AI tools."

Risk of bias

Not explicitly stated in abstract; Selection bias: Study participants were students from higher education context; Potential confounders: Prior experience with AI systems, technical background, language proficiency; Potential confounders: Prior technology experience, domain knowledge, and individual differences in problem-solving approaches not fully characterized in abstract; Measurement bias: Quality assessment of prompts and LLM outputs depends on evaluator judgment criteria

Limitations

  • The paper states that "research on the perspectives of non-experts using LLM-based AI systems through prompt engineering and on how AI literacy affects prompting behavior is lacking," indicating a significant research gap that this study addresses but may not fully resolve given the novelty of the domain.

Open questions raised

  • The paper identifies the lack of research on perspectives of non-experts using LLM-based AI systems through prompt engineering and how AI literacy affects prompting behavior, particularly in higher education contexts. Future research directions focus on AI literacy curriculum integration.
Data: not_statedCode: not_statedExtracted from: pdf

Explore related topics

AI Literacy TrainingDoctoral TrainingEmpirical Study

Related papers

← Browse all papers