AI evidence extraction

A comprehensive AI policy education framework for university teaching and learning

Cecilia Ka Yuk Chan · International Journal of Educational Technology in Higher Education · 2023

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

7/10

Relevance

Evidence

1,160

Citations

41.01

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1186/s41239-023-00408-3

Methodology & findings

Study design

Mixed-methods survey design combining quantitative (closed-ended questions with descriptive analysis) and qualitative (open-ended questions with thematic analysis) data collection from 457 undergraduate and postgraduate students and 180 teachers and staff across various disciplines in Hong Kong universities via online questionnaire..

Sample

N = 637, 2 groups

Primary method

Descriptive analysis (means, standard deviations, medians) for quantitative survey data. Thematic analysis using an inductive approach for qualitative open-ended responses, where themes emerged from the data rather than being predetermined. No inferential statistical tests reported.

Main result

The study found that "both students (mean = 2.28, SD = 1.18) and teachers (mean = 2.02, SD = 1.1) reported relatively low experience" with generative AI technologies, yet "both groups demonstrated a belief in the positive impact of integrating AI technologies into higher education (students: mean = 4, SD = 0.891; teachers: mean = 3.87, SD = 1.32)." Additionally, "both groups expressed doubt about teachers' ability to accurately identify a student's usage of generative AI technologies for completing assignments (students: mean = 3.02, SD = 1.56; teachers: mean = 2.72, SD = 1.62)."

Reports effect sizes.

Research paradigm

Mixed-methods (pragmatism)

Author conclusions

The authors conclude that "This study proposes an AI Ecological Education Policy Framework to address the diverse implications of AI integration in university settings. The framework consists of three dimensions—Pedagogical, Governance, and Operational—each led by a responsible party. This structure allows for a more comprehensive understanding of AI integration implications in teaching and learning settings and ensures stakeholders are aware of their responsibilities. By adopting this framework, educational institutions can align actions with their policy, ensuring responsible and ethical AI usage while maximizing potential benefits."

Risk of bias

Selection bias: convenience sampling based on availability and willingness; Self-report bias: reliance on self-reported data from participants; Sample representativeness: relatively small sample may not represent all educational institutions; Scope limitation: only text-based generative AI (ChatGPT, Bing, Co-Pilot) examined; Temporal limitation: Study conducted during early adoption phase of ChatGPT; findings may not generalize to later periods; Geographic limitation: Data collected only from Hong Kong universities, limiting generalizability; Sample representativeness: potentially small and non-representative sample from Hong Kong universities only; Temporal bias: UNESCO framework used as basis was developed before GPT-3.5 and GPT-4 availability

Limitations

"This study has some limitations, including a relatively small sample size that may not be representative of all educational institutions
Additionally, the research only focused on text-based generative AI technology and did not explore other types or variations
Lastly, the study relied on self-reported data from participants, which may be subject to bias or inaccuracies."

Open questions raised

The authors note that more research is necessary to fully comprehend AI integration in higher education. They highlight that while UNESCO's recommendations provide high-level guidelines, they were formulated before the availability of GPT 3.5 and 4, and do not specifically address university teaching and learning contexts. The authors also note that existing AI education policies tend to emphasize instrumental workforce development but lack focus on transformative potential and ethical considerations.
Further investigation needed into other types and variations of generative AI technology beyond text-based models
Need for longitudinal studies to assess the long-term impacts of AI integration on student learning outcomes
Requirement for implementation and evaluation studies of the proposed AI Ecological Education Policy Framework in actual institutional settings

Extracted from: pdf

Explore related topics

Institutional Policy AI Governance Mixed Methods