AI evidence extraction

Regards croisés sur les IA Génératives dans l'Enseignement Supérieur en Gestion. Volet 2 - Recherche qualitative d’usages précurseurs et préconisations

Cécile Godé, Régis Meissonier · HAL (Le Centre pour la Communication Scientifique Directe) · 2024

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

2/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.13140/rg.2.2.30442.12488

Methodology & findings

Study design

Qualitative case study research using semi-structured interviews (22 interviews of one hour each, conducted between mid-March and mid-June 2024) with 20 faculty members and 2 pedagogical engineers identified as early adopters of generative AI.

Sample

N = 22, 5 groups

Primary method

Thematic content analysis (analyse thématique de contenu). Authors state: "L'ensemble du corpus de données a fait l'objet d'une analyse thématique de contenu afin de repérer puis regrouper des thèmes généraux récurrents" (The entire data corpus underwent thematic content analysis to identify and group recurring general themes). Supplementary analysis methods mentioned for student questionnaires created by some faculty (automatic analysis of responses, comparison across cohorts), but no formal statistical tests reported.

Main result

The study found that "the very large majority of people interviewed have implemented IAg (whether in course construction, delivery, or student evaluation) according to a principle of both innovation and caution regarding potential and possible drifts." A common characteristic is that "users recognize using generative IAg in their quest [for course development], with one of the main advantages they emphasize being the time saved from no longer having to surf the web following many hyperlinks page by page." The research reveals that "early adopters" unanimously advocate for "responsible and measured use of IAg in teaching" and that "all persons interviewed emphasize the necessity of enclosing use with the awareness of possible drifts."

Reports effect sizes.

Research paradigm

interpretivist/constructivist

Author conclusions

The authors conclude: "En d'autres termes, nos propositions ne sont pas de simples items à ajouter à une liste déjà longue de projets visant à maintenir la visibilité et l'attractivité de nos institutions. Elles impliquent avant toute chose un changement de paradigme quant au rôle que nous avons à jouer dans un processus d'apprentissage désormais médiaté par une technologie ayant une capacité de traitement de données supérieure à n'importe quel cerveau humain" (In other words, our proposals are not simple items to add to an already long list of projects aimed at maintaining institutional visibility and attractiveness. They first and foremost require a paradigm shift regarding the role we must play in a learning process now mediated by technology with data processing capacity superior to any human brain). They propose three main recommendations: (1) ecosystem approach linking commerce schools and universities, (2) formation of teacher communities of practice, and (3) development of institutional RAG-based AI systems.

Risk of bias

Selection bias: sample of convenience, non-random selection of early adopters; Survivor bias: only interviewed those who successfully adopted AI (no interviews with resistant or failed adoption cases); Self-selection bias: participants voluntarily agreed to participate; Small sample size (n=22) with low variance; Potential confirmation bias: researchers seeking innovative practices may have emphasized positive findings; Selection bias: Convenience sampling of 'early adopters' only, not representative of broader faculty population; Sampling bias: Small sample size (n=22) with low variance in adoption levels; Survivorship bias: Only includes faculty with successful implementation experiences; Confirmation bias: Potential bias toward seeking experiences that support predetermined themes about innovation and prudence; Selection bias: Sample consists exclusively of early adopters (6% of faculty identified in Phase 1), not representative of general faculty population; Self-selection bias: Participants volunteered and were identified through opportunity sampling; Low variance in adoption levels: All participants are advanced/frequent users of AI; Survivorship bias: Captures only those actively using AI, not those who rejected it; Temporal bias: Data collected over short 4-month window during period of rapid AI evolution; Small sample size (n=22) limits generalizability

Limitations

The authors state: "Sur un plan méthodologique, nos entretiens se résument à un échantillon de 22 répondants (20 enseignants-chercheurs et 2 ingénieurs pédagogiques), précurseurs dans l'utilisation des IAg
Outre son nombre réduit, l'échantillon affiche une faible variance en termes de niveau d'adoption de ces solutions technologiques." (Methodologically, our interviews are limited to a sample of 22 respondents, pioneers in the use of generative AI
Beyond its small size, the sample shows low variance in terms of adoption level.) Additionally: "La collecte a été arrêtée lorsqu'il est apparu que toute nouvelle donnée n'apportait plus d'éléments significatifs à la compréhension des usages précurseurs des IAg
La contrainte de temps de production du rapport ainsi que la difficulté grandissante à identifier des enseignants-chercheurs précurseurs supplémentaires y ont également contribué." (Data collection stopped when new information no longer provided significant elements for understanding early adopter uses
Time constraints for report production and growing difficulty identifying additional early adopter faculty also contributed.)

Open questions raised

Need for research on the long-term impacts of AI on student cognitive abilities and retention
Lack of frameworks for evaluating return on investment (ROI) of AI implementations in higher education
Limited understanding of how AI adoption trajectories differ across institutional types
Insufficient evidence on appropriate pedagogical models for AI-mediated learning
Need for interdisciplinary (not just management-focused) policies on AI in higher education
Gap in understanding the ethical and legal frameworks for AI-generated content evaluation

Extracted from: pdfAgreement 58%

Explore related topics

AI Literacy Training Human-AI Collaboration Qualitative