AI evidence extraction

Ce que l’IA fait à l’écriture scientifique

Nathan Ferret, Patrice Abry, Rémy Cazabet, Philippe Gabriel, Lucie Gournay, Jean-Philippe Magué et al. · Discourse and Writing/Rédactologie · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

2/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.31468/dwr.1173

Methodology & findings

Study design

Ethnographic participatory action research with direct observation.

Sample

N = 32, 16 groups

Primary method

No formal quantitative statistical methods were employed. The research uses qualitative ethnographic analysis with participatory co-construction. Data analysis methods include thematic observation, interpretive analysis of writing practices, and collective validation of findings by participants.

Main result

The study found that "le LLM a, dès lors, joué un rôle de médiateur entre ces disciplines et statuts, en facilitant l'explicitation des savoirs, la mise en langage des hypothèses, et la coordination des démarches analytiques" (the LLM served as a mediator between disciplines and statuses, facilitating the explicitation of knowledge, the formulation of hypotheses in language, and the coordination of analytical approaches). Additionally, the authors observed that "Cette mise en doute permanente structure la trajectoire de recherche et d'écriture scientifiques: elle appelle des stratégies de contournement des limites du modèle" (this permanent questioning structures the trajectory of scientific research and writing: it calls for strategies to circumvent the limits of the model).

Reports effect sizes.

Research paradigm

Interpretivist/qualitative ethnographic

Author conclusions

The authors conclude that "le lien d'écriture au modèle est apparu, tout au long du hackathon, comme une phase critique du travail scientifique: un moment où se concentrent le raisonnement collectif, le dialogue interdisciplinaire et la coopération de groupe" (the link of writing to the model appeared throughout the hackathon as a critical phase of scientific work: a moment where collective reasoning, interdisciplinary dialogue, and group cooperation concentrate). Furthermore, they argue that writing with LLMs "oblige en effet les chercheurs et chercheuses à formuler, expliciter et cadrer ce qu'ils et elles attendent -non seulement de la machine, mais aussi de leur propre démarche collective" (indeed obliges researchers to formulate, make explicit, and frame what they expect—not only from the machine, but also from their own collective approach).

Risk of bias

Selection bias: Participants were self-selected volunteers, primarily from French academic institutions; Observer bias: Single ethnographer with embedded participatory approach may influence group dynamics; Contextual bias: Hackathon setting with explicit legitimization of LLM use creates non-representative research conditions; Social desirability bias: Awareness of being studied may influence reporting and use of tools; Selection bias: Participants were self-selected volunteers interested in LLM experimentation, not a random sample of researchers; Hawthorne effect: Participants' behavior may have been altered by the presence of an observing ethnographer; Limited generalizability: Single event with 32 participants from primarily French academic institutions; Observer bias: The ethnographer's presence and role as 'facilitator and mediator' may have influenced dynamics; Participant reactivity: Self-consciousness about LLM usage due to ethical sensitivity and perceptions of incompetence or cheating; Selection bias: Participants were self-selected volunteers, not randomly recruited; Setting bias: Hackathon context (high-intensity, explicitly LLM-legitimized) differs from routine academic work; Observer effect: Presence of ethnographer may influence behavior; Disciplinary bias: Participants from French academic institutions primarily

Limitations

The authors acknowledge that "l'exploration des usages scientifiques des LLM repose majoritairement sur des analyses de corpus ou des métasynthèses, les observations en situation demeurent rares" (exploration of scientific uses of LLMs relies mainly on corpus analyses or meta-syntheses, and situational observations remain rare)
Additionally, they note that "Cela tient notamment à la nature souvent solitaire et peu visible du travail d'écriture scientifique, mais aussi à la dimension socialement sensible de l'usage de ces outils" (this is partly due to the often solitary and poorly visible nature of scientific writing work, but also to the socially sensitive dimension of using these tools)
The study's scope is limited to a single hackathon event lasting two days.

Open questions raised

The authors identify a gap between majority corpus-based and meta-synthesis studies versus situated observations of LLM use. They note the scarcity of direct observation of scientific writing practices and the social sensitivity surrounding LLM use in academic contexts. The paper suggests that understanding LLM integration requires examining writing practices across all phases of research, from problem formulation to interpretation and discussion of results.
The authors identify that observations of LLM usage in situated research contexts remain rare, despite growing deployment of these tools. They note that the solitary and often invisible nature of scientific writing work, combined with the socially sensitive perception of LLM usage, makes direct observation difficult. The authors propose their empirical anchoring in direct observation and focus on writing practices as an under-explored dimension of LLM integration in scientific research.
The authors identify that "observations in situ remain rare" in studies of LLM use in scientific contexts, and call for more empirical research grounded in direct observation. They suggest future research should examine how LLM integration transforms long-term research practices beyond accelerated hackathon contexts, and how disciplinary norms and epistemological standards shape LLM adoption across different scientific fields.

Data: Five datasets provided by organizing committee or participants during hackathon, referenced at: https://github.com/IXXI-fr/Hackathon25/blob/main/données/Covid.pdf; COVID-19 dataset; Five datasets provided for hackathon projects; Five datasets provided by organizing committee or proposed by participants (specific data not detailed in paper); GitHub repository reference: https://github.com/IXXI-fr/Hackathon25/blob/main/données/Covid.pdfCode: https://github.com/IXXI-fr/Hackathon25; IXXI Hackathon GitHub repositoryExtracted from: pdfAgreement 54%

Explore related topics

AI Manuscript Writing Academic Writing Qualitative