AI evidence extraction

Generative artificial intelligence for academic research: evidence from guidance issued for researchers by higher education institutions in the United States

Amrita Ganguly, Aditya Johri, Areej Ali, Nora McDonald · AI and Ethics · 2025

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

Evidence

Citations

9.62

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1007/s43681-025-00688-7

Methodology & findings

Study design

Thematic analysis of institutional policies and guidelines.

Sample

N = 30, 1 group

Primary method

Qualitative content analysis following Zhang and Wildemuth's approach. Three researchers conducted open coding to develop a codebook with six categories containing multiple codes and subcodes. Inter-rater reliability was achieved through formal coding process with discussion and reconciliation of differences. Frequency counts of codes were calculated for the entire codebook (N = 30 institutions). Overlap analysis was conducted across different categories to identify alignment and divergent actions. Percentages reported are out of N = 30.

Main result

The study found that guidance provided to researchers: (1) "asks them to refer to external sources of information such as funding agencies and publishers to keep updated and use institutional resources for training and education"; (2) "asks them to understand and learn about specific GenAI attributes that shape research such as predictive modeling, knowledge cutoff date, data provenance, and model limitations, and educate themselves about ethical concerns such as authorship, attribution, privacy, and intellectual property issues"; and (3) "includes instructions on how to acknowledge sources and disclose the use of GenAI, how to communicate effectively about their GenAI use, and alerts researchers to long term implications such as over reliance on GenAI, legal consequences, and risks to their institutions from GenAI use."

Reports effect sizes.

Research paradigm

Qualitative interpretivism with inductive thematic analysis

Author conclusions

The authors conclude that institutional guidance on GenAI use for research "asks them to refer to external sources of information such as funding agencies and publishers to keep updated and use institutional resources for training and education; asks them to understand and learn about specific GenAI attributes that shape research such as predictive modeling, knowledge cutoff date, data provenance, and model limitations, and educate themselves about ethical concerns such as authorship, attribution, privacy, and intellectual property issues; and includes instructions on how to acknowledge sources and disclose the use of GenAI, how to communicate effectively about their GenAI use, and alerts researchers to long term implications such as over reliance on GenAI, legal consequences, and risks to their institutions from GenAI use. Overall, guidance places the onus of compliance on individual researchers making them accountable for any lapses, thereby increasing their responsibility."

Risk of bias

Selection bias: Study limited to R1 universities only, which are well-funded and resource-rich, limiting generalizability to less-resourced institutions; Temporal bias: Data collected July 20 - August 27, 2024; policies may have been released or revised since data collection; Document selection bias: Only publicly accessible policies included; non-public SharePoint sites excluded; Coder subjectivity: Qualitative coding vulnerable to interpretation bias despite inter-rater reliability measures; Coding bias: Although inter-rater reliability was assessed, only three researchers conducted analysis; Selection bias: Study limited to R1 research-intensive institutions, excluding institutions with lower research activity levels; Geographic bias: Study limited to institutions in the United States; Exclusion bias: Excluded policies that vaguely mentioned research data or were not solely research-specific, and instructional design-related sources

Limitations

The authors acknowledge that "because R1 universities tend to be well-funded and often resource-rich in the research arena, it limits the generalization of our findings to institutions that are less resourced with lower levels of research activities, i.e
those with less advanced infrastructure and policy-making capacity." Additionally, they note that "Given that this is a fast-developing area, we recognize the limitation that possibly more policies have been released since we collected the data or have been revised
Our data captures a slice in time." Data collection occurred from July 20, 2024, to August 27, 2024.

Open questions raised

The authors note the need to continue understanding the terrain of GenAI guidance as policies are changing frequently. They recommend focusing on practical aspects of GenAI use rather than theoretical exercises, and suggest the importance of developing documentation guidelines and training programs.

Data: Data is included in the Appendix within the manuscript. The dataset consists of 30 public GenAI research policies/guidelines from R1 universities.; Data is included in the Appendix within the manuscript. Appendix lists all 30 institutions analyzed with institutional affiliations, office/department sources, and web links to guidelines.Code: None mentionedExtracted from: pdf

Explore related topics

Responsible AI Use Academic Writing Qualitative