AI evidence extraction

Generative artificial intelligence for academic research: evidence from guidance issued for researchers by higher education institutions in the United States

Amrita Ganguly, Aditya Johri, Areej Ali, Nora McDonald · AI and Ethics · 2025

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

3/4

Quality (LMQS)

Evidence

Citations

9.62

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1007/s43681-025-00688-7

Methodology & findings

Study design

Thematic analysis of institutional policies and guidelines.

Sample

N = 30, 3 groups

Primary method

Qualitative content analysis following Zhang and Wildemuth's approach. Three researchers conducted open coding to develop a codebook with six categories containing multiple codes and subcodes. Inter-rater reliability was achieved through formal coding process with discussion and reconciliation of differences. Frequency counts of codes were calculated for the entire codebook (N = 30 institutions). Overlap analysis was conducted across different categories to identify alignment and divergent actions. Percentages reported are out of N = 30.

Main result

The study found that guidance provided to researchers: (1) "asks them to refer to external sources of information such as funding agencies and publishers to keep updated and use institutional resources for training and education"; (2) "asks them to understand and learn about specific GenAI attributes that shape research such as predictive modeling, knowledge cutoff date, data provenance, and model limitations, and educate themselves about ethical concerns such as authorship, attribution, privacy, and intellectual property issues"; and (3) "includes instructions on how to acknowledge sources and disclose the use of GenAI, how to communicate effectively about their GenAI use, and alerts researchers to long term implications such as over reliance on GenAI, legal consequences, and risks to their institutions from GenAI use."

Reports effect sizes.

Research paradigm

Qualitative interpretivism with inductive thematic analysis

Author conclusions

The authors conclude that institutional guidance on GenAI use for research "asks them to refer to external sources of information such as funding agencies and publishers to keep updated and use institutional resources for training and education; asks them to understand and learn about specific GenAI attributes that shape research such as predictive modeling, knowledge cutoff date, data provenance, and model limitations, and educate themselves about ethical concerns such as authorship, attribution, privacy, and intellectual property issues; and includes instructions on how to acknowledge sources and disclose the use of GenAI, how to communicate effectively about their GenAI use, and alerts researchers to long term implications such as over reliance on GenAI, legal consequences, and risks to their institutions from GenAI use. Overall, guidance places the onus of compliance on individual researchers making them accountable for any lapses, thereby increasing their responsibility."

Risk of bias

Selection bias: Study limited to R1 universities only, which are well-funded and resource-rich, limiting generalizability to less-resourced institutions; Temporal bias: Data collected July 20 - August 27, 2024; policies may have been released or revised since data collection; Document selection bias: Only publicly accessible policies included; non-public SharePoint sites excluded; Coder subjectivity: Qualitative coding vulnerable to interpretation bias despite inter-rater reliability measures; Selection bias: Sample limited to R1 universities only, excluding less resource-rich institutions; Temporal bias: Data collected July-August 2024; policies may have been revised or new policies released; Publication bias: Only publicly available guidelines included; non-public SharePoint materials excluded; Coding bias: Although inter-rater reliability was assessed, only three researchers conducted analysis; Selection bias: Study limited to R1 research-intensive institutions, excluding institutions with lower research activity levels; Temporal bias: Data collection was time-bound (July-August 2024), capturing only a snapshot of rapidly evolving policies; Availability bias: Analysis limited to publicly available policies, excluding non-public resources (e.g., SharePoint sites); Geographic bias: Study limited to institutions in the United States; Exclusion bias: Excluded policies that vaguely mentioned research data or were not solely research-specific, and instructional design-related sources

Limitations

The authors acknowledge that "because R1 universities tend to be well-funded and often resource-rich in the research arena, it limits the generalization of our findings to institutions that are less resourced with lower levels of research activities, i.e
those with less advanced infrastructure and policy-making capacity." Additionally, they note that "Given that this is a fast-developing area, we recognize the limitation that possibly more policies have been released since we collected the data or have been revised
Our data captures a slice in time." Data collection occurred from July 20, 2024, to August 27, 2024.

Open questions raised

The authors note the need to continue understanding the terrain of GenAI guidance as policies are changing frequently. They recommend focusing on practical aspects of GenAI use rather than theoretical exercises, and suggest the importance of developing documentation guidelines and training programs.
The authors identify the need for continued research to understand the terrain as policies evolve. They recommend focusing on practical aspects of GenAI use rather than theoretical exercises. The paper notes that overall guidance for GenAI use in research varies greatly across journals, funding agencies, and professional associations and changes frequently, suggesting ongoing need for monitoring and analysis.
The authors identify the need to better understand practical aspects of GenAI use in research rather than conduct purely theoretical exercises. They note the importance of monitoring how policies evolve over time and the need for continued dialogue about institutional guidance. The paper suggests future research should examine how policies change and how well they are implemented across different institution types and resource levels.

Data: Data is included in the Appendix within the manuscript. The dataset consists of 30 public GenAI research policies/guidelines from R1 universities.; Data is included in the Appendix within the manuscript. Appendix lists all 30 institutions analyzed with institutional affiliations, office/department sources, and web links to guidelines.; Data is included in the Appendix within the manuscript. The appendix lists all 30 R1 institutions analyzed with their office/department affiliations and guideline/policy web links.Code: None mentionedExtracted from: pdfAgreement 53%

Explore related topics

Responsible AI Use Academic Writing Qualitative