AI evidence extraction

Speeding up to keep up: exploring the use of AI in the research process

Jennifer Chubb, Peter Cowling, Darren Reed · AI & Society · 2021

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

3/4

Quality (LMQS)

Evidence

198

Citations

21.33

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1007/s00146-021-01259-0

Methodology & findings

Study design

Deductive thematic analysis of semi-structured interviews with 25 leading academics working in AI futures or applications of AI to human creativity across multiple disciplines (physical/natural/life sciences, social sciences, and arts/humanities).

Main result

The study found that "AI is perceived as helpful with respect to information gathering and other narrow tasks, and in support of impact and interdisciplinarity. However, using AI as a way of 'speeding up-to keep up' with bureaucratic and metricised processes, may proliferate negative aspects of academic culture in that the expansion of AI in research should assist and not replace human creativity." Interviewees identified both positive and negative consequences for research and researchers with respect to collective and individual use.

Research paradigm

Interpretivist/qualitative

Author conclusions

"We are therefore left with a choice as to how far AI is incorporated into future research and to what end. Currently, there is no clear strategy." The authors conclude that "wider stakeholder discussions are needed on the challenges posed by introducing AI into the research process and to reflect on where its use is inappropriate or disadvantageous for research or researchers. If AI is to be deployed responsibly, incentives need to be provided and there needs to be acceptance of the potential for disruption." They emphasize that "AI can help research and researchers, but a deeper debate is required at all levels so as to avoid unintended negative consequences."

Risk of bias

Selection bias: participants identified through literature review and mapping of AI research landscape, potentially favoring prominent scholars; Temporal bias: data collected during COVID-19 lockdown which may have affected responses; Gender representation bias: despite efforts toward gender balance, one gender preponderates in some disciplines; Disciplinary representation bias: unequal distribution across disciplines (8 in physical/natural/life sciences, 12 in social sciences, 5 in arts/humanities); Selection bias: Participants were leading scholars self-selected from literature review on AI futures, not representative sample of all academics; Interviewer bias: Conducted during pandemic lockdown which may have affected responses; Disciplinary representation bias: Overrepresentation of computer science and social sciences (20/25 participants), underrepresentation of arts/humanities (5/25); Gender representation: While 64% female (16/25), this may not reflect natural distribution across disciplines due to gender disparities in computer science; Temporal bias: Conducted during COVID-19 crisis, which may have influenced perspectives on automation and work; Selection bias: Non-random sample of leading scholars; representation limited to specific institutions and geographic regions (UK, Europe, Canada, US); Attrition/retention: Small sample size (n=25) may not be representative of broader academic population; Interview setting bias: Online interviews during pandemic may have affected participant responses; Gender representation: While 64% female, skewed toward overrepresentation in some disciplines due to gender imbalance in AI field itself; Interviewer bias: Potential for leading questions or interpretation bias in thematic coding

Limitations

"Interviews were conducted during a National Lockdown Summer, 2020 and this may have affected participants' responses, at a time of multiple crisis
It can be difficult to develop a rapport with participants online...While generalization of an initial small scale qualitative study is difficult given the representation of disciplines, this research adds richness to existing issues and shows how AI can intersect with research from those at the cutting edge of its development and critique."

Open questions raised

Need for explicit movement of meta-research on the role of AI in research to consider effects on research and researcher creativity
Anticipatory approaches and engagement of diverse and critical voices at policy level and across disciplines needed
Further examination needed of effects of ML and AI in research funding systems to understand what responsible use would look like
Research needed on hidden consequences of AI adoption with respect to research
Need for futures research, anticipatory governance, and forecasting to develop beneficial research culture with AI
Insufficient information to guide best practice or consideration of limits of AI application

Data: Anonymised data can be made available upon request via ethics approval (as stated: 'Anonymised data can be made available upon request via ethics approval'); Anonymised interview data available upon request via ethics approval; Anonymised data can be made available upon request via ethics approvalExtracted from: pdfAgreement 76%

Explore related topics

Responsible AI Use Human-AI Collaboration Qualitative