AI evidence extraction

The dual impact of generative AI on research: structural stability and attention reallocation

Xiaoting Xu, Naixuan Zhao, Jiang Li, Xiao Hu · Information Research an international electronic journal · 2026

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

1/4

Quality (LMQS)

Evidence

Citations

0.00

FWCI

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.47989/ir31iconf64268

Methodology & findings

Study design

Large-scale quantitative analysis using graph clustering algorithms (fast-unfolding algorithm for community detection) on bibliographic coupling networks constructed from 24,309,359 publications in the OpenAlex dataset (2020-2024).

Main result

The study found a dual pattern in GenAI's initial impact: "macro-structural stability coexists with micro-level attention reallocation." Specifically, "the statistical analysis revealed no statistically significant difference between the two years" (2022 vs 2024) for topic variety (t=0.368, p=0.714), and "no statistically significant difference in mean clustering coefficients between 2022 and 2024" (t=1.961, p=0.055). However, "the proportion of GenAI-related publications within these prominent topics increased significantly from 2022 to 2024" (t=2.516, p=0.019), indicating substantial reallocation of scholarly attention toward GenAI within established research domains.

Research paradigm

Positivist/Empiricist - quantitative analysis of bibliometric data using computational methods

Author conclusions

The authors conclude: "The results suggest a dual pattern of influence: macro-level knowledge structures appear largely stable, while indications of a reallocation of research attention are observed at the micro level. These findings tentatively imply that GenAI may be beginning to reshape certain aspects of research focus, though further evidence is required to establish a definitive relationship."

Risk of bias

Publication lag bias (early stages only); Missing data bias (records with missing reference lists or unspecified research fields were excluded); Potential indexing bias in OpenAlex coverage; Selection bias in keyword-based retrieval for GenAI-related publications; Lack of qualitative validation of quantitative findings; Publication lag bias: only capturing early stages of GenAI adoption; Database coverage bias: analysis limited to OpenAlex indexed publications; Selection bias: exclusion of records with missing reference lists; Keyword detection bias: GenAI identification relies on keyword matching in titles, abstracts, and keywords; Publication lag bias - analysis captures only early stages of GenAI impact with potential 1-2 year publication delays; Database selection bias - exclusive reliance on OpenAlex may miss papers in non-indexed venues; Keyword retrieval bias - GenAI-related paper identification depends on consistent terminology usage in titles, abstracts, keywords; Missing data - 'records with missing reference lists or unspecified research fields' were excluded, potentially biasing the sample; Temporal boundary bias - selection of 2022 and 2024 as 'before and after' GenAI is somewhat arbitrary given publication delays

Limitations

The authors state: "First, the time lag in academic publishing means that we are capturing only the very early stages of GenAI's impact
Second, our analysis relies primarily on quantitative indicators and does not reveal qualitative investigation on how GenAI is used in the scientific research cycle."

Open questions raised

Future research should: (1) extend the observation window to examine whether current findings represent short-term fluctuations or long-term trends; (2) employ qualitative methods such as interviews and ethnography to investigate changes in research practices; (3) incorporate diverse metrics such as funding flows and collaboration network changes to comprehensively depict GenAI's transformation effects.
Extend the observation window to examine whether current findings represent short-term fluctuations or long-term trends
Employ qualitative methods such as interviews and ethnography to delve deeper into changes in research practices
Incorporate more diverse metrics, such as funding flows and changes in collaboration networks, to more comprehensively depict the transformation brought about by GenAI
Authors identify three future research directions: (1) extend the observation window to examine whether current findings represent short-term fluctuations or long-term trends; (2) employ qualitative methods such as interviews and ethnography to delve deeper into changes in research practices; and (3) incorporate more diverse metrics, such as funding flows and changes in collaboration networks, to more comprehensively depict the transformation brought about by GenAI.

Data: OpenAlex dataset (https://openalex.org/) - 24,309,359 publications from 2020-2024; OpenAlex dataset (https://openalex.org/) - papers published 2020-2024, 24,309,359 valid entries retained after excluding records with missing reference lists or unspecified research fields; OpenAlex dataset (https://openalex.org/) - papers published 2020-2024; 24,309,359 valid entries after filteringExtracted from: pdfAgreement 63%

Explore related topics

Productivity Effects Research Productivity Empirical Study