AI evidence extraction

Artificial intelligence adoption in the physical sciences, natural sciences, life sciences, social sciences and the arts and humanities: A bibliometric analysis of research publications from 1960-2021

Stefan Hajkowicz, Conrad Sanderson, Sarvnaz Karimi, Alexandra Bratanova, Claire Naughtin · Technology in Society · 2023

AI-generated evidence extraction, verified across multiple analytical personas. Not a substitute for the peer-reviewed original.

9/10

Relevance

2/4

Quality (LMQS)

Evidence

119

Citations

4.24

FWCI

Top 10%

Impact

This is an AI analysis. Read the peer-reviewed original at the publisher: https://doi.org/10.1016/j.techsoc.2023.102260

Methodology & findings

Study design

Bibliometric analysis using The Lens database (version 8.2) containing 137 million peer-reviewed publications from 1960-2021.

Main result

The study found that "3.1 million of the 137 million peer-reviewed research publications during the entire period were AI-related, with a surge in AI adoption across practically all research fields (physical science, natural science, life science, social science and the arts and humanities) in recent years." Additionally, "In 1960 14% of 333 research fields were related to AI (many in computer science), but this increased to cover over half of all research fields by 1972, over 80% by 1986 and over 98% in current times." The analysis also showed that "over 50% of the total volume of AI research has been published in the past 5 years" and "the year-on-year growth in AI publishing has averaged at 26% over the past 5 years, compared to 17% for all preceding history."

Research paradigm

Positivist/Empiricist

Author conclusions

The authors conclude that "We are amid a worldwide surge in AI development and application for research. This is happening in the physical sciences, natural sciences, life sciences, social sciences and the arts and humanities. While AI has surged in the past, none of the prior events come close to the magnitude and breadth of the current situation." They further state: "AI is likely to continue improving the speed, cost-efficiency, safety and overall productivity of scientific research. Beyond mere efficiency gains, over the coming two decades, AI might fundamentally change the scientific method and human approaches to knowledge discovery. The overall implication of this study for researchers, and research organisations, is to invest in the many dimensions of AI capability uplift."

Risk of bias

Publication bias: Failed AI studies are underrepresented in literature; Database coverage bias: The Lens database may have differential coverage across fields and time periods; Search strategy bias: AI definition via 214 phrases may miss relevant publications or include false positives; Classification bias: ASJC classification assigned based on journal ISSN may not reflect actual content; Publication bias - failures in AI applications are underreported; Database coverage bias - The Lens may not capture all peer-reviewed publications uniformly across all disciplines and time periods; Search strategy bias - reliance on specific 214 AI phrases may miss emerging terminology or field-specific AI nomenclature; Classification bias - ASJC assignment based on ISSN descriptions may not perfectly reflect actual research content; Historical data completeness - older publications from 1960s-1980s may be less comprehensively indexed than recent publications; Publication bias: successful AI applications more likely to be published than failures; authors explicitly note 'it is hard to publish a failed AI study'; Database coverage bias: The Lens database composition may not uniformly represent all research fields; data sourced from multiple repositories (Microsoft Academic Graph, CrossRef, PubMed, Impactstory, CORE) with potentially different coverage; Search strategy bias: use of 214 OECD phrases may not capture all AI research or may capture non-AI publications; comparison with Liu, Shapira and Yue phrases showed 113 OECD phrases had no matching entry in prior work; Classification bias: assignment of publications to ASJC fields based on ISSN descriptions in Crossref metadata may misclassify interdisciplinary work; Temporal bias: early years (1960-1980s) likely have lower overall publication volume and potentially less complete database coverage; Discipline representation bias: computer science may dominate early AI records due to field origin

Limitations

The authors acknowledge that "there are some critical uncertainties about how AI development and application by researchers will unfold over the decades ahead." They note that "Evidence of widespread application is not the same as evidence of productivity enhancements." Furthermore, they state "While there is emerging evidence that AI is creating a productivity uplift in business...it is not yet well demonstrated in the science, research, innovation and technology sectors
More research is needed to examine this issue." Additionally, they observe that "it is hard to publish a failed AI study" and cite an evaluation where "62 published scientific studies using machine learning for COVID-19 diagnosis and prognosis found none of the models could be used for clinical purposes as they were subject to methodological flaws or biases."

Open questions raised

Limited understanding of whether current AI surge will be sustained or followed by a winter
Lack of evidence demonstrating actual productivity gains from AI in science and research sectors
Need for more research examining productivity uplift in science, research, innovation and technology sectors
Recognition that failures in AI applications are underreported and more research needed on problem areas unsuitable for AI
Uncertainty about whether the current AI boom will be sustained or followed by another AI winter
Lack of evidence demonstrating productivity gains from AI in science, research, innovation and technology sectors

Data: The Lens database (version 8.2) - authors state "The authors do not have permission to share data." The database contains over 224 million scholarly publications and 137 million intellectual property patents with records dating back to the 1950s.; The Lens database (version 8.2) - scholarly publications data extracted from records published up to 31 December 2021. Data sourced from Microsoft Academic Graph, CrossRef Open Researcher and Contributor IDentifier, PubMed, Impactstory, and Connecting Repositories (CORE). Authors note: "The authors do not have permission to share data."; Data availability statement states: "The authors do not have permission to share data." The Lens database (version 8.2) is publicly accessible at https://www.lens.org/Extracted from: pdfAgreement 61%

Explore related topics

AI Governance Benchmarking